stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 12 Releases Wiki Activity

up
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Details
Build Test Deploy / authority-container (push) Has been cancelled
Details
Build Test Deploy / docs (push) Has been cancelled
Details
Build Test Deploy / deploy (push) Has been cancelled
Details
Docs CI / lint-and-preview (push) Has been cancelled
Details

Browse Source
This commit is contained in:
root
2025-10-10 06:53:40 +00:00
parent 3aed135fb5
commit df5984d07e
1081 changed files with 97764 additions and 61389 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/08_MODULE_SPECIFICATIONS.md
View File

@@ -131,7 +131,11 @@ Each connector ships fixtures/tests under the matching `*.Tests` project.
* JSON exporter mirrors vuln-list layout with per-file digests and manifest.
* Trivy DB exporter shells or native-builds Bolt archives, optionally pushes OCI
layers, and records export cursors.
layers, and records export cursors. Delta runs reuse unchanged blobs from the
previous full baseline, annotating `metadata.json` with `mode`, `baseExportId`,
`baseManifestDigest`, `resetBaseline`, and `delta.changedFiles[]`/`delta.removedPaths[]`.
ORAS pushes honour `publishFull` / `publishDelta`, and offline bundles respect
`includeFull` / `includeDelta` for air-gapped syncs.
### 5.4 Feedser.WebService

697
docs/09_API_CLI_REFERENCE.md
View File

@@ -1,198 +1,198 @@
# API & CLI Reference
*Purpose* – give operators and integrators a single, authoritative spec for REST/GRPC calls **and** first‑party CLI tools (`stella-cli`, `zastava`, `stella`).
Everything here is *source‑of‑truth* for generated Swagger/OpenAPI and the `--help` screens in the CLIs.
---
## 0 Quick Glance
| Area | Call / Flag | Notes |
| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ |
| Scan entry | `POST /scan` | Accepts SBOM or image; sub‑5 s target |
| Delta check | `POST /layers/missing` | <20 ms reply; powers *delta SBOM* feature |
| Rate‑limit / quota | — | Headers **`X‑Stella‑Quota‑Remaining`**, **`X‑Stella‑Reset`** on every response |
| Policy I/O | `GET /policy/export`, `POST /policy/import` | YAML now; Rego coming |
| Policy lint | `POST /policy/validate` | Returns 200 OK if ruleset passes |
| Auth | `POST /connect/token` (OpenIddict) | Client‑credentials preferred |
| Health | `GET /healthz` | Simple liveness probe |
| Attestation * | `POST /attest` (TODO Q1‑2026) | SLSA provenance + Rekor log |
| CLI flags | `--sbom-type` `--delta` `--policy-file` | Added to `stella` |
\* Marked **TODO** → delivered after sixth month (kept on Feature Matrix “To Do” list).
---
## 1 Authentication
Stella Ops uses **OAuth 2.0 / OIDC** (token endpoint mounted via OpenIddict).
```
POST /connect/token
Content‑Type: application/x-www-form-urlencoded
grant_type=client_credentials&
client_id=ci‑bot&
client_secret=REDACTED&
scope=stella.api
```
Successful response:
```json
{
"access_token": "eyJraWQi...",
"token_type": "Bearer",
"expires_in": 3600
}
```
> **Tip** – pass the token via `Authorization: Bearer <token>` on every call.
---
## 2 REST API
### 2.0 Obtain / Refresh Offline‑Token
```text
POST /token/offline
Authorization: Bearer <admin‑token>
```
| Body field | Required | Example | Notes |
|------------|----------|---------|-------|
| `expiresDays` | no | `30` | Max 90 days |
```json
{
"jwt": "eyJhbGciOiJSUzI1NiIsInR5cCI6...",
"expires": "2025‑08‑17T00:00:00Z"
}
```
Token is signed with the backend’s private key and already contains
`"maxScansPerDay": {{ quota_token }}`.
### 2.1 Scan – Upload SBOM **or** Image
```
POST /scan
```
| Param / Header | In | Required | Description |
| -------------------- | ------ | -------- | --------------------------------------------------------------------- |
| `X‑Stella‑Sbom‑Type` | header | no | `trivy-json-v2`, `spdx-json`, `cyclonedx-json`; omitted ➞ auto‑detect |
| `?threshold` | query | no | `low`, `medium`, `high`, `critical`; default **critical** |
| body | body | yes | *Either* SBOM JSON *or* Docker image tarball/upload URL |
Every successful `/scan` response now includes:
| Header | Example |
|--------|---------|
| `X‑Stella‑Quota‑Remaining` | `129` |
| `X‑Stella‑Reset` | `2025‑07‑18T23:59:59Z` |
| `X‑Stella‑Token‑Expires` | `2025‑08‑17T00:00:00Z` |
**Response 200** (scan completed):
```json
{
"digest": "sha256:…",
"summary": {
"Critical": 0,
"High": 3,
"Medium": 12,
"Low": 41
},
"policyStatus": "pass",
"quota": {
"remaining": 131,
"reset": "2025-07-18T00:00:00Z"
}
}
```
**Response 202** – queued; polling URL in `Location` header.
---
### 2.2 Delta SBOM – Layer Cache Check
```
POST /layers/missing
Content‑Type: application/json
Authorization: Bearer <token>
```
```json
{
"layers": [
"sha256:d38b...",
"sha256:af45..."
]
}
```
**Response 200** — <20 ms target:
```json
{
"missing": [
"sha256:af45..."
]
}
```
Client then generates SBOM **only** for the `missing` layers and re‑posts `/scan`.
---
### 2.3 Policy Endpoints
| Method | Path | Purpose |
| ------ | ------------------ | ------------------------------------ |
| `GET` | `/policy/export` | Download live YAML ruleset |
| `POST` | `/policy/import` | Upload YAML or Rego; replaces active |
| `POST` | `/policy/validate` | Lint only; returns 400 on error |
| `GET` | `/policy/history` | Paginated change log (audit trail) |
```yaml
# Example import payload (YAML)
version: "1.0"
rules:
- name: Ignore Low dev
severity: [Low, None]
environments: [dev, staging]
action: ignore
```
Validation errors come back as:
```json
{
"errors": [
{
"path": "$.rules[0].severity",
"msg": "Invalid level 'None'"
}
]
}
```
---
# API & CLI Reference
*Purpose* – give operators and integrators a single, authoritative spec for REST/GRPC calls **and** first‑party CLI tools (`stella-cli`, `zastava`, `stella`).
Everything here is *source‑of‑truth* for generated Swagger/OpenAPI and the `--help` screens in the CLIs.
---
## 0 Quick Glance
| Area | Call / Flag | Notes |
| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ |
| Scan entry | `POST /scan` | Accepts SBOM or image; sub‑5 s target |
| Delta check | `POST /layers/missing` | <20 ms reply; powers *delta SBOM* feature |
| Rate‑limit / quota | — | Headers **`X‑Stella‑Quota‑Remaining`**, **`X‑Stella‑Reset`** on every response |
| Policy I/O | `GET /policy/export`, `POST /policy/import` | YAML now; Rego coming |
| Policy lint | `POST /policy/validate` | Returns 200 OK if ruleset passes |
| Auth | `POST /connect/token` (OpenIddict) | Client‑credentials preferred |
| Health | `GET /healthz` | Simple liveness probe |
| Attestation * | `POST /attest` (TODO Q1‑2026) | SLSA provenance + Rekor log |
| CLI flags | `--sbom-type` `--delta` `--policy-file` | Added to `stella` |
\* Marked **TODO** → delivered after sixth month (kept on Feature Matrix “To Do” list).
---
## 1 Authentication
Stella Ops uses **OAuth 2.0 / OIDC** (token endpoint mounted via OpenIddict).
```
POST /connect/token
Content‑Type: application/x-www-form-urlencoded
grant_type=client_credentials&
client_id=ci‑bot&
client_secret=REDACTED&
scope=stella.api
```
Successful response:
```json
{
"access_token": "eyJraWQi...",
"token_type": "Bearer",
"expires_in": 3600
}
```
> **Tip** – pass the token via `Authorization: Bearer <token>` on every call.
---
## 2 REST API
### 2.0 Obtain / Refresh Offline‑Token
```text
POST /token/offline
Authorization: Bearer <admin‑token>
```
| Body field | Required | Example | Notes |
|------------|----------|---------|-------|
| `expiresDays` | no | `30` | Max 90 days |
```json
{
"jwt": "eyJhbGciOiJSUzI1NiIsInR5cCI6...",
"expires": "2025‑08‑17T00:00:00Z"
}
```
Token is signed with the backend’s private key and already contains
`"maxScansPerDay": {{ quota_token }}`.
### 2.1 Scan – Upload SBOM **or** Image
```
POST /scan
```
| Param / Header | In | Required | Description |
| -------------------- | ------ | -------- | --------------------------------------------------------------------- |
| `X‑Stella‑Sbom‑Type` | header | no | `trivy-json-v2`, `spdx-json`, `cyclonedx-json`; omitted ➞ auto‑detect |
| `?threshold` | query | no | `low`, `medium`, `high`, `critical`; default **critical** |
| body | body | yes | *Either* SBOM JSON *or* Docker image tarball/upload URL |
Every successful `/scan` response now includes:
| Header | Example |
|--------|---------|
| `X‑Stella‑Quota‑Remaining` | `129` |
| `X‑Stella‑Reset` | `2025‑07‑18T23:59:59Z` |
| `X‑Stella‑Token‑Expires` | `2025‑08‑17T00:00:00Z` |
**Response 200** (scan completed):
```json
{
"digest": "sha256:…",
"summary": {
"Critical": 0,
"High": 3,
"Medium": 12,
"Low": 41
},
"policyStatus": "pass",
"quota": {
"remaining": 131,
"reset": "2025-07-18T00:00:00Z"
}
}
```
**Response 202** – queued; polling URL in `Location` header.
---
### 2.2 Delta SBOM – Layer Cache Check
```
POST /layers/missing
Content‑Type: application/json
Authorization: Bearer <token>
```
```json
{
"layers": [
"sha256:d38b...",
"sha256:af45..."
]
}
```
**Response 200** — <20 ms target:
```json
{
"missing": [
"sha256:af45..."
]
}
```
Client then generates SBOM **only** for the `missing` layers and re‑posts `/scan`.
---
### 2.3 Policy Endpoints
| Method | Path | Purpose |
| ------ | ------------------ | ------------------------------------ |
| `GET` | `/policy/export` | Download live YAML ruleset |
| `POST` | `/policy/import` | Upload YAML or Rego; replaces active |
| `POST` | `/policy/validate` | Lint only; returns 400 on error |
| `GET` | `/policy/history` | Paginated change log (audit trail) |
```yaml
# Example import payload (YAML)
version: "1.0"
rules:
- name: Ignore Low dev
severity: [Low, None]
environments: [dev, staging]
action: ignore
```
Validation errors come back as:
```json
{
"errors": [
{
"path": "$.rules[0].severity",
"msg": "Invalid level 'None'"
}
]
}
```
---
### 2.4 Attestation (Planned – Q1‑2026)
```
POST /attest
```
| Param | Purpose |
| ----------- | ------------------------------------- |
| body (JSON) | SLSA v1.0 provenance doc |
| | Signed + stored in local Rekor mirror |
| Param | Purpose |
| ----------- | ------------------------------------- |
| body (JSON) | SLSA v1.0 provenance doc |
| | Signed + stored in local Rekor mirror |
Returns `202 Accepted` and `Location: /attest/{id}` for async verify.
@@ -211,11 +211,14 @@ Configuration follows the same precedence chain everywhere:
| Command | Purpose | Key Flags / Arguments | Notes |
|---------|---------|-----------------------|-------|
| `stellaops-cli scanner download` | Fetch and install scanner container | `--channel <stable\|beta\|nightly>` (default `stable`)<br>`--output <path>`<br>`--overwrite`<br>`--no-install` | Saves artefact under `ScannerCacheDirectory`, verifies digest/signature, and executes `docker load` unless `--no-install` is supplied. |
| `stellaops-cli scan run` | Execute scanner container against a directory (auto-upload) | `--target <directory>` (required)<br>`--runner <docker\|dotnet\|self>` (default from config)<br>`--entry <image-or-entrypoint>`<br>`[scanner-args...]` | Runs the scanner, writes results into `ResultsDirectory`, and automatically uploads the artefact when the exit code is `0`. |
| `stellaops-cli scan run` | Execute scanner container against a directory (auto-upload) | `--target <directory>` (required)<br>`--runner <docker\|dotnet\|self>` (default from config)<br>`--entry <image-or-entrypoint>`<br>`[scanner-args...]` | Runs the scanner, writes results into `ResultsDirectory`, emits a structured `scan-run-*.json` metadata file, and automatically uploads the artefact when the exit code is `0`. |
| `stellaops-cli scan upload` | Re-upload existing scan artefact | `--file <path>` | Useful for retries when automatic upload fails or when operating offline. |
| `stellaops-cli db fetch` | Trigger connector jobs | `--source <id>` (e.g. `redhat`, `osv`)<br>`--stage <fetch\|parse\|map>` (default `fetch`)<br>`--mode <resume|init|cursor>` | Translates to `POST /jobs/source:{source}:{stage}` with `trigger=cli` |
| `stellaops-cli db merge` | Run canonical merge reconcile | — | Calls `POST /jobs/merge:reconcile`; exit code `0` on acceptance, `1` on failures/conflicts |
| `stellaops-cli db export` | Kick JSON / Trivy exports | `--format <json\|trivy-db>` (default `json`)<br>`--delta` | Sets `{ delta = true }` parameter when requested |
| `stellaops-cli db export` | Kick JSON / Trivy exports | `--format <json\|trivy-db>` (default `json`)<br>`--delta`<br>`--publish-full/--publish-delta`<br>`--bundle-full/--bundle-delta` | Sets `{ delta = true }` parameter when requested and can override ORAS/bundle toggles per run |
| `stellaops-cli auth <login\|logout\|status>` | Manage cached tokens for StellaOps Authority | `auth login --force` (ignore cache)<br>`auth status` | Uses `StellaOps.Auth.Client` under the hood; honours `StellaOps:Authority:*` configuration |
When running on an interactive terminal without explicit override flags, the CLI uses Spectre.Console prompts to let you choose per-run ORAS/offline bundle behaviour.
| `stellaops-cli config show` | Display resolved configuration | — | Masks secret values; helpful for air‑gapped installs |
**Logging & exit codes**
@@ -229,12 +232,51 @@ Configuration follows the same precedence chain everywhere:
- Downloads are verified against the `X-StellaOps-Digest` header (SHA-256). When `StellaOps:ScannerSignaturePublicKeyPath` points to a PEM-encoded RSA key, the optional `X-StellaOps-Signature` header is validated as well.
- Metadata for each bundle is written alongside the artefact (`*.metadata.json`) with digest, signature, source URL, and timestamps.
- Retry behaviour is controlled via `StellaOps:ScannerDownloadAttempts` (default **3** with exponential backoff).
- Successful `scan run` executions create timestamped JSON artefacts inside `ResultsDirectory`; these are posted back to Feedser automatically.
- Successful `scan run` executions create timestamped JSON artefacts inside `ResultsDirectory` plus a `scan-run-*.json` metadata envelope documenting the runner, arguments, timing, and stdout/stderr. The artefact is posted back to Feedser automatically.
#### Trivy DB export metadata (`metadata.json`)
`stellaops-cli db export --format trivy-db` (and the backing `POST /jobs/export:trivy-db`) always emits a `metadata.json` document in the OCI layout root. Operators consuming the bundle or delta updates should inspect the following fields:
| Field | Type | Purpose |
| ----- | ---- | ------- |
| `mode` | `full` \| `delta` | Indicates whether the current run rebuilt the entire database (`full`) or only the changed files (`delta`). |
| `baseExportId` | string? | Export ID of the last full baseline that the delta builds upon. Only present for `mode = delta`. |
| `baseManifestDigest` | string? | SHA-256 digest of the manifest belonging to the baseline OCI layout. |
| `resetBaseline` | boolean | `true` when the exporter rotated the baseline (e.g., repo change, delta chain reset). Treat as a full refresh. |
| `treeDigest` | string | Canonical SHA-256 digest of the JSON tree used to build the database. |
| `treeBytes` | number | Total bytes across exported JSON files. |
| `advisoryCount` | number | Count of advisories included in the export. |
| `exporterVersion` | string | Version stamp of `StellaOps.Feedser.Exporter.TrivyDb`. |
| `builder` | object? | Raw metadata emitted by `trivy-db build` (version, update cadence, etc.). |
| `delta.changedFiles[]` | array | Present when `mode = delta`. Each entry lists `{ "path": "<relative json>", "length": <bytes>, "digest": "sha256:..." }`. |
| `delta.removedPaths[]` | array | Paths that existed in the previous manifest but were removed in the new run. |
When the planner opts for a delta run, the exporter copies unmodified blobs from the baseline layout identified by `baseManifestDigest`. Consumers that cache OCI blobs only need to fetch the `changedFiles` and the new manifest/metadata unless `resetBaseline` is true.
When pushing to ORAS, set `feedser:exporters:trivyDb:oras:publishFull` / `publishDelta` to control whether full or delta runs are copied to the registry. Offline bundles follow the analogous `includeFull` / `includeDelta` switches under `offlineBundle`.
Example configuration (`appsettings.yaml`):
```yaml
feedser:
exporters:
trivyDb:
oras:
enabled: true
publishFull: true
publishDelta: false
offlineBundle:
enabled: true
includeFull: true
includeDelta: false
```
**Authentication**
- API key is sent as `Authorization: Bearer <token>` automatically when configured.
- Anonymous operation (empty key) is permitted for offline use cases but backend calls will fail with 401 unless the Feedser instance allows guest access.
- When `StellaOps:Authority:Url` is set the CLI initialises the StellaOps auth client. Use `stellaops-cli auth login` to obtain a token (password grant when `Username`/`Password` are set, otherwise client credentials). Tokens are cached under `~/.stellaops/tokens` by default; `auth status` shows expiry and `auth logout` removes the cached entry.
**Configuration file template**
@@ -247,7 +289,16 @@ Configuration follows the same precedence chain everywhere:
"ResultsDirectory": "results",
"DefaultRunner": "docker",
"ScannerSignaturePublicKeyPath": "",
"ScannerDownloadAttempts": 3
"ScannerDownloadAttempts": 3,
"Authority": {
"Url": "https://authority.example.org",
"ClientId": "feedser-cli",
"ClientSecret": "REDACTED",
"Username": "",
"Password": "",
"Scope": "feedser.jobs.trigger",
"TokenCacheDirectory": ""
}
}
}
```
@@ -256,132 +307,132 @@ Drop `appsettings.local.json` or `.yaml` beside the binary to override per envir
---
### 2.5 Misc Endpoints
| Path | Method | Description |
| ---------- | ------ | ---------------------------- |
| `/healthz` | GET | Liveness; returns `"ok"` |
| `/metrics` | GET | Prometheus exposition (OTel) |
| `/version` | GET | Git SHA + build date |
---
## 3 First‑Party CLI Tools
### 3.1 `stella`
> *Package SBOM + Scan + Exit code* – designed for CI.
```
Usage: stella [OPTIONS] IMAGE_OR_SBOM
```
| Flag / Option | Default | Description |
| --------------- | ----------------------- | -------------------------------------------------- |
| `--server` | `http://localhost:8080` | API root |
| `--token` | *env `STELLA_TOKEN`* | Bearer token |
| `--sbom-type` | *auto* | Force `trivy-json-v2`/`spdx-json`/`cyclonedx-json` |
| `--delta` | `false` | Enable delta layer optimisation |
| `--policy-file` | *none* | Override server rules with local YAML/Rego |
| `--threshold` | `critical` | Fail build if ≥ level found |
| `--output-json` | *none* | Write raw scan result to file |
| `--wait-quota` | `true` | If 429 received, automatically wait `Retry‑After` and retry once. |
**Exit codes**
| Code | Meaning |
| ---- | ------------------------------------------- |
| 0 | Scan OK, policy passed |
| 1 | Vulnerabilities ≥ threshold OR policy block |
| 2 | Internal error (network etc.) |
---
### 3.2 `stella‑zastava`
> *Daemon / K8s DaemonSet* – watch container runtime, push SBOMs.
Core flags (excerpt):
| Flag | Purpose |
| ---------------- | ---------------------------------- |
| `--mode` | `listen` (default) / `enforce` |
| `--filter-image` | Regex; ignore infra/busybox images |
| `--threads` | Worker pool size |
---
### 3.3 `stellopsctl`
> *Admin utility* – policy snapshots, feed status, user CRUD.
Examples:
```
stellopsctl policy export > policies/backup-2025-07-14.yaml
stellopsctl feed refresh # force OSV merge
stellopsctl user add dev-team --role developer
```
---
## 4 Error Model
Uniform problem‑details object (RFC 7807):
```json
{
"type": "https://stella-ops.org/probs/validation",
"title": "Invalid request",
"status": 400,
"detail": "Layer digest malformed",
"traceId": "00-7c39..."
}
```
---
## 5 Rate Limits
Default **40 requests / second / token**.
429 responses include `Retry-After` seconds header.
---
## 6 FAQ & Tips
* **Skip SBOM generation in CI** – supply a *pre‑built* SBOM and add `?sbom-only=true` to `/scan` for <1 s path.
* **Air‑gapped?** – point `--server` to `http://oukgw:8080` inside the Offline Update Kit.
* **YAML vs Rego** – YAML simpler; Rego unlocks time‑based logic (see samples).
* **Cosign verify plug‑ins** – enable `SCANNER_VERIFY_SIG=true` env to refuse unsigned plug‑ins.
---
## 7 Planned Changes (Beyond 6 Months)
These stay in *Feature Matrix → To Do* until design is frozen.
| Epic / Feature | API Impact Sketch |
| ---------------------------- | ---------------------------------- |
| **SLSA L1‑L3** attestation | `/attest` (see §2.4) |
| Rekor transparency log | `/rekor/log/{id}` (GET) |
| Plug‑in Marketplace metadata | `/plugins/market` (catalog) |
| Horizontal scaling controls | `POST /cluster/node` (add/remove) |
| Windows agent support | Update LSAPI to PDE, no API change |
---
## 8 References
* OpenAPI YAML → `/openapi/v1.yaml` (served by backend)
* OAuth2 spec: <https://datatracker.ietf.org/doc/html/rfc6749>
* SLSA spec: <https://slsa.dev/spec/v1.0>
---
## 9 Changelog (truncated)
* **2025‑07‑14** – added *delta SBOM*, policy import/export, CLI `--sbom-type`.
* **2025‑07‑12** – initial public reference.
---
### 2.5 Misc Endpoints
| Path | Method | Description |
| ---------- | ------ | ---------------------------- |
| `/healthz` | GET | Liveness; returns `"ok"` |
| `/metrics` | GET | Prometheus exposition (OTel) |
| `/version` | GET | Git SHA + build date |
---
## 3 First‑Party CLI Tools
### 3.1 `stella`
> *Package SBOM + Scan + Exit code* – designed for CI.
```
Usage: stella [OPTIONS] IMAGE_OR_SBOM
```
| Flag / Option | Default | Description |
| --------------- | ----------------------- | -------------------------------------------------- |
| `--server` | `http://localhost:8080` | API root |
| `--token` | *env `STELLA_TOKEN`* | Bearer token |
| `--sbom-type` | *auto* | Force `trivy-json-v2`/`spdx-json`/`cyclonedx-json` |
| `--delta` | `false` | Enable delta layer optimisation |
| `--policy-file` | *none* | Override server rules with local YAML/Rego |
| `--threshold` | `critical` | Fail build if ≥ level found |
| `--output-json` | *none* | Write raw scan result to file |
| `--wait-quota` | `true` | If 429 received, automatically wait `Retry‑After` and retry once. |
**Exit codes**
| Code | Meaning |
| ---- | ------------------------------------------- |
| 0 | Scan OK, policy passed |
| 1 | Vulnerabilities ≥ threshold OR policy block |
| 2 | Internal error (network etc.) |
---
### 3.2 `stella‑zastava`
> *Daemon / K8s DaemonSet* – watch container runtime, push SBOMs.
Core flags (excerpt):
| Flag | Purpose |
| ---------------- | ---------------------------------- |
| `--mode` | `listen` (default) / `enforce` |
| `--filter-image` | Regex; ignore infra/busybox images |
| `--threads` | Worker pool size |
---
### 3.3 `stellopsctl`
> *Admin utility* – policy snapshots, feed status, user CRUD.
Examples:
```
stellopsctl policy export > policies/backup-2025-07-14.yaml
stellopsctl feed refresh # force OSV merge
stellopsctl user add dev-team --role developer
```
---
## 4 Error Model
Uniform problem‑details object (RFC 7807):
```json
{
"type": "https://stella-ops.org/probs/validation",
"title": "Invalid request",
"status": 400,
"detail": "Layer digest malformed",
"traceId": "00-7c39..."
}
```
---
## 5 Rate Limits
Default **40 requests / second / token**.
429 responses include `Retry-After` seconds header.
---
## 6 FAQ & Tips
* **Skip SBOM generation in CI** – supply a *pre‑built* SBOM and add `?sbom-only=true` to `/scan` for <1 s path.
* **Air‑gapped?** – point `--server` to `http://oukgw:8080` inside the Offline Update Kit.
* **YAML vs Rego** – YAML simpler; Rego unlocks time‑based logic (see samples).
* **Cosign verify plug‑ins** – enable `SCANNER_VERIFY_SIG=true` env to refuse unsigned plug‑ins.
---
## 7 Planned Changes (Beyond 6 Months)
These stay in *Feature Matrix → To Do* until design is frozen.
| Epic / Feature | API Impact Sketch |
| ---------------------------- | ---------------------------------- |
| **SLSA L1‑L3** attestation | `/attest` (see §2.4) |
| Rekor transparency log | `/rekor/log/{id}` (GET) |
| Plug‑in Marketplace metadata | `/plugins/market` (catalog) |
| Horizontal scaling controls | `POST /cluster/node` (add/remove) |
| Windows agent support | Update LSAPI to PDE, no API change |
---
## 8 References
* OpenAPI YAML → `/openapi/v1.yaml` (served by backend)
* OAuth2 spec: <https://datatracker.ietf.org/doc/html/rfc6749>
* SLSA spec: <https://slsa.dev/spec/v1.0>
---
## 9 Changelog (truncated)
* **2025‑07‑14** – added *delta SBOM*, policy import/export, CLI `--sbom-type`.
* **2025‑07‑12** – initial public reference.
---

62
docs/19_TEST_SUITE_OVERVIEW.md
View File

@@ -42,28 +42,46 @@ contributors who need to extend coverage or diagnose failures.
---
## Local runner
```bash
# minimal run: unit + property + frontend tests
./scripts/dev-test.sh
# full stack incl. Playwright and lighthouse
./scripts/dev-test.sh --full
````
The script spins up MongoDB/Redis via Testcontainers and requires:
* Docker ≥ 25
* Node 20 (for Jest/Playwright)
---
## CI job layout
```mermaid
flowchart LR
subgraph fast-path
## Local runner
```bash
# minimal run: unit + property + frontend tests
./scripts/dev-test.sh
# full stack incl. Playwright and lighthouse
./scripts/dev-test.sh --full
````
The script spins up MongoDB/Redis via Testcontainers and requires:
* Docker ≥ 25
* Node 20 (for Jest/Playwright)
---
### Feedser OSV↔GHSA parity fixtures
The Feedser connector suite includes a regression test (`OsvGhsaParityRegressionTests`)
that checks a curated set of GHSA identifiers against OSV responses. The fixture
snapshots live in `src/StellaOps.Feedser.Source.Osv.Tests/Fixtures/` and are kept
deterministic so the parity report remains reproducible.
To refresh the fixtures when GHSA/OSV payloads change:
1. Ensure outbound HTTPS access to `https://api.osv.dev` and `https://api.github.com`.
2. Run `UPDATE_PARITY_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Osv.Tests/StellaOps.Feedser.Source.Osv.Tests.csproj`.
3. Commit the regenerated `osv-ghsa.*.json` files that the test emits (raw snapshots and canonical advisories).
The regen flow logs `[Parity]` messages and normalises `recordedAt` timestamps so the
fixtures stay stable across machines.
---
## CI job layout
```mermaid
flowchart LR
subgraph fast-path
U[xUnit] --> P[FsCheck] --> I1[Testcontainer API]
end

6
docs/ARCHITECTURE_FEEDSER.md
View File

@@ -160,9 +160,9 @@ public interface IFeedConnector {
## 7) Exporters
* JSON exporter mirrors `aquasecurity/vuln-list` layout with deterministic ordering and reproducible timestamps.
* Trivy DB exporter initially shells out to `trivy-db` builder; later will emit BoltDB directly.
* `StellaOps.Feedser.Storage.Mongo` provides cursors for delta exports based on `export_state.exportCursor`.
* Export jobs produce OCI tarballs (layer media type `application/vnd.aquasec.trivy.db.layer.v1.tar+gzip`) and optionally push via ORAS.
* Trivy DB exporter shells out to `trivy-db build`, produces Bolt archives, and reuses unchanged blobs from the last full baseline when running in delta mode. The exporter annotates `metadata.json` with `mode`, `baseExportId`, `baseManifestDigest`, `resetBaseline`, and `delta.changedFiles[]`/`delta.removedPaths[]`, and honours `publishFull` / `publishDelta` (ORAS) plus `includeFull` / `includeDelta` (offline bundle) toggles.
* `StellaOps.Feedser.Storage.Mongo` provides cursors for delta exports based on `export_state.exportCursor` and the persisted per-file manifest (`export_state.files`).
* Export jobs produce OCI tarballs (layer media type `application/vnd.aquasec.trivy.db.layer.v1.tar+gzip`) and optionally push via ORAS; `metadata.json` accompanies each layout so mirrors can decide between full refreshes and deltas.
---

155
docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md Normal file
View File

@@ -0,0 +1,155 @@
# Authority Plug-in Developer Guide
> **Status:** Ready for Docs/DOC4 editorial review as of 2025-10-10. Content aligns with PLG6 acceptance criteria and references stable Authority primitives.
## 1. Overview
Authority plug-ins extend the **StellaOps Authority** service with custom identity providers, credential stores, and client-management logic. Unlike Feedser plug-ins (which ingest or export advisories), Authority plug-ins participate directly in authentication flows:
- **Use cases:** integrate corporate directories (LDAP/AD), delegate to external IDPs, enforce bespoke password/lockout policies, or add client provisioning automation.
- **Constraints:** plug-ins load only during service start (no hot-reload), must function without outbound internet access, and must emit deterministic results for identical configuration and input data.
- **Ship targets:** target the same .NET 10 preview as the host, honour offline-first requirements, and provide clear diagnostics so operators can triage issues from `/ready`.
## 2. Architecture Snapshot
Authority hosts follow a deterministic plug-in lifecycle. The flow below can be rendered as a sequence diagram in the final authored documentation, but all touchpoints are described here for offline viewers:
1. **Configuration load** – `AuthorityPluginConfigurationLoader` resolves YAML manifests under `etc/authority.plugins/`.
2. **Assembly discovery** – the shared `PluginHost` scans `PluginBinaries/Authority` for `StellaOps.Authority.Plugin.*.dll` assemblies.
3. **Registrar execution** – each assembly is searched for `IAuthorityPluginRegistrar` implementations. Registrars bind options, register services, and optionally queue bootstrap tasks.
4. **Runtime** – the host resolves `IIdentityProviderPlugin` instances, uses capability metadata to decide which OAuth grants to expose, and invokes health checks for readiness endpoints.
**Data persistence primer:** the standard Mongo-backed plugin stores users in collections named `authority_users_<pluginName>` and lockout metadata in embedded documents. Additional plugins must document their storage layout and provide deterministic collection naming to honour the Offline Kit replication process.
## 3. Capability Metadata
Capability flags let the host reason about what your plug-in supports:
- Declare capabilities in your descriptor using the string constants from `AuthorityPluginCapabilities` (`password`, `mfa`, `clientProvisioning`, `bootstrap`). The configuration loader now validates these tokens and rejects unknown values at startup.
- `AuthorityIdentityProviderCapabilities.FromCapabilities` projects those strings into strongly typed booleans (`SupportsPassword`, etc.). Authority Core will use these flags when wiring flows such as the password grant. Built-in plugins (e.g., Standard) will fail fast or force-enable required capabilities if the descriptor is misconfigured, so keep manifests accurate.
- Typical configuration (`etc/authority.plugins/standard.yaml`):
```yaml
plugins:
descriptors:
standard:
assemblyName: "StellaOps.Authority.Plugin.Standard"
capabilities:
- password
- bootstrap
```
- Only declare a capability if the plug-in genuinely implements it. For example, if `SupportsClientProvisioning` is `true`, the plug-in must supply a working `IClientProvisioningStore`.
**Operational reminder:** the Authority host surfaces capability summaries during startup (see `AuthorityIdentityProviderRegistry` log lines). Use those logs during smoke tests to ensure manifests align with expectations.
## 4. Project Scaffold
- Target **.NET 10 preview**, enable nullable, treat warnings as errors, and mark Authority plug-ins with `<IsAuthorityPlugin>true</IsAuthorityPlugin>`.
- Minimum references:
- `StellaOps.Authority.Plugins.Abstractions` (contracts & capability helpers)
- `StellaOps.Plugin` (hosting/DI helpers)
- `StellaOps.Auth.*` libraries as needed for shared token utilities (optional today).
- Example `.csproj` (trimmed from `StellaOps.Authority.Plugin.Standard`):
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<IsAuthorityPlugin>true</IsAuthorityPlugin>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Authority.Plugins.Abstractions\StellaOps.Authority.Plugins.Abstractions.csproj" />
<ProjectReference Include="..\..\StellaOps.Plugin\StellaOps.Plugin.csproj" />
</ItemGroup>
</Project>
```
(Add other references—e.g., MongoDB driver, shared auth libraries—according to your implementation.)
## 5. Implementing `IAuthorityPluginRegistrar`
- Create a parameterless registrar class that returns your plug-in type name via `PluginType`.
- Use `AuthorityPluginRegistrationContext` to:
- Bind options (`AddOptions<T>(pluginName).Bind(...)`).
- Register singletons for stores/enrichers using manifest metadata.
- Register any hosted bootstrap tasks (e.g., seed admin users).
- Always validate configuration inside `PostConfigure` and throw meaningful `InvalidOperationException` to fail fast during startup.
- Use the provided `ILoggerFactory` from DI; avoid static loggers or console writes.
- Example skeleton:
```csharp
internal sealed class MyPluginRegistrar : IAuthorityPluginRegistrar
{
public string PluginType => "my-custom";
public void Register(AuthorityPluginRegistrationContext context)
{
var name = context.Plugin.Manifest.Name;
context.Services.AddOptions<MyPluginOptions>(name)
.Bind(context.Plugin.Configuration)
.PostConfigure(opts => opts.Validate(name));
context.Services.AddSingleton<IIdentityProviderPlugin>(sp =>
new MyIdentityProvider(context.Plugin, sp.GetRequiredService<MyCredentialStore>(),
sp.GetRequiredService<MyClaimsEnricher>(),
sp.GetRequiredService<ILogger<MyIdentityProvider>>()));
}
}
```
## 6. Identity Provider Surface
- Implement `IIdentityProviderPlugin` to expose:
- `IUserCredentialStore` for password validation and user CRUD.
- `IClaimsEnricher` to append roles/attributes onto issued principals.
- Optional `IClientProvisioningStore` for machine-to-machine clients.
- `AuthorityIdentityProviderCapabilities` to advertise supported flows.
- Password guidance:
- Prefer Argon2 (Security Guild upcoming recommendation); Standard plug-in currently ships PBKDF2 with easy swap via `IPasswordHasher`.
- Enforce password policies before hashing to avoid storing weak credentials.
- Health checks should probe backing stores (e.g., Mongo `ping`) and return `AuthorityPluginHealthResult` so `/ready` can surface issues.
- When supporting additional factors (e.g., TOTP), implement `SupportsMfa` and document the enrolment flow for resource servers.
## 7. Configuration & Secrets
- Authority looks for manifests under `etc/authority.plugins/`. Each YAML file maps directly to a plug-in name.
- Support environment overrides using `STELLAOPS_AUTHORITY_PLUGINS__DESCRIPTORS__<NAME>__...`.
- Never store raw secrets in git: allow operators to supply them via `.local.yaml`, environment variables, or injected secret files. Document which keys are mandatory.
- Validate configuration as soon as the registrar runs; use explicit error messages to guide operators. The Standard plug-in now enforces complete bootstrap credentials (username + password) and positive lockout windows via `StandardPluginOptions.Validate`.
- Cross-reference bootstrap workflows with `docs/ops/authority_bootstrap.md` (to be published alongside CORE6) so operators can reuse the same payload formats for manual provisioning.
## 8. Logging, Metrics, and Diagnostics
- Always log via the injected `ILogger<T>`; include `pluginName` and correlation IDs where available.
- Activity/metric names should align with `AuthorityTelemetry` constants (`service.name=stellaops-authority`).
- Expose additional diagnostics via structured logging rather than writing custom HTTP endpoints; the host will integrate these into `/health` and `/ready`.
- Emit metrics with stable names (`auth.plugins.<pluginName>.*`) when introducing custom instrumentation; coordinate with the Observability guild to reserve prefixes.
## 9. Testing & Tooling
- Unit tests: use Mongo2Go (or similar) to exercise credential stores without hitting production infrastructure (`StandardUserCredentialStoreTests` is a template).
- Determinism: fix timestamps to UTC and sort outputs consistently; avoid random GUIDs unless stable.
- Smoke tests: launch `dotnet run --project src/StellaOps.Authority/StellaOps.Authority` with your plug-in under `PluginBinaries/Authority` and verify `/ready`.
- Example verification snippet:
```csharp
[Fact]
public async Task VerifyPasswordAsync_ReturnsSuccess()
{
var store = CreateCredentialStore();
await store.UpsertUserAsync(new AuthorityUserRegistration("alice", "Pa55!", null, null, false,
Array.Empty<string>(), new Dictionary<string, string?>()), CancellationToken.None);
var result = await store.VerifyPasswordAsync("alice", "Pa55!", CancellationToken.None);
Assert.True(result.Succeeded);
Assert.True(result.User?.Roles.Count == 0);
}
```
## 10. Packaging & Delivery
- Output assembly should follow `StellaOps.Authority.Plugin.<Name>.dll` so the host’s search pattern picks it up.
- Place the compiled DLL plus dependencies under `PluginBinaries/Authority` for offline deployments; include hashes/signatures in release notes (Security Guild guidance forthcoming).
- Document any external prerequisites (e.g., CA cert bundle) in your plug-in README.
- Update `etc/authority.plugins/<plugin>.yaml` samples and include deterministic SHA256 hashes for optional bootstrap payloads when distributing Offline Kit artefacts.
## 11. Checklist & Handoff
- ✅ Capabilities declared and validated in automated tests.
- ✅ Bootstrap workflows documented (if `bootstrap` capability used) and repeatable.
- ✅ Local smoke test + unit/integration suites green (`dotnet test`).
- ✅ Operational docs updated: configuration keys, secrets guidance, troubleshooting.
- Submit the developer guide update referencing PLG6/DOC4 and tag DevEx + Docs reviewers for sign-off.
---
**Next documentation actions:**
- Add rendered architectural diagram (PlantUML/mermaid) reflecting the lifecycle above once the Docs toolkit pipeline is ready.
- Reference the LDAP RFC (`docs/rfcs/authority-plugin-ldap.md`) in the capability section once review completes.
- Sync terminology with `docs/11_AUTHORITY.md` when that chapter is published to keep glossary terms consistent.

136
docs/rfcs/authority-plugin-ldap.md Normal file
View File

@@ -0,0 +1,136 @@
# RFC: StellaOps.Authority.Plugin.Ldap
**Status:** Draft – for review by Auth Guild, Security Guild, DevEx (2025-10-10)
**Authors:** Plugin Team 4 (Auth Libraries & Identity Providers)
**Related initiatives:** PLG7 backlog, CORE5 event handlers, DOC4 developer guide
## 1. Problem Statement
Many on-prem StellaOps deployments rely on existing LDAP/Active Directory domains for workforce identity. The current Standard Mongo-backed plugin requires duplicating users and secrets, which increases operational overhead and violates corporate policy in some regulated environments. We need a sovereign, offline-friendly LDAP plugin that:
- Supports password grant and bootstrap provisioning flows without storing credentials in Mongo.
- Enforces StellaOps security policies (lockout, password policy hints, audit logging) while delegating credential validation to LDAP.
- Operates deterministically in offline or partially connected environments by caching directory metadata when necessary.
## 2. Goals
- Provide a first-party `StellaOps.Authority.Plugin.Ldap` plugin advertising `password` and optional `clientProvisioning` capabilities at launch.
- Support username/password authentication against LDAP bind operations with configurable DN templates.
- Allow optional bootstrap seeding of service accounts by writing into LDAP (guarded behind explicit configuration) or by mapping to pre-existing entries.
- Surface directory-derived claims (groups, attributes) for downstream authorization via `IClaimsEnricher`.
- Integrate with Authority lockout telemetry and structured logging without persisting secrets locally.
## 3. Non-Goals
- Implement multi-factor authentication out of the box (future enhancement once TOTP/WebAuthn strategy is finalised).
- Provide write-heavy directory management (e.g., user creation workflows) beyond optional bootstrap service account seeding.
- Replace the Standard plugin; both must remain supported and selectable per environment.
## 4. Key Constraints & Assumptions
- Offline-first posture: deployments may operate without outbound internet and with intermittent directory connectivity (e.g., read-only replicas). The plugin must tolerate transient LDAP connectivity failures and degrade gracefully.
- Deterministic behaviour: identical configuration and directory state must yield identical token issuance results. Cached metadata (e.g., group lookups) must have defined expiration.
- Security: No plaintext credential storage; TLS must be enforced for LDAP connections unless explicitly overridden for air-gapped lab environments.
## 5. High-Level Architecture
1. **Configuration binding** (`ldap.yaml`): defines server endpoints, bind strategy, claim mapping, and optional bootstrap overrides.
2. **Connection factory**: pooled LDAP connections using a resilient client (preferred dependency: `Novell.Directory.Ldap.NETStandard`).
3. **Credential validator** (`IUserCredentialStore`): performs bind-as-user flow with optional fallback bind using service account when directories disallow anonymous search.
4. **Claims enricher** (`IClaimsEnricher`): queries group membership/attributes and projects them into canonical roles/claims.
5. **Optional client provisioning** (`IClientProvisioningStore`): maintains machine/service principals either in Mongo (metadata) or via LDAP `serviceConnectionPoint` entries based on configuration.
6. **Health checks**: periodic LDAP `whoami` or `search` probes surfaced through `AuthorityPluginHealthResult`.
```
Authority Host
├── Plugin Manifest (ldap)
├── Registrar → registers ConnectionFactory, LdapCredentialStore, LdapClaimsEnricher
├── Password Grant Handler → CredentialStore.VerifyPasswordAsync → LDAP Bind
└── Claims Pipeline → ClaimsEnricher.EnrichAsync → LDAP group lookup
```
## 6. Configuration Schema (Draft)
```yaml
connection:
host: "ldaps://ldap.example.internal"
port: 636
useStartTls: false
validateCertificates: true
bindDn: "cn=stellaops-bind,ou=service,dc=example,dc=internal"
bindPasswordSecret: "file:/etc/stellaops/secrets/ldap-bind.txt"
searchBase: "dc=example,dc=internal"
usernameAttribute: "uid"
userDnFormat: "uid={username},ou=people,dc=example,dc=internal" # optional template
security:
requireTls: true
allowedCipherSuites: [] # optional allow-list
referralChasing: false
lockout:
useAuthorityPolicies: true # reuse Authority lockout counters
directoryLockoutAttribute: "pwdAccountLockedTime"
claims:
groupAttribute: "memberOf"
groupToRoleMap:
"cn=stellaops-admins,ou=groups,dc=example,dc=internal": "operators"
"cn=stellaops-read,ou=groups,dc=example,dc=internal": "auditors"
extraAttributes:
displayName: "displayName"
email: "mail"
clientProvisioning:
enabled: false
containerDn: "ou=service,dc=example,dc=internal"
secretAttribute: "userPassword"
health:
probeIntervalSeconds: 60
timeoutSeconds: 5
```
## 7. Capability Mapping
| Capability | Implementation Notes |
|------------|---------------------|
| `password` | Bind-as-user validation with Authority lockout integration. Mandatory. |
| `clientProvisioning` | Optional; when enabled, creates/updates LDAP entries for machine clients or stores metadata in Mongo if directory writes are disabled. |
| `bootstrap` | Exposed only when bootstrap manifest provides service account credentials AND directory write permissions are confirmed during startup. |
| `mfa` | Not supported in MVP. Future iteration may integrate TOTP attributes or external MFA providers. |
## 8. Operational Considerations
- **Offline cache:** provide optional Mongo cache for group membership to keep `/ready` responsive if LDAP is temporarily unreachable. Cache entries must include TTL and invalidation hooks.
- **Secrets management:** accept `file:` and environment variable references; integrate with existing `StellaOps.Configuration` secret providers.
- **Observability:** emit structured logs with event IDs (`LDAP_BIND_START`, `LDAP_BIND_FAILURE`, `LDAP_GROUP_LOOKUP`), counters for success/failure, and latency histograms.
- **Throttling:** reuse Authority rate-limiting middleware; add per-connection throttles to avoid saturating directory servers during brute-force attacks.
## 9. Security & Compliance
- Enforce TLS (`ldaps://` or STARTTLS) by default. Provide explicit `allowInsecure` flag gated behind environment variable for lab/testing only.
- Support password hash migration by detecting directory lockout attributes and surfacing `RequiresPasswordReset` when policies demand changes.
- Log distinguished names only at `Debug` level to avoid leaking sensitive structure in default logs.
- Coordinate with Security Guild for penetration testing before GA; incorporate audit log entries for bind attempts and provisioning changes.
## 10. Testing Strategy
- **Unit tests:** mock LDAP connections to validate DN formatting, error mapping, and capability negotiation.
- **Integration tests:** run against an ephemeral OpenLDAP container (seeded via LDIF fixtures) within CI. Include offline cache regression (disconnect LDAP mid-test).
- **Determinism tests:** feed identical LDIF snapshots and configuration to ensure output tokens/claims remain stable across runs.
- **Smoke tests:** `dotnet test` harness plus manual `dotnet run` scenario verifying `/token` password grants and `/internal/users` bootstrap with LDAP-backed store.
## 11. Implementation Plan
1. Scaffold `StellaOps.Authority.Plugin.Ldap` project + tests (net10.0, `<IsAuthorityPlugin>` true).
2. Implement configuration options + validation (mirroring Standard plugin guardrails).
3. Build connection factory + credential store with bind logic.
4. Implement claims enricher and optional cache layer.
5. Add client provisioning store (optional) with toggles for read-only deployments.
6. Wire bootstrapper to validate connectivity/permissions and record findings in startup logs.
7. Extend developer guide with LDAP specifics (post-RFC acceptance).
8. Update Docs and TODO trackers; produce release notes entry once merged.
## 12. Open Questions
- Should client provisioning default to storing metadata in Mongo even when LDAP writes succeed (to preserve audit history)?
- Do we require LDAPS mutual TLS support (client certificates) for regulated environments? If yes, need to extend configuration schema.
- How will we map LDAP groups to Authority scopes/roles when names differ significantly? Consider supporting regex or mapping scripts.
## 13. Timeline (Tentative)
- **Week 1:** RFC review & sign-off.
- **Week 2-3:** Implementation & unit tests.
- **Week 4:** Integration tests + documentation updates.
- **Week 5:** Security review, release candidate packaging.
## 14. Approval
- **Auth Guild Lead:** _TBD_
- **Security Guild Representative:** _TBD_
- **DevEx Docs:** _TBD_
---
Please add comments inline or via PR review. Once approved, track execution under PLG7.