Add StellaOps Authority planning artifacts and config templates
This commit is contained in:
root
232
docs/10_FEEDSER_CLI_QUICKSTART.md
Normal file
232
docs/10_FEEDSER_CLI_QUICKSTART.md
Normal file
@@ -0,0 +1,232 @@
|
||||
# 10 · Feedser + CLI Quickstart
|
||||
|
||||
This guide walks through configuring the Feedser web service and the `stellaops-cli`
|
||||
tool so an operator can ingest advisories, merge them, and publish exports from a
|
||||
single workstation. It focuses on deployment-facing surfaces only (configuration,
|
||||
runtime wiring, CLI usage) and leaves connector/internal customization for later.
|
||||
|
||||
---
|
||||
|
||||
## 0 · Prerequisites
|
||||
|
||||
- .NET SDK **10.0.100-preview** (matches `global.json`)
|
||||
- MongoDB instance reachable from the host (local Docker or managed)
|
||||
- `trivy-db` binary on `PATH` for Trivy exports (and `oras` if publishing to OCI)
|
||||
- Plugin assemblies present in `PluginBinaries/` (already included in the repo)
|
||||
- Optional: Docker/Podman runtime if you plan to run scanners locally
|
||||
|
||||
> **Tip** – air-gapped installs should preload `trivy-db` and `oras` binaries into the
|
||||
> runner image since Feedser never fetches them dynamically.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Configure Feedser
|
||||
|
||||
1. Copy the sample config to the expected location (CI/CD pipelines can stamp values
|
||||
into this file during deployment—see the “Deployment automation” note below):
|
||||
|
||||
```bash
|
||||
mkdir -p etc
|
||||
cp etc/feedser.yaml.sample etc/feedser.yaml
|
||||
```
|
||||
|
||||
2. Edit `etc/feedser.yaml` and update the MongoDB DSN (and optional database name).
|
||||
The default template configures plug-in discovery to look in `PluginBinaries/`
|
||||
and disables remote telemetry exporters by default.
|
||||
|
||||
3. (Optional) Override settings via environment variables. All keys are prefixed with
|
||||
`FEEDSER_`. Example:
|
||||
|
||||
```bash
|
||||
export FEEDSER_STORAGE__DSN="mongodb://user:pass@mongo:27017/feedser"
|
||||
export FEEDSER_TELEMETRY__ENABLETRACING=false
|
||||
```
|
||||
|
||||
4. Start the web service from the repository root:
|
||||
|
||||
```bash
|
||||
dotnet run --project src/StellaOps.Feedser.WebService
|
||||
```
|
||||
|
||||
On startup Feedser validates the options, boots MongoDB indexes, loads plug-ins,
|
||||
and exposes:
|
||||
|
||||
- `GET /health` – returns service status and telemetry settings
|
||||
- `GET /ready` – performs a MongoDB `ping`
|
||||
- `GET /jobs` + `POST /jobs/{kind}` – inspect and trigger connector/export jobs
|
||||
|
||||
> **Security note** – authentication is not wired yet; guard the service with
|
||||
> network controls or a reverse proxy until auth middleware ships.
|
||||
|
||||
### Authority companion configuration (preview)
|
||||
|
||||
1. Copy the Authority sample configuration:
|
||||
|
||||
```bash
|
||||
cp etc/authority.yaml.sample etc/authority.yaml
|
||||
```
|
||||
|
||||
2. Update the issuer URL, token lifetimes, and plug-in descriptors to match your
|
||||
environment. Authority expects per-plugin manifests in `etc/authority.plugins/`;
|
||||
sample `standard.yaml` and `ldap.yaml` files are provided as starting points.
|
||||
For air-gapped installs keep the default plug-in binary directory
|
||||
(`../PluginBinaries/Authority`) so packaged plug-ins load without outbound access.
|
||||
|
||||
3. Environment variables prefixed with `STELLAOPS_AUTHORITY_` override individual
|
||||
fields. Example:
|
||||
|
||||
```bash
|
||||
export STELLAOPS_AUTHORITY__ISSUER="https://authority.stella-ops.local"
|
||||
export STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0="/srv/authority/plugins"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2 · Configure the CLI
|
||||
|
||||
The CLI reads configuration from JSON/YAML files *and* environment variables. The
|
||||
defaults live in `src/StellaOps.Cli/appsettings.json` and expect overrides at runtime.
|
||||
|
||||
| Setting | Environment variable | Default | Purpose |
|
||||
| ------- | -------------------- | ------- | ------- |
|
||||
| `BackendUrl` | `STELLAOPS_BACKEND_URL` | _empty_ | Base URL of the Feedser web service |
|
||||
| `ApiKey` | `API_KEY` | _empty_ | Reserved for future auth; keep empty today |
|
||||
| `ScannerCacheDirectory` | `STELLAOPS_SCANNER_CACHE_DIRECTORY` | `scanners` | Local cache folder |
|
||||
| `ResultsDirectory` | `STELLAOPS_RESULTS_DIRECTORY` | `results` | Where scan outputs are written |
|
||||
|
||||
Example bootstrap:
|
||||
|
||||
```bash
|
||||
export STELLAOPS_BACKEND_URL="http://localhost:5000"
|
||||
export STELLAOPS_RESULTS_DIRECTORY="$HOME/.stellaops/results"
|
||||
dotnet run --project src/StellaOps.Cli -- db merge
|
||||
```
|
||||
|
||||
To persist configuration, you can create `stellaops-cli.yaml` next to the binary or
|
||||
rely on environment variables for ephemeral runners.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Operating Workflow
|
||||
|
||||
1. **Trigger connector fetch stages**
|
||||
|
||||
```bash
|
||||
dotnet run --project src/StellaOps.Cli -- db fetch --source osv --stage fetch
|
||||
dotnet run --project src/StellaOps.Cli -- db fetch --source osv --stage parse
|
||||
dotnet run --project src/StellaOps.Cli -- db fetch --source osv --stage map
|
||||
```
|
||||
|
||||
Use `--mode resume` when continuing from a previous window:
|
||||
|
||||
```bash
|
||||
dotnet run --project src/StellaOps.Cli -- db fetch --source redhat --stage fetch --mode resume
|
||||
```
|
||||
|
||||
2. **Merge canonical advisories**
|
||||
|
||||
```bash
|
||||
dotnet run --project src/StellaOps.Cli -- db merge
|
||||
```
|
||||
|
||||
3. **Produce exports**
|
||||
|
||||
```bash
|
||||
# JSON tree (vuln-list style)
|
||||
dotnet run --project src/StellaOps.Cli -- db export --format json
|
||||
|
||||
# Trivy DB (delta example)
|
||||
dotnet run --project src/StellaOps.Cli -- db export --format trivy-db --delta
|
||||
```
|
||||
|
||||
Feedser always produces a deterministic OCI layout. The first run after a clean
|
||||
bootstrap emits a **full** baseline; subsequent `--delta` runs reuse the previous
|
||||
baseline’s blobs when only JSON manifests change. If the exporter detects that a
|
||||
prior delta is still active (i.e., `LastDeltaDigest` is recorded) it automatically
|
||||
upgrades the next run to a full export and resets the baseline so operators never
|
||||
chain deltas indefinitely. The CLI exposes `--publish-full/--publish-delta` (for
|
||||
ORAS pushes) and `--include-full/--include-delta` (for offline bundles) should you
|
||||
need to override the defaults interactively.
|
||||
|
||||
**Smoke-check delta reuse:** after the first baseline completes, run the export a
|
||||
second time with `--delta` and verify that the new directory reports `mode=delta`
|
||||
while reusing the previous layer blob.
|
||||
|
||||
```bash
|
||||
export_root=${FEEDSER_EXPORT_ROOT:-exports/trivy}
|
||||
base=$(ls -1d "$export_root"/* | sort | tail -n2 | head -n1)
|
||||
delta=$(ls -1d "$export_root"/* | sort | tail -n1)
|
||||
|
||||
jq -r '.mode,.baseExportId' "$delta/metadata.json"
|
||||
|
||||
base_manifest=$(jq -r '.manifests[0].digest' "$base/index.json")
|
||||
delta_manifest=$(jq -r '.manifests[0].digest' "$delta/index.json")
|
||||
printf 'baseline manifest: %s\ndelta manifest: %s\n' "$base_manifest" "$delta_manifest"
|
||||
|
||||
layer_digest=$(jq -r '.layers[0].digest' "$base/blobs/sha256/${base_manifest#sha256:}")
|
||||
cmp "$base/blobs/sha256/${layer_digest#sha256:}" \
|
||||
"$delta/blobs/sha256/${layer_digest#sha256:}"
|
||||
```
|
||||
|
||||
`cmp` returning exit code `0` confirms the delta export reuses the baseline’s
|
||||
`db.tar.gz` layer instead of rebuilding it.
|
||||
|
||||
4. **Manage scanners (optional)**
|
||||
|
||||
```bash
|
||||
dotnet run --project src/StellaOps.Cli -- scanner download --channel stable
|
||||
dotnet run --project src/StellaOps.Cli -- scan run --entry scanners/latest/Scanner.dll --target ./sboms
|
||||
dotnet run --project src/StellaOps.Cli -- scan upload --file results/scan-001.json
|
||||
```
|
||||
|
||||
Add `--verbose` to any command for structured console logs. All commands honour
|
||||
`Ctrl+C` cancellation and exit with non-zero status codes when the backend returns
|
||||
a problem document.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Verification Checklist
|
||||
|
||||
- Feedser `/health` returns `"status":"healthy"` and Storage bootstrap is marked
|
||||
complete after startup.
|
||||
- CLI commands return HTTP 202 with a `Location` header (job tracking URL) when
|
||||
triggering Feedser jobs.
|
||||
- Export artefacts are materialised under the configured output directories and
|
||||
their manifests record digests.
|
||||
- MongoDB contains the expected `document`, `dto`, `advisory`, and `export_state`
|
||||
collections after a run.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Deployment Automation
|
||||
|
||||
- Treat `etc/feedser.yaml.sample` as the canonical template. CI/CD should copy it to
|
||||
the deployment artifact and replace placeholders (DSN, telemetry endpoints, cron
|
||||
overrides) with environment-specific secrets.
|
||||
- Keep secret material (Mongo credentials, OTLP tokens) outside of the repository;
|
||||
inject them via secret stores or pipeline variables at stamp time.
|
||||
- When building container images, include `trivy-db` (and `oras` if used) so air-gapped
|
||||
clusters do not need outbound downloads at runtime.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Next Steps
|
||||
|
||||
- Introduce authentication/authorization in the web service before exposing it on
|
||||
shared networks.
|
||||
- Automate the workflow above via CI/CD (compose stack or Kubernetes CronJobs).
|
||||
- Pair with the Feedser connector teams when enabling additional sources so their
|
||||
module-specific requirements are pulled in safely.
|
||||
|
||||
---
|
||||
|
||||
## 6 · Microsoft Authentication Integration (Planned)
|
||||
|
||||
- The Feedser web service will integrate with the Microsoft identity stack (Entra ID)
|
||||
using OAuth 2.0. Expect additional configuration keys for authority URLs, client
|
||||
IDs/secrets, and audience scopes once the implementation lands.
|
||||
- CLI commands already pass `Authorization` headers when credentials are supplied.
|
||||
When auth is enabled, point `stellaops-cli` at the token issuer (client credentials
|
||||
flow) or run it behind a proxy that injects bearer tokens.
|
||||
- Keep network-facing deployments behind reverse proxies or firewalls until the
|
||||
authentication middleware ships and is fully validated.
|
||||
Reference in New Issue
Block a user