diff --git a/docs/ops/connector-setup-guide.md b/docs/ops/connector-setup-guide.md index bdbd41b38..5ccd9e656 100644 --- a/docs/ops/connector-setup-guide.md +++ b/docs/ops/connector-setup-guide.md @@ -1,111 +1,116 @@ # Connector Setup Guide — Concelier + Excititor -Companion to the UI-driven setup work on 2026-04-22. This guide lists every advisory/VEX connector available in the codebase, whether it needs credentials, and — for the credentialed ones — **exact steps for the operator to create the credentials at the provider's site**. An agent cannot create real accounts on your behalf; you must do these steps yourself with your own accounts, then hand the tokens back so the UI/CLI persists them. +Operator reference for bringing up the Concelier (advisory source) and Excititor (VEX provider) connectors. This guide distinguishes the **aspirational catalog** (~78 entries exposed through `stellaops-cli sources list`) from the **actually backend-wired connectors** that run on ingest today, and lays out credential-creation steps for the few providers that require operator-minted secrets. -## Scope +## Actual state (verified 2026-04-22 against the local dev stack) -- Concelier advisory sources (from `src/Concelier/__Libraries/StellaOps.Concelier.Connector.*`) -- Excititor VEX providers (from `src/Concelier/__Libraries/StellaOps.Excititor.Connectors.*`) -- Already registered in DB on 2026-04-22: Concelier `{auscert, redhat, stella-mirror}`, Excititor `{excititor:redhat, excititor:ubuntu, excititor:cisco, excititor:oracle}` +### Concelier — 9 wired sources, 8 healthy, 787 advisories ingested -## Credential requirements at a glance +| Source | Backend wired? | Health check | Docs in DB | Last updated | +|---|---|---|---|---| +| `redhat` | YES | healthy | **651** | 2026-04-22 17:16 | +| `osv` | YES | healthy (498ms) | 59 | 2026-04-22 17:40 | +| `debian` | YES | healthy | 41 | 2026-04-22 17:07 | +| `suse` | YES | healthy | 26 | 2026-04-22 17:00 | +| `alpine` | YES | healthy | 8 | 2026-04-22 17:06 | +| `ubuntu` | YES | healthy | 2 | 2026-04-22 17:41 | +| `auscert` | YES | healthy | 0 | never | +| `vmware` | YES | healthy | 0 | never | +| `stella-mirror` | YES | unhealthy (404) | 0 | never | -| Connector | Needs creds? | What you need to create | +The `stella-mirror` failure on the dev stack is expected — that source expects a reachable StellaOps infrastructure mirror that's not wired in dev. + +### Excititor — 4 wired providers, all enabled + +| Provider | Kind | Base URIs | |---|---|---| -| **Concelier/GHSA** | YES | GitHub Personal Access Token (classic) with `read:packages` + `read:public_repo` | -| **Concelier/Vndr.Cisco** | YES | Cisco PSIRT openVuln `client_id` + `client_secret` | -| **Concelier/Vndr.Msrc** | YES | Microsoft Entra app `tenant_id` + `client_id` + `client_secret` | -| **Excititor/Cisco.CSAF** | YES | same Cisco PSIRT OAuth creds (can reuse) | -| **Excititor/MSRC.CSAF** | YES | same Microsoft Entra creds (can reuse) | -| All other Concelier connectors | NO | public endpoints | -| All other Excititor connectors | NO | public CSAF / OpenVEX endpoints | +| `excititor:redhat` | distro | _(pulls via CSAF feeds)_ | +| `excititor:ubuntu` | distro | `https://ubuntu.com/security/notices.json` | +| `excititor:cisco` | vendor | `https://www.cisco.com/.well-known/csaf/` (public CSAF — no auth) | +| `excititor:oracle` | vendor | _(pulls via public Oracle Critical Patch Updates)_ | -## Step-by-step — credentialed providers +**Note about Cisco**: the VEX side uses Cisco's public CSAF feed (unauthenticated). The **Concelier advisory** side of Cisco (which WOULD require PSIRT openVuln OAuth) is currently in the aspirational catalog but not backend-wired. -### 1. GitHub Security Advisories (GHSA) +### Aspirational catalog (~65 entries — NOT yet backend-wired) -**What Stella Ops needs**: a GitHub Personal Access Token (classic) to query the GraphQL security-advisories API at scale. +`stellaops-cli sources list` reports 78 entries. The other ~65 (npm, pypi, go, rubygems, maven, crates, packagist, hex, rustsec, pypa, govuln, bundler-audit, exploitdb, metasploit, intel, amd, arm, siemens, kaspersky-ics, cert-ua/pl/in, krcert, fstec-bdu, nkcki, mitre-attack, mitre-d3fend, nuget, poc-github, and more) are present in the static catalog but have no `source_type` → connector mapping in the backend. Calling `stellaops-cli sources enable ` on these returns `[OK]` but does NOT persist the source — the call is a no-op because there's no runtime implementation to register against. -**Steps**: -1. Open (must be logged into your GitHub account). -2. Click **Generate new token** → **Generate new token (classic)**. -3. Name it something like `stella-ops-concelier-ghsa` (note: tokens are per-user, not per-organisation; use a service account if the project has one). -4. Set expiration per your org's policy (**90 days minimum recommended**). -5. Scopes required: - - `read:packages` - - `public_repo` (or `read:public_repo` if you have that fine-grained scope available) -6. Click **Generate token** and copy the `ghp_...` string immediately — GitHub won't show it again. -7. If your GitHub org requires SAML SSO, authorize the token for that org: on the token list page, click **Configure SSO** next to the new token → **Authorize** per org. +Tracked in: +- `docs/implplan/SPRINT_20260422_004_Concelier_full_connector_control_plane.md` (archived — covered Excititor backend; Concelier-catalog mapping remains) +- `docs/implplan/SPRINT_20260422_007_Concelier_excititor_persisted_provider_credentials.md` (open — persisted per-provider configuration) -**Where to paste in Stella Ops**: -- UI: `/setup/integrations` → advisory sources → GHSA → paste token in the PAT field. -- CLI: `stellaops-cli db connectors configure ghsa --credential token=` +## Credential requirements -### 2. Cisco PSIRT openVuln (Concelier/Vndr.Cisco + Excititor/Cisco.CSAF) +Only the following connectors need operator-minted credentials — and **all three are currently in the aspirational catalog only**. You cannot configure them against a running backend until the connector code is wired. Steps are retained here so they're ready when that sprint lands. -**What Stella Ops needs**: a Cisco PSIRT openVuln OAuth 2.0 client (grant_type=client_credentials). Cisco mints these through their developer portal app registration flow. +### GitHub Security Advisories (GHSA) -**Steps**: -1. Open (Cisco developer portal; create/sign into a free Cisco developer account if you don't have one). -2. Click **Register a New App** (or **My Apps & Keys** → **Register a New App**). -3. App details: - - Name: `stella-ops-concelier-psirt` - - OAuth2 Credentials: tick **Client Credentials** grant. - - APIs to grant: **openVuln API**. -4. Submit. Cisco will show the new app's `client_id` and `client_secret` on the detail page. -5. Copy both strings. +**What Stella Ops needs**: a GitHub Personal Access Token (classic). -Cisco ref: +Steps: +1. → **Generate new token (classic)**. +2. Name: `stella-ops-concelier-ghsa`. Expiration: 90 days minimum. +3. Scopes: `read:packages` + `public_repo` (or the fine-grained `read:public_repo` equivalent). +4. If your GitHub org enforces SAML SSO: **Configure SSO** next to the token → authorize per org. -**Where to paste in Stella Ops**: -- UI: `/setup/integrations` → advisory sources → Cisco → paste `clientId` + `clientSecret`. The same credentials are reused by the Excititor Cisco VEX provider. -- CLI: `stellaops-cli db connectors configure cisco --credential clientId= --credential clientSecret=` +### Cisco PSIRT openVuln (Concelier advisory only; VEX uses public CSAF) -### 3. Microsoft MSRC + Microsoft VEX (Concelier/Vndr.Msrc + Excititor/MSRC.CSAF) +**What Stella Ops needs**: a Cisco PSIRT OAuth 2.0 client (grant_type=client_credentials). -**What Stella Ops needs**: a Microsoft Entra (Azure AD) confidential client app with API permission to the MSRC Security Updates API. +Steps: +1. → **Register a New App**. +2. Name: `stella-ops-concelier-psirt`. Tick **Client Credentials** grant. Grant **openVuln API**. +3. Copy `client_id` + `client_secret` from the app detail page. -**Steps**: -1. Open → **App registrations** (must be signed into an Entra tenant; the tenant can be your org or a personal/developer tenant). -2. Click **New registration**. -3. Name: `stella-ops-concelier-msrc`. Supported account types: **Accounts in this organizational directory only** (single tenant). Redirect URI: leave blank. -4. After registration, copy from the Overview page: - - **Directory (tenant) ID** — GUID like `aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee` - - **Application (client) ID** — another GUID -5. Go to **Certificates & secrets** → **New client secret**: - - Description: `stella-ops-concelier-msrc` - - Expiry: 24 months (or per policy) - - Click **Add** and **immediately copy the `Value` column** — it's hidden after you leave the page. -6. Go to **API permissions** → **Add a permission** → **Microsoft APIs** → **Security Updates API** (you may need to type the name; if not visible, enter its Application ID `83b40db2-0d04-4b56-9e77-0e7d76a47d4b` via **APIs my organization uses**). -7. Select **Application permissions** → tick `SecurityUpdates.Read.All` → **Add permissions**. -8. Click **Grant admin consent for ** (requires admin; if you're not admin, have one click this). +Cisco ref: . + +### Microsoft MSRC (Concelier advisory + Excititor VEX — not yet wired for either) + +**What Stella Ops needs**: a Microsoft Entra confidential client app with `SecurityUpdates.Read.All` API permission. + +Steps: +1. → **App registrations** → **New registration**. +2. Name: `stella-ops-concelier-msrc`. Single-tenant. Redirect URI blank. +3. From Overview: copy **Directory (tenant) ID** + **Application (client) ID**. +4. **Certificates & secrets** → **New client secret** → 24-month expiry → copy the `Value` column **immediately**. +5. **API permissions** → **Add a permission** → Security Updates API (App ID `83b40db2-0d04-4b56-9e77-0e7d76a47d4b`) → Application permissions → `SecurityUpdates.Read.All` → Grant admin consent. Microsoft refs: , . -**Where to paste in Stella Ops**: -- UI: `/setup/integrations` → advisory sources → MSRC → `tenantId`, `clientId`, `clientSecret`. -- CLI: `stellaops-cli db connectors configure msrc --credential tenantId= --credential clientId= --credential clientSecret=` +## Day-to-day operator commands -## Unauthenticated connectors — no credentials needed +```bash +# Using the CLI (src/Cli/StellaOps.Cli/bin/Debug/net10.0/StellaOps.Cli.dll): +dotnet StellaOps.Cli.dll sources list --json # catalog view +dotnet StellaOps.Cli.dll sources status # runtime readiness +dotnet StellaOps.Cli.dll sources check # connectivity probe +dotnet StellaOps.Cli.dll sources enable [...] # enable (no-op for unwired) +dotnet StellaOps.Cli.dll sources disable +dotnet StellaOps.Cli.dll db fetch --source --stage fetch # trigger ingest +``` -Just enable them via UI `/setup/integrations` → advisory sources (or Excititor providers) → **Add**: +Environment: +- `STELLAOPS_BACKEND_URL=https://stella-ops.local` (required for `db fetch`) +- Self-signed cert: resolve by either trusting the dev cert system-wide or running the CLI from a container already in the compose network (the CLI currently has no `--insecure` flag). -### Concelier advisory sources -- ACSC, CCCS, CERT-BUND, CERT/CC, CERT-FR, CERT-IN, ICS-CISA, ICS-Kaspersky -- CVE (MITRE), EPSS, KEV, JVN, KISA, NVD (API-key-optional), OSV -- Distro: Alpine, Debian, Red Hat, SUSE, Ubuntu -- Vendor: Adobe, Apple, Chromium, Oracle, VMware -- Regional: RU.BDU (FSTEC), RU.NKCKI +Verify persisted state directly: +```sql +SELECT key, source_type, enabled FROM vuln.sources ORDER BY key; +SELECT s.key, COUNT(d.*) AS docs, MAX(d.updated_at) AS last_update +FROM vuln.sources s LEFT JOIN concelier.source_documents d ON d.source_id = s.id +GROUP BY s.key ORDER BY docs DESC; +SELECT id, kind, enabled, array_to_string(base_uris,' | ') FROM vex.providers ORDER BY id; +``` -### Excititor VEX providers -- Oracle.CSAF, RedHat.CSAF, Ubuntu.CSAF, SUSE.RancherVEXHub, OCI.OpenVEX.Attest +## What the UI workflow looks like -## Retained-secret contract (important) +1. `/setup/integrations` → advisory sources catalog. +2. Each source row shows `enabled`, `readiness` (`ready` / `blocked` / `disabled` / `unsupported`), and the stored configuration schema. +3. `readiness=blocked` means the source is persisted as enabled but is missing required fields (credentials, URIs). The source will be excluded from automatic and manual sync with a blocked-outcome response (contract: `SOURCE_CONFIG_REQUIRED`, delivered via SPRINT_20260422_003 SRC-CREDS-005 work landed earlier today). +4. Paste credentials in the source's detail editor. The server persists them; subsequent reads surface only a masked "secret retained" badge — secrets are never echoed back. +5. Updating an unrelated field doesn't require re-entering the secret; the server detects "no change" and preserves the stored value. -Stella Ops never echoes secrets back to callers once they're persisted. The UI will show something like `••••set••••` or a "secret retained" badge for persisted credentials. Updating an unrelated field on a credentialed source does NOT require re-entering the secret; the server detects "no change" and preserves the stored value. +## Follow-up for this stack -Only provide a new secret when you're **rotating** it. Provide empty string `""` if you want to explicitly **clear** it (which will put the source into `readiness=blocked` state). - -## After you have the credentials - -Hand them back in this turn of the conversation (in a secure channel — these are tokens) and I'll drive the UI to persist them for you. Or paste them yourself through the `/setup/integrations` page. Either way. +- ~65 aspirational catalog entries still need backend connector wiring — tracked in `SPRINT_20260422_004` and follow-up sprints. +- `Concelier.Advisories.Read` policy requires `advisory:read` scope which requires `aoc:verify` scope pairing, and the `stellaops-cli` OAuth client isn't allowed to mint `aoc:verify`. The UI (`stella-ops-ui` client, authorization_code flow) CAN mint it. Result: the CLI can execute the write path (enable, disable, sync) but cannot call the read-only status/catalog endpoints directly — it uses a separate auth path. This is a known asymmetry; documented here for future operators.