stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

Rename Feedser to Concelier

Browse Source
This commit is contained in:
master
2025-10-18 20:04:15 +03:00
parent 7e1b10d3b2
commit 0137856fdb
1208 changed files with 4370 additions and 4370 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/07_HIGH_LEVEL_ARCHITECTURE.md
View File

@@ -36,7 +36,7 @@ It **absorbs** all content from `components.md` so you have a single, authoritat
| **Scanner.Worker** | `stellaops/scanner-worker` | Runs analyzers (OS, Lang: Java/Node/Python/Go/.NET/Rust, Native ELF/PE/Mach‑O, EntryTrace); emits per‑layer SBOMs and composes image SBOMs. | Horizontal; queue‑driven; sharded by layer digest. |
| **Scanner.Sbomer.BuildXPlugin** | `stellaops/sbom-indexer` | BuildKit **generator** for build‑time SBOMs as OCI **referrers**. | CI‑side; ephemeral. |
| **Scanner.Sbomer.DockerImage** | `stellaops/scanner-cli` | CLI‑orchestrated scanner container for post‑build scans. | Local/CI; ephemeral. |
| **Feedser.WebService** | `stellaops/feedser-web` | Vulnerability ingest/normalize/merge/export (JSON + Trivy DB). | HA via Mongo locks. |
| **Concelier.WebService** | `stellaops/concelier-web` | Vulnerability ingest/normalize/merge/export (JSON + Trivy DB). | HA via Mongo locks. |
| **Excititor.WebService** | `stellaops/excititor-web` | VEX ingest/normalize/consensus; conflict retention; exports. | HA via Mongo locks. |
| **Policy Engine** | (in `scanner-web`) | YAML DSL evaluator (waivers, vendor preferences, KEV/EPSS, license, usage‑gating); produces **policy digest**. | In‑process; cache per digest. |
| **Signer** | `stellaops/signer` | **Hard gate:** validates entitlement + release integrity; mints signing cert (Fulcio keyless) or uses KMS; signs DSSE. | Stateless; HPA by QPS. |
@@ -71,7 +71,7 @@ flowchart LR
Auth[Authority (OIDC)\nOpTok (DPoP/mTLS)]
SW[Scanner.WebService]
WK[Scanner.Worker xN]
FEED[Feedser]
FEED[Concelier]
VEX[Excititor]
POL[Policy Engine (in Scanner.Web)]
SGN[Signer\n(entitlement + signing)]
@@ -177,7 +177,7 @@ LS --> IA: PoE (mTLS client cert or JWT with cnf=K_inst), CRL/OCSP/introspect
## 4) Backend evaluation (decider)
### 4.1 Feedser (advisories)
### 4.1 Concelier (advisories)
* Ingests vendor, distro, OSS feeds; normalizes & merges; persists canonical advisories in Mongo; exports **deterministic JSON** and **Trivy DB**.
* Offline kit bundles for air‑gapped sites.
@@ -194,7 +194,7 @@ LS --> IA: PoE (mTLS client cert or JWT with cnf=K_inst), CRL/OCSP/introspect
### 4.4 PASS/FAIL flow
1. SBOM (Inventory / Usage) → join with **Feedser** advisories.
1. SBOM (Inventory / Usage) → join with **Concelier** advisories.
2. Apply **Excititor** consensus (statuses & justifications).
3. Apply **Policy**; compute PASS/FAIL with waiver TTLs.
4. Sign the **final report** (DSSE via **Signer**) and log to **Rekor v2** via **Attestor**.
@@ -349,9 +349,9 @@ services:
image: stellaops/scanner-worker
deploy: { replicas: 4 }
depends_on: [scanner-web]
feedser: { image: stellaops/feedser-web, depends_on: [mongo] }
concelier: { image: stellaops/concelier-web, depends_on: [mongo] }
excititor: { image: stellaops/excititor-web, depends_on: [mongo] }
ui: { image: stellaops/ui, depends_on: [scanner-web, feedser, excititor] }
ui: { image: stellaops/ui, depends_on: [scanner-web, concelier, excititor] }
```
* **Backups:** Mongo dumps; MinIO versioned buckets & replication; Rekor v2 DB snapshots; JWKS/Fulcio/KMS key rotation.

20
docs/09_API_CLI_REFERENCE.md
View File

@@ -200,7 +200,7 @@ Returns `202 Accepted` and `Location: /attest/{id}` for async verify.
## 3 StellaOps CLI (`stellaops-cli`)
The new CLI is built on **System.CommandLine 2.0.0‑beta5** and mirrors the Feedser backend REST API.
The new CLI is built on **System.CommandLine 2.0.0‑beta5** and mirrors the Concelier backend REST API.
Configuration follows the same precedence chain everywhere:
1. Environment variables (e.g. `API_KEY`, `STELLAOPS_BACKEND_URL`, `StellaOps:ApiKey`)
@@ -250,7 +250,7 @@ When running on an interactive terminal without explicit override flags, the CLI
- Downloads are verified against the `X-StellaOps-Digest` header (SHA-256). When `StellaOps:ScannerSignaturePublicKeyPath` points to a PEM-encoded RSA key, the optional `X-StellaOps-Signature` header is validated as well.
- Metadata for each bundle is written alongside the artefact (`*.metadata.json`) with digest, signature, source URL, and timestamps.
- Retry behaviour is controlled via `StellaOps:ScannerDownloadAttempts` (default **3** with exponential backoff).
- Successful `scan run` executions create timestamped JSON artefacts inside `ResultsDirectory` plus a `scan-run-*.json` metadata envelope documenting the runner, arguments, timing, and stdout/stderr. The artefact is posted back to Feedser automatically.
- Successful `scan run` executions create timestamped JSON artefacts inside `ResultsDirectory` plus a `scan-run-*.json` metadata envelope documenting the runner, arguments, timing, and stdout/stderr. The artefact is posted back to Concelier automatically.
#### Trivy DB export metadata (`metadata.json`)
@@ -265,18 +265,18 @@ When running on an interactive terminal without explicit override flags, the CLI
| `treeDigest` | string | Canonical SHA-256 digest of the JSON tree used to build the database. |
| `treeBytes` | number | Total bytes across exported JSON files. |
| `advisoryCount` | number | Count of advisories included in the export. |
| `exporterVersion` | string | Version stamp of `StellaOps.Feedser.Exporter.TrivyDb`. |
| `exporterVersion` | string | Version stamp of `StellaOps.Concelier.Exporter.TrivyDb`. |
| `builder` | object? | Raw metadata emitted by `trivy-db build` (version, update cadence, etc.). |
| `delta.changedFiles[]` | array | Present when `mode = delta`. Each entry lists `{ "path": "<relative json>", "length": <bytes>, "digest": "sha256:..." }`. |
| `delta.removedPaths[]` | array | Paths that existed in the previous manifest but were removed in the new run. |
When the planner opts for a delta run, the exporter copies unmodified blobs from the baseline layout identified by `baseManifestDigest`. Consumers that cache OCI blobs only need to fetch the `changedFiles` and the new manifest/metadata unless `resetBaseline` is true.
When pushing to ORAS, set `feedser:exporters:trivyDb:oras:publishFull` / `publishDelta` to control whether full or delta runs are copied to the registry. Offline bundles follow the analogous `includeFull` / `includeDelta` switches under `offlineBundle`.
When pushing to ORAS, set `concelier:exporters:trivyDb:oras:publishFull` / `publishDelta` to control whether full or delta runs are copied to the registry. Offline bundles follow the analogous `includeFull` / `includeDelta` switches under `offlineBundle`.
Example configuration (`appsettings.yaml`):
```yaml
feedser:
concelier:
exporters:
trivyDb:
oras:
@@ -293,7 +293,7 @@ feedser:
**Authentication**
- API key is sent as `Authorization: Bearer <token>` automatically when configured.
- Anonymous operation is permitted only when Feedser runs with
- Anonymous operation is permitted only when Concelier runs with
`authority.allowAnonymousFallback: true`. This flag is temporary—plan to disable
it before **2025-12-31 UTC** so bearer tokens become mandatory.
@@ -303,7 +303,7 @@ Authority-backed auth workflow:
3. Execute CLI commands as normal—the backend client injects the cached bearer token automatically and retries on transient 401/403 responses with operator guidance.
4. Inspect the cache with `stellaops-cli auth status` (shows expiry, scope, mode) or clear it via `stellaops-cli auth logout`.
5. Run `stellaops-cli auth whoami` to dump token subject, audience, issuer, scopes, and remaining lifetime (verbose mode prints additional claims).
6. Expect Feedser to emit audit logs for each `/jobs*` request showing `subject`,
6. Expect Concelier to emit audit logs for each `/jobs*` request showing `subject`,
`clientId`, `scopes`, `status`, and whether network bypass rules were applied.
Tokens live in `~/.stellaops/tokens` unless `StellaOps:Authority:TokenCacheDirectory` overrides it. Cached tokens are reused offline until they expire; the CLI surfaces clear errors if refresh fails.
@@ -314,7 +314,7 @@ Tokens live in `~/.stellaops/tokens` unless `StellaOps:Authority:TokenCacheDirec
{
"StellaOps": {
"ApiKey": "your-api-token",
"BackendUrl": "https://feedser.example.org",
"BackendUrl": "https://concelier.example.org",
"ScannerCacheDirectory": "scanners",
"ResultsDirectory": "results",
"DefaultRunner": "docker",
@@ -322,11 +322,11 @@ Tokens live in `~/.stellaops/tokens` unless `StellaOps:Authority:TokenCacheDirec
"ScannerDownloadAttempts": 3,
"Authority": {
"Url": "https://authority.example.org",
"ClientId": "feedser-cli",
"ClientId": "concelier-cli",
"ClientSecret": "REDACTED",
"Username": "",
"Password": "",
"Scope": "feedser.jobs.trigger",
"Scope": "concelier.jobs.trigger",
"TokenCacheDirectory": ""
}
}

68
docs/10_FEEDSER_CLI_QUICKSTART.md → docs/10_CONCELIER_CLI_QUICKSTART.md
View File

@@ -1,6 +1,6 @@
# 10 · Feedser + CLI Quickstart
# 10 · Concelier + CLI Quickstart
This guide walks through configuring the Feedser web service and the `stellaops-cli`
This guide walks through configuring the Concelier web service and the `stellaops-cli`
tool so an operator can ingest advisories, merge them, and publish exports from a
single workstation. It focuses on deployment-facing surfaces only (configuration,
runtime wiring, CLI usage) and leaves connector/internal customization for later.
@@ -16,39 +16,39 @@ runtime wiring, CLI usage) and leaves connector/internal customization for later
- Optional: Docker/Podman runtime if you plan to run scanners locally
> **Tip** – air-gapped installs should preload `trivy-db` and `oras` binaries into the
> runner image since Feedser never fetches them dynamically.
> runner image since Concelier never fetches them dynamically.
---
## 1 · Configure Feedser
## 1 · Configure Concelier
1. Copy the sample config to the expected location (CI/CD pipelines can stamp values
into this file during deployment—see the “Deployment automation” note below):
```bash
mkdir -p etc
cp etc/feedser.yaml.sample etc/feedser.yaml
cp etc/concelier.yaml.sample etc/concelier.yaml
```
2. Edit `etc/feedser.yaml` and update the MongoDB DSN (and optional database name).
2. Edit `etc/concelier.yaml` and update the MongoDB DSN (and optional database name).
The default template configures plug-in discovery to look in `PluginBinaries/`
and disables remote telemetry exporters by default.
3. (Optional) Override settings via environment variables. All keys are prefixed with
`FEEDSER_`. Example:
`CONCELIER_`. Example:
```bash
export FEEDSER_STORAGE__DSN="mongodb://user:pass@mongo:27017/feedser"
export FEEDSER_TELEMETRY__ENABLETRACING=false
export CONCELIER_STORAGE__DSN="mongodb://user:pass@mongo:27017/concelier"
export CONCELIER_TELEMETRY__ENABLETRACING=false
```
4. Start the web service from the repository root:
```bash
dotnet run --project src/StellaOps.Feedser.WebService
dotnet run --project src/StellaOps.Concelier.WebService
```
On startup Feedser validates the options, boots MongoDB indexes, loads plug-ins,
On startup Concelier validates the options, boots MongoDB indexes, loads plug-ins,
and exposes:
- `GET /health` – returns service status and telemetry settings
@@ -90,7 +90,7 @@ defaults live in `src/StellaOps.Cli/appsettings.json` and expect overrides at ru
| Setting | Environment variable | Default | Purpose |
| ------- | -------------------- | ------- | ------- |
| `BackendUrl` | `STELLAOPS_BACKEND_URL` | _empty_ | Base URL of the Feedser web service |
| `BackendUrl` | `STELLAOPS_BACKEND_URL` | _empty_ | Base URL of the Concelier web service |
| `ApiKey` | `API_KEY` | _empty_ | Reserved for legacy key auth; leave empty when using Authority |
| `ScannerCacheDirectory` | `STELLAOPS_SCANNER_CACHE_DIRECTORY` | `scanners` | Local cache folder |
| `ResultsDirectory` | `STELLAOPS_RESULTS_DIRECTORY` | `results` | Where scan outputs are written |
@@ -99,7 +99,7 @@ defaults live in `src/StellaOps.Cli/appsettings.json` and expect overrides at ru
| `Authority.ClientSecret` | `STELLAOPS_AUTHORITY_CLIENT_SECRET` | _empty_ | Client secret (omit when using username/password grant) |
| `Authority.Username` | `STELLAOPS_AUTHORITY_USERNAME` | _empty_ | Username for password grant flows |
| `Authority.Password` | `STELLAOPS_AUTHORITY_PASSWORD` | _empty_ | Password for password grant flows |
| `Authority.Scope` | `STELLAOPS_AUTHORITY_SCOPE` | `feedser.jobs.trigger` | OAuth scope requested for backend operations |
| `Authority.Scope` | `STELLAOPS_AUTHORITY_SCOPE` | `concelier.jobs.trigger` | OAuth scope requested for backend operations |
| `Authority.TokenCacheDirectory` | `STELLAOPS_AUTHORITY_TOKEN_CACHE_DIR` | `~/.stellaops/tokens` | Directory that persists cached tokens |
| `Authority.Resilience.EnableRetries` | `STELLAOPS_AUTHORITY_ENABLE_RETRIES` | `true` | Toggle Polly retry handler for Authority HTTP calls |
| `Authority.Resilience.RetryDelays` | `STELLAOPS_AUTHORITY_RETRY_DELAYS` | `1s,2s,5s` | Comma- or space-separated backoff delays (hh:mm:ss) |
@@ -112,7 +112,7 @@ Example bootstrap:
export STELLAOPS_BACKEND_URL="http://localhost:5000"
export STELLAOPS_RESULTS_DIRECTORY="$HOME/.stellaops/results"
export STELLAOPS_AUTHORITY_URL="https://authority.local"
export STELLAOPS_AUTHORITY_CLIENT_ID="feedser-cli"
export STELLAOPS_AUTHORITY_CLIENT_ID="concelier-cli"
export STELLAOPS_AUTHORITY_CLIENT_SECRET="s3cr3t"
dotnet run --project src/StellaOps.Cli -- db merge
@@ -161,7 +161,7 @@ rely on environment variables for ephemeral runners.
dotnet run --project src/StellaOps.Cli -- db export --format trivy-db --delta
```
Feedser always produces a deterministic OCI layout. The first run after a clean
Concelier always produces a deterministic OCI layout. The first run after a clean
bootstrap emits a **full** baseline; subsequent `--delta` runs reuse the previous
baseline’s blobs when only JSON manifests change. If the exporter detects that a
prior delta is still active (i.e., `LastDeltaDigest` is recorded) it automatically
@@ -175,7 +175,7 @@ rely on environment variables for ephemeral runners.
while reusing the previous layer blob.
```bash
export_root=${FEEDSER_EXPORT_ROOT:-exports/trivy}
export_root=${CONCELIER_EXPORT_ROOT:-exports/trivy}
base=$(ls -1d "$export_root"/* | sort | tail -n2 | head -n1)
delta=$(ls -1d "$export_root"/* | sort | tail -n1)
@@ -209,10 +209,10 @@ a problem document.
## 4 · Verification Checklist
- Feedser `/health` returns `"status":"healthy"` and Storage bootstrap is marked
- Concelier `/health` returns `"status":"healthy"` and Storage bootstrap is marked
complete after startup.
- CLI commands return HTTP 202 with a `Location` header (job tracking URL) when
triggering Feedser jobs.
triggering Concelier jobs.
- Export artefacts are materialised under the configured output directories and
their manifests record digests.
- MongoDB contains the expected `document`, `dto`, `advisory`, and `export_state`
@@ -222,7 +222,7 @@ a problem document.
## 5 · Deployment Automation
- Treat `etc/feedser.yaml.sample` as the canonical template. CI/CD should copy it to
- Treat `etc/concelier.yaml.sample` as the canonical template. CI/CD should copy it to
the deployment artifact and replace placeholders (DSN, telemetry endpoints, cron
overrides) with environment-specific secrets.
- Keep secret material (Mongo credentials, OTLP tokens) outside of the repository;
@@ -238,15 +238,15 @@ a problem document.
`authority.enabled: true` while keeping `authority.allowAnonymousFallback: true`
to observe logs, then flip it to `false` before 2025-12-31 UTC to enforce tokens.
- Automate the workflow above via CI/CD (compose stack or Kubernetes CronJobs).
- Pair with the Feedser connector teams when enabling additional sources so their
- Pair with the Concelier connector teams when enabling additional sources so their
module-specific requirements are pulled in safely.
---
## 6 · Authority Integration
- Feedser now authenticates callers through StellaOps Authority using OAuth 2.0
resource server flows. Populate the `authority` block in `feedser.yaml`:
- Concelier now authenticates callers through StellaOps Authority using OAuth 2.0
resource server flows. Populate the `authority` block in `concelier.yaml`:
```yaml
authority:
@@ -254,20 +254,20 @@ a problem document.
allowAnonymousFallback: false # keep true only during the staged rollout window
issuer: "https://authority.example.org"
audiences:
- "api://feedser"
- "api://concelier"
requiredScopes:
- "feedser.jobs.trigger"
clientId: "feedser-jobs"
clientSecretFile: "../secrets/feedser-jobs.secret"
- "concelier.jobs.trigger"
clientId: "concelier-jobs"
clientSecretFile: "../secrets/concelier-jobs.secret"
clientScopes:
- "feedser.jobs.trigger"
- "concelier.jobs.trigger"
bypassNetworks:
- "127.0.0.1/32"
- "::1/128"
```
- Store the client secret outside of source control. Either provide it via
`authority.clientSecret` (environment variable `FEEDSER_AUTHORITY__CLIENTSECRET`)
`authority.clientSecret` (environment variable `CONCELIER_AUTHORITY__CLIENTSECRET`)
or point `authority.clientSecretFile` to a file mounted at runtime.
- Cron jobs running on the same host can keep using the API thanks to the loopback
bypass mask. Add additional CIDR ranges as needed; every bypass is logged.
@@ -275,15 +275,15 @@ a problem document.
variables such as:
```bash
export FEEDSER_AUTHORITY__ENABLED=true
export FEEDSER_AUTHORITY__ALLOWANONYMOUSFALLBACK=false
export FEEDSER_AUTHORITY__ISSUER="https://authority.example.org"
export FEEDSER_AUTHORITY__CLIENTID="feedser-jobs"
export FEEDSER_AUTHORITY__CLIENTSECRETFILE="/var/run/secrets/feedser/authority-client"
export CONCELIER_AUTHORITY__ENABLED=true
export CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=false
export CONCELIER_AUTHORITY__ISSUER="https://authority.example.org"
export CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
export CONCELIER_AUTHORITY__CLIENTSECRETFILE="/var/run/secrets/concelier/authority-client"
```
- CLI commands already pass `Authorization` headers when credentials are supplied.
Configure the CLI with matching Authority settings (`docs/09_API_CLI_REFERENCE.md`)
so that automation can obtain tokens with the same client credentials. Feedser
so that automation can obtain tokens with the same client credentials. Concelier
logs every job request with the client ID, subject (if present), scopes, and
a `bypass` flag so operators can audit cron traffic.

10
docs/11_AUTHORITY.md
View File

@@ -3,14 +3,14 @@
> **Status:** Drafted 2025-10-12 (CORE5B.DOC / DOC1.AUTH) – aligns with Authority revocation store, JWKS rotation, and bootstrap endpoints delivered in Sprint 1.
## 1. Purpose
The **StellaOps Authority** service issues OAuth2/OIDC tokens for every StellaOps module (Feedser, Backend, Agent, Zastava) and exposes the policy controls required in sovereign/offline environments. Authority is built as a minimal ASP.NET host that:
The **StellaOps Authority** service issues OAuth2/OIDC tokens for every StellaOps module (Concelier, Backend, Agent, Zastava) and exposes the policy controls required in sovereign/offline environments. Authority is built as a minimal ASP.NET host that:
- brokers password, client-credentials, and device-code flows through pluggable identity providers;
- persists access/refresh/device tokens in MongoDB with deterministic schemas for replay analysis and air-gapped audit copies;
- distributes revocation bundles and JWKS material so downstream services can enforce lockouts without direct database access;
- offers bootstrap APIs for first-run provisioning and key rotation without redeploying binaries.
Authority is deployed alongside Feedser in air-gapped environments and never requires outbound internet access. All trusted metadata (OpenIddict discovery, JWKS, revocation bundles) is cacheable, signed, and reproducible.
Authority is deployed alongside Concelier in air-gapped environments and never requires outbound internet access. All trusted metadata (OpenIddict discovery, JWKS, revocation bundles) is cacheable, signed, and reproducible.
## 2. Component Architecture
Authority is composed of five cooperating subsystems:
@@ -46,7 +46,7 @@ Authority persists every issued token in MongoDB so operators can audit or revok
- **Expiry maintenance:** `AuthorityTokenStore.DeleteExpiredAsync` prunes non-revoked tokens past their `expiresAt` timestamp. Operators should schedule this in maintenance windows if large volumes of tokens are issued.
### Expectations for resource servers
Resource servers (Feedser WebService, Backend, Agent) **must not** assume in-memory caches are authoritative. They should:
Resource servers (Concelier WebService, Backend, Agent) **must not** assume in-memory caches are authoritative. They should:
- cache `/jwks` and `/revocations/export` responses within configured lifetimes;
- honour `revokedReason` metadata when shaping audit trails;
@@ -73,7 +73,7 @@ Authority centralises revocation in `authority_revocations` with deterministic c
**Consumer guidance:**
1. Mirror `revocation-bundle.json*` alongside Feedser exports. Offline agents fetch both over the existing update channel.
1. Mirror `revocation-bundle.json*` alongside Concelier exports. Offline agents fetch both over the existing update channel.
2. Use bundle `sequence` and `bundleId` to detect replay or monotonicity regressions. Ignore bundles with older sequence numbers unless `bundleId` changes and `issuedAt` advances.
3. Treat `revokedReason` taxonomy as machine-friendly codes (`compromised`, `rotation`, `policy`, `lifecycle`). Translating to human-readable logs is the consumer’s responsibility.
@@ -154,7 +154,7 @@ All administrative calls emit `AuthEventRecord` entries enriched with correlatio
## 9. Operational Checklist
- [ ] Protect the bootstrap API key and disable bootstrap endpoints (`bootstrap.enabled: false`) once initial setup is complete.
- [ ] Schedule `stella auth revoke export` (or `/internal/revocations/export`) at the same cadence as Feedser exports so bundles remain in lockstep.
- [ ] Schedule `stella auth revoke export` (or `/internal/revocations/export`) at the same cadence as Concelier exports so bundles remain in lockstep.
- [ ] Rotate signing keys before expiration; keep at least one retired key until all cached bundles/tokens signed with it have expired.
- [ ] Monitor `/health` and `/ready` plus rate-limiter metrics to detect plugin outages early.
- [ ] Ensure downstream services cache JWKS and revocation bundles within tolerances; stale caches risk accepting revoked tokens.

2
docs/13_SECURITY_POLICY.md
View File

@@ -81,7 +81,7 @@ cosign verify \
## 5 · Private‑feed mirrors 🌐
The **Feedser (vulnerability ingest/merge/export service)** provides signed JSON and Trivy DB snapshots that merge:
The **Concelier (vulnerability ingest/merge/export service)** provides signed JSON and Trivy DB snapshots that merge:
* OSV + GHSA
* (optional) NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU regionals

4
docs/14_GLOSSARY_OF_TERMS.md
View File

@@ -20,7 +20,7 @@ open a PR and append it alphabetically.*
| **ADR** | *Architecture Decision Record* – lightweight Markdown file that captures one irreversible design decision. | ADR template lives at `/docs/adr/` |
| **AIRE** | *AI Risk Evaluator* – optional Plus/Pro plug‑in that suggests mute rules using an ONNX model. | Commercial feature |
| **Azure‑Pipelines** | CI/CD service in Microsoft Azure DevOps. | Recipe in Pipeline Library |
| **BDU** | Russian (FSTEC) national vulnerability database: *База данных уязвимостей*. | Merged with NVD by Feedser (vulnerability ingest/merge/export service) |
| **BDU** | Russian (FSTEC) national vulnerability database: *База данных уязвимостей*. | Merged with NVD by Concelier (vulnerability ingest/merge/export service) |
| **BuildKit** | Modern Docker build engine with caching and concurrency. | Needed for layer cache patterns |
| **CI** | *Continuous Integration* – automated build/test pipeline. | Stella integrates via CLI |
| **Cosign** | Open‑source Sigstore tool that signs & verifies container images **and files**. | Images & OUK tarballs |
@@ -36,7 +36,7 @@ open a PR and append it alphabetically.*
| **Digest (image)** | SHA‑256 hash uniquely identifying a container image or layer. | Pin digests for reproducible builds |
| **Docker‑in‑Docker (DinD)** | Running Docker daemon inside a CI container. | Used in GitHub / GitLab recipes |
| **DTO** | *Data Transfer Object* – C# record serialised to JSON. | Schemas in doc 11 |
| **Feedser** | Vulnerability ingest/merge/export service consolidating OVN, GHSA, NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU feeds into the canonical MongoDB store and export artifacts. | Cron default `0 1 * * *` |
| **Concelier** | Vulnerability ingest/merge/export service consolidating OVN, GHSA, NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU feeds into the canonical MongoDB store and export artifacts. | Cron default `0 1 * * *` |
| **FSTEC** | Russian regulator issuing SOBIT certificates. | Pro GA target |
| **Gitea** | Self‑hosted Git service – mirrors GitHub repo. | OSS hosting |
| **GOST TLS** | TLS cipher‑suites defined by Russian GOST R 34.10‑2012 / 34.11‑2012. | Provided by `OpenSslGost` or CryptoPro |

10
docs/17_SECURITY_HARDENING_GUIDE.md
View File

@@ -145,28 +145,28 @@ cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
| Audit events | Redis stream audit; export daily to SIEM |
| Alert rules | Feed age  ≥ 48 h, P95 wall‑time > 5 s, Redis used memory > 75 % |
###  7.1 Feedser authorization audits
###  7.1 Concelier authorization audits
- Enable the Authority integration for Feedser (`authority.enabled=true`). Keep
- Enable the Authority integration for Concelier (`authority.enabled=true`). Keep
`authority.allowAnonymousFallback` set to `true` only during migration and plan
to disable it before **2025-12-31 UTC** so the `/jobs*` surface always demands
a bearer token.
- Store the Authority client secret using Docker/Kubernetes secrets and point
`authority.clientSecretFile` at the mounted path; the value is read at startup
and never logged.
- Watch the `Feedser.Authorization.Audit` logger. Each entry contains the HTTP
- Watch the `Concelier.Authorization.Audit` logger. Each entry contains the HTTP
status, subject, client ID, scopes, remote IP, and a boolean `bypass` flag
showing whether a network bypass CIDR allowed the request. Configure your SIEM
to alert when unauthenticated requests (`status=401`) appear with
`bypass=true`, or when unexpected scopes invoke job triggers.
Detailed monitoring and response guidance lives in `docs/ops/feedser-authority-audit-runbook.md`.
Detailed monitoring and response guidance lives in `docs/ops/concelier-authority-audit-runbook.md`.
##  8 Update & patch strategy
| Layer | Cadence | Method |
| -------------------- | -------------------------------------------------------- | ------------------------------ |
| Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d |
| Trivy DB | 24 h scheduler via Feedser (vulnerability ingest/merge/export service) | configurable via Feedser scheduler options |
| Trivy DB | 24 h scheduler via Concelier (vulnerability ingest/merge/export service) | configurable via Concelier scheduler options |
| Docker Engine | vendor LTS | distro package manager |
| Host OS | security repos enabled | unattended‑upgrades |

12
docs/19_TEST_SUITE_OVERVIEW.md
View File

@@ -16,7 +16,7 @@ contributors who need to extend coverage or diagnose failures.
| **1. Unit** | `xUnit` (<code>dotnet test</code>) | `*.Tests.csproj` | per PR / push |
| **2. Property‑based** | `FsCheck` | `SbomPropertyTests` | per PR |
| **3. Integration (API)** | `Testcontainers` suite | `test/Api.Integration` | per PR + nightly |
| **4. Integration (DB-merge)** | in-memory Mongo + Redis | `Feedser.Integration` (vulnerability ingest/merge/export service) | per PR |
| **4. Integration (DB-merge)** | in-memory Mongo + Redis | `Concelier.Integration` (vulnerability ingest/merge/export service) | per PR |
| **5. Contract (gRPC)** | `Buf breaking` | `buf.yaml` files | per PR |
| **6. Front‑end unit** | `Jest` | `ui/src/**/*.spec.ts` | per PR |
| **7. Front‑end E2E** | `Playwright` | `ui/e2e/**` | nightly |
@@ -59,17 +59,17 @@ The script spins up MongoDB/Redis via Testcontainers and requires:
---
### Feedser OSV↔GHSA parity fixtures
### Concelier OSV↔GHSA parity fixtures
The Feedser connector suite includes a regression test (`OsvGhsaParityRegressionTests`)
The Concelier connector suite includes a regression test (`OsvGhsaParityRegressionTests`)
that checks a curated set of GHSA identifiers against OSV responses. The fixture
snapshots live in `src/StellaOps.Feedser.Source.Osv.Tests/Fixtures/` and are kept
snapshots live in `src/StellaOps.Concelier.Source.Osv.Tests/Fixtures/` and are kept
deterministic so the parity report remains reproducible.
To refresh the fixtures when GHSA/OSV payloads change:
1. Ensure outbound HTTPS access to `https://api.osv.dev` and `https://api.github.com`.
2. Run `UPDATE_PARITY_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Osv.Tests/StellaOps.Feedser.Source.Osv.Tests.csproj`.
2. Run `UPDATE_PARITY_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Osv.Tests/StellaOps.Concelier.Source.Osv.Tests.csproj`.
3. Commit the regenerated `osv-ghsa.*.json` files that the test emits (raw snapshots and canonical advisories).
The regen flow logs `[Parity]` messages and normalises `recordedAt` timestamps so the
@@ -88,7 +88,7 @@ flowchart LR
I1 --> FE[Jest]
FE --> E2E[Playwright]
E2E --> Lighthouse
Lighthouse --> INTEG2[Feedser]
Lighthouse --> INTEG2[Concelier]
INTEG2 --> LOAD[k6]
LOAD --> CHAOS[pumba]
CHAOS --> RELEASE[Attestation diff]

52
docs/21_INSTALL_GUIDE.md
View File

@@ -76,51 +76,51 @@ UI: [https://\&lt;host\&gt;:8443](https://&lt;host&gt;:8443) (self‑signed cert
> `stella-ops:latest` with the immutable digest printed by
> `docker images --digests`.
### 1.1 · Feedser authority configuration
### 1.1 · Concelier authority configuration
The Feedser container reads configuration from `etc/feedser.yaml` plus
`FEEDSER_` environment variables. To enable the new Authority integration:
The Concelier container reads configuration from `etc/concelier.yaml` plus
`CONCELIER_` environment variables. To enable the new Authority integration:
1. Add the following keys to `.env` (replace values for your environment):
```bash
FEEDSER_AUTHORITY__ENABLED=true
FEEDSER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true # temporary rollout only
FEEDSER_AUTHORITY__ISSUER="https://authority.internal"
FEEDSER_AUTHORITY__AUDIENCES__0="api://feedser"
FEEDSER_AUTHORITY__REQUIREDSCOPES__0="feedser.jobs.trigger"
FEEDSER_AUTHORITY__CLIENTID="feedser-jobs"
FEEDSER_AUTHORITY__CLIENTSECRETFILE="/run/secrets/feedser_authority_client"
FEEDSER_AUTHORITY__BYPASSNETWORKS__0="127.0.0.1/32"
FEEDSER_AUTHORITY__BYPASSNETWORKS__1="::1/128"
FEEDSER_AUTHORITY__RESILIENCE__ENABLERETRIES=true
FEEDSER_AUTHORITY__RESILIENCE__RETRYDELAYS__0="00:00:01"
FEEDSER_AUTHORITY__RESILIENCE__RETRYDELAYS__1="00:00:02"
FEEDSER_AUTHORITY__RESILIENCE__RETRYDELAYS__2="00:00:05"
FEEDSER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK=true
FEEDSER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE="00:10:00"
CONCELIER_AUTHORITY__ENABLED=true
CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true # temporary rollout only
CONCELIER_AUTHORITY__ISSUER="https://authority.internal"
CONCELIER_AUTHORITY__AUDIENCES__0="api://concelier"
CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger"
CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
CONCELIER_AUTHORITY__CLIENTSECRETFILE="/run/secrets/concelier_authority_client"
CONCELIER_AUTHORITY__BYPASSNETWORKS__0="127.0.0.1/32"
CONCELIER_AUTHORITY__BYPASSNETWORKS__1="::1/128"
CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES=true
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0="00:00:01"
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1="00:00:02"
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__2="00:00:05"
CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK=true
CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE="00:10:00"
```
Store the client secret outside source control (Docker secrets, mounted file,
or Kubernetes Secret). Feedser loads the secret during post-configuration, so
or Kubernetes Secret). Concelier loads the secret during post-configuration, so
the value never needs to appear in the YAML template.
Connected sites can keep the retry ladder short (1 s, 2 s, 5 s) so job triggers fail fast when Authority is down. For air‑gapped or intermittently connected deployments, extend `RESILIENCE__OFFLINECACHETOLERANCE` (e.g. `00:30:00`) so cached discovery/JWKS data remains valid while the Offline Kit synchronises upstream changes.
2. Redeploy Feedser:
2. Redeploy Concelier:
```bash
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d feedser
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d concelier
```
3. Tail the logs: `docker compose logs -f feedser`. Successful `/jobs*` calls now
emit `Feedser.Authorization.Audit` entries with `route`, `status`, `subject`,
3. Tail the logs: `docker compose logs -f concelier`. Successful `/jobs*` calls now
emit `Concelier.Authorization.Audit` entries with `route`, `status`, `subject`,
`clientId`, `scopes`, `bypass`, and `remote` fields. 401 denials keep the same
shape—watch for `bypass=True`, which indicates a bypass CIDR accepted an anonymous
call. See `docs/ops/feedser-authority-audit-runbook.md` for a full audit/alerting checklist.
call. See `docs/ops/concelier-authority-audit-runbook.md` for a full audit/alerting checklist.
> **Enforcement deadline** – keep `FEEDSER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true`
> only while validating the rollout. Set it to `false` (and restart Feedser)
> **Enforcement deadline** – keep `CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true`
> only while validating the rollout. Set it to `false` (and restart Concelier)
> before **2025-12-31 UTC** to require tokens in production.
---

4
docs/24_OFFLINE_KIT.md
View File

@@ -18,7 +18,7 @@ completely isolated network:
| **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. |
| **Delta patches** | Daily diff bundles keep size \< 350 MB |
**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `feedser:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
*Scanner core:* C# 12 on **.NET {{ dotnet }}**.
*Imports are idempotent and atomic — no service downtime.*
@@ -110,4 +110,4 @@ See the detailed rules in
* **Install guide:** `/install/#air-gapped`
* **Sovereign mode rationale:** `/sovereign/`
* **Security policy:** `/security/#reporting-a-vulnerability`
* **CERT-Bund snapshots:** `python tools/certbund_offline_snapshot.py --help` (see `docs/ops/feedser-certbund-operations.md`)
* **CERT-Bund snapshots:** `python tools/certbund_offline_snapshot.py --help` (see `docs/ops/concelier-certbund-operations.md`)

6
docs/40_ARCHITECTURE_OVERVIEW.md
View File

@@ -32,7 +32,7 @@ why the system leans *monolith‑plus‑plug‑ins*, and where extension points
graph TD
A(API Gateway)
B1(Scanner Core<br/>.NET latest LTS)
B2(Feedser service\n(vuln ingest/merge/export))
B2(Concelier service\n(vuln ingest/merge/export))
B3(Policy Engine OPA)
C1(Redis 7)
C2(MongoDB 7)
@@ -53,7 +53,7 @@ graph TD
| ---------------------------- | --------------------- | ---------------------------------------------------- |
| **API Gateway** | ASP.NET Minimal API | Auth (JWT), quotas, request routing |
| **Scanner Core** | C# 12, Polly | Layer diffing, SBOM generation, vuln correlation |
| **Feedser (vulnerability ingest/merge/export service)** | C# source-gen workers | Consolidate NVD + regional CVE feeds into the canonical MongoDB store and drive JSON / Trivy DB exports |
| **Concelier (vulnerability ingest/merge/export service)** | C# source-gen workers | Consolidate NVD + regional CVE feeds into the canonical MongoDB store and drive JSON / Trivy DB exports |
| **Policy Engine** | OPA (Rego) | admission decisions, custom org rules |
| **Redis 7** | Key‑DB compatible | LRU cache, quota counters |
| **MongoDB 7** | WiredTiger | SBOM & findings storage |
@@ -121,7 +121,7 @@ Hot‑plugging is deferred until after v 1.0 for security review.
Although the default deployment is a single container, each sub‑service can be
extracted:
* Feedser → standalone cron pod.
* Concelier → standalone cron pod.
* Policy Engine → side‑car (OPA) with gRPC contract.
* ResultSink → queue worker (RabbitMQ or Azure Service Bus).

2
docs/AGENTS.md
View File

@@ -5,7 +5,7 @@ Produce and maintain offline-friendly documentation for StellaOps modules, cover
## Scope Highlights
- Authority docs (`docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, upcoming `docs/11_AUTHORITY.md`).
- Feedser quickstarts, CLI guides, Offline Kit manuals.
- Concelier quickstarts, CLI guides, Offline Kit manuals.
- Release notes and migration playbooks.
## Operating Principles

8
docs/ARCHITECTURE_AUTHORITY.md
View File

@@ -6,7 +6,7 @@
## 0) Mission & boundaries
**Mission.** Provide **fast, local, verifiable** authentication for Stella Ops microservices and tools by minting **very short‑lived** OAuth2/OIDC tokens that are **sender‑constrained** (DPoP or mTLS‑bound). Support RBAC scopes, multi‑tenant claims, and deterministic validation for APIs (Scanner, Signer, Attestor, Excititor, Feedser, UI, CLI, Zastava).
**Mission.** Provide **fast, local, verifiable** authentication for Stella Ops microservices and tools by minting **very short‑lived** OAuth2/OIDC tokens that are **sender‑constrained** (DPoP or mTLS‑bound). Support RBAC scopes, multi‑tenant claims, and deterministic validation for APIs (Scanner, Signer, Attestor, Excititor, Concelier, UI, CLI, Zastava).
**Boundaries.**
@@ -43,7 +43,7 @@
```
iss = https://authority.<domain>
sub = <client_id or user_id>
aud = <service audience: signer|scanner|attestor|feedser|excititor|ui|zastava>
aud = <service audience: signer|scanner|attestor|concelier|excititor|ui|zastava>
exp = <unix ts> (<= 300 s from iat)
iat = <unix ts>
nbf = iat - 30
@@ -140,7 +140,7 @@ plan? = <plan name> // optional hint for UIs; not used for e
### 4.1 Audiences
* `signer` — only the **Signer** service should accept tokens with `aud=signer`.
* `attestor`, `scanner`, `feedser`, `excititor`, `ui`, `zastava` similarly.
* `attestor`, `scanner`, `concelier`, `excititor`, `ui`, `zastava` similarly.
Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their policy.
@@ -154,7 +154,7 @@ Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their p
| `scanner.export` | Scanner.WebService | Export SBOMs |
| `scanner.read` | Scanner.WebService | Read catalog/SBOMs |
| `vex.read` / `vex.admin` | Excititor | Query/operate |
| `feedser.read` / `feedser.export` | Feedser | Query/exports |
| `concelier.read` / `concelier.export` | Concelier | Query/exports |
| `ui.read` / `ui.admin` | UI | View/admin |
| `zastava.emit` / `zastava.enforce` | Scanner/Zastava | Runtime events / admission |

16
docs/ARCHITECTURE_CLI.md
View File

@@ -1,6 +1,6 @@
# component_architecture_cli.md — **Stella Ops CLI** (2025Q4)
> **Scope.** Implementation‑ready architecture for **Stella Ops CLI**: command surface, process model, auth (Authority/DPoP), integration with Scanner/Excititor/Feedser/Signer/Attestor, Buildx plug‑in management, offline kit behavior, packaging, observability, security posture, and CI ergonomics.
> **Scope.** Implementation‑ready architecture for **Stella Ops CLI**: command surface, process model, auth (Authority/DPoP), integration with Scanner/Excititor/Concelier/Signer/Attestor, Buildx plug‑in management, offline kit behavior, packaging, observability, security posture, and CI ergonomics.
---
@@ -18,7 +18,7 @@
* CLI **never** signs; it only calls **Signer**/**Attestor** via backend APIs when needed (e.g., `report --attest`).
* CLI **does not** store long‑lived credentials beyond OS keychain; tokens are **short** (Authority OpToks).
* Heavy work (scanning, merging, policy) is executed **server‑side** (Scanner/Excititor/Feedser).
* Heavy work (scanning, merging, policy) is executed **server‑side** (Scanner/Excititor/Concelier).
---
@@ -76,7 +76,7 @@ src/
### 2.4 Policy & data
* `policy get/set/apply` — fetch active policy, apply staged policy, compute digest.
* `feedser export` — trigger/export canonical JSON or Trivy DB (admin).
* `concelier export` — trigger/export canonical JSON or Trivy DB (admin).
* `excititor export` — trigger/export consensus/raw claims (admin).
### 2.5 Verification
@@ -91,8 +91,8 @@ src/
### 2.7 Offline kit
* `offline kit pull` — fetch latest **Feedser JSON + Trivy DB + Excititor exports** as a tarball from a mirror.
* `offline kit import <tar>` — upload the kit to on‑prem services (Feedser/Excititor).
* `offline kit pull` — fetch latest **Concelier JSON + Trivy DB + Excititor exports** as a tarball from a mirror.
* `offline kit import <tar>` — upload the kit to on‑prem services (Concelier/Excititor).
* `offline kit status` — list current seed versions.
### 2.8 Utilities
@@ -122,7 +122,7 @@ src/
* `scanner` for scan/export/report/diff
* `signer` (indirect; usually backend calls Signer)
* `attestor` for verify
* `feedser`/`excititor` for admin verbs
* `concelier`/`excititor` for admin verbs
CLI rejects verbs if required scopes are missing.
@@ -167,7 +167,7 @@ cli:
backend:
scanner: "https://scanner-web.internal"
attestor: "https://attestor.internal"
feedser: "https://feedser-web.internal"
concelier: "https://concelier-web.internal"
excititor: "https://excititor-web.internal"
auth:
audienceDefault: "scanner"
@@ -263,7 +263,7 @@ Exit code: 2
## 13) Admin & advanced flags
* `--authority`, `--scanner`, `--attestor`, `--feedser`, `--excititor` override config URLs.
* `--authority`, `--scanner`, `--attestor`, `--concelier`, `--excititor` override config URLs.
* `--no-color`, `--quiet`, `--json`.
* `--timeout`, `--retries`, `--retry-backoff-ms`.
* `--ca-bundle`, `--insecure` (dev only; prints warning).

58
docs/ARCHITECTURE_FEEDSER.md → docs/ARCHITECTURE_CONCELIER.md
View File

@@ -1,6 +1,6 @@
# component_architecture_feedser.md — **Stella Ops Feedser** (2025Q4)
# component_architecture_concelier.md — **Stella Ops Concelier** (2025Q4)
> **Scope.** Implementation‑ready architecture for **Feedser**: the vulnerability ingest/normalize/merge/export subsystem that produces deterministic advisory data for the Scanner + Policy + Excititor pipeline. Covers domain model, connectors, merge rules, storage schema, exports, APIs, performance, security, and test matrices.
> **Scope.** Implementation‑ready architecture for **Concelier**: the vulnerability ingest/normalize/merge/export subsystem that produces deterministic advisory data for the Scanner + Policy + Excititor pipeline. Covers domain model, connectors, merge rules, storage schema, exports, APIs, performance, security, and test matrices.
---
@@ -10,15 +10,15 @@
**Boundaries.**
* Feedser **does not** sign with private keys. When attestation is required, the export artifact is handed to the **Signer**/**Attestor** pipeline (out‑of‑process).
* Feedser **does not** decide PASS/FAIL; it provides data to the **Policy** engine.
* Concelier **does not** sign with private keys. When attestation is required, the export artifact is handed to the **Signer**/**Attestor** pipeline (out‑of‑process).
* Concelier **does not** decide PASS/FAIL; it provides data to the **Policy** engine.
* Online operation is **allowlist‑only**; air‑gapped deployments use the **Offline Kit**.
---
## 1) Topology & processes
**Process shape:** single ASP.NET Core service `StellaOps.Feedser.WebService` hosting:
**Process shape:** single ASP.NET Core service `StellaOps.Concelier.WebService` hosting:
* **Scheduler** with distributed locks (Mongo backed).
* **Connectors** (fetch/parse/map).
@@ -32,7 +32,7 @@
## 2) Canonical domain model
> Stored in MongoDB (database `feedser`), serialized with a **canonical JSON** writer (stable order, camelCase, normalized timestamps).
> Stored in MongoDB (database `concelier`), serialized with a **canonical JSON** writer (stable order, camelCase, normalized timestamps).
### 2.1 Core entities
@@ -173,7 +173,7 @@ public interface IFeedConnector {
### 5.1 Keying & identity
* Identity graph: **CVE** is primary node; vendor/distro IDs resolved via **Alias** edges (from connectors and Feedser’s alias tables).
* Identity graph: **CVE** is primary node; vendor/distro IDs resolved via **Alias** edges (from connectors and Concelier’s alias tables).
* `advisoryKey` is the canonical primary key (CVE if present, else vendor/distro key).
### 5.2 Merge algorithm (deterministic)
@@ -255,14 +255,14 @@ public interface IFeedConnector {
### 7.3 Hand‑off to Signer/Attestor (optional)
* On export completion, if `attest: true` is set in job args, Feedser **posts** the artifact metadata to **Signer**/**Attestor**; Feedser itself **does not** hold signing keys.
* On export completion, if `attest: true` is set in job args, Concelier **posts** the artifact metadata to **Signer**/**Attestor**; Concelier itself **does not** hold signing keys.
* Export record stores returned `{ uuid, index, url }` from **Rekor v2**.
---
## 8) REST APIs
All under `/api/v1/feedser`.
All under `/api/v1/concelier`.
**Health & status**
@@ -296,18 +296,18 @@ GET /advisories?scheme=CVE&value=CVE-2025-12345
GET /affected?productKey=pkg:rpm/openssl&limit=100
```
**AuthN/Z:** Authority tokens (OpTok) with roles: `feedser.read`, `feedser.admin`, `feedser.export`.
**AuthN/Z:** Authority tokens (OpTok) with roles: `concelier.read`, `concelier.admin`, `concelier.export`.
---
## 9) Configuration (YAML)
```yaml
feedser:
mongo: { uri: "mongodb://mongo/feedser" }
concelier:
mongo: { uri: "mongodb://mongo/concelier" }
s3:
endpoint: "http://minio:9000"
bucket: "stellaops-feedser"
bucket: "stellaops-concelier"
scheduler:
windowSeconds: 30
maxParallelSources: 4
@@ -337,14 +337,14 @@ feedser:
exporters:
json:
enabled: true
output: s3://stellaops-feedser/json/
output: s3://stellaops-concelier/json/
trivy:
enabled: true
mode: full
output: s3://stellaops-feedser/trivy/
output: s3://stellaops-concelier/trivy/
oras:
enabled: false
repo: ghcr.io/org/feedser
repo: ghcr.io/org/concelier
precedence:
vendorWinsOverDistro: true
distroWinsOverOsv: true
@@ -372,7 +372,7 @@ feedser:
* **Export**: 1M advisories JSON in ≤ 90 s (streamed, zstd), Trivy DB in ≤ 60 s on 8 cores.
* **Memory**: hard cap per job; chunked streaming writers; backpressure to avoid GC spikes.
**Scale pattern**: add Feedser replicas; Mongo scaling via indices and read/write concerns; GridFS only for oversized docs.
**Scale pattern**: add Concelier replicas; Mongo scaling via indices and read/write concerns; GridFS only for oversized docs.
---
@@ -380,13 +380,13 @@ feedser:
* **Metrics**
* `feedser.fetch.docs_total{source}`
* `feedser.fetch.bytes_total{source}`
* `feedser.parse.failures_total{source}`
* `feedser.map.affected_total{source}`
* `feedser.merge.changed_total`
* `feedser.export.bytes{kind}`
* `feedser.export.duration_seconds{kind}`
* `concelier.fetch.docs_total{source}`
* `concelier.fetch.bytes_total{source}`
* `concelier.parse.failures_total{source}`
* `concelier.map.affected_total{source}`
* `concelier.merge.changed_total`
* `concelier.export.bytes{kind}`
* `concelier.export.duration_seconds{kind}`
* **Tracing** around fetch/parse/map/merge/export.
* **Logs**: structured with `source`, `uri`, `docDigest`, `advisoryKey`, `exportId`.
@@ -415,11 +415,11 @@ feedser:
## 15) Operator runbook (quick)
* **Trigger all sources:** `POST /api/v1/feedser/sources/*/trigger`
* **Force full export JSON:** `POST /api/v1/feedser/exports/json { "full": true, "force": true }`
* **Force Trivy DB delta publish:** `POST /api/v1/feedser/exports/trivy { "full": false, "publish": true }`
* **Inspect advisory:** `GET /api/v1/feedser/advisories?scheme=CVE&value=CVE-2025-12345`
* **Pause noisy source:** `POST /api/v1/feedser/sources/osv/pause`
* **Trigger all sources:** `POST /api/v1/concelier/sources/*/trigger`
* **Force full export JSON:** `POST /api/v1/concelier/exports/json { "full": true, "force": true }`
* **Force Trivy DB delta publish:** `POST /api/v1/concelier/exports/trivy { "full": false, "publish": true }`
* **Inspect advisory:** `GET /api/v1/concelier/advisories?scheme=CVE&value=CVE-2025-12345`
* **Pause noisy source:** `POST /api/v1/concelier/sources/osv/pause`
---

26
docs/ARCHITECTURE_DEVOPS.md
View File

@@ -42,7 +42,7 @@ Semantic core + calendar tag:
A release is a **bundle** of image digests + charts + manifests. All services in a bundle are **wire‑compatible**. Mixed minor versions are allowed within a bounded skew:
* **Web UI ↔ backend**: `±1 minor`.
* **Scanner ↔ Policy/Excititor/Feedser**: `±1 minor`.
* **Scanner ↔ Policy/Excititor/Concelier**: `±1 minor`.
* **Authority/Signer/Attestor triangle**: **must** be same minor (crypto and DPoP/mTLS binding rules).
At startup, services **self‑advertise** their semver & channel; the UI surfaces **mismatch warnings**.
@@ -75,7 +75,7 @@ At startup, services **self‑advertise** their semver & channel; the UI surface
* **Static**: linters, codegen checks, protobuf API freeze (backward‑compat tests).
* **Unit/integration**: per‑component, plus **end‑to‑end** flows (scan→vex→policy→sign→attest).
* **Perf SLOs**: hot paths (SBOM compose, diff, export) measured against budgets.
* **Security**: dependency audit vs Feedser export; container hardening tests; minimal caps.
* **Security**: dependency audit vs Concelier export; container hardening tests; minimal caps.
* **Canary cohort**: internal staging + selected customers; one week on **edge** before **stable** tag.
---
@@ -90,7 +90,7 @@ At startup, services **self‑advertise** their semver & channel; the UI surface
**Gating policy**:
* **Core images** (Authority, Scanner, Feedser, Excititor, Attestor, UI): public **read**.
* **Core images** (Authority, Scanner, Concelier, Excititor, Attestor, UI): public **read**.
* **Enterprise add‑ons** (if any) and **pre‑release**: private repos via OAuth2 token service.
> Monetization lever is **signing** (PoE gate), not image pulls, so the core remains simple to consume.
@@ -115,7 +115,7 @@ At startup, services **self‑advertise** their semver & channel; the UI surface
/attest/ DSSE bundles + Rekor proofs
/charts/ Helm charts + values templates
/compose/ docker-compose.yml + .env template
/plugins/ Feedser/Excititor connectors (restart-time)
/plugins/ Concelier/Excititor connectors (restart-time)
/policy/ example policies
/manifest/ release.yaml (see §6.1)
```
@@ -169,7 +169,7 @@ helm install stella stellaops/platform \
--set authority.issuer=https://authority.stella.local \
--set scanner.minio.endpoint=http://minio.stella.local:9000 \
--set scanner.mongo.uri=mongodb://mongo/scanner \
--set feedser.mongo.uri=mongodb://mongo/feedser \
--set concelier.mongo.uri=mongodb://mongo/concelier \
--set excititor.mongo.uri=mongodb://mongo/excititor
```
@@ -185,7 +185,7 @@ helm install stella stellaops/platform \
1. Authority (stateless, dual‑key rotation ready)
2. Signer/Attestor (same minor)
3. Scanner WebService & Workers
4. Feedser, then Excititor (schema migrations are expand/contract)
4. Concelier, then Excititor (schema migrations are expand/contract)
5. UI last
* **DB migrations** are **expand/contract**:
@@ -263,7 +263,7 @@ s3://stellaops/
images/<imgDigest>/usage.cdx.pb
diffs/<old>_<new>/diff.json.zst
attest/<artifactSha256>.dsse.json
feedser/
concelier/
json/<exportId>/...
trivy/<exportId>/...
excititor/
@@ -289,14 +289,14 @@ s3://stellaops/
### 7.4 Mongo retention
* **Scanner**: `runtime.events` use TTL (e.g., 30–90 days); **catalog** permanent.
* **Feedser/Excititor**: raw docs keep **last N windows**; canonical stores permanent.
* **Concelier/Excititor**: raw docs keep **last N windows**; canonical stores permanent.
* **Attestor**: `entries` permanent; `dedupe` TTL 24–48h.
---
## 8) Observability & SLOs (operations)
* **Uptime SLO**: 99.9% for Signer/Authority/Attestor; 99.5% for Scanner WebService; Excititor/Feedser 99.0%.
* **Uptime SLO**: 99.9% for Signer/Authority/Attestor; 99.5% for Scanner WebService; Excititor/Concelier 99.0%.
* **Error budgets**: tracked per month; dashboards show burn rates.
* **Golden signals**:
@@ -324,7 +324,7 @@ Prometheus + OTLP; Grafana dashboards ship in the charts.
* **Vulnerability response**:
* Feedser red‑flag advisories trigger accelerated **stable** patch rollout; UI/CLI “security patch available” notice.
* Concelier red‑flag advisories trigger accelerated **stable** patch rollout; UI/CLI “security patch available” notice.
* **Backups/DR**:
@@ -408,8 +408,8 @@ services:
scanner-worker:
image: registry.stella-ops.org/stellaops/scanner-worker@sha256:...
deploy: { replicas: 4 }
feedser:
image: registry.stella-ops.org/stellaops/feedser@sha256:...
concelier:
image: registry.stella-ops.org/stellaops/concelier@sha256:...
excititor:
image: registry.stella-ops.org/stellaops/excititor@sha256:...
web-ui:
@@ -446,7 +446,7 @@ services:
* `signer.requests_total{result="success"}/minute` > 0 (when scans occur).
* `attestor.submit_latency_seconds{quantile=0.95}` < 0.3.
* `scanner.scan_latency_seconds{quantile=0.95}` < target per image size.
* `feedser.export.duration_seconds` stable; `excititor.consensus.conflicts_total` not exploding after policy changes.
* `concelier.export.duration_seconds` stable; `excititor.consensus.conflicts_total` not exploding after policy changes.
* MinIO `s3_requests_errors_total` near zero; Mongo `opcounters` hit expected baseline.
### Appendix B — Upgrade safety checklist

6
docs/ARCHITECTURE_EXCITITOR.md
View File

@@ -1,6 +1,6 @@
# component_architecture_excititor.md — **Stella Ops Excititor** (2025Q4)
> **Scope.** This document specifies the **Excititor** service: its purpose, trust model, data structures, APIs, plug‑in contracts, storage schema, normalization/consensus algorithms, performance budgets, testing matrix, and how it integrates with Scanner, Policy, Feedser, and the attestation chain. It is implementation‑ready.
> **Scope.** This document specifies the **Excititor** service: its purpose, trust model, data structures, APIs, plug‑in contracts, storage schema, normalization/consensus algorithms, performance budgets, testing matrix, and how it integrates with Scanner, Policy, Concelier, and the attestation chain. It is implementation‑ready.
---
@@ -67,7 +67,7 @@ All exports are **deterministic**, and (optionally) **attested** via DSSE and lo
### 2.1 Vuln identity
* Accepts **CVE**, **GHSA**, vendor IDs (MSRC, RHSA…), distro IDs (DSA/USN/RHSA…) — normalized to `vulnId` with alias sets.
* **Alias graph** maintained (from Feedser) to map vendor/distro IDs → CVE (primary) and to **GHSA** where applicable.
* **Alias graph** maintained (from Concelier) to map vendor/distro IDs → CVE (primary) and to **GHSA** where applicable.
### 2.2 Product identity (`productKey`)
@@ -427,7 +427,7 @@ excititor:
## 14) Integration points
* **Backend Policy Engine** (in Scanner.WebService): calls `POST /resolve` with batched `(purl, vulnId)` pairs to fetch `rollupStatus + sources`.
* **Feedser**: provides alias graph (CVE↔vendor IDs) and may supply VEX‑adjacent metadata (e.g., KEV flag) for policy escalation.
* **Concelier**: provides alias graph (CVE↔vendor IDs) and may supply VEX‑adjacent metadata (e.g., KEV flag) for policy escalation.
* **UI**: VEX explorer screens use `/claims/search` and `/consensus/search`; show conflicts & provenance.
* **CLI**: `stellaops vex export --consensus --since 7d --out vex.json` for audits.

4
docs/ARCHITECTURE_SCANNER.md
View File

@@ -1,6 +1,6 @@
# component_architecture_scanner.md — **Stella Ops Scanner** (2025Q4)
> **Scope.** Implementation‑ready architecture for the **Scanner** subsystem: WebService, Workers, analyzers, SBOM assembly (inventory & usage), per‑layer caching, three‑way diffs, artifact catalog (MinIO+Mongo), attestation hand‑off, and scale/security posture. This document is the contract between the scanning plane and everything else (Policy, Excititor, Feedser, UI, CLI).
> **Scope.** Implementation‑ready architecture for the **Scanner** subsystem: WebService, Workers, analyzers, SBOM assembly (inventory & usage), per‑layer caching, three‑way diffs, artifact catalog (MinIO+Mongo), attestation hand‑off, and scale/security posture. This document is the contract between the scanning plane and everything else (Policy, Excititor, Concelier, UI, CLI).
---
@@ -10,7 +10,7 @@
**Boundaries.**
* Scanner **does not** produce PASS/FAIL. The backend (Policy + Excititor + Feedser) decides presentation and verdicts.
* Scanner **does not** produce PASS/FAIL. The backend (Policy + Excititor + Concelier) decides presentation and verdicts.
* Scanner **does not** keep third‑party SBOM warehouses. It may **bind** to existing attestations for exact hashes.
* Core analyzers are **deterministic** (no fuzzy identity). Optional heuristic plug‑ins (e.g., patch‑presence) run under explicit flags and never contaminate the core SBOM.

14
docs/ARCHITECTURE_UI.md
View File

@@ -1,6 +1,6 @@
# component_architecture_web_ui.md — **Stella Ops Web UI** (2025Q4)
> **Scope.** Implementation‑ready architecture for the **Angular SPA** that operators and developers use to drive Stella Ops. This document defines UX surfaces, module boundaries, data flows, auth, RBAC, real‑time updates, performance targets, i18n/a11y, security headers, testing and deployment. The UI is a *consumer* of backend APIs (Scanner, Policy, Excititor, Feedser, Attestor, Authority) and never performs scanning, merging, or signing on its own.
> **Scope.** Implementation‑ready architecture for the **Angular SPA** that operators and developers use to drive Stella Ops. This document defines UX surfaces, module boundaries, data flows, auth, RBAC, real‑time updates, performance targets, i18n/a11y, security headers, testing and deployment. The UI is a *consumer* of backend APIs (Scanner, Policy, Excititor, Concelier, Attestor, Authority) and never performs scanning, merging, or signing on its own.
---
@@ -10,7 +10,7 @@
* Scans (status, SBOMs, diffs, EntryTrace, attestation).
* Policy management (rules, exemptions, VEX consumption view).
* Vulnerability intel (Feedser status), VEX consensus exploration (Excititor).
* Vulnerability intel (Concelier status), VEX consensus exploration (Excititor).
* Runtime posture (Zastava observer + admission).
* Admin operations (tenants, tokens, quotas, licensing posture).
@@ -42,7 +42,7 @@
├─ runtime/ # Zastava posture, drift events, admission decisions
├─ policy/ # rules editor (YAML/Rego), exemptions, previews
├─ vex/ # VEX explorer (claims, consensus, conflicts)
├─ feedser/ # source health, export cursors, rebuild/export triggers
├─ concelier/ # source health, export cursors, rebuild/export triggers
├─ attest/ # attestation proofs, verification bundles, Rekor links
├─ admin/ # tenants, roles, clients, quotas, licensing posture
└─ plugins/ # route plug-ins (lazy remote modules, governed)
@@ -92,7 +92,7 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
* **Consensus view**: rollup per (vuln, product) with accepted/rejected sources, weights, timestamps.
* **Conflicts**: grid of top conflicts; filters for justification gates failed.
### 3.6 Feedser
### 3.6 Concelier
* **Sources** table: staleness, last run, errors.
* **Advisory search**: by CVE/alias; show normalized affected ranges.
@@ -136,7 +136,7 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
* **`core/http/api-client.ts`** centralizes:
* Base URLs (Scanner, Excititor, Feedser, Attestor).
* Base URLs (Scanner, Excititor, Concelier, Attestor).
* **Retry** policies on idempotent GETs (backoff + jitter).
* **Problem+JSON** parser → uniform error toasts with correlation ID.
* **SSE** helper (EventSource) with auto‑reconnect & backpressure.
@@ -144,7 +144,7 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
* Typed API clients (DTOs in `core/api/models.ts`):
* `ScannerApi`, `PolicyApi`, `ExcititorApi`, `FeedserApi`, `AttestorApi`, `AuthorityApi`.
* `ScannerApi`, `PolicyApi`, `ExcititorApi`, `ConcelierApi`, `AttestorApi`, `AuthorityApi`.
**DTO examples (abbrev):**
@@ -184,7 +184,7 @@ export interface VexConsensus {
* **Huge tables** rendered with **virtual scrolling** (CDK Virtual Scroll); sort/filter performed client‑side for ≤ 20k rows; beyond that, server‑side queries via BOM‑Index endpoints.
* **Component row** shows purl, version, origin (OS pkg / metadata / linker / attested), licenses, and **used** badge (Usage view).
* **Diff**: compact heatmap per layer; clicking opens a right‑pane with evidence: introducing paths, file hashes, VEX notes (from Excititor consensus) and links to advisories (Feedser).
* **Diff**: compact heatmap per layer; clicking opens a right‑pane with evidence: introducing paths, file hashes, VEX notes (from Excititor consensus) and links to advisories (Concelier).
---

8
docs/EXCITITOR_SCORRING.md
View File

@@ -8,7 +8,7 @@ This document tracks the future-looking risk scoring model for Excititor. The ca
* **Gate(VEX_status)**: `affected`/`under_investigation` → 1, `not_affected`/`fixed` → 0. A trusted “not affected” or “fixed” still zeroes the score.
* **W_trust(source)**: normalized policy weight (baseline 0‒1). Policies may opt into >1 boosts for signed vendor feeds once Phase 1 closes.
* **Severity_base**: canonical numeric severity from Feedser (CVSS or org-defined scale).
* **Severity_base**: canonical numeric severity from Concelier (CVSS or org-defined scale).
* **KEV flag**: 0/1 boost when CISA Known Exploited Vulnerabilities applies.
* **EPSS**: probability [0,1]; bounded multiplier.
* **α, β**: configurable coefficients (default α=0.25, β=0.5) stored in policy.
@@ -19,9 +19,9 @@ Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0,
| Phase | Scope | Artifacts |
| --- | --- | --- |
| **Phase 1 – Schema foundations** | Extend Excititor consensus/claims and Feedser canonical advisories with severity, KEV, EPSS, and expose α/β + weight ceilings in policy. | Sprint 7 tasks `EXCITITOR-CORE-02-001`, `EXCITITOR-POLICY-02-001`, `EXCITITOR-STORAGE-02-001`, `FEEDCORE-ENGINE-07-001`. |
| **Phase 1 – Schema foundations** | Extend Excititor consensus/claims and Concelier canonical advisories with severity, KEV, EPSS, and expose α/β + weight ceilings in policy. | Sprint 7 tasks `EXCITITOR-CORE-02-001`, `EXCITITOR-POLICY-02-001`, `EXCITITOR-STORAGE-02-001`, `FEEDCORE-ENGINE-07-001`. |
| **Phase 2 – Deterministic score engine** | Implement a scoring component that executes alongside consensus and persists score envelopes with hashes. | Planned task `EXCITITOR-CORE-02-002` (backlog). |
| **Phase 3 – Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Feedser noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
| **Phase 3 – Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Concelier noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
## Data model (after Phase 1)
@@ -60,7 +60,7 @@ Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0,
## Operational guidance
* **Inputs**: Feedser delivers severity/KEV/EPSS via the advisory event log; Excititor connectors load VEX statements. Policy owns trust tiers and coefficients.
* **Inputs**: Concelier delivers severity/KEV/EPSS via the advisory event log; Excititor connectors load VEX statements. Policy owns trust tiers and coefficients.
* **Processing**: the scoring engine (Phase 2) runs next to consensus, storing results with deterministic hashes so exports and attestations can reference them.
* **Consumption**: WebService/CLI will return consensus plus score; scanners may suppress findings only when policy-authorized VEX gating and signed score envelopes agree.

14
docs/README.md
View File

@@ -36,7 +36,7 @@ Everything here is open‑source and versioned — when you check out a git ta
- **07 – [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)**
- **08 – Module Architecture Dossiers**
- [Scanner](ARCHITECTURE_SCANNER.md)
- [Feedser](ARCHITECTURE_FEEDSER.md)
- [Concelier](ARCHITECTURE_CONCELIER.md)
- [Excititor](ARCHITECTURE_EXCITITOR.md)
- [Signer](ARCHITECTURE_SIGNER.md)
- [Attestor](ARCHITECTURE_ATTESTOR.md)
@@ -47,7 +47,7 @@ Everything here is open‑source and versioned — when you check out a git ta
- [Release & Operations](ARCHITECTURE_DEVOPS.md)
- **09 – [API & CLI Reference](09_API_CLI_REFERENCE.md)**
- **10 – [Plug‑in SDK Guide](10_PLUGIN_SDK_GUIDE.md)**
- **10 – [Feedser CLI Quickstart](10_FEEDSER_CLI_QUICKSTART.md)**
- **10 – [Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)**
- **30 – [Excititor Connector Packaging Guide](dev/30_EXCITITOR_CONNECTOR_GUIDE.md)**
- **30 – Developer Templates**
- [Excititor Connector Skeleton](dev/templates/excititor-connector/)
@@ -67,12 +67,12 @@ Everything here is open‑source and versioned — when you check out a git ta
- **22 – [CI/CD Recipes Library](ci/20_CI_RECIPES.md)**
- **23 – [FAQ](23_FAQ_MATRIX.md)**
- **24 – [Offline Update Kit Admin Guide](24_OFFLINE_KIT.md)**
- **25 – [Feedser Apple Connector Operations](ops/feedser-apple-operations.md)**
- **25 – [Concelier Apple Connector Operations](ops/concelier-apple-operations.md)**
- **26 – [Authority Key Rotation Playbook](ops/authority-key-rotation.md)**
- **27 – [Feedser CCCS Connector Operations](ops/feedser-cccs-operations.md)**
- **28 – [Feedser CISA ICS Connector Operations](ops/feedser-icscisa-operations.md)**
- **29 – [Feedser CERT-Bund Connector Operations](ops/feedser-certbund-operations.md)**
- **30 – [Feedser MSRC Connector – AAD Onboarding](ops/feedser-msrc-operations.md)**
- **27 – [Concelier CCCS Connector Operations](ops/concelier-cccs-operations.md)**
- **28 – [Concelier CISA ICS Connector Operations](ops/concelier-icscisa-operations.md)**
- **29 – [Concelier CERT-Bund Connector Operations](ops/concelier-certbund-operations.md)**
- **30 – [Concelier MSRC Connector – AAD Onboarding](ops/concelier-msrc-operations.md)**
### Legal & licence
- **31 – [Legal & Quota FAQ](29_LEGAL_FAQ_QUOTA.md)**

8
docs/TASKS.md
View File

@@ -5,10 +5,10 @@
| DOC7.README-INDEX | DONE (2025-10-17) | Docs Guild | — | Refresh index docs (docs/README.md + root README) after architecture dossier split and Offline Kit overhaul. | ✅ ToC reflects new component architecture docs; ✅ root README highlights updated doc set; ✅ Offline Kit guide linked correctly. |
| DOC4.AUTH-PDG | REVIEW | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. |
| DOC1.AUTH | DONE (2025-10-12) | Docs Guild, Authority Core | CORE5B.DOC | Draft `docs/11_AUTHORITY.md` covering architecture, configuration, bootstrap flows. | ✅ Architecture + config sections approved by Core; ✅ Samples reference latest options; ✅ Offline note added. |
| DOC3.Feedser-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Feedser authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. |
| DOC5.Feedser-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Feedser-Authority | Produce dedicated Feedser authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. |
| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Feedser conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/ops/feedser-conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Feedser team. |
| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Feedser Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/ops/feedser-authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. |
| DOC3.Concelier-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Concelier authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. |
| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. |
| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Concelier conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/ops/concelier-conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Concelier team. |
| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/ops/concelier-authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. |
> Update statuses (TODO/DOING/REVIEW/DONE/BLOCKED) as progress changes. Keep guides in sync with configuration samples under `etc/`.

2
docs/assets/authority/authority-rate-limit-flow.svg
View File

@@ -101,5 +101,5 @@
<!-- Notes -->
<text class="note" x="40" y="210">429 path → add `authority.client_id`, `authority.remote_ip` tags for dashboards.</text>
<text class="note" x="40" y="240">Lockout path → reuse precedence strategy from Feedser dedup (see DEDUP_CONFLICTS_RESOLUTION_ALGO.md).</text>
<text class="note" x="40" y="240">Lockout path → reuse precedence strategy from Concelier dedup (see DEDUP_CONFLICTS_RESOLUTION_ALGO.md).</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.8 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB

10
docs/feedser-connector-research-20251011.md → docs/concelier-connector-research-20251011.md
View File

@@ -1,4 +1,4 @@
# Feedser Connector Research – 2025-10-11
# Concelier Connector Research – 2025-10-11
Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national/vendor connectors in scope. Use alongside each module’s `TASKS.md` notes.
@@ -8,7 +8,7 @@ Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national
## CCCS (Canada)
- JSON endpoint (`https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=<lang>&content_type=cccs_threat`) returns ~5 100 records per language; `page=<n>` still works for segmented pulls and the earliest `date_created` seen is 2018‑06‑08 (EN) / 2018‑06‑08 (FR). Use an explicit `User-Agent` to avoid 403 responses.
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/ops/feedser-cccs-operations.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/ops/concelier-cccs-operations.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
## CERT-Bund (Germany)
- `https://wid.cert-bund.de/content/public/securityAdvisory/rss` responds 200 without cookies (≈250-item window, German taxonomy). Detail links load an Angular SPA that fetches JSON behind the bootstrap session.
@@ -16,7 +16,7 @@ Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national
- Historical advisories accessible through the SPA search/export endpoints once the `XSRF-TOKEN` cookie (exposed via `GET /portal/api/security/csrf`) is supplied with the `X-XSRF-TOKEN` header:
- `POST /portal/api/securityadvisory/search` (`{"page":N,"size":100,"sort":["published,desc"]}`) pages data back to 2014.
- `GET /portal/api/securityadvisory/export?format=json&from=YYYY-MM-DD` emits JSON bundles suitable for Offline Kit mirrors.
- Locale note: content is German-only; Feedser preserves `language=de` and Docs will publish a CERT-Bund glossary so operators can bridge terminology without machine translation.
- Locale note: content is German-only; Concelier preserves `language=de` and Docs will publish a CERT-Bund glossary so operators can bridge terminology without machine translation.
## KISA / KNVD (Korea)
- `https://knvd.krcert.or.kr/rss/securityInfo.do` and `/rss/securityNotice.do` return UTF-8 RSS (10-item window) with `detailDos.do?IDX=` links. No cookies required for feed fetch.
@@ -24,11 +24,11 @@ Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national
## BDU (Russia / FSTEC)
- Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml/json`) return 403/404; TLS chain requires Russian Trusted Sub CA and WAF expects additional headers.
- Next actions: acquire official PEM chain, point `feedser:httpClients:source.bdu:trustedRootPaths` (or `feedser:sources:bdu:http:trustedRootPaths`) at the Offline Kit PEM, keep `allowInvalidCertificates=false`, script session bootstrap, then capture RSS/HTML schema for parser work.
- Next actions: acquire official PEM chain, point `concelier:httpClients:source.bdu:trustedRootPaths` (or `concelier:sources:bdu:http:trustedRootPaths`) at the Offline Kit PEM, keep `allowInvalidCertificates=false`, script session bootstrap, then capture RSS/HTML schema for parser work.
## NKTsKI / cert.gov.ru (Russia)
- `https://cert.gov.ru/rss/advisories.xml` served via Bitrix returns 403/404 even with `Accept-Language: ru-RU`; TLS chain also requires Russian trust anchors.
- Next actions: source trust store, configure `feedser:httpClients:source.nkcki:trustedRootPaths` (Offline Kit root via `feedser:offline:root`), prepare proxy fallback, and once accessible document taxonomy/retention plus attachment handling.
- Next actions: source trust store, configure `concelier:httpClients:source.nkcki:trustedRootPaths` (Offline Kit root via `concelier:offline:root`), prepare proxy fallback, and once accessible document taxonomy/retention plus attachment handling.
## CISA ICS (United States)
- `curl -I https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` returns HTTP 403 + `x-reference-error` (Akamai). Same for legacy feed paths.

2
docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md
View File

@@ -3,7 +3,7 @@
> **Status:** Updated 2025-10-11 (AUTHPLUG-DOCS-01-001) with lifecycle + limiter diagrams and refreshed rate-limit guidance aligned to PLG6 acceptance criteria.
## 1. Overview
Authority plug-ins extend the **StellaOps Authority** service with custom identity providers, credential stores, and client-management logic. Unlike Feedser plug-ins (which ingest or export advisories), Authority plug-ins participate directly in authentication flows:
Authority plug-ins extend the **StellaOps Authority** service with custom identity providers, credential stores, and client-management logic. Unlike Concelier plug-ins (which ingest or export advisories), Authority plug-ins participate directly in authentication flows:
- **Use cases:** integrate corporate directories (LDAP/AD), delegate to external IDPs, enforce bespoke password/lockout policies, or add client provisioning automation.
- **Constraints:** plug-ins load only during service start (no hot-reload), must function without outbound internet access, and must emit deterministic results for identical configuration and input data.

10
docs/dev/32_AUTH_CLIENT_GUIDE.md
View File

@@ -1,6 +1,6 @@
# StellaOps Auth Client — Integration Guide
> **Status:** Drafted 2025-10-10 as part of LIB5. Consumer teams (Feedser, CLI, Agent) should review before wiring the new options into their configuration surfaces.
> **Status:** Drafted 2025-10-10 as part of LIB5. Consumer teams (Concelier, CLI, Agent) should review before wiring the new options into their configuration surfaces.
The `StellaOps.Auth.Client` library provides a resilient OpenID Connect client for services and tools that talk to **StellaOps Authority**. LIB5 introduced configurable HTTP retry/backoff policies and an offline-fallback window so downstream components stay deterministic even when Authority is briefly unavailable.
@@ -14,7 +14,7 @@ services.AddStellaOpsAuthClient(options =>
options.Authority = configuration["StellaOps:Authority:Url"]!;
options.ClientId = configuration["StellaOps:Authority:ClientId"]!;
options.ClientSecret = configuration["StellaOps:Authority:ClientSecret"];
options.DefaultScopes.Add("feedser.jobs.trigger");
options.DefaultScopes.Add("concelier.jobs.trigger");
options.EnableRetries = true;
options.RetryDelays.Clear();
@@ -53,7 +53,7 @@ Suggested configuration keys (coordinate with consuming teams before finalising)
StellaOps:
Authority:
Url: "https://authority.stella-ops.local"
ClientId: "feedser"
ClientId: "concelier"
ClientSecret: "change-me"
AuthClient:
EnableRetries: true
@@ -72,7 +72,7 @@ STELLAOPS__AUTHORITY__AUTHCLIENT__RETRYDELAYS__0=00:00:02
STELLAOPS__AUTHORITY__AUTHCLIENT__OFFLINECACHETOLERANCE=00:05:00
```
CLI and Feedser teams should expose these knobs once they adopt the auth client.
CLI and Concelier teams should expose these knobs once they adopt the auth client.
## 4. Testing recommendations
@@ -88,4 +88,4 @@ CLI and Feedser teams should expose these knobs once they adopt the auth client.
- [ ] Extend smoke tests to cover Authority outage scenarios.
- [ ] Coordinate with Docs Guild so user-facing quickstarts reference the new knobs.
Once Feedser and CLI integrate these changes, we can mark LIB5 **DONE**; further packaging work is deferred until the backlog reintroduces it.
Once Concelier and CLI integrate these changes, we can mark LIB5 **DONE**; further packaging work is deferred until the backlog reintroduces it.

22
docs/dev/fixtures.md
View File

@@ -1,45 +1,45 @@
# Feedser Fixture Maintenance
# Concelier Fixture Maintenance
Feedser uses a handful of deterministic fixtures to keep connector regressions in check. This guide lists the
Concelier uses a handful of deterministic fixtures to keep connector regressions in check. This guide lists the
fixture sets, where they live, and how to regenerate them safely.
---
## GHSA ↔ OSV parity fixtures
- **Location:** `src/StellaOps.Feedser.Source.Osv.Tests/Fixtures/osv-ghsa.*.json`
- **Location:** `src/StellaOps.Concelier.Source.Osv.Tests/Fixtures/osv-ghsa.*.json`
- **Purpose:** Exercised by `OsvGhsaParityRegressionTests` to ensure OSV + GHSA outputs stay aligned on aliases,
ranges, references, and credits.
- **Regeneration:** Either run the test harness with online regeneration (`UPDATE_PARITY_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Osv.Tests/StellaOps.Feedser.Source.Osv.Tests.csproj`)
- **Regeneration:** Either run the test harness with online regeneration (`UPDATE_PARITY_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Osv.Tests/StellaOps.Concelier.Source.Osv.Tests.csproj`)
or execute the fixture updater (`dotnet run --project tools/FixtureUpdater/FixtureUpdater.csproj`). Both paths
normalise timestamps and canonical ordering.
- **SemVer provenance:** The regenerated fixtures should show `normalizedVersions[].notes` in the
`osv:{ecosystem}:{advisoryId}:{identifier}` shape emitted by `SemVerRangeRuleBuilder`. Confirm the
constraints and notes line up with GHSA/NVD composites before committing.
- **Verification:** Inspect the diff, then re-run `dotnet test src/StellaOps.Feedser.Source.Osv.Tests/StellaOps.Feedser.Source.Osv.Tests.csproj` to confirm parity.
- **Verification:** Inspect the diff, then re-run `dotnet test src/StellaOps.Concelier.Source.Osv.Tests/StellaOps.Concelier.Source.Osv.Tests.csproj` to confirm parity.
## GHSA credit parity fixtures
- **Location:** `src/StellaOps.Feedser.Source.Ghsa.Tests/Fixtures/credit-parity.{ghsa,osv,nvd}.json`
- **Location:** `src/StellaOps.Concelier.Source.Ghsa.Tests/Fixtures/credit-parity.{ghsa,osv,nvd}.json`
- **Purpose:** Exercised by `GhsaCreditParityRegressionTests` to guarantee GHSA/NVD/OSV acknowledgements remain in lockstep.
- **Regeneration:** `dotnet run --project tools/FixtureUpdater/FixtureUpdater.csproj` rewrites all three canonical snapshots.
- **Verification:** `dotnet test src/StellaOps.Feedser.Source.Ghsa.Tests/StellaOps.Feedser.Source.Ghsa.Tests.csproj`.
- **Verification:** `dotnet test src/StellaOps.Concelier.Source.Ghsa.Tests/StellaOps.Concelier.Source.Ghsa.Tests.csproj`.
> Always commit fixture changes together with the code that motivated them and reference the regression test that guards the behaviour.
## Apple security update fixtures
- **Location:** `src/StellaOps.Feedser.Source.Vndr.Apple.Tests/Apple/Fixtures/*.html` and `.expected.json`.
- **Location:** `src/StellaOps.Concelier.Source.Vndr.Apple.Tests/Apple/Fixtures/*.html` and `.expected.json`.
- **Purpose:** Exercised by `AppleLiveRegressionTests` to guarantee the Apple HTML parser and mapper stay deterministic while covering Rapid Security Responses and multi-device advisories.
- **Regeneration:** Use the helper scripts (`scripts/update-apple-fixtures.sh` or `scripts/update-apple-fixtures.ps1`). They export `UPDATE_APPLE_FIXTURES=1`, propagate the flag through `WSLENV`, touch `.update-apple-fixtures`, and then run the Apple test project. This keeps WSL/VSCode test invocations in sync while the refresh workflow fetches live Apple support pages, sanitises them, and rewrites both the HTML and expected DTO snapshots with normalised ordering.
- **Verification:** Inspect the generated diffs and re-run `dotnet test src/StellaOps.Feedser.Source.Vndr.Apple.Tests/StellaOps.Feedser.Source.Vndr.Apple.Tests.csproj` without the env var to confirm determinism.
- **Verification:** Inspect the generated diffs and re-run `dotnet test src/StellaOps.Concelier.Source.Vndr.Apple.Tests/StellaOps.Concelier.Source.Vndr.Apple.Tests.csproj` without the env var to confirm determinism.
> **Tip for other connector owners:** mirror the sentinel + `WSLENV` pattern (`touch .update-<connector>-fixtures`, append the env var via `WSLENV`) when you add fixture refresh scripts so contributors running under WSL inherit the regeneration flag automatically.
## KISA advisory fixtures
- **Location:** `src/StellaOps.Feedser.Source.Kisa.Tests/Fixtures/kisa-{feed,detail}.(xml|json)`
- **Location:** `src/StellaOps.Concelier.Source.Kisa.Tests/Fixtures/kisa-{feed,detail}.(xml|json)`
- **Purpose:** Used by `KisaConnectorTests` to verify Hangul-aware fetch → parse → map flows and to assert telemetry counters stay wired.
- **Regeneration:** `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Kisa.Tests/StellaOps.Feedser.Source.Kisa.Tests.csproj`
- **Regeneration:** `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Kisa.Tests/StellaOps.Concelier.Source.Kisa.Tests.csproj`
- **Verification:** Re-run the same test suite without the env var; confirm advisory content remains NFC-normalised and HTML is sanitised. Metrics assertions will fail if counters drift.
- **Localisation note:** RSS `category` values (e.g. `취약점정보`) remain in Hangul—do not translate them in fixtures; they feed directly into metrics/log tags.

4
docs/dev/kisa_connector_notes.md
View File

@@ -4,7 +4,7 @@ The KISA/KNVD connector now ships with structured telemetry, richer logging, and
## Telemetry counters
All metrics are emitted from `KisaDiagnostics` (`Meter` name `StellaOps.Feedser.Source.Kisa`).
All metrics are emitted from `KisaDiagnostics` (`Meter` name `StellaOps.Concelier.Source.Kisa`).
| Metric | Description | Tags |
| --- | --- | --- |
@@ -39,7 +39,7 @@ The messages use structured properties (`Idx`, `Category`, `DocumentId`, `Severi
- Hangul fields (`title`, `summary`, `category`, `reference.label`, product vendor/name) are normalised to NFC before storage. Sample category `취약점정보` roughly translates to “vulnerability information”.
- Advisory HTML is sanitised via `HtmlContentSanitizer`, stripping script/style while preserving inline anchors for translation pipelines.
- Metrics carry Hangul `category` tags and logging keeps Hangul strings intact; this ensures air-gapped operators can validate native-language content without relying on MT.
- Fixtures live under `src/StellaOps.Feedser.Source.Kisa.Tests/Fixtures/`. Regenerate with `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Kisa.Tests/StellaOps.Feedser.Source.Kisa.Tests.csproj`.
- Fixtures live under `src/StellaOps.Concelier.Source.Kisa.Tests/Fixtures/`. Regenerate with `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Kisa.Tests/StellaOps.Concelier.Source.Kisa.Tests.csproj`.
- The regression suite asserts canonical mapping, state cleanup, and telemetry counters (`KisaConnectorTests.Telemetry_RecordsMetrics`) so QA can track instrumentation drift.
For operator docs, link to this brief when documenting Hangul handling or counter dashboards so localisation reviewers have a single reference point.

8
docs/dev/merge_semver_playbook.md
View File

@@ -1,4 +1,4 @@
# Feedser SemVer Merge Playbook (Sprint 1–2)
# Concelier SemVer Merge Playbook (Sprint 1–2)
This playbook describes how the merge layer and connector teams should emit the new SemVer primitives introduced in Sprint 1–2, how those primitives become normalized version rules, and how downstream jobs query them deterministically.
@@ -8,7 +8,7 @@ This playbook describes how the merge layer and connector teams should emit the
- `NormalizedVersionRule` documents the analytics-friendly projection of each `AffectedPackage` coverage entry and is persisted alongside legacy `versionRanges`.
- `AdvisoryProvenance.decisionReason` records whether merge resolution favored precedence, freshness, or a tie-breaker comparison.
See `src/StellaOps.Feedser.Models/CANONICAL_RECORDS.md` for the full schema and field descriptions.
See `src/StellaOps.Concelier.Models/CANONICAL_RECORDS.md` for the full schema and field descriptions.
## 2. Mapper pattern
@@ -150,5 +150,5 @@ If a new scheme is required (for example, `apple.build` or `ios.semver`), raise
## 9. Observability signals
- `feedser.merge.normalized_rules` (counter, tags: `package_type`, `scheme`) – increments once per normalized rule retained after precedence merge.
- `feedser.merge.normalized_rules_missing` (counter, tags: `package_type`) – increments when a merged package still carries version ranges but no normalized rules; watch for spikes to catch connectors that have not emitted normalized arrays yet.
- `concelier.merge.normalized_rules` (counter, tags: `package_type`, `scheme`) – increments once per normalized rule retained after precedence merge.
- `concelier.merge.normalized_rules_missing` (counter, tags: `package_type`) – increments when a merged package still carries version ranges but no normalized rules; watch for spikes to catch connectors that have not emitted normalized arrays yet.

6
docs/dev/mongo_indices.md
View File

@@ -2,11 +2,11 @@
This guide complements the Sprint 1–2 normalized versions rollout. It documents recommended indexes and aggregation patterns for querying `AffectedPackage.normalizedVersions`.
For a field-by-field look at how normalized rules persist in MongoDB (including provenance metadata), see Section 8 of the [Feedser SemVer Merge Playbook](merge_semver_playbook.md).
For a field-by-field look at how normalized rules persist in MongoDB (including provenance metadata), see Section 8 of the [Concelier SemVer Merge Playbook](merge_semver_playbook.md).
## 1. Recommended indexes
When `feedser.storage.enableSemVerStyle` is enabled, advisories expose a flattened
When `concelier.storage.enableSemVerStyle` is enabled, advisories expose a flattened
`normalizedVersions` array at the document root. Create these indexes in `mongosh`
after the migration completes (adjust collection name if you use a prefix):
@@ -105,4 +105,4 @@ Use this to confirm the merge dedupe logic keeps only one normalized rule per un
- [ ] Notify downstream services that consume advisory snapshots about the new `normalizedVersions` array.
- [ ] Update export fixtures once dedupe verification passes.
Additional background and mapper examples live in [Feedser SemVer Merge Playbook](merge_semver_playbook.md).
Additional background and mapper examples live in [Concelier SemVer Merge Playbook](merge_semver_playbook.md).

34
docs/dev/normalized_versions_rollout.md
View File

@@ -1,11 +1,11 @@
# Normalized Versions Rollout Dashboard (Sprint 2 – Feedser)
# Normalized Versions Rollout Dashboard (Sprint 2 – Concelier)
_Status date: 2025-10-12 17:05 UTC_
This dashboard tracks connector readiness for emitting `AffectedPackage.NormalizedVersions` arrays and highlights upcoming coordination checkpoints. Use it alongside:
- [`src/StellaOps.Feedser.Merge/RANGE_PRIMITIVES_COORDINATION.md`](../../src/StellaOps.Feedser.Merge/RANGE_PRIMITIVES_COORDINATION.md) for detailed guidance and timelines.
- [Feedser SemVer Merge Playbook](merge_semver_playbook.md) §8 for persisted Mongo document shapes.
- [`src/StellaOps.Concelier.Merge/RANGE_PRIMITIVES_COORDINATION.md`](../../src/StellaOps.Concelier.Merge/RANGE_PRIMITIVES_COORDINATION.md) for detailed guidance and timelines.
- [Concelier SemVer Merge Playbook](merge_semver_playbook.md) §8 for persisted Mongo document shapes.
- [Normalized Versions Query Guide](mongo_indices.md) for index/query validation steps.
## Key milestones
@@ -18,28 +18,28 @@ This dashboard tracks connector readiness for emitting `AffectedPackage.Normaliz
| Connector | Owner team | Normalized versions status | Last update | Next action / link |
|-----------|------------|---------------------------|-------------|--------------------|
| Acsc | BE-Conn-ACSC | ❌ Not started – mapper pending | 2025-10-11 | Design DTOs + mapper with normalized rule array; see `src/StellaOps.Feedser.Source.Acsc/TASKS.md`. |
| Acsc | BE-Conn-ACSC | ❌ Not started – mapper pending | 2025-10-11 | Design DTOs + mapper with normalized rule array; see `src/StellaOps.Concelier.Source.Acsc/TASKS.md`. |
| Cccs | BE-Conn-CCCS | ❌ Not started – mapper pending | 2025-10-11 | Add normalized SemVer array in canonical mapper; coordinate fixtures per `TASKS.md`. |
| CertBund | BE-Conn-CERTBUND | ✅ Canonical mapper emitting vendor ranges | 2025-10-14 | Normalized vendor range payloads landed alongside telemetry/docs updates; see `src/StellaOps.Feedser.Source.CertBund/TASKS.md`. |
| CertCc | BE-Conn-CERTCC | ⚠️ In progress – fetch pipeline DOING | 2025-10-11 | Implement VINCE mapper with SemVer/NEVRA rules; unblock snapshot regeneration; `src/StellaOps.Feedser.Source.CertCc/TASKS.md`. |
| Kev | BE-Conn-KEV | ✅ Normalized catalog/due-date rules verified | 2025-10-12 | Fixtures reconfirmed via `dotnet test src/StellaOps.Feedser.Source.Kev.Tests`; `src/StellaOps.Feedser.Source.Kev/TASKS.md`. |
| Cve | BE-Conn-CVE | ✅ Normalized SemVer rules verified | 2025-10-12 | Snapshot parity green (`dotnet test src/StellaOps.Feedser.Source.Cve.Tests`); `src/StellaOps.Feedser.Source.Cve/TASKS.md`. |
| Ghsa | BE-Conn-GHSA | ⚠️ DOING – normalized rollout task active | 2025-10-11 18:45 UTC | Wire `SemVerRangeRuleBuilder` + refresh fixtures; `src/StellaOps.Feedser.Source.Ghsa/TASKS.md`. |
| Osv | BE-Conn-OSV | ✅ SemVer mapper & parity fixtures verified | 2025-10-12 | GHSA parity regression passing (`dotnet test src/StellaOps.Feedser.Source.Osv.Tests`); `src/StellaOps.Feedser.Source.Osv/TASKS.md`. |
| Ics.Cisa | BE-Conn-ICS-CISA | ❌ Not started – mapper TODO | 2025-10-11 | Plan SemVer/firmware scheme selection; `src/StellaOps.Feedser.Source.Ics.Cisa/TASKS.md`. |
| CertBund | BE-Conn-CERTBUND | ✅ Canonical mapper emitting vendor ranges | 2025-10-14 | Normalized vendor range payloads landed alongside telemetry/docs updates; see `src/StellaOps.Concelier.Source.CertBund/TASKS.md`. |
| CertCc | BE-Conn-CERTCC | ⚠️ In progress – fetch pipeline DOING | 2025-10-11 | Implement VINCE mapper with SemVer/NEVRA rules; unblock snapshot regeneration; `src/StellaOps.Concelier.Source.CertCc/TASKS.md`. |
| Kev | BE-Conn-KEV | ✅ Normalized catalog/due-date rules verified | 2025-10-12 | Fixtures reconfirmed via `dotnet test src/StellaOps.Concelier.Source.Kev.Tests`; `src/StellaOps.Concelier.Source.Kev/TASKS.md`. |
| Cve | BE-Conn-CVE | ✅ Normalized SemVer rules verified | 2025-10-12 | Snapshot parity green (`dotnet test src/StellaOps.Concelier.Source.Cve.Tests`); `src/StellaOps.Concelier.Source.Cve/TASKS.md`. |
| Ghsa | BE-Conn-GHSA | ⚠️ DOING – normalized rollout task active | 2025-10-11 18:45 UTC | Wire `SemVerRangeRuleBuilder` + refresh fixtures; `src/StellaOps.Concelier.Source.Ghsa/TASKS.md`. |
| Osv | BE-Conn-OSV | ✅ SemVer mapper & parity fixtures verified | 2025-10-12 | GHSA parity regression passing (`dotnet test src/StellaOps.Concelier.Source.Osv.Tests`); `src/StellaOps.Concelier.Source.Osv/TASKS.md`. |
| Ics.Cisa | BE-Conn-ICS-CISA | ❌ Not started – mapper TODO | 2025-10-11 | Plan SemVer/firmware scheme selection; `src/StellaOps.Concelier.Source.Ics.Cisa/TASKS.md`. |
| Kisa | BE-Conn-KISA | ✅ Landed 2025-10-14 (mapper + telemetry) | 2025-10-11 | Hangul-aware mapper emits normalized rules; see `docs/dev/kisa_connector_notes.md` for localisation/metric details. |
| Ru.Bdu | BE-Conn-BDU | ✅ Raw scheme emitted | 2025-10-14 | Mapper now writes `ru-bdu.raw` normalized rules with provenance + telemetry; `src/StellaOps.Feedser.Source.Ru.Bdu/TASKS.md`. |
| Ru.Nkcki | BE-Conn-Nkcki | ❌ Not started – mapper TODO | 2025-10-11 | Similar to BDU; ensure Cyrillic provenance preserved; `src/StellaOps.Feedser.Source.Ru.Nkcki/TASKS.md`. |
| Vndr.Apple | BE-Conn-Apple | ✅ Shipped – emitting normalized arrays | 2025-10-11 | Continue fixture/tooling work; `src/StellaOps.Feedser.Source.Vndr.Apple/TASKS.md`. |
| Ru.Bdu | BE-Conn-BDU | ✅ Raw scheme emitted | 2025-10-14 | Mapper now writes `ru-bdu.raw` normalized rules with provenance + telemetry; `src/StellaOps.Concelier.Source.Ru.Bdu/TASKS.md`. |
| Ru.Nkcki | BE-Conn-Nkcki | ❌ Not started – mapper TODO | 2025-10-11 | Similar to BDU; ensure Cyrillic provenance preserved; `src/StellaOps.Concelier.Source.Ru.Nkcki/TASKS.md`. |
| Vndr.Apple | BE-Conn-Apple | ✅ Shipped – emitting normalized arrays | 2025-10-11 | Continue fixture/tooling work; `src/StellaOps.Concelier.Source.Vndr.Apple/TASKS.md`. |
| Vndr.Cisco | BE-Conn-Cisco | ✅ SemVer + vendor extensions emitted | 2025-10-14 | Connector outputs SemVer primitives with `cisco.productId` notes; see `CiscoMapper` and fixtures for coverage. |
| Vndr.Msrc | BE-Conn-MSRC | ✅ Map + normalized build rules landed | 2025-10-15 | `MsrcMapper` emits `msrc.build` normalized rules with CVRF references; see `src/StellaOps.Feedser.Source.Vndr.Msrc/TASKS.md`. |
| Nvd | BE-Conn-NVD | ⚠️ Needs follow-up – mapper complete but normalized array MR pending | 2025-10-11 | Align CVE notes + normalized payload flag; `src/StellaOps.Feedser.Source.Nvd/TASKS.md`. |
| Vndr.Msrc | BE-Conn-MSRC | ✅ Map + normalized build rules landed | 2025-10-15 | `MsrcMapper` emits `msrc.build` normalized rules with CVRF references; see `src/StellaOps.Concelier.Source.Vndr.Msrc/TASKS.md`. |
| Nvd | BE-Conn-NVD | ⚠️ Needs follow-up – mapper complete but normalized array MR pending | 2025-10-11 | Align CVE notes + normalized payload flag; `src/StellaOps.Concelier.Source.Nvd/TASKS.md`. |
Legend: ✅ complete, ⚠️ in progress/partial, ❌ not started.
## Monitoring
- Merge now emits `feedser.merge.normalized_rules` (tags: `package_type`, `scheme`) and `feedser.merge.normalized_rules_missing` (tags: `package_type`). Track these counters to confirm normalized arrays land as connectors roll out.
- Merge now emits `concelier.merge.normalized_rules` (tags: `package_type`, `scheme`) and `concelier.merge.normalized_rules_missing` (tags: `package_type`). Track these counters to confirm normalized arrays land as connectors roll out.
- Expect `normalized_rules_missing` to trend toward zero as each connector flips on normalized output. Investigate any sustained counts by checking the corresponding module `TASKS.md`.
## Implementation tips

28
docs/ops/feedser-apple-operations.md → docs/ops/concelier-apple-operations.md
View File

@@ -1,4 +1,4 @@
# Feedser Apple Security Update Connector Operations
# Concelier Apple Security Update Connector Operations
This runbook covers staging and production rollout for the Apple security updates connector (`source:vndr-apple:*`), including observability checks and fixture maintenance.
@@ -6,10 +6,10 @@ This runbook covers staging and production rollout for the Apple security update
- Network egress (or mirrored cache) for `https://gdmf.apple.com/v2/pmv` and the Apple Support domain (`https://support.apple.com/`).
- Optional: corporate proxy exclusions for the Apple hosts if outbound traffic is normally filtered.
- Updated configuration (environment variables or `feedser.yaml`) with an `apple` section. Example baseline:
- Updated configuration (environment variables or `concelier.yaml`) with an `apple` section. Example baseline:
```yaml
feedser:
concelier:
sources:
apple:
softwareLookupUri: "https://gdmf.apple.com/v2/pmv"
@@ -21,23 +21,23 @@ feedser:
failureBackoff: "00:05:00"
```
> ℹ️ `softwareLookupUri` and `advisoryBaseUri` must stay absolute and aligned with the HTTP allow-list; Feedser automatically adds both hosts to the connector HttpClient.
> ℹ️ `softwareLookupUri` and `advisoryBaseUri` must stay absolute and aligned with the HTTP allow-list; Concelier automatically adds both hosts to the connector HttpClient.
## 2. Staging Smoke Test
1. Deploy the configuration and restart the Feedser workers to ensure the Apple connector options are bound.
1. Deploy the configuration and restart the Concelier workers to ensure the Apple connector options are bound.
2. Trigger a full connector cycle:
- CLI: `stella db jobs run source:vndr-apple:fetch --and-then source:vndr-apple:parse --and-then source:vndr-apple:map`
- REST: `POST /jobs/run { "kind": "source:vndr-apple:fetch", "chain": ["source:vndr-apple:parse", "source:vndr-apple:map"] }`
3. Validate metrics exported under meter `StellaOps.Feedser.Source.Vndr.Apple`:
3. Validate metrics exported under meter `StellaOps.Concelier.Source.Vndr.Apple`:
- `apple.fetch.items` (documents fetched)
- `apple.fetch.failures`
- `apple.fetch.unchanged`
- `apple.parse.failures`
- `apple.map.affected.count` (histogram of affected package counts)
4. Cross-check the shared HTTP counters:
- `feedser.source.http.requests_total{feedser_source="vndr-apple"}` should increase for both index and detail phases.
- `feedser.source.http.failures_total{feedser_source="vndr-apple"}` should remain flat (0) during a healthy run.
- `concelier.source.http.requests_total{concelier_source="vndr-apple"}` should increase for both index and detail phases.
- `concelier.source.http.failures_total{concelier_source="vndr-apple"}` should remain flat (0) during a healthy run.
5. Inspect the info logs:
- `Apple software index fetch … processed=X newDocuments=Y`
- `Apple advisory parse complete … aliases=… affected=…`
@@ -50,28 +50,28 @@ feedser:
## 3. Production Monitoring
- **Dashboards** – Add the following expressions to your Feedser Grafana board (OTLP/Prometheus naming assumed):
- `rate(apple_fetch_items_total[15m])` vs `rate(feedser_source_http_requests_total{feedser_source="vndr-apple"}[15m])`
- **Dashboards** – Add the following expressions to your Concelier Grafana board (OTLP/Prometheus naming assumed):
- `rate(apple_fetch_items_total[15m])` vs `rate(concelier_source_http_requests_total{concelier_source="vndr-apple"}[15m])`
- `rate(apple_fetch_failures_total[5m])` for error spikes (`severity=warning` at `>0`)
- `histogram_quantile(0.95, rate(apple_map_affected_count_bucket[1h]))` to watch affected-package fan-out
- `increase(apple_parse_failures_total[6h])` to catch parser drift (alerts at `>0`)
- **Alerts** – Page if `rate(apple_fetch_items_total[2h]) == 0` during business hours while other connectors are active. This often indicates lookup feed failures or misconfigured allow-lists.
- **Logs** – Surface warnings `Apple document {DocumentId} missing GridFS payload` or `Apple parse failed`—repeated hits imply storage issues or HTML regressions.
- **Telemetry pipeline** – `StellaOps.Feedser.WebService` now exports `StellaOps.Feedser.Source.Vndr.Apple` alongside existing Feedser meters; ensure your OTEL collector or Prometheus scraper includes it.
- **Telemetry pipeline** – `StellaOps.Concelier.WebService` now exports `StellaOps.Concelier.Source.Vndr.Apple` alongside existing Concelier meters; ensure your OTEL collector or Prometheus scraper includes it.
## 4. Fixture Maintenance
Regression fixtures live under `src/StellaOps.Feedser.Source.Vndr.Apple.Tests/Apple/Fixtures`. Refresh them whenever Apple reshapes the HT layout or when new platforms appear.
Regression fixtures live under `src/StellaOps.Concelier.Source.Vndr.Apple.Tests/Apple/Fixtures`. Refresh them whenever Apple reshapes the HT layout or when new platforms appear.
1. Run the helper script matching your platform:
- Bash: `./scripts/update-apple-fixtures.sh`
- PowerShell: `./scripts/update-apple-fixtures.ps1`
2. Each script exports `UPDATE_APPLE_FIXTURES=1`, updates the `WSLENV` passthrough, and touches `.update-apple-fixtures` so WSL+VS Code test runs observe the flag. The subsequent test execution fetches the live HT articles listed in `AppleFixtureManager`, sanitises the HTML, and rewrites the `.expected.json` DTO snapshots.
3. Review the diff for localisation or nav noise. Once satisfied, re-run the tests without the env var (`dotnet test src/StellaOps.Feedser.Source.Vndr.Apple.Tests/StellaOps.Feedser.Source.Vndr.Apple.Tests.csproj`) to verify determinism.
3. Review the diff for localisation or nav noise. Once satisfied, re-run the tests without the env var (`dotnet test src/StellaOps.Concelier.Source.Vndr.Apple.Tests/StellaOps.Concelier.Source.Vndr.Apple.Tests.csproj`) to verify determinism.
4. Commit fixture updates together with any parser/mapping changes that motivated them.
## 5. Known Issues & Follow-up Tasks
- Apple occasionally throttles anonymous requests after bursts. The connector backs off automatically, but persistent `apple.fetch.failures` spikes might require mirroring the HT content or scheduling wider fetch windows.
- Rapid Security Responses may appear before the general patch notes surface in the lookup JSON. When that happens, the fetch run will log `detailFailures>0`. Collect sample HTML and refresh fixtures to confirm parser coverage.
- Multi-locale content is still under regression sweep (`src/StellaOps.Feedser.Source.Vndr.Apple/TASKS.md`). Capture non-`en-us` snapshots once the fixture tooling stabilises.
- Multi-locale content is still under regression sweep (`src/StellaOps.Concelier.Source.Vndr.Apple/TASKS.md`). Capture non-`en-us` snapshots once the fixture tooling stabilises.

74
docs/ops/feedser-authority-audit-runbook.md → docs/ops/concelier-authority-audit-runbook.md
View File

@@ -1,32 +1,32 @@
# Feedser Authority Audit Runbook
# Concelier Authority Audit Runbook
_Last updated: 2025-10-12_
This runbook helps operators verify and monitor the StellaOps Feedser ⇆ Authority integration. It focuses on the `/jobs*` surface, which now requires StellaOps Authority tokens, and the corresponding audit/metric signals that expose authentication and bypass activity.
This runbook helps operators verify and monitor the StellaOps Concelier ⇆ Authority integration. It focuses on the `/jobs*` surface, which now requires StellaOps Authority tokens, and the corresponding audit/metric signals that expose authentication and bypass activity.
## 1. Prerequisites
- Authority integration is enabled in `feedser.yaml` (or via `FEEDSER_AUTHORITY__*` environment variables) with a valid `clientId`, secret, audience, and required scopes.
- OTLP metrics/log exporters are configured (`feedser.telemetry.*`) or container stdout is shipped to your SIEM.
- Operators have access to the Feedser job trigger endpoints via CLI or REST for smoke tests.
- Authority integration is enabled in `concelier.yaml` (or via `CONCELIER_AUTHORITY__*` environment variables) with a valid `clientId`, secret, audience, and required scopes.
- OTLP metrics/log exporters are configured (`concelier.telemetry.*`) or container stdout is shipped to your SIEM.
- Operators have access to the Concelier job trigger endpoints via CLI or REST for smoke tests.
### Configuration snippet
```yaml
feedser:
concelier:
authority:
enabled: true
allowAnonymousFallback: false # keep true only during initial rollout
issuer: "https://authority.internal"
audiences:
- "api://feedser"
- "api://concelier"
requiredScopes:
- "feedser.jobs.trigger"
- "concelier.jobs.trigger"
bypassNetworks:
- "127.0.0.1/32"
- "::1/128"
clientId: "feedser-jobs"
clientSecretFile: "/run/secrets/feedser_authority_client"
clientId: "concelier-jobs"
clientSecretFile: "/run/secrets/concelier_authority_client"
tokenClockSkewSeconds: 60
resilience:
enableRetries: true
@@ -38,22 +38,22 @@ feedser:
offlineCacheTolerance: "00:10:00"
```
> Store secrets outside source control. Feedser reads `clientSecretFile` on startup; rotate by updating the mounted file and restarting the service.
> Store secrets outside source control. Concelier reads `clientSecretFile` on startup; rotate by updating the mounted file and restarting the service.
### Resilience tuning
- **Connected sites:** keep the default 1 s / 2 s / 5 s retry ladder so Feedser retries transient Authority hiccups but still surfaces outages quickly. Leave `allowOfflineCacheFallback=true` so cached discovery/JWKS data can bridge short Pathfinder restarts.
- **Air-gapped/Offline Kit installs:** extend `offlineCacheTolerance` (15–30 minutes) to keep the cached metadata valid between manual synchronisations. You can also disable retries (`enableRetries=false`) if infrastructure teams prefer to handle exponential backoff at the network layer; Feedser will fail fast but keep deterministic logs.
- Feedser resolves these knobs through `IOptionsMonitor<StellaOpsAuthClientOptions>`. Edits to `feedser.yaml` are applied on configuration reload; restart the container if you change environment variables or do not have file-watch reloads enabled.
- **Connected sites:** keep the default 1 s / 2 s / 5 s retry ladder so Concelier retries transient Authority hiccups but still surfaces outages quickly. Leave `allowOfflineCacheFallback=true` so cached discovery/JWKS data can bridge short Pathfinder restarts.
- **Air-gapped/Offline Kit installs:** extend `offlineCacheTolerance` (15–30 minutes) to keep the cached metadata valid between manual synchronisations. You can also disable retries (`enableRetries=false`) if infrastructure teams prefer to handle exponential backoff at the network layer; Concelier will fail fast but keep deterministic logs.
- Concelier resolves these knobs through `IOptionsMonitor<StellaOpsAuthClientOptions>`. Edits to `concelier.yaml` are applied on configuration reload; restart the container if you change environment variables or do not have file-watch reloads enabled.
## 2. Key Signals
### 2.1 Audit log channel
Feedser emits structured audit entries via the `Feedser.Authorization.Audit` logger for every `/jobs*` request once Authority enforcement is active.
Concelier emits structured audit entries via the `Concelier.Authorization.Audit` logger for every `/jobs*` request once Authority enforcement is active.
```
Feedser authorization audit route=/jobs/definitions status=200 subject=ops@example.com clientId=feedser-cli scopes=feedser.jobs.trigger bypass=False remote=10.1.4.7
Concelier authorization audit route=/jobs/definitions status=200 subject=ops@example.com clientId=concelier-cli scopes=concelier.jobs.trigger bypass=False remote=10.1.4.7
```
| Field | Sample value | Meaning |
@@ -61,8 +61,8 @@ Feedser authorization audit route=/jobs/definitions status=200 subject=ops@examp
| `route` | `/jobs/definitions` | Endpoint that processed the request. |
| `status` | `200` / `401` / `409` | Final HTTP status code returned to the caller. |
| `subject` | `ops@example.com` | User or service principal subject (falls back to `(anonymous)` when unauthenticated). |
| `clientId` | `feedser-cli` | OAuth client ID provided by Authority ( `(none)` if the token lacked the claim). |
| `scopes` | `feedser.jobs.trigger` | Normalised scope list extracted from token claims; `(none)` if the token carried none. |
| `clientId` | `concelier-cli` | OAuth client ID provided by Authority ( `(none)` if the token lacked the claim). |
| `scopes` | `concelier.jobs.trigger` | Normalised scope list extracted from token claims; `(none)` if the token carried none. |
| `bypass` | `True` / `False` | Indicates whether the request succeeded because its source IP matched a bypass CIDR. |
| `remote` | `10.1.4.7` | Remote IP recorded from the connection / forwarded header test hooks. |
@@ -74,7 +74,7 @@ Use your logging backend (e.g., Loki) to index the logger name and filter for su
### 2.2 Metrics
Feedser publishes counters under the OTEL meter `StellaOps.Feedser.WebService.Jobs`. Tags: `job.kind`, `job.trigger`, `job.outcome`.
Concelier publishes counters under the OTEL meter `StellaOps.Concelier.WebService.Jobs`. Tags: `job.kind`, `job.trigger`, `job.outcome`.
| Metric name | Description | PromQL example |
|-------------------------------|----------------------------------------------------|----------------|
@@ -84,42 +84,42 @@ Feedser publishes counters under the OTEL meter `StellaOps.Feedser.WebService.Jo
> Prometheus/OTEL collectors typically surface counters with `_total` suffix. Adjust queries to match your pipeline’s generated metric names.
Correlate audit logs with the following global meter exported via `Feedser.SourceDiagnostics`:
Correlate audit logs with the following global meter exported via `Concelier.SourceDiagnostics`:
- `feedser.source.http.requests_total{feedser_source="jobs-run"}` – ensures REST/manual triggers route through Authority.
- If Grafana dashboards are deployed, extend the “Feedser Jobs” board with the above counters plus a table of recent audit log entries.
- `concelier.source.http.requests_total{concelier_source="jobs-run"}` – ensures REST/manual triggers route through Authority.
- If Grafana dashboards are deployed, extend the “Concelier Jobs” board with the above counters plus a table of recent audit log entries.
## 3. Alerting Guidance
1. **Unauthorized bypass attempt**
- Query: `sum(rate(log_messages_total{logger="Feedser.Authorization.Audit", status="401", bypass="True"}[5m])) > 0`
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", status="401", bypass="True"}[5m])) > 0`
- Action: verify `bypassNetworks` list; confirm expected maintenance windows; rotate credentials if suspicious.
2. **Missing scopes**
- Query: `sum(rate(log_messages_total{logger="Feedser.Authorization.Audit", scopes="(none)", status="200"}[5m])) > 0`
- Action: audit Authority client registration; ensure `requiredScopes` includes `feedser.jobs.trigger`.
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", scopes="(none)", status="200"}[5m])) > 0`
- Action: audit Authority client registration; ensure `requiredScopes` includes `concelier.jobs.trigger`.
3. **Trigger failure surge**
- Query: `sum(rate(web_jobs_trigger_failed_total[10m])) > 0` with severity `warning` if sustained for 10 minutes.
- Action: inspect correlated audit entries and `Feedser.Telemetry` traces for job execution errors.
- Action: inspect correlated audit entries and `Concelier.Telemetry` traces for job execution errors.
4. **Conflict spike**
- Query: `sum(rate(web_jobs_trigger_conflict_total[10m])) > 5` (tune threshold).
- Action: downstream scheduling may be firing repetitive triggers; ensure precedence is configured properly.
5. **Authority offline**
- Watch `Feedser.Authorization.Audit` logs for `status=503` or `status=500` along with `clientId="(none)"`. Investigate Authority availability before re-enabling anonymous fallback.
- Watch `Concelier.Authorization.Audit` logs for `status=503` or `status=500` along with `clientId="(none)"`. Investigate Authority availability before re-enabling anonymous fallback.
## 4. Rollout & Verification Procedure
1. **Pre-checks**
- Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation.
- Validate Authority issuer metadata is reachable from Feedser (`curl https://authority.internal/.well-known/openid-configuration` from the host).
- Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host).
2. **Smoke test with valid token**
- Obtain a token via CLI: `stella auth login --scope feedser.jobs.trigger`.
- Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://feedser.internal/jobs/definitions`.
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=feedser.jobs.trigger`.
- Obtain a token via CLI: `stella auth login --scope concelier.jobs.trigger`.
- Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://concelier.internal/jobs/definitions`.
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger`.
3. **Negative test without token**
- Call the same endpoint without a token. Expect HTTP 401, `bypass=False`.
@@ -130,21 +130,21 @@ Correlate audit logs with the following global meter exported via `Feedser.Sourc
5. **Metrics validation**
- Ensure `web.jobs.triggered` counter increments during accepted runs.
- Exporters should show corresponding spans (`feedser.job.trigger`) if tracing is enabled.
- Exporters should show corresponding spans (`concelier.job.trigger`) if tracing is enabled.
## 5. Troubleshooting
| Symptom | Probable cause | Remediation |
|---------|----------------|-------------|
| Audit log shows `clientId=(none)` for all requests | Authority not issuing `client_id` claim or CLI outdated | Update StellaOps Authority configuration (`StellaOpsAuthorityOptions.Token.Claims.ClientId`), or upgrade the CLI token acquisition flow. |
| Requests succeed with `bypass=True` unexpectedly | Local network added to `bypassNetworks` or fallback still enabled | Remove/adjust the CIDR list, disable anonymous fallback, restart Feedser. |
| HTTP 401 with valid token | `requiredScopes` missing from client registration or token audience mismatch | Verify Authority client scopes (`feedser.jobs.trigger`) and ensure the token audience matches `audiences` config. |
| Metrics missing from Prometheus | Telemetry exporters disabled or filter missing OTEL meter | Set `feedser.telemetry.enableMetrics=true`, ensure collector includes `StellaOps.Feedser.WebService.Jobs` meter. |
| Sudden spike in `web.jobs.trigger.failed` | Downstream job failure or Authority timeout mid-request | Inspect Feedser job logs, re-run with tracing enabled, validate Authority latency. |
| Requests succeed with `bypass=True` unexpectedly | Local network added to `bypassNetworks` or fallback still enabled | Remove/adjust the CIDR list, disable anonymous fallback, restart Concelier. |
| HTTP 401 with valid token | `requiredScopes` missing from client registration or token audience mismatch | Verify Authority client scopes (`concelier.jobs.trigger`) and ensure the token audience matches `audiences` config. |
| Metrics missing from Prometheus | Telemetry exporters disabled or filter missing OTEL meter | Set `concelier.telemetry.enableMetrics=true`, ensure collector includes `StellaOps.Concelier.WebService.Jobs` meter. |
| Sudden spike in `web.jobs.trigger.failed` | Downstream job failure or Authority timeout mid-request | Inspect Concelier job logs, re-run with tracing enabled, validate Authority latency. |
## 6. References
- `docs/21_INSTALL_GUIDE.md` – Authority configuration quick start.
- `docs/17_SECURITY_HARDENING_GUIDE.md` – Security guardrails and enforcement deadlines.
- `docs/ops/authority-monitoring.md` – Authority-side monitoring and alerting playbook.
- `StellaOps.Feedser.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields.
- `StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields.

22
docs/ops/feedser-cccs-operations.md → docs/ops/concelier-cccs-operations.md
View File

@@ -1,14 +1,14 @@
# Feedser CCCS Connector Operations
# Concelier CCCS Connector Operations
This runbook covers day‑to‑day operation of the Canadian Centre for Cyber Security (`source:cccs:*`) connector, including configuration, telemetry, and historical backfill guidance for English/French advisories.
## 1. Configuration Checklist
- Network egress (or mirrored cache) for `https://www.cyber.gc.ca/` and the JSON API endpoints under `/api/cccs/`.
- Set the Feedser options before restarting workers. Example `feedser.yaml` snippet:
- Set the Concelier options before restarting workers. Example `concelier.yaml` snippet:
```yaml
feedser:
concelier:
sources:
cccs:
feeds:
@@ -27,15 +27,15 @@ feedser:
## 2. Telemetry & Logging
- **Metrics (Meter `StellaOps.Feedser.Source.Cccs`):**
- **Metrics (Meter `StellaOps.Concelier.Source.Cccs`):**
- `cccs.fetch.attempts`, `cccs.fetch.success`, `cccs.fetch.failures`
- `cccs.fetch.documents`, `cccs.fetch.unchanged`
- `cccs.parse.success`, `cccs.parse.failures`, `cccs.parse.quarantine`
- `cccs.map.success`, `cccs.map.failures`
- **Shared HTTP metrics** via `SourceDiagnostics`:
- `feedser.source.http.requests{feedser.source="cccs"}`
- `feedser.source.http.failures{feedser.source="cccs"}`
- `feedser.source.http.duration{feedser.source="cccs"}`
- `concelier.source.http.requests{concelier.source="cccs"}`
- `concelier.source.http.failures{concelier.source="cccs"}`
- `concelier.source.http.duration{concelier.source="cccs"}`
- **Structured logs**
- `CCCS fetch completed feeds=… items=… newDocuments=… pendingDocuments=…`
- `CCCS parse completed parsed=… failures=…`
@@ -45,13 +45,13 @@ feedser:
Suggested Grafana alerts:
- `increase(cccs.fetch.failures_total[15m]) > 0`
- `rate(cccs.map.success_total[1h]) == 0` while other connectors are active
- `histogram_quantile(0.95, rate(feedser_source_http_duration_bucket{feedser_source="cccs"}[1h])) > 5s`
- `histogram_quantile(0.95, rate(concelier_source_http_duration_bucket{concelier_source="cccs"}[1h])) > 5s`
## 3. Historical Backfill Plan
1. **Snapshot the source** – the API accepts `page=<n>` and `lang=<en|fr>` query parameters. `page=0` returns the full dataset (observed earliest `date_created`: 2018‑06‑08 for EN, 2018‑06‑08 for FR). Mirror those responses into Offline Kit storage when operating air‑gapped.
2. **Stage ingestion**:
- Temporarily raise `maxEntriesPerFetch` (e.g. 500) and restart Feedser workers.
- Temporarily raise `maxEntriesPerFetch` (e.g. 500) and restart Concelier workers.
- Run chained jobs until `pendingDocuments` drains:
`stella db jobs run source:cccs:fetch --and-then source:cccs:parse --and-then source:cccs:map`
- Monitor `cccs.fetch.unchanged` growth; once it approaches dataset size the backfill is complete.
@@ -67,6 +67,6 @@ Suggested Grafana alerts:
## 5. Fixture Maintenance
- Regression fixtures live in `src/StellaOps.Feedser.Source.Cccs.Tests/Fixtures`.
- Refresh via `UPDATE_CCCS_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Cccs.Tests/StellaOps.Feedser.Source.Cccs.Tests.csproj`.
- Regression fixtures live in `src/StellaOps.Concelier.Source.Cccs.Tests/Fixtures`.
- Refresh via `UPDATE_CCCS_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Cccs.Tests/StellaOps.Concelier.Source.Cccs.Tests.csproj`.
- Fixtures capture both EN/FR advisories with nested lists to guard against sanitiser regressions; review diffs for heading/list changes before committing.

16
docs/ops/feedser-certbund-operations.md → docs/ops/concelier-certbund-operations.md
View File

@@ -1,8 +1,8 @@
# Feedser CERT-Bund Connector Operations
# Concelier CERT-Bund Connector Operations
_Last updated: 2025-10-17_
Germany’s Federal Office for Information Security (BSI) operates the Warn- und Informationsdienst (WID) portal. The Feedser CERT-Bund connector (`source:cert-bund:*`) ingests the public RSS feed, hydrates the portal’s JSON detail endpoint, and maps the result into canonical advisories while preserving the original German content.
Germany’s Federal Office for Information Security (BSI) operates the Warn- und Informationsdienst (WID) portal. The Concelier CERT-Bund connector (`source:cert-bund:*`) ingests the public RSS feed, hydrates the portal’s JSON detail endpoint, and maps the result into canonical advisories while preserving the original German content.
---
@@ -14,10 +14,10 @@ Germany’s Federal Office for Information Security (BSI) operates the Warn- und
- `https://wid.cert-bund.de/portal/api/securityadvisory` (detail/search/export JSON)
- Ensure the HTTP client reuses a cookie container (the connector’s dependency injection wiring already sets this up).
Example `feedser.yaml` fragment:
Example `concelier.yaml` fragment:
```yaml
feedser:
concelier:
sources:
cert-bund:
feedUri: "https://wid.cert-bund.de/content/public/securityAdvisory/rss"
@@ -36,7 +36,7 @@ feedser:
## 2. Telemetry & Logging
- **Meter**: `StellaOps.Feedser.Source.CertBund`
- **Meter**: `StellaOps.Concelier.Source.CertBund`
- **Counters / histograms**:
- `certbund.feed.fetch.attempts|success|failures`
- `certbund.feed.items.count`
@@ -47,7 +47,7 @@ feedser:
- `certbund.parse.products.count`, `certbund.parse.cve.count`
- `certbund.map.success|failures{reason}`
- `certbund.map.affected.count`, `certbund.map.aliases.count`
- Shared HTTP metrics remain available through `feedser.source.http.*`.
- Shared HTTP metrics remain available through `concelier.source.http.*`.
**Structured logs** (all emitted at information level when work occurs):
@@ -59,7 +59,7 @@ Alerting ideas:
1. `increase(certbund.detail.fetch.failures_total[10m]) > 0`
2. `rate(certbund.map.success_total[30m]) == 0`
3. `histogram_quantile(0.95, rate(feedser_source_http_duration_bucket{feedser_source="cert-bund"}[15m])) > 5s`
3. `histogram_quantile(0.95, rate(concelier_source_http_duration_bucket{concelier_source="cert-bund"}[15m])) > 5s`
The WebService now registers the meter so metrics surface automatically once OpenTelemetry metrics are enabled.
@@ -114,7 +114,7 @@ The helper stores yearly exports under `seed-data/cert-bund/export/`,
captures paginated search snapshots in `seed-data/cert-bund/search/`,
and generates the manifest + SHA files in `seed-data/cert-bund/manifest/`.
Split ranges according to your compliance window (default: one file per
calendar year). Feedser can ingest these JSON payloads directly when
calendar year). Concelier can ingest these JSON payloads directly when
operating offline.
> When automatic bootstrap fails (e.g. portal introduces CAPTCHA), run the

30
docs/ops/feedser-cisco-operations.md → docs/ops/concelier-cisco-operations.md
View File

@@ -1,16 +1,16 @@
# Feedser Cisco PSIRT Connector – OAuth Provisioning SOP
# Concelier Cisco PSIRT Connector – OAuth Provisioning SOP
_Last updated: 2025-10-14_
## 1. Scope
This runbook describes how Ops provisions, rotates, and distributes Cisco PSIRT openVuln OAuth client credentials for the Feedser Cisco connector. It covers online and air-gapped (Offline Kit) environments, quota-aware execution, and escalation paths.
This runbook describes how Ops provisions, rotates, and distributes Cisco PSIRT openVuln OAuth client credentials for the Concelier Cisco connector. It covers online and air-gapped (Offline Kit) environments, quota-aware execution, and escalation paths.
## 2. Prerequisites
- Active Cisco.com (CCO) account with access to the Cisco API Console.
- Cisco PSIRT openVuln API entitlement (visible under “My Apps & Keys” once granted).citeturn3search0
- Feedser configuration location (typically `/etc/stella/feedser.yaml` in production) or Offline Kit secret bundle staging directory.
- Concelier configuration location (typically `/etc/stella/concelier.yaml` in production) or Offline Kit secret bundle staging directory.
## 3. Provisioning workflow
@@ -29,11 +29,11 @@ This runbook describes how Ops provisions, rotates, and distributes Cisco PSIRT
```
- Confirm HTTP 200 and an `expires_in` value of 3600 seconds (tokens live for one hour).citeturn3search0turn3search7
- Preserve the response only long enough to validate syntax; do **not** persist tokens.
3. **Authorize Feedser runtime**
- Update `feedser:sources:cisco:auth` (or the module-specific secret template) with the stored credentials.
3. **Authorize Concelier runtime**
- Update `concelier:sources:cisco:auth` (or the module-specific secret template) with the stored credentials.
- For Offline Kit delivery, export encrypted secrets into `offline-kit/secrets/cisco-openvuln.json` using the platform’s sealed secret format.
4. **Connectivity validation**
- From the Feedser control plane, run `stella db jobs run source:vndr-cisco:fetch --dry-run`.
- From the Concelier control plane, run `stella db jobs run source:vndr-cisco:fetch --dry-run`.
- Ensure the Source HTTP diagnostics record `Bearer` authorization headers and no 401/403 responses.
## 4. Rotation SOP
@@ -42,12 +42,12 @@ This runbook describes how Ops provisions, rotates, and distributes Cisco PSIRT
| --- | --- | --- |
| 1. Schedule rotation | Ops (monthly board) | Rotate every 90 days or immediately after suspected credential exposure. |
| 2. Create replacement app | Ops | Repeat §3.1 with “-next” suffix; verify token issuance. |
| 3. Stage dual credentials | Ops + Feedser On-Call | Publish new credentials to secret store alongside current pair. |
| 4. Cut over | Feedser On-Call | Restart connector workers during a low-traffic window (<10 min) to pick up the new secret. |
| 3. Stage dual credentials | Ops + Concelier On-Call | Publish new credentials to secret store alongside current pair. |
| 4. Cut over | Concelier On-Call | Restart connector workers during a low-traffic window (<10 min) to pick up the new secret. |
| 5. Deactivate legacy app | Ops | Delete prior app in Cisco API Console once telemetry confirms successful fetch/parse cycles for 2 consecutive hours. |
**Automation hooks**
- Rotation reminders are tracked in OpsRunbookOps board (`OPS-RUN-KEYS` swim lane); add checklist items for Feedser Cisco when opening a rotation task.
- Rotation reminders are tracked in OpsRunbookOps board (`OPS-RUN-KEYS` swim lane); add checklist items for Concelier Cisco when opening a rotation task.
- Use the secret management pipeline (`ops/secrets/rotate.sh --connector cisco`) to template vault updates; the script renders a redacted diff for audit.
## 5. Offline Kit packaging
@@ -61,19 +61,19 @@ This runbook describes how Ops provisions, rotates, and distributes Cisco PSIRT
## 6. Quota and throttling guidance
- Cisco enforces combined limits of 5 requests/second, 30 requests/minute, and 5 000 requests/day per application.citeturn0search0turn3search6
- Feedser fetch jobs must respect `Retry-After` headers on HTTP 429 responses; Ops should monitor for sustained quota saturation and consider paging window adjustments.
- Telemetry to watch: `feedser.source.http.requests{feedser.source="vndr-cisco"}`, `feedser.source.http.failures{...}`, and connector-specific metrics once implemented.
- Concelier fetch jobs must respect `Retry-After` headers on HTTP 429 responses; Ops should monitor for sustained quota saturation and consider paging window adjustments.
- Telemetry to watch: `concelier.source.http.requests{concelier.source="vndr-cisco"}`, `concelier.source.http.failures{...}`, and connector-specific metrics once implemented.
## 7. Telemetry & Monitoring
- **Metrics (Meter `StellaOps.Feedser.Source.Vndr.Cisco`)**
- **Metrics (Meter `StellaOps.Concelier.Source.Vndr.Cisco`)**
- `cisco.fetch.documents`, `cisco.fetch.failures`, `cisco.fetch.unchanged`
- `cisco.parse.success`, `cisco.parse.failures`
- `cisco.map.success`, `cisco.map.failures`, `cisco.map.affected.packages`
- **Shared HTTP metrics** via `SourceDiagnostics`:
- `feedser.source.http.requests{feedser.source="vndr-cisco"}`
- `feedser.source.http.failures{feedser.source="vndr-cisco"}`
- `feedser.source.http.duration{feedser.source="vndr-cisco"}`
- `concelier.source.http.requests{concelier.source="vndr-cisco"}`
- `concelier.source.http.failures{concelier.source="vndr-cisco"}`
- `concelier.source.http.duration{concelier.source="vndr-cisco"}`
- **Structured logs**
- `Cisco fetch completed date=… pages=… added=…` (info)
- `Cisco parse completed parsed=… failures=…` (info)

76
docs/ops/feedser-conflict-resolution.md → docs/ops/concelier-conflict-resolution.md
View File

@@ -1,12 +1,12 @@
# Feedser Conflict Resolution Runbook (Sprint 3)
# Concelier Conflict Resolution Runbook (Sprint 3)
This runbook equips Feedser operators to detect, triage, and resolve advisory conflicts now that the Sprint 3 merge engine landed (`AdvisoryPrecedenceMerger`, merge-event hashing, and telemetry counters). It builds on the canonical rules defined in `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md` and the metrics/logging instrumentation delivered this sprint.
This runbook equips Concelier operators to detect, triage, and resolve advisory conflicts now that the Sprint 3 merge engine landed (`AdvisoryPrecedenceMerger`, merge-event hashing, and telemetry counters). It builds on the canonical rules defined in `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md` and the metrics/logging instrumentation delivered this sprint.
---
## 1. Precedence Model (recap)
- **Default ranking:** `GHSA -> NVD -> OSV`, with distro/vendor PSIRTs outranking ecosystem feeds (`AdvisoryPrecedenceDefaults`). Use `feedser:merge:precedence:ranks` to override per source when incident response requires it.
- **Default ranking:** `GHSA -> NVD -> OSV`, with distro/vendor PSIRTs outranking ecosystem feeds (`AdvisoryPrecedenceDefaults`). Use `concelier:merge:precedence:ranks` to override per source when incident response requires it.
- **Freshness override:** if a lower-ranked source is >= 48 hours newer for a freshness-sensitive field (title, summary, affected ranges, references, credits), it wins. Every override stamps `provenance[].decisionReason = freshness`.
- **Tie-breakers:** when precedence and freshness tie, the engine falls back to (1) primary source order, (2) shortest normalized text, (3) lowest stable hash. Merge-generated provenance records set `decisionReason = tie-breaker`.
- **Audit trail:** each merged advisory receives a `merge` provenance entry listing the participating sources plus a `merge_event` record with canonical before/after SHA-256 hashes.
@@ -17,41 +17,41 @@ This runbook equips Feedser operators to detect, triage, and resolve advisory co
| Instrument | Type | Key Tags | Purpose |
|------------|------|----------|---------|
| `feedser.merge.operations` | Counter | `inputs` | Total precedence merges executed. |
| `feedser.merge.overrides` | Counter | `primary_source`, `suppressed_source`, `primary_rank`, `suppressed_rank` | Field-level overrides chosen by precedence. |
| `feedser.merge.range_overrides` | Counter | `advisory_key`, `package_type`, `primary_source`, `suppressed_source`, `primary_range_count`, `suppressed_range_count` | Package range overrides emitted by `AffectedPackagePrecedenceResolver`. |
| `feedser.merge.conflicts` | Counter | `type` (`severity`, `precedence_tie`), `reason` (`mismatch`, `primary_missing`, `equal_rank`) | Conflicts requiring operator review. |
| `feedser.merge.identity_conflicts` | Counter | `scheme`, `alias_value`, `advisory_count` | Alias collisions surfaced by the identity graph. |
| `concelier.merge.operations` | Counter | `inputs` | Total precedence merges executed. |
| `concelier.merge.overrides` | Counter | `primary_source`, `suppressed_source`, `primary_rank`, `suppressed_rank` | Field-level overrides chosen by precedence. |
| `concelier.merge.range_overrides` | Counter | `advisory_key`, `package_type`, `primary_source`, `suppressed_source`, `primary_range_count`, `suppressed_range_count` | Package range overrides emitted by `AffectedPackagePrecedenceResolver`. |
| `concelier.merge.conflicts` | Counter | `type` (`severity`, `precedence_tie`), `reason` (`mismatch`, `primary_missing`, `equal_rank`) | Conflicts requiring operator review. |
| `concelier.merge.identity_conflicts` | Counter | `scheme`, `alias_value`, `advisory_count` | Alias collisions surfaced by the identity graph. |
### Structured logs
- `AdvisoryOverride` (EventId 1000) - logs merge suppressions with alias/provenance counts.
- `PackageRangeOverride` (EventId 1001) - logs package-level precedence decisions.
- `PrecedenceConflict` (EventId 1002) - logs mismatched severity or equal-rank scenarios.
- `Alias collision ...` (no EventId) - emitted when `feedser.merge.identity_conflicts` increments.
- `Alias collision ...` (no EventId) - emitted when `concelier.merge.identity_conflicts` increments.
Expect all logs at `Information`. Ensure OTEL exporters include the scope `StellaOps.Feedser.Merge`.
Expect all logs at `Information`. Ensure OTEL exporters include the scope `StellaOps.Concelier.Merge`.
---
## 3. Detection & Alerting
1. **Dashboard panels**
- `feedser.merge.conflicts` - table grouped by `type/reason`. Alert when > 0 in a 15 minute window.
- `feedser.merge.range_overrides` - stacked bar by `package_type`. Spikes highlight vendor PSIRT overrides over registry data.
- `feedser.merge.overrides` with `primary_source|suppressed_source` - catches unexpected precedence flips (e.g., OSV overtaking GHSA).
- `feedser.merge.identity_conflicts` - single-stat; alert when alias collisions occur more than once per day.
- `concelier.merge.conflicts` - table grouped by `type/reason`. Alert when > 0 in a 15 minute window.
- `concelier.merge.range_overrides` - stacked bar by `package_type`. Spikes highlight vendor PSIRT overrides over registry data.
- `concelier.merge.overrides` with `primary_source|suppressed_source` - catches unexpected precedence flips (e.g., OSV overtaking GHSA).
- `concelier.merge.identity_conflicts` - single-stat; alert when alias collisions occur more than once per day.
2. **Log based alerts**
- `eventId=1002` with `reason="equal_rank"` - indicates precedence table gaps; page merge owners.
- `eventId=1002` with `reason="mismatch"` - severity disagreement; open connector bug if sustained.
3. **Job health**
- `stellaops-cli db merge` exit code `1` signifies unresolved conflicts. Pipe to automation that captures logs and notifies #feedser-ops.
- `stellaops-cli db merge` exit code `1` signifies unresolved conflicts. Pipe to automation that captures logs and notifies #concelier-ops.
### Threshold updates (2025-10-12)
- `feedser.merge.conflicts` – Page only when ≥ 2 events fire within 30 minutes; the synthetic conflict fixture run produces 0 conflicts, so the first event now routes to Slack for manual review instead of paging.
- `feedser.merge.overrides` – Raise a warning when the 30-minute sum exceeds 10 (canonical triple yields exactly 1 summary override with `primary_source=osv`, `suppressed_source=ghsa`).
- `feedser.merge.range_overrides` – Maintain the 15-minute alert at ≥ 3 but annotate dashboards that the regression triple emits a single `package_type=semver` override so ops can spot unexpected spikes.
- `concelier.merge.conflicts` – Page only when ≥ 2 events fire within 30 minutes; the synthetic conflict fixture run produces 0 conflicts, so the first event now routes to Slack for manual review instead of paging.
- `concelier.merge.overrides` – Raise a warning when the 30-minute sum exceeds 10 (canonical triple yields exactly 1 summary override with `primary_source=osv`, `suppressed_source=ghsa`).
- `concelier.merge.range_overrides` – Maintain the 15-minute alert at ≥ 3 but annotate dashboards that the regression triple emits a single `package_type=semver` override so ops can spot unexpected spikes.
---
@@ -60,16 +60,16 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
1. **Confirm job context**
- `stellaops-cli db merge` (CLI) or `POST /jobs/merge:reconcile` (API) to rehydrate the merge job. Use `--verbose` to stream structured logs during triage.
2. **Inspect metrics**
- Correlate spikes in `feedser.merge.conflicts` with `primary_source`/`suppressed_source` tags from `feedser.merge.overrides`.
- Correlate spikes in `concelier.merge.conflicts` with `primary_source`/`suppressed_source` tags from `concelier.merge.overrides`.
3. **Pull structured logs**
- Example (vector output):
```
jq 'select(.EventId.Name=="PrecedenceConflict") | {advisory: .State[0].Value, type: .ConflictType, reason: .Reason, primary: .PrimarySources, suppressed: .SuppressedSources}' stellaops-feedser.log
jq 'select(.EventId.Name=="PrecedenceConflict") | {advisory: .State[0].Value, type: .ConflictType, reason: .Reason, primary: .PrimarySources, suppressed: .SuppressedSources}' stellaops-concelier.log
```
4. **Review merge events**
- `mongosh`:
```javascript
use feedser;
use concelier;
db.merge_event.find({ advisoryKey: "CVE-2025-1234" }).sort({ mergedAt: -1 }).limit(5);
```
- Compare `beforeHash` vs `afterHash` to confirm the merge actually changed canonical output.
@@ -85,9 +85,9 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
|--------|--------------|------------------|
| `reason="mismatch"` with `type="severity"` | Upstream feeds disagree on CVSS vector/severity. | Verify which feed is freshest; if correctness is known, adjust connector mapping or precedence override. |
| `reason="primary_missing"` | Higher-ranked source lacks the field entirely. | Backfill connector data or temporarily allow lower-ranked source via precedence override. |
| `reason="equal_rank"` | Two feeds share the same precedence rank (custom config or missing entry). | Update `feedser:merge:precedence:ranks` to break the tie; restart merge job. |
| Rising `feedser.merge.range_overrides` for a package type | Vendor PSIRT now supplies richer ranges. | Validate connectors emit `decisionReason="precedence"` and update dashboards to treat registry ranges as fallback. |
| `feedser.merge.identity_conflicts` > 0 | Alias scheme mapping produced collisions (duplicate CVE <-> advisory pairs). | Inspect `Alias collision` log payload; reconcile the alias graph by adjusting connector alias output. |
| `reason="equal_rank"` | Two feeds share the same precedence rank (custom config or missing entry). | Update `concelier:merge:precedence:ranks` to break the tie; restart merge job. |
| Rising `concelier.merge.range_overrides` for a package type | Vendor PSIRT now supplies richer ranges. | Validate connectors emit `decisionReason="precedence"` and update dashboards to treat registry ranges as fallback. |
| `concelier.merge.identity_conflicts` > 0 | Alias scheme mapping produced collisions (duplicate CVE <-> advisory pairs). | Inspect `Alias collision` log payload; reconcile the alias graph by adjusting connector alias output. |
---
@@ -97,16 +97,16 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
- Re-run the offending connector stages (`stellaops-cli db fetch --source ghsa --stage map` etc.).
- Once fixed, rerun merge and verify `decisionReason` reflects `freshness` or `precedence` as expected.
2. **Temporary precedence override**
- Edit `etc/feedser.yaml`:
- Edit `etc/concelier.yaml`:
```yaml
feedser:
concelier:
merge:
precedence:
ranks:
osv: 1
ghsa: 0
```
- Restart Feedser workers; confirm tags in `feedser.merge.overrides` show the new ranks.
- Restart Concelier workers; confirm tags in `concelier.merge.overrides` show the new ranks.
- Document the override with expiry in the change log.
3. **Alias remediation**
- Update connector mapping rules to weed out duplicate aliases (e.g., skip GHSA aliases that mirror CVE IDs).
@@ -119,7 +119,7 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
## 7. Validation Checklist
- [ ] Merge job rerun returns exit code `0`.
- [ ] `feedser.merge.conflicts` baseline returns to zero after corrective action.
- [ ] `concelier.merge.conflicts` baseline returns to zero after corrective action.
- [ ] Latest `merge_event` entry shows expected hash delta.
- [ ] Affected advisory document shows updated `provenance[].decisionReason`.
- [ ] Ops change log updated with incident summary, config overrides, and rollback plan.
@@ -129,9 +129,9 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
## 8. Reference Material
- Canonical conflict rules: `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md`.
- Merge engine internals: `src/StellaOps.Feedser.Merge/Services/AdvisoryPrecedenceMerger.cs`.
- Metrics definitions: `src/StellaOps.Feedser.Merge/Services/AdvisoryMergeService.cs` (identity conflicts) and `AdvisoryPrecedenceMerger`.
- Storage audit trail: `src/StellaOps.Feedser.Merge/Services/MergeEventWriter.cs`, `src/StellaOps.Feedser.Storage.Mongo/MergeEvents`.
- Merge engine internals: `src/StellaOps.Concelier.Merge/Services/AdvisoryPrecedenceMerger.cs`.
- Metrics definitions: `src/StellaOps.Concelier.Merge/Services/AdvisoryMergeService.cs` (identity conflicts) and `AdvisoryPrecedenceMerger`.
- Storage audit trail: `src/StellaOps.Concelier.Merge/Services/MergeEventWriter.cs`, `src/StellaOps.Concelier.Storage.Mongo/MergeEvents`.
Keep this runbook synchronized with future sprint notes and update alert thresholds as baseline volumes change.
@@ -139,17 +139,17 @@ Keep this runbook synchronized with future sprint notes and update alert thresho
## 9. Synthetic Regression Fixtures
- **Locations** – Canonical conflict snapshots now live at `src/StellaOps.Feedser.Source.Ghsa.Tests/Fixtures/conflict-ghsa.canonical.json`, `src/StellaOps.Feedser.Source.Nvd.Tests/Nvd/Fixtures/conflict-nvd.canonical.json`, and `src/StellaOps.Feedser.Source.Osv.Tests/Fixtures/conflict-osv.canonical.json`.
- **Locations** – Canonical conflict snapshots now live at `src/StellaOps.Concelier.Source.Ghsa.Tests/Fixtures/conflict-ghsa.canonical.json`, `src/StellaOps.Concelier.Source.Nvd.Tests/Nvd/Fixtures/conflict-nvd.canonical.json`, and `src/StellaOps.Concelier.Source.Osv.Tests/Fixtures/conflict-osv.canonical.json`.
- **Validation commands** – To regenerate and verify the fixtures offline, run:
```bash
dotnet test src/StellaOps.Feedser.Source.Ghsa.Tests/StellaOps.Feedser.Source.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests
dotnet test src/StellaOps.Feedser.Source.Nvd.Tests/StellaOps.Feedser.Source.Nvd.Tests.csproj --filter NvdConflictFixtureTests
dotnet test src/StellaOps.Feedser.Source.Osv.Tests/StellaOps.Feedser.Source.Osv.Tests.csproj --filter OsvConflictFixtureTests
dotnet test src/StellaOps.Feedser.Merge.Tests/StellaOps.Feedser.Merge.Tests.csproj --filter MergeAsync_AppliesCanonicalRulesAndPersistsDecisions
dotnet test src/StellaOps.Concelier.Source.Ghsa.Tests/StellaOps.Concelier.Source.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests
dotnet test src/StellaOps.Concelier.Source.Nvd.Tests/StellaOps.Concelier.Source.Nvd.Tests.csproj --filter NvdConflictFixtureTests
dotnet test src/StellaOps.Concelier.Source.Osv.Tests/StellaOps.Concelier.Source.Osv.Tests.csproj --filter OsvConflictFixtureTests
dotnet test src/StellaOps.Concelier.Merge.Tests/StellaOps.Concelier.Merge.Tests.csproj --filter MergeAsync_AppliesCanonicalRulesAndPersistsDecisions
```
- **Expected signals** – The triple produces one freshness-driven summary override (`primary_source=osv`, `suppressed_source=ghsa`) and one range override for the npm SemVer package while leaving `feedser.merge.conflicts` at zero. Use these values as the baseline when tuning dashboards or load-testing alert pipelines.
- **Expected signals** – The triple produces one freshness-driven summary override (`primary_source=osv`, `suppressed_source=ghsa`) and one range override for the npm SemVer package while leaving `concelier.merge.conflicts` at zero. Use these values as the baseline when tuning dashboards or load-testing alert pipelines.
---
@@ -157,4 +157,4 @@ dotnet test src/StellaOps.Feedser.Merge.Tests/StellaOps.Feedser.Merge.Tests.cspr
| Date (UTC) | Change | Notes |
|------------|--------|-------|
| 2025-10-16 | Ops review signed off after connector expansion (CCCS, CERT-Bund, KISA, ICS CISA, MSRC) landed. Alert thresholds from §3 reaffirmed; dashboards updated to watch attachment signals emitted by ICS CISA connector. | Ops sign-off recorded by Feedser Ops Guild; no additional overrides required. |
| 2025-10-16 | Ops review signed off after connector expansion (CCCS, CERT-Bund, KISA, ICS CISA, MSRC) landed. Alert thresholds from §3 reaffirmed; dashboards updated to watch attachment signals emitted by ICS CISA connector. | Ops sign-off recorded by Concelier Ops Guild; no additional overrides required. |

4
docs/ops/feedser-cve-kev-grafana-dashboard.json → docs/ops/concelier-cve-kev-grafana-dashboard.json
View File

@@ -1,6 +1,6 @@
{
"title": "Feedser CVE & KEV Observability",
"uid": "feedser-cve-kev",
"title": "Concelier CVE & KEV Observability",
"uid": "concelier-cve-kev",
"schemaVersion": 38,
"version": 1,
"editable": true,

46
docs/ops/feedser-cve-kev-operations.md → docs/ops/concelier-cve-kev-operations.md
View File

@@ -1,4 +1,4 @@
# Feedser CVE & KEV Connector Operations
# Concelier CVE & KEV Connector Operations
This playbook equips operators with the steps required to roll out and monitor the CVE Services and CISA KEV connectors across environments.
@@ -7,17 +7,17 @@ This playbook equips operators with the steps required to roll out and monitor t
### 1.1 Prerequisites
- CVE Services API credentials (organisation ID, user ID, API key) with access to the JSON 5 API.
- Network egress to `https://cveawg.mitre.org` (or a mirrored endpoint) from the Feedser workers.
- Updated `feedser.yaml` (or the matching environment variables) with the following section:
- Network egress to `https://cveawg.mitre.org` (or a mirrored endpoint) from the Concelier workers.
- Updated `concelier.yaml` (or the matching environment variables) with the following section:
```yaml
feedser:
concelier:
sources:
cve:
baseEndpoint: "https://cveawg.mitre.org/api/"
apiOrg: "ORG123"
apiUser: "user@example.org"
apiKeyFile: "/var/run/secrets/feedser/cve-api-key"
apiKeyFile: "/var/run/secrets/concelier/cve-api-key"
seedDirectory: "./seed-data/cve"
pageSize: 200
maxPagesPerFetch: 5
@@ -26,44 +26,44 @@ feedser:
failureBackoff: "00:10:00"
```
> ℹ️ Store the API key outside source control. When using `apiKeyFile`, mount the secret file into the container/host; alternatively supply `apiKey` via `FEEDSER_SOURCES__CVE__APIKEY`.
> ℹ️ Store the API key outside source control. When using `apiKeyFile`, mount the secret file into the container/host; alternatively supply `apiKey` via `CONCELIER_SOURCES__CVE__APIKEY`.
> 🪙 When credentials are not yet available, configure `seedDirectory` to point at mirrored CVE JSON (for example, the repo’s `seed-data/cve/` bundle). The connector will ingest those records and log a warning instead of failing the job; live fetching resumes automatically once `apiOrg` / `apiUser` / `apiKey` are supplied.
### 1.2 Smoke Test (staging)
1. Deploy the updated configuration and restart the Feedser service so the connector picks up the credentials.
1. Deploy the updated configuration and restart the Concelier service so the connector picks up the credentials.
2. Trigger one end-to-end cycle:
- Feedser CLI: `stella db jobs run source:cve:fetch --and-then source:cve:parse --and-then source:cve:map`
- Concelier CLI: `stella db jobs run source:cve:fetch --and-then source:cve:parse --and-then source:cve:map`
- REST fallback: `POST /jobs/run { "kind": "source:cve:fetch", "chain": ["source:cve:parse", "source:cve:map"] }`
3. Observe the following metrics (exported via OTEL meter `StellaOps.Feedser.Source.Cve`):
3. Observe the following metrics (exported via OTEL meter `StellaOps.Concelier.Source.Cve`):
- `cve.fetch.attempts`, `cve.fetch.success`, `cve.fetch.documents`, `cve.fetch.failures`, `cve.fetch.unchanged`
- `cve.parse.success`, `cve.parse.failures`, `cve.parse.quarantine`
- `cve.map.success`
4. Verify Prometheus shows matching `feedser.source.http.requests_total{feedser_source="cve"}` deltas (list vs detail phases) while `feedser.source.http.failures_total{feedser_source="cve"}` stays flat.
4. Verify Prometheus shows matching `concelier.source.http.requests_total{concelier_source="cve"}` deltas (list vs detail phases) while `concelier.source.http.failures_total{concelier_source="cve"}` stays flat.
5. Confirm the info-level summary log `CVEs fetch window … pages=X detailDocuments=Y detailFailures=Z` appears once per fetch run and shows `detailFailures=0`.
6. Verify the MongoDB advisory store contains fresh CVE advisories (`advisoryKey` prefix `cve/`) and that the source cursor (`source_states` collection) advanced.
### 1.3 Production Monitoring
- **Dashboards** – Plot `rate(cve_fetch_success_total[5m])`, `rate(cve_fetch_failures_total[5m])`, and `rate(cve_fetch_documents_total[5m])` alongside `feedser_source_http_requests_total{feedser_source="cve"}` to confirm HTTP and connector counters stay aligned. Keep `feedser.range.primitives{scheme=~"semver|vendor"}` on the same board for range coverage. Example alerts:
- **Dashboards** – Plot `rate(cve_fetch_success_total[5m])`, `rate(cve_fetch_failures_total[5m])`, and `rate(cve_fetch_documents_total[5m])` alongside `concelier_source_http_requests_total{concelier_source="cve"}` to confirm HTTP and connector counters stay aligned. Keep `concelier.range.primitives{scheme=~"semver|vendor"}` on the same board for range coverage. Example alerts:
- `rate(cve_fetch_failures_total[5m]) > 0` for 10 minutes (`severity=warning`)
- `rate(cve_map_success_total[15m]) == 0` while `rate(cve_fetch_success_total[15m]) > 0` (`severity=critical`)
- `sum_over_time(cve_parse_quarantine_total[1h]) > 0` to catch schema anomalies
- **Logs** – Monitor warnings such as `Failed fetching CVE record {CveId}` and `Malformed CVE JSON`, and surface the summary info log `CVEs fetch window … detailFailures=0 detailUnchanged=0` on dashboards. A non-zero `detailFailures` usually indicates rate-limit or auth issues on detail requests.
- **Grafana pack** – Import `docs/ops/feedser-cve-kev-grafana-dashboard.json` and filter by panel legend (`CVE`, `KEV`) to reuse the canned layout.
- **Backfill window** – Operators can tighten or widen `initialBackfill` / `maxPagesPerFetch` after validating throughput. Update config and restart Feedser to apply changes.
- **Grafana pack** – Import `docs/ops/concelier-cve-kev-grafana-dashboard.json` and filter by panel legend (`CVE`, `KEV`) to reuse the canned layout.
- **Backfill window** – Operators can tighten or widen `initialBackfill` / `maxPagesPerFetch` after validating throughput. Update config and restart Concelier to apply changes.
### 1.4 Staging smoke log (2025-10-15)
While Ops finalises long-lived CVE Services credentials, we validated the connector end-to-end against the recorded CVE-2024-0001 payloads used in regression tests:
- Command: `dotnet test src/StellaOps.Feedser.Source.Cve.Tests/StellaOps.Feedser.Source.Cve.Tests.csproj -l "console;verbosity=detailed"`
- Command: `dotnet test src/StellaOps.Concelier.Source.Cve.Tests/StellaOps.Concelier.Source.Cve.Tests.csproj -l "console;verbosity=detailed"`
- Summary log emitted by the connector:
```
CVEs fetch window 2024-09-01T00:00:00Z->2024-10-01T00:00:00Z pages=1 listSuccess=1 detailDocuments=1 detailFailures=0 detailUnchanged=0 pendingDocuments=0->1 pendingMappings=0->1 hasMorePages=False nextWindowStart=2024-09-15T12:00:00Z nextWindowEnd=(none) nextPage=1
```
- Telemetry captured by `Meter` `StellaOps.Feedser.Source.Cve`:
- Telemetry captured by `Meter` `StellaOps.Concelier.Source.Cve`:
| Metric | Value |
|--------|-------|
| `cve.fetch.attempts` | 1 |
@@ -72,7 +72,7 @@ While Ops finalises long-lived CVE Services credentials, we validated the connec
| `cve.parse.success` | 1 |
| `cve.map.success` | 1 |
The Grafana pack `docs/ops/feedser-cve-kev-grafana-dashboard.json` has been imported into staging so the panels referenced above render against these counters once the live API keys are in place.
The Grafana pack `docs/ops/concelier-cve-kev-grafana-dashboard.json` has been imported into staging so the panels referenced above render against these counters once the live API keys are in place.
## 2. CISA KEV Connector (`source:kev:*`)
@@ -80,10 +80,10 @@ The Grafana pack `docs/ops/feedser-cve-kev-grafana-dashboard.json` has been impo
- Network egress (or mirrored content) for `https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json`.
- No credentials are required, but the HTTP allow-list must include `www.cisa.gov`.
- Confirm the following snippet in `feedser.yaml` (defaults shown; tune as needed):
- Confirm the following snippet in `concelier.yaml` (defaults shown; tune as needed):
```yaml
feedser:
concelier:
sources:
kev:
feedUri: "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"
@@ -105,15 +105,15 @@ Treat repeated schema failures or growing anomaly counts as an upstream regressi
### 2.3 Smoke Test (staging)
1. Deploy the configuration and restart Feedser.
1. Deploy the configuration and restart Concelier.
2. Trigger a pipeline run:
- CLI: `stella db jobs run source:kev:fetch --and-then source:kev:parse --and-then source:kev:map`
- REST: `POST /jobs/run { "kind": "source:kev:fetch", "chain": ["source:kev:parse", "source:kev:map"] }`
3. Verify the metrics exposed by meter `StellaOps.Feedser.Source.Kev`:
3. Verify the metrics exposed by meter `StellaOps.Concelier.Source.Kev`:
- `kev.fetch.attempts`, `kev.fetch.success`, `kev.fetch.unchanged`, `kev.fetch.failures`
- `kev.parse.entries` (tag `catalogVersion`), `kev.parse.failures`, `kev.parse.anomalies` (tag `reason`)
- `kev.map.advisories` (tag `catalogVersion`)
4. Confirm `feedser.source.http.requests_total{feedser_source="kev"}` increments once per fetch and that the paired `feedser.source.http.failures_total` stays flat (zero increase).
4. Confirm `concelier.source.http.requests_total{concelier_source="kev"}` increments once per fetch and that the paired `concelier.source.http.failures_total` stays flat (zero increase).
5. Inspect the info logs `Fetched KEV catalog document … pendingDocuments=…` and `Parsed KEV catalog document … entries=…`—they should appear exactly once per run and `Mapped X/Y… skipped=0` should match the `kev.map.advisories` delta.
6. Confirm MongoDB documents exist for the catalog JSON (`raw_documents` & `dtos`) and that advisories with prefix `kev/` are written.
@@ -126,7 +126,7 @@ Treat repeated schema failures or growing anomaly counts as an upstream regressi
### 2.5 Known good dashboard tiles
Add the following panels to the Feedser observability board:
Add the following panels to the Concelier observability board:
| Metric | Recommended visualisation |
|--------|---------------------------|
@@ -140,4 +140,4 @@ Add the following panels to the Feedser observability board:
- Record staging/production smoke test results (date, catalog version, advisory counts) in your team’s change log.
- Add the CVE/KEV job kinds to the standard maintenance checklist so operators can manually trigger them after planned downtime.
- Keep this document in sync with future connector changes (for example, new anomaly reasons or additional metrics).
- Version-control dashboard tweaks alongside `docs/ops/feedser-cve-kev-grafana-dashboard.json` so operations can re-import the observability pack during restores.
- Version-control dashboard tweaks alongside `docs/ops/concelier-cve-kev-grafana-dashboard.json` so operations can re-import the observability pack during restores.

28
docs/ops/feedser-ghsa-operations.md → docs/ops/concelier-ghsa-operations.md
View File

@@ -1,4 +1,4 @@
# Feedser GHSA Connector – Operations Runbook
# Concelier GHSA Connector – Operations Runbook
_Last updated: 2025-10-16_
@@ -32,9 +32,9 @@ When GitHub reports zero remaining calls, the connector logs and sleeps for the
After the quota recovers above the warning threshold the connector writes an informational log with the refreshed remaining/headroom, letting operators clear alerts quickly.
## 4. Configuration knobs (`feedser.yaml`)
## 4. Configuration knobs (`concelier.yaml`)
```yaml
feedser:
concelier:
sources:
ghsa:
apiToken: "${GITHUB_PAT}"
@@ -58,18 +58,18 @@ feedser:
| `source:ghsa:parse` | `3,13,23,33,43,53 * * * *` | 5 minutes | 4 minutes |
| `source:ghsa:map` | `5,15,25,35,45,55 * * * *` | 5 minutes | 4 minutes |
These defaults spread GHSA stages across the hour so fetch completes before parse/map fire. Override them via `feedser.jobs.definitions[...]` when coordinating multiple connectors on the same runner.
These defaults spread GHSA stages across the hour so fetch completes before parse/map fire. Override them via `concelier.jobs.definitions[...]` when coordinating multiple connectors on the same runner.
## 5. Provisioning credentials
Feedser requires a GitHub personal access token (classic) with the **`read:org`** and **`security_events`** scopes to pull GHSA data. Store it as a secret and reference it via `feedser.sources.ghsa.apiToken`.
Concelier requires a GitHub personal access token (classic) with the **`read:org`** and **`security_events`** scopes to pull GHSA data. Store it as a secret and reference it via `concelier.sources.ghsa.apiToken`.
### Docker Compose (stack operators)
```yaml
services:
feedser:
concelier:
environment:
FEEDSER__SOURCES__GHSA__APITOKEN: /run/secrets/ghsa_pat
CONCELIER__SOURCES__GHSA__APITOKEN: /run/secrets/ghsa_pat
secrets:
- ghsa_pat
@@ -80,25 +80,25 @@ secrets:
### Helm values (cluster operators)
```yaml
feedser:
concelier:
extraEnv:
- name: FEEDSER__SOURCES__GHSA__APITOKEN
- name: CONCELIER__SOURCES__GHSA__APITOKEN
valueFrom:
secretKeyRef:
name: feedser-ghsa
name: concelier-ghsa
key: apiToken
extraSecrets:
feedser-ghsa:
concelier-ghsa:
apiToken: "<paste PAT here or source from external secret store>"
```
After rotating the PAT, restart the Feedser workers (or run `kubectl rollout restart deployment/feedser`) to ensure the configuration reloads.
After rotating the PAT, restart the Concelier workers (or run `kubectl rollout restart deployment/concelier`) to ensure the configuration reloads.
When enabling GHSA the first time, run a staged backfill:
1. Trigger `source:ghsa:fetch` manually (CLI or API) outside of peak hours.
2. Watch `feedser.jobs.health` for the GHSA jobs until they report `healthy`.
2. Watch `concelier.jobs.health` for the GHSA jobs until they report `healthy`.
3. Allow the scheduled cron cadence to resume once the initial backlog drains (typically < 30 minutes).
## 6. Runbook steps when throttled
@@ -107,7 +107,7 @@ When enabling GHSA the first time, run a staged backfill:
3. If rate limits stay exhausted:
- Verify no other jobs are sharing the PAT.
- Temporarily reduce `MaxPagesPerFetch` or `PageSize` to shrink burst size.
- Consider provisioning a dedicated PAT (GHSA permissions only) for Feedser.
- Consider provisioning a dedicated PAT (GHSA permissions only) for Concelier.
4. After the quota resets, reset `rateLimitWarningThreshold`/`requestDelay` to their normal values and monitor the histograms for at least one hour.
## 7. Alert integration quick reference

40
docs/ops/feedser-icscisa-operations.md → docs/ops/concelier-icscisa-operations.md
View File

@@ -1,4 +1,4 @@
# Feedser CISA ICS Connector Operations
# Concelier CISA ICS Connector Operations
This runbook documents how to provision, rotate, and validate credentials for the CISA Industrial Control Systems (ICS) connector (`source:ics-cisa:*`). Follow it before enabling the connector in staging or offline installations.
@@ -10,16 +10,16 @@ This runbook documents how to provision, rotate, and validate credentials for th
- `USDHSCISA_19` — ICS medical advisories (`ICSMA-YY-###`).
- `USDHSCISA_17` — ICS alerts (`IR-ALERT-YY-###`) for completeness.
3. Complete the verification email. After confirmation, note the **personalised subscription code** included in the “Manage Preferences” link. It has the shape `code=AB12CD34EF`.
4. Store the code in the shared secret vault (or Offline Kit secrets bundle) as `feedser/sources/icscisa/govdelivery/code`.
4. Store the code in the shared secret vault (or Offline Kit secrets bundle) as `concelier/sources/icscisa/govdelivery/code`.
> ℹ️ GovDelivery does not expose a one-time API key; the personalised code is what authenticates the RSS pull. Never commit it to git.
## 2. Feed Validation
Use the following command to confirm the feed is reachable before wiring it into Feedser (substitute `<CODE>` with the personalised value):
Use the following command to confirm the feed is reachable before wiring it into Concelier (substitute `<CODE>` with the personalised value):
```bash
curl -H "User-Agent: StellaOpsFeedser/ics-cisa" \
curl -H "User-Agent: StellaOpsConcelier/ics-cisa" \
"https://content.govdelivery.com/accounts/USDHSCISA/topics/ICS-CERT/feed.rss?format=xml&code=<CODE>"
```
@@ -27,14 +27,14 @@ If the endpoint returns HTTP 200 and an RSS payload, record the sample response
## 3. Configuration Snippet
Add the connector configuration to `feedser.yaml` (or equivalent environment variables):
Add the connector configuration to `concelier.yaml` (or equivalent environment variables):
```yaml
feedser:
concelier:
sources:
icscisa:
govDelivery:
code: "${FEEDSER_ICS_CISA_GOVDELIVERY_CODE}"
code: "${CONCELIER_ICS_CISA_GOVDELIVERY_CODE}"
topics:
- "USDHSCISA_16"
- "USDHSCISA_19"
@@ -47,10 +47,10 @@ feedser:
Environment variable example:
```bash
export FEEDSER_SOURCES_ICSCISA_GOVDELIVERY_CODE="AB12CD34EF"
export CONCELIER_SOURCES_ICSCISA_GOVDELIVERY_CODE="AB12CD34EF"
```
Feedser automatically register the host with the Source.Common HTTP allow-list when the connector assembly is loaded.
Concelier automatically register the host with the Source.Common HTTP allow-list when the connector assembly is loaded.
Optional tuning keys (set only when needed):
@@ -68,21 +68,21 @@ If credentials are still pending, populate the connector with the community CSV
1. Run `./scripts/fetch-ics-cisa-seed.sh` (or `.ps1`) to download the latest `CISA_ICS_ADV_*.csv` files into `seed-data/ics-cisa/`.
2. Copy the CSVs (and the generated `.sha256` files) into your Offline Kit staging area so they ship alongside the other feeds.
3. Import the kit as usual. The connector can parse the seed data for historical context, but **live GovDelivery credentials are still required** for fresh advisories.
4. Once credentials arrive, update `feedser:sources:icscisa:govDelivery:code` and re-trigger `source:ics-cisa:fetch` so the connector switches to the authorised feed.
4. Once credentials arrive, update `concelier:sources:icscisa:govDelivery:code` and re-trigger `source:ics-cisa:fetch` so the connector switches to the authorised feed.
> The CSVs are licensed under ODbL 1.0 by the ICS Advisory Project. Preserve the attribution when redistributing them.
## 4. Integration Validation
1. Ensure secrets are in place and restart the Feedser workers.
1. Ensure secrets are in place and restart the Concelier workers.
2. Run a dry-run fetch/parse/map chain against an Akamai-protected topic:
```bash
FEEDSER_SOURCES_ICSCISA_GOVDELIVERY_CODE=... \
FEEDSER_SOURCES_ICSCISA_ENABLEDETAILSCRAPE=1 \
CONCELIER_SOURCES_ICSCISA_GOVDELIVERY_CODE=... \
CONCELIER_SOURCES_ICSCISA_ENABLEDETAILSCRAPE=1 \
stella db jobs run source:ics-cisa:fetch --and-then source:ics-cisa:parse --and-then source:ics-cisa:map
```
3. Confirm logs contain `ics-cisa detail fetch` entries and that new documents/DTOs include attachments (see `docs/artifacts/icscisa`). Canonical advisories should expose PDF links as `references.kind == "attachment"` and affected packages should surface `primitives.semVer.exactValue` for single-version hits.
4. If Akamai blocks direct fetches, set `feedser:sources:icscisa:proxyUri` to your allow-listed egress proxy and rerun the dry-run.
4. If Akamai blocks direct fetches, set `concelier:sources:icscisa:proxyUri` to your allow-listed egress proxy and rerun the dry-run.
## 4. Rotation & Incident Response
@@ -92,19 +92,19 @@ If credentials are still pending, populate the connector with the community CSV
## 5. Offline Kit Handling
Include the personalised code in `offline-kit/secrets/feedser/icscisa.env`:
Include the personalised code in `offline-kit/secrets/concelier/icscisa.env`:
```
FEEDSER_SOURCES_ICSCISA_GOVDELIVERY_CODE=AB12CD34EF
CONCELIER_SOURCES_ICSCISA_GOVDELIVERY_CODE=AB12CD34EF
```
The Offline Kit deployment script copies this file into the container secret directory mounted at `/run/secrets/feedser`. Ensure permissions are `600` and ownership matches the Feedser runtime user.
The Offline Kit deployment script copies this file into the container secret directory mounted at `/run/secrets/concelier`. Ensure permissions are `600` and ownership matches the Concelier runtime user.
## 6. Telemetry & Monitoring
The connector emits metrics under the meter `StellaOps.Feedser.Source.Ics.Cisa`. They allow operators to track Akamai fallbacks, detail enrichment health, and advisory fan-out.
The connector emits metrics under the meter `StellaOps.Concelier.Source.Ics.Cisa`. They allow operators to track Akamai fallbacks, detail enrichment health, and advisory fan-out.
- `icscisa.fetch.*` – counters for `attempts`, `success`, `failures`, `not_modified`, and `fallbacks`, plus histogram `icscisa.fetch.documents` showing documents added per topic pull (tags: `feedser.source`, `icscisa.topic`).
- `icscisa.fetch.*` – counters for `attempts`, `success`, `failures`, `not_modified`, and `fallbacks`, plus histogram `icscisa.fetch.documents` showing documents added per topic pull (tags: `concelier.source`, `icscisa.topic`).
- `icscisa.parse.*` – counters for `success`/`failures` and histograms `icscisa.parse.advisories`, `icscisa.parse.attachments`, `icscisa.parse.detail_fetches` to monitor enrichment workload per feed document.
- `icscisa.detail.*` – counters `success` / `failures` per advisory (tagged with `icscisa.advisory`) to alert when Akamai blocks detail pages.
- `icscisa.map.*` – counters for `success`/`failures` and histograms `icscisa.map.references`, `icscisa.map.packages`, `icscisa.map.aliases` capturing canonical fan-out.
@@ -114,7 +114,7 @@ Suggested alerts:
- `increase(icscisa.fetch.failures_total[15m]) > 0` or `increase(icscisa.fetch.fallbacks_total[15m]) > 5` — sustained Akamai or proxy issues.
- `increase(icscisa.detail.failures_total[30m]) > 0` — detail enrichment breaking (potential HTML layout change).
- `histogram_quantile(0.95, rate(icscisa.map.references_bucket[1h]))` trending sharply higher — sudden advisory reference explosion worth investigating.
- Keep an eye on shared HTTP metrics (`feedser.source.http.*{feedser.source="ics-cisa"}`) for request latency and retry patterns.
- Keep an eye on shared HTTP metrics (`concelier.source.http.*{concelier.source="ics-cisa"}`) for request latency and retry patterns.
## 6. Related Tasks

18
docs/ops/feedser-kisa-operations.md → docs/ops/concelier-kisa-operations.md
View File

@@ -1,14 +1,14 @@
# Feedser KISA Connector Operations
# Concelier KISA Connector Operations
Operational guidance for the Korea Internet & Security Agency (KISA / KNVD) connector (`source:kisa:*`). Pair this with the engineering brief in `docs/dev/kisa_connector_notes.md`.
## 1. Prerequisites
- Outbound HTTPS (or mirrored cache) for `https://knvd.krcert.or.kr/`.
- Connector options defined under `feedser:sources:kisa`:
- Connector options defined under `concelier:sources:kisa`:
```yaml
feedser:
concelier:
sources:
kisa:
feedUri: "https://knvd.krcert.or.kr/rss/securityInfo.do"
@@ -19,15 +19,15 @@ feedser:
failureBackoff: "00:05:00"
```
> Ensure the URIs stay absolute—Feedser adds the `feedUri`/`detailApiUri` hosts to the HttpClient allow-list automatically.
> Ensure the URIs stay absolute—Concelier adds the `feedUri`/`detailApiUri` hosts to the HttpClient allow-list automatically.
## 2. Staging Smoke Test
1. Restart the Feedser workers so the KISA options bind.
1. Restart the Concelier workers so the KISA options bind.
2. Run a full connector cycle:
- CLI: `stella db jobs run source:kisa:fetch --and-then source:kisa:parse --and-then source:kisa:map`
- REST: `POST /jobs/run { "kind": "source:kisa:fetch", "chain": ["source:kisa:parse", "source:kisa:map"] }`
3. Confirm telemetry (Meter `StellaOps.Feedser.Source.Kisa`):
3. Confirm telemetry (Meter `StellaOps.Concelier.Source.Kisa`):
- `kisa.feed.success`, `kisa.feed.items`
- `kisa.detail.success` / `.failures`
- `kisa.parse.success` / `.failures`
@@ -47,7 +47,7 @@ feedser:
## 3. Production Monitoring
- **Dashboards** – Add the following Prometheus/OTEL expressions:
- `rate(kisa_feed_items_total[15m])` versus `rate(feedser_source_http_requests_total{feedser_source="kisa"}[15m])`
- `rate(kisa_feed_items_total[15m])` versus `rate(concelier_source_http_requests_total{concelier_source="kisa"}[15m])`
- `increase(kisa_detail_failures_total{reason!="empty-document"}[1h])` alert at `>0`
- `increase(kisa_parse_failures_total[1h])` for storage/JSON issues
- `increase(kisa_map_failures_total[1h])` to flag schema drift
@@ -63,8 +63,8 @@ feedser:
## 5. Fixture & Regression Maintenance
- Regression fixtures: `src/StellaOps.Feedser.Source.Kisa.Tests/Fixtures/kisa-feed.xml` and `kisa-detail.json`.
- Refresh via `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Feedser.Source.Kisa.Tests/StellaOps.Feedser.Source.Kisa.Tests.csproj`.
- Regression fixtures: `src/StellaOps.Concelier.Source.Kisa.Tests/Fixtures/kisa-feed.xml` and `kisa-detail.json`.
- Refresh via `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Kisa.Tests/StellaOps.Concelier.Source.Kisa.Tests.csproj`.
- The telemetry regression (`KisaConnectorTests.Telemetry_RecordsMetrics`) will fail if counters/log wiring drifts—treat failures as gating.
## 6. Known Issues

14
docs/ops/feedser-msrc-operations.md → docs/ops/concelier-msrc-operations.md
View File

@@ -1,4 +1,4 @@
# Feedser MSRC Connector – Azure AD Onboarding Brief
# Concelier MSRC Connector – Azure AD Onboarding Brief
_Drafted: 2025-10-15_
@@ -8,18 +8,18 @@ _Drafted: 2025-10-15_
- **Application type**: confidential client (web/API) issuing client credentials.
- **API permissions**: `api://api.msrc.microsoft.com/.default` (Application). Admin consent required once.
- **Token audience**: `https://api.msrc.microsoft.com/`.
- **Grant type**: client credentials. Feedser will request tokens via `POST https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token`.
- **Grant type**: client credentials. Concelier will request tokens via `POST https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token`.
## 2. Secret/credential policy
- Maintain two client secrets (primary + standby) rotating every 90 days.
- Store secrets in the Feedser secrets vault; Offline Kit deployments must mirror the secret payloads in their encrypted store.
- Record rotation cadence in Ops runbook and update Feedser configuration (`FEEDSER__SOURCES__VNDR__MSRC__CLIENTSECRET`) ahead of expiry.
- Store secrets in the Concelier secrets vault; Offline Kit deployments must mirror the secret payloads in their encrypted store.
- Record rotation cadence in Ops runbook and update Concelier configuration (`CONCELIER__SOURCES__VNDR__MSRC__CLIENTSECRET`) ahead of expiry.
## 3. Feedser configuration sample
## 3. Concelier configuration sample
```yaml
feedser:
concelier:
sources:
vndr.msrc:
tenantId: "<azure-tenant-guid>"
@@ -73,7 +73,7 @@ Run the helper:
```bash
dotnet run --project tools/SourceStateSeeder -- \
--connection-string "mongodb://localhost:27017" \
--database feedser \
--database concelier \
--input seeds/msrc-backfill.json
```

12
docs/ops/feedser-nkcki-operations.md → docs/ops/concelier-nkcki-operations.md
View File

@@ -6,7 +6,7 @@ The NKCKI connector ingests JSON bulletin archives from cert.gov.ru, expanding e
## Configuration
Key options exposed through `feedser:sources:ru-nkcki:http`:
Key options exposed through `concelier:sources:ru-nkcki:http`:
- `maxBulletinsPerFetch` – limits new bulletin downloads in a single run (default `5`).
- `maxListingPagesPerFetch` – maximum listing pages visited during pagination (default `3`).
@@ -14,11 +14,11 @@ Key options exposed through `feedser:sources:ru-nkcki:http`:
- `cacheDirectory` – optional path for persisted bulletin archives used during offline or failure scenarios.
- `requestDelay` – delay inserted between bulletin downloads to respect upstream politeness.
When operating in offline-first mode, set `cacheDirectory` to a writable path (e.g. `/var/lib/feedser/cache/ru-nkcki`) and pre-populate bulletin archives via the offline kit.
When operating in offline-first mode, set `cacheDirectory` to a writable path (e.g. `/var/lib/concelier/cache/ru-nkcki`) and pre-populate bulletin archives via the offline kit.
## Telemetry
`RuNkckiDiagnostics` emits the following metrics under meter `StellaOps.Feedser.Source.Ru.Nkcki`:
`RuNkckiDiagnostics` emits the following metrics under meter `StellaOps.Concelier.Source.Ru.Nkcki`:
- `nkcki.listing.fetch.attempts` / `nkcki.listing.fetch.success` / `nkcki.listing.fetch.failures`
- `nkcki.listing.pages.visited` (histogram, `pages`)
@@ -26,7 +26,7 @@ When operating in offline-first mode, set `cacheDirectory` to a writable path (e
- `nkcki.bulletin.fetch.success` / `nkcki.bulletin.fetch.cached` / `nkcki.bulletin.fetch.failures`
- `nkcki.entries.processed` (histogram, `entries`)
Integrate these counters into standard Feedser observability dashboards to track crawl coverage and cache hit rates.
Integrate these counters into standard Concelier observability dashboards to track crawl coverage and cache hit rates.
## Archive Backfill Strategy
@@ -35,7 +35,7 @@ Bitrix pagination surfaces archives via `?PAGEN_1=n`. The connector now walks up
1. Enumerate pages from newest to oldest, respecting `maxListingPagesPerFetch` and `listingCacheDuration` to avoid refetch storms.
2. Persist every `*.json.zip` attachment to the configured cache directory. This enables replay when listing access is temporarily blocked.
3. During archive replay, `ProcessCachedBulletinsAsync` enqueues missing documents while respecting `maxVulnerabilitiesPerFetch`.
4. For historical HTML-only advisories, collect page URLs and metadata while offline (future work: HTML and PDF extraction pipeline documented in `docs/feedser-connector-research-20251011.md`).
4. For historical HTML-only advisories, collect page URLs and metadata while offline (future work: HTML and PDF extraction pipeline documented in `docs/concelier-connector-research-20251011.md`).
For large migrations, seed caches with archived zip bundles, then run fetch/parse/map cycles in chronological order to maintain deterministic outputs.
@@ -45,4 +45,4 @@ For large migrations, seed caches with archived zip bundles, then run fetch/pars
- Bulletin fetches fall back to cached copies before surfacing an error.
- Mongo integration tests rely on bundled OpenSSL 1.1 libraries (`tools/openssl/linux-x64`) to keep `Mongo2Go` operational on modern distros.
Refer to `ru-nkcki` entries in `src/StellaOps.Feedser.Source.Ru.Nkcki/TASKS.md` for outstanding items.
Refer to `ru-nkcki` entries in `src/StellaOps.Concelier.Source.Ru.Nkcki/TASKS.md` for outstanding items.

4
docs/ops/feedser-osv-operations.md → docs/ops/concelier-osv-operations.md
View File

@@ -1,4 +1,4 @@
# Feedser OSV Connector – Operations Notes
# Concelier OSV Connector – Operations Notes
_Last updated: 2025-10-16_
@@ -20,5 +20,5 @@ The OSV connector ingests advisories from OSV.dev across OSS ecosystems. This no
- Exporters already surface `canonicalMetricId`; no schema change is required, but ORAS/Trivy bundles should be spot-checked after deploying the connector update.
## 4. Runbook updates
- Fixture parity suites (`osv-ghsa.*`) now assert the fallback id and provenance notes. Regenerate via `dotnet test src/StellaOps.Feedser.Source.Osv.Tests/StellaOps.Feedser.Source.Osv.Tests.csproj`.
- Fixture parity suites (`osv-ghsa.*`) now assert the fallback id and provenance notes. Regenerate via `dotnet test src/StellaOps.Concelier.Source.Osv.Tests/StellaOps.Concelier.Source.Osv.Tests.csproj`.
- When investigating merge severity conflicts, include the fallback counter and confirm OSV advisories carry the expected `osv:severity/<level>` id before raising connector bugs.

14
docs/ops/migrations/SEMVER_STYLE.md
View File

@@ -6,12 +6,12 @@ _Last updated: 2025-10-11_
The SemVer style migration populates the new `normalizedVersions` field on advisory documents and ensures
provenance `decisionReason` values are preserved during future reads. The migration is idempotent and only
runs when the feature flag `feedser:storage:enableSemVerStyle` is enabled.
runs when the feature flag `concelier:storage:enableSemVerStyle` is enabled.
## Preconditions
1. **Review configuration** – set `feedser.storage.enableSemVerStyle` to `true` on all Feedser services.
2. **Confirm batch size** – adjust `feedser.storage.backfillBatchSize` if you need smaller batches for older
1. **Review configuration** – set `concelier.storage.enableSemVerStyle` to `true` on all Concelier services.
2. **Confirm batch size** – adjust `concelier.storage.backfillBatchSize` if you need smaller batches for older
deployments (default: `250`).
3. **Back up** – capture a fresh snapshot of the `advisory` collection or a full MongoDB backup.
4. **Staging dry-run** – enable the flag in a staging environment and observe the migration output before
@@ -19,7 +19,7 @@ runs when the feature flag `feedser:storage:enableSemVerStyle` is enabled.
## Execution
No manual command is required. After deploying the configuration change, restart the Feedser WebService or
No manual command is required. After deploying the configuration change, restart the Concelier WebService or
any component that hosts the Mongo migration runner. During startup you will see log entries similar to:
```
@@ -27,7 +27,7 @@ Applying Mongo migration 20251011-semver-style-backfill: Populate advisory.norma
Mongo migration 20251011-semver-style-backfill applied
```
The migration reads advisories in batches (`feedser.storage.backfillBatchSize`) and writes flattened
The migration reads advisories in batches (`concelier.storage.backfillBatchSize`) and writes flattened
`normalizedVersions` arrays. Existing documents without SemVer ranges remain untouched.
## Post-checks
@@ -39,12 +39,12 @@ The migration reads advisories in batches (`feedser.storage.backfillBatchSize`)
You should see `advisory_normalizedVersions_pkg_scheme_type` and `advisory_normalizedVersions_value`.
2. Spot check a few advisories to confirm the top-level `normalizedVersions` array exists and matches
the embedded package data.
3. Run `dotnet test` for `StellaOps.Feedser.Storage.Mongo.Tests` (optional but recommended) in CI to confirm
3. Run `dotnet test` for `StellaOps.Concelier.Storage.Mongo.Tests` (optional but recommended) in CI to confirm
the storage suite passes with the feature flag enabled.
## Rollback
Set `feedser.storage.enableSemVerStyle` back to `false` and redeploy. The migration will be skipped on
Set `concelier.storage.enableSemVerStyle` back to `false` and redeploy. The migration will be skipped on
subsequent startups. You can leave the populated `normalizedVersions` arrays in place; they are ignored when
the feature flag is off. If you must remove them entirely, restore from the backup captured during
preparation.

4
docs/security/authority-threat-model.md
View File

@@ -14,7 +14,7 @@
|-----------------|-------------|----------------|
| Token issuance APIs (`/token`, `/authorize`) | OAuth/OIDC endpoints mediated by OpenIddict | CLI, UI, automation agents |
| Bootstrap channel | Initial admin invite + bootstrap CLI workflow | Platform operators |
| Revocation bundle | Offline JSON + detached JWS consumed by agents | Feedser, Agents, Zastava |
| Revocation bundle | Offline JSON + detached JWS consumed by agents | Concelier, Agents, Zastava |
| Plug-in manifests | Standard plug-in configuration and password policy overrides | Operators, DevOps |
| Signing keys | ES256 signing keys backing tokens and revocation manifests | Security Guild, HSM/KeyOps |
| Audit telemetry | Structured login/audit stream persisted to Mongo/observability stack | SOC, SecOps |
@@ -68,7 +68,7 @@ flowchart LR
end
subgraph Distribution
OFFKIT[Offline Kit Bundle]
AGENT[Authorized Agent / Feedser]
AGENT[Authorized Agent / Concelier]
end
OPS -->|Bootstrap CLI (`stellaops auth bootstrap`)| AUTH
AUTH -->|One-time invite + Argon2 hash| STORE

6
docs/security/revocation-bundle-example.json
View File

@@ -12,14 +12,14 @@
"id": "7ad4f3d2c21b461d9b3420e1151be9c4",
"category": "token",
"tokenType": "access_token",
"clientId": "feedser-cli",
"clientId": "concelier-cli",
"subjectId": "user:ops-admin",
"reason": "compromised",
"reasonDescription": "Access token reported by SOC automation run R-2045.",
"revokedAt": "2025-10-12T14:32:05Z",
"scopes": [
"feedser:export",
"feedser:jobs"
"concelier:export",
"concelier:jobs"
],
"fingerprint": "AD35E719C12204D7E7C92ED3F6DEBF0A44642D41AAF94233F9A47E183F4C5F18",
"metadata": {

4
docs/security/revocation-bundle.md
View File

@@ -1,6 +1,6 @@
# Authority Revocation Bundle
The Authority service exports revocation information as an offline-friendly JSON document plus a detached JWS signature. Operators can mirror the bundle alongside Feedser exports to ensure air-gapped scanners receive the latest token, subject, and client revocations.
The Authority service exports revocation information as an offline-friendly JSON document plus a detached JWS signature. Operators can mirror the bundle alongside Concelier exports to ensure air-gapped scanners receive the latest token, subject, and client revocations.
## File layout
@@ -88,4 +88,4 @@ The repository contains an [example bundle](revocation-bundle-example.json) demo
- `stella auth revoke verify` validates a bundle using cached JWKS or an offline PEM key, honours the `provider` metadata embedded in the signature, and reports digest mismatches before distribution.
- `POST /internal/revocations/export` provides the same payload for orchestrators that already talk to the bootstrap API.
- `POST /internal/signing/rotate` rotates JWKS material without downtime; always export a fresh bundle afterward so downstream mirrors receive signatures from the new `kid`.
- Offline Kit automation should mirror `revocation-bundle.json*` alongside Feedser exports so agents ingest revocations during the same sync pass.
- Offline Kit automation should mirror `revocation-bundle.json*` alongside Concelier exports so agents ingest revocations during the same sync pass.