stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Align AOC tasks for Excititor and Concelier

Browse Source
This commit is contained in:
master
2025-10-31 18:50:15 +02:00
committed by root
parent 9e6d9fbae8
commit 8da4e12a90
334 changed files with 35528 additions and 34546 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
.gitignore vendored
View File

@@ -1,34 +1,34 @@
# Build outputs # Build outputs
bin/ bin/
obj/ obj/
*.pdb *.pdb
*.dll *.dll
# IDE state # IDE state
.vs/ .vs/
*.user *.user
*.suo *.suo
*.userprefs *.userprefs
# Rider/VSCode # Rider/VSCode
.idea/ .idea/
.vscode/ .vscode/
# Packages and logs # Packages and logs
*.log *.log
TestResults/ TestResults/
.dotnet .dotnet
.DS_Store .DS_Store
seed-data/ics-cisa/*.csv seed-data/ics-cisa/*.csv
seed-data/ics-cisa/*.xlsx seed-data/ics-cisa/*.xlsx
seed-data/ics-cisa/*.sha256 seed-data/ics-cisa/*.sha256
seed-data/cert-bund/**/*.json seed-data/cert-bund/**/*.json
seed-data/cert-bund/**/*.sha256 seed-data/cert-bund/**/*.sha256
out/offline-kit/web/**/* out/offline-kit/web/**/*
**/node_modules/**/* **/node_modules/**/*
**/.angular/**/* **/.angular/**/*
**/.cache/**/* **/.cache/**/*
**/dist/**/* **/dist/**/*
tmp/**/* tmp/**/*

412
AGENTS.md
View File

@@ -1,206 +1,206 @@
# 1) What is StellaOps? # 1) What is StellaOps?
**StellaOps** an next-gen and sovereign container-security toolkit built for high-speed, offline operation, released under AGPL-3.0-or-later. **StellaOps** an next-gen and sovereign container-security toolkit built for high-speed, offline operation, released under AGPL-3.0-or-later.
Stella Ops is a self-hostable, sovereign container-security platform that makes proof—not promises—default. It binds every container digest to content-addressed SBOMs (SBOM 3.0.0 and CycloneDX 1.6), in-toto/DSSE attestations, and optional Sigstore Rekor transparency, then layers deterministic, replayable scanning with entry-trace and VEX-first decisioning. “Next-gen” means findings are reproducible and explainable, exploitability is modeled in OpenVEX and merged with lattice logic for stable outcomes, and the same workflow runs online or fully air-gapped. “Sovereign” means cryptographic and operational independence: bring-your-own trust roots, regional crypto readiness (eIDAS/FIPS/GOST/SM), offline bundles, and post-quantum-ready modes—so regulated orgs can comply without phoning home. Stella Ops is a self-hostable, sovereign container-security platform that makes proof—not promises—default. It binds every container digest to content-addressed SBOMs (SBOM 3.0.0 and CycloneDX 1.6), in-toto/DSSE attestations, and optional Sigstore Rekor transparency, then layers deterministic, replayable scanning with entry-trace and VEX-first decisioning. “Next-gen” means findings are reproducible and explainable, exploitability is modeled in OpenVEX and merged with lattice logic for stable outcomes, and the same workflow runs online or fully air-gapped. “Sovereign” means cryptographic and operational independence: bring-your-own trust roots, regional crypto readiness (eIDAS/FIPS/GOST/SM), offline bundles, and post-quantum-ready modes—so regulated orgs can comply without phoning home.
Our principles and goals are simple: authenticity & integrity by default, provenance attached to digests, transparency for tamper-evidence, determinism & replay for audits, explainability engineers can act on, and exploitability over enumeration to cut noise. We minimize trust and blast radius with short-lived keys, least-privilege, and content-addressed caches; we stay air-gap friendly with mirrored feeds; and we keep governance honest with reviewable OPA/Rego policy gates and VEX-based waivers. The result is a platform that shortens time-to-truth, makes risk measurable, and lets you ship with confidence—anywhere, under any sovereignty requirement. Our principles and goals are simple: authenticity & integrity by default, provenance attached to digests, transparency for tamper-evidence, determinism & replay for audits, explainability engineers can act on, and exploitability over enumeration to cut noise. We minimize trust and blast radius with short-lived keys, least-privilege, and content-addressed caches; we stay air-gap friendly with mirrored feeds; and we keep governance honest with reviewable OPA/Rego policy gates and VEX-based waivers. The result is a platform that shortens time-to-truth, makes risk measurable, and lets you ship with confidence—anywhere, under any sovereignty requirement.
More documention is available ./docs/*.md files. Read `docs/README.md` to gather information about the available documentation. You could inquiry specific documents as your work requires it More documention is available ./docs/*.md files. Read `docs/README.md` to gather information about the available documentation. You could inquiry specific documents as your work requires it
--- ---
# 3) Practices # 3) Practices
## 3.1) Naming ## 3.1) Naming
All modules are .NET projects based on .NET 10 (preview). Exclussion is the UI. It is based on Angular All modules are .NET projects based on .NET 10 (preview). Exclussion is the UI. It is based on Angular
All modules are contained by one or more projects. Each project goes in its dedicated folder. Each project starts with StellaOps.<ModuleName>. In case it is common for for all StellaOps modules it is library or plugin and it is named StellaOps.<LibraryOrPlugin>. All modules are contained by one or more projects. Each project goes in its dedicated folder. Each project starts with StellaOps.<ModuleName>. In case it is common for for all StellaOps modules it is library or plugin and it is named StellaOps.<LibraryOrPlugin>.
## 3.2) Key technologies & integrations ## 3.2) Key technologies & integrations
- **Runtime**: .NET 10 (`net10.0`) preview SDK; C# latest preview features. Any dependencies like Microsoft.* should strive to be closests version. - **Runtime**: .NET 10 (`net10.0`) preview SDK; C# latest preview features. Any dependencies like Microsoft.* should strive to be closests version.
- **Nuget**: Try to re-use / cache nugets to /local-nugets - **Nuget**: Try to re-use / cache nugets to /local-nugets
- **Data**: MongoDB (canonical store and job/export state). MongoDB driver version should be > 3.0 - **Data**: MongoDB (canonical store and job/export state). MongoDB driver version should be > 3.0
- **Observability**: structured logs, counters, and (optional) OpenTelemetry traces. - **Observability**: structured logs, counters, and (optional) OpenTelemetry traces.
- **Ops posture**: offline‑first, allowlist for remote hosts, strict schema validation, gated LLM fallback (only where explicitly configured). - **Ops posture**: offline‑first, allowlist for remote hosts, strict schema validation, gated LLM fallback (only where explicitly configured).
# 4) Modules # 4) Modules
StellaOps ships as containerised building blocks; each module owns a clear boundary and has its own code folder, deployable image, and deep-dive architecture dossier. StellaOps ships as containerised building blocks; each module owns a clear boundary and has its own code folder, deployable image, and deep-dive architecture dossier.
| Module | Primary path(s) | Key doc | | Module | Primary path(s) | Key doc |
|--------|-----------------|---------| |--------|-----------------|---------|
| Authority | `src/Authority/StellaOps.Authority`<br>`src/Authority/StellaOps.Authority.Plugin.*` | `docs/modules/authority/architecture.md` | | Authority | `src/Authority/StellaOps.Authority`<br>`src/Authority/StellaOps.Authority.Plugin.*` | `docs/modules/authority/architecture.md` |
| Signer | `src/Signer/StellaOps.Signer` | `docs/modules/signer/architecture.md` | | Signer | `src/Signer/StellaOps.Signer` | `docs/modules/signer/architecture.md` |
| Attestor | `src/Attestor/StellaOps.Attestor`<br>`src/Attestor/StellaOps.Attestor.Verify` | `docs/modules/attestor/architecture.md` | | Attestor | `src/Attestor/StellaOps.Attestor`<br>`src/Attestor/StellaOps.Attestor.Verify` | `docs/modules/attestor/architecture.md` |
| Concelier | `src/Concelier/StellaOps.Concelier.WebService`<br>`src/Concelier/__Libraries/StellaOps.Concelier.*` | `docs/modules/concelier/architecture.md` | | Concelier | `src/Concelier/StellaOps.Concelier.WebService`<br>`src/Concelier/__Libraries/StellaOps.Concelier.*` | `docs/modules/concelier/architecture.md` |
| Excititor | `src/Excititor/StellaOps.Excititor.WebService`<br>`src/Excititor/__Libraries/StellaOps.Excititor.*` | `docs/modules/excititor/architecture.md` | | Excititor | `src/Excititor/StellaOps.Excititor.WebService`<br>`src/Excititor/__Libraries/StellaOps.Excititor.*` | `docs/modules/excititor/architecture.md` |
| Policy Engine | `src/Policy/StellaOps.Policy.Engine`<br>`src/Policy/__Libraries/StellaOps.Policy.*` | `docs/modules/policy/architecture.md` | | Policy Engine | `src/Policy/StellaOps.Policy.Engine`<br>`src/Policy/__Libraries/StellaOps.Policy.*` | `docs/modules/policy/architecture.md` |
| Scanner | `src/Scanner/StellaOps.Scanner.WebService`<br>`src/Scanner/StellaOps.Scanner.Worker`<br>`src/Scanner/__Libraries/StellaOps.Scanner.*` | `docs/modules/scanner/architecture.md` | | Scanner | `src/Scanner/StellaOps.Scanner.WebService`<br>`src/Scanner/StellaOps.Scanner.Worker`<br>`src/Scanner/__Libraries/StellaOps.Scanner.*` | `docs/modules/scanner/architecture.md` |
| Scheduler | `src/Scheduler/StellaOps.Scheduler.WebService`<br>`src/Scheduler/StellaOps.Scheduler.Worker` | `docs/modules/scheduler/architecture.md` | | Scheduler | `src/Scheduler/StellaOps.Scheduler.WebService`<br>`src/Scheduler/StellaOps.Scheduler.Worker` | `docs/modules/scheduler/architecture.md` |
| CLI | `src/Cli/StellaOps.Cli`<br>`src/Cli/StellaOps.Cli.Core`<br>`src/Cli/StellaOps.Cli.Plugins.*` | `docs/modules/cli/architecture.md` | | CLI | `src/Cli/StellaOps.Cli`<br>`src/Cli/StellaOps.Cli.Core`<br>`src/Cli/StellaOps.Cli.Plugins.*` | `docs/modules/cli/architecture.md` |
| UI / Console | `src/UI/StellaOps.UI` | `docs/modules/ui/architecture.md` | | UI / Console | `src/UI/StellaOps.UI` | `docs/modules/ui/architecture.md` |
| Notify | `src/Notify/StellaOps.Notify.WebService`<br>`src/Notify/StellaOps.Notify.Worker` | `docs/modules/notify/architecture.md` | | Notify | `src/Notify/StellaOps.Notify.WebService`<br>`src/Notify/StellaOps.Notify.Worker` | `docs/modules/notify/architecture.md` |
| Export Center | `src/ExportCenter/StellaOps.ExportCenter.WebService`<br>`src/ExportCenter/StellaOps.ExportCenter.Worker` | `docs/modules/export-center/architecture.md` | | Export Center | `src/ExportCenter/StellaOps.ExportCenter.WebService`<br>`src/ExportCenter/StellaOps.ExportCenter.Worker` | `docs/modules/export-center/architecture.md` |
| Registry Token Service | `src/Registry/StellaOps.Registry.TokenService`<br>`src/Registry/__Tests/StellaOps.Registry.TokenService.Tests` | `docs/modules/registry/architecture.md` | | Registry Token Service | `src/Registry/StellaOps.Registry.TokenService`<br>`src/Registry/__Tests/StellaOps.Registry.TokenService.Tests` | `docs/modules/registry/architecture.md` |
| Advisory AI | `src/AdvisoryAI/StellaOps.AdvisoryAI` | `docs/modules/advisory-ai/architecture.md` | | Advisory AI | `src/AdvisoryAI/StellaOps.AdvisoryAI` | `docs/modules/advisory-ai/architecture.md` |
| Orchestrator | `src/Orchestrator/StellaOps.Orchestrator` | `docs/modules/orchestrator/architecture.md` | | Orchestrator | `src/Orchestrator/StellaOps.Orchestrator` | `docs/modules/orchestrator/architecture.md` |
| Vulnerability Explorer | `src/VulnExplorer/StellaOps.VulnExplorer.Api` | `docs/modules/vuln-explorer/architecture.md` | | Vulnerability Explorer | `src/VulnExplorer/StellaOps.VulnExplorer.Api` | `docs/modules/vuln-explorer/architecture.md` |
| VEX Lens | `src/VexLens/StellaOps.VexLens` | `docs/modules/vex-lens/architecture.md` | | VEX Lens | `src/VexLens/StellaOps.VexLens` | `docs/modules/vex-lens/architecture.md` |
| Graph Explorer | `src/Graph/StellaOps.Graph.Api`<br>`src/Graph/StellaOps.Graph.Indexer` | `docs/modules/graph/architecture.md` | | Graph Explorer | `src/Graph/StellaOps.Graph.Api`<br>`src/Graph/StellaOps.Graph.Indexer` | `docs/modules/graph/architecture.md` |
| Telemetry Stack | `ops/devops/telemetry` | `docs/modules/telemetry/architecture.md` | | Telemetry Stack | `ops/devops/telemetry` | `docs/modules/telemetry/architecture.md` |
| DevOps / Release | `ops/devops` | `docs/modules/devops/architecture.md` | | DevOps / Release | `ops/devops` | `docs/modules/devops/architecture.md` |
| Platform | *(cross-cutting docs)* | `docs/modules/platform/architecture-overview.md` | | Platform | *(cross-cutting docs)* | `docs/modules/platform/architecture-overview.md` |
| CI Recipes | *(pipeline templates)* | `docs/modules/ci/architecture.md` | | CI Recipes | *(pipeline templates)* | `docs/modules/ci/architecture.md` |
| Zastava | `src/Zastava/StellaOps.Zastava.Observer`<br>`src/Zastava/StellaOps.Zastava.Webhook`<br>`src/Zastava/StellaOps.Zastava.Core` | `docs/modules/zastava/architecture.md` | | Zastava | `src/Zastava/StellaOps.Zastava.Observer`<br>`src/Zastava/StellaOps.Zastava.Webhook`<br>`src/Zastava/StellaOps.Zastava.Core` | `docs/modules/zastava/architecture.md` |
## 4.1 Module cheat sheet ## 4.1 Module cheat sheet
### Authority ### Authority
- **Path:** `src/Authority/StellaOps.Authority`, plugins in `src/Authority/StellaOps.Authority.Plugin.*`. - **Path:** `src/Authority/StellaOps.Authority`, plugins in `src/Authority/StellaOps.Authority.Plugin.*`.
- **Docs:** `docs/modules/authority/architecture.md`. - **Docs:** `docs/modules/authority/architecture.md`.
- **Responsibilities:** Issues short-lived, sender-constrained OpToks (DPoP/mTLS) for services, CLI, and UI; exposes OIDC discovery, device-code, and auth-code flows. - **Responsibilities:** Issues short-lived, sender-constrained OpToks (DPoP/mTLS) for services, CLI, and UI; exposes OIDC discovery, device-code, and auth-code flows.
- **Key traits:** Ed25519/ES256 signing with JWKS rotation, tenant-aware scopes, stateless JWT validation, optional introspection, and structured audit trails. - **Key traits:** Ed25519/ES256 signing with JWKS rotation, tenant-aware scopes, stateless JWT validation, optional introspection, and structured audit trails.
### Signer ### Signer
- **Path:** `src/Signer/StellaOps.Signer`. - **Path:** `src/Signer/StellaOps.Signer`.
- **Docs:** `docs/modules/signer/architecture.md`. - **Docs:** `docs/modules/signer/architecture.md`.
- **Responsibilities:** Authenticates callers, enforces Proof-of-Entitlement, verifies scanner release signatures, and returns DSSE bundles for SBOMs and reports. - **Responsibilities:** Authenticates callers, enforces Proof-of-Entitlement, verifies scanner release signatures, and returns DSSE bundles for SBOMs and reports.
- **Key traits:** Supports keyless (Fulcio) and keyful (KMS/HSM) signing, applies plan quotas, stores audit trails, and delegates Rekor logging to the Attestor. - **Key traits:** Supports keyless (Fulcio) and keyful (KMS/HSM) signing, applies plan quotas, stores audit trails, and delegates Rekor logging to the Attestor.
### Attestor ### Attestor
- **Path:** `src/Attestor/StellaOps.Attestor`, proof helpers in `src/Attestor/StellaOps.Attestor.Verify`. - **Path:** `src/Attestor/StellaOps.Attestor`, proof helpers in `src/Attestor/StellaOps.Attestor.Verify`.
- **Docs:** `docs/modules/attestor/architecture.md`. - **Docs:** `docs/modules/attestor/architecture.md`.
- **Responsibilities:** Submits DSSE bundles to Rekor v2, caches `{uuid, index, proof}`, and serves verification bundles to Scanner, UI, CLI, and Export Center. - **Responsibilities:** Submits DSSE bundles to Rekor v2, caches `{uuid, index, proof}`, and serves verification bundles to Scanner, UI, CLI, and Export Center.
- **Key traits:** mTLS + OpTok enforcement for Signer-only submissions, Mongo/Redis idempotency, optional DSSE archive mirroring, and resilient retry/backoff. - **Key traits:** mTLS + OpTok enforcement for Signer-only submissions, Mongo/Redis idempotency, optional DSSE archive mirroring, and resilient retry/backoff.
### Concelier ### Concelier
- **Path:** `src/Concelier/StellaOps.Concelier.WebService` with connectors/exporters under `src/Concelier/__Libraries/StellaOps.Concelier.*`. - **Path:** `src/Concelier/StellaOps.Concelier.WebService` with connectors/exporters under `src/Concelier/__Libraries/StellaOps.Concelier.*`.
- **Docs:** `docs/modules/concelier/architecture.md`. - **Docs:** `docs/modules/concelier/architecture.md`.
- **Responsibilities:** Applies the Aggregation-Only Contract to ingest advisories, produce immutable observations, correlate linksets, and publish deterministic exports. - **Responsibilities:** Applies the Aggregation-Only Contract to ingest advisories, produce immutable observations, correlate linksets, and publish deterministic exports.
- **Key traits:** Restart-time connectors/exporters, Mongo-backed scheduling, canonical JSON/Trivy outputs, Offline Kit parity, and hash-stable manifests. - **Key traits:** Restart-time connectors/exporters, Mongo-backed scheduling, canonical JSON/Trivy outputs, Offline Kit parity, and hash-stable manifests.
### Excititor ### Excititor
- **Path:** `src/Excititor/StellaOps.Excititor.WebService`, connectors/adapters in `src/Excititor/__Libraries/StellaOps.Excititor.*`. - **Path:** `src/Excititor/StellaOps.Excititor.WebService`, connectors/adapters in `src/Excititor/__Libraries/StellaOps.Excititor.*`.
- **Docs:** `docs/modules/excititor/architecture.md`. - **Docs:** `docs/modules/excititor/architecture.md`.
- **Responsibilities:** Normalises VEX statements into observations, builds provenance-rich linksets, and surfaces consensus/conflicts for policy suppression. - **Responsibilities:** Normalises VEX statements into observations, builds provenance-rich linksets, and surfaces consensus/conflicts for policy suppression.
- **Key traits:** Aggregation-only guardrails, restart-time plug-ins, Mongo persistence, deterministic exports, and Offline Kit-ready bundles. - **Key traits:** Aggregation-only guardrails, restart-time plug-ins, Mongo persistence, deterministic exports, and Offline Kit-ready bundles.
### Policy Engine ### Policy Engine
- **Path:** `src/Policy/StellaOps.Policy.Engine`, shared libraries under `src/Policy/__Libraries/StellaOps.Policy.*`. - **Path:** `src/Policy/StellaOps.Policy.Engine`, shared libraries under `src/Policy/__Libraries/StellaOps.Policy.*`.
- **Docs:** `docs/modules/policy/architecture.md`. - **Docs:** `docs/modules/policy/architecture.md`.
- **Responsibilities:** Evaluates `stella-dsl@1` policies, joins SBOM/advisory/VEX evidence, materialises effective findings, and emits explain traces. - **Responsibilities:** Evaluates `stella-dsl@1` policies, joins SBOM/advisory/VEX evidence, materialises effective findings, and emits explain traces.
- **Key traits:** Deterministic evaluation (no wall clock), change-stream driven increments, simulation endpoints, and Authority-scoped tenancy/RBAC enforcement. - **Key traits:** Deterministic evaluation (no wall clock), change-stream driven increments, simulation endpoints, and Authority-scoped tenancy/RBAC enforcement.
### Scanner.WebService ### Scanner.WebService
- **Path:** `src/Scanner/StellaOps.Scanner.WebService`. - **Path:** `src/Scanner/StellaOps.Scanner.WebService`.
- **Docs:** `docs/modules/scanner/architecture.md`. - **Docs:** `docs/modules/scanner/architecture.md`.
- **Responsibilities:** Hosts scan/diff/export APIs, enqueues work, serves SBOM and diff artifacts, and publishes DSSE-ready report metadata. - **Responsibilities:** Hosts scan/diff/export APIs, enqueues work, serves SBOM and diff artifacts, and publishes DSSE-ready report metadata.
- **Key traits:** Minimal APIs with Redis/NATS queue clients, RustFS artifact integration, BOM-index lookups, and DSSE hand-off to Signer/Attestor. - **Key traits:** Minimal APIs with Redis/NATS queue clients, RustFS artifact integration, BOM-index lookups, and DSSE hand-off to Signer/Attestor.
### Scanner.Worker ### Scanner.Worker
- **Path:** `src/Scanner/StellaOps.Scanner.Worker` with analyzers/caches in `src/Scanner/__Libraries/StellaOps.Scanner.*`. - **Path:** `src/Scanner/StellaOps.Scanner.Worker` with analyzers/caches in `src/Scanner/__Libraries/StellaOps.Scanner.*`.
- **Docs:** `docs/modules/scanner/architecture.md`. - **Docs:** `docs/modules/scanner/architecture.md`.
- **Responsibilities:** Runs deterministic OS/language/native analyzers per layer, composes inventory and usage SBOM fragments, and streams them back to the catalog. - **Responsibilities:** Runs deterministic OS/language/native analyzers per layer, composes inventory and usage SBOM fragments, and streams them back to the catalog.
- **Key traits:** Layer/file CAS caching, restart-time analyzer plug-ins under `plugins/scanner/**`, bounded retries with lease renewals, and DSSE-ready outputs. - **Key traits:** Layer/file CAS caching, restart-time analyzer plug-ins under `plugins/scanner/**`, bounded retries with lease renewals, and DSSE-ready outputs.
### Scheduler ### Scheduler
- **Path:** `src/Scheduler/StellaOps.Scheduler.WebService`, `src/Scheduler/StellaOps.Scheduler.Worker`. - **Path:** `src/Scheduler/StellaOps.Scheduler.WebService`, `src/Scheduler/StellaOps.Scheduler.Worker`.
- **Docs:** `docs/modules/scheduler/architecture.md`. - **Docs:** `docs/modules/scheduler/architecture.md`.
- **Responsibilities:** Detects advisory/VEX deltas, selects impacted assets via BOM index, and schedules analysis-only runs toward Scanner and Policy Engine. - **Responsibilities:** Detects advisory/VEX deltas, selects impacted assets via BOM index, and schedules analysis-only runs toward Scanner and Policy Engine.
- **Key traits:** Mongo impact cursors, Redis/NATS orchestration, webhook fan-out (Policy/Notify/Runtime), and deterministic evaluation windows. - **Key traits:** Mongo impact cursors, Redis/NATS orchestration, webhook fan-out (Policy/Notify/Runtime), and deterministic evaluation windows.
### CLI ### CLI
- **Path:** `src/Cli/StellaOps.Cli`, helpers in `src/Cli/StellaOps.Cli.Core`, plug-ins in `src/Cli/StellaOps.Cli.Plugins.*`. - **Path:** `src/Cli/StellaOps.Cli`, helpers in `src/Cli/StellaOps.Cli.Core`, plug-ins in `src/Cli/StellaOps.Cli.Plugins.*`.
- **Docs:** `docs/modules/cli/architecture.md`. - **Docs:** `docs/modules/cli/architecture.md`.
- **Responsibilities:** Provides deterministic verbs for scan/diff/export/report, Buildx SBOM orchestration, policy/VEX administration, and offline kit workflows. - **Responsibilities:** Provides deterministic verbs for scan/diff/export/report, Buildx SBOM orchestration, policy/VEX administration, and offline kit workflows.
- **Key traits:** Native AOT binaries, device-code/client-credential login with DPoP storage, golden-output tests, and restart-time plug-in manifests in `plugins/cli/**`. - **Key traits:** Native AOT binaries, device-code/client-credential login with DPoP storage, golden-output tests, and restart-time plug-in manifests in `plugins/cli/**`.
### UI ### UI
- **Path:** `src/UI/StellaOps.UI`. - **Path:** `src/UI/StellaOps.UI`.
- **Docs:** `docs/modules/ui/architecture.md`. - **Docs:** `docs/modules/ui/architecture.md`.
- **Responsibilities:** Angular SPA for scans, policy authoring, VEX evidence exploration, runtime posture, and admin tooling via backend APIs. - **Responsibilities:** Angular SPA for scans, policy authoring, VEX evidence exploration, runtime posture, and admin tooling via backend APIs.
- **Key traits:** Angular Signals with `@ngrx/signals`, typed API clients handling DPoP + SSE, Tailwind theming, and immutable content-hashed bundles. - **Key traits:** Angular Signals with `@ngrx/signals`, typed API clients handling DPoP + SSE, Tailwind theming, and immutable content-hashed bundles.
### Notify ### Notify
- **Path:** `src/Notify/StellaOps.Notify.WebService`, `src/Notify/StellaOps.Notify.Worker`, connectors in `src/Notify/__Libraries`. - **Path:** `src/Notify/StellaOps.Notify.WebService`, `src/Notify/StellaOps.Notify.Worker`, connectors in `src/Notify/__Libraries`.
- **Docs:** `docs/modules/notify/architecture.md`. - **Docs:** `docs/modules/notify/architecture.md`.
- **Responsibilities:** Evaluates notification rules on platform events, renders channel-specific payloads, and delivers messages with throttling/digests. - **Responsibilities:** Evaluates notification rules on platform events, renders channel-specific payloads, and delivers messages with throttling/digests.
- **Key traits:** Tenant-scoped rule engine, idempotent delivery queues, secrets referenced rather than stored, and comprehensive audit/metrics coverage. - **Key traits:** Tenant-scoped rule engine, idempotent delivery queues, secrets referenced rather than stored, and comprehensive audit/metrics coverage.
### Export Center ### Export Center
- **Path:** `src/ExportCenter/StellaOps.ExportCenter.WebService`, `src/ExportCenter/StellaOps.ExportCenter.Worker`, adapters in `src/ExportCenter/StellaOps.ExportCenter.*`. - **Path:** `src/ExportCenter/StellaOps.ExportCenter.WebService`, `src/ExportCenter/StellaOps.ExportCenter.Worker`, adapters in `src/ExportCenter/StellaOps.ExportCenter.*`.
- **Docs:** `docs/modules/export-center/architecture.md`. - **Docs:** `docs/modules/export-center/architecture.md`.
- **Responsibilities:** Packages reproducible evidence bundles (JSON, Trivy, mirror) with provenance, signing, and distribution manifests for offline or mirror deployments. - **Responsibilities:** Packages reproducible evidence bundles (JSON, Trivy, mirror) with provenance, signing, and distribution manifests for offline or mirror deployments.
- **Key traits:** Profile-driven exports, Orchestrator-backed job leases, Mongo/object storage staging, and cosign-compatible provenance/signature emission. - **Key traits:** Profile-driven exports, Orchestrator-backed job leases, Mongo/object storage staging, and cosign-compatible provenance/signature emission.
### Registry Token Service ### Registry Token Service
- **Path:** `src/Registry/StellaOps.Registry.TokenService`, with integration tests in `src/Registry/__Tests/StellaOps.Registry.TokenService.Tests`. - **Path:** `src/Registry/StellaOps.Registry.TokenService`, with integration tests in `src/Registry/__Tests/StellaOps.Registry.TokenService.Tests`.
- **Docs:** `docs/modules/registry/operations/token-service.md`. - **Docs:** `docs/modules/registry/operations/token-service.md`.
- **Responsibilities:** Issues scoped pull tokens for container/image registries, enforces licence/plan constraints, and publishes audit telemetry for token usage. - **Responsibilities:** Issues scoped pull tokens for container/image registries, enforces licence/plan constraints, and publishes audit telemetry for token usage.
- **Key traits:** Authority-issued OpTok validation, Mongo-backed issuance ledger, deterministic checksum manifests for Offline Kit bundles, and emergency revoke/rotation tooling. - **Key traits:** Authority-issued OpTok validation, Mongo-backed issuance ledger, deterministic checksum manifests for Offline Kit bundles, and emergency revoke/rotation tooling.
### Zastava ### Zastava
- **Path:** `src/Zastava/StellaOps.Zastava.Observer`, `src/Zastava/StellaOps.Zastava.Webhook`, shared contracts in `src/Zastava/StellaOps.Zastava.Core`. - **Path:** `src/Zastava/StellaOps.Zastava.Observer`, `src/Zastava/StellaOps.Zastava.Webhook`, shared contracts in `src/Zastava/StellaOps.Zastava.Core`.
- **Docs:** `docs/modules/zastava/architecture.md`. - **Docs:** `docs/modules/zastava/architecture.md`.
- **Responsibilities:** Observes running workloads, emits runtime posture events, and enforces admission-time policy (signed images, SBOM availability, policy verdict). - **Responsibilities:** Observes running workloads, emits runtime posture events, and enforces admission-time policy (signed images, SBOM availability, policy verdict).
- **Key traits:** Authority-issued OpToks with DPoP/mTLS, ND-JSON batching with local buffering, delta-scan triggers on drift, and Kubernetes webhook enforcement. - **Key traits:** Authority-issued OpToks with DPoP/mTLS, ND-JSON batching with local buffering, delta-scan triggers on drift, and Kubernetes webhook enforcement.
--- ---
### 4.1.4) Glossary (quick) ### 4.1.4) Glossary (quick)
- **OVAL** — Vendor/distro security definition format; authoritative for OS packages. - **OVAL** — Vendor/distro security definition format; authoritative for OS packages.
- **NEVRA / EVR** — RPM and Debian version semantics for OS packages. - **NEVRA / EVR** — RPM and Debian version semantics for OS packages.
- **PURL / SemVer** — Coordinates and version semantics for OSS ecosystems. - **PURL / SemVer** — Coordinates and version semantics for OSS ecosystems.
- **KEV** — Known Exploited Vulnerabilities (flag only). - **KEV** — Known Exploited Vulnerabilities (flag only).
--- ---
# 5) Your role as StellaOps contributor # 5) Your role as StellaOps contributor
You acting as information technology engineer that will take different type of roles in goal achieving StellaOps production implementation You acting as information technology engineer that will take different type of roles in goal achieving StellaOps production implementation
In order you to work - you have to be supplied with directory that contains `AGENTS.md`,`TASKS.md` files. There will you have more information about the role you have, the scope of your work and the tasks you will have. In order you to work - you have to be supplied with directory that contains `AGENTS.md`,`TASKS.md` files. There will you have more information about the role you have, the scope of your work and the tasks you will have.
Boundaries: Boundaries:
- You operate only in the working directories I gave you, unless there is dependencies that makes you to work on dependency in shared directory. Then you ask for confirmation. - You operate only in the working directories I gave you, unless there is dependencies that makes you to work on dependency in shared directory. Then you ask for confirmation.
You main characteristics: You main characteristics:
- Keep endpoints small, deterministic, and cancellation-aware. - Keep endpoints small, deterministic, and cancellation-aware.
- Improve logs/metrics as per tasks. - Improve logs/metrics as per tasks.
- Update `TASKS.md` when moving tasks forward. - Update `TASKS.md` when moving tasks forward.
- When you are done with all task you state explicitly you are done. - When you are done with all task you state explicitly you are done.
- Impersonate the role described on working directory `AGENTS.md` you will read, if role is not available - take role of the CTO of the StellaOps in early stages. - Impersonate the role described on working directory `AGENTS.md` you will read, if role is not available - take role of the CTO of the StellaOps in early stages.
- You always strive for best practices - You always strive for best practices
- You always strive for re-usability - You always strive for re-usability
- When in doubt of design decision - you ask then act - When in doubt of design decision - you ask then act
- You are autonomus - meaning that you will work for long time alone and achieve maximum without stopping for stupid questions - You are autonomus - meaning that you will work for long time alone and achieve maximum without stopping for stupid questions
- You operate on the same directory where other agents will work. In case you need to work on directory that is dependency on provided `AGENTS.md`,`TASKS.md` files you have to ask for confirmation first. - You operate on the same directory where other agents will work. In case you need to work on directory that is dependency on provided `AGENTS.md`,`TASKS.md` files you have to ask for confirmation first.
## 5.1) Type of contributions ## 5.1) Type of contributions
- **BE‑Base (Platform & Pipeline)** - **BE‑Base (Platform & Pipeline)**
Owns DI, plugin host, job scheduler/coordinator, configuration binding, minimal API endpoints, and Mongo bootstrapping. Owns DI, plugin host, job scheduler/coordinator, configuration binding, minimal API endpoints, and Mongo bootstrapping.
- **BE‑Conn‑X (Connectors)** - **BE‑Conn‑X (Connectors)**
One agent per source family (NVD, Red Hat, Ubuntu, Debian, SUSE, GHSA, OSV, PSIRTs, CERTs, KEV, ICS). Implements fetch/parse/map with incremental watermarks. One agent per source family (NVD, Red Hat, Ubuntu, Debian, SUSE, GHSA, OSV, PSIRTs, CERTs, KEV, ICS). Implements fetch/parse/map with incremental watermarks.
- **BE‑Merge (Canonical Merge & Dedupe)** - **BE‑Merge (Canonical Merge & Dedupe)**
Identity graph, precedence policies, canonical JSON serializer, and deterministic hashing (`merge_event`). Identity graph, precedence policies, canonical JSON serializer, and deterministic hashing (`merge_event`).
- **BE‑Export (JSON & Trivy DB)** - **BE‑Export (JSON & Trivy DB)**
Deterministic export trees, Trivy DB packaging, optional ORAS push, and offline bundle. Deterministic export trees, Trivy DB packaging, optional ORAS push, and offline bundle.
- **QA (Validation & Observability)** - **QA (Validation & Observability)**
Schema tests, fixture goldens, determinism checks, metrics/logs/traces, e2e reproducibility runs. Schema tests, fixture goldens, determinism checks, metrics/logs/traces, e2e reproducibility runs.
- **DevEx/Docs** - **DevEx/Docs**
Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews. Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews.
## 5.2) Work rules (important) ## 5.2) Work rules (important)
- **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description. - **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description.
- **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting. - **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting.
- **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests. - **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests.
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED). Before starting of actual work - ensure you have set the task to DOING. When complete or stop update the status in corresponding TASKS.md and in ./SPRINTS.md file. - **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED). Before starting of actual work - ensure you have set the task to DOING. When complete or stop update the status in corresponding TASKS.md and in ./SPRINTS.md file.
- **Coordination**: In case task is discovered as blocked on other team or task, according TASKS.md files that dependency is on needs to be changed by adding new tasks describing the requirement. the current task must be updated as completed. In case task changes, scope or requirements or rules - other documentations needs be updated accordingly. - **Coordination**: In case task is discovered as blocked on other team or task, according TASKS.md files that dependency is on needs to be changed by adding new tasks describing the requirement. the current task must be updated as completed. In case task changes, scope or requirements or rules - other documentations needs be updated accordingly.
- **Sprint synchronization**: When given task seek for relevant directory to work on from SPRINTS.md. Confirm its state on both SPRINTS.md and the relevant TASKS.md file. Always check the AGENTS.md in the relevant TASKS.md directory. - **Sprint synchronization**: When given task seek for relevant directory to work on from SPRINTS.md. Confirm its state on both SPRINTS.md and the relevant TASKS.md file. Always check the AGENTS.md in the relevant TASKS.md directory.
- **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence. - **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence.
- **Test layout**: Use module-specific projects in `StellaOps.Concelier.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Concelier.Testing`. - **Test layout**: Use module-specific projects in `StellaOps.Concelier.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Concelier.Testing`.
- **Execution autonomous**: In case you need to continue with more than one options just continue sequentially, unless the continue requires design decision. - **Execution autonomous**: In case you need to continue with more than one options just continue sequentially, unless the continue requires design decision.
- **Additional references**: When a task mentions historical epics, consult the corresponding module guides or domain playbooks under `docs/modules/**`, `docs/api/`, `docs/risk/`, or `docs/airgap/` for the latest specification. - **Additional references**: When a task mentions historical epics, consult the corresponding module guides or domain playbooks under `docs/modules/**`, `docs/api/`, `docs/risk/`, or `docs/airgap/` for the latest specification.
--- ---

66
README.md
View File

@@ -1,33 +1,33 @@
# StellaOps Concelier & CLI # StellaOps Concelier & CLI
This repository hosts the StellaOps Concelier service, its plug-in ecosystem, and the This repository hosts the StellaOps Concelier service, its plug-in ecosystem, and the
first-party CLI (`stellaops-cli`). Concelier ingests vulnerability advisories from first-party CLI (`stellaops-cli`). Concelier ingests vulnerability advisories from
authoritative sources, stores them in MongoDB, and exports deterministic JSON and authoritative sources, stores them in MongoDB, and exports deterministic JSON and
Trivy DB artefacts. The CLI drives scanner distribution, scan execution, and job Trivy DB artefacts. The CLI drives scanner distribution, scan execution, and job
control against the Concelier API. control against the Concelier API.
## Quickstart ## Quickstart
1. Prepare a MongoDB instance and (optionally) install `trivy-db`/`oras`. 1. Prepare a MongoDB instance and (optionally) install `trivy-db`/`oras`.
2. Copy `etc/concelier.yaml.sample` to `etc/concelier.yaml` and update the storage + telemetry 2. Copy `etc/concelier.yaml.sample` to `etc/concelier.yaml` and update the storage + telemetry
settings. settings.
3. Copy `etc/authority.yaml.sample` to `etc/authority.yaml`, review the issuer, token 3. Copy `etc/authority.yaml.sample` to `etc/authority.yaml`, review the issuer, token
lifetimes, and plug-in descriptors, then edit the companion manifests under lifetimes, and plug-in descriptors, then edit the companion manifests under
`etc/authority.plugins/*.yaml` to match your deployment. `etc/authority.plugins/*.yaml` to match your deployment.
4. Start the web service with `dotnet run --project src/Concelier/StellaOps.Concelier.WebService`. 4. Start the web service with `dotnet run --project src/Concelier/StellaOps.Concelier.WebService`.
5. Configure the CLI via environment variables (e.g. `STELLAOPS_BACKEND_URL`) and trigger 5. Configure the CLI via environment variables (e.g. `STELLAOPS_BACKEND_URL`) and trigger
jobs with `dotnet run --project src/Cli/StellaOps.Cli -- db merge`. jobs with `dotnet run --project src/Cli/StellaOps.Cli -- db merge`.
Detailed operator guidance is available in `docs/10_CONCELIER_CLI_QUICKSTART.md`. API and Detailed operator guidance is available in `docs/10_CONCELIER_CLI_QUICKSTART.md`. API and
command reference material lives in `docs/09_API_CLI_REFERENCE.md`. command reference material lives in `docs/09_API_CLI_REFERENCE.md`.
Pipeline note: deployment workflows should template `etc/concelier.yaml` during CI/CD, Pipeline note: deployment workflows should template `etc/concelier.yaml` during CI/CD,
injecting environment-specific Mongo credentials and telemetry endpoints. Upcoming injecting environment-specific Mongo credentials and telemetry endpoints. Upcoming
releases will add Microsoft OAuth (Entra ID) authentication support—track the quickstart releases will add Microsoft OAuth (Entra ID) authentication support—track the quickstart
for integration steps once available. for integration steps once available.
## Documentation ## Documentation
- `docs/README.md` now consolidates the platform index and points to the updated high-level architecture. - `docs/README.md` now consolidates the platform index and points to the updated high-level architecture.
- Module architecture dossiers now live under `docs/modules/<module>/`. The most relevant here are `docs/modules/concelier/ARCHITECTURE.md` (service layout, merge engine, exports) and `docs/modules/cli/ARCHITECTURE.md` (command surface, AOT packaging, auth flows). Related services such as the Signer, Attestor, Authority, Scanner, UI, Excititor, Zastava, and DevOps pipeline each have their own dossier in the same hierarchy. - Module architecture dossiers now live under `docs/modules/<module>/`. The most relevant here are `docs/modules/concelier/ARCHITECTURE.md` (service layout, merge engine, exports) and `docs/modules/cli/ARCHITECTURE.md` (command surface, AOT packaging, auth flows). Related services such as the Signer, Attestor, Authority, Scanner, UI, Excititor, Zastava, and DevOps pipeline each have their own dossier in the same hierarchy.
- Offline operation guidance moved to `docs/24_OFFLINE_KIT.md`, which details bundle composition, verification, and delta workflows. Concelier-specific connector operations stay in `docs/modules/concelier/operations/connectors/*.md` with companion runbooks in `docs/modules/concelier/operations/`. - Offline operation guidance moved to `docs/24_OFFLINE_KIT.md`, which details bundle composition, verification, and delta workflows. Concelier-specific connector operations stay in `docs/modules/concelier/operations/connectors/*.md` with companion runbooks in `docs/modules/concelier/operations/`.

102
deploy/README.md
View File

@@ -1,51 +1,51 @@
# Deployment Profiles # Deployment Profiles
This directory contains deterministic deployment bundles for the core Stella Ops stack. All manifests reference immutable image digests and map 1:1 to the release manifests stored under `deploy/releases/`. This directory contains deterministic deployment bundles for the core Stella Ops stack. All manifests reference immutable image digests and map 1:1 to the release manifests stored under `deploy/releases/`.
## Structure ## Structure
- `releases/` – canonical release manifests (edge, stable, airgap) used to source image digests. - `releases/` – canonical release manifests (edge, stable, airgap) used to source image digests.
- `compose/` – Docker Compose bundles for dev/stage/airgap targets plus `.env` seed files. - `compose/` – Docker Compose bundles for dev/stage/airgap targets plus `.env` seed files.
- `compose/docker-compose.mirror.yaml` – managed mirror bundle for `*.stella-ops.org` with gateway cache and multi-tenant auth. - `compose/docker-compose.mirror.yaml` – managed mirror bundle for `*.stella-ops.org` with gateway cache and multi-tenant auth.
- `compose/docker-compose.telemetry.yaml` – optional OpenTelemetry collector overlay (mutual TLS, OTLP pipelines). - `compose/docker-compose.telemetry.yaml` – optional OpenTelemetry collector overlay (mutual TLS, OTLP pipelines).
- `compose/docker-compose.telemetry-storage.yaml` – optional Prometheus/Tempo/Loki stack for observability backends. - `compose/docker-compose.telemetry-storage.yaml` – optional Prometheus/Tempo/Loki stack for observability backends.
- `helm/stellaops/` – multi-profile Helm chart with values files for dev/stage/airgap. - `helm/stellaops/` – multi-profile Helm chart with values files for dev/stage/airgap.
- `telemetry/` – shared OpenTelemetry collector configuration and certificate artefacts (generated via tooling). - `telemetry/` – shared OpenTelemetry collector configuration and certificate artefacts (generated via tooling).
- `tools/validate-profiles.sh` – helper that runs `docker compose config` and `helm lint/template` for every profile. - `tools/validate-profiles.sh` – helper that runs `docker compose config` and `helm lint/template` for every profile.
## Workflow ## Workflow
1. Update or add a release manifest under `releases/` with the new digests. 1. Update or add a release manifest under `releases/` with the new digests.
2. Mirror the digests into the Compose and Helm profiles that correspond to that channel. 2. Mirror the digests into the Compose and Helm profiles that correspond to that channel.
3. Run `deploy/tools/validate-profiles.sh` (requires Docker CLI and Helm) to ensure the bundles lint and template cleanly. 3. Run `deploy/tools/validate-profiles.sh` (requires Docker CLI and Helm) to ensure the bundles lint and template cleanly.
4. If telemetry ingest is required for the release, generate development certificates using 4. If telemetry ingest is required for the release, generate development certificates using
`./ops/devops/telemetry/generate_dev_tls.sh` and run the collector smoke test with `./ops/devops/telemetry/generate_dev_tls.sh` and run the collector smoke test with
`python ./ops/devops/telemetry/smoke_otel_collector.py` to verify the OTLP endpoints. `python ./ops/devops/telemetry/smoke_otel_collector.py` to verify the OTLP endpoints.
5. Commit the change alongside any documentation updates (e.g. install guide cross-links). 5. Commit the change alongside any documentation updates (e.g. install guide cross-links).
Maintaining the digest linkage keeps offline/air-gapped installs reproducible and avoids tag drift between environments. Maintaining the digest linkage keeps offline/air-gapped installs reproducible and avoids tag drift between environments.
### Additional tooling ### Additional tooling
- `deploy/tools/check-channel-alignment.py` – verifies that Helm/Compose profiles reference the exact images listed in a release manifest. Run it for each channel before promoting a release. - `deploy/tools/check-channel-alignment.py` – verifies that Helm/Compose profiles reference the exact images listed in a release manifest. Run it for each channel before promoting a release.
- `ops/devops/telemetry/generate_dev_tls.sh` – produces local CA/server/client certificates for Compose-based collector testing. - `ops/devops/telemetry/generate_dev_tls.sh` – produces local CA/server/client certificates for Compose-based collector testing.
- `ops/devops/telemetry/smoke_otel_collector.py` – sends OTLP traffic and asserts the collector accepted traces, metrics, and logs. - `ops/devops/telemetry/smoke_otel_collector.py` – sends OTLP traffic and asserts the collector accepted traces, metrics, and logs.
- `ops/devops/telemetry/package_offline_bundle.py` – packages telemetry assets (config/Helm/Compose) into a signed tarball for air-gapped installs. - `ops/devops/telemetry/package_offline_bundle.py` – packages telemetry assets (config/Helm/Compose) into a signed tarball for air-gapped installs.
- `docs/modules/devops/runbooks/deployment-upgrade.md` – end-to-end instructions for upgrade, rollback, and channel promotion workflows (Helm + Compose). - `docs/modules/devops/runbooks/deployment-upgrade.md` – end-to-end instructions for upgrade, rollback, and channel promotion workflows (Helm + Compose).
## CI smoke checks ## CI smoke checks
The `.gitea/workflows/build-test-deploy.yml` pipeline includes a `notify-smoke` stage that validates scanner event propagation after staging deployments. Configure the following repository secrets (or environment-level secrets) so the job can connect to Redis and the Notify API: The `.gitea/workflows/build-test-deploy.yml` pipeline includes a `notify-smoke` stage that validates scanner event propagation after staging deployments. Configure the following repository secrets (or environment-level secrets) so the job can connect to Redis and the Notify API:
- `NOTIFY_SMOKE_REDIS_DSN` – Redis connection string (`redis://user:pass@host:port/db`). - `NOTIFY_SMOKE_REDIS_DSN` – Redis connection string (`redis://user:pass@host:port/db`).
- `NOTIFY_SMOKE_NOTIFY_BASEURL` – Base URL for the staging Notify WebService (e.g. `https://notify.stage.stella-ops.internal`). - `NOTIFY_SMOKE_NOTIFY_BASEURL` – Base URL for the staging Notify WebService (e.g. `https://notify.stage.stella-ops.internal`).
- `NOTIFY_SMOKE_NOTIFY_TOKEN` – OAuth bearer token (service account) with permission to read deliveries. - `NOTIFY_SMOKE_NOTIFY_TOKEN` – OAuth bearer token (service account) with permission to read deliveries.
- `NOTIFY_SMOKE_NOTIFY_TENANT` – Tenant identifier used for the smoke validation requests. - `NOTIFY_SMOKE_NOTIFY_TENANT` – Tenant identifier used for the smoke validation requests.
- *(Optional)* `NOTIFY_SMOKE_NOTIFY_TENANT_HEADER` – Override for the tenant header name (defaults to `X-StellaOps-Tenant`). - *(Optional)* `NOTIFY_SMOKE_NOTIFY_TENANT_HEADER` – Override for the tenant header name (defaults to `X-StellaOps-Tenant`).
Define the following repository variables (or secrets) to drive the assertions performed by the smoke check: Define the following repository variables (or secrets) to drive the assertions performed by the smoke check:
- `NOTIFY_SMOKE_EXPECT_KINDS` – Comma-separated event kinds the checker must observe (for example `scanner.report.ready,scanner.scan.completed`). - `NOTIFY_SMOKE_EXPECT_KINDS` – Comma-separated event kinds the checker must observe (for example `scanner.report.ready,scanner.scan.completed`).
- `NOTIFY_SMOKE_LOOKBACK_MINUTES` – Time window (in minutes) used when scanning the Redis stream for recent events (for example `30`). - `NOTIFY_SMOKE_LOOKBACK_MINUTES` – Time window (in minutes) used when scanning the Redis stream for recent events (for example `30`).
All of the above values are required—the workflow fails fast with a descriptive error if any are missing or empty. Provide the variables at the organisation or repository scope before enabling the smoke stage. All of the above values are required—the workflow fails fast with a descriptive error if any are missing or empty. Provide the variables at the organisation or repository scope before enabling the smoke stage.

36
deploy/telemetry/storage/README.md
View File

@@ -1,19 +1,19 @@
# Telemetry Storage Stack # Telemetry Storage Stack
Configuration snippets for the default StellaOps observability backends used in Configuration snippets for the default StellaOps observability backends used in
staging and production environments. The stack comprises: staging and production environments. The stack comprises:
- **Prometheus** for metrics (scraping the collector's Prometheus exporter) - **Prometheus** for metrics (scraping the collector's Prometheus exporter)
- **Tempo** for traces (OTLP ingest via mTLS) - **Tempo** for traces (OTLP ingest via mTLS)
- **Loki** for logs (HTTP ingest with tenant isolation) - **Loki** for logs (HTTP ingest with tenant isolation)
## Files ## Files
| Path | Description | | Path | Description |
| ---- | ----------- | | ---- | ----------- |
| `prometheus.yaml` | Scrape configuration for the collector (mTLS + bearer token placeholder). | | `prometheus.yaml` | Scrape configuration for the collector (mTLS + bearer token placeholder). |
| `tempo.yaml` | Tempo configuration with multitenancy enabled and local storage paths. | | `tempo.yaml` | Tempo configuration with multitenancy enabled and local storage paths. |
| `loki.yaml` | Loki configuration enabling per-tenant overrides and boltdb-shipper storage. | | `loki.yaml` | Loki configuration enabling per-tenant overrides and boltdb-shipper storage. |
| `tenants/tempo-overrides.yaml` | Example tenant overrides for Tempo (retention, limits). | | `tenants/tempo-overrides.yaml` | Example tenant overrides for Tempo (retention, limits). |
| `tenants/loki-overrides.yaml` | Example tenant overrides for Loki (rate limits, retention). | | `tenants/loki-overrides.yaml` | Example tenant overrides for Loki (rate limits, retention). |
| `auth/` | Placeholder directory for Prometheus bearer token files (e.g., `token`). | | `auth/` | Placeholder directory for Prometheus bearer token files (e.g., `token`). |
@@ -32,5 +32,5 @@ ensure TLS, multitenancy, and override references remain intact.
- Both Tempo and Loki require mutual TLS. - Both Tempo and Loki require mutual TLS.
- Prometheus uses mTLS plus a bearer token that should be minted by Authority. - Prometheus uses mTLS plus a bearer token that should be minted by Authority.
- Update the overrides files to enforce per-tenant retention/ingestion limits. - Update the overrides files to enforce per-tenant retention/ingestion limits.
For comprehensive deployment steps see `docs/modules/telemetry/operations/storage.md`. For comprehensive deployment steps see `docs/modules/telemetry/operations/storage.md`.

708
docs/10_CONCELIER_CLI_QUICKSTART.md
View File

@@ -1,354 +1,354 @@
# 10 · Concelier + CLI Quickstart # 10 · Concelier + CLI Quickstart
This guide walks through configuring the Concelier web service and the `stellaops-cli` This guide walks through configuring the Concelier web service and the `stellaops-cli`
tool so an operator can ingest advisories, merge them, and publish exports from a tool so an operator can ingest advisories, merge them, and publish exports from a
single workstation. It focuses on deployment-facing surfaces only (configuration, single workstation. It focuses on deployment-facing surfaces only (configuration,
runtime wiring, CLI usage) and leaves connector/internal customization for later. runtime wiring, CLI usage) and leaves connector/internal customization for later.
--- ---
## 0 · Prerequisites ## 0 · Prerequisites
- .NET SDK **10.0.100-preview** (matches `global.json`) - .NET SDK **10.0.100-preview** (matches `global.json`)
- MongoDB instance reachable from the host (local Docker or managed) - MongoDB instance reachable from the host (local Docker or managed)
- `trivy-db` binary on `PATH` for Trivy exports (and `oras` if publishing to OCI) - `trivy-db` binary on `PATH` for Trivy exports (and `oras` if publishing to OCI)
- Plugin assemblies present in `StellaOps.Concelier.PluginBinaries/` (already included in the repo) - Plugin assemblies present in `StellaOps.Concelier.PluginBinaries/` (already included in the repo)
- Optional: Docker/Podman runtime if you plan to run scanners locally - Optional: Docker/Podman runtime if you plan to run scanners locally
> **Tip** – air-gapped installs should preload `trivy-db` and `oras` binaries into the > **Tip** – air-gapped installs should preload `trivy-db` and `oras` binaries into the
> runner image since Concelier never fetches them dynamically. > runner image since Concelier never fetches them dynamically.
--- ---
## 1 · Configure Concelier ## 1 · Configure Concelier
1. Copy the sample config to the expected location (CI/CD pipelines can stamp values 1. Copy the sample config to the expected location (CI/CD pipelines can stamp values
into this file during deployment—see the “Deployment automation” note below): into this file during deployment—see the “Deployment automation” note below):
```bash ```bash
mkdir -p etc mkdir -p etc
cp etc/concelier.yaml.sample etc/concelier.yaml cp etc/concelier.yaml.sample etc/concelier.yaml
``` ```
2. Edit `etc/concelier.yaml` and update the MongoDB DSN (and optional database name). 2. Edit `etc/concelier.yaml` and update the MongoDB DSN (and optional database name).
The default template configures plug-in discovery to look in `StellaOps.Concelier.PluginBinaries/` The default template configures plug-in discovery to look in `StellaOps.Concelier.PluginBinaries/`
and disables remote telemetry exporters by default. and disables remote telemetry exporters by default.
3. (Optional) Override settings via environment variables. All keys are prefixed with 3. (Optional) Override settings via environment variables. All keys are prefixed with
`CONCELIER_`. Example: `CONCELIER_`. Example:
```bash ```bash
export CONCELIER_STORAGE__DSN="mongodb://user:pass@mongo:27017/concelier" export CONCELIER_STORAGE__DSN="mongodb://user:pass@mongo:27017/concelier"
export CONCELIER_TELEMETRY__ENABLETRACING=false export CONCELIER_TELEMETRY__ENABLETRACING=false
``` ```
4. Start the web service from the repository root: 4. Start the web service from the repository root:
```bash ```bash
dotnet run --project src/Concelier/StellaOps.Concelier.WebService dotnet run --project src/Concelier/StellaOps.Concelier.WebService
``` ```
On startup Concelier validates the options, boots MongoDB indexes, loads plug-ins, On startup Concelier validates the options, boots MongoDB indexes, loads plug-ins,
and exposes: and exposes:
- `GET /health` – returns service status and telemetry settings - `GET /health` – returns service status and telemetry settings
- `GET /ready` – performs a MongoDB `ping` - `GET /ready` – performs a MongoDB `ping`
- `GET /jobs` + `POST /jobs/{kind}` – inspect and trigger connector/export jobs - `GET /jobs` + `POST /jobs/{kind}` – inspect and trigger connector/export jobs
> **Security note** – authentication now ships via StellaOps Authority. Keep > **Security note** – authentication now ships via StellaOps Authority. Keep
> `authority.allowAnonymousFallback: true` only during the staged rollout and > `authority.allowAnonymousFallback: true` only during the staged rollout and
> disable it before **2025-12-31 UTC** so tokens become mandatory. > disable it before **2025-12-31 UTC** so tokens become mandatory.
Rollout checkpoints for the two Authority toggles: Rollout checkpoints for the two Authority toggles:
| Phase | `authority.enabled` | `authority.allowAnonymousFallback` | Goal | Observability focus | | Phase | `authority.enabled` | `authority.allowAnonymousFallback` | Goal | Observability focus |
| ----- | ------------------- | ---------------------------------- | ---- | ------------------- | | ----- | ------------------- | ---------------------------------- | ---- | ------------------- |
| **Validation (staging)** | `true` | `true` | Verify token issuance, CLI scopes, and audit log noise without breaking cron jobs. | Watch `Concelier.Authorization.Audit` for `bypass=True` events and scope gaps; confirm CLI `auth status` succeeds. | | **Validation (staging)** | `true` | `true` | Verify token issuance, CLI scopes, and audit log noise without breaking cron jobs. | Watch `Concelier.Authorization.Audit` for `bypass=True` events and scope gaps; confirm CLI `auth status` succeeds. |
| **Cutover rehearsal** | `true` | `false` | Exercise production-style enforcement before the deadline; ensure only approved maintenance ranges remain in `bypassNetworks`. | Expect some HTTP 401s; verify `web.jobs.triggered` metrics flatten for unauthenticated calls and audit logs highlight missing tokens. | | **Cutover rehearsal** | `true` | `false` | Exercise production-style enforcement before the deadline; ensure only approved maintenance ranges remain in `bypassNetworks`. | Expect some HTTP 401s; verify `web.jobs.triggered` metrics flatten for unauthenticated calls and audit logs highlight missing tokens. |
| **Enforced (steady state)** | `true` | `false` | Production baseline after the 2025-12-31 UTC cutoff. | Alert on new `bypass=True` entries and on repeated 401 bursts; correlate with Authority availability dashboards. | | **Enforced (steady state)** | `true` | `false` | Production baseline after the 2025-12-31 UTC cutoff. | Alert on new `bypass=True` entries and on repeated 401 bursts; correlate with Authority availability dashboards. |
### Authority companion configuration (preview) ### Authority companion configuration (preview)
1. Copy the Authority sample configuration: 1. Copy the Authority sample configuration:
```bash ```bash
cp etc/authority.yaml.sample etc/authority.yaml cp etc/authority.yaml.sample etc/authority.yaml
``` ```
2. Update the issuer URL, token lifetimes, and plug-in descriptors to match your 2. Update the issuer URL, token lifetimes, and plug-in descriptors to match your
environment. Authority expects per-plugin manifests in `etc/authority.plugins/`; environment. Authority expects per-plugin manifests in `etc/authority.plugins/`;
sample `standard.yaml` and `ldap.yaml` files are provided as starting points. sample `standard.yaml` and `ldap.yaml` files are provided as starting points.
For air-gapped installs keep the default plug-in binary directory For air-gapped installs keep the default plug-in binary directory
(`../StellaOps.Authority.PluginBinaries`) so packaged plug-ins load without outbound access. (`../StellaOps.Authority.PluginBinaries`) so packaged plug-ins load without outbound access.
3. Environment variables prefixed with `STELLAOPS_AUTHORITY_` override individual 3. Environment variables prefixed with `STELLAOPS_AUTHORITY_` override individual
fields. Example: fields. Example:
```bash ```bash
export STELLAOPS_AUTHORITY__ISSUER="https://authority.stella-ops.local" export STELLAOPS_AUTHORITY__ISSUER="https://authority.stella-ops.local"
export STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0="/srv/authority/plugins" export STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0="/srv/authority/plugins"
``` ```
--- ---
## 2 · Configure the CLI ## 2 · Configure the CLI
The CLI reads configuration from JSON/YAML files *and* environment variables. The The CLI reads configuration from JSON/YAML files *and* environment variables. The
defaults live in `src/Cli/StellaOps.Cli/appsettings.json` and expect overrides at runtime. defaults live in `src/Cli/StellaOps.Cli/appsettings.json` and expect overrides at runtime.
| Setting | Environment variable | Default | Purpose | | Setting | Environment variable | Default | Purpose |
| ------- | -------------------- | ------- | ------- | | ------- | -------------------- | ------- | ------- |
| `BackendUrl` | `STELLAOPS_BACKEND_URL` | _empty_ | Base URL of the Concelier web service | | `BackendUrl` | `STELLAOPS_BACKEND_URL` | _empty_ | Base URL of the Concelier web service |
| `ApiKey` | `API_KEY` | _empty_ | Reserved for legacy key auth; leave empty when using Authority | | `ApiKey` | `API_KEY` | _empty_ | Reserved for legacy key auth; leave empty when using Authority |
| `ScannerCacheDirectory` | `STELLAOPS_SCANNER_CACHE_DIRECTORY` | `scanners` | Local cache folder | | `ScannerCacheDirectory` | `STELLAOPS_SCANNER_CACHE_DIRECTORY` | `scanners` | Local cache folder |
| `ResultsDirectory` | `STELLAOPS_RESULTS_DIRECTORY` | `results` | Where scan outputs are written | | `ResultsDirectory` | `STELLAOPS_RESULTS_DIRECTORY` | `results` | Where scan outputs are written |
| `Authority.Url` | `STELLAOPS_AUTHORITY_URL` | _empty_ | StellaOps Authority issuer/token endpoint | | `Authority.Url` | `STELLAOPS_AUTHORITY_URL` | _empty_ | StellaOps Authority issuer/token endpoint |
| `Authority.ClientId` | `STELLAOPS_AUTHORITY_CLIENT_ID` | _empty_ | Client identifier for the CLI | | `Authority.ClientId` | `STELLAOPS_AUTHORITY_CLIENT_ID` | _empty_ | Client identifier for the CLI |
| `Authority.ClientSecret` | `STELLAOPS_AUTHORITY_CLIENT_SECRET` | _empty_ | Client secret (omit when using username/password grant) | | `Authority.ClientSecret` | `STELLAOPS_AUTHORITY_CLIENT_SECRET` | _empty_ | Client secret (omit when using username/password grant) |
| `Authority.Username` | `STELLAOPS_AUTHORITY_USERNAME` | _empty_ | Username for password grant flows | | `Authority.Username` | `STELLAOPS_AUTHORITY_USERNAME` | _empty_ | Username for password grant flows |
| `Authority.Password` | `STELLAOPS_AUTHORITY_PASSWORD` | _empty_ | Password for password grant flows | | `Authority.Password` | `STELLAOPS_AUTHORITY_PASSWORD` | _empty_ | Password for password grant flows |
| `Authority.Scope` | `STELLAOPS_AUTHORITY_SCOPE` | `concelier.jobs.trigger advisory:ingest` | Space-separated OAuth scopes requested for backend operations | | `Authority.Scope` | `STELLAOPS_AUTHORITY_SCOPE` | `concelier.jobs.trigger advisory:ingest` | Space-separated OAuth scopes requested for backend operations |
| `Authority.TokenCacheDirectory` | `STELLAOPS_AUTHORITY_TOKEN_CACHE_DIR` | `~/.stellaops/tokens` | Directory that persists cached tokens | | `Authority.TokenCacheDirectory` | `STELLAOPS_AUTHORITY_TOKEN_CACHE_DIR` | `~/.stellaops/tokens` | Directory that persists cached tokens |
| `Authority.Resilience.EnableRetries` | `STELLAOPS_AUTHORITY_ENABLE_RETRIES` | `true` | Toggle Polly retry handler for Authority HTTP calls | | `Authority.Resilience.EnableRetries` | `STELLAOPS_AUTHORITY_ENABLE_RETRIES` | `true` | Toggle Polly retry handler for Authority HTTP calls |
| `Authority.Resilience.RetryDelays` | `STELLAOPS_AUTHORITY_RETRY_DELAYS` | `1s,2s,5s` | Comma- or space-separated backoff delays (hh:mm:ss) | | `Authority.Resilience.RetryDelays` | `STELLAOPS_AUTHORITY_RETRY_DELAYS` | `1s,2s,5s` | Comma- or space-separated backoff delays (hh:mm:ss) |
| `Authority.Resilience.AllowOfflineCacheFallback` | `STELLAOPS_AUTHORITY_ALLOW_OFFLINE_CACHE_FALLBACK` | `true` | Allow CLI to reuse cached discovery/JWKS metadata when Authority is offline | | `Authority.Resilience.AllowOfflineCacheFallback` | `STELLAOPS_AUTHORITY_ALLOW_OFFLINE_CACHE_FALLBACK` | `true` | Allow CLI to reuse cached discovery/JWKS metadata when Authority is offline |
| `Authority.Resilience.OfflineCacheTolerance` | `STELLAOPS_AUTHORITY_OFFLINE_CACHE_TOLERANCE` | `00:10:00` | Additional tolerance window applied to cached metadata | | `Authority.Resilience.OfflineCacheTolerance` | `STELLAOPS_AUTHORITY_OFFLINE_CACHE_TOLERANCE` | `00:10:00` | Additional tolerance window applied to cached metadata |
Example bootstrap: Example bootstrap:
```bash ```bash
export STELLAOPS_BACKEND_URL="http://localhost:5000" export STELLAOPS_BACKEND_URL="http://localhost:5000"
export STELLAOPS_RESULTS_DIRECTORY="$HOME/.stellaops/results" export STELLAOPS_RESULTS_DIRECTORY="$HOME/.stellaops/results"
export STELLAOPS_AUTHORITY_URL="https://authority.local" export STELLAOPS_AUTHORITY_URL="https://authority.local"
export STELLAOPS_AUTHORITY_CLIENT_ID="concelier-cli" export STELLAOPS_AUTHORITY_CLIENT_ID="concelier-cli"
export STELLAOPS_AUTHORITY_CLIENT_SECRET="s3cr3t" export STELLAOPS_AUTHORITY_CLIENT_SECRET="s3cr3t"
export STELLAOPS_AUTHORITY_SCOPE="concelier.jobs.trigger advisory:ingest advisory:read" export STELLAOPS_AUTHORITY_SCOPE="concelier.jobs.trigger advisory:ingest advisory:read"
dotnet run --project src/Cli/StellaOps.Cli -- db merge dotnet run --project src/Cli/StellaOps.Cli -- db merge
# Acquire a bearer token and confirm cache state # Acquire a bearer token and confirm cache state
dotnet run --project src/Cli/StellaOps.Cli -- auth login dotnet run --project src/Cli/StellaOps.Cli -- auth login
dotnet run --project src/Cli/StellaOps.Cli -- auth status dotnet run --project src/Cli/StellaOps.Cli -- auth status
dotnet run --project src/Cli/StellaOps.Cli -- auth whoami dotnet run --project src/Cli/StellaOps.Cli -- auth whoami
``` ```
Refer to `docs/dev/32_AUTH_CLIENT_GUIDE.md` for deeper guidance on tuning retry/offline settings and rollout checklists. Refer to `docs/dev/32_AUTH_CLIENT_GUIDE.md` for deeper guidance on tuning retry/offline settings and rollout checklists.
To persist configuration, you can create `stellaops-cli.yaml` next to the binary or To persist configuration, you can create `stellaops-cli.yaml` next to the binary or
rely on environment variables for ephemeral runners. rely on environment variables for ephemeral runners.
--- ---
## 3 · Operating Workflow ## 3 · Operating Workflow
1. **Trigger connector fetch stages** 1. **Trigger connector fetch stages**
```bash ```bash
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage fetch dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage fetch
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage parse dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage parse
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage map dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage map
``` ```
Use `--mode resume` when continuing from a previous window: Use `--mode resume` when continuing from a previous window:
```bash ```bash
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source redhat --stage fetch --mode resume dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source redhat --stage fetch --mode resume
``` ```
2. **Merge canonical advisories** 2. **Merge canonical advisories**
```bash ```bash
dotnet run --project src/Cli/StellaOps.Cli -- db merge dotnet run --project src/Cli/StellaOps.Cli -- db merge
``` ```
3. **Produce exports** 3. **Produce exports**
```bash ```bash
# JSON tree (vuln-list style) # JSON tree (vuln-list style)
dotnet run --project src/Cli/StellaOps.Cli -- db export --format json dotnet run --project src/Cli/StellaOps.Cli -- db export --format json
# Trivy DB (delta example) # Trivy DB (delta example)
dotnet run --project src/Cli/StellaOps.Cli -- db export --format trivy-db --delta dotnet run --project src/Cli/StellaOps.Cli -- db export --format trivy-db --delta
``` ```
Concelier always produces a deterministic OCI layout. The first run after a clean Concelier always produces a deterministic OCI layout. The first run after a clean
bootstrap emits a **full** baseline; subsequent `--delta` runs reuse the previous bootstrap emits a **full** baseline; subsequent `--delta` runs reuse the previous
baseline’s blobs when only JSON manifests change. If the exporter detects that a baseline’s blobs when only JSON manifests change. If the exporter detects that a
prior delta is still active (i.e., `LastDeltaDigest` is recorded) it automatically prior delta is still active (i.e., `LastDeltaDigest` is recorded) it automatically
upgrades the next run to a full export and resets the baseline so operators never upgrades the next run to a full export and resets the baseline so operators never
chain deltas indefinitely. The CLI exposes `--publish-full/--publish-delta` (for chain deltas indefinitely. The CLI exposes `--publish-full/--publish-delta` (for
ORAS pushes) and `--include-full/--include-delta` (for offline bundles) should you ORAS pushes) and `--include-full/--include-delta` (for offline bundles) should you
need to override the defaults interactively. need to override the defaults interactively.
**Smoke-check delta reuse:** after the first baseline completes, run the export a **Smoke-check delta reuse:** after the first baseline completes, run the export a
second time with `--delta` and verify that the new directory reports `mode=delta` second time with `--delta` and verify that the new directory reports `mode=delta`
while reusing the previous layer blob. while reusing the previous layer blob.
```bash ```bash
export_root=${CONCELIER_EXPORT_ROOT:-exports/trivy} export_root=${CONCELIER_EXPORT_ROOT:-exports/trivy}
base=$(ls -1d "$export_root"/* | sort | tail -n2 | head -n1) base=$(ls -1d "$export_root"/* | sort | tail -n2 | head -n1)
delta=$(ls -1d "$export_root"/* | sort | tail -n1) delta=$(ls -1d "$export_root"/* | sort | tail -n1)
jq -r '.mode,.baseExportId' "$delta/metadata.json" jq -r '.mode,.baseExportId' "$delta/metadata.json"
base_manifest=$(jq -r '.manifests[0].digest' "$base/index.json") base_manifest=$(jq -r '.manifests[0].digest' "$base/index.json")
delta_manifest=$(jq -r '.manifests[0].digest' "$delta/index.json") delta_manifest=$(jq -r '.manifests[0].digest' "$delta/index.json")
printf 'baseline manifest: %s\ndelta manifest: %s\n' "$base_manifest" "$delta_manifest" printf 'baseline manifest: %s\ndelta manifest: %s\n' "$base_manifest" "$delta_manifest"
layer_digest=$(jq -r '.layers[0].digest' "$base/blobs/sha256/${base_manifest#sha256:}") layer_digest=$(jq -r '.layers[0].digest' "$base/blobs/sha256/${base_manifest#sha256:}")
cmp "$base/blobs/sha256/${layer_digest#sha256:}" \ cmp "$base/blobs/sha256/${layer_digest#sha256:}" \
"$delta/blobs/sha256/${layer_digest#sha256:}" "$delta/blobs/sha256/${layer_digest#sha256:}"
``` ```
`cmp` returning exit code `0` confirms the delta export reuses the baseline’s `cmp` returning exit code `0` confirms the delta export reuses the baseline’s
`db.tar.gz` layer instead of rebuilding it. `db.tar.gz` layer instead of rebuilding it.
4. **Verify guard compliance** 4. **Verify guard compliance**
```bash ```bash
export STELLA_TENANT="${STELLA_TENANT:-tenant-a}" export STELLA_TENANT="${STELLA_TENANT:-tenant-a}"
dotnet run --project src/Cli/StellaOps.Cli -- aoc verify \ dotnet run --project src/Cli/StellaOps.Cli -- aoc verify \
--since 24h \ --since 24h \
--format table \ --format table \
--tenant "$STELLA_TENANT" --tenant "$STELLA_TENANT"
# Optional: capture JSON evidence for pipelines/audits # Optional: capture JSON evidence for pipelines/audits
dotnet run --project src/Cli/StellaOps.Cli -- aoc verify \ dotnet run --project src/Cli/StellaOps.Cli -- aoc verify \
--since 7d \ --since 7d \
--limit 100 \ --limit 100 \
--format json \ --format json \
--export artifacts/aoc-verify.json \ --export artifacts/aoc-verify.json \
--tenant "$STELLA_TENANT" --tenant "$STELLA_TENANT"
``` ```
The CLI exits with `0` when no violations are detected. Guard failures map The CLI exits with `0` when no violations are detected. Guard failures map
to `ERR_AOC_00x` codes (`11…17`), while truncated results return `18`. Use to `ERR_AOC_00x` codes (`11…17`), while truncated results return `18`. Use
`--sources`/`--codes` to focus on noisy connectors and feed the exported JSON `--sources`/`--codes` to focus on noisy connectors and feed the exported JSON
into dashboards or evidence lockers for compliance reviews. into dashboards or evidence lockers for compliance reviews.
5. **Pre-flight individual payloads** 5. **Pre-flight individual payloads**
```bash ```bash
stella sources ingest --dry-run \ stella sources ingest --dry-run \
--source redhat \ --source redhat \
--input ./fixtures/redhat/RHSA-2025-9999.json \ --input ./fixtures/redhat/RHSA-2025-9999.json \
--tenant "$STELLA_TENANT" \ --tenant "$STELLA_TENANT" \
--format json \ --format json \
--output artifacts/redhat-dry-run.json --output artifacts/redhat-dry-run.json
``` ```
Exit code `0` confirms the candidate document is AOC compliant. Any guard Exit code `0` confirms the candidate document is AOC compliant. Any guard
violation is emitted as deterministic `ERR_AOC_00x` exit codes (`11…17`); violation is emitted as deterministic `ERR_AOC_00x` exit codes (`11…17`);
reuse the exported JSON in PRs or incident timelines to show offending paths. reuse the exported JSON in PRs or incident timelines to show offending paths.
6. **Manage scanners (optional)** 6. **Manage scanners (optional)**
```bash ```bash
dotnet run --project src/Cli/StellaOps.Cli -- scanner download --channel stable dotnet run --project src/Cli/StellaOps.Cli -- scanner download --channel stable
dotnet run --project src/Cli/StellaOps.Cli -- scan run --entry scanners/latest/Scanner.dll --target ./sboms dotnet run --project src/Cli/StellaOps.Cli -- scan run --entry scanners/latest/Scanner.dll --target ./sboms
dotnet run --project src/Cli/StellaOps.Cli -- scan upload --file results/scan-001.json dotnet run --project src/Cli/StellaOps.Cli -- scan upload --file results/scan-001.json
``` ```
Add `--verbose` to any command for structured console logs. All commands honour Add `--verbose` to any command for structured console logs. All commands honour
`Ctrl+C` cancellation and exit with non-zero status codes when the backend returns `Ctrl+C` cancellation and exit with non-zero status codes when the backend returns
a problem document. a problem document.
--- ---
## 4 · Verification Checklist ## 4 · Verification Checklist
- Concelier `/health` returns `"status":"healthy"` and Storage bootstrap is marked - Concelier `/health` returns `"status":"healthy"` and Storage bootstrap is marked
complete after startup. complete after startup.
- CLI commands return HTTP 202 with a `Location` header (job tracking URL) when - CLI commands return HTTP 202 with a `Location` header (job tracking URL) when
triggering Concelier jobs. triggering Concelier jobs.
- Export artefacts are materialised under the configured output directories and - Export artefacts are materialised under the configured output directories and
their manifests record digests. their manifests record digests.
- MongoDB contains the expected `document`, `dto`, `advisory`, and `export_state` - MongoDB contains the expected `document`, `dto`, `advisory`, and `export_state`
collections after a run. collections after a run.
--- ---
## 5 · Deployment Automation ## 5 · Deployment Automation
- Treat `etc/concelier.yaml.sample` as the canonical template. CI/CD should copy it to - Treat `etc/concelier.yaml.sample` as the canonical template. CI/CD should copy it to
the deployment artifact and replace placeholders (DSN, telemetry endpoints, cron the deployment artifact and replace placeholders (DSN, telemetry endpoints, cron
overrides) with environment-specific secrets. overrides) with environment-specific secrets.
- Keep secret material (Mongo credentials, OTLP tokens) outside of the repository; - Keep secret material (Mongo credentials, OTLP tokens) outside of the repository;
inject them via secret stores or pipeline variables at stamp time. inject them via secret stores or pipeline variables at stamp time.
- When building container images, include `trivy-db` (and `oras` if used) so air-gapped - When building container images, include `trivy-db` (and `oras` if used) so air-gapped
clusters do not need outbound downloads at runtime. clusters do not need outbound downloads at runtime.
--- ---
## 5 · Next Steps ## 5 · Next Steps
- Enable authority-backed authentication in non-production first. Set - Enable authority-backed authentication in non-production first. Set
`authority.enabled: true` while keeping `authority.allowAnonymousFallback: true` `authority.enabled: true` while keeping `authority.allowAnonymousFallback: true`
to observe logs, then flip it to `false` before 2025-12-31 UTC to enforce tokens. to observe logs, then flip it to `false` before 2025-12-31 UTC to enforce tokens.
- Automate the workflow above via CI/CD (compose stack or Kubernetes CronJobs). - Automate the workflow above via CI/CD (compose stack or Kubernetes CronJobs).
- Pair with the Concelier connector teams when enabling additional sources so their - Pair with the Concelier connector teams when enabling additional sources so their
module-specific requirements are pulled in safely. module-specific requirements are pulled in safely.
--- ---
## 6 · Authority Integration ## 6 · Authority Integration
- Concelier now authenticates callers through StellaOps Authority using OAuth 2.0 - Concelier now authenticates callers through StellaOps Authority using OAuth 2.0
resource server flows. Populate the `authority` block in `concelier.yaml`: resource server flows. Populate the `authority` block in `concelier.yaml`:
```yaml ```yaml
authority: authority:
enabled: true enabled: true
allowAnonymousFallback: false # keep true only during the staged rollout window allowAnonymousFallback: false # keep true only during the staged rollout window
issuer: "https://authority.example.org" issuer: "https://authority.example.org"
audiences: audiences:
- "api://concelier" - "api://concelier"
requiredScopes: requiredScopes:
- "concelier.jobs.trigger" - "concelier.jobs.trigger"
- "advisory:read" - "advisory:read"
- "advisory:ingest" - "advisory:ingest"
requiredTenants: requiredTenants:
- "tenant-default" - "tenant-default"
clientId: "concelier-jobs" clientId: "concelier-jobs"
clientSecretFile: "../secrets/concelier-jobs.secret" clientSecretFile: "../secrets/concelier-jobs.secret"
clientScopes: clientScopes:
- "concelier.jobs.trigger" - "concelier.jobs.trigger"
- "advisory:read" - "advisory:read"
- "advisory:ingest" - "advisory:ingest"
bypassNetworks: bypassNetworks:
- "127.0.0.1/32" - "127.0.0.1/32"
- "::1/128" - "::1/128"
``` ```
- Store the client secret outside of source control. Either provide it via - Store the client secret outside of source control. Either provide it via
`authority.clientSecret` (environment variable `CONCELIER_AUTHORITY__CLIENTSECRET`) `authority.clientSecret` (environment variable `CONCELIER_AUTHORITY__CLIENTSECRET`)
or point `authority.clientSecretFile` to a file mounted at runtime. or point `authority.clientSecretFile` to a file mounted at runtime.
- Cron jobs running on the same host can keep using the API thanks to the loopback - Cron jobs running on the same host can keep using the API thanks to the loopback
bypass mask. Add additional CIDR ranges as needed; every bypass is logged. bypass mask. Add additional CIDR ranges as needed; every bypass is logged.
- Export the same configuration to Kubernetes or systemd by setting environment - Export the same configuration to Kubernetes or systemd by setting environment
variables such as: variables such as:
```bash ```bash
export CONCELIER_AUTHORITY__ENABLED=true export CONCELIER_AUTHORITY__ENABLED=true
export CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=false export CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=false
export CONCELIER_AUTHORITY__ISSUER="https://authority.example.org" export CONCELIER_AUTHORITY__ISSUER="https://authority.example.org"
export CONCELIER_AUTHORITY__CLIENTID="concelier-jobs" export CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
export CONCELIER_AUTHORITY__CLIENTSECRETFILE="/var/run/secrets/concelier/authority-client" export CONCELIER_AUTHORITY__CLIENTSECRETFILE="/var/run/secrets/concelier/authority-client"
export CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger" export CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger"
export CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read" export CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read"
export CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest" export CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest"
export CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger" export CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger"
export CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read" export CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read"
export CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest" export CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest"
export CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default" export CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default"
``` ```
- CLI commands already pass `Authorization` headers when credentials are supplied. - CLI commands already pass `Authorization` headers when credentials are supplied.
Configure the CLI with matching Authority settings (`docs/09_API_CLI_REFERENCE.md`) Configure the CLI with matching Authority settings (`docs/09_API_CLI_REFERENCE.md`)
so that automation can obtain tokens with the same client credentials. Concelier so that automation can obtain tokens with the same client credentials. Concelier
logs every job request with the client ID, subject (if present), scopes, and logs every job request with the client ID, subject (if present), scopes, and
a `bypass` flag so operators can audit cron traffic. a `bypass` flag so operators can audit cron traffic.
- **Rollout checklist.** - **Rollout checklist.**
1. Stage the integration with fallback enabled (`allowAnonymousFallback=true`) and confirm CLI/token issuance using `stella auth status`. 1. Stage the integration with fallback enabled (`allowAnonymousFallback=true`) and confirm CLI/token issuance using `stella auth status`.
2. Follow the rehearsal pattern (`allowAnonymousFallback=false`) while monitoring `Concelier.Authorization.Audit` and `web.jobs.triggered`/`web.jobs.trigger.failed` metrics. 2. Follow the rehearsal pattern (`allowAnonymousFallback=false`) while monitoring `Concelier.Authorization.Audit` and `web.jobs.triggered`/`web.jobs.trigger.failed` metrics.
3. Lock in enforcement, review the audit runbook (`docs/modules/concelier/operations/authority-audit-runbook.md`), and document the bypass CIDR approvals in your change log. 3. Lock in enforcement, review the audit runbook (`docs/modules/concelier/operations/authority-audit-runbook.md`), and document the bypass CIDR approvals in your change log.

1100
docs/11_DATA_SCHEMAS.md
View File

File diff suppressed because it is too large Load Diff

338
docs/12_PERFORMANCE_WORKBOOK.md
View File

@@ -1,170 +1,170 @@
# 12 - Performance Workbook # 12 - Performance Workbook
*Purpose* – define **repeatable, data‑driven** benchmarks that guard Stella Ops’ core pledge: *Purpose* – define **repeatable, data‑driven** benchmarks that guard Stella Ops’ core pledge:
> *“P95 vulnerability feedback in ≤ 5 seconds.”* > *“P95 vulnerability feedback in ≤ 5 seconds.”*
--- ---
## 0 Benchmark Scope ## 0 Benchmark Scope
| Area | Included | Excluded | | Area | Included | Excluded |
|------------------|----------------------------------|---------------------------| |------------------|----------------------------------|---------------------------|
| SBOM‑first scan | Trivy engine w/ warmed DB | Full image unpack ≥ 300 MB | | SBOM‑first scan | Trivy engine w/ warmed DB | Full image unpack ≥ 300 MB |
| Delta SBOM ⭑ | Missing‑layer lookup & merge | Multi‑arch images | | Delta SBOM ⭑ | Missing‑layer lookup & merge | Multi‑arch images |
| Policy eval ⭑ | YAML → JSON → rule match | Rego (until GA) | | Policy eval ⭑ | YAML → JSON → rule match | Rego (until GA) |
| Feed merge | NVD JSON 2023–2025 | GHSA GraphQL (plugin) | | Feed merge | NVD JSON 2023–2025 | GHSA GraphQL (plugin) |
| Quota wait‑path | 5 s soft‑wait, 60 s hard‑wait behaviour | Paid tiers (unlimited) | | Quota wait‑path | 5 s soft‑wait, 60 s hard‑wait behaviour | Paid tiers (unlimited) |
| API latency | REST `/scan`, `/layers/missing` | UI SPA calls | | API latency | REST `/scan`, `/layers/missing` | UI SPA calls |
⭑ = new in July 2025. ⭑ = new in July 2025.
--- ---
## 1 Hardware Baseline (Reference Rig) ## 1 Hardware Baseline (Reference Rig)
| Element | Spec | | Element | Spec |
|-------------|------------------------------------| |-------------|------------------------------------|
| CPU | 8 vCPU (Intel Ice‑Lake equiv.) | | CPU | 8 vCPU (Intel Ice‑Lake equiv.) |
| Memory | 16 GiB | | Memory | 16 GiB |
| Disk | NVMe SSD, 3 GB/s R/W | | Disk | NVMe SSD, 3 GB/s R/W |
| Network | 1 Gbit virt. switch | | Network | 1 Gbit virt. switch |
| Container | Docker 25.0 + overlay2 | | Container | Docker 25.0 + overlay2 |
| OS | Ubuntu 22.04 LTS (kernel 6.8) | | OS | Ubuntu 22.04 LTS (kernel 6.8) |
*All P95 targets assume a **single‑node** deployment on this rig unless stated.* *All P95 targets assume a **single‑node** deployment on this rig unless stated.*
--- ---
## 2 Phase Targets & Gates ## 2 Phase Targets & Gates
| Phase (ID) | Target P95 | Gate (CI) | Rationale | | Phase (ID) | Target P95 | Gate (CI) | Rationale |
|-----------------------|-----------:|-----------|----------------------------------------| |-----------------------|-----------:|-----------|----------------------------------------|
| **SBOM_FIRST** | ≤ 5 s | `hard` | Core UX promise. | | **SBOM_FIRST** | ≤ 5 s | `hard` | Core UX promise. |
| **IMAGE_UNPACK** | ≤ 10 s | `soft` | Fallback path for legacy flows. | | **IMAGE_UNPACK** | ≤ 10 s | `soft` | Fallback path for legacy flows. |
| **DELTA_SBOM** ⭑ | ≤ 1 s | `hard` | Needed to stay sub‑5 s for big bases. | | **DELTA_SBOM** ⭑ | ≤ 1 s | `hard` | Needed to stay sub‑5 s for big bases. |
| **POLICY_EVAL** ⭑ | ≤ 50 ms | `hard` | Keeps gate latency invisible to users. | | **POLICY_EVAL** ⭑ | ≤ 50 ms | `hard` | Keeps gate latency invisible to users. |
| **QUOTA_WAIT** ⭑ | *soft* ≤ 5 s<br>*hard* ≤ 60 s | `hard` | Ensures graceful Free‑tier throttling. | | **QUOTA_WAIT** ⭑ | *soft* ≤ 5 s<br>*hard* ≤ 60 s | `hard` | Ensures graceful Free‑tier throttling. |
| **SCHED_RESCAN** | ≤ 30 s | `soft` | Nightly batch – not user‑facing. | | **SCHED_RESCAN** | ≤ 30 s | `soft` | Nightly batch – not user‑facing. |
| **FEED_MERGE** | ≤ 60 s | `soft` | Off‑peak cron @ 01:00. | | **FEED_MERGE** | ≤ 60 s | `soft` | Off‑peak cron @ 01:00. |
| **API_P95** | ≤ 200 ms | `hard` | UI snappiness. | | **API_P95** | ≤ 200 ms | `hard` | UI snappiness. |
*Gate* legend — `hard`: break CI if regression > 3 × target, *Gate* legend — `hard`: break CI if regression > 3 × target,
`soft`: raise warning & issue ticket. `soft`: raise warning & issue ticket.
--- ---
## 3 Test Harness ## 3 Test Harness
* **Runner** – `perf/run.sh`, accepts `--phase` and `--samples`. * **Runner** – `perf/run.sh`, accepts `--phase` and `--samples`.
* **Language analyzers microbench** – `dotnet run --project src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj -- --repo-root . --out src/Bench/StellaOps.Bench/Scanner.Analyzers/baseline.csv --json out/bench/scanner-analyzers/latest.json --prom out/bench/scanner-analyzers/latest.prom --commit $(git rev-parse HEAD)` produces CSV + JSON + Prometheus gauges for analyzer scenarios. Runs fail if `max_ms` regresses ≥ 20 % against `baseline.csv` or if thresholds are exceeded. * **Language analyzers microbench** – `dotnet run --project src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj -- --repo-root . --out src/Bench/StellaOps.Bench/Scanner.Analyzers/baseline.csv --json out/bench/scanner-analyzers/latest.json --prom out/bench/scanner-analyzers/latest.prom --commit $(git rev-parse HEAD)` produces CSV + JSON + Prometheus gauges for analyzer scenarios. Runs fail if `max_ms` regresses ≥ 20 % against `baseline.csv` or if thresholds are exceeded.
* **Metrics** – Prometheus + `jq` extracts; aggregated via `scripts/aggregate.ts`. * **Metrics** – Prometheus + `jq` extracts; aggregated via `scripts/aggregate.ts`.
* **CI** – GitLab CI job *benchmark* publishes JSON to `bench‑artifacts/`. * **CI** – GitLab CI job *benchmark* publishes JSON to `bench‑artifacts/`.
* **Visualisation** – Grafana dashboard *Stella‑Perf* (provisioned JSON). * **Visualisation** – Grafana dashboard *Stella‑Perf* (provisioned JSON).
> **Note** – harness mounts `/var/cache/trivy` tmpfs to avoid disk noise. > **Note** – harness mounts `/var/cache/trivy` tmpfs to avoid disk noise.
--- ---
## 4 Current Results (July 2025) ## 4 Current Results (July 2025)
| Phase | Samples | Mean (s) | P95 (s) | Target OK? | | Phase | Samples | Mean (s) | P95 (s) | Target OK? |
|---------------|--------:|---------:|--------:|-----------:| |---------------|--------:|---------:|--------:|-----------:|
| SBOM_FIRST | 100 | 3.7 | 4.9 | ✅ | | SBOM_FIRST | 100 | 3.7 | 4.9 | ✅ |
| IMAGE_UNPACK | 50 | 6.4 | 9.2 | ✅ | | IMAGE_UNPACK | 50 | 6.4 | 9.2 | ✅ |
| **DELTA_SBOM**| 100 | 0.46 | 0.83 | ✅ | | **DELTA_SBOM**| 100 | 0.46 | 0.83 | ✅ |
| **POLICY_EVAL** | 1 000 | 0.021 | 0.041 | ✅ | | **POLICY_EVAL** | 1 000 | 0.021 | 0.041 | ✅ |
| **QUOTA_WAIT** | 80 | 4.0* | 4.9* | ✅ | | **QUOTA_WAIT** | 80 | 4.0* | 4.9* | ✅ |
| SCHED_RESCAN | 10 | 18.3 | 24.9 | ✅ | | SCHED_RESCAN | 10 | 18.3 | 24.9 | ✅ |
| FEED_MERGE | 3 | 38.1 | 41.0 | ✅ | | FEED_MERGE | 3 | 38.1 | 41.0 | ✅ |
| API_P95 | 20 000 | 0.087 | 0.143 | ✅ | | API_P95 | 20 000 | 0.087 | 0.143 | ✅ |
*Data files:* `bench-artifacts/2025‑07‑14/phase‑stats.json`. *Data files:* `bench-artifacts/2025‑07‑14/phase‑stats.json`.
--- ---
## 5 Δ‑SBOM Micro‑Benchmark Detail ## 5 Δ‑SBOM Micro‑Benchmark Detail
### 5.1 Scenario ### 5.1 Scenario
1. Base image `python:3.12-slim` already scanned (all layers cached). 1. Base image `python:3.12-slim` already scanned (all layers cached).
2. Application layer (`COPY . /app`) triggers new digest. 2. Application layer (`COPY . /app`) triggers new digest.
3. `Stella CLI` lists **7** layers, backend replies *6 hit*, *1 miss*. 3. `Stella CLI` lists **7** layers, backend replies *6 hit*, *1 miss*.
4. Builder scans **only 1 layer** (~9 MiB, 217 files) & uploads delta. 4. Builder scans **only 1 layer** (~9 MiB, 217 files) & uploads delta.
### 5.2 Key Timings ### 5.2 Key Timings
| Step | Time (ms) | | Step | Time (ms) |
|---------------------|----------:| |---------------------|----------:|
| `/layers/missing` | 13 | | `/layers/missing` | 13 |
| Trivy single layer | 655 | | Trivy single layer | 655 |
| Upload delta blob | 88 | | Upload delta blob | 88 |
| Backend merge + CVE | 74 | | Backend merge + CVE | 74 |
| **Total wall‑time** | **830 ms** | | **Total wall‑time** | **830 ms** |
--- ---
## 6 Quota Wait‑Path Benchmark Detail ## 6 Quota Wait‑Path Benchmark Detail
### 6.1 Scenario ### 6.1 Scenario
1. Free‑tier token reaches **scan #200** – dashboard shows yellow banner. 1. Free‑tier token reaches **scan #200** – dashboard shows yellow banner.
### 6.2 Key Timings ### 6.2 Key Timings
| Step | Time (ms) | | Step | Time (ms) |
|------------------------------------|----------:| |------------------------------------|----------:|
| `/quota/check` Redis LUA INCR | 0.8 | | `/quota/check` Redis LUA INCR | 0.8 |
| Soft wait sleep (server) | 5 000 | | Soft wait sleep (server) | 5 000 |
| Hard wait sleep (server) | 60 000 | | Hard wait sleep (server) | 60 000 |
| End‑to‑end wall‑time (soft‑hit) | 5 003 | | End‑to‑end wall‑time (soft‑hit) | 5 003 |
| End‑to‑end wall‑time (hard‑hit) | 60 004 | | End‑to‑end wall‑time (hard‑hit) | 60 004 |
--- ---
## 7 Policy Eval Bench ## 7 Policy Eval Bench
### 7.1 Setup ### 7.1 Setup
* Policy YAML: **28** rules, mix severity & package conditions. * Policy YAML: **28** rules, mix severity & package conditions.
* Input: scan result JSON with **1 026** findings. * Input: scan result JSON with **1 026** findings.
* Evaluator: custom rules engine (Go structs → map look‑ups). * Evaluator: custom rules engine (Go structs → map look‑ups).
### 7.2 Latency Histogram ### 7.2 Latency Histogram
``` ```
0‑10 ms ▇▇▇▇▇▇▇▇▇▇ 38 % 0‑10 ms ▇▇▇▇▇▇▇▇▇▇ 38 %
10‑20 ms ▇▇▇▇▇▇▇▇▇▇ 42 % 10‑20 ms ▇▇▇▇▇▇▇▇▇▇ 42 %
20‑40 ms ▇▇▇▇▇▇ 17 % 20‑40 ms ▇▇▇▇▇▇ 17 %
40‑50 ms ▇ 3 % 40‑50 ms ▇ 3 %
``` ```
P99 = 48 ms. Meets 50 ms gate. P99 = 48 ms. Meets 50 ms gate.
--- ---
## 8 Trend Snapshot ## 8 Trend Snapshot
> _Perf trend spark‑line screenshot pending upload._ > _Perf trend spark‑line screenshot pending upload._
> **Grafana/Alerting** – Import `docs/modules/scanner/operations/analyzers-grafana-dashboard.json` and point it at the Prometheus datasource storing `scanner_analyzer_bench_*` metrics. Configure an alert on `scanner_analyzer_bench_regression_ratio` ≥ 1.20 (default limit); the bundled Stat panel surfaces breached scenarios (non-zero values). On-call runbook: `docs/modules/scanner/operations/analyzers.md`. > **Grafana/Alerting** – Import `docs/modules/scanner/operations/analyzers-grafana-dashboard.json` and point it at the Prometheus datasource storing `scanner_analyzer_bench_*` metrics. Configure an alert on `scanner_analyzer_bench_regression_ratio` ≥ 1.20 (default limit); the bundled Stat panel surfaces breached scenarios (non-zero values). On-call runbook: `docs/modules/scanner/operations/analyzers.md`.
_Plot generated weekly by `scripts/update‑trend.py`; shows last 12 weeks P95 per phase._ _Plot generated weekly by `scripts/update‑trend.py`; shows last 12 weeks P95 per phase._
--- ---
## 9 Action Items ## 9 Action Items
1. **Image Unpack** – Evaluate zstd for layer decompress; aim to shave 1 s. 1. **Image Unpack** – Evaluate zstd for layer decompress; aim to shave 1 s.
2. **Feed Merge** – Parallelise regional XML feed parse (plugin) once stable. 2. **Feed Merge** – Parallelise regional XML feed parse (plugin) once stable.
3. **Rego Support** – Prototype OPA side‑car; target ≤ 100 ms eval. 3. **Rego Support** – Prototype OPA side‑car; target ≤ 100 ms eval.
4. **Concurrency** – Stress‑test 100 rps on 4‑node Redis cluster (Q4‑2025). 4. **Concurrency** – Stress‑test 100 rps on 4‑node Redis cluster (Q4‑2025).
--- ---
## 10 Change Log ## 10 Change Log
| Date | Note | | Date | Note |
|------------|-------------------------------------------------------------------------| |------------|-------------------------------------------------------------------------|
| 2025‑07‑14 | Added Δ‑SBOM & Policy Eval phases; updated targets & current results. | | 2025‑07‑14 | Added Δ‑SBOM & Policy Eval phases; updated targets & current results. |
| 2025‑07‑12 | First public workbook (SBOM‑first, image‑unpack, feed merge). | | 2025‑07‑12 | First public workbook (SBOM‑first, image‑unpack, feed merge). |
--- ---

460
docs/13_RELEASE_ENGINEERING_PLAYBOOK.md
View File

@@ -1,230 +1,230 @@
# 13 · Release Engineering Playbook — Stella Ops # 13 · Release Engineering Playbook — Stella Ops
A concise, automation‑first guide describing **how source code on `main` becomes a verifiably signed, air‑gap‑friendly release**. A concise, automation‑first guide describing **how source code on `main` becomes a verifiably signed, air‑gap‑friendly release**.
It is opinionated for offline use‑cases and supply‑chain security (SLSA ≥ level 2 today, aiming for level 3). It is opinionated for offline use‑cases and supply‑chain security (SLSA ≥ level 2 today, aiming for level 3).
--- ---
## 0 Release Philosophy ## 0 Release Philosophy
* **Fast but fearless** – every commit on `main` must be releasable; broken builds break the build, not the team. * **Fast but fearless** – every commit on `main` must be releasable; broken builds break the build, not the team.
* **Reproducible** – anyone can rebuild byte‑identical artefacts with a single `make release` offline. * **Reproducible** – anyone can rebuild byte‑identical artefacts with a single `make release` offline.
* **Secure by default** – every artefact ships with a SBOM, Cosign signature and (future) Rekor log entry. * **Secure by default** – every artefact ships with a SBOM, Cosign signature and (future) Rekor log entry.
* **Offline‑first** – all dependencies are vendored or mirrored into the internal registry; no Internet required at runtime. * **Offline‑first** – all dependencies are vendored or mirrored into the internal registry; no Internet required at runtime.
--- ---
## 1 Versioning & Branching ## 1 Versioning & Branching
| Branch | Purpose | Auto‑publish? | | Branch | Purpose | Auto‑publish? |
| ------------- | ------------------------------ | --------------------------------------- | | ------------- | ------------------------------ | --------------------------------------- |
| `main` | Always‑green development trunk | `nightly-*` images | | `main` | Always‑green development trunk | `nightly-*` images |
| `release/X.Y` | Stabilise a minor line | `stella:X.Y-rcN` | | `release/X.Y` | Stabilise a minor line | `stella:X.Y-rcN` |
| Tags | `X.Y.Z` = SemVer | `stella:X.Y.Z`, OUK tarball, Helm chart | | Tags | `X.Y.Z` = SemVer | `stella:X.Y.Z`, OUK tarball, Helm chart |
* **SemVer** – MAJOR for breaking API/CLI changes, MINOR for features, PATCH for fixes. * **SemVer** – MAJOR for breaking API/CLI changes, MINOR for features, PATCH for fixes.
* Release tags are **signed** (`git tag -s`) with the Stella Ops GPG key (`0x90C4…`). * Release tags are **signed** (`git tag -s`) with the Stella Ops GPG key (`0x90C4…`).
--- ---
## 2 CI/CD Overview (GitLab CI + GitLab Runner) ## 2 CI/CD Overview (GitLab CI + GitLab Runner)
```mermaid ```mermaid
graph LR graph LR
A[push / MR] --> Lint A[push / MR] --> Lint
Lint --> Unit Lint --> Unit
Unit --> Build Unit --> Build
Build --> Test-Container Build --> Test-Container
Test-Container --> SBOM Test-Container --> SBOM
SBOM --> Sign SBOM --> Sign
Sign --> Publish Sign --> Publish
Publish --> E2E Publish --> E2E
Publish --> Notify Publish --> Notify
``` ```
### Pipeline Stages ### Pipeline Stages
| Stage | Key tasks | | Stage | Key tasks |
| ------------------ | ------------------------------------------------------------------------------------------------ | | ------------------ | ------------------------------------------------------------------------------------------------ |
| **Lint** | ESLint, golangci‑lint, hadolint, markdown‑lint. | | **Lint** | ESLint, golangci‑lint, hadolint, markdown‑lint. |
| **Unit** | `dotnet test`, `go test`, Jest UI tests. | | **Unit** | `dotnet test`, `go test`, Jest UI tests. |
| **Quota unit‑tests 🏷** | Validate QuotaService logic: reset at UTC, 5 s vs 60 s waits, header correctness. | | **Quota unit‑tests 🏷** | Validate QuotaService logic: reset at UTC, 5 s vs 60 s waits, header correctness. |
| **Build** | Multi‑arch container build (`linux/amd64`, `linux/arm64`) using **BuildKit** + `--provenance` 📌. | | **Build** | Multi‑arch container build (`linux/amd64`, `linux/arm64`) using **BuildKit** + `--provenance` 📌. |
| **Test‑Container** | Spin up compose file, run smoke APIs. | | **Test‑Container** | Spin up compose file, run smoke APIs. |
| **SBOM** 📌 | Invoke **StellaOps.SBOMBuilder** to generate SPDX JSON + attach `.sbom` label to image. | | **SBOM** 📌 | Invoke **StellaOps.SBOMBuilder** to generate SPDX JSON + attach `.sbom` label to image. |
| **Sign** | Sign image with **Cosign** (`cosign sign --key cosign.key`). | | **Sign** | Sign image with **Cosign** (`cosign sign --key cosign.key`). |
| **Publish** | Push to `registry.git.stella-ops.org`. | | **Publish** | Push to `registry.git.stella-ops.org`. |
| **E2E** | Kind‑based Kubernetes test incl. Zastava DaemonSet; verify sub‑5 s scan SLA. | | **E2E** | Kind‑based Kubernetes test incl. Zastava DaemonSet; verify sub‑5 s scan SLA. |
| **Notify** | Report to Mattermost & GitLab Slack app. | | **Notify** | Report to Mattermost & GitLab Slack app. |
| **OfflineToken** | Call `JwtIssuer.Generate(exp=30d)` → store `client.jwt` artefact → attach to OUK build context | | **OfflineToken** | Call `JwtIssuer.Generate(exp=30d)` → store `client.jwt` artefact → attach to OUK build context |
*All stages run in parallel where possible; max wall‑time < 15 min.* *All stages run in parallel where possible; max wall‑time < 15 min.*
**Implementation note.** `.gitea/workflows/release.yml` executes **Implementation note.** `.gitea/workflows/release.yml` executes
`ops/devops/release/build_release.py` to build multi-arch images, attach `ops/devops/release/build_release.py` to build multi-arch images, attach
CycloneDX SBOMs and SLSA provenance with Cosign, and emit CycloneDX SBOMs and SLSA provenance with Cosign, and emit
`out/release/release.yaml` for downstream packaging (Helm, Compose, Offline Kit). `out/release/release.yaml` for downstream packaging (Helm, Compose, Offline Kit).
The `build-test-deploy` workflow also runs The `build-test-deploy` workflow also runs
`python ops/devops/release/test_verify_release.py` so release verifier `python ops/devops/release/test_verify_release.py` so release verifier
regressions fail fast during every CI pass. regressions fail fast during every CI pass.
--- ---
## 3 Container Image Strategy ## 3 Container Image Strategy
| Image | Registry Tag | Contents | | Image | Registry Tag | Contents |
| ------------------------------ | --------------------------- | ---------------------------------------------------------------------- | | ------------------------------ | --------------------------- | ---------------------------------------------------------------------- |
| **backend** | `stella/backend:{ver}` | ASP.NET API, plugin loader. | | **backend** | `stella/backend:{ver}` | ASP.NET API, plugin loader. |
| **ui** | `stella/ui:{ver}` | Pre‑built Angular SPA. | | **ui** | `stella/ui:{ver}` | Pre‑built Angular SPA. |
| **runner-trivy** | `stella/runner-trivy:{ver}` | Trivy CLI + SPDX/CycloneDX 🛠. | | **runner-trivy** | `stella/runner-trivy:{ver}` | Trivy CLI + SPDX/CycloneDX 🛠. |
| **runner-grype** | `stella/runner-grype:{ver}` | Optional plug‑in scanner. | | **runner-grype** | `stella/runner-grype:{ver}` | Optional plug‑in scanner. |
| **🏷️ StellaOps.Registry** 📌 | `stella/registry:{ver}` | Scratch image embedding Docker Registry v2 + Cosign policy controller. | | **🏷️ StellaOps.Registry** 📌 | `stella/registry:{ver}` | Scratch image embedding Docker Registry v2 + Cosign policy controller. |
| **🏷️ StellaOps.MutePolicies** 📌 | `stella/policies:{ver}` | Sidecar serving policy bundles. | | **🏷️ StellaOps.MutePolicies** 📌 | `stella/policies:{ver}` | Sidecar serving policy bundles. |
| **🏷️ StellaOps.Attestor** 📌 | `stella/attestor:{ver}` | SLSA provenance & Rekor signer (future). | | **🏷️ StellaOps.Attestor** 📌 | `stella/attestor:{ver}` | SLSA provenance & Rekor signer (future). |
*Images are **`--label org.opencontainers.image.source=git.stella-ops.ru`** and include SBOMs generated at build time.* *Images are **`--label org.opencontainers.image.source=git.stella-ops.ru`** and include SBOMs generated at build time.*
--- ---
## 4 📌 Offline Update Kit (OUK) Build & Distribution ## 4 📌 Offline Update Kit (OUK) Build & Distribution
**Purpose** – deliver updated CVE feeds & Trivy DB to air‑gapped clusters. **Purpose** – deliver updated CVE feeds & Trivy DB to air‑gapped clusters.
### 4.1 CLI Tool ### 4.1 CLI Tool
*Go binary `ouk` lives in `src/Tools/ouk/`.* *Go binary `ouk` lives in `src/Tools/ouk/`.*
```sh ```sh
ouk fetch \ ouk fetch \
--nvd --osv \ --nvd --osv \
--trivy-db --date $(date -I) \ --trivy-db --date $(date -I) \
--output ouk-$(date +%Y%m%d).tar.gz \ --output ouk-$(date +%Y%m%d).tar.gz \
--sign cosign.key --sign cosign.key
``` ```
### 4.2 Pipeline Hook ### 4.2 Pipeline Hook
* Runs on **first Friday** each month (cron). * Runs on **first Friday** each month (cron).
* Generates tarball, signs it, uploads to **GitLab Release asset**. * Generates tarball, signs it, uploads to **GitLab Release asset**.
* SHA‑256 + signature published alongside. * SHA‑256 + signature published alongside.
* Release job must emit `out/release/debug/` with `debug-manifest.json` and `.sha256` so `ops/offline-kit/mirror_debug_store.py` can mirror symbols into the Offline Kit (see `DEVOPS-REL-17-004`). * Release job must emit `out/release/debug/` with `debug-manifest.json` and `.sha256` so `ops/offline-kit/mirror_debug_store.py` can mirror symbols into the Offline Kit (see `DEVOPS-REL-17-004`).
### 4.3 Activation Flow (runtime) ### 4.3 Activation Flow (runtime)
1. Admin uploads `.tar.gz` via **UI → Settings → Offline Updates (OUK)**. 1. Admin uploads `.tar.gz` via **UI → Settings → Offline Updates (OUK)**.
2. Backend verifies Cosign signature & digest. 2. Backend verifies Cosign signature & digest.
3. Files extracted into `var/lib/stella/db`. 3. Files extracted into `var/lib/stella/db`.
4. Redis caches invalidated; Dashboard “Feed Age” ticks green. 4. Redis caches invalidated; Dashboard “Feed Age” ticks green.
5. Audit event `ouk_update` stored. 5. Audit event `ouk_update` stored.
### 4.4 Token Detail ### 4.4 Token Detail
client.jwt placed under /root/ inside the tarball. client.jwt placed under /root/ inside the tarball.
CI job fails if token expiry < 29 days (guard against stale caches). CI job fails if token expiry < 29 days (guard against stale caches).
--- ---
## 5 Artifact Signing & Transparency ## 5 Artifact Signing & Transparency
| Artefact | Signer | Tool/Notes | | Artefact | Signer | Tool/Notes |
| ------------ | --------------- | ---------------------------------- | | ------------ | --------------- | ---------------------------------- |
| Git tags | GPG (`0x90C4…`) | `git tag -s` | | Git tags | GPG (`0x90C4…`) | `git tag -s` |
| Containers | Cosign key pair | `cosign sign` | | Containers | Cosign key pair | `cosign sign` |
| Helm Charts | prov file | `helm package --sign` | | Helm Charts | prov file | `helm package --sign` |
| OUK tarballs | Cosign | `cosign sign-blob` | | OUK tarballs | Cosign | `cosign sign-blob` |
| Debug store | — | `debug/debug-manifest.json` hashed | | Debug store | — | `debug/debug-manifest.json` hashed |
**Rekor** integration is **TODO** – once the internal Rekor mirror is online (`StellaOpsAttestor`) a post‑publish job will submit transparency log entries. **Rekor** integration is **TODO** – once the internal Rekor mirror is online (`StellaOpsAttestor`) a post‑publish job will submit transparency log entries.
--- ---
## 6 Release Checklist ## 6 Release Checklist
1. CI pipeline green. 1. CI pipeline green.
2. Bump `VERSION` file. 2. Bump `VERSION` file.
3. Tag `git tag -s X.Y.Z -m "Release X.Y.Z"` & push. 3. Tag `git tag -s X.Y.Z -m "Release X.Y.Z"` & push.
4. GitLab CI auto‑publishes images & charts. 4. GitLab CI auto‑publishes images & charts.
5. Draft GitLab **Release Notes** using `src/Tools/release-notes-gen`. 5. Draft GitLab **Release Notes** using `src/Tools/release-notes-gen`.
6. Verify SBOM attachment with `stella sbom verify stella/backend:X.Y.Z`. 6. Verify SBOM attachment with `stella sbom verify stella/backend:X.Y.Z`.
7. Run the release verifier locally if CI isn’t available (mirrors the workflow step): 7. Run the release verifier locally if CI isn’t available (mirrors the workflow step):
`python ops/devops/release/test_verify_release.py` `python ops/devops/release/test_verify_release.py`
8. Mirror the release debug store into the Offline Kit staging tree and re-check the manifest: 8. Mirror the release debug store into the Offline Kit staging tree and re-check the manifest:
```bash ```bash
./ops/offline-kit/mirror_debug_store.py \ ./ops/offline-kit/mirror_debug_store.py \
--release-dir out/release \ --release-dir out/release \
--offline-kit-dir out/offline-kit --offline-kit-dir out/offline-kit
jq '.artifacts | length' out/offline-kit/debug/debug-manifest.json jq '.artifacts | length' out/offline-kit/debug/debug-manifest.json
readelf -n /app/... | grep -i 'Build ID' readelf -n /app/... | grep -i 'Build ID'
``` ```
Validate that the hash from `readelf` matches the `.build-id/<aa>/<rest>.debug` path created by the script. Validate that the hash from `readelf` matches the `.build-id/<aa>/<rest>.debug` path created by the script.
9. Smoke-test OUK tarball in offline lab. 9. Smoke-test OUK tarball in offline lab.
10. Announce in `#stella-release` Mattermost channel. 10. Announce in `#stella-release` Mattermost channel.
--- ---
## 7 Hot‑fix Procedure ## 7 Hot‑fix Procedure
* Branch from latest tag → `hotfix/X.Y.Z+1-hf1`. * Branch from latest tag → `hotfix/X.Y.Z+1-hf1`.
* Apply minimal patch, add regression test. * Apply minimal patch, add regression test.
* CI pipeline (with reduced stages) must pass. * CI pipeline (with reduced stages) must pass.
* Tag `X.Y.Z+1`. * Tag `X.Y.Z+1`.
* Publish only container + Helm chart; OUK not rebuilt. * Publish only container + Helm chart; OUK not rebuilt.
* Cherry‑pick back to `main`. * Cherry‑pick back to `main`.
--- ---
## 8 Deprecation & End‑of‑Life Policy ## 8 Deprecation & End‑of‑Life Policy
| Feature | Deprecation notice | Removal earliest | | Feature | Deprecation notice | Removal earliest |
| ------------------------ | ------------------ | ---------------- | | ------------------------ | ------------------ | ---------------- |
| Legacy CSV policy import | 2025‑10‑01 | 2026‑04‑01 | | Legacy CSV policy import | 2025‑10‑01 | 2026‑04‑01 |
| Docker v1 Registry auth | 2025‑12‑01 | 2026‑06‑01 | | Docker v1 Registry auth | 2025‑12‑01 | 2026‑06‑01 |
| In‑image Trivy DB | 2025‑12‑15 | 2026‑03‑15 | | In‑image Trivy DB | 2025‑12‑15 | 2026‑03‑15 |
*At least 6 months notice; removal requires major version bump.* *At least 6 months notice; removal requires major version bump.*
--- ---
## 9 📌 Non‑Commercial Usage Rules (English canonical) ## 9 📌 Non‑Commercial Usage Rules (English canonical)
1. **Free for internal security assessments** (company or personal). 1. **Free for internal security assessments** (company or personal).
2. **SaaS resale / re‑hosting prohibited** without prior written consent (AGPL §13). 2. **SaaS resale / re‑hosting prohibited** without prior written consent (AGPL §13).
3. If you distribute a fork with UI or backend modifications **you must**: 3. If you distribute a fork with UI or backend modifications **you must**:
* Publish the complete modified source code. * Publish the complete modified source code.
* Retain the original Stella Ops attribution in UI footer and CLI `--version`. * Retain the original Stella Ops attribution in UI footer and CLI `--version`.
4. All third‑party dependencies remain under their respective licences (MIT, Apache‑2.0, ISC, BSD). 4. All third‑party dependencies remain under their respective licences (MIT, Apache‑2.0, ISC, BSD).
5. Deployments in state‑regulated or classified environments must obey**applicable local regulations** governing cryptography and software distribution. 5. Deployments in state‑regulated or classified environments must obey**applicable local regulations** governing cryptography and software distribution.
--- ---
## 10 Best Practices Snapshot 📌 ## 10 Best Practices Snapshot 📌
* **SBOM‑per‑image** → attach at build time; store as OCI artifact for supply‑chain introspection. * **SBOM‑per‑image** → attach at build time; store as OCI artifact for supply‑chain introspection.
* **Provenance flag** (`--provenance=true`) in BuildKit fulfils SLSA 2 requirement. * **Provenance flag** (`--provenance=true`) in BuildKit fulfils SLSA 2 requirement.
* Use **multi‑arch, reproducible builds** (`SOURCE_DATE_EPOCH` pins timestamps). * Use **multi‑arch, reproducible builds** (`SOURCE_DATE_EPOCH` pins timestamps).
* All pipelines enforce **Signed‑off‑by (DCO)**; CI fails if trailer missing. * All pipelines enforce **Signed‑off‑by (DCO)**; CI fails if trailer missing.
* `cosign policy` ensures only images signed by the project key run in production. * `cosign policy` ensures only images signed by the project key run in production.
--- ---
## 11 Contributing to Release Engineering ## 11 Contributing to Release Engineering
* Fork & create MR to `infra/release-*`. * Fork & create MR to `infra/release-*`.
* All infra changes require green **`integration-e2e-offline`** job. * All infra changes require green **`integration-e2e-offline`** job.
* Discuss larger infra migrations in `#sig-release` Mattermost; decisions recorded in `ADR/` folder. * Discuss larger infra migrations in `#sig-release` Mattermost; decisions recorded in `ADR/` folder.
--- ---
## 12 Change Log (high‑level) ## 12 Change Log (high‑level)
| Version | Date | Note | | Version | Date | Note |
| ------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | ------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| v2.1 | 2025‑07‑15 | Added OUK build/publish pipeline, internal registry image (`StellaOps.Registry`), non‑commercial usage rules extraction, SBOM stage, BuildKit provenance. | | v2.1 | 2025‑07‑15 | Added OUK build/publish pipeline, internal registry image (`StellaOps.Registry`), non‑commercial usage rules extraction, SBOM stage, BuildKit provenance. |
| v2.0 | 2025‑07‑12 | Initial open‑sourcing of Release Engineering guide. | | v2.0 | 2025‑07‑12 | Initial open‑sourcing of Release Engineering guide. |
| v1.1 | 2025‑07‑09 | Fixed inner fencing; added retention policy | | v1.1 | 2025‑07‑09 | Fixed inner fencing; added retention policy |
| v1.0 | 2025‑07‑09 | Initial playbook | | v1.0 | 2025‑07‑09 | Initial playbook |
--- ---
*(End of Release Engineering Playbook v1.1)* *(End of Release Engineering Playbook v1.1)*

404
docs/17_SECURITY_HARDENING_GUIDE.md
View File

@@ -1,202 +1,202 @@
#  17 · Security Hardening Guide — **Stella Ops** #  17 · Security Hardening Guide — **Stella Ops**
*(v2.0 — 12 Jul 2025)* *(v2.0 — 12 Jul 2025)*
> **Audience** — Site‑reliability and platform teams deploying **the open‑source Core** in production or restricted networks. > **Audience** — Site‑reliability and platform teams deploying **the open‑source Core** in production or restricted networks.
--- ---
##  0 Table of Contents ##  0 Table of Contents
1. Threat model (summary) 1. Threat model (summary)
2. Host‑OS baseline 2. Host‑OS baseline
3. Container & runtime hardening 3. Container & runtime hardening
4. Network‑plane guidance 4. Network‑plane guidance
5. Secrets & key management 5. Secrets & key management
6. Image, SBOM & plug‑in supply‑chain controls 6. Image, SBOM & plug‑in supply‑chain controls
7. Logging, monitoring & audit 7. Logging, monitoring & audit
8. Update & patch strategy 8. Update & patch strategy
9. Incident‑response workflow 9. Incident‑response workflow
10. Pen‑testing & continuous assurance 10. Pen‑testing & continuous assurance
11. Contacts & vulnerability disclosure 11. Contacts & vulnerability disclosure
12. Change log 12. Change log
--- ---
##  1 Threat model (summary) ##  1 Threat model (summary)
| Asset | Threats | Mitigations | | Asset | Threats | Mitigations |
| -------------------- | --------------------- | ---------------------------------------------------------------------- | | -------------------- | --------------------- | ---------------------------------------------------------------------- |
| SBOMs & scan results | Disclosure, tamper | TLS‑in‑transit, read‑only Redis volume, RBAC, Cosign‑verified plug‑ins | | SBOMs & scan results | Disclosure, tamper | TLS‑in‑transit, read‑only Redis volume, RBAC, Cosign‑verified plug‑ins |
| Backend container | RCE, code‑injection | Distroless image, non‑root UID, read‑only FS, seccomp + `CAP_DROP:ALL` | | Backend container | RCE, code‑injection | Distroless image, non‑root UID, read‑only FS, seccomp + `CAP_DROP:ALL` |
| Update artefacts | Supply‑chain attack | Cosign‑signed images & SBOMs, enforced by admission controller | | Update artefacts | Supply‑chain attack | Cosign‑signed images & SBOMs, enforced by admission controller |
| Admin credentials | Phishing, brute force | OAuth 2.0 with 12‑h token TTL, optional mTLS | | Admin credentials | Phishing, brute force | OAuth 2.0 with 12‑h token TTL, optional mTLS |
--- ---
##  2 Host‑OS baseline checklist ##  2 Host‑OS baseline checklist
| Item | Recommended setting | | Item | Recommended setting |
| ------------- | --------------------------------------------------------- | | ------------- | --------------------------------------------------------- |
| OS | Ubuntu 22.04 LTS (kernel ≥ 5.15) or Alma 9 | | OS | Ubuntu 22.04 LTS (kernel ≥ 5.15) or Alma 9 |
| Patches | `unattended‑upgrades` or vendor‑equivalent enabled | | Patches | `unattended‑upgrades` or vendor‑equivalent enabled |
| Filesystem | `noexec,nosuid` on `/tmp`, `/var/tmp` | | Filesystem | `noexec,nosuid` on `/tmp`, `/var/tmp` |
| Docker Engine | v24.*, API socket root‑owned (`0660`) | | Docker Engine | v24.*, API socket root‑owned (`0660`) |
| Auditd | Watch `/etc/docker`, `/usr/bin/docker*` and Compose files | | Auditd | Watch `/etc/docker`, `/usr/bin/docker*` and Compose files |
| Time sync | `chrony` or `systemd‑timesyncd` | | Time sync | `chrony` or `systemd‑timesyncd` |
--- ---
##  3 Container & runtime hardening ##  3 Container & runtime hardening
###  3.1 Docker Compose reference (`compose-core.yml`) ###  3.1 Docker Compose reference (`compose-core.yml`)
```yaml ```yaml
services: services:
backend: backend:
image: registry.stella-ops.org/stella-ops/stella-ops:<PINNED_TAG_OR_DIGEST> image: registry.stella-ops.org/stella-ops/stella-ops:<PINNED_TAG_OR_DIGEST>
user: "101:101" # non‑root user: "101:101" # non‑root
read_only: true read_only: true
security_opt: security_opt:
- "no-new-privileges:true" - "no-new-privileges:true"
- "seccomp:./seccomp-backend.json" - "seccomp:./seccomp-backend.json"
cap_drop: [ALL] cap_drop: [ALL]
tmpfs: tmpfs:
- /tmp:size=64m,exec,nosymlink - /tmp:size=64m,exec,nosymlink
environment: environment:
- ASPNETCORE_URLS=https://+:8080 - ASPNETCORE_URLS=https://+:8080
- TLSPROVIDER=OpenSslGost - TLSPROVIDER=OpenSslGost
depends_on: [redis] depends_on: [redis]
networks: [core-net] networks: [core-net]
healthcheck: healthcheck:
test: ["CMD", "wget", "-qO-", "https://localhost:8080/health"] test: ["CMD", "wget", "-qO-", "https://localhost:8080/health"]
interval: 30s interval: 30s
timeout: 5s timeout: 5s
retries: 5 retries: 5
redis: redis:
image: redis:7.2-alpine image: redis:7.2-alpine
command: ["redis-server", "--requirepass", "${REDIS_PASS}", "--rename-command", "FLUSHALL", ""] command: ["redis-server", "--requirepass", "${REDIS_PASS}", "--rename-command", "FLUSHALL", ""]
user: "redis" user: "redis"
read_only: true read_only: true
cap_drop: [ALL] cap_drop: [ALL]
tmpfs: tmpfs:
- /data - /data
networks: [core-net] networks: [core-net]
networks: networks:
core-net: core-net:
driver: bridge driver: bridge
``` ```
No dedicated “Redis” or “Mongo” sub‑nets are declared; the single bridge network suffices for the default stack. No dedicated “Redis” or “Mongo” sub‑nets are declared; the single bridge network suffices for the default stack.
###  3.2 Kubernetes deployment highlights ###  3.2 Kubernetes deployment highlights
Use a separate NetworkPolicy that only allows egress from backend to Redis :6379. Use a separate NetworkPolicy that only allows egress from backend to Redis :6379.
securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, drop all capabilities. securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, drop all capabilities.
PodDisruptionBudget of minAvailable: 1. PodDisruptionBudget of minAvailable: 1.
Optionally add CosignVerified=true label enforced by an admission controller (e.g. Kyverno or Connaisseur). Optionally add CosignVerified=true label enforced by an admission controller (e.g. Kyverno or Connaisseur).
##  4 Network‑plane guidance ##  4 Network‑plane guidance
| Plane | Recommendation | | Plane | Recommendation |
| ------------------ | -------------------------------------------------------------------------- | | ------------------ | -------------------------------------------------------------------------- |
| North‑south | Terminate TLS 1.2+ (OpenSSL‑GOST default). Use LetsEncrypt or internal CA. | | North‑south | Terminate TLS 1.2+ (OpenSSL‑GOST default). Use LetsEncrypt or internal CA. |
| East‑west | Compose bridge or K8s ClusterIP only; no public Redis/Mongo ports. | | East‑west | Compose bridge or K8s ClusterIP only; no public Redis/Mongo ports. |
| Ingress controller | Limit methods to GET, POST, PATCH (no TRACE). | | Ingress controller | Limit methods to GET, POST, PATCH (no TRACE). |
| Rate‑limits | 40 rps default; tune ScannerPool.Workers and ingress limit‑req to match. | | Rate‑limits | 40 rps default; tune ScannerPool.Workers and ingress limit‑req to match. |
##  5 Secrets & key management ##  5 Secrets & key management
| Secret | Storage | Rotation | | Secret | Storage | Rotation |
| --------------------------------- | ---------------------------------- | ----------------------------- | | --------------------------------- | ---------------------------------- | ----------------------------- |
| **Client‑JWT (offline)** | `/var/lib/stella/tokens/client.jwt` (root : 600) | **30 days** – provided by each OUK | | **Client‑JWT (offline)** | `/var/lib/stella/tokens/client.jwt` (root : 600) | **30 days** – provided by each OUK |
| REDIS_PASS | Docker/K8s secret | 90 days | | REDIS_PASS | Docker/K8s secret | 90 days |
| OAuth signing key | /keys/jwt.pem (read‑only mount) | 180 days | | OAuth signing key | /keys/jwt.pem (read‑only mount) | 180 days |
| Cosign public key | /keys/cosign.pub baked into image; | change on every major release | | Cosign public key | /keys/cosign.pub baked into image; | change on every major release |
| Trivy DB mirror token (if remote) | Secret + read‑only | 30 days | | Trivy DB mirror token (if remote) | Secret + read‑only | 30 days |
Never bake secrets into images; always inject at runtime. Never bake secrets into images; always inject at runtime.
> **Operational tip:** schedule a cron reminding ops 5 days before > **Operational tip:** schedule a cron reminding ops 5 days before
> `client.jwt` expiry. The backend also emits a Prometheus metric > `client.jwt` expiry. The backend also emits a Prometheus metric
> `stella_quota_token_days_remaining`. > `stella_quota_token_days_remaining`.
##  6 Image, SBOM & plug‑in supply‑chain controls ##  6 Image, SBOM & plug‑in supply‑chain controls
* Images — Pull by digest not latest; verify: * Images — Pull by digest not latest; verify:
```bash ```bash
cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \ cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
--key https://stella-ops.org/keys/cosign.pub --key https://stella-ops.org/keys/cosign.pub
``` ```
* SBOM — Each release ships an SPDX file; store alongside images for audit. * SBOM — Each release ships an SPDX file; store alongside images for audit.
* Third‑party plug‑ins — Place in /plugins/; backend will: * Third‑party plug‑ins — Place in /plugins/; backend will:
* Validate Cosign signature. * Validate Cosign signature.
* Check [StellaPluginVersion("major.minor")]. * Check [StellaPluginVersion("major.minor")].
* Refuse to start if Security.DisablePluginUnsigned=false (default). * Refuse to start if Security.DisablePluginUnsigned=false (default).
##  7 Logging, monitoring & audit ##  7 Logging, monitoring & audit
| Control | Implementation | | Control | Implementation |
| ------------ | ----------------------------------------------------------------- | | ------------ | ----------------------------------------------------------------- |
| Log format | Serilog JSON; ship via Fluent‑Bit to ELK or Loki | | Log format | Serilog JSON; ship via Fluent‑Bit to ELK or Loki |
| Metrics | Prometheus /metrics endpoint; default Grafana dashboard in infra/ | | Metrics | Prometheus /metrics endpoint; default Grafana dashboard in infra/ |
| Audit events | Redis stream audit; export daily to SIEM | | Audit events | Redis stream audit; export daily to SIEM |
| Alert rules | Feed age  ≥ 48 h, P95 wall‑time > 5 s, Redis used memory > 75 % | | Alert rules | Feed age  ≥ 48 h, P95 wall‑time > 5 s, Redis used memory > 75 % |
###  7.1 Concelier authorization audits ###  7.1 Concelier authorization audits
- Enable the Authority integration for Concelier (`authority.enabled=true`). Keep - Enable the Authority integration for Concelier (`authority.enabled=true`). Keep
`authority.allowAnonymousFallback` set to `true` only during migration and plan `authority.allowAnonymousFallback` set to `true` only during migration and plan
to disable it before **2025-12-31 UTC** so the `/jobs*` surface always demands to disable it before **2025-12-31 UTC** so the `/jobs*` surface always demands
a bearer token. a bearer token.
- Store the Authority client secret using Docker/Kubernetes secrets and point - Store the Authority client secret using Docker/Kubernetes secrets and point
`authority.clientSecretFile` at the mounted path; the value is read at startup `authority.clientSecretFile` at the mounted path; the value is read at startup
and never logged. and never logged.
- Watch the `Concelier.Authorization.Audit` logger. Each entry contains the HTTP - Watch the `Concelier.Authorization.Audit` logger. Each entry contains the HTTP
status, subject, client ID, scopes, remote IP, and a boolean `bypass` flag status, subject, client ID, scopes, remote IP, and a boolean `bypass` flag
showing whether a network bypass CIDR allowed the request. Configure your SIEM showing whether a network bypass CIDR allowed the request. Configure your SIEM
to alert when unauthenticated requests (`status=401`) appear with to alert when unauthenticated requests (`status=401`) appear with
`bypass=true`, or when unexpected scopes invoke job triggers. `bypass=true`, or when unexpected scopes invoke job triggers.
Detailed monitoring and response guidance lives in `docs/modules/concelier/operations/authority-audit-runbook.md`. Detailed monitoring and response guidance lives in `docs/modules/concelier/operations/authority-audit-runbook.md`.
##  8 Update & patch strategy ##  8 Update & patch strategy
| Layer | Cadence | Method | | Layer | Cadence | Method |
| -------------------- | -------------------------------------------------------- | ------------------------------ | | -------------------- | -------------------------------------------------------- | ------------------------------ |
| Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d | | Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d |
| Trivy DB | 24 h scheduler via Concelier (vulnerability ingest/merge/export service) | configurable via Concelier scheduler options | | Trivy DB | 24 h scheduler via Concelier (vulnerability ingest/merge/export service) | configurable via Concelier scheduler options |
| Docker Engine | vendor LTS | distro package manager | | Docker Engine | vendor LTS | distro package manager |
| Host OS | security repos enabled | unattended‑upgrades | | Host OS | security repos enabled | unattended‑upgrades |
##  9 Incident‑response workflow ##  9 Incident‑response workflow
* Detect — PagerDuty alert from Prometheus or SIEM. * Detect — PagerDuty alert from Prometheus or SIEM.
* Contain — Stop affected Backend container; isolate Redis RDB snapshot. * Contain — Stop affected Backend container; isolate Redis RDB snapshot.
* Eradicate — Pull verified images, redeploy, rotate secrets. * Eradicate — Pull verified images, redeploy, rotate secrets.
* Recover — Restore RDB, replay SBOMs if history lost. * Recover — Restore RDB, replay SBOMs if history lost.
* Review — Post‑mortem within 72 h; create follow‑up issues. * Review — Post‑mortem within 72 h; create follow‑up issues.
* Escalate P1 incidents to <security@stella‑ops.org> (24 × 7). * Escalate P1 incidents to <security@stella‑ops.org> (24 × 7).
##  10 Pen‑testing & continuous assurance ##  10 Pen‑testing & continuous assurance
| Control | Frequency | Tool/Runner | | Control | Frequency | Tool/Runner |
|----------------------|-----------------------|-------------------------------------------| |----------------------|-----------------------|-------------------------------------------|
| OWASP ZAP baseline | Each merge to `main` | GitHub Action `zap-baseline-scan` | | OWASP ZAP baseline | Each merge to `main` | GitHub Action `zap-baseline-scan` |
| Dependency scanning | Per pull request | Trivy FS + Dependabot | | Dependency scanning | Per pull request | Trivy FS + Dependabot |
| External red‑team | Annual or pre‑GA | CREST‑accredited third‑party | | External red‑team | Annual or pre‑GA | CREST‑accredited third‑party |
##  11 Vulnerability disclosure & contact ##  11 Vulnerability disclosure & contact
* Preferred channel: security@stella‑ops.org (GPG key on website). * Preferred channel: security@stella‑ops.org (GPG key on website).
* Coordinated disclosure reward: public credit and swag (no monetary bounty at this time). * Coordinated disclosure reward: public credit and swag (no monetary bounty at this time).
##  12 Change log ##  12 Change log
| Version | Date | Notes | | Version | Date | Notes |
| ------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- | | ------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
| v2.0 | 2025‑07‑12 | Full overhaul: host‑OS baseline, supply‑chain signing, removal of unnecessary sub‑nets, role‑based contact e‑mail, K8s guidance. | | v2.0 | 2025‑07‑12 | Full overhaul: host‑OS baseline, supply‑chain signing, removal of unnecessary sub‑nets, role‑based contact e‑mail, K8s guidance. |
| v1.1 | 2025‑07‑09 | Minor fence fixes. | | v1.1 | 2025‑07‑09 | Minor fence fixes. |
| v1.0 | 2025‑07‑09 | Original draft. | | v1.0 | 2025‑07‑09 | Original draft. |

380
docs/21_INSTALL_GUIDE.md
View File

@@ -1,190 +1,190 @@
# Stella Ops — Installation Guide (Docker & Air‑Gap) # Stella Ops — Installation Guide (Docker & Air‑Gap)
<!-- <!--
This file is processed by the Eleventy build. This file is processed by the Eleventy build.
Do **not** hard‑code versions or quota numbers; inherit from Do **not** hard‑code versions or quota numbers; inherit from
docs/_includes/CONSTANTS.md instead. docs/_includes/CONSTANTS.md instead.
{{ dotnet }}     → ".NET 10 LTS" {{ dotnet }}     → ".NET 10 LTS"
{{ angular }}    → "20" {{ angular }}    → "20"
--> -->
> **Status — public α not yet published.** > **Status — public α not yet published.**
> The commands below will work as soon as the first image is tagged > The commands below will work as soon as the first image is tagged
> `registry.stella-ops.org/stella-ops/stella-ops:0.1.0-alpha` > `registry.stella-ops.org/stella-ops/stella-ops:0.1.0-alpha`
> (target date: **late 2025**). Track progress on the > (target date: **late 2025**). Track progress on the
> [road‑map](/roadmap/). > [road‑map](/roadmap/).
--- ---
## 0 · Prerequisites ## 0 · Prerequisites
| Item | Minimum | Notes | | Item | Minimum | Notes |
|------|---------|-------| |------|---------|-------|
| Linux | Ubuntu 22.04 LTS / Alma 9 | x86‑64 or arm64 | | Linux | Ubuntu 22.04 LTS / Alma 9 | x86‑64 or arm64 |
| CPU / RAM | 2 vCPU / 2 GiB | Laptop baseline | | CPU / RAM | 2 vCPU / 2 GiB | Laptop baseline |
| Disk | 10 GiB SSD | SBOM + vuln DB cache | | Disk | 10 GiB SSD | SBOM + vuln DB cache |
| Docker | **Engine 25 + Compose v2** | `docker -v` | | Docker | **Engine 25 + Compose v2** | `docker -v` |
| TLS | OpenSSL 1.1 +  | Self‑signed cert generated at first run | | TLS | OpenSSL 1.1 +  | Self‑signed cert generated at first run |
--- ---
## 1 · Connected‑host install (Docker Compose) ## 1 · Connected‑host install (Docker Compose)
```bash ```bash
# 1. Make a working directory # 1. Make a working directory
mkdir stella && cd stella mkdir stella && cd stella
# 2. Download the signed Compose bundle + example .env # 2. Download the signed Compose bundle + example .env
curl -LO https://get.stella-ops.org/releases/latest/.env.example curl -LO https://get.stella-ops.org/releases/latest/.env.example
curl -LO https://get.stella-ops.org/releases/latest/.env.example.sig curl -LO https://get.stella-ops.org/releases/latest/.env.example.sig
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml.sig curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml.sig
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml.sig curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml.sig
# 3. Verify provenance (Cosign public key is stable) # 3. Verify provenance (Cosign public key is stable)
cosign verify-blob \ cosign verify-blob \
--key https://stella-ops.org/keys/cosign.pub \ --key https://stella-ops.org/keys/cosign.pub \
--signature .env.example.sig \ --signature .env.example.sig \
.env.example .env.example
cosign verify-blob \ cosign verify-blob \
--key https://stella-ops.org/keys/cosign.pub \ --key https://stella-ops.org/keys/cosign.pub \
--signature docker-compose.infrastructure.yml.sig \ --signature docker-compose.infrastructure.yml.sig \
docker-compose.infrastructure.yml docker-compose.infrastructure.yml
cosign verify-blob \ cosign verify-blob \
--key https://stella-ops.org/keys/cosign.pub \ --key https://stella-ops.org/keys/cosign.pub \
--signature docker-compose.stella-ops.yml.sig \ --signature docker-compose.stella-ops.yml.sig \
docker-compose.stella-ops.yml docker-compose.stella-ops.yml
# 4. Copy .env.example → .env and edit secrets # 4. Copy .env.example → .env and edit secrets
cp .env.example .env cp .env.example .env
$EDITOR .env $EDITOR .env
# 5. Launch databases (MongoDB + Redis) # 5. Launch databases (MongoDB + Redis)
docker compose --env-file .env -f docker-compose.infrastructure.yml up -d docker compose --env-file .env -f docker-compose.infrastructure.yml up -d
# 6. Launch Stella Ops (first run pulls ~50 MB merged vuln DB) # 6. Launch Stella Ops (first run pulls ~50 MB merged vuln DB)
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d docker compose --env-file .env -f docker-compose.stella-ops.yml up -d
```` ````
*Default login:* `admin / changeme` *Default login:* `admin / changeme`
UI: [https://\&lt;host\&gt;:8443](https://&lt;host&gt;:8443) (self‑signed certificate) UI: [https://\&lt;host\&gt;:8443](https://&lt;host&gt;:8443) (self‑signed certificate)
> **Pinning best‑practice** – in production environments replace > **Pinning best‑practice** – in production environments replace
> `stella-ops:latest` with the immutable digest printed by > `stella-ops:latest` with the immutable digest printed by
> `docker images --digests`. > `docker images --digests`.
> **Repo bundles** – Development, staging, and air‑gapped Compose profiles live > **Repo bundles** – Development, staging, and air‑gapped Compose profiles live
> under `deploy/compose/`, already tied to the release manifests in > under `deploy/compose/`, already tied to the release manifests in
> `deploy/releases/`. Helm users can pull the same channel overlays from > `deploy/releases/`. Helm users can pull the same channel overlays from
> `deploy/helm/stellaops/values-*.yaml` and validate everything with > `deploy/helm/stellaops/values-*.yaml` and validate everything with
> `deploy/tools/validate-profiles.sh`. > `deploy/tools/validate-profiles.sh`.
### 1.1 · Concelier authority configuration ### 1.1 · Concelier authority configuration
The Concelier container reads configuration from `etc/concelier.yaml` plus The Concelier container reads configuration from `etc/concelier.yaml` plus
`CONCELIER_` environment variables. To enable the new Authority integration: `CONCELIER_` environment variables. To enable the new Authority integration:
1. Add the following keys to `.env` (replace values for your environment): 1. Add the following keys to `.env` (replace values for your environment):
```bash ```bash
CONCELIER_AUTHORITY__ENABLED=true CONCELIER_AUTHORITY__ENABLED=true
CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true # temporary rollout only CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true # temporary rollout only
CONCELIER_AUTHORITY__ISSUER="https://authority.internal" CONCELIER_AUTHORITY__ISSUER="https://authority.internal"
CONCELIER_AUTHORITY__AUDIENCES__0="api://concelier" CONCELIER_AUTHORITY__AUDIENCES__0="api://concelier"
CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger" CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger"
CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read" CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read"
CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest" CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest"
CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default" CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default"
CONCELIER_AUTHORITY__CLIENTID="concelier-jobs" CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger" CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger"
CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read" CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read"
CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest" CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest"
CONCELIER_AUTHORITY__CLIENTSECRETFILE="/run/secrets/concelier_authority_client" CONCELIER_AUTHORITY__CLIENTSECRETFILE="/run/secrets/concelier_authority_client"
CONCELIER_AUTHORITY__BYPASSNETWORKS__0="127.0.0.1/32" CONCELIER_AUTHORITY__BYPASSNETWORKS__0="127.0.0.1/32"
CONCELIER_AUTHORITY__BYPASSNETWORKS__1="::1/128" CONCELIER_AUTHORITY__BYPASSNETWORKS__1="::1/128"
CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES=true CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES=true
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0="00:00:01" CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0="00:00:01"
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1="00:00:02" CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1="00:00:02"
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__2="00:00:05" CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__2="00:00:05"
CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK=true CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK=true
CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE="00:10:00" CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE="00:10:00"
``` ```
Store the client secret outside source control (Docker secrets, mounted file, Store the client secret outside source control (Docker secrets, mounted file,
or Kubernetes Secret). Concelier loads the secret during post-configuration, so or Kubernetes Secret). Concelier loads the secret during post-configuration, so
the value never needs to appear in the YAML template. the value never needs to appear in the YAML template.
Connected sites can keep the retry ladder short (1 s, 2 s, 5 s) so job triggers fail fast when Authority is down. For air‑gapped or intermittently connected deployments, extend `RESILIENCE__OFFLINECACHETOLERANCE` (e.g. `00:30:00`) so cached discovery/JWKS data remains valid while the Offline Kit synchronises upstream changes. Connected sites can keep the retry ladder short (1 s, 2 s, 5 s) so job triggers fail fast when Authority is down. For air‑gapped or intermittently connected deployments, extend `RESILIENCE__OFFLINECACHETOLERANCE` (e.g. `00:30:00`) so cached discovery/JWKS data remains valid while the Offline Kit synchronises upstream changes.
2. Redeploy Concelier: 2. Redeploy Concelier:
```bash ```bash
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d concelier docker compose --env-file .env -f docker-compose.stella-ops.yml up -d concelier
``` ```
3. Tail the logs: `docker compose logs -f concelier`. Successful `/jobs*` calls now 3. Tail the logs: `docker compose logs -f concelier`. Successful `/jobs*` calls now
emit `Concelier.Authorization.Audit` entries with `route`, `status`, `subject`, emit `Concelier.Authorization.Audit` entries with `route`, `status`, `subject`,
`clientId`, `scopes`, `bypass`, and `remote` fields. 401 denials keep the same `clientId`, `scopes`, `bypass`, and `remote` fields. 401 denials keep the same
shape—watch for `bypass=True`, which indicates a bypass CIDR accepted an anonymous shape—watch for `bypass=True`, which indicates a bypass CIDR accepted an anonymous
call. See `docs/modules/concelier/operations/authority-audit-runbook.md` for a full audit/alerting checklist. call. See `docs/modules/concelier/operations/authority-audit-runbook.md` for a full audit/alerting checklist.
> **Enforcement deadline** – keep `CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true` > **Enforcement deadline** – keep `CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true`
> only while validating the rollout. Set it to `false` (and restart Concelier) > only while validating the rollout. Set it to `false` (and restart Concelier)
> before **2025-12-31 UTC** to require tokens in production. > before **2025-12-31 UTC** to require tokens in production.
--- ---
## 2 · Optional: request a free quota token ## 2 · Optional: request a free quota token
Anonymous installs allow **{{ quota\_anon }} scans per UTC day**. Anonymous installs allow **{{ quota\_anon }} scans per UTC day**.
Email `token@stella-ops.org` to receive a signed JWT that raises the limit to Email `token@stella-ops.org` to receive a signed JWT that raises the limit to
**{{ quota\_token }} scans/day**. Insert it into `.env`: **{{ quota\_token }} scans/day**. Insert it into `.env`:
```bash ```bash
STELLA_JWT="paste‑token‑here" STELLA_JWT="paste‑token‑here"
docker compose --env-file .env -f docker-compose.stella-ops.yml \ docker compose --env-file .env -f docker-compose.stella-ops.yml \
exec stella-ops stella set-jwt "$STELLA_JWT" exec stella-ops stella set-jwt "$STELLA_JWT"
``` ```
>  The UI shows a reminder at 200 scans and throttles above the limit but will >  The UI shows a reminder at 200 scans and throttles above the limit but will
>  **never block** your pipeline. >  **never block** your pipeline.
--- ---
## 3 · Air‑gapped install (Offline Update Kit) ## 3 · Air‑gapped install (Offline Update Kit)
When running on an isolated network use the **Offline Update Kit (OUK)**: When running on an isolated network use the **Offline Update Kit (OUK)**:
```bash ```bash
# Download & verify on a connected host # Download & verify on a connected host
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz.sig curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz.sig
cosign verify-blob \ cosign verify-blob \
--key https://stella-ops.org/keys/cosign.pub \ --key https://stella-ops.org/keys/cosign.pub \
--signature stella-ops-offline-kit-v0.1a.tgz.sig \ --signature stella-ops-offline-kit-v0.1a.tgz.sig \
stella-ops-offline-kit-v0.1a.tgz stella-ops-offline-kit-v0.1a.tgz
# Transfer → air‑gap → import # Transfer → air‑gap → import
docker compose --env-file .env -f docker-compose.stella-ops.yml \ docker compose --env-file .env -f docker-compose.stella-ops.yml \
exec stella admin import-offline-usage-kit stella-ops-offline-kit-v0.1a.tgz exec stella admin import-offline-usage-kit stella-ops-offline-kit-v0.1a.tgz
``` ```
*Import is atomic; no service downtime.* *Import is atomic; no service downtime.*
For details see the dedicated [Offline Kit guide](/offline/). For details see the dedicated [Offline Kit guide](/offline/).
--- ---
## 4 · Next steps ## 4 · Next steps
* **5‑min Quick‑Start:** `/quickstart/` * **5‑min Quick‑Start:** `/quickstart/`
* **CI recipes:** `docs/ci/20_CI_RECIPES.md` * **CI recipes:** `docs/ci/20_CI_RECIPES.md`
* **Plug‑in SDK:** `/plugins/` * **Plug‑in SDK:** `/plugins/`
--- ---
*Generated {{ "now" | date: "%Y‑%m‑%d" }} — build tags inserted at render time.* *Generated {{ "now" | date: "%Y‑%m‑%d" }} — build tags inserted at render time.*

572
docs/24_OFFLINE_KIT.md
View File

@@ -1,286 +1,286 @@
# Offline Update Kit (OUK) — Air‑Gap Bundle # Offline Update Kit (OUK) — Air‑Gap Bundle
<!-- <!--
Build‑time variable injection: Build‑time variable injection:
{{ quota_anon }} = 33 {{ quota_anon }} = 33
{{ quota_token }} = 333 {{ quota_token }} = 333
{{ dotnet }} = "10 LTS" {{ dotnet }} = "10 LTS"
--> -->
The **Offline Update Kit** packages everything Stella Ops needs to run on a The **Offline Update Kit** packages everything Stella Ops needs to run on a
completely isolated network: completely isolated network:
| Component | Contents | | Component | Contents |
|-----------|----------| |-----------|----------|
| **Merged vulnerability feeds** | OSV, GHSA plus optional NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU | | **Merged vulnerability feeds** | OSV, GHSA plus optional NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU |
| **Container images** | `stella-ops`, *Zastava* sidecar (x86‑64 & arm64) | | **Container images** | `stella-ops`, *Zastava* sidecar (x86‑64 & arm64) |
| **Provenance** | Cosign signature, SPDX 2.3 SBOM, in‑toto SLSA attestation | | **Provenance** | Cosign signature, SPDX 2.3 SBOM, in‑toto SLSA attestation |
| **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. | | **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. |
| **Delta patches** | Daily diff bundles keep size \< 350 MB | | **Delta patches** | Daily diff bundles keep size \< 350 MB |
| **Scanner plug-ins** | OS analyzers plus the Node.js, Go, .NET, and Python language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. | | **Scanner plug-ins** | OS analyzers plus the Node.js, Go, .NET, and Python language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. |
| **Debug store** | `.debug` artefacts laid out under `debug/.build-id/<aa>/<rest>.debug` with `debug/debug-manifest.json` mapping build-ids to originating images for symbol retrieval. | | **Debug store** | `.debug` artefacts laid out under `debug/.build-id/<aa>/<rest>.debug` with `debug/debug-manifest.json` mapping build-ids to originating images for symbol retrieval. |
| **Telemetry collector bundle** | `telemetry/telemetry-offline-bundle.tar.gz` plus `.sha256`, containing OTLP collector config, Helm/Compose overlays, and operator instructions. | | **Telemetry collector bundle** | `telemetry/telemetry-offline-bundle.tar.gz` plus `.sha256`, containing OTLP collector config, Helm/Compose overlays, and operator instructions. |
**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache. **RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
**Language analyzers:** the kit now carries the restart-only Node.js, Go, .NET, and Python analyzer plug-ins (`plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Node/`, `...Lang.Go/`, `...Lang.DotNet/`, `...Lang.Python/`). Drop the directories alongside Worker binaries so the unified plug-in catalog can load them without outbound fetches; Rust remains on the Wave 4 roadmap. **Language analyzers:** the kit now carries the restart-only Node.js, Go, .NET, and Python analyzer plug-ins (`plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Node/`, `...Lang.Go/`, `...Lang.DotNet/`, `...Lang.Python/`). Drop the directories alongside Worker binaries so the unified plug-in catalog can load them without outbound fetches; Rust remains on the Wave 4 roadmap.
*Scanner core:* C# 12 on **.NET {{ dotnet }}**. *Scanner core:* C# 12 on **.NET {{ dotnet }}**.
*Imports are idempotent and atomic — no service downtime.* *Imports are idempotent and atomic — no service downtime.*
## 0 · Prepare the debug store ## 0 · Prepare the debug store
Before packaging the Offline Kit, mirror the release debug artefacts (GNU build-id `.debug` files and the associated manifest) into the staging directory: Before packaging the Offline Kit, mirror the release debug artefacts (GNU build-id `.debug` files and the associated manifest) into the staging directory:
```bash ```bash
./ops/offline-kit/mirror_debug_store.py \ ./ops/offline-kit/mirror_debug_store.py \
--release-dir out/release \ --release-dir out/release \
--offline-kit-dir out/offline-kit --offline-kit-dir out/offline-kit
``` ```
The helper copies `debug/.build-id/**`, validates `debug/debug-manifest.json` against its recorded SHA-256, and writes `out/offline-kit/metadata/debug-store.json` with a short summary (platforms, artefact counts, sample build-ids). The command exits non-zero if an artefact referenced by the manifest is missing or has the wrong digest, so run it as part of every kit build. The helper copies `debug/.build-id/**`, validates `debug/debug-manifest.json` against its recorded SHA-256, and writes `out/offline-kit/metadata/debug-store.json` with a short summary (platforms, artefact counts, sample build-ids). The command exits non-zero if an artefact referenced by the manifest is missing or has the wrong digest, so run it as part of every kit build.
--- ---
## 0.1 · Automated packaging ## 0.1 · Automated packaging
The packaging workflow is scripted via `ops/offline-kit/build_offline_kit.py`. The packaging workflow is scripted via `ops/offline-kit/build_offline_kit.py`.
It verifies the release artefacts, runs the Python analyzer smoke suite, mirrors the debug store, and emits a deterministic tarball + manifest set. It verifies the release artefacts, runs the Python analyzer smoke suite, mirrors the debug store, and emits a deterministic tarball + manifest set.
```bash ```bash
python ops/offline-kit/build_offline_kit.py \ python ops/offline-kit/build_offline_kit.py \
--version 2025.10.0 \ --version 2025.10.0 \
--channel edge \ --channel edge \
--release-dir out/release \ --release-dir out/release \
--staging-dir out/offline-kit/staging \ --staging-dir out/offline-kit/staging \
--output-dir out/offline-kit/dist --output-dir out/offline-kit/dist
# Optional: regenerate the telemetry collector bundle prior to packaging. # Optional: regenerate the telemetry collector bundle prior to packaging.
python ops/devops/telemetry/package_offline_bundle.py --output out/telemetry/telemetry-offline-bundle.tar.gz python ops/devops/telemetry/package_offline_bundle.py --output out/telemetry/telemetry-offline-bundle.tar.gz
``` ```
Outputs: Outputs:
- `stella-ops-offline-kit-<version>-<channel>.tar.gz` — bundle (mtime/uid/gid forced to zero for reproducibility) - `stella-ops-offline-kit-<version>-<channel>.tar.gz` — bundle (mtime/uid/gid forced to zero for reproducibility)
- `stella-ops-offline-kit-<version>-<channel>.tar.gz.sha256` — bundle digest - `stella-ops-offline-kit-<version>-<channel>.tar.gz.sha256` — bundle digest
- `manifest/offline-manifest.json` + `.sha256` — inventories every file in the bundle - `manifest/offline-manifest.json` + `.sha256` — inventories every file in the bundle
- `<bundle>.metadata.json` — descriptor consumed by the CLI/Console import tooling - `<bundle>.metadata.json` — descriptor consumed by the CLI/Console import tooling
- `telemetry/telemetry-offline-bundle.tar.gz` + `.sha256` — packaged OTLP collector assets for environments without upstream access - `telemetry/telemetry-offline-bundle.tar.gz` + `.sha256` — packaged OTLP collector assets for environments without upstream access
- `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*.sig` (+ `.sha256`) — Cosign signatures for the Python analyzer DLL and manifest - `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*.sig` (+ `.sha256`) — Cosign signatures for the Python analyzer DLL and manifest
### Policy Gateway configuration bundle ### Policy Gateway configuration bundle
- Copy `etc/policy-gateway.yaml` (or the `*.sample` template if you expect operators to override values) into `config/policy-gateway/policy-gateway.yaml` within the staging tree. - Copy `etc/policy-gateway.yaml` (or the `*.sample` template if you expect operators to override values) into `config/policy-gateway/policy-gateway.yaml` within the staging tree.
- Include the gateway DPoP private key under `secrets/policy-gateway/policy-gateway-dpop.pem` and reference the location inside the manifest notes. Set the permissions explicitly (`chmod 600 secrets/policy-gateway/policy-gateway-dpop.pem`) so only the kit importer can read it; the importer will refuse keys that are broader. - Include the gateway DPoP private key under `secrets/policy-gateway/policy-gateway-dpop.pem` and reference the location inside the manifest notes. Set the permissions explicitly (`chmod 600 secrets/policy-gateway/policy-gateway-dpop.pem`) so only the kit importer can read it; the importer will refuse keys that are broader.
- Document the gateway base URL and activation verification steps in `docs/policy/gateway.md` (bundled alongside the kit). Operators can use those curl snippets to smoke-test pack CRUD once the Offline Kit is imported. - Document the gateway base URL and activation verification steps in `docs/policy/gateway.md` (bundled alongside the kit). Operators can use those curl snippets to smoke-test pack CRUD once the Offline Kit is imported.
- Ensure the Prometheus snapshot captured during packaging contains `policy_gateway_activation_requests_total` so auditors can reconcile activation attempts performed via the gateway during the validation window. - Ensure the Prometheus snapshot captured during packaging contains `policy_gateway_activation_requests_total` so auditors can reconcile activation attempts performed via the gateway during the validation window.
Provide `--cosign-key` / `--cosign-identity-token` (and optional `--cosign-password`) to generate Cosign signatures for both the tarball and manifest. Provide `--cosign-key` / `--cosign-identity-token` (and optional `--cosign-password`) to generate Cosign signatures for both the tarball and manifest.
--- ---
## 1 · Download & verify ## 1 · Download & verify
```bash ```bash
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz.sig curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz.sig
curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json
curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json.jws curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json.jws
cosign verify-blob \ cosign verify-blob \
--key https://stella-ops.org/keys/cosign.pub \ --key https://stella-ops.org/keys/cosign.pub \
--signature stella-ops-offline-kit-<DATE>.tgz.sig \ --signature stella-ops-offline-kit-<DATE>.tgz.sig \
stella-ops-offline-kit-<DATE>.tgz stella-ops-offline-kit-<DATE>.tgz
```` ````
**CLI shortcut.** `stellaops-cli offline kit pull --destination ./offline-kit` downloads the bundle, manifest, and detached signatures in one step, resumes partial transfers, and writes a `.metadata.json` summary for later import. **CLI shortcut.** `stellaops-cli offline kit pull --destination ./offline-kit` downloads the bundle, manifest, and detached signatures in one step, resumes partial transfers, and writes a `.metadata.json` summary for later import.
Verification prints **OK** and the SHA‑256 digest; cross‑check against the Verification prints **OK** and the SHA‑256 digest; cross‑check against the
[changelog](https://git.stella-ops.org/stella-ops/offline-kit/-/releases). [changelog](https://git.stella-ops.org/stella-ops/offline-kit/-/releases).
Validate the attested manifest before distribution: Validate the attested manifest before distribution:
```bash ```bash
cosign verify-blob \ cosign verify-blob \
--key https://stella-ops.org/keys/cosign.pub \ --key https://stella-ops.org/keys/cosign.pub \
--signature offline-manifest-<DATE>.json.jws \ --signature offline-manifest-<DATE>.json.jws \
offline-manifest-<DATE>.json offline-manifest-<DATE>.json
jq '.artifacts[] | {name, sha256, size, capturedAt}' offline-manifest-<DATE>.json jq '.artifacts[] | {name, sha256, size, capturedAt}' offline-manifest-<DATE>.json
``` ```
The manifest enumerates every artefact (`name`, `sha256`, `size`, `capturedAt`) and is signed with the same key registry as Authority revocation bundles. Operators can ship the manifest alongside the tarball so downstream mirrors can re-verify without unpacking the kit. The manifest enumerates every artefact (`name`, `sha256`, `size`, `capturedAt`) and is signed with the same key registry as Authority revocation bundles. Operators can ship the manifest alongside the tarball so downstream mirrors can re-verify without unpacking the kit.
Example excerpt (2025-10-23 kit) showing the Go and .NET analyzer plug-in payloads: Example excerpt (2025-10-23 kit) showing the Go and .NET analyzer plug-in payloads:
```json ```json
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.dll", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.dll",
"sha256": "a6dc850fc51151c8967ef46a3c4730f08b549667e041079431f39a8a72d0b641", "sha256": "a6dc850fc51151c8967ef46a3c4730f08b549667e041079431f39a8a72d0b641",
"size": 33792, "size": 33792,
"capturedAt": "2025-10-23T00:00:00Z" "capturedAt": "2025-10-23T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.pdb", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.pdb",
"sha256": "6cbdabf155282f458b89edf267e7f6bb2441a93029aad7aad45c8a9ec58b1b3b", "sha256": "6cbdabf155282f458b89edf267e7f6bb2441a93029aad7aad45c8a9ec58b1b3b",
"size": 32152, "size": 32152,
"capturedAt": "2025-10-23T00:00:00Z" "capturedAt": "2025-10-23T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/manifest.json", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/manifest.json",
"sha256": "c19bfca2fcbb7cb18f1082b5d0d5a8f15fc799c648b50e95fce8d8b109ce48c9", "sha256": "c19bfca2fcbb7cb18f1082b5d0d5a8f15fc799c648b50e95fce8d8b109ce48c9",
"size": 622, "size": 622,
"capturedAt": "2025-10-23T00:00:00Z" "capturedAt": "2025-10-23T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.dll", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.dll",
"sha256": "0734d23e33277ce2ccb596782d2d42cfe394b3d372dc34da9cb28b59df9b9d22", "sha256": "0734d23e33277ce2ccb596782d2d42cfe394b3d372dc34da9cb28b59df9b9d22",
"size": 70144, "size": 70144,
"capturedAt": "2025-10-23T00:00:00Z" "capturedAt": "2025-10-23T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.pdb", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.pdb",
"sha256": "b853c1ff4b196715f5bd1447e1a13edeb4940917527ec9bf153b5048da49abaf", "sha256": "b853c1ff4b196715f5bd1447e1a13edeb4940917527ec9bf153b5048da49abaf",
"size": 40400, "size": 40400,
"capturedAt": "2025-10-23T00:00:00Z" "capturedAt": "2025-10-23T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/manifest.json", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/manifest.json",
"sha256": "5d483885f825f01bfd9943dcf2889ec2e0beba38ede92ecfe67d4f506cf14e37", "sha256": "5d483885f825f01bfd9943dcf2889ec2e0beba38ede92ecfe67d4f506cf14e37",
"size": 647, "size": 647,
"capturedAt": "2025-10-23T00:00:00Z" "capturedAt": "2025-10-23T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.dll", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.dll",
"sha256": "a4f558f363394096e3dd6263f35b180b93b4112f9cf616c05872da8a8657d518", "sha256": "a4f558f363394096e3dd6263f35b180b93b4112f9cf616c05872da8a8657d518",
"size": 47104, "size": 47104,
"capturedAt": "2025-10-26T00:00:00Z" "capturedAt": "2025-10-26T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.pdb", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.pdb",
"sha256": "ef2ad78bc2cd1d7e99bae000b92357aa9a9c32938501899e9033d001096196d0", "sha256": "ef2ad78bc2cd1d7e99bae000b92357aa9a9c32938501899e9033d001096196d0",
"size": 31896, "size": 31896,
"capturedAt": "2025-10-26T00:00:00Z" "capturedAt": "2025-10-26T00:00:00Z"
} }
{ {
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/manifest.json", "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/manifest.json",
"sha256": "668ad9a1a35485628677b639db4d996d1e25f62021680a81a22482483800e557", "sha256": "668ad9a1a35485628677b639db4d996d1e25f62021680a81a22482483800e557",
"size": 648, "size": 648,
"capturedAt": "2025-10-26T00:00:00Z" "capturedAt": "2025-10-26T00:00:00Z"
} }
``` ```
--- ---
## 2 · Import on the air‑gapped host ## 2 · Import on the air‑gapped host
```bash ```bash
docker compose --env-file .env \ docker compose --env-file .env \
-f docker-compose.stella-ops.yml \ -f docker-compose.stella-ops.yml \
exec stella-ops \ exec stella-ops \
stella admin import-offline-usage-kit stella-ops-offline-kit-<DATE>.tgz stella admin import-offline-usage-kit stella-ops-offline-kit-<DATE>.tgz
``` ```
Alternatively, run Alternatively, run
```bash ```bash
stellaops-cli offline kit import stella-ops-offline-kit-<DATE>.tgz \ stellaops-cli offline kit import stella-ops-offline-kit-<DATE>.tgz \
--manifest offline-manifest-<DATE>.json \ --manifest offline-manifest-<DATE>.json \
--bundle-signature stella-ops-offline-kit-<DATE>.tgz.sig \ --bundle-signature stella-ops-offline-kit-<DATE>.tgz.sig \
--manifest-signature offline-manifest-<DATE>.json.jws --manifest-signature offline-manifest-<DATE>.json.jws
``` ```
The CLI validates recorded digests (when `.metadata.json` is present) before streaming the multipart payload to `/api/offline-kit/import`. The CLI validates recorded digests (when `.metadata.json` is present) before streaming the multipart payload to `/api/offline-kit/import`.
* The CLI validates the Cosign signature **before** activation. * The CLI validates the Cosign signature **before** activation.
* Old feeds are kept until the new bundle is fully verified. * Old feeds are kept until the new bundle is fully verified.
* Import time on a SATA SSD: ≈ 25 s for a 300 MB kit. * Import time on a SATA SSD: ≈ 25 s for a 300 MB kit.
### 2.1 Validator + idempotency enablement (air-gap) ### 2.1 Validator + idempotency enablement (air-gap)
The Offline Kit carries the same helper scripts under `scripts/`: The Offline Kit carries the same helper scripts under `scripts/`:
1. **Duplicate audit:** run 1. **Duplicate audit:** run
```bash ```bash
mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;' mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'
``` ```
to verify no `(vendor, upstream_id, content_hash, tenant)` conflicts remain before enabling the idempotency index. to verify no `(vendor, upstream_id, content_hash, tenant)` conflicts remain before enabling the idempotency index.
2. **Apply validators:** execute `mongo concelier ops/devops/scripts/apply-aoc-validators.js` (and the Excititor equivalent) with `validationLevel: "moderate"` in maintenance mode. 2. **Apply validators:** execute `mongo concelier ops/devops/scripts/apply-aoc-validators.js` (and the Excititor equivalent) with `validationLevel: "moderate"` in maintenance mode.
3. **Restart Concelier** so migrations `20251028_advisory_raw_idempotency_index` and `20251028_advisory_supersedes_backfill` run automatically. After the restart: 3. **Restart Concelier** so migrations `20251028_advisory_raw_idempotency_index` and `20251028_advisory_supersedes_backfill` run automatically. After the restart:
- Confirm `db.advisory` resolves to a view on `advisory_backup_20251028`. - Confirm `db.advisory` resolves to a view on `advisory_backup_20251028`.
- Spot-check a few `advisory_raw` entries to ensure `supersedes` chains are populated deterministically. - Spot-check a few `advisory_raw` entries to ensure `supersedes` chains are populated deterministically.
4. **Smoke test:** run `stella sources ingest --dry-run --fixture advisory` (bundled fixtures) to confirm ingestion succeeds post-guard and the CLI reports zero violations. 4. **Smoke test:** run `stella sources ingest --dry-run --fixture advisory` (bundled fixtures) to confirm ingestion succeeds post-guard and the CLI reports zero violations.
### Authority scope sanity check ### Authority scope sanity check
Offline installs rely on the bundled `etc/authority.yaml.sample`. Before promoting the kit, confirm the sample clients keep the Aggregation-Only guardrails: Offline installs rely on the bundled `etc/authority.yaml.sample`. Before promoting the kit, confirm the sample clients keep the Aggregation-Only guardrails:
- `aoc-verifier` requests `aoc:verify`, `advisory:read`, and `vex:read`. - `aoc-verifier` requests `aoc:verify`, `advisory:read`, and `vex:read`.
- `signals-uploader` requests `signals:write`, `signals:read`, and `aoc:verify`. - `signals-uploader` requests `signals:write`, `signals:read`, and `aoc:verify`.
Authority now rejects tokens that request `advisory:read`, `vex:read`, or any `signals:*` scope without `aoc:verify`; the sample has been updated to match. If you maintain tenant-specific overlays, mirror the same pairing so air-gapped automation fails deterministically with `invalid_scope` when misconfigured. Authority now rejects tokens that request `advisory:read`, `vex:read`, or any `signals:*` scope without `aoc:verify`; the sample has been updated to match. If you maintain tenant-specific overlays, mirror the same pairing so air-gapped automation fails deterministically with `invalid_scope` when misconfigured.
**Quick smoke test:** before import, verify the tarball carries the Go analyzer plug-in: **Quick smoke test:** before import, verify the tarball carries the Go analyzer plug-in:
```bash ```bash
tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*' tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*'
``` ```
The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted. The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted.
> **Release guardrail.** The automated release pipeline now publishes the Python plug-in from source and executes `dotnet run --project src/Tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle. > **Release guardrail.** The automated release pipeline now publishes the Python plug-in from source and executes `dotnet run --project src/Tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle.
### Debug store mirror ### Debug store mirror
Offline symbols (`debug/.build-id/**`) must accompany every Offline Kit to keep symbol lookup deterministic. The release workflow is expected to emit `out/release/debug/` containing the build-id tree plus `debug-manifest.json` and its `.sha256` companion. After a release completes: Offline symbols (`debug/.build-id/**`) must accompany every Offline Kit to keep symbol lookup deterministic. The release workflow is expected to emit `out/release/debug/` containing the build-id tree plus `debug-manifest.json` and its `.sha256` companion. After a release completes:
```bash ```bash
python ops/offline-kit/mirror_debug_store.py \ python ops/offline-kit/mirror_debug_store.py \
--release-dir out/release \ --release-dir out/release \
--offline-dir out/offline-kit \ --offline-dir out/offline-kit \
--summary out/offline-kit/metadata/debug-store.json --summary out/offline-kit/metadata/debug-store.json
``` ```
The script mirrors the debug tree into the Offline Kit staging directory, verifies SHA-256 values against the manifest, and writes a summary under `metadata/debug-store.json` for audit logs. If the release pipeline does not populate `out/release/debug`, the tooling now logs a warning (`DEVOPS-REL-17-004`)—treat it as a build failure and re-run the release once symbol extraction is enabled. The script mirrors the debug tree into the Offline Kit staging directory, verifies SHA-256 values against the manifest, and writes a summary under `metadata/debug-store.json` for audit logs. If the release pipeline does not populate `out/release/debug`, the tooling now logs a warning (`DEVOPS-REL-17-004`)—treat it as a build failure and re-run the release once symbol extraction is enabled.
--- ---
## 3 · Delta patch workflow ## 3 · Delta patch workflow
1. **Connected site** fetches `stella-ouk-YYYY‑MM‑DD.delta.tgz`. 1. **Connected site** fetches `stella-ouk-YYYY‑MM‑DD.delta.tgz`.
2. Transfer via any medium (USB, portable disk). 2. Transfer via any medium (USB, portable disk).
3. `stella admin import-offline-usage-kit <delta>` applies only changed CVE rows & images. 3. `stella admin import-offline-usage-kit <delta>` applies only changed CVE rows & images.
Daily deltas are **< 30 MB**; weekly roll‑up produces a fresh full kit. Daily deltas are **< 30 MB**; weekly roll‑up produces a fresh full kit.
--- ---
## 4 · Quota behaviour offline ## 4 · Quota behaviour offline
The scanner enforces the same fair‑use limits offline: The scanner enforces the same fair‑use limits offline:
* **Anonymous:** {{ quota\_anon }} scans per UTC day * **Anonymous:** {{ quota\_anon }} scans per UTC day
* **Free JWT:** {{ quota\_token }} scans per UTC day * **Free JWT:** {{ quota\_token }} scans per UTC day
Soft reminder at 200 scans; throttle above the ceiling but **never block**. Soft reminder at 200 scans; throttle above the ceiling but **never block**.
See the detailed rules in See the detailed rules in
[`33_333_QUOTA_OVERVIEW.md`](33_333_QUOTA_OVERVIEW.md). [`33_333_QUOTA_OVERVIEW.md`](33_333_QUOTA_OVERVIEW.md).
--- ---
## 5 · Troubleshooting ## 5 · Troubleshooting
| Symptom | Explanation | Fix | | Symptom | Explanation | Fix |
| -------------------------------------- | ---------------------------------------- | ------------------------------------- | | -------------------------------------- | ---------------------------------------- | ------------------------------------- |
| `could not verify SBOM hash` | Bundle corrupted in transit | Re‑download / re‑copy | | `could not verify SBOM hash` | Bundle corrupted in transit | Re‑download / re‑copy |
| Import hangs at `Applying feeds…` | Low disk space in `/var/lib/stella` | Free ≥ 2 GiB before retry | | Import hangs at `Applying feeds…` | Low disk space in `/var/lib/stella` | Free ≥ 2 GiB before retry |
| `quota exceeded` same day after import | Import resets counters at UTC 00:00 only | Wait until next UTC day or load a JWT | | `quota exceeded` same day after import | Import resets counters at UTC 00:00 only | Wait until next UTC day or load a JWT |
--- ---
## 6 · Related documentation ## 6 · Related documentation
* **Install guide:** `/install/#air-gapped` * **Install guide:** `/install/#air-gapped`
* **Sovereign mode rationale:** `/sovereign/` * **Sovereign mode rationale:** `/sovereign/`
* **Security policy:** `/security/#reporting-a-vulnerability` * **Security policy:** `/security/#reporting-a-vulnerability`
* **CERT-Bund snapshots:** `python src/Tools/certbund_offline_snapshot.py --help` (see `docs/modules/concelier/operations/connectors/certbund.md`) * **CERT-Bund snapshots:** `python src/Tools/certbund_offline_snapshot.py --help` (see `docs/modules/concelier/operations/connectors/certbund.md`)

40
docs/AGENTS.md
View File

@@ -1,20 +1,20 @@
# Docs & Enablement Guild # Docs & Enablement Guild
## Mission ## Mission
Produce and maintain offline-friendly documentation for StellaOps modules, covering architecture, configuration, operator workflows, and developer onboarding. Produce and maintain offline-friendly documentation for StellaOps modules, covering architecture, configuration, operator workflows, and developer onboarding.
## Scope Highlights ## Scope Highlights
- Authority docs (`docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, upcoming `docs/11_AUTHORITY.md`). - Authority docs (`docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, upcoming `docs/11_AUTHORITY.md`).
- Concelier quickstarts, CLI guides, Offline Kit manuals. - Concelier quickstarts, CLI guides, Offline Kit manuals.
- Release notes and migration playbooks. - Release notes and migration playbooks.
## Operating Principles ## Operating Principles
- Keep guides deterministic and in sync with shipped configuration samples. - Keep guides deterministic and in sync with shipped configuration samples.
- Prefer tables/checklists for operator steps; flag security-sensitive actions. - Prefer tables/checklists for operator steps; flag security-sensitive actions.
- When work involves a specific `StellaOps.<Component>` project, consult both `docs/07_HIGH_LEVEL_ARCHITECTURE.md` and the matching dossier `docs/modules/<component>/architecture.md` before drafting or editing content. - When work involves a specific `StellaOps.<Component>` project, consult both `docs/07_HIGH_LEVEL_ARCHITECTURE.md` and the matching dossier `docs/modules/<component>/architecture.md` before drafting or editing content.
- Update `docs/TASKS.md` whenever work items change status (TODO/DOING/REVIEW/DONE/BLOCKED). - Update `docs/TASKS.md` whenever work items change status (TODO/DOING/REVIEW/DONE/BLOCKED).
## Coordination ## Coordination
- Authority Core & Plugin teams for auth-related changes. - Authority Core & Plugin teams for auth-related changes.
- Security Guild for threat-model outputs and mitigations. - Security Guild for threat-model outputs and mitigations.
- DevEx for tooling diagrams and documentation pipeline. - DevEx for tooling diagrams and documentation pipeline.

262
docs/accessibility.md
View File

@@ -1,131 +1,131 @@
# StellaOps Console Accessibility Guide # StellaOps Console Accessibility Guide
> **Audience:** Accessibility Guild, Console Guild, Docs Guild, QA. > **Audience:** Accessibility Guild, Console Guild, Docs Guild, QA.
> **Scope:** Keyboard interaction model, screen-reader behaviour, colour & focus tokens, testing workflows, offline considerations, and compliance checklist for the StellaOps Console (Sprint 23). > **Scope:** Keyboard interaction model, screen-reader behaviour, colour & focus tokens, testing workflows, offline considerations, and compliance checklist for the StellaOps Console (Sprint 23).
The console targets **WCAG 2.2 AA** across all supported browsers (Chromium, Firefox ESR) and honours StellaOps’ sovereign/offline constraints. Every build must keep keyboard-only users, screen-reader users, and high-contrast operators productive without relying on third-party services. The console targets **WCAG 2.2 AA** across all supported browsers (Chromium, Firefox ESR) and honours StellaOps’ sovereign/offline constraints. Every build must keep keyboard-only users, screen-reader users, and high-contrast operators productive without relying on third-party services.
--- ---
## 1 · Accessibility Principles ## 1 · Accessibility Principles
1. **Deterministic navigation** – Focus order, shortcuts, and announcements remain stable across releases; URLs encode state for deep links. 1. **Deterministic navigation** – Focus order, shortcuts, and announcements remain stable across releases; URLs encode state for deep links.
2. **Keyboard-first design** – Every actionable element is reachable via keyboard; shortcuts provide accelerators, and remapping is available via *Settings → Accessibility → Keyboard shortcuts*. 2. **Keyboard-first design** – Every actionable element is reachable via keyboard; shortcuts provide accelerators, and remapping is available via *Settings → Accessibility → Keyboard shortcuts*.
3. **Assistive technology parity** – ARIA roles and live regions mirror visual affordances (status banners, SSE tickers, progress drawers). Screen readers receive polite/atomic updates to avoid chatter. 3. **Assistive technology parity** – ARIA roles and live regions mirror visual affordances (status banners, SSE tickers, progress drawers). Screen readers receive polite/atomic updates to avoid chatter.
4. **Colour & contrast tokens** – All palettes derive from design tokens that achieve ≥ 4.5:1 contrast (text) and ≥ 3:1 for graphical indicators; tokens pass automated contrast linting. 4. **Colour & contrast tokens** – All palettes derive from design tokens that achieve ≥ 4.5:1 contrast (text) and ≥ 3:1 for graphical indicators; tokens pass automated contrast linting.
5. **Offline equivalence** – Accessibility features (shortcuts, offline banners, focus restoration) behave the same in sealed environments, with guidance when actions require online authority. 5. **Offline equivalence** – Accessibility features (shortcuts, offline banners, focus restoration) behave the same in sealed environments, with guidance when actions require online authority.
--- ---
## 2 · Keyboard Interaction Map ## 2 · Keyboard Interaction Map
### 2.1 Global shortcuts ### 2.1 Global shortcuts
| Action | Macs | Windows/Linux | Notes | | Action | Macs | Windows/Linux | Notes |
|--------|------|---------------|-------| |--------|------|---------------|-------|
| Command palette | `⌘ K` | `Ctrl K` | Focuses palette search; respects tenant scope. | | Command palette | `⌘ K` | `Ctrl K` | Focuses palette search; respects tenant scope. |
| Tenant picker | `⌘ T` | `Ctrl T` | Opens modal; `Enter` confirms, `Esc` cancels. | | Tenant picker | `⌘ T` | `Ctrl T` | Opens modal; `Enter` confirms, `Esc` cancels. |
| Filter tray toggle | `⇧ F` | `Shift F` | Focus lands on first filter; `Tab` cycles filters before returning to page. | | Filter tray toggle | `⇧ F` | `Shift F` | Focus lands on first filter; `Tab` cycles filters before returning to page. |
| Saved view presets | `⌘ 1-9` | `Ctrl 1-9` | Bound per tenant; missing preset triggers tooltip. | | Saved view presets | `⌘ 1-9` | `Ctrl 1-9` | Bound per tenant; missing preset triggers tooltip. |
| Keyboard reference | `?` | `?` | Opens overlay listing context-specific shortcuts; `Esc` closes. | | Keyboard reference | `?` | `?` | Opens overlay listing context-specific shortcuts; `Esc` closes. |
| Global search (context) | `/` | `/` | When the filter tray is closed, focuses inline search field. | | Global search (context) | `/` | `/` | When the filter tray is closed, focuses inline search field. |
### 2.2 Module-specific shortcuts ### 2.2 Module-specific shortcuts
| Module | Action | Macs | Windows/Linux | Notes | | Module | Action | Macs | Windows/Linux | Notes |
|--------|--------|------|---------------|-------| |--------|--------|------|---------------|-------|
| Findings | Explain search | `⌘ /` | `Ctrl /` | Only when Explain drawer open; announces results via live region. | | Findings | Explain search | `⌘ /` | `Ctrl /` | Only when Explain drawer open; announces results via live region. |
| SBOM Explorer | Toggle overlays | `⌘ G` | `Ctrl G` | Persists per session (see `/docs/ui/sbom-explorer.md`). | | SBOM Explorer | Toggle overlays | `⌘ G` | `Ctrl G` | Persists per session (see `/docs/ui/sbom-explorer.md`). |
| Advisories & VEX | Provider filter | `⌘ ⌥ F` | `Ctrl Alt F` | Moves focus to provider chip row. | | Advisories & VEX | Provider filter | `⌘ ⌥ F` | `Ctrl Alt F` | Moves focus to provider chip row. |
| Runs | Refresh snapshot | `⌘ R` | `Ctrl R` | Soft refresh of SSE state; no full page reload. | | Runs | Refresh snapshot | `⌘ R` | `Ctrl R` | Soft refresh of SSE state; no full page reload. |
| Policies | Save draft | `⌘ S` | `Ctrl S` | Requires edit scope; exposes toast + status live update. | | Policies | Save draft | `⌘ S` | `Ctrl S` | Requires edit scope; exposes toast + status live update. |
| Downloads | Copy CLI command | `⇧ D` | `Shift D` | Copies manifest or export command; toast announces scope hints. | | Downloads | Copy CLI command | `⇧ D` | `Shift D` | Copies manifest or export command; toast announces scope hints. |
All shortcuts are remappable. Remaps persist in IndexedDB (per tenant) and export as part of profile bundles so operators can restore preferences offline. All shortcuts are remappable. Remaps persist in IndexedDB (per tenant) and export as part of profile bundles so operators can restore preferences offline.
--- ---
## 3 · Screen Reader & Focus Behaviour ## 3 · Screen Reader & Focus Behaviour
- **Skip navigation** – Each route exposes a “Skip to content” link revealed on keyboard focus. Focus order: global header → page breadcrumb → action shelf → data grid/list → drawers/dialogs. - **Skip navigation** – Each route exposes a “Skip to content” link revealed on keyboard focus. Focus order: global header → page breadcrumb → action shelf → data grid/list → drawers/dialogs.
- **Live regions** – Status ticker and SSE progress bars use `aria-live="polite"` with throttling to avoid flooding AT. Error toasts use `aria-live="assertive"` and auto-focus dismiss buttons. - **Live regions** – Status ticker and SSE progress bars use `aria-live="polite"` with throttling to avoid flooding AT. Error toasts use `aria-live="assertive"` and auto-focus dismiss buttons.
- **Drawers & modals** – Dialog components trap focus, support `Esc` to close, and restore focus to the launching control. Screen readers announce title + purpose. - **Drawers & modals** – Dialog components trap focus, support `Esc` to close, and restore focus to the launching control. Screen readers announce title + purpose.
- **Tables & grids** – Large tables (Findings, SBOM inventory) switch to virtualised rows but retain ARIA grid semantics (`aria-rowcount`, `aria-colindex`). Column headers include sorting state via `aria-sort`. - **Tables & grids** – Large tables (Findings, SBOM inventory) switch to virtualised rows but retain ARIA grid semantics (`aria-rowcount`, `aria-colindex`). Column headers include sorting state via `aria-sort`.
- **Tenancy context** – Tenant badge exposes `aria-describedby` linking to context summary (environment, offline snapshot). Switching tenant queues a polite announcement summarising new scope. - **Tenancy context** – Tenant badge exposes `aria-describedby` linking to context summary (environment, offline snapshot). Switching tenant queues a polite announcement summarising new scope.
- **Command palette** – Uses `role="dialog"` with search input labelled. Keyboard navigation within results uses `Up/Down`; screen readers announce result category + command. - **Command palette** – Uses `role="dialog"` with search input labelled. Keyboard navigation within results uses `Up/Down`; screen readers announce result category + command.
- **Offline banner** – When offline, a dismissible banner announces reason and includes instructions for CLI fallback. The banner has `role="status"` so it announces once without stealing focus. - **Offline banner** – When offline, a dismissible banner announces reason and includes instructions for CLI fallback. The banner has `role="status"` so it announces once without stealing focus.
--- ---
## 4 · Colour & Focus Tokens ## 4 · Colour & Focus Tokens
Console consumes design tokens published by the Console Guild (tracked via CONSOLE-FEAT-23-102). Tokens live in the design system bundle (`ui/design/tokens/colors.json`, mirrored at build time). Key tokens: Console consumes design tokens published by the Console Guild (tracked via CONSOLE-FEAT-23-102). Tokens live in the design system bundle (`ui/design/tokens/colors.json`, mirrored at build time). Key tokens:
| Token | Purpose | Contrast target | | Token | Purpose | Contrast target |
|-------|---------|-----------------| |-------|---------|-----------------|
| `so-color-surface-base` | Primary surface/background | ≥ 4.5:1 against `so-color-text-primary`. | | `so-color-surface-base` | Primary surface/background | ≥ 4.5:1 against `so-color-text-primary`. |
| `so-color-surface-raised` | Cards, drawers, modals | ≥ 3:1 against surrounding surfaces. | | `so-color-surface-raised` | Cards, drawers, modals | ≥ 3:1 against surrounding surfaces. |
| `so-color-text-primary` | Default text colour | ≥ 4.5:1 against base surfaces. | | `so-color-text-primary` | Default text colour | ≥ 4.5:1 against base surfaces. |
| `so-color-text-inverted` | Text on accent buttons | ≥ 4.5:1 against accent fills. | | `so-color-text-inverted` | Text on accent buttons | ≥ 4.5:1 against accent fills. |
| `so-color-accent-primary` | Action buttons, focus headings | ≥ 3:1 against surface. | | `so-color-accent-primary` | Action buttons, focus headings | ≥ 3:1 against surface. |
| `so-color-status-critical` | Error toasts, violation chips | ≥ 4.5:1 for text; `critical-bg` provides >3:1 on neutral surface. | | `so-color-status-critical` | Error toasts, violation chips | ≥ 4.5:1 for text; `critical-bg` provides >3:1 on neutral surface. |
| `so-color-status-warning` | Warning banners | Meets 3:1 on surface and 4.5:1 for text overlays. | | `so-color-status-warning` | Warning banners | Meets 3:1 on surface and 4.5:1 for text overlays. |
| `so-color-status-success` | Success toasts, pass badges | ≥ 3:1 for iconography; text uses `text-primary`. | | `so-color-status-success` | Success toasts, pass badges | ≥ 3:1 for iconography; text uses `text-primary`. |
| `so-focus-ring` | 2 px outline used across focusable elements | 3:1 against both light/dark surfaces. | | `so-focus-ring` | 2 px outline used across focusable elements | 3:1 against both light/dark surfaces. |
Colour tokens undergo automated linting (**axe-core contrast checks** + custom luminance script) during build. Any new token must include dark/light variants and pass the token contract tests. Colour tokens undergo automated linting (**axe-core contrast checks** + custom luminance script) during build. Any new token must include dark/light variants and pass the token contract tests.
--- ---
## 5 · Testing Workflow ## 5 · Testing Workflow
| Layer | Tooling | Frequency | Notes | | Layer | Tooling | Frequency | Notes |
|-------|---------|-----------|-------| |-------|---------|-----------|-------|
| Component a11y | Storybook + axe-core addon | On PR (story CI) | Fails when axe detects violations. | | Component a11y | Storybook + axe-core addon | On PR (story CI) | Fails when axe detects violations. |
| Route regression | Playwright a11y sweep (`pnpm test:a11y`) | Nightly & release pipeline | Executes keyboard navigation, checks focus trap, runs Axe on key routes (Dashboard, Findings, SBOM, Admin). | | Route regression | Playwright a11y sweep (`pnpm test:a11y`) | Nightly & release pipeline | Executes keyboard navigation, checks focus trap, runs Axe on key routes (Dashboard, Findings, SBOM, Admin). |
| Colour contrast lint | Token validator (`src/Tools/a11y/check-contrast.ts`) | On token change | Guards design token updates. | | Colour contrast lint | Token validator (`src/Tools/a11y/check-contrast.ts`) | On token change | Guards design token updates. |
| CI parity | Pending `scripts/check-console-cli-parity.sh` (CONSOLE-DOC-23-502) | Release CI | Ensures CLI commands documented for parity features. | | CI parity | Pending `scripts/check-console-cli-parity.sh` (CONSOLE-DOC-23-502) | Release CI | Ensures CLI commands documented for parity features. |
| Screen-reader spot checks | Manual NVDA + VoiceOver scripts | Pre-release checklist | Scenarios: tenant switch, explain drawer, downloads parity copy. | | Screen-reader spot checks | Manual NVDA + VoiceOver scripts | Pre-release checklist | Scenarios: tenant switch, explain drawer, downloads parity copy. |
| Offline smoke | `stella offline kit import` + Playwright sealed-mode run | Prior to Offline Kit cut | Validates offline banners, disabled actions, keyboard flows without Authority. | | Offline smoke | `stella offline kit import` + Playwright sealed-mode run | Prior to Offline Kit cut | Validates offline banners, disabled actions, keyboard flows without Authority. |
Accessibility QA (CONSOLE-QA-23-402) tracks failing scenarios via Playwright snapshots and publishes reports in the Downloads parity channel (`kind = "parity.report"` placeholder until CLI parity CI lands). Accessibility QA (CONSOLE-QA-23-402) tracks failing scenarios via Playwright snapshots and publishes reports in the Downloads parity channel (`kind = "parity.report"` placeholder until CLI parity CI lands).
--- ---
## 6 · Offline & Internationalisation Considerations ## 6 · Offline & Internationalisation Considerations
- Offline mode surfaces staleness badges and disables remote-only palette entries; keyboard focus skips disabled controls. - Offline mode surfaces staleness badges and disables remote-only palette entries; keyboard focus skips disabled controls.
- Saved shortcuts, presets, and remaps serialise into Offline Kit bundles so operators can restore preferences post-import. - Saved shortcuts, presets, and remaps serialise into Offline Kit bundles so operators can restore preferences post-import.
- Locale switching (future feature flag) will load translations at runtime; ensure ARIA labels use i18n tokens rather than hard-coded strings. - Locale switching (future feature flag) will load translations at runtime; ensure ARIA labels use i18n tokens rather than hard-coded strings.
- For sealed installs, guidance panels include CLI equivalents (`stella auth fresh-auth`, `stella runs export`) to unblock tasks when Authority is unavailable. - For sealed installs, guidance panels include CLI equivalents (`stella auth fresh-auth`, `stella runs export`) to unblock tasks when Authority is unavailable.
--- ---
## 7 · Compliance Checklist ## 7 · Compliance Checklist
- [ ] Keyboard shortcut matrix validated (default + remapped) and documented. - [ ] Keyboard shortcut matrix validated (default + remapped) and documented.
- [ ] Screen-reader pass recorded for tenant switch, Explain drawer, Downloads copy-to-clipboard. - [ ] Screen-reader pass recorded for tenant switch, Explain drawer, Downloads copy-to-clipboard.
- [ ] Colour tokens audited; contrast reports stored with release artifacts. - [ ] Colour tokens audited; contrast reports stored with release artifacts.
- [ ] Automated a11y pipelines (Storybook axe, Playwright a11y) green; failures feed the `#console-qa` channel. - [ ] Automated a11y pipelines (Storybook axe, Playwright a11y) green; failures feed the `#console-qa` channel.
- [ ] Offline kit a11y smoke executed before publishing each bundle. - [ ] Offline kit a11y smoke executed before publishing each bundle.
- [ ] CLI parity gaps logged in `/docs/cli-vs-ui-parity.md`; UI callouts reference fallback commands until parity closes. - [ ] CLI parity gaps logged in `/docs/cli-vs-ui-parity.md`; UI callouts reference fallback commands until parity closes.
- [ ] Accessibility Guild sign-off captured in sprint log and release notes reference this guide. - [ ] Accessibility Guild sign-off captured in sprint log and release notes reference this guide.
- [ ] References cross-checked (`/docs/ui/navigation.md`, `/docs/ui/downloads.md`, `/docs/security/console-security.md`, `/docs/observability/ui-telemetry.md`). - [ ] References cross-checked (`/docs/ui/navigation.md`, `/docs/ui/downloads.md`, `/docs/security/console-security.md`, `/docs/observability/ui-telemetry.md`).
--- ---
## 8 · References ## 8 · References
- `/docs/ui/navigation.md` – shortcut definitions, URL schema. - `/docs/ui/navigation.md` – shortcut definitions, URL schema.
- `/docs/ui/downloads.md` – CLI parity and offline copy workflows. - `/docs/ui/downloads.md` – CLI parity and offline copy workflows.
- `/docs/ui/console-overview.md` – tenant model, filter behaviours. - `/docs/ui/console-overview.md` – tenant model, filter behaviours.
- `/docs/security/console-security.md` – security metrics and DPoP/fresh-auth requirements. - `/docs/security/console-security.md` – security metrics and DPoP/fresh-auth requirements.
- `/docs/observability/ui-telemetry.md` – telemetry metrics mapped to accessibility features. - `/docs/observability/ui-telemetry.md` – telemetry metrics mapped to accessibility features.
- `/docs/cli-vs-ui-parity.md` – parity status per console feature. - `/docs/cli-vs-ui-parity.md` – parity status per console feature.
- `CONSOLE-QA-23-402` – Accessibility QA backlog (Playwright + manual checks). - `CONSOLE-QA-23-402` – Accessibility QA backlog (Playwright + manual checks).
- `CONSOLE-FEAT-23-102` – Design tokens & theming delivery. - `CONSOLE-FEAT-23-102` – Design tokens & theming delivery.
--- ---
*Last updated: 2025-10-28 (Sprint 23).* *Last updated: 2025-10-28 (Sprint 23).*

142
docs/airgap/airgap-mode.md
View File

@@ -1,71 +1,71 @@
# Air-Gapped Mode Playbook # Air-Gapped Mode Playbook
> Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. > Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
## Overview ## Overview
Air-Gapped Mode is the supported operating profile for deployments with **zero external egress**. All inputs arrive via signed mirror bundles, and every surface (CLI, Console, APIs, schedulers, scanners) operates under sealed-network constraints while preserving Aggregation-Only Contract invariants. Air-Gapped Mode is the supported operating profile for deployments with **zero external egress**. All inputs arrive via signed mirror bundles, and every surface (CLI, Console, APIs, schedulers, scanners) operates under sealed-network constraints while preserving Aggregation-Only Contract invariants.
- **Primary components:** Web Services API, Console, CLI, Orchestrator, Task Runner, Conseiller (Feedser), Excitator (VEXer), Policy Engine, Findings Ledger, Export Center, Authority & Tenancy, Notifications, Observability & Forensics. - **Primary components:** Web Services API, Console, CLI, Orchestrator, Task Runner, Conseiller (Feedser), Excitator (VEXer), Policy Engine, Findings Ledger, Export Center, Authority & Tenancy, Notifications, Observability & Forensics.
- **Surfaces:** offline bootstrap, mirror ingestion, deterministic jobs, offline advisories/VEX/policy packs/notifications, evidence exports. - **Surfaces:** offline bootstrap, mirror ingestion, deterministic jobs, offline advisories/VEX/policy packs/notifications, evidence exports.
- **Dependencies:** Export Center, Containerized Distribution, Authority-backed scopes & tenancy, Observability & Forensics, Policy Studio. - **Dependencies:** Export Center, Containerized Distribution, Authority-backed scopes & tenancy, Observability & Forensics, Policy Studio.
## Guiding principles ## Guiding principles
1. **Zero egress:** all outbound network calls are disabled unless explicitly allowed. Any feature requiring online data must degrade gracefully with clear UX messaging. 1. **Zero egress:** all outbound network calls are disabled unless explicitly allowed. Any feature requiring online data must degrade gracefully with clear UX messaging.
2. **Deterministic inputs:** the platform accepts only signed Mirror Bundles (advisories, VEX, policy packs, vendor feeds, images, dashboards). Bundles carry provenance attestations and chain-of-custody manifests. 2. **Deterministic inputs:** the platform accepts only signed Mirror Bundles (advisories, VEX, policy packs, vendor feeds, images, dashboards). Bundles carry provenance attestations and chain-of-custody manifests.
3. **Auditable exchange:** every import/export records provenance, signatures, and operator identity. Evidence bundles and reports remain verifiable offline. 3. **Auditable exchange:** every import/export records provenance, signatures, and operator identity. Evidence bundles and reports remain verifiable offline.
4. **Aggregation-Only Contract compliance:** Conseiller and Excitator continue to aggregate without mutating source records, even when ingesting mirrored feeds. 4. **Aggregation-Only Contract compliance:** Conseiller and Excitator continue to aggregate without mutating source records, even when ingesting mirrored feeds.
5. **Operator ergonomics:** offline bootstrap, upgrade, and verification steps are reproducible and scripted. 5. **Operator ergonomics:** offline bootstrap, upgrade, and verification steps are reproducible and scripted.
## Lifecycle & modes ## Lifecycle & modes
| Mode | Description | Tooling | | Mode | Description | Tooling |
| --- | --- | --- | | --- | --- | --- |
| Connected | Standard deployment with online feeds. Operators use Export Center to build mirror bundles for offline environments. | `stella export bundle create --profile mirror:full` | | Connected | Standard deployment with online feeds. Operators use Export Center to build mirror bundles for offline environments. | `stella export bundle create --profile mirror:full` |
| Staging mirror | Sealed host that fetches upstream feeds, runs validation, and signs mirror bundles. | Export Center, cosign, bundle validation scripts | | Staging mirror | Sealed host that fetches upstream feeds, runs validation, and signs mirror bundles. | Export Center, cosign, bundle validation scripts |
| Air-gapped | Production cluster with egress sealed, consuming validated bundles, issuing provenance for inward/outward transfers. | Mirror import CLI, sealed-mode runtime flags | | Air-gapped | Production cluster with egress sealed, consuming validated bundles, issuing provenance for inward/outward transfers. | Mirror import CLI, sealed-mode runtime flags |
### Installation & bootstrap ### Installation & bootstrap
1. Prepare mirror bundles (images, charts, advisories/VEX, policy packs, dashboards, telemetry configs). 1. Prepare mirror bundles (images, charts, advisories/VEX, policy packs, dashboards, telemetry configs).
2. Transfer bundles via approved media and validate signatures (`cosign verify`, bundle manifest hash). 2. Transfer bundles via approved media and validate signatures (`cosign verify`, bundle manifest hash).
3. Deploy platform using offline artefacts (`helm install --set airgap.enabled=true`), referencing local registry/object storage. 3. Deploy platform using offline artefacts (`helm install --set airgap.enabled=true`), referencing local registry/object storage.
### Updates ### Updates
1. Staging host generates incremental bundles (mirror delta) with provenance. 1. Staging host generates incremental bundles (mirror delta) with provenance.
2. Offline site imports bundles via the CLI (`stella airgap import --bundle`) and records chain-of-custody. 2. Offline site imports bundles via the CLI (`stella airgap import --bundle`) and records chain-of-custody.
3. Scheduler triggers replay jobs with deterministic timelines; results remain reproducible across imports. 3. Scheduler triggers replay jobs with deterministic timelines; results remain reproducible across imports.
## Component responsibilities ## Component responsibilities
| Component | Offline duties | | Component | Offline duties |
| --- | --- | | --- | --- |
| Export Center | Produce full/delta mirror bundles, signed manifests, provenance attestations. | | Export Center | Produce full/delta mirror bundles, signed manifests, provenance attestations. |
| Authority & Tenancy | Provide offline scope enforcement, short-lived tokens, revocation via local CRLs. | | Authority & Tenancy | Provide offline scope enforcement, short-lived tokens, revocation via local CRLs. |
| Conseiller / Excitator | Ingest mirrored advisories/VEX, enforce AOC, versioned observations. | | Conseiller / Excitator | Ingest mirrored advisories/VEX, enforce AOC, versioned observations. |
| Policy Engine & Findings Ledger | Replay evaluations using offline feeds, emit explain traces, support sealed-mode hints. | | Policy Engine & Findings Ledger | Replay evaluations using offline feeds, emit explain traces, support sealed-mode hints. |
| Notifications | Deliver locally via approved channels (email relay, webhook proxies) or queue for manual export. | | Notifications | Deliver locally via approved channels (email relay, webhook proxies) or queue for manual export. |
| Observability | Collect metrics/logs/traces locally, generate forensic bundles for external analysis. | | Observability | Collect metrics/logs/traces locally, generate forensic bundles for external analysis. |
## Operational guardrails ## Operational guardrails
- **Network policy:** enforce allowlists (`airgap.egressAllowlist=[]`). Any unexpected outbound request raises an alert. - **Network policy:** enforce allowlists (`airgap.egressAllowlist=[]`). Any unexpected outbound request raises an alert.
- **Bundle validation:** double-sign manifests (bundle signer + site-specific cosign key); reject on mismatch. - **Bundle validation:** double-sign manifests (bundle signer + site-specific cosign key); reject on mismatch.
- **Time synchronization:** rely on local NTP or manual clock audits; many signatures require monotonic time. - **Time synchronization:** rely on local NTP or manual clock audits; many signatures require monotonic time.
- **Key rotation:** plan for offline key ceremonies; Export Center and Authority document rotation playbooks. - **Key rotation:** plan for offline key ceremonies; Export Center and Authority document rotation playbooks.
- **Incident response:** maintain scripts for replaying imports, regenerating manifests, and exporting forensic data without egress. - **Incident response:** maintain scripts for replaying imports, regenerating manifests, and exporting forensic data without egress.
## Testing & verification ## Testing & verification
- Integration tests mimic offline installs by running with `AIRGAP_ENABLED=true` in CI. - Integration tests mimic offline installs by running with `AIRGAP_ENABLED=true` in CI.
- Mirror bundles include validation scripts to compare hash manifests across staging and production. - Mirror bundles include validation scripts to compare hash manifests across staging and production.
- Sealed-mode smoke tests ensure services fail closed when attempting egress. - Sealed-mode smoke tests ensure services fail closed when attempting egress.
## References ## References
- Export workflows: `docs/modules/export-center/overview.md` - Export workflows: `docs/modules/export-center/overview.md`
- Policy sealed-mode hints: `docs/policy/overview.md` - Policy sealed-mode hints: `docs/policy/overview.md`
- Observability forensic bundles: `docs/modules/telemetry/architecture.md` - Observability forensic bundles: `docs/modules/telemetry/architecture.md`
- Runtime posture enforcement: `docs/modules/zastava/operations/runtime.md` - Runtime posture enforcement: `docs/modules/zastava/operations/runtime.md`

26
docs/aoc/aoc-guardrails.md
View File

@@ -1,13 +1,13 @@
# Aggregation-Only Contract (AOC) Guardrails # Aggregation-Only Contract (AOC) Guardrails
The Aggregation-Only Contract keeps ingestion services deterministic and policy-neutral. Use these checkpoints whenever you add or modify backlog items: The Aggregation-Only Contract keeps ingestion services deterministic and policy-neutral. Use these checkpoints whenever you add or modify backlog items:
1. **Ingestion writes raw facts only.** Concelier and Excititor append immutable observations/linksets. No precedence, severity, suppression, or "safe fix" hints may be computed at ingest time. 1. **Ingestion writes raw facts only.** Concelier and Excititor append immutable observations/linksets. No precedence, severity, suppression, or "safe fix" hints may be computed at ingest time.
2. **Derived semantics live elsewhere.** Policy Engine overlays, Vuln Explorer composition, and downstream reporting layers attach severity, precedence, policy verdicts, and UI hints. 2. **Derived semantics live elsewhere.** Policy Engine overlays, Vuln Explorer composition, and downstream reporting layers attach severity, precedence, policy verdicts, and UI hints.
3. **Provenance is mandatory.** Every ingestion write must include original source metadata, digests, and signing/provenance evidence when available. Reject writes lacking provenance. 3. **Provenance is mandatory.** Every ingestion write must include original source metadata, digests, and signing/provenance evidence when available. Reject writes lacking provenance.
4. **Deterministic outputs.** Given the same inputs, ingestion must produce identical documents, hashes, and event payloads across reruns. 4. **Deterministic outputs.** Given the same inputs, ingestion must produce identical documents, hashes, and event payloads across reruns.
5. **Guardrails everywhere.** Roslyn analyzers, schema validators, and CI smoke tests should fail builds that attempt forbidden writes. 5. **Guardrails everywhere.** Roslyn analyzers, schema validators, and CI smoke tests should fail builds that attempt forbidden writes.
For detailed roles and ownership boundaries, see `AGENTS.md` at the repo root and the module-specific dossiers under `docs/modules/<module>/architecture.md`. For detailed roles and ownership boundaries, see `AGENTS.md` at the repo root and the module-specific dossiers under `docs/modules/<module>/architecture.md`.
Need the full contract? Read the [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md) for schemas, error codes, and migration guidance. Need the full contract? Read the [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md) for schemas, error codes, and migration guidance.

102
docs/api/sdk-openapi-program.md
View File

@@ -1,51 +1,51 @@
# SDK & OpenAPI Program # SDK & OpenAPI Program
> Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. > Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
## Overview ## Overview
The SDK & OpenAPI program delivers canonical OpenAPI 3.1 contracts for every Stella Ops surface, plus officially supported SDKs (TypeScript/Node, Python, Go, Java, C#). It ensures backwards-compatible evolution, documentation, and offline availability. The SDK & OpenAPI program delivers canonical OpenAPI 3.1 contracts for every Stella Ops surface, plus officially supported SDKs (TypeScript/Node, Python, Go, Java, C#). It ensures backwards-compatible evolution, documentation, and offline availability.
- **Primary components:** API Gateway, Web Services, Policy Engine, Conseiller, Excitator, Orchestrator, Findings Ledger, Export Center, Authority & Tenancy, Console, CLI. - **Primary components:** API Gateway, Web Services, Policy Engine, Conseiller, Excitator, Orchestrator, Findings Ledger, Export Center, Authority & Tenancy, Console, CLI.
- **Surfaces:** OpenAPI specs, language SDKs, developer portal, examples, mock server, conformance tests, changelog feeds, deprecation notices. - **Surfaces:** OpenAPI specs, language SDKs, developer portal, examples, mock server, conformance tests, changelog feeds, deprecation notices.
- **Dependencies:** Authority scopes/tenancy, CLI parity, Export Center, Notifications, Air-Gapped Mode, Observability. - **Dependencies:** Authority scopes/tenancy, CLI parity, Export Center, Notifications, Air-Gapped Mode, Observability.
## Program pillars ## Program pillars
1. **Contract-first:** treat OpenAPI specs as the source of truth. CI validates schemas, compatibility, and documentation generation. 1. **Contract-first:** treat OpenAPI specs as the source of truth. CI validates schemas, compatibility, and documentation generation.
2. **SDK parity:** language SDKs cover the same surfaces with deterministic clients, pagination helpers, and typed models mirroring Aggregation-Only Contract semantics. 2. **SDK parity:** language SDKs cover the same surfaces with deterministic clients, pagination helpers, and typed models mirroring Aggregation-Only Contract semantics.
3. **Version discipline:** semantic versioning for specs and SDKs, release notes, deprecation windows, and automated change alerts via Notifications. 3. **Version discipline:** semantic versioning for specs and SDKs, release notes, deprecation windows, and automated change alerts via Notifications.
4. **Offline readiness:** specs and SDK bundles ship in Mirror Bundles for air-gapped environments; examples include smoke tests. 4. **Offline readiness:** specs and SDK bundles ship in Mirror Bundles for air-gapped environments; examples include smoke tests.
5. **Observability:** telemetry around SDK usage, spec download metrics, and error reporting funnels back into product decisions. 5. **Observability:** telemetry around SDK usage, spec download metrics, and error reporting funnels back into product decisions.
## Deliverables ## Deliverables
| Workstream | Deliverable | | Workstream | Deliverable |
| --- | --- | | --- | --- |
| Spec authoring | Unified OpenAPI 3.1 documents per service plus aggregate spec; lint rules; schema registries. | | Spec authoring | Unified OpenAPI 3.1 documents per service plus aggregate spec; lint rules; schema registries. |
| SDK generation | Language-specific clients with idiomatic ergonomics, retries, pagination, long-running operation helpers, unit + integration tests. | | SDK generation | Language-specific clients with idiomatic ergonomics, retries, pagination, long-running operation helpers, unit + integration tests. |
| Dev portal | Consolidated documentation, guides, changelog, copy/paste examples, quickstart scripts. | | Dev portal | Consolidated documentation, guides, changelog, copy/paste examples, quickstart scripts. |
| Testing | Contract tests against staging, mock server for integration tests, compatibility verification per release. | | Testing | Contract tests against staging, mock server for integration tests, compatibility verification per release. |
| Release ops | Automated CI pipelines, version bump workflows, release notes, deprecation policies. | | Release ops | Automated CI pipelines, version bump workflows, release notes, deprecation policies. |
## Guardrails ## Guardrails
- **Aggregation-Only Contract compliance:** SDKs expose raw advisory/VEX objects without hidden merges; all derived fields require explicit Policy Engine calls. - **Aggregation-Only Contract compliance:** SDKs expose raw advisory/VEX objects without hidden merges; all derived fields require explicit Policy Engine calls.
- **Security:** enforce scopes via SDK configuration; redact secrets; support DPoP/mTLS and offline token provisioning. - **Security:** enforce scopes via SDK configuration; redact secrets; support DPoP/mTLS and offline token provisioning.
- **Compatibility:** maintain backwards-compatible paths for at least two minor releases; log warnings on deprecated endpoints. - **Compatibility:** maintain backwards-compatible paths for at least two minor releases; log warnings on deprecated endpoints.
- **Documentation:** publish examples for common workflows (scan, policy evaluate, export, attestation) with language parity. - **Documentation:** publish examples for common workflows (scan, policy evaluate, export, attestation) with language parity.
## Roadmap checkpoints ## Roadmap checkpoints
1. Baseline OpenAPI specs extracted from gateway, validated, and published. 1. Baseline OpenAPI specs extracted from gateway, validated, and published.
2. TypeScript/Node SDK as pilot, followed by Python and Go. 2. TypeScript/Node SDK as pilot, followed by Python and Go.
3. Developer portal launch with SDK docs, quickstarts, and mock server. 3. Developer portal launch with SDK docs, quickstarts, and mock server.
4. Offline kit integration (mirror bundles include specs + SDK tarballs). 4. Offline kit integration (mirror bundles include specs + SDK tarballs).
5. Runtime alerting for breaking changes and dependency vulnerabilities. 5. Runtime alerting for breaking changes and dependency vulnerabilities.
## References ## References
- API gateway integration: `docs/modules/platform/architecture-overview.md` - API gateway integration: `docs/modules/platform/architecture-overview.md`
- Policy/Findings models: `docs/modules/policy/architecture.md`, `docs/modules/vuln-explorer/architecture.md` - Policy/Findings models: `docs/modules/policy/architecture.md`, `docs/modules/vuln-explorer/architecture.md`
- Export bundle distribution: `docs/modules/export-center/overview.md` - Export bundle distribution: `docs/modules/export-center/overview.md`
- Offline workflows: `docs/airgap/airgap-mode.md` - Offline workflows: `docs/airgap/airgap-mode.md`

310
docs/cli-vs-ui-parity.md
View File

@@ -1,155 +1,155 @@
# Console CLI ↔ UI Parity Matrix # Console CLI ↔ UI Parity Matrix
> **Audience:** Docs Guild, Console Guild, CLI Guild, DevOps automation. > **Audience:** Docs Guild, Console Guild, CLI Guild, DevOps automation.
> **Scope:** Track feature-level parity between the StellaOps Console and the `stella` CLI, surface pending work, and describe the parity CI check owned by CONSOLE-DOC-23-502. > **Scope:** Track feature-level parity between the StellaOps Console and the `stella` CLI, surface pending work, and describe the parity CI check owned by CONSOLE-DOC-23-502.
Status key: Status key:
- **✅ Available** – command exists in `StellaOps.Cli` and is documented. - **✅ Available** – command exists in `StellaOps.Cli` and is documented.
- **🟡 In progress** – command implemented but still under active delivery (task status `DOING`). - **🟡 In progress** – command implemented but still under active delivery (task status `DOING`).
- **🟩 Planned** – command spec’d but not yet implemented (task `TODO`). - **🟩 Planned** – command spec’d but not yet implemented (task `TODO`).
- **⚪ UI-only** – no CLI equivalent required. - **⚪ UI-only** – no CLI equivalent required.
- **🔴 Gap** – CLI feature missing with no active task; file a task before sprint exit. - **🔴 Gap** – CLI feature missing with no active task; file a task before sprint exit.
--- ---
## 1 · Navigation & Tenancy ## 1 · Navigation & Tenancy
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Login / token cache status (`/console/profile`) | `stella auth login`, `stella auth status`, `stella auth whoami` | ✅ Available | Command definitions in `CommandFactory.BuildAuthCommand`. | | Login / token cache status (`/console/profile`) | `stella auth login`, `stella auth status`, `stella auth whoami` | ✅ Available | Command definitions in `CommandFactory.BuildAuthCommand`. |
| Fresh-auth challenge for sensitive actions | `stella auth fresh-auth` | ✅ Available | Referenced in `/docs/ui/admin.md`. | | Fresh-auth challenge for sensitive actions | `stella auth fresh-auth` | ✅ Available | Referenced in `/docs/ui/admin.md`. |
| Tenant switcher (UI shell) | `--tenant` flag across CLI commands | ✅ Available | All multi-tenant commands require explicit `--tenant`. | | Tenant switcher (UI shell) | `--tenant` flag across CLI commands | ✅ Available | All multi-tenant commands require explicit `--tenant`. |
| Tenant creation / suspension | *(pending CLI)* | 🟩 Planned | No `stella auth tenant *` commands yet – track via `CLI-TEN-47-001` (scopes & tenancy). | | Tenant creation / suspension | *(pending CLI)* | 🟩 Planned | No `stella auth tenant *` commands yet – track via `CLI-TEN-47-001` (scopes & tenancy). |
--- ---
## 2 · Policies & Findings ## 2 · Policies & Findings
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Policy simulation diff, explain | `stella policy simulate` | 🟡 In progress | Implementation present; task `CLI-POLICY-20-002` marked DOING. | | Policy simulation diff, explain | `stella policy simulate` | 🟡 In progress | Implementation present; task `CLI-POLICY-20-002` marked DOING. |
| Promote / activate policy | `stella policy promote`, `stella policy activate` | 🟩 Planned | Spec tracked under `CLI-POLICY-23-005`. | | Promote / activate policy | `stella policy promote`, `stella policy activate` | 🟩 Planned | Spec tracked under `CLI-POLICY-23-005`. |
| History & explain trees | `stella policy history`, `stella policy explain` | 🟩 Planned | `CLI-POLICY-23-006`. | | History & explain trees | `stella policy history`, `stella policy explain` | 🟩 Planned | `CLI-POLICY-23-006`. |
| Findings explorer export | `stella findings get`, `stella findings export` | 🟩 Planned | Part of `CLI-POLICY-20-003`. | | Findings explorer export | `stella findings get`, `stella findings export` | 🟩 Planned | Part of `CLI-POLICY-20-003`. |
| Explain drawer JSON | `stella policy simulate --format json` | 🟡 In progress | Same command; JSON output flagged for CLI tests. | | Explain drawer JSON | `stella policy simulate --format json` | 🟡 In progress | Same command; JSON output flagged for CLI tests. |
--- ---
## 3 · Runs & Evidence ## 3 · Runs & Evidence
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Run retry / cancel | `stella runs retry`, `stella runs cancel` | 🟩 Planned | Included in export suite task `CLI-EXPORT-35-001`. | | Run retry / cancel | `stella runs retry`, `stella runs cancel` | 🟩 Planned | Included in export suite task `CLI-EXPORT-35-001`. |
| Manual run submit / preview | `stella runs submit`, `stella runs preview` | 🟩 Planned | `CLI-EXPORT-35-001`. | | Manual run submit / preview | `stella runs submit`, `stella runs preview` | 🟩 Planned | `CLI-EXPORT-35-001`. |
| Evidence bundle export | `stella runs export --run <id> --bundle` | 🟩 Planned | `CLI-EXPORT-35-001`. | | Evidence bundle export | `stella runs export --run <id> --bundle` | 🟩 Planned | `CLI-EXPORT-35-001`. |
| Run status polling | `stella runs status` | 🟩 Planned | Same task. | | Run status polling | `stella runs status` | 🟩 Planned | Same task. |
--- ---
## 4 · Advisories, VEX, SBOM ## 4 · Advisories, VEX, SBOM
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Advisory observations search | `stella vuln observations` | ✅ Available | Implemented via `BuildVulnCommand`. | | Advisory observations search | `stella vuln observations` | ✅ Available | Implemented via `BuildVulnCommand`. |
| Advisory linkset export | `stella advisory linkset show/export` | 🟩 Planned | `CLI-LNM-22-001`. | | Advisory linkset export | `stella advisory linkset show/export` | 🟩 Planned | `CLI-LNM-22-001`. |
| VEX observations / linksets | `stella vex obs get/linkset show` | 🟩 Planned | `CLI-LNM-22-002`. | | VEX observations / linksets | `stella vex obs get/linkset show` | 🟩 Planned | `CLI-LNM-22-002`. |
| SBOM overlay export | `stella sbom overlay apply/export` | 🟩 Planned | Scoped to upcoming SBOM CLI sprint (`SBOM-CONSOLE-23-001/002` + CLI backlog). | | SBOM overlay export | `stella sbom overlay apply/export` | 🟩 Planned | Scoped to upcoming SBOM CLI sprint (`SBOM-CONSOLE-23-001/002` + CLI backlog). |
--- ---
## 5 · Downloads & Offline Kit ## 5 · Downloads & Offline Kit
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Manifest lookup (Console Downloads) | `stella downloads manifest show --artifact <id>` | 🟩 Planned | Delivered with `CONSOLE-DOC-23-502` + CLI parity commands. | | Manifest lookup (Console Downloads) | `stella downloads manifest show --artifact <id>` | 🟩 Planned | Delivered with `CONSOLE-DOC-23-502` + CLI parity commands. |
| Mirror digest to OCI archive | `stella downloads mirror --artifact <id> --to <target>` | 🟩 Planned | Same task bundle (`CONSOLE-DOC-23-502`). | | Mirror digest to OCI archive | `stella downloads mirror --artifact <id> --to <target>` | 🟩 Planned | Same task bundle (`CONSOLE-DOC-23-502`). |
| Console health check | `stella console status --endpoint <url>` | 🟩 Planned | Tracked in `CONSOLE-DOC-23-502`; interim use `curl` as documented. | | Console health check | `stella console status --endpoint <url>` | 🟩 Planned | Tracked in `CONSOLE-DOC-23-502`; interim use `curl` as documented. |
| Offline kit import/export | `stella offline kit import`, `stella offline kit export` | ✅ Available | Implemented (see `CommandHandlers.HandleOfflineKitImportAsync/HandleOfflineKitPullAsync`). | | Offline kit import/export | `stella offline kit import`, `stella offline kit export` | ✅ Available | Implemented (see `CommandHandlers.HandleOfflineKitImportAsync/HandleOfflineKitPullAsync`). |
--- ---
## 6 · Admin & Security ## 6 · Admin & Security
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Client creation / rotation | `stella auth client create` *(planned)* | 🟩 Planned | Pending tenancy backlog `CLI-TEN-47-001`. | | Client creation / rotation | `stella auth client create` *(planned)* | 🟩 Planned | Pending tenancy backlog `CLI-TEN-47-001`. |
| Token revoke | `stella auth revoke export/verify` | ✅ Available | Already implemented. | | Token revoke | `stella auth revoke export/verify` | ✅ Available | Already implemented. |
| Audit export | `stella auth audit export` | 🟩 Planned | Needs CLI work item (Authority guild). | | Audit export | `stella auth audit export` | 🟩 Planned | Needs CLI work item (Authority guild). |
| Signing key rotation | `stella auth signing rotate` | 🟩 Planned | To be added with AUTH-CONSOLE-23-003 follow-up. | | Signing key rotation | `stella auth signing rotate` | 🟩 Planned | To be added with AUTH-CONSOLE-23-003 follow-up. |
--- ---
## 7 · Telemetry & Observability ## 7 · Telemetry & Observability
| UI capability | CLI command(s) | Status | Notes / Tasks | | UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------| |---------------|----------------|--------|---------------|
| Telemetry dashboard parity | `stella obs top`, `stella obs trace`, `stella obs logs` | 🟩 Planned | CLI observability epic (`CLI-OBS-51-001`, `CLI-OBS-52-001`). | | Telemetry dashboard parity | `stella obs top`, `stella obs trace`, `stella obs logs` | 🟩 Planned | CLI observability epic (`CLI-OBS-51-001`, `CLI-OBS-52-001`). |
| Incident mode toggle | `stella obs incident-mode enable|disable|status` | 🟩 Planned | CLI task `CLI-OBS-55-001`. | | Incident mode toggle | `stella obs incident-mode enable|disable|status` | 🟩 Planned | CLI task `CLI-OBS-55-001`. |
| Verify console telemetry health | `stella console status --telemetry` | 🟩 Planned | Part of `CONSOLE-DOC-23-502`. | | Verify console telemetry health | `stella console status --telemetry` | 🟩 Planned | Part of `CONSOLE-DOC-23-502`. |
--- ---
## 8 · Parity Gaps & Follow-up ## 8 · Parity Gaps & Follow-up
- **Tenant and client lifecycle CLI**: create/suspend tenants, manage clients. Coordinate with Authority CLI epic (`CLI-TEN-47-001`, `CLI-TEN-49-001`). - **Tenant and client lifecycle CLI**: create/suspend tenants, manage clients. Coordinate with Authority CLI epic (`CLI-TEN-47-001`, `CLI-TEN-49-001`).
- **Downloads parity commands**: blocked on `CONSOLE-DOC-23-502` and DevOps pipeline `DOWNLOADS-CONSOLE-23-001`. - **Downloads parity commands**: blocked on `CONSOLE-DOC-23-502` and DevOps pipeline `DOWNLOADS-CONSOLE-23-001`.
- **Policy promotion/history**: requires completion of CLI policy epic (`CLI-POLICY-23-005`/`23-006`). - **Policy promotion/history**: requires completion of CLI policy epic (`CLI-POLICY-23-005`/`23-006`).
- **Runs/evidence exports**: waiting on `CLI-EXPORT-35-001`. - **Runs/evidence exports**: waiting on `CLI-EXPORT-35-001`.
- **Observability tooling**: deliver `stella obs` commands before enabling parity CI checks for telemetry. - **Observability tooling**: deliver `stella obs` commands before enabling parity CI checks for telemetry.
Document updates should occur whenever a row changes status. When promoting a command from Planned → Available, ensure: Document updates should occur whenever a row changes status. When promoting a command from Planned → Available, ensure:
1. CLI command merged with help text. 1. CLI command merged with help text.
2. Relevant UI doc references updated to remove “pending” callouts. 2. Relevant UI doc references updated to remove “pending” callouts.
3. This matrix row status updated to ✅ and task IDs moved to release notes. 3. This matrix row status updated to ✅ and task IDs moved to release notes.
--- ---
## 9 · Parity CI Check (CONSOLE-DOC-23-502) ## 9 · Parity CI Check (CONSOLE-DOC-23-502)
- **Owner:** Docs Guild + DevEx/CLI Guild. - **Owner:** Docs Guild + DevEx/CLI Guild.
- **Artefact:** Planned `.gitea/workflows/cli-parity-console.yml`. - **Artefact:** Planned `.gitea/workflows/cli-parity-console.yml`.
- **What it does:** Runs `scripts/check-console-cli-parity.sh` (to be committed with the workflow) which: - **What it does:** Runs `scripts/check-console-cli-parity.sh` (to be committed with the workflow) which:
1. Parses this matrix (YAML view exported from Markdown) to identify rows marked ✅. 1. Parses this matrix (YAML view exported from Markdown) to identify rows marked ✅.
2. Executes `stella --help` to confirm listed commands exist. 2. Executes `stella --help` to confirm listed commands exist.
3. Optionally triggers smoke commands in sandbox mode (e.g., `stella policy simulate --help`). 3. Optionally triggers smoke commands in sandbox mode (e.g., `stella policy simulate --help`).
- **Failure action:** Workflow fails when a listed command is missing or when a row marked ✅ still contains “pending” notes. Update the matrix or fix CLI implementation before merging. - **Failure action:** Workflow fails when a listed command is missing or when a row marked ✅ still contains “pending” notes. Update the matrix or fix CLI implementation before merging.
Until the workflow lands, run the checker locally: Until the workflow lands, run the checker locally:
```bash ```bash
# Pending CONSOLE-DOC-23-502 – placeholder command # Pending CONSOLE-DOC-23-502 – placeholder command
./scripts/check-console-cli-parity.sh ./scripts/check-console-cli-parity.sh
``` ```
The script should emit a parity report that feeds into the Downloads workspace (`kind = "parity.report"`). The script should emit a parity report that feeds into the Downloads workspace (`kind = "parity.report"`).
--- ---
## 10 · Compliance checklist ## 10 · Compliance checklist
- [ ] Matrix reflects latest command availability (statuses accurate, task IDs linked). - [ ] Matrix reflects latest command availability (statuses accurate, task IDs linked).
- [ ] Notes include owning backlog items for every 🟩 / 🟡 row. - [ ] Notes include owning backlog items for every 🟩 / 🟡 row.
- [ ] CLI commands marked ✅ have corresponding entries in `/docs/modules/cli/guides/*.md` or module-specific docs. - [ ] CLI commands marked ✅ have corresponding entries in `/docs/modules/cli/guides/*.md` or module-specific docs.
- [ ] CI parity workflow description kept in sync with CONSOLE-DOC-23-502 implementation. - [ ] CI parity workflow description kept in sync with CONSOLE-DOC-23-502 implementation.
- [ ] Downloads workspace links to latest parity report. - [ ] Downloads workspace links to latest parity report.
- [ ] Install / observability guides reference this matrix for pending CLI parity. - [ ] Install / observability guides reference this matrix for pending CLI parity.
- [ ] Offline workflows capture CLI fallbacks when commands are pending. - [ ] Offline workflows capture CLI fallbacks when commands are pending.
- [ ] Docs Guild review recorded in sprint log once parity CI lands. - [ ] Docs Guild review recorded in sprint log once parity CI lands.
--- ---
## 11 · References ## 11 · References
- `/docs/ui/*.md` – per-surface UI parity callouts. - `/docs/ui/*.md` – per-surface UI parity callouts.
- `/docs/install/docker.md` – CLI parity section for deployments. - `/docs/install/docker.md` – CLI parity section for deployments.
- `/docs/observability/ui-telemetry.md` – telemetry metrics referencing CLI checks. - `/docs/observability/ui-telemetry.md` – telemetry metrics referencing CLI checks.
- `/docs/security/console-security.md` – security metrics & CLI parity expectations. - `/docs/security/console-security.md` – security metrics & CLI parity expectations.
- `src/Cli/StellaOps.Cli/TASKS.md` – authoritative status for CLI backlog. - `src/Cli/StellaOps.Cli/TASKS.md` – authoritative status for CLI backlog.
- `/docs/updates/2025-10-28-docs-guild.md` – coordination note for Authority/Security follow-up. - `/docs/updates/2025-10-28-docs-guild.md` – coordination note for Authority/Security follow-up.
--- ---
*Last updated: 2025-10-28 (Sprint 23).* *Last updated: 2025-10-28 (Sprint 23).*

86
docs/concelier-connector-research-20251011.md
View File

@@ -1,43 +1,43 @@
# Concelier Connector Research – 2025-10-11 # Concelier Connector Research – 2025-10-11
Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national/vendor connectors in scope. Use alongside each module’s `TASKS.md` notes. Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national/vendor connectors in scope. Use alongside each module’s `TASKS.md` notes.
## ACSC (Australia) ## ACSC (Australia)
- Enumerated feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92). Forcing HTTP/1.1 hangs >600 s and sitemap/HTML fetches fail the same way. - Enumerated feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92). Forcing HTTP/1.1 hangs >600 s and sitemap/HTML fetches fail the same way.
- Next actions: prototype `SocketsHttpHandler` settings (`RequestVersionOrLower`, allow fallback to relay), capture successful headers from partner vantage (need retention + cache semantics), and keep `FEEDCONN-SHARED-HTTP2-001` open for downgrade work. - Next actions: prototype `SocketsHttpHandler` settings (`RequestVersionOrLower`, allow fallback to relay), capture successful headers from partner vantage (need retention + cache semantics), and keep `FEEDCONN-SHARED-HTTP2-001` open for downgrade work.
## CCCS (Canada) ## CCCS (Canada)
- JSON endpoint (`https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=<lang>&content_type=cccs_threat`) returns ~5 100 records per language; `page=<n>` still works for segmented pulls and the earliest `date_created` seen is 2018‑06‑08 (EN) / 2018‑06‑08 (FR). Use an explicit `User-Agent` to avoid 403 responses. - JSON endpoint (`https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=<lang>&content_type=cccs_threat`) returns ~5 100 records per language; `page=<n>` still works for segmented pulls and the earliest `date_created` seen is 2018‑06‑08 (EN) / 2018‑06‑08 (FR). Use an explicit `User-Agent` to avoid 403 responses.
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/modules/concelier/operations/connectors/cccs.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive. - Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/modules/concelier/operations/connectors/cccs.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
## CERT-Bund (Germany) ## CERT-Bund (Germany)
- `https://wid.cert-bund.de/content/public/securityAdvisory/rss` responds 200 without cookies (≈250-item window, German taxonomy). Detail links load an Angular SPA that fetches JSON behind the bootstrap session. - `https://wid.cert-bund.de/content/public/securityAdvisory/rss` responds 200 without cookies (≈250-item window, German taxonomy). Detail links load an Angular SPA that fetches JSON behind the bootstrap session.
- Confirmed `GET https://wid.cert-bund.de/portal/api/securityadvisory?name=<WID-SEC-…>` returns JSON once the portal cookie container is primed; payload includes severity, CVEs, products, and references used by the connector fixtures. - Confirmed `GET https://wid.cert-bund.de/portal/api/securityadvisory?name=<WID-SEC-…>` returns JSON once the portal cookie container is primed; payload includes severity, CVEs, products, and references used by the connector fixtures.
- Historical advisories accessible through the SPA search/export endpoints once the `XSRF-TOKEN` cookie (exposed via `GET /portal/api/security/csrf`) is supplied with the `X-XSRF-TOKEN` header: - Historical advisories accessible through the SPA search/export endpoints once the `XSRF-TOKEN` cookie (exposed via `GET /portal/api/security/csrf`) is supplied with the `X-XSRF-TOKEN` header:
- `POST /portal/api/securityadvisory/search` (`{"page":N,"size":100,"sort":["published,desc"]}`) pages data back to 2014. - `POST /portal/api/securityadvisory/search` (`{"page":N,"size":100,"sort":["published,desc"]}`) pages data back to 2014.
- `GET /portal/api/securityadvisory/export?format=json&from=YYYY-MM-DD` emits JSON bundles suitable for Offline Kit mirrors. - `GET /portal/api/securityadvisory/export?format=json&from=YYYY-MM-DD` emits JSON bundles suitable for Offline Kit mirrors.
- Locale note: content is German-only; Concelier preserves `language=de` and Docs will publish a CERT-Bund glossary so operators can bridge terminology without machine translation. - Locale note: content is German-only; Concelier preserves `language=de` and Docs will publish a CERT-Bund glossary so operators can bridge terminology without machine translation.
## KISA / KNVD (Korea) ## KISA / KNVD (Korea)
- `https://knvd.krcert.or.kr/rss/securityInfo.do` and `/rss/securityNotice.do` return UTF-8 RSS (10-item window) with `detailDos.do?IDX=` links. No cookies required for feed fetch. - `https://knvd.krcert.or.kr/rss/securityInfo.do` and `/rss/securityNotice.do` return UTF-8 RSS (10-item window) with `detailDos.do?IDX=` links. No cookies required for feed fetch.
- Detail SPA calls resolve to `rssDetailData.do?IDX=` JSON payloads; connector fetches those directly, sanitises HTML, and records Hangul metadata (NFC). See `docs/dev/kisa_connector_notes.md` for telemetry + localisation guidance. - Detail SPA calls resolve to `rssDetailData.do?IDX=` JSON payloads; connector fetches those directly, sanitises HTML, and records Hangul metadata (NFC). See `docs/dev/kisa_connector_notes.md` for telemetry + localisation guidance.
## BDU (Russia / FSTEC) ## BDU (Russia / FSTEC)
- Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml/json`) return 403/404; TLS chain requires Russian Trusted Sub CA and WAF expects additional headers. - Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml/json`) return 403/404; TLS chain requires Russian Trusted Sub CA and WAF expects additional headers.
- Next actions: acquire official PEM chain, point `concelier:httpClients:source.bdu:trustedRootPaths` (or `concelier:sources:bdu:http:trustedRootPaths`) at the Offline Kit PEM, keep `allowInvalidCertificates=false`, script session bootstrap, then capture RSS/HTML schema for parser work. - Next actions: acquire official PEM chain, point `concelier:httpClients:source.bdu:trustedRootPaths` (or `concelier:sources:bdu:http:trustedRootPaths`) at the Offline Kit PEM, keep `allowInvalidCertificates=false`, script session bootstrap, then capture RSS/HTML schema for parser work.
## NKTsKI / cert.gov.ru (Russia) ## NKTsKI / cert.gov.ru (Russia)
- `https://cert.gov.ru/rss/advisories.xml` served via Bitrix returns 403/404 even with `Accept-Language: ru-RU`; TLS chain also requires Russian trust anchors. - `https://cert.gov.ru/rss/advisories.xml` served via Bitrix returns 403/404 even with `Accept-Language: ru-RU`; TLS chain also requires Russian trust anchors.
- Next actions: source trust store, configure `concelier:httpClients:source.nkcki:trustedRootPaths` (Offline Kit root via `concelier:offline:root`), prepare proxy fallback, and once accessible document taxonomy/retention plus attachment handling. - Next actions: source trust store, configure `concelier:httpClients:source.nkcki:trustedRootPaths` (Offline Kit root via `concelier:offline:root`), prepare proxy fallback, and once accessible document taxonomy/retention plus attachment handling.
## CISA ICS (United States) ## CISA ICS (United States)
- `curl -I https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` returns HTTP 403 + `x-reference-error` (Akamai). Same for legacy feed paths. - `curl -I https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` returns HTTP 403 + `x-reference-error` (Akamai). Same for legacy feed paths.
- Next actions: secure GovDelivery access, document token rotation, and build HTML/email fallback with throttling. - Next actions: secure GovDelivery access, document token rotation, and build HTML/email fallback with throttling.
## Cisco PSIRT ## Cisco PSIRT
- `https://api.cisco.com/security/advisories/latest` returns `ERR_596_SERVICE_NOT_FOUND` when unauthenticated. openVuln REST requires Mashery OAuth (client credentials) with quotas ~5 req/s, 30/min, 5 000/day; supports `pageIndex/pageSize` pagination. - `https://api.cisco.com/security/advisories/latest` returns `ERR_596_SERVICE_NOT_FOUND` when unauthenticated. openVuln REST requires Mashery OAuth (client credentials) with quotas ~5 req/s, 30/min, 5 000/day; supports `pageIndex/pageSize` pagination.
- Next actions: register OAuth app, capture pagination/delta parameters, and compare API vs RSS coverage. - Next actions: register OAuth app, capture pagination/delta parameters, and compare API vs RSS coverage.
## Microsoft MSRC ## Microsoft MSRC
- REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) requires Azure AD token + `api-version` (current `2024-08-01`) and supports delta filters (`lastModifiedStartDateTime`). CVRF ZIP remains available for offline use. - REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) requires Azure AD token + `api-version` (current `2024-08-01`) and supports delta filters (`lastModifiedStartDateTime`). CVRF ZIP remains available for offline use.
- Next actions: finalise AAD app registration, implement token cache, and design combined REST+CVRF ingestion path for determinism. - Next actions: finalise AAD app registration, implement token cache, and design combined REST+CVRF ingestion path for determinism.

456
docs/deploy/console.md
View File

@@ -1,228 +1,228 @@
# Deploying the StellaOps Console # Deploying the StellaOps Console
> **Audience:** Deployment Guild, Console Guild, operators rolling out the web console. > **Audience:** Deployment Guild, Console Guild, operators rolling out the web console.
> **Scope:** Helm and Docker Compose deployment steps, ingress/TLS configuration, required environment variables, health checks, offline/air-gap operation, and compliance checklist (Sprint 23). > **Scope:** Helm and Docker Compose deployment steps, ingress/TLS configuration, required environment variables, health checks, offline/air-gap operation, and compliance checklist (Sprint 23).
The StellaOps Console ships as part of the `stellaops` stack Helm chart and Compose bundles maintained under `deploy/`. This guide describes the supported deployment paths, the configuration surface, and operational checks needed to run the console in connected or air-gapped environments. The StellaOps Console ships as part of the `stellaops` stack Helm chart and Compose bundles maintained under `deploy/`. This guide describes the supported deployment paths, the configuration surface, and operational checks needed to run the console in connected or air-gapped environments.
--- ---
## 1. Prerequisites ## 1. Prerequisites
- Kubernetes cluster (v1.28+) with ingress controller (NGINX, Traefik, or equivalent) and Cert-Manager for automated TLS, or Docker host for Compose deployments. - Kubernetes cluster (v1.28+) with ingress controller (NGINX, Traefik, or equivalent) and Cert-Manager for automated TLS, or Docker host for Compose deployments.
- Container registry access to `registry.stella-ops.org` (or mirrored registry) for all images listed in `deploy/releases/*.yaml`. - Container registry access to `registry.stella-ops.org` (or mirrored registry) for all images listed in `deploy/releases/*.yaml`.
- Authority service configured with console client (`aud=ui`, scopes `ui.read`, `ui.admin`). - Authority service configured with console client (`aud=ui`, scopes `ui.read`, `ui.admin`).
- DNS entry pointing to the console hostname (for example, `console.acme.internal`). - DNS entry pointing to the console hostname (for example, `console.acme.internal`).
- Cosign public key for manifest verification (`deploy/releases/manifest.json.sig`). - Cosign public key for manifest verification (`deploy/releases/manifest.json.sig`).
- Optional: Offline Kit bundle for air-gapped sites (`stella-ops-offline-kit-<ver>.tar.gz`). - Optional: Offline Kit bundle for air-gapped sites (`stella-ops-offline-kit-<ver>.tar.gz`).
--- ---
## 2. Helm deployment (recommended) ## 2. Helm deployment (recommended)
### 2.1 Install chart repository ### 2.1 Install chart repository
```bash ```bash
helm repo add stellaops https://downloads.stella-ops.org/helm helm repo add stellaops https://downloads.stella-ops.org/helm
helm repo update stellaops helm repo update stellaops
``` ```
If operating offline, copy the chart archive from the Offline Kit (`deploy/helm/stellaops-<ver>.tgz`) and run: If operating offline, copy the chart archive from the Offline Kit (`deploy/helm/stellaops-<ver>.tgz`) and run:
```bash ```bash
helm install stellaops ./stellaops-<ver>.tgz --namespace stellaops --create-namespace helm install stellaops ./stellaops-<ver>.tgz --namespace stellaops --create-namespace
``` ```
### 2.2 Base installation ### 2.2 Base installation
```bash ```bash
helm install stellaops stellaops/stellaops \ helm install stellaops stellaops/stellaops \
--namespace stellaops \ --namespace stellaops \
--create-namespace \ --create-namespace \
--values deploy/helm/stellaops/values-prod.yaml --values deploy/helm/stellaops/values-prod.yaml
``` ```
The chart deploys Authority, Console web/API gateway, Scanner API, Scheduler, and supporting services. The console frontend pod is labelled `app=stellaops-web-ui`. The chart deploys Authority, Console web/API gateway, Scanner API, Scheduler, and supporting services. The console frontend pod is labelled `app=stellaops-web-ui`.
### 2.3 Helm values highlights ### 2.3 Helm values highlights
Key sections in `deploy/helm/stellaops/values-prod.yaml`: Key sections in `deploy/helm/stellaops/values-prod.yaml`:
| Path | Description | | Path | Description |
|------|-------------| |------|-------------|
| `console.ingress.host` | Hostname served by the console (`console.example.com`). | | `console.ingress.host` | Hostname served by the console (`console.example.com`). |
| `console.ingress.tls.secretName` | Kubernetes secret containing TLS certificate (generated by Cert-Manager or uploaded manually). | | `console.ingress.tls.secretName` | Kubernetes secret containing TLS certificate (generated by Cert-Manager or uploaded manually). |
| `console.config.apiGateway.baseUrl` | Internal base URL the UI uses to reach the gateway (defaults to `https://stellaops-web`). | | `console.config.apiGateway.baseUrl` | Internal base URL the UI uses to reach the gateway (defaults to `https://stellaops-web`). |
| `console.env.AUTHORITY_ISSUER` | Authority issuer URL (for example, `https://authority.example.com`). | | `console.env.AUTHORITY_ISSUER` | Authority issuer URL (for example, `https://authority.example.com`). |
| `console.env.AUTHORITY_CLIENT_ID` | Authority client ID for the console UI. | | `console.env.AUTHORITY_CLIENT_ID` | Authority client ID for the console UI. |
| `console.env.AUTHORITY_SCOPES` | Space-separated scopes required by UI (`ui.read ui.admin`). | | `console.env.AUTHORITY_SCOPES` | Space-separated scopes required by UI (`ui.read ui.admin`). |
| `console.resources` | CPU/memory requests and limits (default 250m CPU / 512Mi memory). | | `console.resources` | CPU/memory requests and limits (default 250m CPU / 512Mi memory). |
| `console.podAnnotations` | Optional annotations for service mesh or monitoring. | | `console.podAnnotations` | Optional annotations for service mesh or monitoring. |
Use `values-stage.yaml`, `values-dev.yaml`, or `values-airgap.yaml` as templates for other environments. Use `values-stage.yaml`, `values-dev.yaml`, or `values-airgap.yaml` as templates for other environments.
### 2.4 TLS and ingress ### 2.4 TLS and ingress
Example ingress override: Example ingress override:
```yaml ```yaml
console: console:
ingress: ingress:
enabled: true enabled: true
className: nginx className: nginx
host: console.acme.internal host: console.acme.internal
tls: tls:
enabled: true enabled: true
secretName: console-tls secretName: console-tls
``` ```
Generate certificates using Cert-Manager or provide an existing secret. For air-gapped deployments, pre-create the secret with the mirrored CA chain. Generate certificates using Cert-Manager or provide an existing secret. For air-gapped deployments, pre-create the secret with the mirrored CA chain.
### 2.5 Health checks ### 2.5 Health checks
Console pods expose: Console pods expose:
| Path | Purpose | Notes | | Path | Purpose | Notes |
|------|---------|-------| |------|---------|-------|
| `/health/live` | Liveness probe | Confirms process responsive. | | `/health/live` | Liveness probe | Confirms process responsive. |
| `/health/ready` | Readiness probe | Verifies configuration bootstrap and Authority reachability. | | `/health/ready` | Readiness probe | Verifies configuration bootstrap and Authority reachability. |
| `/metrics` | Prometheus metrics | Enabled when `console.metrics.enabled=true`. | | `/metrics` | Prometheus metrics | Enabled when `console.metrics.enabled=true`. |
Helm chart sets default probes (`initialDelaySeconds: 10`, `periodSeconds: 15`). Adjust via `console.livenessProbe` and `console.readinessProbe`. Helm chart sets default probes (`initialDelaySeconds: 10`, `periodSeconds: 15`). Adjust via `console.livenessProbe` and `console.readinessProbe`.
--- ---
## 3. Docker Compose deployment ## 3. Docker Compose deployment
Located in `deploy/compose/docker-compose.console.yaml`. Quick start: Located in `deploy/compose/docker-compose.console.yaml`. Quick start:
```bash ```bash
cd deploy/compose cd deploy/compose
docker compose -f docker-compose.console.yaml --env-file console.env up -d docker compose -f docker-compose.console.yaml --env-file console.env up -d
``` ```
`console.env` should define: `console.env` should define:
``` ```
CONSOLE_PUBLIC_BASE_URL=https://console.acme.internal CONSOLE_PUBLIC_BASE_URL=https://console.acme.internal
AUTHORITY_ISSUER=https://authority.acme.internal AUTHORITY_ISSUER=https://authority.acme.internal
AUTHORITY_CLIENT_ID=console-ui AUTHORITY_CLIENT_ID=console-ui
AUTHORITY_CLIENT_SECRET=<if using confidential client> AUTHORITY_CLIENT_SECRET=<if using confidential client>
AUTHORITY_SCOPES=ui.read ui.admin AUTHORITY_SCOPES=ui.read ui.admin
CONSOLE_GATEWAY_BASE_URL=https://api.acme.internal CONSOLE_GATEWAY_BASE_URL=https://api.acme.internal
``` ```
The compose bundle includes Traefik as reverse proxy with TLS termination. Update `traefik/dynamic/console.yml` for custom certificates or additional middlewares (CSP headers, rate limits). The compose bundle includes Traefik as reverse proxy with TLS termination. Update `traefik/dynamic/console.yml` for custom certificates or additional middlewares (CSP headers, rate limits).
--- ---
## 4. Environment variables ## 4. Environment variables
| Variable | Description | Default | | Variable | Description | Default |
|----------|-------------|---------| |----------|-------------|---------|
| `CONSOLE_PUBLIC_BASE_URL` | External URL used for redirects, deep links, and telemetry. | None (required). | | `CONSOLE_PUBLIC_BASE_URL` | External URL used for redirects, deep links, and telemetry. | None (required). |
| `CONSOLE_GATEWAY_BASE_URL` | URL of the web gateway that proxies API calls (`/console/*`). | Chart service name. | | `CONSOLE_GATEWAY_BASE_URL` | URL of the web gateway that proxies API calls (`/console/*`). | Chart service name. |
| `AUTHORITY_ISSUER` | Authority issuer (`https://authority.example.com`). | None (required). | | `AUTHORITY_ISSUER` | Authority issuer (`https://authority.example.com`). | None (required). |
| `AUTHORITY_CLIENT_ID` | OIDC client configured in Authority. | None (required). | | `AUTHORITY_CLIENT_ID` | OIDC client configured in Authority. | None (required). |
| `AUTHORITY_SCOPES` | Space-separated scopes assigned to the console client. | `ui.read ui.admin`. | | `AUTHORITY_SCOPES` | Space-separated scopes assigned to the console client. | `ui.read ui.admin`. |
| `AUTHORITY_DPOP_ENABLED` | Enables DPoP challenge/response (recommended true). | `true`. | | `AUTHORITY_DPOP_ENABLED` | Enables DPoP challenge/response (recommended true). | `true`. |
| `CONSOLE_FEATURE_FLAGS` | Comma-separated feature flags (`runs`, `downloads.offline`, etc.). | `runs,downloads,policies`. | | `CONSOLE_FEATURE_FLAGS` | Comma-separated feature flags (`runs`, `downloads.offline`, etc.). | `runs,downloads,policies`. |
| `CONSOLE_LOG_LEVEL` | Minimum log level (`Information`, `Debug`, etc.). | `Information`. | | `CONSOLE_LOG_LEVEL` | Minimum log level (`Information`, `Debug`, etc.). | `Information`. |
| `CONSOLE_METRICS_ENABLED` | Expose `/metrics` endpoint. | `true`. | | `CONSOLE_METRICS_ENABLED` | Expose `/metrics` endpoint. | `true`. |
| `CONSOLE_SENTRY_DSN` | Optional error reporting DSN. | Blank. | | `CONSOLE_SENTRY_DSN` | Optional error reporting DSN. | Blank. |
When running behind additional proxies, set `ASPNETCORE_FORWARDEDHEADERS_ENABLED=true` to honour `X-Forwarded-*` headers. When running behind additional proxies, set `ASPNETCORE_FORWARDEDHEADERS_ENABLED=true` to honour `X-Forwarded-*` headers.
--- ---
## 5. Security headers and CSP ## 5. Security headers and CSP
The console serves a strict Content Security Policy (CSP) by default: The console serves a strict Content Security Policy (CSP) by default:
``` ```
default-src 'self'; default-src 'self';
connect-src 'self' https://*.stella-ops.local; connect-src 'self' https://*.stella-ops.local;
script-src 'self'; script-src 'self';
style-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';
img-src 'self' data:; img-src 'self' data:;
font-src 'self'; font-src 'self';
frame-ancestors 'none'; frame-ancestors 'none';
``` ```
Adjust via `console.config.cspOverrides` if additional domains are required. For integrations embedding the console, update OIDC redirect URIs and Authority scopes accordingly. Adjust via `console.config.cspOverrides` if additional domains are required. For integrations embedding the console, update OIDC redirect URIs and Authority scopes accordingly.
TLS recommendations: TLS recommendations:
- Use TLS 1.2+ with modern cipher suite policy. - Use TLS 1.2+ with modern cipher suite policy.
- Enable HSTS (`Strict-Transport-Security: max-age=31536000; includeSubDomains`). - Enable HSTS (`Strict-Transport-Security: max-age=31536000; includeSubDomains`).
- Provide custom trust bundles via `console.config.trustBundleSecret` when using private CAs. - Provide custom trust bundles via `console.config.trustBundleSecret` when using private CAs.
--- ---
## 6. Logging and metrics ## 6. Logging and metrics
- Structured logs emitted to stdout with correlation IDs. Configure log shipping via Fluent Bit or similar. - Structured logs emitted to stdout with correlation IDs. Configure log shipping via Fluent Bit or similar.
- Metrics available at `/metrics` in Prometheus format. Key metrics include `ui_request_duration_seconds`, `ui_tenant_switch_total`, and `ui_download_manifest_refresh_seconds`. - Metrics available at `/metrics` in Prometheus format. Key metrics include `ui_request_duration_seconds`, `ui_tenant_switch_total`, and `ui_download_manifest_refresh_seconds`.
- Enable OpenTelemetry exporter by setting `OTEL_EXPORTER_OTLP_ENDPOINT` and associated headers in environment variables. - Enable OpenTelemetry exporter by setting `OTEL_EXPORTER_OTLP_ENDPOINT` and associated headers in environment variables.
--- ---
## 7. Offline and air-gap deployment ## 7. Offline and air-gap deployment
- Mirror container images using the Downloads workspace or Offline Kit manifest. Example: - Mirror container images using the Downloads workspace or Offline Kit manifest. Example:
```bash ```bash
oras copy registry.stella-ops.org/stellaops/web-ui@sha256:<digest> \ oras copy registry.stella-ops.org/stellaops/web-ui@sha256:<digest> \
registry.airgap.local/stellaops/web-ui:2025.10.0 registry.airgap.local/stellaops/web-ui:2025.10.0
``` ```
- Import Offline Kit using `stella ouk import` before starting the console so manifest parity checks succeed. - Import Offline Kit using `stella ouk import` before starting the console so manifest parity checks succeed.
- Use `values-airgap.yaml` to disable external telemetry endpoints and configure internal certificate chains. - Use `values-airgap.yaml` to disable external telemetry endpoints and configure internal certificate chains.
- Run `helm upgrade --install` using the mirrored chart (`stellaops-<ver>.tgz`) and set `console.offlineMode=true` to surface offline banners. - Run `helm upgrade --install` using the mirrored chart (`stellaops-<ver>.tgz`) and set `console.offlineMode=true` to surface offline banners.
--- ---
## 8. Health checks and remediation ## 8. Health checks and remediation
| Check | Command | Expected result | | Check | Command | Expected result |
|-------|---------|-----------------| |-------|---------|-----------------|
| Pod status | `kubectl get pods -n stellaops` | `Running` state with restarts = 0. | | Pod status | `kubectl get pods -n stellaops` | `Running` state with restarts = 0. |
| Liveness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/live` | Returns `{"status":"Healthy"}`. | | Liveness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/live` | Returns `{"status":"Healthy"}`. |
| Readiness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/ready` | Returns `{"status":"Ready"}`. | | Readiness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/ready` | Returns `{"status":"Ready"}`. |
| Gateway reachability | `curl -I https://console.example.com/api/console/status` | `200 OK` with CSP headers. | | Gateway reachability | `curl -I https://console.example.com/api/console/status` | `200 OK` with CSP headers. |
| Static assets | `curl -I https://console.example.com/static/assets/app.js` | `200 OK` with long cache headers. | | Static assets | `curl -I https://console.example.com/static/assets/app.js` | `200 OK` with long cache headers. |
Troubleshooting steps: Troubleshooting steps:
- **Authority unreachable:** readiness fails with `AUTHORITY_UNREACHABLE`. Check DNS, trust bundles, and Authority service health. - **Authority unreachable:** readiness fails with `AUTHORITY_UNREACHABLE`. Check DNS, trust bundles, and Authority service health.
- **Manifest mismatch:** console logs `DOWNLOAD_MANIFEST_SIGNATURE_INVALID`. Verify cosign key and re-sync manifest. - **Manifest mismatch:** console logs `DOWNLOAD_MANIFEST_SIGNATURE_INVALID`. Verify cosign key and re-sync manifest.
- **Ingress 404:** ensure ingress controller routes host to `stellaops-web-ui` service; check TLS secret name. - **Ingress 404:** ensure ingress controller routes host to `stellaops-web-ui` service; check TLS secret name.
- **SSE blocked:** confirm proxy allows HTTP/1.1 and disables buffering on `/console/runs/*`. - **SSE blocked:** confirm proxy allows HTTP/1.1 and disables buffering on `/console/runs/*`.
--- ---
## 9. References ## 9. References
- `deploy/helm/stellaops/values-*.yaml` - environment-specific overrides. - `deploy/helm/stellaops/values-*.yaml` - environment-specific overrides.
- `deploy/compose/docker-compose.console.yaml` - Compose bundle. - `deploy/compose/docker-compose.console.yaml` - Compose bundle.
- `/docs/ui/downloads.md` - manifest and offline bundle guidance. - `/docs/ui/downloads.md` - manifest and offline bundle guidance.
- `/docs/security/console-security.md` - CSP and Authority scopes. - `/docs/security/console-security.md` - CSP and Authority scopes.
- `/docs/24_OFFLINE_KIT.md` - Offline kit packaging and verification. - `/docs/24_OFFLINE_KIT.md` - Offline kit packaging and verification.
- `/docs/modules/devops/runbooks/deployment-runbook.md` (pending) - wider platform deployment steps. - `/docs/modules/devops/runbooks/deployment-runbook.md` (pending) - wider platform deployment steps.
--- ---
## 10. Compliance checklist ## 10. Compliance checklist
- [ ] Helm and Compose instructions verified against `deploy/` assets. - [ ] Helm and Compose instructions verified against `deploy/` assets.
- [ ] Ingress/TLS guidance aligns with Security Guild recommendations. - [ ] Ingress/TLS guidance aligns with Security Guild recommendations.
- [ ] Environment variables documented with defaults and required values. - [ ] Environment variables documented with defaults and required values.
- [ ] Health/liveness/readiness endpoints tested and listed. - [ ] Health/liveness/readiness endpoints tested and listed.
- [ ] Offline workflow (mirrors, manifest parity) captured. - [ ] Offline workflow (mirrors, manifest parity) captured.
- [ ] Logging and metrics surface documented metrics. - [ ] Logging and metrics surface documented metrics.
- [ ] CSP and security header defaults stated alongside override guidance. - [ ] CSP and security header defaults stated alongside override guidance.
- [ ] Troubleshooting section linked to relevant runbooks. - [ ] Troubleshooting section linked to relevant runbooks.
--- ---
*Last updated: 2025-10-27 (Sprint 23).* *Last updated: 2025-10-27 (Sprint 23).*

320
docs/deploy/containers.md
View File

@@ -1,160 +1,160 @@
# Container Deployment Guide — AOC Update # Container Deployment Guide — AOC Update
> **Audience:** DevOps Guild, platform operators deploying StellaOps services. > **Audience:** DevOps Guild, platform operators deploying StellaOps services.
> **Scope:** Deployment configuration changes required by the Aggregation-Only Contract (AOC), including schema validators, guard environment flags, and verifier identities. > **Scope:** Deployment configuration changes required by the Aggregation-Only Contract (AOC), including schema validators, guard environment flags, and verifier identities.
This guide supplements existing deployment manuals with AOC-specific configuration. It assumes familiarity with the base Compose/Helm manifests described in `ops/deployment/` and `docs/modules/devops/architecture.md`. This guide supplements existing deployment manuals with AOC-specific configuration. It assumes familiarity with the base Compose/Helm manifests described in `ops/deployment/` and `docs/modules/devops/architecture.md`.
--- ---
## 1 · Schema validator enablement ## 1 · Schema validator enablement
### 1.1 MongoDB validators ### 1.1 MongoDB validators
- Apply JSON schema validators to `advisory_raw` and `vex_raw` collections before enabling AOC guards. - Apply JSON schema validators to `advisory_raw` and `vex_raw` collections before enabling AOC guards.
- Before enabling validators or the idempotency index, run the duplicate audit helper to confirm no conflicting raw advisories remain: - Before enabling validators or the idempotency index, run the duplicate audit helper to confirm no conflicting raw advisories remain:
```bash ```bash
mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;' mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'
``` ```
Resolve any reported rows prior to rollout. Resolve any reported rows prior to rollout.
- Use the migration script provided in `ops/devops/scripts/apply-aoc-validators.js`: - Use the migration script provided in `ops/devops/scripts/apply-aoc-validators.js`:
```bash ```bash
kubectl exec -n concelier deploy/concelier-mongo -- \ kubectl exec -n concelier deploy/concelier-mongo -- \
mongo concelier ops/devops/scripts/apply-aoc-validators.js mongo concelier ops/devops/scripts/apply-aoc-validators.js
kubectl exec -n excititor deploy/excititor-mongo -- \ kubectl exec -n excititor deploy/excititor-mongo -- \
mongo excititor ops/devops/scripts/apply-aoc-validators.js mongo excititor ops/devops/scripts/apply-aoc-validators.js
``` ```
- Validators enforce required fields (`tenant`, `source`, `upstream`, `linkset`) and reject forbidden keys at DB level. - Validators enforce required fields (`tenant`, `source`, `upstream`, `linkset`) and reject forbidden keys at DB level.
- Rollback plan: validators are applied with `validationLevel: moderate`—downgrade via the same script with `--remove` if required. - Rollback plan: validators are applied with `validationLevel: moderate`—downgrade via the same script with `--remove` if required.
### 1.2 Migration order ### 1.2 Migration order
1. Deploy validators in maintenance window. 1. Deploy validators in maintenance window.
2. Roll out Concelier/Excititor images with guard middleware enabled (`AOC_GUARD_ENABLED=true`). 2. Roll out Concelier/Excititor images with guard middleware enabled (`AOC_GUARD_ENABLED=true`).
3. Run smoke tests (`stella sources ingest --dry-run` fixtures) before resuming production ingestion. 3. Run smoke tests (`stella sources ingest --dry-run` fixtures) before resuming production ingestion.
### 1.3 Supersedes backfill verification ### 1.3 Supersedes backfill verification
1. **Duplicate audit:** Confirm `mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'` reports no conflicts before restarting Concelier with the new migrations. 1. **Duplicate audit:** Confirm `mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'` reports no conflicts before restarting Concelier with the new migrations.
2. **Post-migration check:** After the service restarts, validate that `db.advisory` is a view pointing to `advisory_backup_20251028`: 2. **Post-migration check:** After the service restarts, validate that `db.advisory` is a view pointing to `advisory_backup_20251028`:
```bash ```bash
mongo concelier --quiet --eval 'db.getCollectionInfos({ name: "advisory" })[0]' mongo concelier --quiet --eval 'db.getCollectionInfos({ name: "advisory" })[0]'
``` ```
The `type` should be `"view"` and `options.viewOn` should equal `"advisory_backup_20251028"`. The `type` should be `"view"` and `options.viewOn` should equal `"advisory_backup_20251028"`.
3. **Supersedes chain spot-check:** Inspect a sample set to ensure deterministic chaining: 3. **Supersedes chain spot-check:** Inspect a sample set to ensure deterministic chaining:
```bash ```bash
mongo concelier --quiet --eval ' mongo concelier --quiet --eval '
db.advisory_raw.aggregate([ db.advisory_raw.aggregate([
{ $match: { "upstream.upstream_id": { $exists: true } } }, { $match: { "upstream.upstream_id": { $exists: true } } },
{ $sort: { "tenant": 1, "source.vendor": 1, "upstream.upstream_id": 1, "upstream.retrieved_at": 1 } }, { $sort: { "tenant": 1, "source.vendor": 1, "upstream.upstream_id": 1, "upstream.retrieved_at": 1 } },
{ $limit: 5 }, { $limit: 5 },
{ $project: { _id: 1, supersedes: 1 } } { $project: { _id: 1, supersedes: 1 } }
]).forEach(printjson)' ]).forEach(printjson)'
``` ```
Each revision should reference the previous `_id` (or `null` for the first revision). Record findings in the change ticket before proceeding to production. Each revision should reference the previous `_id` (or `null` for the first revision). Record findings in the change ticket before proceeding to production.
--- ---
## 2 · Container environment flags ## 2 · Container environment flags
Add the following environment variables to Concelier/Excititor deployments: Add the following environment variables to Concelier/Excititor deployments:
| Variable | Default | Description | | Variable | Default | Description |
|----------|---------|-------------| |----------|---------|-------------|
| `AOC_GUARD_ENABLED` | `true` | Enables `AOCWriteGuard` interception. Set `false` only for controlled rollback. | | `AOC_GUARD_ENABLED` | `true` | Enables `AOCWriteGuard` interception. Set `false` only for controlled rollback. |
| `AOC_ALLOW_SUPERSEDES_RETROFIT` | `false` | Allows temporary supersedes backfill during migration. Remove after cutover. | | `AOC_ALLOW_SUPERSEDES_RETROFIT` | `false` | Allows temporary supersedes backfill during migration. Remove after cutover. |
| `AOC_METRICS_ENABLED` | `true` | Emits `ingestion_write_total`, `aoc_violation_total`, etc. | | `AOC_METRICS_ENABLED` | `true` | Emits `ingestion_write_total`, `aoc_violation_total`, etc. |
| `AOC_TENANT_HEADER` | `X-Stella-Tenant` | Header name expected from Gateway. | | `AOC_TENANT_HEADER` | `X-Stella-Tenant` | Header name expected from Gateway. |
| `AOC_VERIFIER_USER` | `stella-aoc-verify` | Read-only service user used by UI/CLI verification. | | `AOC_VERIFIER_USER` | `stella-aoc-verify` | Read-only service user used by UI/CLI verification. |
Compose snippet: Compose snippet:
```yaml ```yaml
environment: environment:
- AOC_GUARD_ENABLED=true - AOC_GUARD_ENABLED=true
- AOC_ALLOW_SUPERSEDES_RETROFIT=false - AOC_ALLOW_SUPERSEDES_RETROFIT=false
- AOC_METRICS_ENABLED=true - AOC_METRICS_ENABLED=true
- AOC_TENANT_HEADER=X-Stella-Tenant - AOC_TENANT_HEADER=X-Stella-Tenant
- AOC_VERIFIER_USER=stella-aoc-verify - AOC_VERIFIER_USER=stella-aoc-verify
``` ```
Ensure `AOC_VERIFIER_USER` exists in Authority with `aoc:verify` scope and no write permissions. Ensure `AOC_VERIFIER_USER` exists in Authority with `aoc:verify` scope and no write permissions.
--- ---
## 3 · Verifier identity ## 3 · Verifier identity
- Create a dedicated client (`stella-aoc-verify`) via Authority bootstrap: - Create a dedicated client (`stella-aoc-verify`) via Authority bootstrap:
```yaml ```yaml
clients: clients:
- clientId: stella-aoc-verify - clientId: stella-aoc-verify
grantTypes: [client_credentials] grantTypes: [client_credentials]
scopes: [aoc:verify, advisory:read, vex:read] scopes: [aoc:verify, advisory:read, vex:read]
tenants: [default] tenants: [default]
``` ```
- Store credentials in secret store (`Kubernetes Secret`, `Docker swarm secret`). - Store credentials in secret store (`Kubernetes Secret`, `Docker swarm secret`).
- Bind credentials to `stella aoc verify` CI jobs and Console verification service. - Bind credentials to `stella aoc verify` CI jobs and Console verification service.
- Rotate quarterly; document in `ops/authority-key-rotation.md`. - Rotate quarterly; document in `ops/authority-key-rotation.md`.
--- ---
## 4 · Deployment steps ## 4 · Deployment steps
1. **Pre-checks:** Confirm database backups, alerting in maintenance mode, and staging environment validated. 1. **Pre-checks:** Confirm database backups, alerting in maintenance mode, and staging environment validated.
2. **Apply validators:** Run scripts per § 1.1. 2. **Apply validators:** Run scripts per § 1.1.
3. **Update manifests:** Inject environment variables (§ 2) and mount guard configuration configmaps. 3. **Update manifests:** Inject environment variables (§ 2) and mount guard configuration configmaps.
4. **Redeploy services:** Rolling restart Concelier/Excititor pods. Monitor `ingestion_write_total` for steady throughput. 4. **Redeploy services:** Rolling restart Concelier/Excititor pods. Monitor `ingestion_write_total` for steady throughput.
5. **Seed verifier:** Deploy read-only verifier user and store credentials. 5. **Seed verifier:** Deploy read-only verifier user and store credentials.
6. **Run verification:** Execute `stella aoc verify --since 24h` and ensure exit code `0`. 6. **Run verification:** Execute `stella aoc verify --since 24h` and ensure exit code `0`.
7. **Update dashboards:** Point Grafana panels to new metrics (`aoc_violation_total`). 7. **Update dashboards:** Point Grafana panels to new metrics (`aoc_violation_total`).
8. **Record handoff:** Capture console screenshots and verification logs for release notes. 8. **Record handoff:** Capture console screenshots and verification logs for release notes.
--- ---
## 5 · Offline Kit updates ## 5 · Offline Kit updates
- Ship validator scripts with Offline Kit (`offline-kit/scripts/apply-aoc-validators.js`). - Ship validator scripts with Offline Kit (`offline-kit/scripts/apply-aoc-validators.js`).
- Include pre-generated verification reports for air-gapped deployments. - Include pre-generated verification reports for air-gapped deployments.
- Document offline CLI workflow in bundle README referencing `docs/modules/cli/guides/cli-reference.md`. - Document offline CLI workflow in bundle README referencing `docs/modules/cli/guides/cli-reference.md`.
- Ensure `stella-aoc-verify` credentials are scoped to offline tenant and rotated during bundle refresh. - Ensure `stella-aoc-verify` credentials are scoped to offline tenant and rotated during bundle refresh.
--- ---
## 6 · Rollback plan ## 6 · Rollback plan
1. Disable guard via `AOC_GUARD_ENABLED=false` on Concelier/Excititor and rollout. 1. Disable guard via `AOC_GUARD_ENABLED=false` on Concelier/Excititor and rollout.
2. Remove validators with the migration script (`--remove`). 2. Remove validators with the migration script (`--remove`).
3. Pause verification jobs to prevent noise. 3. Pause verification jobs to prevent noise.
4. Investigate and remediate upstream issues before re-enabling guards. 4. Investigate and remediate upstream issues before re-enabling guards.
--- ---
## 7 · References ## 7 · References
- [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md) - [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md)
- [Authority scopes & tenancy](../security/authority-scopes.md) - [Authority scopes & tenancy](../security/authority-scopes.md)
- [Observability guide](../observability/observability.md) - [Observability guide](../observability/observability.md)
- [CLI AOC commands](../modules/cli/guides/cli-reference.md) - [CLI AOC commands](../modules/cli/guides/cli-reference.md)
- [Concelier architecture](../modules/concelier/architecture.md) - [Concelier architecture](../modules/concelier/architecture.md)
- [Excititor architecture](../modules/excititor/architecture.md) - [Excititor architecture](../modules/excititor/architecture.md)
--- ---
## 8 · Compliance checklist ## 8 · Compliance checklist
- [ ] Validators documented and scripts referenced for online/offline deployments. - [ ] Validators documented and scripts referenced for online/offline deployments.
- [ ] Environment variables cover guard enablement, metrics, and tenant header. - [ ] Environment variables cover guard enablement, metrics, and tenant header.
- [ ] Read-only verifier user installation steps included. - [ ] Read-only verifier user installation steps included.
- [ ] Offline kit instructions align with validator/verification workflow. - [ ] Offline kit instructions align with validator/verification workflow.
- [ ] Rollback procedure captured. - [ ] Rollback procedure captured.
- [ ] Cross-links to AOC docs, Authority scopes, and observability guides present. - [ ] Cross-links to AOC docs, Authority scopes, and observability guides present.
- [ ] DevOps Guild sign-off tracked (owner: @devops-guild, due 2025-10-29). - [ ] DevOps Guild sign-off tracked (owner: @devops-guild, due 2025-10-29).
--- ---
*Last updated: 2025-10-26 (Sprint 19).* *Last updated: 2025-10-26 (Sprint 19).*

440
docs/dev/30_EXCITITOR_CONNECTOR_GUIDE.md
View File

@@ -1,220 +1,220 @@
# Excititor Connector Packaging Guide # Excititor Connector Packaging Guide
> **Audience:** teams implementing new Excititor provider plug‑ins (CSAF feeds, > **Audience:** teams implementing new Excititor provider plug‑ins (CSAF feeds,
> OpenVEX attestations, etc.) > OpenVEX attestations, etc.)
> **Prerequisites:** read `docs/modules/excititor/architecture.md` and the module > **Prerequisites:** read `docs/modules/excititor/architecture.md` and the module
> `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`. > `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`.
The Excititor connector SDK gives you: The Excititor connector SDK gives you:
- `VexConnectorBase` – deterministic logging, SHA‑256 helpers, time provider. - `VexConnectorBase` – deterministic logging, SHA‑256 helpers, time provider.
- `VexConnectorOptionsBinder` – strongly typed YAML/JSON configuration binding. - `VexConnectorOptionsBinder` – strongly typed YAML/JSON configuration binding.
- `IVexConnectorOptionsValidator<T>` – custom validation hooks (offline defaults, auth invariants). - `IVexConnectorOptionsValidator<T>` – custom validation hooks (offline defaults, auth invariants).
- `VexConnectorDescriptor` & metadata helpers for consistent telemetry. - `VexConnectorDescriptor` & metadata helpers for consistent telemetry.
This guide explains how to package a connector so the Excititor Worker/WebService This guide explains how to package a connector so the Excititor Worker/WebService
can load it via the plugin host. can load it via the plugin host.
--- ---
## 1. Project layout ## 1. Project layout
Start from the template under Start from the template under
`docs/dev/templates/excititor-connector/`. It contains: `docs/dev/templates/excititor-connector/`. It contains:
``` ```
Excititor.MyConnector/ Excititor.MyConnector/
├── src/ ├── src/
│ ├── Excititor.MyConnector.csproj │ ├── Excititor.MyConnector.csproj
│ ├── MyConnectorOptions.cs │ ├── MyConnectorOptions.cs
│ ├── MyConnector.cs │ ├── MyConnector.cs
│ └── MyConnectorPlugin.cs │ └── MyConnectorPlugin.cs
└── manifest/ └── manifest/
└── connector.manifest.yaml └── connector.manifest.yaml
``` ```
Key points: Key points:
- Target `net10.0`, enable `TreatWarningsAsErrors`, reference the - Target `net10.0`, enable `TreatWarningsAsErrors`, reference the
`StellaOps.Excititor.Connectors.Abstractions` project (or NuGet once published). `StellaOps.Excititor.Connectors.Abstractions` project (or NuGet once published).
- Keep project ID prefix `StellaOps.Excititor.Connectors.<Provider>` so the - Keep project ID prefix `StellaOps.Excititor.Connectors.<Provider>` so the
plugin loader can discover it with the default search pattern. plugin loader can discover it with the default search pattern.
### 1.1 csproj snippet ### 1.1 csproj snippet
```xml ```xml
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>net10.0</TargetFramework> <TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable> <Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings> <ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors> <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\..\..\src\StellaOps.Excititor.Connectors.Abstractions\StellaOps.Excititor.Connectors.Abstractions.csproj" /> <ProjectReference Include="..\..\..\src\StellaOps.Excititor.Connectors.Abstractions\StellaOps.Excititor.Connectors.Abstractions.csproj" />
</ItemGroup> </ItemGroup>
</Project> </Project>
``` ```
Adjust the `ProjectReference` for your checkout (or switch to a NuGet package Adjust the `ProjectReference` for your checkout (or switch to a NuGet package
once published). once published).
--- ---
## 2. Implement the connector ## 2. Implement the connector
1. **Options model** – create an options POCO with data-annotation attributes. 1. **Options model** – create an options POCO with data-annotation attributes.
Bind it via `VexConnectorOptionsBinder.Bind<TOptions>` in your connector Bind it via `VexConnectorOptionsBinder.Bind<TOptions>` in your connector
constructor or `ValidateAsync`. constructor or `ValidateAsync`.
2. **Validator** – implement `IVexConnectorOptionsValidator<TOptions>` to add 2. **Validator** – implement `IVexConnectorOptionsValidator<TOptions>` to add
complex checks (e.g., ensure both `clientId` and `clientSecret` are present). complex checks (e.g., ensure both `clientId` and `clientSecret` are present).
3. **Connector** – inherit from `VexConnectorBase`. Implement: 3. **Connector** – inherit from `VexConnectorBase`. Implement:
- `ValidateAsync` – run binder/validators, log configuration summary. - `ValidateAsync` – run binder/validators, log configuration summary.
- `FetchAsync` – stream raw documents to `context.RawSink`. - `FetchAsync` – stream raw documents to `context.RawSink`.
- `NormalizeAsync` – convert raw documents into `VexClaimBatch` via - `NormalizeAsync` – convert raw documents into `VexClaimBatch` via
format-specific normalizers (`context.Normalizers`). format-specific normalizers (`context.Normalizers`).
4. **Plugin adapter** – expose the connector via a plugin entry point so the 4. **Plugin adapter** – expose the connector via a plugin entry point so the
host can instantiate it. host can instantiate it.
### 2.1 Options binding example ### 2.1 Options binding example
```csharp ```csharp
public sealed class MyConnectorOptions public sealed class MyConnectorOptions
{ {
[Required] [Required]
[Url] [Url]
public string CatalogUri { get; set; } = default!; public string CatalogUri { get; set; } = default!;
[Required] [Required]
public string ApiKey { get; set; } = default!; public string ApiKey { get; set; } = default!;
[Range(1, 64)] [Range(1, 64)]
public int MaxParallelRequests { get; set; } = 4; public int MaxParallelRequests { get; set; } = 4;
} }
public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions> public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions>
{ {
public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors) public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors)
{ {
if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase)) if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
{ {
errors.Add("CatalogUri must use HTTPS."); errors.Add("CatalogUri must use HTTPS.");
} }
} }
} }
``` ```
Bind inside the connector: Bind inside the connector:
```csharp ```csharp
private readonly MyConnectorOptions _options; private readonly MyConnectorOptions _options;
public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider) public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider)
: base(descriptor, logger, timeProvider) : base(descriptor, logger, timeProvider)
{ {
// `settings` comes from the orchestrator; validators registered via DI. // `settings` comes from the orchestrator; validators registered via DI.
_options = VexConnectorOptionsBinder.Bind<MyConnectorOptions>( _options = VexConnectorOptionsBinder.Bind<MyConnectorOptions>(
descriptor, descriptor,
VexConnectorSettings.Empty, VexConnectorSettings.Empty,
validators: new[] { new MyConnectorOptionsValidator() }); validators: new[] { new MyConnectorOptionsValidator() });
} }
``` ```
Replace `VexConnectorSettings.Empty` with the actual settings from context Replace `VexConnectorSettings.Empty` with the actual settings from context
inside `ValidateAsync`. inside `ValidateAsync`.
--- ---
## 3. Plugin adapter & manifest ## 3. Plugin adapter & manifest
Create a simple plugin class that implements Create a simple plugin class that implements
`StellaOps.Plugin.IConnectorPlugin`. The Worker/WebService plugin host uses `StellaOps.Plugin.IConnectorPlugin`. The Worker/WebService plugin host uses
this contract today. this contract today.
```csharp ```csharp
public sealed class MyConnectorPlugin : IConnectorPlugin public sealed class MyConnectorPlugin : IConnectorPlugin
{ {
private static readonly VexConnectorDescriptor Descriptor = private static readonly VexConnectorDescriptor Descriptor =
new("excititor:my-provider", VexProviderKind.Vendor, "My Provider VEX"); new("excititor:my-provider", VexProviderKind.Vendor, "My Provider VEX");
public string Name => Descriptor.DisplayName; public string Name => Descriptor.DisplayName;
public bool IsAvailable(IServiceProvider services) => true; // inject feature flags if needed public bool IsAvailable(IServiceProvider services) => true; // inject feature flags if needed
public IFeedConnector Create(IServiceProvider services) public IFeedConnector Create(IServiceProvider services)
{ {
var logger = services.GetRequiredService<ILogger<MyConnector>>(); var logger = services.GetRequiredService<ILogger<MyConnector>>();
var timeProvider = services.GetRequiredService<TimeProvider>(); var timeProvider = services.GetRequiredService<TimeProvider>();
return new MyConnector(Descriptor, logger, timeProvider); return new MyConnector(Descriptor, logger, timeProvider);
} }
} }
``` ```
> **Note:** the Excititor Worker currently instantiates connectors through the > **Note:** the Excititor Worker currently instantiates connectors through the
> shared `IConnectorPlugin` contract. Once a dedicated Excititor plugin interface > shared `IConnectorPlugin` contract. Once a dedicated Excititor plugin interface
> lands you simply swap the base interface; the descriptor/connector code > lands you simply swap the base interface; the descriptor/connector code
> remains unchanged. > remains unchanged.
Provide a manifest describing the assembly for operational tooling: Provide a manifest describing the assembly for operational tooling:
```yaml ```yaml
# manifest/connector.manifest.yaml # manifest/connector.manifest.yaml
id: excititor-my-provider id: excititor-my-provider
assembly: StellaOps.Excititor.Connectors.MyProvider.dll assembly: StellaOps.Excititor.Connectors.MyProvider.dll
entryPoint: StellaOps.Excititor.Connectors.MyProvider.MyConnectorPlugin entryPoint: StellaOps.Excititor.Connectors.MyProvider.MyConnectorPlugin
description: > description: >
Official VEX feed for ExampleCorp products (CSAF JSON, daily updates). Official VEX feed for ExampleCorp products (CSAF JSON, daily updates).
tags: tags:
- excititor - excititor
- csaf - csaf
- vendor - vendor
``` ```
Store manifests under `/opt/stella/excititor/plugins/<connector>/manifest/` in Store manifests under `/opt/stella/excititor/plugins/<connector>/manifest/` in
production so the deployment tooling can inventory and verify plug‑ins. production so the deployment tooling can inventory and verify plug‑ins.
--- ---
## 4. Packaging workflow ## 4. Packaging workflow
1. `dotnet publish -c Release` → copy the published DLLs to 1. `dotnet publish -c Release` → copy the published DLLs to
`/opt/stella/excititor/plugins/<Provider>/`. `/opt/stella/excititor/plugins/<Provider>/`.
2. Place `connector.manifest.yaml` next to the binaries. 2. Place `connector.manifest.yaml` next to the binaries.
3. Restart the Excititor Worker or WebService (hot reload not supported yet). 3. Restart the Excititor Worker or WebService (hot reload not supported yet).
4. Verify logs: `VEX-ConnectorLoader` should list the connector descriptor. 4. Verify logs: `VEX-ConnectorLoader` should list the connector descriptor.
### 4.1 Offline kits ### 4.1 Offline kits
- Add the connector folder (binaries + manifest) to the Offline Kit bundle. - Add the connector folder (binaries + manifest) to the Offline Kit bundle.
- Include a `settings.sample.yaml` demonstrating offline-friendly defaults. - Include a `settings.sample.yaml` demonstrating offline-friendly defaults.
- Document any external dependencies (e.g., SHA mirrors) in the manifest `notes` - Document any external dependencies (e.g., SHA mirrors) in the manifest `notes`
field. field.
--- ---
## 5. Testing checklist ## 5. Testing checklist
- Unit tests around options binding & validators. - Unit tests around options binding & validators.
- Integration tests (future `StellaOps.Excititor.Connectors.Abstractions.Tests`) - Integration tests (future `StellaOps.Excititor.Connectors.Abstractions.Tests`)
verifying deterministic logging scopes: verifying deterministic logging scopes:
`logger.BeginScope` should produce `vex.connector.id`, `vex.connector.kind`, `logger.BeginScope` should produce `vex.connector.id`, `vex.connector.kind`,
and `vex.connector.operation`. and `vex.connector.operation`.
- Deterministic SHA tests: repeated `CreateRawDocument` calls with identical - Deterministic SHA tests: repeated `CreateRawDocument` calls with identical
content must return the same digest. content must return the same digest.
--- ---
## 6. Reference template ## 6. Reference template
See `docs/dev/templates/excititor-connector/` for the full quick‑start including: See `docs/dev/templates/excititor-connector/` for the full quick‑start including:
- Sample options class + validator. - Sample options class + validator.
- Connector implementation inheriting from `VexConnectorBase`. - Connector implementation inheriting from `VexConnectorBase`.
- Plugin adapter + manifest. - Plugin adapter + manifest.
Copy the directory, rename namespaces/IDs, then iterate on provider-specific Copy the directory, rename namespaces/IDs, then iterate on provider-specific
logic. logic.
--- ---
*Last updated: 2025-10-17* *Last updated: 2025-10-17*

440
docs/dev/30_VEXER_CONNECTOR_GUIDE.md
View File

@@ -1,220 +1,220 @@
# Vexer Connector Packaging Guide # Vexer Connector Packaging Guide
> **Audience:** teams implementing new Vexer provider plug‑ins (CSAF feeds, > **Audience:** teams implementing new Vexer provider plug‑ins (CSAF feeds,
> OpenVEX attestations, etc.) > OpenVEX attestations, etc.)
> **Prerequisites:** read `docs/modules/vexer/architecture.md` and the module > **Prerequisites:** read `docs/modules/vexer/architecture.md` and the module
> `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`. > `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`.
The Vexer connector SDK gives you: The Vexer connector SDK gives you:
- `VexConnectorBase` – deterministic logging, SHA‑256 helpers, time provider. - `VexConnectorBase` – deterministic logging, SHA‑256 helpers, time provider.
- `VexConnectorOptionsBinder` – strongly typed YAML/JSON configuration binding. - `VexConnectorOptionsBinder` – strongly typed YAML/JSON configuration binding.
- `IVexConnectorOptionsValidator<T>` – custom validation hooks (offline defaults, auth invariants). - `IVexConnectorOptionsValidator<T>` – custom validation hooks (offline defaults, auth invariants).
- `VexConnectorDescriptor` & metadata helpers for consistent telemetry. - `VexConnectorDescriptor` & metadata helpers for consistent telemetry.
This guide explains how to package a connector so the Vexer Worker/WebService This guide explains how to package a connector so the Vexer Worker/WebService
can load it via the plugin host. can load it via the plugin host.
--- ---
## 1. Project layout ## 1. Project layout
Start from the template under Start from the template under
`docs/dev/templates/vexer-connector/`. It contains: `docs/dev/templates/vexer-connector/`. It contains:
``` ```
Vexer.MyConnector/ Vexer.MyConnector/
├── src/ ├── src/
│ ├── Vexer.MyConnector.csproj │ ├── Vexer.MyConnector.csproj
│ ├── MyConnectorOptions.cs │ ├── MyConnectorOptions.cs
│ ├── MyConnector.cs │ ├── MyConnector.cs
│ └── MyConnectorPlugin.cs │ └── MyConnectorPlugin.cs
└── manifest/ └── manifest/
└── connector.manifest.yaml └── connector.manifest.yaml
``` ```
Key points: Key points:
- Target `net10.0`, enable `TreatWarningsAsErrors`, reference the - Target `net10.0`, enable `TreatWarningsAsErrors`, reference the
`StellaOps.Vexer.Connectors.Abstractions` project (or NuGet once published). `StellaOps.Vexer.Connectors.Abstractions` project (or NuGet once published).
- Keep project ID prefix `StellaOps.Vexer.Connectors.<Provider>` so the - Keep project ID prefix `StellaOps.Vexer.Connectors.<Provider>` so the
plugin loader can discover it with the default search pattern. plugin loader can discover it with the default search pattern.
### 1.1 csproj snippet ### 1.1 csproj snippet
```xml ```xml
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>net10.0</TargetFramework> <TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable> <Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings> <ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors> <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\..\..\src\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> <ProjectReference Include="..\..\..\src\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
</ItemGroup> </ItemGroup>
</Project> </Project>
``` ```
Adjust the `ProjectReference` for your checkout (or switch to a NuGet package Adjust the `ProjectReference` for your checkout (or switch to a NuGet package
once published). once published).
--- ---
## 2. Implement the connector ## 2. Implement the connector
1. **Options model** – create an options POCO with data-annotation attributes. 1. **Options model** – create an options POCO with data-annotation attributes.
Bind it via `VexConnectorOptionsBinder.Bind<TOptions>` in your connector Bind it via `VexConnectorOptionsBinder.Bind<TOptions>` in your connector
constructor or `ValidateAsync`. constructor or `ValidateAsync`.
2. **Validator** – implement `IVexConnectorOptionsValidator<TOptions>` to add 2. **Validator** – implement `IVexConnectorOptionsValidator<TOptions>` to add
complex checks (e.g., ensure both `clientId` and `clientSecret` are present). complex checks (e.g., ensure both `clientId` and `clientSecret` are present).
3. **Connector** – inherit from `VexConnectorBase`. Implement: 3. **Connector** – inherit from `VexConnectorBase`. Implement:
- `ValidateAsync` – run binder/validators, log configuration summary. - `ValidateAsync` – run binder/validators, log configuration summary.
- `FetchAsync` – stream raw documents to `context.RawSink`. - `FetchAsync` – stream raw documents to `context.RawSink`.
- `NormalizeAsync` – convert raw documents into `VexClaimBatch` via - `NormalizeAsync` – convert raw documents into `VexClaimBatch` via
format-specific normalizers (`context.Normalizers`). format-specific normalizers (`context.Normalizers`).
4. **Plugin adapter** – expose the connector via a plugin entry point so the 4. **Plugin adapter** – expose the connector via a plugin entry point so the
host can instantiate it. host can instantiate it.
### 2.1 Options binding example ### 2.1 Options binding example
```csharp ```csharp
public sealed class MyConnectorOptions public sealed class MyConnectorOptions
{ {
[Required] [Required]
[Url] [Url]
public string CatalogUri { get; set; } = default!; public string CatalogUri { get; set; } = default!;
[Required] [Required]
public string ApiKey { get; set; } = default!; public string ApiKey { get; set; } = default!;
[Range(1, 64)] [Range(1, 64)]
public int MaxParallelRequests { get; set; } = 4; public int MaxParallelRequests { get; set; } = 4;
} }
public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions> public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions>
{ {
public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors) public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors)
{ {
if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase)) if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
{ {
errors.Add("CatalogUri must use HTTPS."); errors.Add("CatalogUri must use HTTPS.");
} }
} }
} }
``` ```
Bind inside the connector: Bind inside the connector:
```csharp ```csharp
private readonly MyConnectorOptions _options; private readonly MyConnectorOptions _options;
public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider) public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider)
: base(descriptor, logger, timeProvider) : base(descriptor, logger, timeProvider)
{ {
// `settings` comes from the orchestrator; validators registered via DI. // `settings` comes from the orchestrator; validators registered via DI.
_options = VexConnectorOptionsBinder.Bind<MyConnectorOptions>( _options = VexConnectorOptionsBinder.Bind<MyConnectorOptions>(
descriptor, descriptor,
VexConnectorSettings.Empty, VexConnectorSettings.Empty,
validators: new[] { new MyConnectorOptionsValidator() }); validators: new[] { new MyConnectorOptionsValidator() });
} }
``` ```
Replace `VexConnectorSettings.Empty` with the actual settings from context Replace `VexConnectorSettings.Empty` with the actual settings from context
inside `ValidateAsync`. inside `ValidateAsync`.
--- ---
## 3. Plugin adapter & manifest ## 3. Plugin adapter & manifest
Create a simple plugin class that implements Create a simple plugin class that implements
`StellaOps.Plugin.IConnectorPlugin`. The Worker/WebService plugin host uses `StellaOps.Plugin.IConnectorPlugin`. The Worker/WebService plugin host uses
this contract today. this contract today.
```csharp ```csharp
public sealed class MyConnectorPlugin : IConnectorPlugin public sealed class MyConnectorPlugin : IConnectorPlugin
{ {
private static readonly VexConnectorDescriptor Descriptor = private static readonly VexConnectorDescriptor Descriptor =
new("vexer:my-provider", VexProviderKind.Vendor, "My Provider VEX"); new("vexer:my-provider", VexProviderKind.Vendor, "My Provider VEX");
public string Name => Descriptor.DisplayName; public string Name => Descriptor.DisplayName;
public bool IsAvailable(IServiceProvider services) => true; // inject feature flags if needed public bool IsAvailable(IServiceProvider services) => true; // inject feature flags if needed
public IFeedConnector Create(IServiceProvider services) public IFeedConnector Create(IServiceProvider services)
{ {
var logger = services.GetRequiredService<ILogger<MyConnector>>(); var logger = services.GetRequiredService<ILogger<MyConnector>>();
var timeProvider = services.GetRequiredService<TimeProvider>(); var timeProvider = services.GetRequiredService<TimeProvider>();
return new MyConnector(Descriptor, logger, timeProvider); return new MyConnector(Descriptor, logger, timeProvider);
} }
} }
``` ```
> **Note:** the Vexer Worker currently instantiates connectors through the > **Note:** the Vexer Worker currently instantiates connectors through the
> shared `IConnectorPlugin` contract. Once a dedicated Vexer plugin interface > shared `IConnectorPlugin` contract. Once a dedicated Vexer plugin interface
> lands you simply swap the base interface; the descriptor/connector code > lands you simply swap the base interface; the descriptor/connector code
> remains unchanged. > remains unchanged.
Provide a manifest describing the assembly for operational tooling: Provide a manifest describing the assembly for operational tooling:
```yaml ```yaml
# manifest/connector.manifest.yaml # manifest/connector.manifest.yaml
id: vexer-my-provider id: vexer-my-provider
assembly: StellaOps.Vexer.Connectors.MyProvider.dll assembly: StellaOps.Vexer.Connectors.MyProvider.dll
entryPoint: StellaOps.Vexer.Connectors.MyProvider.MyConnectorPlugin entryPoint: StellaOps.Vexer.Connectors.MyProvider.MyConnectorPlugin
description: > description: >
Official VEX feed for ExampleCorp products (CSAF JSON, daily updates). Official VEX feed for ExampleCorp products (CSAF JSON, daily updates).
tags: tags:
- vexer - vexer
- csaf - csaf
- vendor - vendor
``` ```
Store manifests under `/opt/stella/vexer/plugins/<connector>/manifest/` in Store manifests under `/opt/stella/vexer/plugins/<connector>/manifest/` in
production so the deployment tooling can inventory and verify plug‑ins. production so the deployment tooling can inventory and verify plug‑ins.
--- ---
## 4. Packaging workflow ## 4. Packaging workflow
1. `dotnet publish -c Release` → copy the published DLLs to 1. `dotnet publish -c Release` → copy the published DLLs to
`/opt/stella/vexer/plugins/<Provider>/`. `/opt/stella/vexer/plugins/<Provider>/`.
2. Place `connector.manifest.yaml` next to the binaries. 2. Place `connector.manifest.yaml` next to the binaries.
3. Restart the Vexer Worker or WebService (hot reload not supported yet). 3. Restart the Vexer Worker or WebService (hot reload not supported yet).
4. Verify logs: `VEX-ConnectorLoader` should list the connector descriptor. 4. Verify logs: `VEX-ConnectorLoader` should list the connector descriptor.
### 4.1 Offline kits ### 4.1 Offline kits
- Add the connector folder (binaries + manifest) to the Offline Kit bundle. - Add the connector folder (binaries + manifest) to the Offline Kit bundle.
- Include a `settings.sample.yaml` demonstrating offline-friendly defaults. - Include a `settings.sample.yaml` demonstrating offline-friendly defaults.
- Document any external dependencies (e.g., SHA mirrors) in the manifest `notes` - Document any external dependencies (e.g., SHA mirrors) in the manifest `notes`
field. field.
--- ---
## 5. Testing checklist ## 5. Testing checklist
- Unit tests around options binding & validators. - Unit tests around options binding & validators.
- Integration tests (future `StellaOps.Vexer.Connectors.Abstractions.Tests`) - Integration tests (future `StellaOps.Vexer.Connectors.Abstractions.Tests`)
verifying deterministic logging scopes: verifying deterministic logging scopes:
`logger.BeginScope` should produce `vex.connector.id`, `vex.connector.kind`, `logger.BeginScope` should produce `vex.connector.id`, `vex.connector.kind`,
and `vex.connector.operation`. and `vex.connector.operation`.
- Deterministic SHA tests: repeated `CreateRawDocument` calls with identical - Deterministic SHA tests: repeated `CreateRawDocument` calls with identical
content must return the same digest. content must return the same digest.
--- ---
## 6. Reference template ## 6. Reference template
See `docs/dev/templates/vexer-connector/` for the full quick‑start including: See `docs/dev/templates/vexer-connector/` for the full quick‑start including:
- Sample options class + validator. - Sample options class + validator.
- Connector implementation inheriting from `VexConnectorBase`. - Connector implementation inheriting from `VexConnectorBase`.
- Plugin adapter + manifest. - Plugin adapter + manifest.
Copy the directory, rename namespaces/IDs, then iterate on provider-specific Copy the directory, rename namespaces/IDs, then iterate on provider-specific
logic. logic.
--- ---
*Last updated: 2025-10-17* *Last updated: 2025-10-17*

424
docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md
View File

@@ -1,212 +1,212 @@
# Authority Plug-in Developer Guide # Authority Plug-in Developer Guide
> **Status:** Updated 2025-10-11 (AUTHPLUG-DOCS-01-001) with lifecycle + limiter diagrams and refreshed rate-limit guidance aligned to PLG6 acceptance criteria. > **Status:** Updated 2025-10-11 (AUTHPLUG-DOCS-01-001) with lifecycle + limiter diagrams and refreshed rate-limit guidance aligned to PLG6 acceptance criteria.
## 1. Overview ## 1. Overview
Authority plug-ins extend the **StellaOps Authority** service with custom identity providers, credential stores, and client-management logic. Unlike Concelier plug-ins (which ingest or export advisories), Authority plug-ins participate directly in authentication flows: Authority plug-ins extend the **StellaOps Authority** service with custom identity providers, credential stores, and client-management logic. Unlike Concelier plug-ins (which ingest or export advisories), Authority plug-ins participate directly in authentication flows:
- **Use cases:** integrate corporate directories (LDAP/AD)[^ldap-rfc], delegate to external IDPs, enforce bespoke password/lockout policies, or add client provisioning automation. - **Use cases:** integrate corporate directories (LDAP/AD)[^ldap-rfc], delegate to external IDPs, enforce bespoke password/lockout policies, or add client provisioning automation.
- **Constraints:** plug-ins load only during service start (no hot-reload), must function without outbound internet access, and must emit deterministic results for identical configuration input. - **Constraints:** plug-ins load only during service start (no hot-reload), must function without outbound internet access, and must emit deterministic results for identical configuration input.
- **Ship targets:** build against the host’s .NET 10 preview SDK, honour offline-first requirements, and surface actionable diagnostics so operators can triage issues from `/ready`. - **Ship targets:** build against the host’s .NET 10 preview SDK, honour offline-first requirements, and surface actionable diagnostics so operators can triage issues from `/ready`.
## 2. Architecture Snapshot ## 2. Architecture Snapshot
Authority hosts follow a deterministic plug-in lifecycle. The exported diagram (`docs/assets/authority/authority-plugin-lifecycle.svg`) mirrors the steps below; regenerate it from the Mermaid source if you update the flow. Authority hosts follow a deterministic plug-in lifecycle. The exported diagram (`docs/assets/authority/authority-plugin-lifecycle.svg`) mirrors the steps below; regenerate it from the Mermaid source if you update the flow.
1. **Configuration load** – `AuthorityPluginConfigurationLoader` resolves YAML manifests under `etc/authority.plugins/`. 1. **Configuration load** – `AuthorityPluginConfigurationLoader` resolves YAML manifests under `etc/authority.plugins/`.
2. **Assembly discovery** – the shared `PluginHost` scans `StellaOps.Authority.PluginBinaries` for `StellaOps.Authority.Plugin.*.dll` assemblies. 2. **Assembly discovery** – the shared `PluginHost` scans `StellaOps.Authority.PluginBinaries` for `StellaOps.Authority.Plugin.*.dll` assemblies.
3. **Registrar execution** – each assembly is searched for `IAuthorityPluginRegistrar` implementations. Registrars bind options, register services, and optionally queue bootstrap tasks. 3. **Registrar execution** – each assembly is searched for `IAuthorityPluginRegistrar` implementations. Registrars bind options, register services, and optionally queue bootstrap tasks.
4. **Runtime** – the host resolves `IIdentityProviderPlugin` instances, uses capability metadata to decide which OAuth grants to expose, and invokes health checks for readiness endpoints. 4. **Runtime** – the host resolves `IIdentityProviderPlugin` instances, uses capability metadata to decide which OAuth grants to expose, and invokes health checks for readiness endpoints.
![Authority plug-in lifecycle diagram](../assets/authority/authority-plugin-lifecycle.svg) ![Authority plug-in lifecycle diagram](../assets/authority/authority-plugin-lifecycle.svg)
_Source:_ `docs/assets/authority/authority-plugin-lifecycle.mmd` _Source:_ `docs/assets/authority/authority-plugin-lifecycle.mmd`
**Data persistence primer:** the standard Mongo-backed plugin stores users in collections named `authority_users_<pluginName>` and lockout metadata in embedded documents. Additional plugins must document their storage layout and provide deterministic collection naming to honour the Offline Kit replication process. **Data persistence primer:** the standard Mongo-backed plugin stores users in collections named `authority_users_<pluginName>` and lockout metadata in embedded documents. Additional plugins must document their storage layout and provide deterministic collection naming to honour the Offline Kit replication process.
## 3. Capability Metadata ## 3. Capability Metadata
Capability flags let the host reason about what your plug-in supports: Capability flags let the host reason about what your plug-in supports:
- Declare capabilities in your descriptor using the string constants from `AuthorityPluginCapabilities` (`password`, `mfa`, `clientProvisioning`, `bootstrap`). The configuration loader now validates these tokens and rejects unknown values at startup. - Declare capabilities in your descriptor using the string constants from `AuthorityPluginCapabilities` (`password`, `mfa`, `clientProvisioning`, `bootstrap`). The configuration loader now validates these tokens and rejects unknown values at startup.
- `AuthorityIdentityProviderCapabilities.FromCapabilities` projects those strings into strongly typed booleans (`SupportsPassword`, etc.). Authority Core will use these flags when wiring flows such as the password grant. Built-in plugins (e.g., Standard) will fail fast or force-enable required capabilities if the descriptor is misconfigured, so keep manifests accurate. - `AuthorityIdentityProviderCapabilities.FromCapabilities` projects those strings into strongly typed booleans (`SupportsPassword`, etc.). Authority Core will use these flags when wiring flows such as the password grant. Built-in plugins (e.g., Standard) will fail fast or force-enable required capabilities if the descriptor is misconfigured, so keep manifests accurate.
- Typical configuration (`etc/authority.plugins/standard.yaml`): - Typical configuration (`etc/authority.plugins/standard.yaml`):
```yaml ```yaml
plugins: plugins:
descriptors: descriptors:
standard: standard:
assemblyName: "StellaOps.Authority.Plugin.Standard" assemblyName: "StellaOps.Authority.Plugin.Standard"
capabilities: capabilities:
- password - password
- bootstrap - bootstrap
``` ```
- Only declare a capability if the plug-in genuinely implements it. For example, if `SupportsClientProvisioning` is `true`, the plug-in must supply a working `IClientProvisioningStore`. - Only declare a capability if the plug-in genuinely implements it. For example, if `SupportsClientProvisioning` is `true`, the plug-in must supply a working `IClientProvisioningStore`.
**Operational reminder:** the Authority host surfaces capability summaries during startup (see `AuthorityIdentityProviderRegistry` log lines). Use those logs during smoke tests to ensure manifests align with expectations. **Operational reminder:** the Authority host surfaces capability summaries during startup (see `AuthorityIdentityProviderRegistry` log lines). Use those logs during smoke tests to ensure manifests align with expectations.
**Configuration path normalisation:** Manifest-relative paths (e.g., `tokenSigning.keyDirectory: "../keys"`) are resolved against the YAML file location and environment variables are expanded before validation. Plug-ins should expect to receive an absolute, canonical path when options are injected. **Configuration path normalisation:** Manifest-relative paths (e.g., `tokenSigning.keyDirectory: "../keys"`) are resolved against the YAML file location and environment variables are expanded before validation. Plug-ins should expect to receive an absolute, canonical path when options are injected.
**Password policy guardrails:** The Standard registrar logs a warning when a plug-in weakens the default password policy (minimum length or required character classes). Keep overrides at least as strong as the compiled defaults—operators treat the warning as an actionable security deviation. **Password policy guardrails:** The Standard registrar logs a warning when a plug-in weakens the default password policy (minimum length or required character classes). Keep overrides at least as strong as the compiled defaults—operators treat the warning as an actionable security deviation.
## 4. Project Scaffold ## 4. Project Scaffold
- Target **.NET 10 preview**, enable nullable, treat warnings as errors, and mark Authority plug-ins with `<IsAuthorityPlugin>true</IsAuthorityPlugin>`. - Target **.NET 10 preview**, enable nullable, treat warnings as errors, and mark Authority plug-ins with `<IsAuthorityPlugin>true</IsAuthorityPlugin>`.
- Minimum references: - Minimum references:
- `StellaOps.Authority.Plugins.Abstractions` (contracts & capability helpers) - `StellaOps.Authority.Plugins.Abstractions` (contracts & capability helpers)
- `StellaOps.Plugin` (hosting/DI helpers) - `StellaOps.Plugin` (hosting/DI helpers)
- `StellaOps.Auth.*` libraries as needed for shared token utilities (optional today). - `StellaOps.Auth.*` libraries as needed for shared token utilities (optional today).
- Example `.csproj` (trimmed from `StellaOps.Authority.Plugin.Standard`): - Example `.csproj` (trimmed from `StellaOps.Authority.Plugin.Standard`):
```xml ```xml
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFramework>net10.0</TargetFramework> <TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable> <Nullable>enable</Nullable>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors> <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<IsAuthorityPlugin>true</IsAuthorityPlugin> <IsAuthorityPlugin>true</IsAuthorityPlugin>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\StellaOps.Authority.Plugins.Abstractions\StellaOps.Authority.Plugins.Abstractions.csproj" /> <ProjectReference Include="..\StellaOps.Authority.Plugins.Abstractions\StellaOps.Authority.Plugins.Abstractions.csproj" />
<ProjectReference Include="..\..\StellaOps.Plugin\StellaOps.Plugin.csproj" /> <ProjectReference Include="..\..\StellaOps.Plugin\StellaOps.Plugin.csproj" />
</ItemGroup> </ItemGroup>
</Project> </Project>
``` ```
(Add other references—e.g., MongoDB driver, shared auth libraries—according to your implementation.) (Add other references—e.g., MongoDB driver, shared auth libraries—according to your implementation.)
## 5. Implementing `IAuthorityPluginRegistrar` ## 5. Implementing `IAuthorityPluginRegistrar`
- Create a parameterless registrar class that returns your plug-in type name via `PluginType`. - Create a parameterless registrar class that returns your plug-in type name via `PluginType`.
- Use `AuthorityPluginRegistrationContext` to: - Use `AuthorityPluginRegistrationContext` to:
- Bind options (`AddOptions<T>(pluginName).Bind(...)`). - Bind options (`AddOptions<T>(pluginName).Bind(...)`).
- Register singletons for stores/enrichers using manifest metadata. - Register singletons for stores/enrichers using manifest metadata.
- Register any hosted bootstrap tasks (e.g., seed admin users). - Register any hosted bootstrap tasks (e.g., seed admin users).
- Always validate configuration inside `PostConfigure` and throw meaningful `InvalidOperationException` to fail fast during startup. - Always validate configuration inside `PostConfigure` and throw meaningful `InvalidOperationException` to fail fast during startup.
- Use the provided `ILoggerFactory` from DI; avoid static loggers or console writes. - Use the provided `ILoggerFactory` from DI; avoid static loggers or console writes.
- Example skeleton: - Example skeleton:
```csharp ```csharp
internal sealed class MyPluginRegistrar : IAuthorityPluginRegistrar internal sealed class MyPluginRegistrar : IAuthorityPluginRegistrar
{ {
public string PluginType => "my-custom"; public string PluginType => "my-custom";
public void Register(AuthorityPluginRegistrationContext context) public void Register(AuthorityPluginRegistrationContext context)
{ {
var name = context.Plugin.Manifest.Name; var name = context.Plugin.Manifest.Name;
context.Services.AddOptions<MyPluginOptions>(name) context.Services.AddOptions<MyPluginOptions>(name)
.Bind(context.Plugin.Configuration) .Bind(context.Plugin.Configuration)
.PostConfigure(opts => opts.Validate(name)); .PostConfigure(opts => opts.Validate(name));
context.Services.AddSingleton<IIdentityProviderPlugin>(sp => context.Services.AddSingleton<IIdentityProviderPlugin>(sp =>
new MyIdentityProvider(context.Plugin, sp.GetRequiredService<MyCredentialStore>(), new MyIdentityProvider(context.Plugin, sp.GetRequiredService<MyCredentialStore>(),
sp.GetRequiredService<MyClaimsEnricher>(), sp.GetRequiredService<MyClaimsEnricher>(),
sp.GetRequiredService<ILogger<MyIdentityProvider>>())); sp.GetRequiredService<ILogger<MyIdentityProvider>>()));
} }
} }
``` ```
## 6. Identity Provider Surface ## 6. Identity Provider Surface
- Implement `IIdentityProviderPlugin` to expose: - Implement `IIdentityProviderPlugin` to expose:
- `IUserCredentialStore` for password validation and user CRUD. - `IUserCredentialStore` for password validation and user CRUD.
- `IClaimsEnricher` to append roles/attributes onto issued principals. - `IClaimsEnricher` to append roles/attributes onto issued principals.
- Optional `IClientProvisioningStore` for machine-to-machine clients. - Optional `IClientProvisioningStore` for machine-to-machine clients.
- `AuthorityIdentityProviderCapabilities` to advertise supported flows. - `AuthorityIdentityProviderCapabilities` to advertise supported flows.
- Password guidance: - Password guidance:
- Standard plug-in hashes via `ICryptoProvider` using Argon2id by default and emits PHC-compliant strings. Successful PBKDF2 logins trigger automatic rehashes so migrations complete gradually. See `docs/security/password-hashing.md` for tuning advice. - Standard plug-in hashes via `ICryptoProvider` using Argon2id by default and emits PHC-compliant strings. Successful PBKDF2 logins trigger automatic rehashes so migrations complete gradually. See `docs/security/password-hashing.md` for tuning advice.
- Enforce password policies before hashing to avoid storing weak credentials. - Enforce password policies before hashing to avoid storing weak credentials.
- Health checks should probe backing stores (e.g., Mongo `ping`) and return `AuthorityPluginHealthResult` so `/ready` can surface issues. - Health checks should probe backing stores (e.g., Mongo `ping`) and return `AuthorityPluginHealthResult` so `/ready` can surface issues.
- When supporting additional factors (e.g., TOTP), implement `SupportsMfa` and document the enrolment flow for resource servers. - When supporting additional factors (e.g., TOTP), implement `SupportsMfa` and document the enrolment flow for resource servers.
## 7. Configuration & Secrets ## 7. Configuration & Secrets
- Authority looks for manifests under `etc/authority.plugins/`. Each YAML file maps directly to a plug-in name. - Authority looks for manifests under `etc/authority.plugins/`. Each YAML file maps directly to a plug-in name.
- Support environment overrides using `STELLAOPS_AUTHORITY_PLUGINS__DESCRIPTORS__<NAME>__...`. - Support environment overrides using `STELLAOPS_AUTHORITY_PLUGINS__DESCRIPTORS__<NAME>__...`.
- Never store raw secrets in git: allow operators to supply them via `.local.yaml`, environment variables, or injected secret files. Document which keys are mandatory. - Never store raw secrets in git: allow operators to supply them via `.local.yaml`, environment variables, or injected secret files. Document which keys are mandatory.
- Validate configuration as soon as the registrar runs; use explicit error messages to guide operators. The Standard plug-in now enforces complete bootstrap credentials (username + password) and positive lockout windows via `StandardPluginOptions.Validate`. - Validate configuration as soon as the registrar runs; use explicit error messages to guide operators. The Standard plug-in now enforces complete bootstrap credentials (username + password) and positive lockout windows via `StandardPluginOptions.Validate`.
- Cross-reference bootstrap workflows with `docs/modules/authority/operations/bootstrap.md` (to be published alongside CORE6) so operators can reuse the same payload formats for manual provisioning. - Cross-reference bootstrap workflows with `docs/modules/authority/operations/bootstrap.md` (to be published alongside CORE6) so operators can reuse the same payload formats for manual provisioning.
- `passwordHashing` inherits defaults from `authority.security.passwordHashing`. Override only when hardware constraints differ per plug-in: - `passwordHashing` inherits defaults from `authority.security.passwordHashing`. Override only when hardware constraints differ per plug-in:
```yaml ```yaml
passwordHashing: passwordHashing:
algorithm: Argon2id algorithm: Argon2id
memorySizeInKib: 19456 memorySizeInKib: 19456
iterations: 2 iterations: 2
parallelism: 1 parallelism: 1
``` ```
Invalid values (≤0) fail fast during startup, and legacy PBKDF2 hashes rehash automatically once the new algorithm succeeds. Invalid values (≤0) fail fast during startup, and legacy PBKDF2 hashes rehash automatically once the new algorithm succeeds.
### 7.1 Token Persistence Contract ### 7.1 Token Persistence Contract
- The host automatically persists every issued principal (access, refresh, device, authorization code) in `authority_tokens`. Plug-in code **must not** bypass this store; use the provided `IAuthorityTokenStore` helpers when implementing custom flows. - The host automatically persists every issued principal (access, refresh, device, authorization code) in `authority_tokens`. Plug-in code **must not** bypass this store; use the provided `IAuthorityTokenStore` helpers when implementing custom flows.
- When a plug-in disables a subject or client outside the standard handlers, call `IAuthorityTokenStore.UpdateStatusAsync(...)` for each affected token so revocation bundles stay consistent. - When a plug-in disables a subject or client outside the standard handlers, call `IAuthorityTokenStore.UpdateStatusAsync(...)` for each affected token so revocation bundles stay consistent.
- Supply machine-friendly `revokedReason` codes (`compromised`, `rotation`, `policy`, `lifecycle`, etc.) and optional `revokedMetadata` entries when invalidating credentials. These flow straight into `revocation-bundle.json` and should remain deterministic. - Supply machine-friendly `revokedReason` codes (`compromised`, `rotation`, `policy`, `lifecycle`, etc.) and optional `revokedMetadata` entries when invalidating credentials. These flow straight into `revocation-bundle.json` and should remain deterministic.
- Token scopes should be normalised (trimmed, unique, ordinal sort) before returning from plug-in verification paths. `TokenPersistenceHandlers` will keep that ordering for downstream consumers. - Token scopes should be normalised (trimmed, unique, ordinal sort) before returning from plug-in verification paths. `TokenPersistenceHandlers` will keep that ordering for downstream consumers.
### 7.2 Claims & Enrichment Checklist ### 7.2 Claims & Enrichment Checklist
- Authority always sets the OpenID Connect basics: `sub`, `client_id`, `preferred_username`, optional `name`, and `role` (for password flows). Plug-ins must use `IClaimsEnricher` to append additional claims in a **deterministic** order (sort arrays, normalise casing) so resource servers can rely on stable shapes. - Authority always sets the OpenID Connect basics: `sub`, `client_id`, `preferred_username`, optional `name`, and `role` (for password flows). Plug-ins must use `IClaimsEnricher` to append additional claims in a **deterministic** order (sort arrays, normalise casing) so resource servers can rely on stable shapes.
- Recommended enrichment keys: - Recommended enrichment keys:
- `stellaops.realm` – plug-in/tenant identifier so services can scope policies. - `stellaops.realm` – plug-in/tenant identifier so services can scope policies.
- `stellaops.subject.type` – values such as `human`, `service`, `bootstrap`. - `stellaops.subject.type` – values such as `human`, `service`, `bootstrap`.
- `groups` / `projects` – sorted arrays describing operator entitlements. - `groups` / `projects` – sorted arrays describing operator entitlements.
- Claims visible in tokens should mirror what `/token` and `/userinfo` emit. Avoid injecting sensitive PII directly; mark values with `ClassifiedString.Personal` inside the plug-in so audit sinks can tag them appropriately. - Claims visible in tokens should mirror what `/token` and `/userinfo` emit. Avoid injecting sensitive PII directly; mark values with `ClassifiedString.Personal` inside the plug-in so audit sinks can tag them appropriately.
- For client-credential flows, remember to enrich both the client principal and the validation path (`TokenValidationHandlers`) so refresh flows keep the same metadata. - For client-credential flows, remember to enrich both the client principal and the validation path (`TokenValidationHandlers`) so refresh flows keep the same metadata.
### 7.3 Revocation Bundles & Reasons ### 7.3 Revocation Bundles & Reasons
- Use `IAuthorityRevocationStore` to record subject/client/token revocations when credentials are deleted or rotated. Stick to the standard categories (`token`, `subject`, `client`, `key`). - Use `IAuthorityRevocationStore` to record subject/client/token revocations when credentials are deleted or rotated. Stick to the standard categories (`token`, `subject`, `client`, `key`).
- Include a deterministic `reason` string and optional `reasonDescription` so operators understand *why* a subject was revoked when inspecting bundles offline. - Include a deterministic `reason` string and optional `reasonDescription` so operators understand *why* a subject was revoked when inspecting bundles offline.
- Plug-ins should populate `metadata` with stable keys (e.g., `revokedBy`, `sourcePlugin`, `ticketId`) to simplify SOC correlation. The keys must be lowercase, ASCII, and free of secrets—bundles are mirrored to air-gapped agents. - Plug-ins should populate `metadata` with stable keys (e.g., `revokedBy`, `sourcePlugin`, `ticketId`) to simplify SOC correlation. The keys must be lowercase, ASCII, and free of secrets—bundles are mirrored to air-gapped agents.
## 8. Rate Limiting & Lockout Interplay ## 8. Rate Limiting & Lockout Interplay
Rate limiting and account lockouts are complementary controls. Plug-ins must surface both deterministically so operators can correlate limiter hits with credential rejections. Rate limiting and account lockouts are complementary controls. Plug-ins must surface both deterministically so operators can correlate limiter hits with credential rejections.
**Baseline quotas** (from `docs/dev/authority-rate-limit-tuning-outline.md`): **Baseline quotas** (from `docs/dev/authority-rate-limit-tuning-outline.md`):
| Endpoint | Default policy | Notes | | Endpoint | Default policy | Notes |
|----------|----------------|-------| |----------|----------------|-------|
| `/token` | 30 requests / 60s, queue 0 | Drop to 10/60s for untrusted ranges; raise only with WAF + monitoring. | | `/token` | 30 requests / 60s, queue 0 | Drop to 10/60s for untrusted ranges; raise only with WAF + monitoring. |
| `/authorize` | 60 requests / 60s, queue 10 | Reduce carefully; interactive UX depends on headroom. | | `/authorize` | 60 requests / 60s, queue 10 | Reduce carefully; interactive UX depends on headroom. |
| `/internal/*` | Disabled by default; recommended 5/60s when enabled | Keep queue 0 for bootstrap APIs. | | `/internal/*` | Disabled by default; recommended 5/60s when enabled | Keep queue 0 for bootstrap APIs. |
**Retry metadata:** The middleware stamps `Retry-After` plus tags `authority.client_id`, `authority.remote_ip`, and `authority.endpoint`. Plug-ins should keep these tags intact when crafting responses or telemetry so dashboards remain consistent. **Retry metadata:** The middleware stamps `Retry-After` plus tags `authority.client_id`, `authority.remote_ip`, and `authority.endpoint`. Plug-ins should keep these tags intact when crafting responses or telemetry so dashboards remain consistent.
**Lockout counters:** Treat lockouts as **subject-scoped** decisions. When multiple instances update counters, reuse the deterministic tie-breakers documented in `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md` (freshness overrides, precedence, and stable hashes) to avoid divergent lockout states across replicas. **Lockout counters:** Treat lockouts as **subject-scoped** decisions. When multiple instances update counters, reuse the deterministic tie-breakers documented in `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md` (freshness overrides, precedence, and stable hashes) to avoid divergent lockout states across replicas.
**Alerting hooks:** Emit structured logs/metrics when either the limiter or credential store rejects access. Suggested gauges include `aspnetcore_rate_limiting_rejections_total{limiter="authority-token"}` and any custom `auth.plugins.<pluginName>.lockouts_total` counter. **Alerting hooks:** Emit structured logs/metrics when either the limiter or credential store rejects access. Suggested gauges include `aspnetcore_rate_limiting_rejections_total{limiter="authority-token"}` and any custom `auth.plugins.<pluginName>.lockouts_total` counter.
![Authority rate limit and lockout flow](../assets/authority/authority-rate-limit-flow.svg) ![Authority rate limit and lockout flow](../assets/authority/authority-rate-limit-flow.svg)
_Source:_ `docs/assets/authority/authority-rate-limit-flow.mmd` _Source:_ `docs/assets/authority/authority-rate-limit-flow.mmd`
## 9. Logging, Metrics, and Diagnostics ## 9. Logging, Metrics, and Diagnostics
- Always log via the injected `ILogger<T>`; include `pluginName` and correlation IDs where available. - Always log via the injected `ILogger<T>`; include `pluginName` and correlation IDs where available.
- Activity/metric names should align with `AuthorityTelemetry` constants (`service.name=stellaops-authority`). - Activity/metric names should align with `AuthorityTelemetry` constants (`service.name=stellaops-authority`).
- Expose additional diagnostics via structured logging rather than writing custom HTTP endpoints; the host will integrate these into `/health` and `/ready`. - Expose additional diagnostics via structured logging rather than writing custom HTTP endpoints; the host will integrate these into `/health` and `/ready`.
- Emit metrics with stable names (`auth.plugins.<pluginName>.*`) when introducing custom instrumentation; coordinate with the Observability guild to reserve prefixes. - Emit metrics with stable names (`auth.plugins.<pluginName>.*`) when introducing custom instrumentation; coordinate with the Observability guild to reserve prefixes.
## 10. Testing & Tooling ## 10. Testing & Tooling
- Unit tests: use Mongo2Go (or similar) to exercise credential stores without hitting production infrastructure (`StandardUserCredentialStoreTests` is a template). - Unit tests: use Mongo2Go (or similar) to exercise credential stores without hitting production infrastructure (`StandardUserCredentialStoreTests` is a template).
- Determinism: fix timestamps to UTC and sort outputs consistently; avoid random GUIDs unless stable. - Determinism: fix timestamps to UTC and sort outputs consistently; avoid random GUIDs unless stable.
- Smoke tests: launch `dotnet run --project src/Authority/StellaOps.Authority/StellaOps.Authority` with your plug-in under `StellaOps.Authority.PluginBinaries` and verify `/ready`. - Smoke tests: launch `dotnet run --project src/Authority/StellaOps.Authority/StellaOps.Authority` with your plug-in under `StellaOps.Authority.PluginBinaries` and verify `/ready`.
- Example verification snippet: - Example verification snippet:
```csharp ```csharp
[Fact] [Fact]
public async Task VerifyPasswordAsync_ReturnsSuccess() public async Task VerifyPasswordAsync_ReturnsSuccess()
{ {
var store = CreateCredentialStore(); var store = CreateCredentialStore();
await store.UpsertUserAsync(new AuthorityUserRegistration("alice", "Pa55!", null, null, false, await store.UpsertUserAsync(new AuthorityUserRegistration("alice", "Pa55!", null, null, false,
Array.Empty<string>(), new Dictionary<string, string?>()), CancellationToken.None); Array.Empty<string>(), new Dictionary<string, string?>()), CancellationToken.None);
var result = await store.VerifyPasswordAsync("alice", "Pa55!", CancellationToken.None); var result = await store.VerifyPasswordAsync("alice", "Pa55!", CancellationToken.None);
Assert.True(result.Succeeded); Assert.True(result.Succeeded);
Assert.True(result.User?.Roles.Count == 0); Assert.True(result.User?.Roles.Count == 0);
} }
``` ```
## 11. Packaging & Delivery ## 11. Packaging & Delivery
- Output assembly should follow `StellaOps.Authority.Plugin.<Name>.dll` so the host’s search pattern picks it up. - Output assembly should follow `StellaOps.Authority.Plugin.<Name>.dll` so the host’s search pattern picks it up.
- Place the compiled DLL plus dependencies under `StellaOps.Authority.PluginBinaries` for offline deployments; include hashes/signatures in release notes (Security Guild guidance forthcoming). - Place the compiled DLL plus dependencies under `StellaOps.Authority.PluginBinaries` for offline deployments; include hashes/signatures in release notes (Security Guild guidance forthcoming).
- Document any external prerequisites (e.g., CA cert bundle) in your plug-in README. - Document any external prerequisites (e.g., CA cert bundle) in your plug-in README.
- Update `etc/authority.plugins/<plugin>.yaml` samples and include deterministic SHA256 hashes for optional bootstrap payloads when distributing Offline Kit artefacts. - Update `etc/authority.plugins/<plugin>.yaml` samples and include deterministic SHA256 hashes for optional bootstrap payloads when distributing Offline Kit artefacts.
[^ldap-rfc]: Lightweight Directory Access Protocol (LDAPv3) specification — [RFC 4511](https://datatracker.ietf.org/doc/html/rfc4511). [^ldap-rfc]: Lightweight Directory Access Protocol (LDAPv3) specification — [RFC 4511](https://datatracker.ietf.org/doc/html/rfc4511).
## 12. Checklist & Handoff ## 12. Checklist & Handoff
- ✅ Capabilities declared and validated in automated tests. - ✅ Capabilities declared and validated in automated tests.
- ✅ Bootstrap workflows documented (if `bootstrap` capability used) and repeatable. - ✅ Bootstrap workflows documented (if `bootstrap` capability used) and repeatable.
- ✅ Local smoke test + unit/integration suites green (`dotnet test`). - ✅ Local smoke test + unit/integration suites green (`dotnet test`).
- ✅ Operational docs updated: configuration keys, secrets guidance, troubleshooting. - ✅ Operational docs updated: configuration keys, secrets guidance, troubleshooting.
- Submit the developer guide update referencing PLG6/DOC4 and tag DevEx + Docs reviewers for sign-off. - Submit the developer guide update referencing PLG6/DOC4 and tag DevEx + Docs reviewers for sign-off.
--- ---
Mermaid sources for the embedded diagrams live under `docs/assets/authority/`. Regenerate the SVG assets with your preferred renderer before committing future updates so the visuals stay in sync with the `.mmd` definitions. Mermaid sources for the embedded diagrams live under `docs/assets/authority/`. Regenerate the SVG assets with your preferred renderer before committing future updates so the visuals stay in sync with the `.mmd` definitions.

238
docs/dev/BUILDX_PLUGIN_QUICKSTART.md
View File

@@ -1,119 +1,119 @@
# BuildX Generator Quickstart # BuildX Generator Quickstart
This quickstart explains how to run the StellaOps **BuildX SBOM generator** offline, verify the CAS handshake, and emit OCI descriptors that downstream services can attest. This quickstart explains how to run the StellaOps **BuildX SBOM generator** offline, verify the CAS handshake, and emit OCI descriptors that downstream services can attest.
## 1. Prerequisites ## 1. Prerequisites
- Docker 25+ with BuildKit enabled (`docker buildx` available). - Docker 25+ with BuildKit enabled (`docker buildx` available).
- .NET 10 (preview) SDK matching the repository `global.json`. - .NET 10 (preview) SDK matching the repository `global.json`.
- Optional: network access to a StellaOps Attestor endpoint (the quickstart uses a mock service). - Optional: network access to a StellaOps Attestor endpoint (the quickstart uses a mock service).
## 2. Publish the plug-in binaries ## 2. Publish the plug-in binaries
The BuildX generator publishes as a .NET self-contained executable with its manifest under `plugins/scanner/buildx/`. The BuildX generator publishes as a .NET self-contained executable with its manifest under `plugins/scanner/buildx/`.
```bash ```bash
# From the repository root # From the repository root
DOTNET_CLI_HOME="${PWD}/.dotnet" \ DOTNET_CLI_HOME="${PWD}/.dotnet" \
dotnet publish src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/StellaOps.Scanner.Sbomer.BuildXPlugin.csproj \ dotnet publish src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/StellaOps.Scanner.Sbomer.BuildXPlugin.csproj \
-c Release \ -c Release \
-o out/buildx -o out/buildx
``` ```
- `out/buildx/` now contains `StellaOps.Scanner.Sbomer.BuildXPlugin.dll` and the manifest `stellaops.sbom-indexer.manifest.json`. - `out/buildx/` now contains `StellaOps.Scanner.Sbomer.BuildXPlugin.dll` and the manifest `stellaops.sbom-indexer.manifest.json`.
- `plugins/scanner/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin/` receives the same artefacts for release packaging. - `plugins/scanner/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin/` receives the same artefacts for release packaging.
- The CI pipeline also tars and signs (SHA-256 manifest) the OS analyzer plug-ins located under - The CI pipeline also tars and signs (SHA-256 manifest) the OS analyzer plug-ins located under
`plugins/scanner/analyzers/os/` so they ship alongside the BuildX generator artefacts. `plugins/scanner/analyzers/os/` so they ship alongside the BuildX generator artefacts.
## 3. Verify the CAS handshake ## 3. Verify the CAS handshake
```bash ```bash
dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll handshake \ dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll handshake \
--manifest out/buildx \ --manifest out/buildx \
--cas out/cas --cas out/cas
``` ```
The command performs a deterministic probe write (`sha256`) into the provided CAS directory and prints the resolved path. The command performs a deterministic probe write (`sha256`) into the provided CAS directory and prints the resolved path.
## 4. Emit a descriptor + provenance placeholder ## 4. Emit a descriptor + provenance placeholder
1. Build or identify the image you want to describe and capture its digest: 1. Build or identify the image you want to describe and capture its digest:
```bash ```bash
docker buildx build --load -t stellaops/buildx-demo:ci samples/ci/buildx-demo docker buildx build --load -t stellaops/buildx-demo:ci samples/ci/buildx-demo
DIGEST=$(docker image inspect stellaops/buildx-demo:ci --format '{{index .RepoDigests 0}}') DIGEST=$(docker image inspect stellaops/buildx-demo:ci --format '{{index .RepoDigests 0}}')
``` ```
2. Generate a CycloneDX SBOM for the built image (any tool works; here we use `docker sbom`): 2. Generate a CycloneDX SBOM for the built image (any tool works; here we use `docker sbom`):
```bash ```bash
docker sbom stellaops/buildx-demo:ci --format cyclonedx-json > out/buildx-sbom.cdx.json docker sbom stellaops/buildx-demo:ci --format cyclonedx-json > out/buildx-sbom.cdx.json
``` ```
3. Invoke the `descriptor` command, pointing at the SBOM file and optional metadata: 3. Invoke the `descriptor` command, pointing at the SBOM file and optional metadata:
```bash ```bash
dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll descriptor \ dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll descriptor \
--manifest out/buildx \ --manifest out/buildx \
--image "$DIGEST" \ --image "$DIGEST" \
--sbom out/buildx-sbom.cdx.json \ --sbom out/buildx-sbom.cdx.json \
--sbom-name buildx-sbom.cdx.json \ --sbom-name buildx-sbom.cdx.json \
--artifact-type application/vnd.stellaops.sbom.layer+json \ --artifact-type application/vnd.stellaops.sbom.layer+json \
--sbom-format cyclonedx-json \ --sbom-format cyclonedx-json \
--sbom-kind inventory \ --sbom-kind inventory \
--repository git.stella-ops.org/stellaops/buildx-demo \ --repository git.stella-ops.org/stellaops/buildx-demo \
--build-ref $(git rev-parse HEAD) \ --build-ref $(git rev-parse HEAD) \
> out/buildx-descriptor.json > out/buildx-descriptor.json
``` ```
The output JSON captures: The output JSON captures:
- OCI artifact descriptor including size, digest, and annotations (`org.stellaops.*`). - OCI artifact descriptor including size, digest, and annotations (`org.stellaops.*`).
- Provenance placeholder (`expectedDsseSha256`, `nonce`, `attestorUri` when provided). `nonce` is derived deterministically from the image + SBOM metadata so repeated runs produce identical placeholders for identical inputs. - Provenance placeholder (`expectedDsseSha256`, `nonce`, `attestorUri` when provided). `nonce` is derived deterministically from the image + SBOM metadata so repeated runs produce identical placeholders for identical inputs.
- Generator metadata and deterministic timestamps. - Generator metadata and deterministic timestamps.
## 5. (Optional) Send the placeholder to an Attestor ## 5. (Optional) Send the placeholder to an Attestor
The plug-in can POST the descriptor metadata to an Attestor endpoint, returning once it receives an HTTP 202. The plug-in can POST the descriptor metadata to an Attestor endpoint, returning once it receives an HTTP 202.
```bash ```bash
python3 - <<'PY' & python3 - <<'PY' &
from http.server import BaseHTTPRequestHandler, HTTPServer from http.server import BaseHTTPRequestHandler, HTTPServer
class Handler(BaseHTTPRequestHandler): class Handler(BaseHTTPRequestHandler):
def do_POST(self): def do_POST(self):
_ = self.rfile.read(int(self.headers.get('Content-Length', 0))) _ = self.rfile.read(int(self.headers.get('Content-Length', 0)))
self.send_response(202); self.end_headers(); self.wfile.write(b'accepted') self.send_response(202); self.end_headers(); self.wfile.write(b'accepted')
def log_message(self, fmt, *args): def log_message(self, fmt, *args):
return return
server = HTTPServer(('127.0.0.1', 8085), Handler) server = HTTPServer(('127.0.0.1', 8085), Handler)
try: try:
server.serve_forever() server.serve_forever()
except KeyboardInterrupt: except KeyboardInterrupt:
pass pass
finally: finally:
server.server_close() server.server_close()
PY PY
MOCK_PID=$! MOCK_PID=$!
dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll descriptor \ dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll descriptor \
--manifest out/buildx \ --manifest out/buildx \
--image "$DIGEST" \ --image "$DIGEST" \
--sbom out/buildx-sbom.cdx.json \ --sbom out/buildx-sbom.cdx.json \
--attestor http://127.0.0.1:8085/provenance \ --attestor http://127.0.0.1:8085/provenance \
--attestor-token "$STELLAOPS_ATTESTOR_TOKEN" \ --attestor-token "$STELLAOPS_ATTESTOR_TOKEN" \
> out/buildx-descriptor.json > out/buildx-descriptor.json
kill $MOCK_PID kill $MOCK_PID
``` ```
Set `STELLAOPS_ATTESTOR_TOKEN` (or pass `--attestor-token`) when the Attestor requires bearer authentication. Use `--attestor-insecure` for lab environments with self-signed certificates. Set `STELLAOPS_ATTESTOR_TOKEN` (or pass `--attestor-token`) when the Attestor requires bearer authentication. Use `--attestor-insecure` for lab environments with self-signed certificates.
## 6. CI workflow example ## 6. CI workflow example
A reusable GitHub Actions workflow is provided under `samples/ci/buildx-demo/github-actions-buildx-demo.yml`. It publishes the plug-in, runs the handshake, builds the demo image, emits a descriptor, and uploads both the descriptor and the mock-Attestor request as artefacts. A reusable GitHub Actions workflow is provided under `samples/ci/buildx-demo/github-actions-buildx-demo.yml`. It publishes the plug-in, runs the handshake, builds the demo image, emits a descriptor, and uploads both the descriptor and the mock-Attestor request as artefacts.
Add the workflow to your repository (or call it via `workflow_call`) and adjust the SBOM path + Attestor URL as needed. The workflow also re-runs the `descriptor` command and diffs the results (ignoring the `generatedAt` timestamp) so you catch regressions that would break deterministic CI use. Add the workflow to your repository (or call it via `workflow_call`) and adjust the SBOM path + Attestor URL as needed. The workflow also re-runs the `descriptor` command and diffs the results (ignoring the `generatedAt` timestamp) so you catch regressions that would break deterministic CI use.
--- ---
For deeper integration guidance (custom SBOM builders, exporting DSSE bundles), track ADRs in `docs/modules/scanner/architecture.md` §7 and follow upcoming Attestor API releases. For deeper integration guidance (custom SBOM builders, exporting DSSE bundles), track ADRs in `docs/modules/scanner/architecture.md` §7 and follow upcoming Attestor API releases.

40
docs/dev/aoc-normalization-removal-notes.md
View File

@@ -1,21 +1,21 @@
# AOC Normalization Removal Notes # AOC Normalization Removal Notes
_Last updated: 2025-10-29_ _Last updated: 2025-10-29_
## Goal ## Goal
Document follow-up actions for CONCELIER-CORE-AOC-19-004 as we unwind the final pieces of normalization from the ingestion/runtime path. Document follow-up actions for CONCELIER-CORE-AOC-19-004 as we unwind the final pieces of normalization from the ingestion/runtime path.
## Current Findings ## Current Findings
- `AdvisoryRawService` and `MongoAdvisoryRawRepository` already preserve upstream ordering and duplicate aliases (trim-only). No additional code changes required there. - `AdvisoryRawService` and `MongoAdvisoryRawRepository` already preserve upstream ordering and duplicate aliases (trim-only). No additional code changes required there.
- Observation layers (`AdvisoryObservationFactory`, `AdvisoryObservationQueryService`) still canonicalise aliases, PURLs, CPEs, and references. These need to be relaxed so Policy/overlays receive raw linksets and can own dedupe logic. - Observation layers (`AdvisoryObservationFactory`, `AdvisoryObservationQueryService`) still canonicalise aliases, PURLs, CPEs, and references. These need to be relaxed so Policy/overlays receive raw linksets and can own dedupe logic.
- Linkset mapper continues to emit deterministic hints. We will keep the mapper but ensure observation output can surface both raw and canonical views for downstream services. - Linkset mapper continues to emit deterministic hints. We will keep the mapper but ensure observation output can surface both raw and canonical views for downstream services.
## Next Steps ## Next Steps
1. Introduce a raw linkset projection alongside the existing canonical mapper so Policy Engine can choose which flavour to consume. 1. Introduce a raw linkset projection alongside the existing canonical mapper so Policy Engine can choose which flavour to consume. ✅ 2025-10-31: `AdvisoryObservation` now surfaces `RawLinkset`; Mongo documents store both canonical & raw shapes; tests/goldens updated.
2. Update observation factory/query tests to assert duplicate handling and ordering with the relaxed projection. 2. Update observation factory/query tests to assert duplicate handling and ordering with the relaxed projection. ✅ 2025-10-31.
3. Refresh docs (`docs/ingestion/aggregation-only-contract.md`) once behaviour lands to explain the “raw vs canonical linkset” split. 3. Refresh docs (`docs/ingestion/aggregation-only-contract.md`) once behaviour lands to explain the “raw vs canonical linkset” split.
4. Coordinate with Policy Guild to validate consumers against the new raw projection before flipping defaults. 4. Coordinate with Policy Guild to validate consumers against the new raw projection before flipping defaults. ↺ Ongoing — see action items in `docs/dev/raw-linkset-backfill-plan.md` (2025-10-31 handshake with POLICY-ENGINE-20-003 owners).

292
docs/dev/authority-dpop-mtls-plan.md
View File

@@ -1,146 +1,146 @@
# Authority DPoP & mTLS Implementation Plan (2025-10-19) # Authority DPoP & mTLS Implementation Plan (2025-10-19)
## Purpose ## Purpose
- Provide the implementation blueprint for AUTH-DPOP-11-001 and AUTH-MTLS-11-002. - Provide the implementation blueprint for AUTH-DPOP-11-001 and AUTH-MTLS-11-002.
- Unify sender-constraint validation across Authority, downstream services, and clients. - Unify sender-constraint validation across Authority, downstream services, and clients.
- Capture deterministic, testable steps that unblock UI/Signer guilds depending on DPoP/mTLS hardening. - Capture deterministic, testable steps that unblock UI/Signer guilds depending on DPoP/mTLS hardening.
## Scope ## Scope
- Token endpoint validation, issuance, and storage changes inside `StellaOps.Authority`. - Token endpoint validation, issuance, and storage changes inside `StellaOps.Authority`.
- Shared security primitives consumed by Authority, Scanner, Signer, CLI, and UI. - Shared security primitives consumed by Authority, Scanner, Signer, CLI, and UI.
- Operator-facing configuration, auditing, and observability. - Operator-facing configuration, auditing, and observability.
- Out of scope: PoE enforcement (Signer) and CLI/UI client UX; those teams consume the new capabilities. - Out of scope: PoE enforcement (Signer) and CLI/UI client UX; those teams consume the new capabilities.
> **Status update (2025-10-19):** `ValidateDpopProofHandler`, `AuthorityClientCertificateValidator`, and the supporting storage/audit plumbing now live in `src/Authority/StellaOps.Authority`. DPoP proofs populate `cnf.jkt`, mTLS bindings enforce certificate thumbprints via `cnf.x5t#S256`, and token documents persist the sender constraint metadata. In-memory nonce issuance is wired (Redis implementation to follow). Documentation and configuration references were updated (`docs/11_AUTHORITY.md`). Targeted unit/integration tests were added; running the broader test suite is currently blocked by pre-existing `StellaOps.Concelier.Storage.Mongo` build errors. > **Status update (2025-10-19):** `ValidateDpopProofHandler`, `AuthorityClientCertificateValidator`, and the supporting storage/audit plumbing now live in `src/Authority/StellaOps.Authority`. DPoP proofs populate `cnf.jkt`, mTLS bindings enforce certificate thumbprints via `cnf.x5t#S256`, and token documents persist the sender constraint metadata. In-memory nonce issuance is wired (Redis implementation to follow). Documentation and configuration references were updated (`docs/11_AUTHORITY.md`). Targeted unit/integration tests were added; running the broader test suite is currently blocked by pre-existing `StellaOps.Concelier.Storage.Mongo` build errors.
> >
> **Status update (2025-10-20):** Redis-backed nonce configuration is exposed through `security.senderConstraints.dpop.nonce` with sample YAML (`etc/authority.yaml.sample`) and architecture docs refreshed. Operator guide now includes concrete Redis/required audiences snippet; nonce challenge regression remains covered by `ValidateDpopProof_IssuesNonceChallenge_WhenNonceMissing`. > **Status update (2025-10-20):** Redis-backed nonce configuration is exposed through `security.senderConstraints.dpop.nonce` with sample YAML (`etc/authority.yaml.sample`) and architecture docs refreshed. Operator guide now includes concrete Redis/required audiences snippet; nonce challenge regression remains covered by `ValidateDpopProof_IssuesNonceChallenge_WhenNonceMissing`.
> >
> **Status update (2025-10-23):** mTLS enforcement now honours `security.senderConstraints.mtls.enforceForAudiences`, automatically rejecting non-mTLS clients targeting audiences such as `signer`. Certificate bindings validate thumbprint, issuer, subject, serial number, and SAN values, producing deterministic error codes for operators. Introspection responses include `cnf.x5t#S256`, and new unit tests cover audience enforcement, binding mismatches, and bootstrap storage. Docs/sample config updated accordingly. > **Status update (2025-10-23):** mTLS enforcement now honours `security.senderConstraints.mtls.enforceForAudiences`, automatically rejecting non-mTLS clients targeting audiences such as `signer`. Certificate bindings validate thumbprint, issuer, subject, serial number, and SAN values, producing deterministic error codes for operators. Introspection responses include `cnf.x5t#S256`, and new unit tests cover audience enforcement, binding mismatches, and bootstrap storage. Docs/sample config updated accordingly.
## Design Summary ## Design Summary
- Extract the existing Scanner `DpopProofValidator` stack into a shared `StellaOps.Auth.Security` library used by Authority and resource servers. - Extract the existing Scanner `DpopProofValidator` stack into a shared `StellaOps.Auth.Security` library used by Authority and resource servers.
- Extend Authority configuration (`authority.yaml`) with strongly-typed `senderConstraints.dpop` and `senderConstraints.mtls` sections (map to sample already shown in architecture doc). - Extend Authority configuration (`authority.yaml`) with strongly-typed `senderConstraints.dpop` and `senderConstraints.mtls` sections (map to sample already shown in architecture doc).
- Require DPoP proofs on `/token` when the registered client policy is `senderConstraint=dpop`; bind issued access tokens via `cnf.jkt`. - Require DPoP proofs on `/token` when the registered client policy is `senderConstraint=dpop`; bind issued access tokens via `cnf.jkt`.
- Introduce Authority-managed nonce issuance for “high value” audiences (default: `signer`, `attestor`) with Redis-backed persistence and deterministic auditing. - Introduce Authority-managed nonce issuance for “high value” audiences (default: `signer`, `attestor`) with Redis-backed persistence and deterministic auditing.
- Enable OAuth 2.0 mTLS (RFC 8705) by storing certificate bindings per client, requesting client certificates at TLS termination, and stamping `cnf.x5t#S256` into issued tokens plus introspection output. - Enable OAuth 2.0 mTLS (RFC 8705) by storing certificate bindings per client, requesting client certificates at TLS termination, and stamping `cnf.x5t#S256` into issued tokens plus introspection output.
- Surface structured logs and counters for both DPoP and mTLS flows; provide integration tests that cover success, replay, invalid proof, and certificate mismatch cases. - Surface structured logs and counters for both DPoP and mTLS flows; provide integration tests that cover success, replay, invalid proof, and certificate mismatch cases.
## AUTH-DPOP-11-001 — Proof Validation & Nonce Handling ## AUTH-DPOP-11-001 — Proof Validation & Nonce Handling
**Shared validator** **Shared validator**
- Move `DpopProofValidator`, option types, and replay cache interfaces from `StellaOps.Scanner.Core` into a new assembly `StellaOps.Auth.Security`. - Move `DpopProofValidator`, option types, and replay cache interfaces from `StellaOps.Scanner.Core` into a new assembly `StellaOps.Auth.Security`.
- Provide pluggable caches: `InMemoryDpopReplayCache` (existing) and new `RedisDpopReplayCache` (leveraging the Authority Redis connection). - Provide pluggable caches: `InMemoryDpopReplayCache` (existing) and new `RedisDpopReplayCache` (leveraging the Authority Redis connection).
- Ensure the validator exposes the validated `SecurityKey`, `jti`, and `iat` so Authority can construct the `cnf` claim and compute nonce expiry. - Ensure the validator exposes the validated `SecurityKey`, `jti`, and `iat` so Authority can construct the `cnf` claim and compute nonce expiry.
**Configuration model** **Configuration model**
- Extend `StellaOpsAuthorityOptions.Security` with a `SenderConstraints` property containing: - Extend `StellaOpsAuthorityOptions.Security` with a `SenderConstraints` property containing:
- `Dpop` (`enabled`, `allowedAlgorithms`, `maxAgeSeconds`, `clockSkewSeconds`, `replayWindowSeconds`, `nonce` settings with `enabled`, `ttlSeconds`, `requiredAudiences`, `maxIssuancePerMinute`). - `Dpop` (`enabled`, `allowedAlgorithms`, `maxAgeSeconds`, `clockSkewSeconds`, `replayWindowSeconds`, `nonce` settings with `enabled`, `ttlSeconds`, `requiredAudiences`, `maxIssuancePerMinute`).
- `Mtls` (`enabled`, `requireChainValidation`, `clientCaBundle`, `allowedSubjectPatterns`, `allowedSanTypes`). - `Mtls` (`enabled`, `requireChainValidation`, `clientCaBundle`, `allowedSubjectPatterns`, `allowedSanTypes`).
- Bind from YAML (`authority.security.senderConstraints.*`) while preserving backwards compatibility (defaults keep both disabled). - Bind from YAML (`authority.security.senderConstraints.*`) while preserving backwards compatibility (defaults keep both disabled).
**Token endpoint pipeline** **Token endpoint pipeline**
- Introduce a scoped OpenIddict handler `ValidateDpopProofHandler` inserted before `ValidateClientCredentialsHandler`. - Introduce a scoped OpenIddict handler `ValidateDpopProofHandler` inserted before `ValidateClientCredentialsHandler`.
- Determine the required sender constraint from client metadata: - Determine the required sender constraint from client metadata:
- Add `AuthorityClientMetadataKeys.SenderConstraint` storing `dpop` or `mtls`. - Add `AuthorityClientMetadataKeys.SenderConstraint` storing `dpop` or `mtls`.
- Optionally allow per-client overrides for nonce requirement. - Optionally allow per-client overrides for nonce requirement.
- When `dpop` is required: - When `dpop` is required:
- Read the `DPoP` header from the ASP.NET request, reject with `invalid_token` + `WWW-Authenticate: DPoP error="invalid_dpop_proof"` if absent. - Read the `DPoP` header from the ASP.NET request, reject with `invalid_token` + `WWW-Authenticate: DPoP error="invalid_dpop_proof"` if absent.
- Call the shared validator with method/URI. Enforce algorithm allowlist and `iat` window from options. - Call the shared validator with method/URI. Enforce algorithm allowlist and `iat` window from options.
- Persist the `jkt` thumbprint plus replay cache state in the OpenIddict transaction (`AuthorityOpenIddictConstants.DpopKeyThumbprintProperty`, `DpopIssuedAtProperty`). - Persist the `jkt` thumbprint plus replay cache state in the OpenIddict transaction (`AuthorityOpenIddictConstants.DpopKeyThumbprintProperty`, `DpopIssuedAtProperty`).
- When the requested audience intersects `SenderConstraints.Dpop.Nonce.RequiredAudiences`, require `nonce` in the proof; on first failure respond with HTTP 401, `error="use_dpop_nonce"`, and include `DPoP-Nonce` header (see nonce note below). Cache the rejection reason for audit logging. - When the requested audience intersects `SenderConstraints.Dpop.Nonce.RequiredAudiences`, require `nonce` in the proof; on first failure respond with HTTP 401, `error="use_dpop_nonce"`, and include `DPoP-Nonce` header (see nonce note below). Cache the rejection reason for audit logging.
**Nonce service** **Nonce service**
- Add `IDpopNonceStore` with methods `IssueAsync(audience, clientId, jkt)` and `TryConsumeAsync(nonce, audience, clientId, jkt)`. - Add `IDpopNonceStore` with methods `IssueAsync(audience, clientId, jkt)` and `TryConsumeAsync(nonce, audience, clientId, jkt)`.
- Default implementation `RedisDpopNonceStore` storing SHA-256 hashes of nonces keyed by `audience:clientId:jkt`. TTL comes from `SenderConstraints.Dpop.Nonce.Ttl`. - Default implementation `RedisDpopNonceStore` storing SHA-256 hashes of nonces keyed by `audience:clientId:jkt`. TTL comes from `SenderConstraints.Dpop.Nonce.Ttl`.
- Create helper `DpopNonceIssuer` used by `ValidateDpopProofHandler` to issue nonces when missing/expired, enforcing issuance rate limits (per options) and tagging audit/log records. - Create helper `DpopNonceIssuer` used by `ValidateDpopProofHandler` to issue nonces when missing/expired, enforcing issuance rate limits (per options) and tagging audit/log records.
- On successful validation (nonce supplied and consumed) stamp metadata into the transaction for auditing. - On successful validation (nonce supplied and consumed) stamp metadata into the transaction for auditing.
- Update `ClientCredentialsHandlers` to observe nonce enforcement: when a nonce challenge was sent, emit structured audit with `nonce_issued`, `audiences`, and `retry`. - Update `ClientCredentialsHandlers` to observe nonce enforcement: when a nonce challenge was sent, emit structured audit with `nonce_issued`, `audiences`, and `retry`.
**Token issuance** **Token issuance**
- In `HandleClientCredentialsHandler`, if the transaction contains a validated DPoP key: - In `HandleClientCredentialsHandler`, if the transaction contains a validated DPoP key:
- Build `cnf.jkt` using thumbprint from validator. - Build `cnf.jkt` using thumbprint from validator.
- Include `auth_time`/`dpop_jti` as needed for diagnostics. - Include `auth_time`/`dpop_jti` as needed for diagnostics.
- Persist the thumbprint alongside token metadata in Mongo (extend `AuthorityTokenDocument` with `SenderConstraint`, `KeyThumbprint`, `Nonce` fields). - Persist the thumbprint alongside token metadata in Mongo (extend `AuthorityTokenDocument` with `SenderConstraint`, `KeyThumbprint`, `Nonce` fields).
**Auditing & observability** **Auditing & observability**
- Emit new audit events: - Emit new audit events:
- `authority.dpop.proof.validated` (success/failure, clientId, audience, thumbprint, nonce status, jti). - `authority.dpop.proof.validated` (success/failure, clientId, audience, thumbprint, nonce status, jti).
- `authority.dpop.nonce.issued` and `authority.dpop.nonce.consumed`. - `authority.dpop.nonce.issued` and `authority.dpop.nonce.consumed`.
- Metrics (Prometheus style): - Metrics (Prometheus style):
- `authority_dpop_validations_total{result,reason}`. - `authority_dpop_validations_total{result,reason}`.
- `authority_dpop_nonce_issued_total{audience}` and `authority_dpop_nonce_fails_total{reason}`. - `authority_dpop_nonce_issued_total{audience}` and `authority_dpop_nonce_fails_total{reason}`.
- Structured logs include `authority.sender_constraint=dpop`, `authority.dpop_thumbprint`, `authority.dpop_nonce`. - Structured logs include `authority.sender_constraint=dpop`, `authority.dpop_thumbprint`, `authority.dpop_nonce`.
**Testing** **Testing**
- Unit tests for the handler pipeline using fake OpenIddict transactions. - Unit tests for the handler pipeline using fake OpenIddict transactions.
- Replay/nonce tests with in-memory and Redis stores. - Replay/nonce tests with in-memory and Redis stores.
- Integration tests in `StellaOps.Authority.Tests` covering: - Integration tests in `StellaOps.Authority.Tests` covering:
- Valid DPoP proof issuing `cnf.jkt`. - Valid DPoP proof issuing `cnf.jkt`.
- Missing header → challenge with nonce. - Missing header → challenge with nonce.
- Replayed `jti` rejected. - Replayed `jti` rejected.
- Invalid nonce rejected even after issuance. - Invalid nonce rejected even after issuance.
- Contract tests to ensure `/.well-known/openid-configuration` advertises `dpop_signing_alg_values_supported` and `dpop_nonce_supported` when enabled. - Contract tests to ensure `/.well-known/openid-configuration` advertises `dpop_signing_alg_values_supported` and `dpop_nonce_supported` when enabled.
## AUTH-MTLS-11-002 — Certificate-Bound Tokens ## AUTH-MTLS-11-002 — Certificate-Bound Tokens
**Configuration model** **Configuration model**
- Reuse `SenderConstraints.Mtls` described above; include: - Reuse `SenderConstraints.Mtls` described above; include:
- `enforceForAudiences` list (defaults `signer`, `attestor`, `scheduler`). - `enforceForAudiences` list (defaults `signer`, `attestor`, `scheduler`).
- `certificateRotationGraceSeconds` for overlap. - `certificateRotationGraceSeconds` for overlap.
- `allowedClientCertificateAuthorities` absolute paths. - `allowedClientCertificateAuthorities` absolute paths.
**Kestrel/TLS pipeline** **Kestrel/TLS pipeline**
- Configure Kestrel with `ClientCertificateMode.AllowCertificate` globally and implement middleware that enforces certificate presence only when the resolved client requires mTLS. - Configure Kestrel with `ClientCertificateMode.AllowCertificate` globally and implement middleware that enforces certificate presence only when the resolved client requires mTLS.
- Add `IAuthorityClientCertificateValidator` that validates presented certificate chain, SANs (`dns`, `uri`, optional SPIFFE), and thumbprint matches one of the stored bindings. - Add `IAuthorityClientCertificateValidator` that validates presented certificate chain, SANs (`dns`, `uri`, optional SPIFFE), and thumbprint matches one of the stored bindings.
- Cache validation results per connection id to avoid rehashing on every request. - Cache validation results per connection id to avoid rehashing on every request.
**Client registration & storage** **Client registration & storage**
- Extend `AuthorityClientDocument` with `List<AuthorityClientCertificateBinding>` containing: - Extend `AuthorityClientDocument` with `List<AuthorityClientCertificateBinding>` containing:
- `Thumbprint`, `SerialNumber`, `Subject`, `NotBefore`, `NotAfter`, `Sans`, `CreatedAt`, `UpdatedAt`, `Label`. - `Thumbprint`, `SerialNumber`, `Subject`, `NotBefore`, `NotAfter`, `Sans`, `CreatedAt`, `UpdatedAt`, `Label`.
- Provide admin API mutations (`/admin/clients/{id}/certificates`) for ops tooling (deferred implementation but schema ready). - Provide admin API mutations (`/admin/clients/{id}/certificates`) for ops tooling (deferred implementation but schema ready).
- Update plugin provisioning store (`StandardClientProvisioningStore`) to map descriptors with certificate bindings and `senderConstraint`. - Update plugin provisioning store (`StandardClientProvisioningStore`) to map descriptors with certificate bindings and `senderConstraint`.
- Persist binding state in Mongo migrations (index on `{clientId, thumbprint}`). - Persist binding state in Mongo migrations (index on `{clientId, thumbprint}`).
**Token issuance & introspection** **Token issuance & introspection**
- Add a transaction property capturing the validated client certificate thumbprint. - Add a transaction property capturing the validated client certificate thumbprint.
- `HandleClientCredentialsHandler`: - `HandleClientCredentialsHandler`:
- When mTLS required, ensure certificate info present; reject otherwise. - When mTLS required, ensure certificate info present; reject otherwise.
- Stamp `cnf` claim: `principal.SetClaim("cnf", JsonSerializer.Serialize(new { x5t#S256 = thumbprint }))`. - Stamp `cnf` claim: `principal.SetClaim("cnf", JsonSerializer.Serialize(new { x5t#S256 = thumbprint }))`.
- Store binding metadata in issued token document for audit. - Store binding metadata in issued token document for audit.
- Update `ValidateAccessTokenHandler` and introspection responses to surface `cnf.x5t#S256`. - Update `ValidateAccessTokenHandler` and introspection responses to surface `cnf.x5t#S256`.
- Ensure refresh tokens (if ever enabled) copy the binding data. - Ensure refresh tokens (if ever enabled) copy the binding data.
**Auditing & observability** **Auditing & observability**
- Audit events: - Audit events:
- `authority.mtls.handshake` (success/failure, clientId, thumbprint, issuer, subject). - `authority.mtls.handshake` (success/failure, clientId, thumbprint, issuer, subject).
- `authority.mtls.binding.missing` when a required client posts without a cert. - `authority.mtls.binding.missing` when a required client posts without a cert.
- Metrics: - Metrics:
- `authority_mtls_handshakes_total{result}`. - `authority_mtls_handshakes_total{result}`.
- `authority_mtls_certificate_rotations_total`. - `authority_mtls_certificate_rotations_total`.
- Logs include `authority.sender_constraint=mtls`, `authority.mtls_thumbprint`, `authority.mtls_subject`. - Logs include `authority.sender_constraint=mtls`, `authority.mtls_thumbprint`, `authority.mtls_subject`.
**Testing** **Testing**
- Unit tests for certificate validation rules (SAN mismatches, expiry, CA trust). - Unit tests for certificate validation rules (SAN mismatches, expiry, CA trust).
- Integration tests running Kestrel with test certificates: - Integration tests running Kestrel with test certificates:
- Successful token issuance with bound certificate. - Successful token issuance with bound certificate.
- Request without certificate → `invalid_client`. - Request without certificate → `invalid_client`.
- Token introspection reveals `cnf.x5t#S256`. - Token introspection reveals `cnf.x5t#S256`.
- Rotation scenario (old + new cert allowed during grace window). - Rotation scenario (old + new cert allowed during grace window).
## Implementation Checklist ## Implementation Checklist
**DPoP work-stream** **DPoP work-stream**
1. Extract shared validator into `StellaOps.Auth.Security`; update Scanner references. 1. Extract shared validator into `StellaOps.Auth.Security`; update Scanner references.
2. Introduce configuration classes and bind from YAML/environment. 2. Introduce configuration classes and bind from YAML/environment.
3. Implement nonce store (Redis + in-memory), handler integration, and OpenIddict transaction plumbing. 3. Implement nonce store (Redis + in-memory), handler integration, and OpenIddict transaction plumbing.
4. Stamp `cnf.jkt`, audit events, and metrics; update Mongo documents and migrations. 4. Stamp `cnf.jkt`, audit events, and metrics; update Mongo documents and migrations.
5. Extend docs: `docs/modules/authority/architecture.md`, `docs/security/audit-events.md`, `docs/security/rate-limits.md`, CLI/UI references. 5. Extend docs: `docs/modules/authority/architecture.md`, `docs/security/audit-events.md`, `docs/security/rate-limits.md`, CLI/UI references.
**mTLS work-stream** **mTLS work-stream**
1. Extend client document/schema and provisioning stores with certificate bindings + sender constraint flag. 1. Extend client document/schema and provisioning stores with certificate bindings + sender constraint flag.
2. Configure Kestrel/middleware for optional client certificates and validation service. 2. Configure Kestrel/middleware for optional client certificates and validation service.
3. Update token issuance/introspection to honour certificate bindings and emit `cnf.x5t#S256`. 3. Update token issuance/introspection to honour certificate bindings and emit `cnf.x5t#S256`.
4. Add auditing/metrics and integration tests (happy path + failure). 4. Add auditing/metrics and integration tests (happy path + failure).
5. Refresh operator documentation (`docs/modules/authority/operations/backup-restore.md`, `docs/modules/authority/operations/monitoring.md`, sample `authority.yaml`) to cover certificate lifecycle. 5. Refresh operator documentation (`docs/modules/authority/operations/backup-restore.md`, `docs/modules/authority/operations/monitoring.md`, sample `authority.yaml`) to cover certificate lifecycle.
Both streams should conclude with `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.sln` and documentation cross-links so dependent guilds can unblock UI/Signer work. Both streams should conclude with `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.sln` and documentation cross-links so dependent guilds can unblock UI/Signer work.

90
docs/dev/fixtures.md
View File

@@ -1,45 +1,45 @@
# Concelier Fixture Maintenance # Concelier Fixture Maintenance
Concelier uses a handful of deterministic fixtures to keep connector regressions in check. This guide lists the Concelier uses a handful of deterministic fixtures to keep connector regressions in check. This guide lists the
fixture sets, where they live, and how to regenerate them safely. fixture sets, where they live, and how to regenerate them safely.
--- ---
## GHSA ↔ OSV parity fixtures ## GHSA ↔ OSV parity fixtures
- **Location:** `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/Fixtures/osv-ghsa.*.json` - **Location:** `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/Fixtures/osv-ghsa.*.json`
- **Purpose:** Exercised by `OsvGhsaParityRegressionTests` to ensure OSV + GHSA outputs stay aligned on aliases, - **Purpose:** Exercised by `OsvGhsaParityRegressionTests` to ensure OSV + GHSA outputs stay aligned on aliases,
ranges, references, and credits. ranges, references, and credits.
- **Regeneration:** Either run the test harness with online regeneration (`UPDATE_PARITY_FIXTURES=1 dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`) - **Regeneration:** Either run the test harness with online regeneration (`UPDATE_PARITY_FIXTURES=1 dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`)
or execute the fixture updater (`dotnet run --project src/Tools/FixtureUpdater/FixtureUpdater.csproj`). Both paths or execute the fixture updater (`dotnet run --project src/Tools/FixtureUpdater/FixtureUpdater.csproj`). Both paths
normalise timestamps and canonical ordering. normalise timestamps and canonical ordering.
- **SemVer provenance:** The regenerated fixtures should show `normalizedVersions[].notes` in the - **SemVer provenance:** The regenerated fixtures should show `normalizedVersions[].notes` in the
`osv:{ecosystem}:{advisoryId}:{identifier}` shape emitted by `SemVerRangeRuleBuilder`. Confirm the `osv:{ecosystem}:{advisoryId}:{identifier}` shape emitted by `SemVerRangeRuleBuilder`. Confirm the
constraints and notes line up with GHSA/NVD composites before committing. constraints and notes line up with GHSA/NVD composites before committing.
- **Verification:** Inspect the diff, then re-run `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj` to confirm parity. - **Verification:** Inspect the diff, then re-run `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj` to confirm parity.
## GHSA credit parity fixtures ## GHSA credit parity fixtures
- **Location:** `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/Fixtures/credit-parity.{ghsa,osv,nvd}.json` - **Location:** `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/Fixtures/credit-parity.{ghsa,osv,nvd}.json`
- **Purpose:** Exercised by `GhsaCreditParityRegressionTests` to guarantee GHSA/NVD/OSV acknowledgements remain in lockstep. - **Purpose:** Exercised by `GhsaCreditParityRegressionTests` to guarantee GHSA/NVD/OSV acknowledgements remain in lockstep.
- **Regeneration:** `dotnet run --project src/Tools/FixtureUpdater/FixtureUpdater.csproj` rewrites all three canonical snapshots. - **Regeneration:** `dotnet run --project src/Tools/FixtureUpdater/FixtureUpdater.csproj` rewrites all three canonical snapshots.
- **Verification:** `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`. - **Verification:** `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`.
> Always commit fixture changes together with the code that motivated them and reference the regression test that guards the behaviour. > Always commit fixture changes together with the code that motivated them and reference the regression test that guards the behaviour.
## Apple security update fixtures ## Apple security update fixtures
- **Location:** `src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/Apple/Fixtures/*.html` and `.expected.json`. - **Location:** `src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/Apple/Fixtures/*.html` and `.expected.json`.
- **Purpose:** Exercised by `AppleLiveRegressionTests` to guarantee the Apple HTML parser and mapper stay deterministic while covering Rapid Security Responses and multi-device advisories. - **Purpose:** Exercised by `AppleLiveRegressionTests` to guarantee the Apple HTML parser and mapper stay deterministic while covering Rapid Security Responses and multi-device advisories.
- **Regeneration:** Use the helper scripts (`scripts/update-apple-fixtures.sh` or `scripts/update-apple-fixtures.ps1`). They export `UPDATE_APPLE_FIXTURES=1`, propagate the flag through `WSLENV`, touch `.update-apple-fixtures`, and then run the Apple test project. This keeps WSL/VSCode test invocations in sync while the refresh workflow fetches live Apple support pages, sanitises them, and rewrites both the HTML and expected DTO snapshots with normalised ordering. - **Regeneration:** Use the helper scripts (`scripts/update-apple-fixtures.sh` or `scripts/update-apple-fixtures.ps1`). They export `UPDATE_APPLE_FIXTURES=1`, propagate the flag through `WSLENV`, touch `.update-apple-fixtures`, and then run the Apple test project. This keeps WSL/VSCode test invocations in sync while the refresh workflow fetches live Apple support pages, sanitises them, and rewrites both the HTML and expected DTO snapshots with normalised ordering.
- **Verification:** Inspect the generated diffs and re-run `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests.csproj` without the env var to confirm determinism. - **Verification:** Inspect the generated diffs and re-run `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests.csproj` without the env var to confirm determinism.
> **Tip for other connector owners:** mirror the sentinel + `WSLENV` pattern (`touch .update-<connector>-fixtures`, append the env var via `WSLENV`) when you add fixture refresh scripts so contributors running under WSL inherit the regeneration flag automatically. > **Tip for other connector owners:** mirror the sentinel + `WSLENV` pattern (`touch .update-<connector>-fixtures`, append the env var via `WSLENV`) when you add fixture refresh scripts so contributors running under WSL inherit the regeneration flag automatically.
## KISA advisory fixtures ## KISA advisory fixtures
- **Location:** `src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/Fixtures/kisa-{feed,detail}.(xml|json)` - **Location:** `src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/Fixtures/kisa-{feed,detail}.(xml|json)`
- **Purpose:** Used by `KisaConnectorTests` to verify Hangul-aware fetch → parse → map flows and to assert telemetry counters stay wired. - **Purpose:** Used by `KisaConnectorTests` to verify Hangul-aware fetch → parse → map flows and to assert telemetry counters stay wired.
- **Regeneration:** `UPDATE_KISA_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/StellaOps.Concelier.Connector.Kisa.Tests.csproj` - **Regeneration:** `UPDATE_KISA_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/StellaOps.Concelier.Connector.Kisa.Tests.csproj`
- **Verification:** Re-run the same test suite without the env var; confirm advisory content remains NFC-normalised and HTML is sanitised. Metrics assertions will fail if counters drift. - **Verification:** Re-run the same test suite without the env var; confirm advisory content remains NFC-normalised and HTML is sanitised. Metrics assertions will fail if counters drift.
- **Localisation note:** RSS `category` values (e.g. `취약점정보`) remain in Hangul—do not translate them in fixtures; they feed directly into metrics/log tags. - **Localisation note:** RSS `category` values (e.g. `취약점정보`) remain in Hangul—do not translate them in fixtures; they feed directly into metrics/log tags.

308
docs/dev/merge_semver_playbook.md
View File

@@ -1,154 +1,154 @@
# Concelier SemVer Merge Playbook (Sprint 1–2) # Concelier SemVer Merge Playbook (Sprint 1–2)
This playbook describes how the merge layer and connector teams should emit the new SemVer primitives introduced in Sprint 1–2, how those primitives become normalized version rules, and how downstream jobs query them deterministically. This playbook describes how the merge layer and connector teams should emit the new SemVer primitives introduced in Sprint 1–2, how those primitives become normalized version rules, and how downstream jobs query them deterministically.
## 1. What landed in Sprint 1–2 ## 1. What landed in Sprint 1–2
- `RangePrimitives.SemVer` now infers a canonical `style` (`range`, `exact`, `lt`, `lte`, `gt`, `gte`) and captures `exactValue` when the constraint is a single version. - `RangePrimitives.SemVer` now infers a canonical `style` (`range`, `exact`, `lt`, `lte`, `gt`, `gte`) and captures `exactValue` when the constraint is a single version.
- `NormalizedVersionRule` documents the analytics-friendly projection of each `AffectedPackage` coverage entry and is persisted alongside legacy `versionRanges`. - `NormalizedVersionRule` documents the analytics-friendly projection of each `AffectedPackage` coverage entry and is persisted alongside legacy `versionRanges`.
- `AdvisoryProvenance.decisionReason` records whether merge resolution favored precedence, freshness, or a tie-breaker comparison. - `AdvisoryProvenance.decisionReason` records whether merge resolution favored precedence, freshness, or a tie-breaker comparison.
See `src/Concelier/__Libraries/StellaOps.Concelier.Models/CANONICAL_RECORDS.md` for the full schema and field descriptions. See `src/Concelier/__Libraries/StellaOps.Concelier.Models/CANONICAL_RECORDS.md` for the full schema and field descriptions.
## 2. Mapper pattern ## 2. Mapper pattern
Connectors should emit SemVer primitives as soon as they can normalize a vendor constraint. The helper `SemVerPrimitiveExtensions.ToNormalizedVersionRule` turns those primitives into the persisted rules: Connectors should emit SemVer primitives as soon as they can normalize a vendor constraint. The helper `SemVerPrimitiveExtensions.ToNormalizedVersionRule` turns those primitives into the persisted rules:
```csharp ```csharp
var primitive = new SemVerPrimitive( var primitive = new SemVerPrimitive(
introduced: "1.2.3", introduced: "1.2.3",
introducedInclusive: true, introducedInclusive: true,
fixed: "2.0.0", fixed: "2.0.0",
fixedInclusive: false, fixedInclusive: false,
lastAffected: null, lastAffected: null,
lastAffectedInclusive: false, lastAffectedInclusive: false,
constraintExpression: ">=1.2.3 <2.0.0", constraintExpression: ">=1.2.3 <2.0.0",
exactValue: null); exactValue: null);
var rule = primitive.ToNormalizedVersionRule(notes: "nvd:CVE-2025-1234"); var rule = primitive.ToNormalizedVersionRule(notes: "nvd:CVE-2025-1234");
// rule => scheme=semver, type=range, min=1.2.3, minInclusive=true, max=2.0.0, maxInclusive=false // rule => scheme=semver, type=range, min=1.2.3, minInclusive=true, max=2.0.0, maxInclusive=false
``` ```
If you omit the optional `notes` argument, `ToNormalizedVersionRule` now falls back to the primitive’s `ConstraintExpression`, ensuring the original comparator expression is preserved for provenance/audit queries. If you omit the optional `notes` argument, `ToNormalizedVersionRule` now falls back to the primitive’s `ConstraintExpression`, ensuring the original comparator expression is preserved for provenance/audit queries.
Emit the resulting rule inside `AffectedPackage.NormalizedVersions` while continuing to populate `AffectedVersionRange.RangeExpression` for backward compatibility. Emit the resulting rule inside `AffectedPackage.NormalizedVersions` while continuing to populate `AffectedVersionRange.RangeExpression` for backward compatibility.
## 3. Merge dedupe flow ## 3. Merge dedupe flow
During merge, feed all package candidates through `NormalizedVersionRuleComparer.Instance` prior to persistence. The comparer orders by scheme → type → min → minInclusive → max → maxInclusive → value → notes, guaranteeing consistent document layout and making `$unwind` pipelines deterministic. During merge, feed all package candidates through `NormalizedVersionRuleComparer.Instance` prior to persistence. The comparer orders by scheme → type → min → minInclusive → max → maxInclusive → value → notes, guaranteeing consistent document layout and making `$unwind` pipelines deterministic.
If multiple connectors emit identical constraints, the merge layer should: If multiple connectors emit identical constraints, the merge layer should:
1. Combine provenance entries (preserving one per source). 1. Combine provenance entries (preserving one per source).
2. Preserve a single normalized rule instance (thanks to `NormalizedVersionRuleEqualityComparer.Instance`). 2. Preserve a single normalized rule instance (thanks to `NormalizedVersionRuleEqualityComparer.Instance`).
3. Attach `decisionReason="precedence"` if one source overrides another. 3. Attach `decisionReason="precedence"` if one source overrides another.
## 4. Example Mongo pipeline ## 4. Example Mongo pipeline
Use the following aggregation to locate advisories that affect a specific SemVer: Use the following aggregation to locate advisories that affect a specific SemVer:
```javascript ```javascript
db.advisories.aggregate([ db.advisories.aggregate([
{ $match: { "affectedPackages.type": "semver", "affectedPackages.identifier": "pkg:npm/lodash" } }, { $match: { "affectedPackages.type": "semver", "affectedPackages.identifier": "pkg:npm/lodash" } },
{ $unwind: "$affectedPackages" }, { $unwind: "$affectedPackages" },
{ $unwind: "$affectedPackages.normalizedVersions" }, { $unwind: "$affectedPackages.normalizedVersions" },
{ $match: { { $match: {
$or: [ $or: [
{ "affectedPackages.normalizedVersions.type": "exact", { "affectedPackages.normalizedVersions.type": "exact",
"affectedPackages.normalizedVersions.value": "4.17.21" }, "affectedPackages.normalizedVersions.value": "4.17.21" },
{ "affectedPackages.normalizedVersions.type": "range", { "affectedPackages.normalizedVersions.type": "range",
"affectedPackages.normalizedVersions.min": { $lte: "4.17.21" }, "affectedPackages.normalizedVersions.min": { $lte: "4.17.21" },
"affectedPackages.normalizedVersions.max": { $gt: "4.17.21" } }, "affectedPackages.normalizedVersions.max": { $gt: "4.17.21" } },
{ "affectedPackages.normalizedVersions.type": "gte", { "affectedPackages.normalizedVersions.type": "gte",
"affectedPackages.normalizedVersions.min": { $lte: "4.17.21" } }, "affectedPackages.normalizedVersions.min": { $lte: "4.17.21" } },
{ "affectedPackages.normalizedVersions.type": "lte", { "affectedPackages.normalizedVersions.type": "lte",
"affectedPackages.normalizedVersions.max": { $gte: "4.17.21" } } "affectedPackages.normalizedVersions.max": { $gte: "4.17.21" } }
] ]
}}, }},
{ $project: { advisoryKey: 1, title: 1, "affectedPackages.identifier": 1 } } { $project: { advisoryKey: 1, title: 1, "affectedPackages.identifier": 1 } }
]); ]);
``` ```
Pair this query with the indexes listed in [Normalized Versions Query Guide](mongo_indices.md). Pair this query with the indexes listed in [Normalized Versions Query Guide](mongo_indices.md).
## 5. Recommended indexes ## 5. Recommended indexes
| Collection | Index | Purpose | | Collection | Index | Purpose |
|------------|-------|---------| |------------|-------|---------|
| `advisory` | `{ "affectedPackages.identifier": 1, "affectedPackages.normalizedVersions.scheme": 1, "affectedPackages.normalizedVersions.type": 1 }` (compound, multikey) | Speeds up `$match` on identifier + rule style. | | `advisory` | `{ "affectedPackages.identifier": 1, "affectedPackages.normalizedVersions.scheme": 1, "affectedPackages.normalizedVersions.type": 1 }` (compound, multikey) | Speeds up `$match` on identifier + rule style. |
| `advisory` | `{ "affectedPackages.normalizedVersions.value": 1 }` (sparse) | Optimizes lookups for exact version hits. | | `advisory` | `{ "affectedPackages.normalizedVersions.value": 1 }` (sparse) | Optimizes lookups for exact version hits. |
Coordinate with the Storage team when enabling these indexes so deployment windows account for collection size. Coordinate with the Storage team when enabling these indexes so deployment windows account for collection size.
## 6. Dual-write rollout ## 6. Dual-write rollout
Follow the operational checklist in `docs/modules/devops/migrations/semver-style.md`. The summary: Follow the operational checklist in `docs/modules/devops/migrations/semver-style.md`. The summary:
1. **Dual write (now)** – emit both legacy `versionRanges` and the new `normalizedVersions`. 1. **Dual write (now)** – emit both legacy `versionRanges` and the new `normalizedVersions`.
2. **Backfill** – follow the storage migration in `docs/modules/devops/migrations/semver-style.md` to rewrite historical advisories before switching consumers. 2. **Backfill** – follow the storage migration in `docs/modules/devops/migrations/semver-style.md` to rewrite historical advisories before switching consumers.
3. **Verify** – run the aggregation above (with `explain("executionStats")`) to ensure the new indexes are used. 3. **Verify** – run the aggregation above (with `explain("executionStats")`) to ensure the new indexes are used.
4. **Cutover** – after consumers switch to normalized rules, mark the old `rangeExpression` as deprecated. 4. **Cutover** – after consumers switch to normalized rules, mark the old `rangeExpression` as deprecated.
## 7. Checklist for connectors & merge ## 7. Checklist for connectors & merge
- [ ] Populate `SemVerPrimitive` for every SemVer-friendly constraint. - [ ] Populate `SemVerPrimitive` for every SemVer-friendly constraint.
- [ ] Call `ToNormalizedVersionRule` and store the result. - [ ] Call `ToNormalizedVersionRule` and store the result.
- [ ] Emit provenance masks covering both `versionRanges[].primitives.semver` and `normalizedVersions[]`. - [ ] Emit provenance masks covering both `versionRanges[].primitives.semver` and `normalizedVersions[]`.
- [ ] Ensure merge deduping relies on the canonical comparer. - [ ] Ensure merge deduping relies on the canonical comparer.
- [ ] Capture merge decisions via `decisionReason`. - [ ] Capture merge decisions via `decisionReason`.
- [ ] Confirm integration tests include fixtures with normalized rules and SemVer styles. - [ ] Confirm integration tests include fixtures with normalized rules and SemVer styles.
For deeper query examples and maintenance tasks, continue with [Normalized Versions Query Guide](mongo_indices.md). For deeper query examples and maintenance tasks, continue with [Normalized Versions Query Guide](mongo_indices.md).
## 8. Storage projection reference ## 8. Storage projection reference
`NormalizedVersionDocumentFactory` copies each normalized rule into MongoDB using the shape below. Use this as a contract when reviewing connector fixtures or diagnosing merge/storage diffs: `NormalizedVersionDocumentFactory` copies each normalized rule into MongoDB using the shape below. Use this as a contract when reviewing connector fixtures or diagnosing merge/storage diffs:
```json ```json
{ {
"packageId": "pkg:npm/example", "packageId": "pkg:npm/example",
"packageType": "npm", "packageType": "npm",
"scheme": "semver", "scheme": "semver",
"type": "range", "type": "range",
"style": "range", "style": "range",
"min": "1.2.3", "min": "1.2.3",
"minInclusive": true, "minInclusive": true,
"max": "2.0.0", "max": "2.0.0",
"maxInclusive": false, "maxInclusive": false,
"value": null, "value": null,
"notes": "ghsa:GHSA-xxxx-yyyy", "notes": "ghsa:GHSA-xxxx-yyyy",
"decisionReason": "ghsa-precedence-over-nvd", "decisionReason": "ghsa-precedence-over-nvd",
"constraint": ">= 1.2.3 < 2.0.0", "constraint": ">= 1.2.3 < 2.0.0",
"source": "ghsa", "source": "ghsa",
"recordedAt": "2025-10-11T00:00:00Z" "recordedAt": "2025-10-11T00:00:00Z"
} }
``` ```
For distro-specific ranges (`nevra`, `evr`) the same envelope applies with `scheme` switched accordingly. Example: For distro-specific ranges (`nevra`, `evr`) the same envelope applies with `scheme` switched accordingly. Example:
```json ```json
{ {
"packageId": "bash", "packageId": "bash",
"packageType": "rpm", "packageType": "rpm",
"scheme": "nevra", "scheme": "nevra",
"type": "range", "type": "range",
"style": "range", "style": "range",
"min": "0:4.4.18-2.el7", "min": "0:4.4.18-2.el7",
"minInclusive": true, "minInclusive": true,
"max": "0:4.4.20-1.el7", "max": "0:4.4.20-1.el7",
"maxInclusive": false, "maxInclusive": false,
"value": null, "value": null,
"notes": "redhat:RHSA-2025:1234", "notes": "redhat:RHSA-2025:1234",
"decisionReason": "rhel-priority-over-nvd", "decisionReason": "rhel-priority-over-nvd",
"constraint": "<= 0:4.4.20-1.el7", "constraint": "<= 0:4.4.20-1.el7",
"source": "redhat", "source": "redhat",
"recordedAt": "2025-10-11T00:00:00Z" "recordedAt": "2025-10-11T00:00:00Z"
} }
``` ```
If a new scheme is required (for example, `apple.build` or `ios.semver`), raise it with the Models team before emitting documents so merge comparers and hashing logic can incorporate the change deterministically. If a new scheme is required (for example, `apple.build` or `ios.semver`), raise it with the Models team before emitting documents so merge comparers and hashing logic can incorporate the change deterministically.
## 9. Observability signals ## 9. Observability signals
- `concelier.merge.normalized_rules` (counter, tags: `package_type`, `scheme`) – increments once per normalized rule retained after precedence merge. - `concelier.merge.normalized_rules` (counter, tags: `package_type`, `scheme`) – increments once per normalized rule retained after precedence merge.
- `concelier.merge.normalized_rules_missing` (counter, tags: `package_type`) – increments when a merged package still carries version ranges but no normalized rules; watch for spikes to catch connectors that have not emitted normalized arrays yet. - `concelier.merge.normalized_rules_missing` (counter, tags: `package_type`) – increments when a merged package still carries version ranges but no normalized rules; watch for spikes to catch connectors that have not emitted normalized arrays yet.

56
docs/dev/raw-linkset-backfill-plan.md Normal file
View File

@@ -0,0 +1,56 @@
# Raw Linkset Backfill & Adoption Plan
_Last updated: 2025-10-31_
Owners: Concelier Storage Guild, DevOps Guild, Policy Guild
## Context
- Concelier observations now emit both a **canonical linkset** (deduped, normalised identifiers) and a **raw linkset** (`rawLinkset`) that preserves upstream ordering, duplicates, and original pointer metadata.
- Existing `concelier.advisory_observations` documents created before 2025-10-31 do **not** contain the `rawLinkset` field.
- Policy Engine selection joiners (`POLICY-ENGINE-20-003`) will switch to the raw projection once backfill completes and consumers validate fixtures.
## Objectives
1. Populate `rawLinkset` for historical observations across online clusters and Offline Kit bundles without breaking append-only guarantees.
2. Provide migration scripts + runbook so operators can rehearse in staging (and air-gapped deployments) before production rollout.
3. Unblock Policy Engine adoption by guaranteeing dual projections exist for all tenants.
## Deliverables
- [ ] **Migration script** (`20251104_advisory_observations_raw_linkset_backfill.csx`)
- Iterates observations lacking `rawLinkset`
- Rehydrates raw document via existing snapshot (or cached DTO)
- Reuses `AdvisoryObservationFactory.CreateRawLinkset`
- Writes using `$set` with optimistic retry; preserves `updatedAt` via `setOnInsert`
- [ ] **Offline Kit updater** (extend `ops/offline-kit/scripts/export_offline_bundle.py`) to patch bundles in-place
- [ ] **Runbook** covering:
- Pre-check query: `db.concelier.advisory_observations.countDocuments({ rawLinkset: { $exists: false } })`
- Backup procedure (`mongodump` or snapshot requirement)
- Dry-run mode limiting batches by tenant
- Metrics/telemetry expectations (`concelier.migrations.documents_processed_total`)
- Rollback (no-op because field addition; note to retain snapshot for verification)
- [ ] **Fixture updates** ensuring storage/CLI/Policy tests include `rawLinkset`
- [ ] **Policy Engine follow-up** to flip joiners once `rawLinkset` population reaches 100% (tracked via metrics).
## Timeline
| Date (UTC) | Milestone | Notes |
|------------|-----------|-------|
| 2025-10-31 | Handshake w/ Policy | Agreement to consume `rawLinkset`; this document created. |
| 2025-11-01 | Draft migration script | Validate against staging dataset snapshots. |
| 2025-11-04 | Storage task CONCELIER-STORE-AOC-19-005 due | Deliver script + runbook for review. |
| 2025-11-06 | Staging backfill rehearsal | Target < 30 min runtime on 5M observations. |
| 2025-11-08 | Policy fixtures updated | POL engine branch consumes `rawLinkset`. |
| 2025-11-11 | Production rollout window | Pending DevOps sign-off after rehearsals. |
## Open Questions
- Do we need archival of the canonical-only projection for backwards compatibility exports? (Policy to confirm.)
- Offline Kit delta: should we regenerate entire bundle or ship incremental patch? (DevOps reviewing.)
- Metrics: add `raw_linkset_missing_total` counter to detect regressions post-backfill?
## Next Actions
- [ ] Concelier Storage Guild: prototype migration script, share for review (`2025-11-01`).
- [ ] DevOps Guild: schedule staging rehearsal + update `docs/deploy/containers.md` with new runbook section.
- [ ] Policy Guild: prepare feature flag/branch to switch joiners once metrics show zero missing `rawLinkset`.

230
docs/events/orchestrator-scanner-events.md
View File

@@ -1,39 +1,39 @@
# Scanner Orchestrator Events (ORCH-SVC-38-101) # Scanner Orchestrator Events (ORCH-SVC-38-101)
Last updated: 2025-10-26 Last updated: 2025-10-26
The Notifications Studio initiative (NOTIFY-SVC-38-001) and orchestrator backlog (ORCH-SVC-38-101) standardise how platform services emit lifecycle events. This document describes the Scanner WebService contract for the new **orchestrator envelopes** (`scanner.event.*`) and how they supersede the legacy Redis-backed `scanner.report.ready` / `scanner.scan.completed` events. The Notifications Studio initiative (NOTIFY-SVC-38-001) and orchestrator backlog (ORCH-SVC-38-101) standardise how platform services emit lifecycle events. This document describes the Scanner WebService contract for the new **orchestrator envelopes** (`scanner.event.*`) and how they supersede the legacy Redis-backed `scanner.report.ready` / `scanner.scan.completed` events.
## 1. Envelope overview ## 1. Envelope overview
Orchestrator events share a deterministic JSON envelope: Orchestrator events share a deterministic JSON envelope:
| Field | Type | Notes | | Field | Type | Notes |
|-------|------|-------| |-------|------|-------|
| `eventId` | `uuid` | Globally unique identifier generated per occurrence. | | `eventId` | `uuid` | Globally unique identifier generated per occurrence. |
| `kind` | `string` | Event identifier; Scanner emits `scanner.event.report.ready` and `scanner.event.scan.completed`. | | `kind` | `string` | Event identifier; Scanner emits `scanner.event.report.ready` and `scanner.event.scan.completed`. |
| `version` | `integer` | Schema version. Initial release uses `1`. | | `version` | `integer` | Schema version. Initial release uses `1`. |
| `tenant` | `string` | Tenant that owns the scan/report. Mirrors Authority claims. | | `tenant` | `string` | Tenant that owns the scan/report. Mirrors Authority claims. |
| `occurredAt` | `date-time` | UTC instant when the underlying state transition happened (e.g., report persisted). | | `occurredAt` | `date-time` | UTC instant when the underlying state transition happened (e.g., report persisted). |
| `recordedAt` | `date-time` | UTC instant when the event was durably written. Optional but recommended. | | `recordedAt` | `date-time` | UTC instant when the event was durably written. Optional but recommended. |
| `source` | `string` | Producer identifier (`scanner.webservice`). | | `source` | `string` | Producer identifier (`scanner.webservice`). |
| `idempotencyKey` | `string` | Deterministic key for duplicate suppression (see §4). | | `idempotencyKey` | `string` | Deterministic key for duplicate suppression (see §4). |
| `correlationId` | `string` | Maps back to the API request or scan identifier. | | `correlationId` | `string` | Maps back to the API request or scan identifier. |
| `traceId` / `spanId` | `string` | W3C trace context propagated into downstream telemetry. | | `traceId` / `spanId` | `string` | W3C trace context propagated into downstream telemetry. |
| `scope` | `object` | Describes the affected artefact. Requires `repo` and `digest`; optional `namespace`, `component`, `image`. | | `scope` | `object` | Describes the affected artefact. Requires `repo` and `digest`; optional `namespace`, `component`, `image`. |
| `attributes` | `object` | Flat string map for frequently queried metadata (e.g., policy revision). | | `attributes` | `object` | Flat string map for frequently queried metadata (e.g., policy revision). |
| `payload` | `object` | Event-specific body (see §2). | | `payload` | `object` | Event-specific body (see §2). |
Canonical schemas live under `docs/events/scanner.event.*@1.json`. Samples that round-trip through `NotifyCanonicalJsonSerializer` are stored in `docs/events/samples/`. Canonical schemas live under `docs/events/scanner.event.*@1.json`. Samples that round-trip through `NotifyCanonicalJsonSerializer` are stored in `docs/events/samples/`.
## 2. Event kinds and payloads ## 2. Event kinds and payloads
### 2.1 `scanner.event.report.ready` ### 2.1 `scanner.event.report.ready`
Emitted once a signed report is persisted and attested. Payload highlights: Emitted once a signed report is persisted and attested. Payload highlights:
- `reportId` / `scanId` — identifiers for the persisted report and originating scan. Until Scan IDs are surfaced by the API, `scanId` mirrors `reportId` so downstream correlators can stabilise on a single key. - `reportId` / `scanId` — identifiers for the persisted report and originating scan. Until Scan IDs are surfaced by the API, `scanId` mirrors `reportId` so downstream correlators can stabilise on a single key.
- **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict` — pre-sorted for deterministic routing. - **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict` — pre-sorted for deterministic routing.
- **Links:** - **Links:**
- `report.ui` → `/ui/reports/{reportId}` on the current host. - `report.ui` → `/ui/reports/{reportId}` on the current host.
- `report.api` → `{apiBasePath}/{reportsSegment}/{reportId}` (defaults to `/api/v1/reports/{reportId}`). - `report.api` → `{apiBasePath}/{reportsSegment}/{reportId}` (defaults to `/api/v1/reports/{reportId}`).
@@ -41,83 +41,83 @@ Emitted once a signed report is persisted and attested. Payload highlights:
- `policy.api` → `{apiBasePath}/{policySegment}/revisions/{revisionId}` when a revision is present. - `policy.api` → `{apiBasePath}/{policySegment}/revisions/{revisionId}` when a revision is present.
- `attestation.ui` → `/ui/attestations/{reportId}` when a DSSE envelope is included. - `attestation.ui` → `/ui/attestations/{reportId}` when a DSSE envelope is included.
- `attestation.api` → `{apiBasePath}/{reportsSegment}/{reportId}/attestation` when a DSSE envelope is included. - `attestation.api` → `{apiBasePath}/{reportsSegment}/{reportId}/attestation` when a DSSE envelope is included.
- `imageDigest` — OCI image digest associated with the analysis. - `imageDigest` — OCI image digest associated with the analysis.
- `generatedAt` — report generation timestamp (ISO-8601 UTC). - `generatedAt` — report generation timestamp (ISO-8601 UTC).
- `verdict` — `pass`, `warn`, or `fail` after policy evaluation. - `verdict` — `pass`, `warn`, or `fail` after policy evaluation.
- `summary` — blocked/warned/ignored/quieted counters (all non-negative integers). - `summary` — blocked/warned/ignored/quieted counters (all non-negative integers).
- `delta` — newly critical/high counts and optional `kev` array. - `delta` — newly critical/high counts and optional `kev` array.
- `quietedFindingCount` — mirrors `summary.quieted`. - `quietedFindingCount` — mirrors `summary.quieted`.
- `policy` — revision metadata (`digest`, `revisionId`) surfaced for routing. - `policy` — revision metadata (`digest`, `revisionId`) surfaced for routing.
- `links` — UI/report/policy URLs suitable for operators. - `links` — UI/report/policy URLs suitable for operators.
- `dsse` — embedded DSSE envelope (payload, type, signature list). - `dsse` — embedded DSSE envelope (payload, type, signature list).
- `report` — canonical report document; identical to the DSSE payload. - `report` — canonical report document; identical to the DSSE payload.
Schema: `docs/events/scanner.event.report.ready@1.json` Schema: `docs/events/scanner.event.report.ready@1.json`
Sample: `docs/events/samples/scanner.event.report.ready@1.sample.json` Sample: `docs/events/samples/scanner.event.report.ready@1.sample.json`
### 2.2 `scanner.event.scan.completed` ### 2.2 `scanner.event.scan.completed`
Emitted after scan execution finishes (success or policy failure). Payload highlights: Emitted after scan execution finishes (success or policy failure). Payload highlights:
- `reportId` / `scanId` / `imageDigest` — identifiers mirroring the report-ready event. As with the report-ready payload, `scanId` currently mirrors `reportId` as a temporary shim. - `reportId` / `scanId` / `imageDigest` — identifiers mirroring the report-ready event. As with the report-ready payload, `scanId` currently mirrors `reportId` as a temporary shim.
- **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict`. - **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict`.
- **Links:** same as above (`report.*`, `policy.*`) with `attestation.*` populated when DSSE metadata exists. - **Links:** same as above (`report.*`, `policy.*`) with `attestation.*` populated when DSSE metadata exists.
- `verdict`, `summary`, `delta`, `policy` — same semantics as above. - `verdict`, `summary`, `delta`, `policy` — same semantics as above.
- `findings` — array of surfaced findings with `id`, `severity`, optional `cve`, `purl`, and `reachability`. - `findings` — array of surfaced findings with `id`, `severity`, optional `cve`, `purl`, and `reachability`.
- `links`, `dsse`, `report` — same structure as §2.1 (allows Notifier to reuse signatures). - `links`, `dsse`, `report` — same structure as §2.1 (allows Notifier to reuse signatures).
Schema: `docs/events/scanner.event.scan.completed@1.json` Schema: `docs/events/scanner.event.scan.completed@1.json`
Sample: `docs/events/samples/scanner.event.scan.completed@1.sample.json` Sample: `docs/events/samples/scanner.event.scan.completed@1.sample.json`
### 2.3 Relationship to legacy events ### 2.3 Relationship to legacy events
| Legacy Redis event | Replacement orchestrator event | Notes | | Legacy Redis event | Replacement orchestrator event | Notes |
|--------------------|-------------------------------|-------| |--------------------|-------------------------------|-------|
| `scanner.report.ready` | `scanner.event.report.ready` | Adds versioning, idempotency, trace context. Payload is a superset of the legacy fields. | | `scanner.report.ready` | `scanner.event.report.ready` | Adds versioning, idempotency, trace context. Payload is a superset of the legacy fields. |
| `scanner.scan.completed` | `scanner.event.scan.completed` | Same data plus explicit scan identifiers and orchestrator metadata. | | `scanner.scan.completed` | `scanner.event.scan.completed` | Same data plus explicit scan identifiers and orchestrator metadata. |
Legacy schemas remain for backwards-compatibility during migration, but new integrations **must** target the orchestrator variants. Legacy schemas remain for backwards-compatibility during migration, but new integrations **must** target the orchestrator variants.
## 3. Deterministic serialization ## 3. Deterministic serialization
- Producers must serialise events using `NotifyCanonicalJsonSerializer` to guarantee consistent key ordering and whitespace. - Producers must serialise events using `NotifyCanonicalJsonSerializer` to guarantee consistent key ordering and whitespace.
- Timestamps (`occurredAt`, `recordedAt`, `payload.generatedAt`) use `DateTimeOffset.UtcDateTime.ToString("O")`. - Timestamps (`occurredAt`, `recordedAt`, `payload.generatedAt`) use `DateTimeOffset.UtcDateTime.ToString("O")`.
- Payload arrays (`delta.kev`, `findings`) should be pre-sorted (e.g., alphabetical CVE order) so hash-based consumers remain stable. - Payload arrays (`delta.kev`, `findings`) should be pre-sorted (e.g., alphabetical CVE order) so hash-based consumers remain stable.
- Optional fields are omitted rather than emitted as `null`. - Optional fields are omitted rather than emitted as `null`.
## 4. Idempotency and correlation ## 4. Idempotency and correlation
Idempotency keys dedupe repeated publishes and align with the orchestrator’s outbox pattern: Idempotency keys dedupe repeated publishes and align with the orchestrator’s outbox pattern:
| Event kind | Idempotency key template | | Event kind | Idempotency key template |
|------------|-------------------------| |------------|-------------------------|
| `scanner.event.report.ready` | `scanner.event.report.ready:<tenant>:<reportId>` | | `scanner.event.report.ready` | `scanner.event.report.ready:<tenant>:<reportId>` |
| `scanner.event.scan.completed` | `scanner.event.scan.completed:<tenant>:<scanId>` | | `scanner.event.scan.completed` | `scanner.event.scan.completed:<tenant>:<scanId>` |
Keys are ASCII lowercase; components should be trimmed and validated before concatenation. Retries must reuse the same key. Keys are ASCII lowercase; components should be trimmed and validated before concatenation. Retries must reuse the same key.
`correlationId` should match the scan identifier that appears in REST responses (`scanId`). Re-using the same value across the pair of events allows Notifier and orchestrator analytics to stitch lifecycle data together. `correlationId` should match the scan identifier that appears in REST responses (`scanId`). Re-using the same value across the pair of events allows Notifier and orchestrator analytics to stitch lifecycle data together.
## 5. Versioning and evolution ## 5. Versioning and evolution
- Increment the `version` field and the `@<version>` suffix for **breaking** changes (field removals, type changes, semantic shifts). - Increment the `version` field and the `@<version>` suffix for **breaking** changes (field removals, type changes, semantic shifts).
- Additive optional fields may remain within version 1; update the JSON schema and samples accordingly. - Additive optional fields may remain within version 1; update the JSON schema and samples accordingly.
- When introducing `@2`, keep the `@1` schema/docs in place until orchestrator subscribers confirm migration. - When introducing `@2`, keep the `@1` schema/docs in place until orchestrator subscribers confirm migration.
## 6. Consumer checklist ## 6. Consumer checklist
1. Validate incoming payloads against the schema for the targeted version. 1. Validate incoming payloads against the schema for the targeted version.
2. Use `idempotencyKey` for dedupe, not `eventId`. 2. Use `idempotencyKey` for dedupe, not `eventId`.
3. Map `traceId`/`spanId` into telemetry spans to preserve causality. 3. Map `traceId`/`spanId` into telemetry spans to preserve causality.
4. Prefer `payload.report` → `policy.revisionId` when populating templates; the top-level `attributes` are convenience duplicates for quick routing. 4. Prefer `payload.report` → `policy.revisionId` when populating templates; the top-level `attributes` are convenience duplicates for quick routing.
5. Reserve the legacy Redis events for transitional compatibility only; downstream systems should subscribe to the orchestrator bus exposed by ORCH-SVC-38-101. 5. Reserve the legacy Redis events for transitional compatibility only; downstream systems should subscribe to the orchestrator bus exposed by ORCH-SVC-38-101.
## 7. Implementation status and next actions ## 7. Implementation status and next actions
- **Scanner WebService** — `SCANNER-EVENTS-16-301` (blocked) and `SCANNER-EVENTS-16-302` (doing) track the production of these envelopes. The remaining blocker is the .NET 10 preview OpenAPI/Auth dependency drift that currently breaks `dotnet test`. Once Gateway and Notifier owners land the replacement packages, rerun the full test suite and capture fresh fixtures under `docs/events/samples/`. - **Scanner WebService** — `SCANNER-EVENTS-16-301` (blocked) and `SCANNER-EVENTS-16-302` (doing) track the production of these envelopes. The remaining blocker is the .NET 10 preview OpenAPI/Auth dependency drift that currently breaks `dotnet test`. Once Gateway and Notifier owners land the replacement packages, rerun the full test suite and capture fresh fixtures under `docs/events/samples/`.
- **Gateway/Notifier consumers** — subscribe to the orchestrator stream documented in ORCH-SVC-38-101. When the Scanner tasks unblock, regenerate notifier contract tests against the sample events included here. - **Gateway/Notifier consumers** — subscribe to the orchestrator stream documented in ORCH-SVC-38-101. When the Scanner tasks unblock, regenerate notifier contract tests against the sample events included here.
- **Docs cadence** — update this file and the matching JSON schemas whenever payload fields change. Use the rehearsal checklist in `docs/modules/devops/runbooks/launch-cutover.md` to confirm downstream validation before the production cutover. Record gaps or newly required fields in `docs/modules/devops/runbooks/launch-readiness.md` so they land in the launch checklist. - **Docs cadence** — update this file and the matching JSON schemas whenever payload fields change. Use the rehearsal checklist in `docs/modules/devops/runbooks/launch-cutover.md` to confirm downstream validation before the production cutover. Record gaps or newly required fields in `docs/modules/devops/runbooks/launch-readiness.md` so they land in the launch checklist.
--- ---
**Imposed rule reminder:** work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. **Imposed rule reminder:** work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

3662
docs/implplan/EXECPLAN.md
View File

File diff suppressed because it is too large Load Diff

2186
docs/implplan/SPRINTS.md
View File

File diff suppressed because it is too large Load Diff

416
docs/implplan/SPRINTS_PRIOR_20251019.md
View File

@@ -1,208 +1,208 @@
Closed sprint tasks archived from SPRINTS.md on 2025-10-19. Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description | | Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
| --- | --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- | --- |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-12) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-001 | SemVer primitive range-style metadata<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/__Libraries/StellaOps.Concelier.Models/AGENTS.md. This task lays the groundwork—complete the SemVer helper updates before teammates pick up FEEDMODELS-SCHEMA-01-002/003 and FEEDMODELS-SCHEMA-02-900. Use ./src/FASTER_MODELING_AND_NORMALIZATION.md for the target rule structure. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-12) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-001 | SemVer primitive range-style metadata<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/__Libraries/StellaOps.Concelier.Models/AGENTS.md. This task lays the groundwork—complete the SemVer helper updates before teammates pick up FEEDMODELS-SCHEMA-01-002/003 and FEEDMODELS-SCHEMA-02-900. Use ./src/FASTER_MODELING_AND_NORMALIZATION.md for the target rule structure. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-11) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-002 | Provenance decision rationale field<br>Instructions to work:<br>AdvisoryProvenance now carries `decisionReason` and docs/tests were updated. Connectors and merge tasks should populate the field when applying precedence/freshness/tie-breaker logic; see src/Concelier/__Libraries/StellaOps.Concelier.Models/PROVENANCE_GUIDELINES.md for usage guidance. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-11) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-002 | Provenance decision rationale field<br>Instructions to work:<br>AdvisoryProvenance now carries `decisionReason` and docs/tests were updated. Connectors and merge tasks should populate the field when applying precedence/freshness/tie-breaker logic; see src/Concelier/__Libraries/StellaOps.Concelier.Models/PROVENANCE_GUIDELINES.md for usage guidance. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-11) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-003 | Normalized version rules collection<br>Instructions to work:<br>`AffectedPackage.NormalizedVersions` and supporting comparer/docs/tests shipped. Connector owners must emit rule arrays per ./src/FASTER_MODELING_AND_NORMALIZATION.md and report progress via FEEDMERGE-COORD-02-900 so merge/storage backfills can proceed. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-11) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-003 | Normalized version rules collection<br>Instructions to work:<br>`AffectedPackage.NormalizedVersions` and supporting comparer/docs/tests shipped. Connector owners must emit rule arrays per ./src/FASTER_MODELING_AND_NORMALIZATION.md and report progress via FEEDMERGE-COORD-02-900 so merge/storage backfills can proceed. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-12) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-02-900 | Range primitives for SemVer/EVR/NEVRA metadata<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/__Libraries/StellaOps.Concelier.Models/AGENTS.md before resuming this stalled effort. Confirm helpers align with the new `NormalizedVersions` representation so connectors finishing in Sprint 2 can emit consistent metadata. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-12) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-02-900 | Range primitives for SemVer/EVR/NEVRA metadata<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/__Libraries/StellaOps.Concelier.Models/AGENTS.md before resuming this stalled effort. Confirm helpers align with the new `NormalizedVersions` representation so connectors finishing in Sprint 2 can emit consistent metadata. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDNORM-NORM-02-001 | SemVer normalized rule emitter<br>Shared `SemVerRangeRuleBuilder` now outputs primitives + normalized rules per `FASTER_MODELING_AND_NORMALIZATION.md`; CVE/GHSA connectors consuming the API have verified fixtures. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDNORM-NORM-02-001 | SemVer normalized rule emitter<br>Shared `SemVerRangeRuleBuilder` now outputs primitives + normalized rules per `FASTER_MODELING_AND_NORMALIZATION.md`; CVE/GHSA connectors consuming the API have verified fixtures. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill<br>AdvisoryStore dual-writes flattened `normalizedVersions` when `concelier.storage.enableSemVerStyle` is set; migration `20251011-semver-style-backfill` updates historical records and docs outline the rollout. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill<br>AdvisoryStore dual-writes flattened `normalizedVersions` when `concelier.storage.enableSemVerStyle` is set; migration `20251011-semver-style-backfill` updates historical records and docs outline the rollout. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-002 | Provenance decision reason persistence<br>Storage now persists `provenance.decisionReason` for advisories and merge events; tests cover round-trips. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-002 | Provenance decision reason persistence<br>Storage now persists `provenance.decisionReason` for advisories and merge events; tests cover round-trips. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-003 | Normalized versions indexing<br>Bootstrapper seeds compound/sparse indexes for flattened normalized rules and `docs/dev/mongo_indices.md` documents query guidance. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-003 | Normalized versions indexing<br>Bootstrapper seeds compound/sparse indexes for flattened normalized rules and `docs/dev/mongo_indices.md` documents query guidance. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-TESTS-02-004 | Restore AdvisoryStore build after normalized versions refactor<br>Updated constructors/tests keep storage suites passing with the new feature flag defaults. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-TESTS-02-004 | Restore AdvisoryStore build after normalized versions refactor<br>Updated constructors/tests keep storage suites passing with the new feature flag defaults. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-ENGINE-01-002 | Plumb Authority client resilience options<br>WebService wires `authority.resilience.*` into `AddStellaOpsAuthClient` and adds binding coverage via `AuthorityClientResilienceOptionsAreBound`. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-ENGINE-01-002 | Plumb Authority client resilience options<br>WebService wires `authority.resilience.*` into `AddStellaOpsAuthClient` and adds binding coverage via `AuthorityClientResilienceOptionsAreBound`. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-003 | Author ops guidance for resilience tuning<br>Install/runbooks document connected vs air-gapped resilience profiles and monitoring hooks. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-003 | Author ops guidance for resilience tuning<br>Install/runbooks document connected vs air-gapped resilience profiles and monitoring hooks. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-004 | Document authority bypass logging patterns<br>Operator guides now call out `route/status/subject/clientId/scopes/bypass/remote` audit fields and SIEM triggers. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-004 | Document authority bypass logging patterns<br>Operator guides now call out `route/status/subject/clientId/scopes/bypass/remote` audit fields and SIEM triggers. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-005 | Update Concelier operator guide for enforcement cutoff<br>Install guide reiterates the 2025-12-31 cutoff and links audit signals to the rollout checklist. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-005 | Update Concelier operator guide for enforcement cutoff<br>Install guide reiterates the 2025-12-31 cutoff and links audit signals to the rollout checklist. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | SEC3.HOST | Rate limiter policy binding<br>Authority host now applies configuration-driven fixed windows to `/token`, `/authorize`, and `/internal/*`; integration tests assert 429 + `Retry-After` headers; docs/config samples refreshed for Docs guild diagrams. | | Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | SEC3.HOST | Rate limiter policy binding<br>Authority host now applies configuration-driven fixed windows to `/token`, `/authorize`, and `/internal/*`; integration tests assert 429 + `Retry-After` headers; docs/config samples refreshed for Docs guild diagrams. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | SEC3.BUILD | Authority rate-limiter follow-through<br>`Security.RateLimiting` now fronts token/authorize/internal limiters; Authority + Configuration matrices (`dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.sln`, `dotnet test src/__Libraries/__Tests/StellaOps.Configuration.Tests/StellaOps.Configuration.Tests.csproj`) passed on 2025-10-11; awaiting #authority-core broadcast. | | Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | SEC3.BUILD | Authority rate-limiter follow-through<br>`Security.RateLimiting` now fronts token/authorize/internal limiters; Authority + Configuration matrices (`dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.sln`, `dotnet test src/__Libraries/__Tests/StellaOps.Configuration.Tests/StellaOps.Configuration.Tests.csproj`) passed on 2025-10-11; awaiting #authority-core broadcast. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-14) | Team Authority Platform & Security Guild | AUTHCORE-BUILD-OPENIDDICT / AUTHCORE-STORAGE-DEVICE-TOKENS / AUTHCORE-BOOTSTRAP-INVITES | Address remaining Authority compile blockers (OpenIddict transaction shim, token device document, bootstrap invite cleanup) so `dotnet build src/Authority/StellaOps.Authority/StellaOps.Authority.sln` returns success. | | Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-14) | Team Authority Platform & Security Guild | AUTHCORE-BUILD-OPENIDDICT / AUTHCORE-STORAGE-DEVICE-TOKENS / AUTHCORE-BOOTSTRAP-INVITES | Address remaining Authority compile blockers (OpenIddict transaction shim, token device document, bootstrap invite cleanup) so `dotnet build src/Authority/StellaOps.Authority/StellaOps.Authority.sln` returns success. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | PLG6.DOC | Plugin developer guide polish<br>Section 9 now documents rate limiter metadata, config keys, and lockout interplay; YAML samples updated alongside Authority config templates. | | Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | PLG6.DOC | Plugin developer guide polish<br>Section 9 now documents rate limiter metadata, config keys, and lockout interplay; YAML samples updated alongside Authority config templates. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-001 | Fetch pipeline & state tracking<br>Summary planner now drives monthly/yearly VINCE fetches, persists pending summaries/notes, and hydrates VINCE detail queue with telemetry.<br>Team instructions: Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/AGENTS.md. Coordinate daily with Models/Merge leads so new normalizedVersions output and provenance tags stay aligned with ./src/FASTER_MODELING_AND_NORMALIZATION.md. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-001 | Fetch pipeline & state tracking<br>Summary planner now drives monthly/yearly VINCE fetches, persists pending summaries/notes, and hydrates VINCE detail queue with telemetry.<br>Team instructions: Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/AGENTS.md. Coordinate daily with Models/Merge leads so new normalizedVersions output and provenance tags stay aligned with ./src/FASTER_MODELING_AND_NORMALIZATION.md. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-002 | VINCE note detail fetcher<br>Summary planner queues VINCE note detail endpoints, persists raw JSON with SHA/ETag metadata, and records retry/backoff metrics. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-002 | VINCE note detail fetcher<br>Summary planner queues VINCE note detail endpoints, persists raw JSON with SHA/ETag metadata, and records retry/backoff metrics. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-003 | DTO & parser implementation<br>Added VINCE DTO aggregate, Markdown→text sanitizer, vendor/status/vulnerability parsers, and parser regression fixture. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-003 | DTO & parser implementation<br>Added VINCE DTO aggregate, Markdown→text sanitizer, vendor/status/vulnerability parsers, and parser regression fixture. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-004 | Canonical mapping & range primitives<br>VINCE DTO aggregate flows through `CertCcMapper`, emitting vendor range primitives + normalized version rules that persist via `_advisoryStore`. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-004 | Canonical mapping & range primitives<br>VINCE DTO aggregate flows through `CertCcMapper`, emitting vendor range primitives + normalized version rules that persist via `_advisoryStore`. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-005 | Deterministic fixtures/tests<br>Snapshot harness refreshed 2025-10-12; `certcc-*.snapshot.json` regenerated and regression suite green without UPDATE flag drift. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-005 | Deterministic fixtures/tests<br>Snapshot harness refreshed 2025-10-12; `certcc-*.snapshot.json` regenerated and regression suite green without UPDATE flag drift. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-006 | Telemetry & documentation<br>`CertCcDiagnostics` publishes summary/detail/parse/map metrics (meter `StellaOps.Concelier.Connector.CertCc`), README documents instruments, and log guidance captured for Ops on 2025-10-12. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-006 | Telemetry & documentation<br>`CertCcDiagnostics` publishes summary/detail/parse/map metrics (meter `StellaOps.Concelier.Connector.CertCc`), README documents instruments, and log guidance captured for Ops on 2025-10-12. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-007 | Connector test harness remediation<br>Harness now wires `AddSourceCommon`, resets `FakeTimeProvider`, and passes canned-response regression run dated 2025-10-12. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-007 | Connector test harness remediation<br>Harness now wires `AddSourceCommon`, resets `FakeTimeProvider`, and passes canned-response regression run dated 2025-10-12. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-008 | Snapshot coverage handoff<br>Fixtures regenerated with normalized ranges + provenance fields on 2025-10-11; QA handoff notes published and merge backfill unblocked. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-008 | Snapshot coverage handoff<br>Fixtures regenerated with normalized ranges + provenance fields on 2025-10-11; QA handoff notes published and merge backfill unblocked. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-012 | Schema sync & snapshot regen follow-up<br>Fixtures regenerated with normalizedVersions + provenance decision reasons; handoff notes updated for Merge backfill 2025-10-12. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-012 | Schema sync & snapshot regen follow-up<br>Fixtures regenerated with normalizedVersions + provenance decision reasons; handoff notes updated for Merge backfill 2025-10-12. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-009 | Detail/map reintegration plan<br>Staged reintegration plan published in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/FEEDCONN-CERTCC-02-009_PLAN.md`; coordinates enablement with FEEDCONN-CERTCC-02-004. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-009 | Detail/map reintegration plan<br>Staged reintegration plan published in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/FEEDCONN-CERTCC-02-009_PLAN.md`; coordinates enablement with FEEDCONN-CERTCC-02-004. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-010 | Partial-detail graceful degradation<br>Detail fetch now tolerates 404/403/410 responses and regression tests cover mixed endpoint availability. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-010 | Partial-detail graceful degradation<br>Detail fetch now tolerates 404/403/410 responses and regression tests cover mixed endpoint availability. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-REDHAT-02-001 | Fixture validation sweep<br>Instructions to work:<br>Fixtures regenerated post-model-helper rollout; provenance ordering and normalizedVersions scaffolding verified via tests. Conflict resolver deltas logged in src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Distro.RedHat/CONFLICT_RESOLVER_NOTES.md for Sprint 3 consumers. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-REDHAT-02-001 | Fixture validation sweep<br>Instructions to work:<br>Fixtures regenerated post-model-helper rollout; provenance ordering and normalizedVersions scaffolding verified via tests. Conflict resolver deltas logged in src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Distro.RedHat/CONFLICT_RESOLVER_NOTES.md for Sprint 3 consumers. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-001 | Canonical mapping & range primitives<br>Mapper emits SemVer rules (`scheme=apple:*`); fixtures regenerated with trimmed references + new RSR coverage, update tooling finalized. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-001 | Canonical mapping & range primitives<br>Mapper emits SemVer rules (`scheme=apple:*`); fixtures regenerated with trimmed references + new RSR coverage, update tooling finalized. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-002 | Deterministic fixtures/tests<br>Sanitized live fixtures + regression snapshots wired into tests; normalized rule coverage asserted. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-002 | Deterministic fixtures/tests<br>Sanitized live fixtures + regression snapshots wired into tests; normalized rule coverage asserted. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-003 | Telemetry & documentation<br>Apple meter metrics wired into Concelier WebService OpenTelemetry configuration; README and fixtures document normalizedVersions coverage. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-003 | Telemetry & documentation<br>Apple meter metrics wired into Concelier WebService OpenTelemetry configuration; README and fixtures document normalizedVersions coverage. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-004 | Live HTML regression sweep<br>Sanitised HT125326/HT125328/HT106355/HT214108/HT215500 fixtures recorded and regression tests green on 2025-10-12. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-004 | Live HTML regression sweep<br>Sanitised HT125326/HT125328/HT106355/HT214108/HT215500 fixtures recorded and regression tests green on 2025-10-12. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-005 | Fixture regeneration tooling<br>`UPDATE_APPLE_FIXTURES=1` flow fetches & rewrites fixtures; README documents usage.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/AGENTS.md. Resume stalled tasks, ensuring normalizedVersions output and fixtures align with ./src/FASTER_MODELING_AND_NORMALIZATION.md before handing data to the conflict sprint. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-005 | Fixture regeneration tooling<br>`UPDATE_APPLE_FIXTURES=1` flow fetches & rewrites fixtures; README documents usage.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/AGENTS.md. Resume stalled tasks, ensuring normalizedVersions output and fixtures align with ./src/FASTER_MODELING_AND_NORMALIZATION.md before handing data to the conflict sprint. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `src/Tools/FixtureUpdater` updates across connectors. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `src/Tools/FixtureUpdater` updates across connectors. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-CVE-02-003 | CVE normalized versions uplift | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-CVE-02-003 | CVE normalized versions uplift |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-KEV-02-003 | KEV normalized versions propagation | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-KEV-02-003 | KEV normalized versions propagation |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-04-003 | OSV parity fixture refresh | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-04-003 | OSV parity fixture refresh |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-10) | Team WebService & Authority | FEEDWEB-DOCS-01-001 | Document authority toggle & scope requirements<br>Quickstart carries toggle/scope guidance pending docs guild review (no change this sprint). | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-10) | Team WebService & Authority | FEEDWEB-DOCS-01-001 | Document authority toggle & scope requirements<br>Quickstart carries toggle/scope guidance pending docs guild review (no change this sprint). |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-003 | Author ops guidance for resilience tuning<br>Operator docs now outline connected vs air-gapped resilience profiles and monitoring cues. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-003 | Author ops guidance for resilience tuning<br>Operator docs now outline connected vs air-gapped resilience profiles and monitoring cues. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-004 | Document authority bypass logging patterns<br>Audit logging guidance highlights `route/status/subject/clientId/scopes/bypass/remote` fields and SIEM alerts. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-004 | Document authority bypass logging patterns<br>Audit logging guidance highlights `route/status/subject/clientId/scopes/bypass/remote` fields and SIEM alerts. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-005 | Update Concelier operator guide for enforcement cutoff<br>Install guide reiterates the 2025-12-31 cutoff and ties audit signals to rollout checks. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-005 | Update Concelier operator guide for enforcement cutoff<br>Install guide reiterates the 2025-12-31 cutoff and ties audit signals to rollout checks. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs, tests, and docs now target `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs, tests, and docs now target `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | FEEDWEB-OPS-01-007 | Authority resilience adoption<br>Deployment docs and CLI notes explain the LIB5 resilience knobs for rollout.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.WebService/AGENTS.md. These items were mid-flight; resume implementation ensuring docs/operators receive timely updates. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | FEEDWEB-OPS-01-007 | Authority resilience adoption<br>Deployment docs and CLI notes explain the LIB5 resilience knobs for rollout.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.WebService/AGENTS.md. These items were mid-flight; resume implementation ensuring docs/operators receive timely updates. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHCORE-ENGINE-01-001 | CORE8.RL — Rate limiter plumbing validated; integration tests green and docs handoff recorded for middleware ordering + Retry-After headers (see `docs/dev/authority-rate-limit-tuning-outline.md` for continuing guidance). | | Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHCORE-ENGINE-01-001 | CORE8.RL — Rate limiter plumbing validated; integration tests green and docs handoff recorded for middleware ordering + Retry-After headers (see `docs/dev/authority-rate-limit-tuning-outline.md` for continuing guidance). |
| Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHCRYPTO-ENGINE-01-001 | SEC3.A — Shared metadata resolver confirmed via host test run; SEC3.B now unblocked for tuning guidance (outline captured in `docs/dev/authority-rate-limit-tuning-outline.md`). | | Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHCRYPTO-ENGINE-01-001 | SEC3.A — Shared metadata resolver confirmed via host test run; SEC3.B now unblocked for tuning guidance (outline captured in `docs/dev/authority-rate-limit-tuning-outline.md`). |
| Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-13) | Team Authority Platform & Security Guild | AUTHSEC-DOCS-01-002 | SEC3.B — Published `docs/security/rate-limits.md` with tuning matrix, alert thresholds, and lockout interplay guidance; Docs guild can lift copy into plugin guide. | | Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-13) | Team Authority Platform & Security Guild | AUTHSEC-DOCS-01-002 | SEC3.B — Published `docs/security/rate-limits.md` with tuning matrix, alert thresholds, and lockout interplay guidance; Docs guild can lift copy into plugin guide. |
| Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-14) | Team Authority Platform & Security Guild | AUTHSEC-CRYPTO-02-001 | SEC5.B1 — Introduce libsodium signing provider and parity tests to unblock CLI verification enhancements. | | Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-14) | Team Authority Platform & Security Guild | AUTHSEC-CRYPTO-02-001 | SEC5.B1 — Introduce libsodium signing provider and parity tests to unblock CLI verification enhancements. |
| Sprint 1 | Bootstrap & Replay Hardening | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-14) | Security Guild | AUTHSEC-CRYPTO-02-004 | SEC5.D/E — Finish bootstrap invite lifecycle (API/store/cleanup) and token device heuristics; build currently red due to pending handler integration. | | Sprint 1 | Bootstrap & Replay Hardening | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-14) | Security Guild | AUTHSEC-CRYPTO-02-004 | SEC5.D/E — Finish bootstrap invite lifecycle (API/store/cleanup) and token device heuristics; build currently red due to pending handler integration. |
| Sprint 1 | Developer Tooling | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-15) | DevEx/CLI | AUTHCLI-DIAG-01-001 | Surface password policy diagnostics in CLI startup/output so operators see weakened overrides immediately.<br>CLI now loads Authority plug-ins at startup, logs weakened password policies (length/complexity), and regression coverage lives in `StellaOps.Cli.Tests/Services/AuthorityDiagnosticsReporterTests`. | | Sprint 1 | Developer Tooling | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-15) | DevEx/CLI | AUTHCLI-DIAG-01-001 | Surface password policy diagnostics in CLI startup/output so operators see weakened overrides immediately.<br>CLI now loads Authority plug-ins at startup, logs weakened password policies (length/complexity), and regression coverage lives in `StellaOps.Cli.Tests/Services/AuthorityDiagnosticsReporterTests`. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHPLUG-DOCS-01-001 | PLG6.DOC — Developer guide copy + diagrams merged 2025-10-11; limiter guidance incorporated and handed to Docs guild for asset export. | | Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHPLUG-DOCS-01-001 | PLG6.DOC — Developer guide copy + diagrams merged 2025-10-11; limiter guidance incorporated and handed to Docs guild for asset export. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md | DONE (2025-10-12) | Team Normalization & Storage Backbone | FEEDNORM-NORM-02-001 | SemVer normalized rule emitter<br>`SemVerRangeRuleBuilder` shipped 2025-10-12 with comparator/`||` support and fixtures aligning to `FASTER_MODELING_AND_NORMALIZATION.md`. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md | DONE (2025-10-12) | Team Normalization & Storage Backbone | FEEDNORM-NORM-02-001 | SemVer normalized rule emitter<br>`SemVerRangeRuleBuilder` shipped 2025-10-12 with comparator/`||` support and fixtures aligning to `FASTER_MODELING_AND_NORMALIZATION.md`. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-002 | Provenance decision reason persistence | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-002 | Provenance decision reason persistence |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-003 | Normalized versions indexing<br>Indexes seeded + docs updated 2025-10-11 to cover flattened normalized rules for connector adoption. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-003 | Normalized versions indexing<br>Indexes seeded + docs updated 2025-10-11 to cover flattened normalized rules for connector adoption. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDMERGE-ENGINE-02-002 | Normalized versions union & dedupe<br>Affected package resolver unions/dedupes normalized rules, stamps merge provenance with `decisionReason`, and tests cover the rollout. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDMERGE-ENGINE-02-002 | Normalized versions union & dedupe<br>Affected package resolver unions/dedupes normalized rules, stamps merge provenance with `decisionReason`, and tests cover the rollout. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-004 | GHSA credits & ecosystem severity mapping | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-004 | GHSA credits & ecosystem severity mapping |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-005 | GitHub quota monitoring & retries | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-005 | GitHub quota monitoring & retries |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-006 | Production credential & scheduler rollout | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-006 | Production credential & scheduler rollout |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-007 | Credit parity regression fixtures | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-007 | Credit parity regression fixtures |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-004 | NVD CVSS & CWE precedence payloads | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-004 | NVD CVSS & CWE precedence payloads |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-005 | NVD merge/export parity regression | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-005 | NVD merge/export parity regression |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-004 | OSV references & credits alignment | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-004 | OSV references & credits alignment |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-005 | Fixture updater workflow<br>Resolved 2025-10-12: OSV mapper now derives canonical PURLs for Go + scoped npm packages when raw payloads omit `purl`; conflict fixtures unchanged for invalid npm names. Verified via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests`, and backbone normalization/storage suites. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-005 | Fixture updater workflow<br>Resolved 2025-10-12: OSV mapper now derives canonical PURLs for Go + scoped npm packages when raw payloads omit `purl`; conflict fixtures unchanged for invalid npm names. Verified via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests`, and backbone normalization/storage suites. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Acsc/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ACSC-02-001 … 02-008 | Fetch→parse→map pipeline, fixtures, diagnostics, and README finished 2025-10-12; downstream export parity captured via FEEDEXPORT-JSON-04-001 / FEEDEXPORT-TRIVY-04-001 (completed). | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Acsc/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ACSC-02-001 … 02-008 | Fetch→parse→map pipeline, fixtures, diagnostics, and README finished 2025-10-12; downstream export parity captured via FEEDEXPORT-JSON-04-001 / FEEDEXPORT-TRIVY-04-001 (completed). |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cccs/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CCCS-02-001 … 02-008 | Observability meter, historical harvest plan, and DOM sanitizer refinements wrapped; ops notes live under `docs/modules/concelier/operations/connectors/cccs.md` with fixtures validating EN/FR list handling. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cccs/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CCCS-02-001 … 02-008 | Observability meter, historical harvest plan, and DOM sanitizer refinements wrapped; ops notes live under `docs/modules/concelier/operations/connectors/cccs.md` with fixtures validating EN/FR list handling. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertBund/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CERTBUND-02-001 … 02-008 | Telemetry/docs (02-006) and history/locale sweep (02-007) completed alongside pipeline; runbook `docs/modules/concelier/operations/connectors/certbund.md` captures locale guidance and offline packaging. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertBund/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CERTBUND-02-001 … 02-008 | Telemetry/docs (02-006) and history/locale sweep (02-007) completed alongside pipeline; runbook `docs/modules/concelier/operations/connectors/certbund.md` captures locale guidance and offline packaging. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kisa/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-KISA-02-001 … 02-007 | Connector, tests, and telemetry/docs (02-006) finalized; localisation notes in `docs/dev/kisa_connector_notes.md` complete rollout. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kisa/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-KISA-02-001 … 02-007 | Connector, tests, and telemetry/docs (02-006) finalized; localisation notes in `docs/dev/kisa_connector_notes.md` complete rollout. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-RUBDU-02-001 … 02-008 | Fetch/parser/mapper refinements, regression fixtures, telemetry/docs, access options, and trusted root packaging all landed; README documents offline access strategy. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-RUBDU-02-001 … 02-008 | Fetch/parser/mapper refinements, regression fixtures, telemetry/docs, access options, and trusted root packaging all landed; README documents offline access strategy. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md | DONE (2025-10-13) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-NKCKI-02-001 … 02-008 | Listing fetch, parser, mapper, fixtures, telemetry/docs, and archive plan finished; Mongo2Go/libcrypto dependency resolved via bundled OpenSSL noted in ops guide. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md | DONE (2025-10-13) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-NKCKI-02-001 … 02-008 | Listing fetch, parser, mapper, fixtures, telemetry/docs, and archive plan finished; Mongo2Go/libcrypto dependency resolved via bundled OpenSSL noted in ops guide. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ICSCISA-02-001 … 02-011 | Feed parser attachment fixes, SemVer exact values, regression suites, telemetry/docs updates, and handover complete; ops runbook now details attachment verification + proxy usage. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ICSCISA-02-001 … 02-011 | Feed parser attachment fixes, SemVer exact values, regression suites, telemetry/docs updates, and handover complete; ops runbook now details attachment verification + proxy usage. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CISCO-02-001 … 02-007 | OAuth fetch pipeline, DTO/mapping, tests, and telemetry/docs shipped; monitoring/export integration follow-ups recorded in Ops docs and exporter backlog (completed). | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CISCO-02-001 … 02-007 | OAuth fetch pipeline, DTO/mapping, tests, and telemetry/docs shipped; monitoring/export integration follow-ups recorded in Ops docs and exporter backlog (completed). |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-MSRC-02-001 … 02-008 | Azure AD onboarding (02-008) unblocked fetch/parse/map pipeline; fixtures, telemetry/docs, and Offline Kit guidance published in `docs/modules/concelier/operations/connectors/msrc.md`. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-MSRC-02-001 … 02-008 | Azure AD onboarding (02-008) unblocked fetch/parse/map pipeline; fixtures, telemetry/docs, and Offline Kit guidance published in `docs/modules/concelier/operations/connectors/msrc.md`. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-15) | Team Connector Support & Monitoring | FEEDCONN-CVE-02-001 … 02-002 | CVE data-source selection, fetch pipeline, and docs landed 2025-10-10. 2025-10-15: smoke verified using the seeded mirror fallback; connector now logs a warning and pulls from `seed-data/cve/` until live CVE Services credentials arrive. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-15) | Team Connector Support & Monitoring | FEEDCONN-CVE-02-001 … 02-002 | CVE data-source selection, fetch pipeline, and docs landed 2025-10-10. 2025-10-15: smoke verified using the seeded mirror fallback; connector now logs a warning and pulls from `seed-data/cve/` until live CVE Services credentials arrive. |
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Support & Monitoring | FEEDCONN-KEV-02-001 … 02-002 | KEV catalog ingestion, fixtures, telemetry, and schema validation completed 2025-10-12; ops dashboard published. | | Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Support & Monitoring | FEEDCONN-KEV-02-001 … 02-002 | KEV catalog ingestion, fixtures, telemetry, and schema validation completed 2025-10-12; ops dashboard published. |
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-01-001 | Canonical schema docs refresh<br>Updated canonical schema + provenance guides with SemVer style, normalized version rules, decision reason change log, and migration notes. | | Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-01-001 | Canonical schema docs refresh<br>Updated canonical schema + provenance guides with SemVer style, normalized version rules, decision reason change log, and migration notes. |
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-02-001 | Concelier-SemVer Playbook<br>Published merge playbook covering mapper patterns, dedupe flow, indexes, and rollout checklist. | | Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-02-001 | Concelier-SemVer Playbook<br>Published merge playbook covering mapper patterns, dedupe flow, indexes, and rollout checklist. |
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-02-002 | Normalized versions query guide<br>Delivered Mongo index/query addendum with `$unwind` recipes, dedupe checks, and operational checklist.<br>Instructions to work:<br>DONE Read ./AGENTS.md and docs/AGENTS.md. Document every schema/index/query change produced in Sprint 1-2 leveraging ./src/FASTER_MODELING_AND_NORMALIZATION.md. | | Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-02-002 | Normalized versions query guide<br>Delivered Mongo index/query addendum with `$unwind` recipes, dedupe checks, and operational checklist.<br>Instructions to work:<br>DONE Read ./AGENTS.md and docs/AGENTS.md. Document every schema/index/query change produced in Sprint 1-2 leveraging ./src/FASTER_MODELING_AND_NORMALIZATION.md. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-03-001 | Canonical merger implementation<br>`CanonicalMerger` ships with freshness/tie-breaker logic, provenance, and unit coverage feeding Merge. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-03-001 | Canonical merger implementation<br>`CanonicalMerger` ships with freshness/tie-breaker logic, provenance, and unit coverage feeding Merge. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-03-002 | Field precedence and tie-breaker map<br>Field precedence tables and tie-breaker metrics wired into the canonical merge flow; docs/tests updated.<br>Instructions to work:<br>Read ./AGENTS.md and core AGENTS. Implement the conflict resolver exactly as specified in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md, coordinating with Merge and Storage teammates. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-03-002 | Field precedence and tie-breaker map<br>Field precedence tables and tie-breaker metrics wired into the canonical merge flow; docs/tests updated.<br>Instructions to work:<br>Read ./AGENTS.md and core AGENTS. Implement the conflict resolver exactly as specified in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md, coordinating with Merge and Storage teammates. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-DATA-03-001 | Merge event provenance audit prep<br>Merge events now persist `fieldDecisions` and analytics-ready provenance snapshots. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-DATA-03-001 | Merge event provenance audit prep<br>Merge events now persist `fieldDecisions` and analytics-ready provenance snapshots. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill<br>Dual-write/backfill flag delivered; migration + options validated in tests. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill<br>Dual-write/backfill flag delivered; migration + options validated in tests. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-TESTS-02-004 | Restore AdvisoryStore build after normalized versions refactor<br>Storage tests adjusted for normalized versions/decision reasons.<br>Instructions to work:<br>Read ./AGENTS.md and storage AGENTS. Extend merge events with decision reasons and analytics views to support the conflict rules, and deliver the dual-write/backfill for `NormalizedVersions` + `decisionReason` so connectors can roll out safely. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-TESTS-02-004 | Restore AdvisoryStore build after normalized versions refactor<br>Storage tests adjusted for normalized versions/decision reasons.<br>Instructions to work:<br>Read ./AGENTS.md and storage AGENTS. Extend merge events with decision reasons and analytics views to support the conflict rules, and deliver the dual-write/backfill for `NormalizedVersions` + `decisionReason` so connectors can roll out safely. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-001 | GHSA/NVD/OSV conflict rules<br>Merge pipeline consumes `CanonicalMerger` output prior to precedence merge. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-001 | GHSA/NVD/OSV conflict rules<br>Merge pipeline consumes `CanonicalMerger` output prior to precedence merge. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-002 | Override metrics instrumentation<br>Merge events capture per-field decisions; counters/logs align with conflict rules. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-002 | Override metrics instrumentation<br>Merge events capture per-field decisions; counters/logs align with conflict rules. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-003 | Reference & credit union pipeline<br>Canonical merge preserves unions with updated tests. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-003 | Reference & credit union pipeline<br>Canonical merge preserves unions with updated tests. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-QA-04-001 | End-to-end conflict regression suite<br>Added regression tests (`AdvisoryMergeServiceTests`) covering canonical + precedence flow.<br>Instructions to work:<br>Read ./AGENTS.md and merge AGENTS. Integrate the canonical merger, instrument metrics, and deliver comprehensive regression tests following ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-QA-04-001 | End-to-end conflict regression suite<br>Added regression tests (`AdvisoryMergeServiceTests`) covering canonical + precedence flow.<br>Instructions to work:<br>Read ./AGENTS.md and merge AGENTS. Integrate the canonical merger, instrument metrics, and deliver comprehensive regression tests following ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md. |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-002 | GHSA conflict regression fixtures | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-002 | GHSA conflict regression fixtures |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-NVD-04-002 | NVD conflict regression fixtures | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-NVD-04-002 | NVD conflict regression fixtures |
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-OSV-04-002 | OSV conflict regression fixtures<br>Instructions to work:<br>Read ./AGENTS.md and module AGENTS. Produce fixture triples supporting the precedence/tie-breaker paths defined in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md and hand them to Merge QA. | | Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-OSV-04-002 | OSV conflict regression fixtures<br>Instructions to work:<br>Read ./AGENTS.md and module AGENTS. Produce fixture triples supporting the precedence/tie-breaker paths defined in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md and hand them to Merge QA. |
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-11) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-001 | Concelier Conflict Rules<br>Runbook published at `docs/modules/concelier/operations/conflict-resolution.md`; metrics/log guidance aligned with Sprint 3 merge counters. | | Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-11) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-001 | Concelier Conflict Rules<br>Runbook published at `docs/modules/concelier/operations/conflict-resolution.md`; metrics/log guidance aligned with Sprint 3 merge counters. |
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-16) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-002 | Conflict runbook ops rollout<br>Ops review completed, alert thresholds applied, and change log appended in `docs/modules/concelier/operations/conflict-resolution.md`; task closed after connector signals verified. | | Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-16) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-002 | Conflict runbook ops rollout<br>Ops review completed, alert thresholds applied, and change log appended in `docs/modules/concelier/operations/conflict-resolution.md`; task closed after connector signals verified. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-15) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-04-001 | Advisory schema parity (description/CWE/canonical metric)<br>Extend `Advisory` and related records with description text, CWE collection, and canonical metric pointer; refresh validation + serializer determinism tests. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-15) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-04-001 | Advisory schema parity (description/CWE/canonical metric)<br>Extend `Advisory` and related records with description text, CWE collection, and canonical metric pointer; refresh validation + serializer determinism tests. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-003 | Canonical merger parity for new fields<br>Teach `CanonicalMerger` to populate description, CWEResults, and canonical metric pointer with provenance + regression coverage. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-003 | Canonical merger parity for new fields<br>Teach `CanonicalMerger` to populate description, CWEResults, and canonical metric pointer with provenance + regression coverage. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-004 | Reference normalization & freshness instrumentation cleanup<br>Implement URL normalization for reference dedupe, align freshness-sensitive instrumentation, and add analytics tests. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-004 | Reference normalization & freshness instrumentation cleanup<br>Implement URL normalization for reference dedupe, align freshness-sensitive instrumentation, and add analytics tests. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-004 | Merge pipeline parity for new advisory fields<br>Ensure merge service + merge events surface description/CWE/canonical metric decisions with updated metrics/tests. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-004 | Merge pipeline parity for new advisory fields<br>Ensure merge service + merge events surface description/CWE/canonical metric decisions with updated metrics/tests. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` validated canonical metric/CWE emission. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` validated canonical metric/CWE emission. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-16) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-16) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. | | Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-15) | Team Excititor Policy | EXCITITOR-POLICY-01-001 | Established policy options & snapshot provider covering baseline weights/overrides. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-15) | Team Excititor Policy | EXCITITOR-POLICY-01-001 | Established policy options & snapshot provider covering baseline weights/overrides. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-15) | Team Excititor Policy | EXCITITOR-POLICY-01-002 | Policy evaluator now feeds consensus resolver with immutable snapshots. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-15) | Team Excititor Policy | EXCITITOR-POLICY-01-002 | Policy evaluator now feeds consensus resolver with immutable snapshots. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-003 | Author policy diagnostics, CLI/WebService surfacing, and documentation updates. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-003 | Author policy diagnostics, CLI/WebService surfacing, and documentation updates. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Excititor Storage | EXCITITOR-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Excititor Storage | EXCITITOR-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-16) | Team Excititor Storage | EXCITITOR-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-16) | Team Excititor Storage | EXCITITOR-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-15) | Team Excititor Export | EXCITITOR-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-15) | Team Excititor Export | EXCITITOR-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-17) | Team Excititor Export | EXCITITOR-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-17) | Team Excititor Export | EXCITITOR-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors | EXCITITOR-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. | | Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors | EXCITITOR-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. |
| Sprint 5 | Excititor Core Foundations | src/Excititor/StellaOps.Excititor.WebService/TASKS.md | DONE (2025-10-17) | Team Excititor WebService | EXCITITOR-WEB-01-001 | Scaffold minimal API host, DI, and `/excititor/status` endpoint integrating policy, storage, export, and attestation services. | | Sprint 5 | Excititor Core Foundations | src/Excititor/StellaOps.Excititor.WebService/TASKS.md | DONE (2025-10-17) | Team Excititor WebService | EXCITITOR-WEB-01-001 | Scaffold minimal API host, DI, and `/excititor/status` endpoint integrating policy, storage, export, and attestation services. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/StellaOps.Excititor.Worker/TASKS.md | DONE (2025-10-17) | Team Excititor Worker | EXCITITOR-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/StellaOps.Excititor.Worker/TASKS.md | DONE (2025-10-17) | Team Excititor Worker | EXCITITOR-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-002 | Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-002 | Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-003 | Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-003 | Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-004 | Persist resume cursors (last updated timestamp/document hashes) in storage and reload during fetch to avoid duplicates. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-004 | Persist resume cursors (last updated timestamp/document hashes) in storage and reload during fetch to avoid duplicates. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-005 | Register connector in Worker/WebService DI, add scheduled jobs, and document CLI triggers for Red Hat CSAF pulls. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-005 | Register connector in Worker/WebService DI, add scheduled jobs, and document CLI triggers for Red Hat CSAF pulls. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-006 | Add CSAF normalization parity fixtures ensuring RHSA-specific metadata is preserved. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-006 | Add CSAF normalization parity fixtures ensuring RHSA-specific metadata is preserved. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Cisco | EXCITITOR-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Cisco | EXCITITOR-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Cisco | EXCITITOR-CONN-CISCO-01-002 | Implement Cisco CSAF paginated fetch loop with dedupe and raw persistence support. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Cisco | EXCITITOR-CONN-CISCO-01-002 | Implement Cisco CSAF paginated fetch loop with dedupe and raw persistence support. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – SUSE | EXCITITOR-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – SUSE | EXCITITOR-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – MSRC | EXCITITOR-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – MSRC | EXCITITOR-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Oracle | EXCITITOR-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Oracle | EXCITITOR-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Ubuntu | EXCITITOR-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Ubuntu | EXCITITOR-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-002 | Attestation fetch & verify loop – download DSSE attestations, trigger verification, handle retries/backoff, persist raw statements. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-002 | Attestation fetch & verify loop – download DSSE attestations, trigger verification, handle retries/backoff, persist raw statements. |
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-003 | Provenance metadata & policy hooks – emit image, subject digest, issuer, and trust metadata for policy weighting/logging. | | Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-003 | Provenance metadata & policy hooks – emit image, subject digest, issuer, and trust metadata for policy weighting/logging. |
| Sprint 6 | Excititor Ingest & Formats | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-18) | DevEx/CLI | EXCITITOR-CLI-01-001 | Add `excititor` CLI verbs bridging to WebService with consistent auth and offline UX. | | Sprint 6 | Excititor Ingest & Formats | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-18) | DevEx/CLI | EXCITITOR-CLI-01-001 | Add `excititor` CLI verbs bridging to WebService with consistent auth and offline UX. |
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-19) | Team Excititor Core & Policy | EXCITITOR-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. | | Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-19) | Team Excititor Core & Policy | EXCITITOR-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. |
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-19) | Team Excititor Policy | EXCITITOR-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. | | Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-19) | Team Excititor Policy | EXCITITOR-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. |
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. | | Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-501 | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `modules/scanner/architecture.md` §3–§4. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-501 | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `modules/scanner/architecture.md` §3–§4. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-502 | Observability helpers (correlation IDs, logging scopes, metric namespacing, deterministic hashes) consumed by WebService/Worker. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-502 | Observability helpers (correlation IDs, logging scopes, metric namespacing, deterministic hashes) consumed by WebService/Worker. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-503 | Security utilities: Authority client factory, OpTok caching, DPoP verifier, restart-time plug-in guardrails for scanner components. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-503 | Security utilities: Authority client factory, OpTok caching, DPoP verifier, restart-time plug-in guardrails for scanner components. |
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-001 | Buildx driver scaffold + handshake with Scanner.Emit (local CAS). | | Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-001 | Buildx driver scaffold + handshake with Scanner.Emit (local CAS). |
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-002 | OCI annotations + provenance hand-off to Attestor. | | Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-002 | OCI annotations + provenance hand-off to Attestor. |
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-003 | CI demo: minimal SBOM push & backend report wiring. | | Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-003 | CI demo: minimal SBOM push & backend report wiring. |
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-004 | Stabilize descriptor nonce derivation so repeated builds emit deterministic placeholders. | | Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-004 | Stabilize descriptor nonce derivation so repeated builds emit deterministic placeholders. |
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-005 | Integrate determinism guard into GitHub/Gitea workflows and archive proof artifacts. | | Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-005 | Integrate determinism guard into GitHub/Gitea workflows and archive proof artifacts. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-18) | Team Scanner WebService | SCANNER-WEB-09-101 | Minimal API host with Authority enforcement, health/ready endpoints, and restart-time plug-in loader per architecture §1, §4. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-18) | Team Scanner WebService | SCANNER-WEB-09-101 | Minimal API host with Authority enforcement, health/ready endpoints, and restart-time plug-in loader per architecture §1, §4. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-18) | Team Scanner WebService | SCANNER-WEB-09-102 | `/api/v1/scans` submission/status endpoints with deterministic IDs, validation, and cancellation support. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-18) | Team Scanner WebService | SCANNER-WEB-09-102 | `/api/v1/scans` submission/status endpoints with deterministic IDs, validation, and cancellation support. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-WEB-09-104 | Configuration binding for Mongo, MinIO, queue, feature flags; startup diagnostics and fail-fast policy. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-WEB-09-104 | Configuration binding for Mongo, MinIO, queue, feature flags; startup diagnostics and fail-fast policy. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-201 | Worker host bootstrap with Authority auth, hosted services, and graceful shutdown semantics. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-201 | Worker host bootstrap with Authority auth, hosted services, and graceful shutdown semantics. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-202 | Lease/heartbeat loop with retry+jitter, poison-job quarantine, structured logging. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-202 | Lease/heartbeat loop with retry+jitter, poison-job quarantine, structured logging. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-203 | Analyzer dispatch skeleton emitting deterministic stage progress and honoring cancellation tokens. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-203 | Analyzer dispatch skeleton emitting deterministic stage progress and honoring cancellation tokens. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-204 | Worker metrics (queue latency, stage duration, failure counts) with OpenTelemetry resource wiring. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-204 | Worker metrics (queue latency, stage duration, failure counts) with OpenTelemetry resource wiring. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-205 | Harden heartbeat jitter so lease safety margin stays ≥3× and cover with regression tests + optional live queue smoke run. | | Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-205 | Harden heartbeat jitter so lease safety margin stays ≥3× and cover with regression tests + optional live queue smoke run. |
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-001 | Policy schema + binder + diagnostics. | | Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-001 | Policy schema + binder + diagnostics. |
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-002 | Policy snapshot store + revision digests. | | Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-002 | Policy snapshot store + revision digests. |
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-003 | `/policy/preview` API (image digest → projected verdict diff). | | Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-003 | `/policy/preview` API (image digest → projected verdict diff). |
| Sprint 9 | DevOps Foundations | ops/devops/TASKS.md | DONE (2025-10-19) | DevOps Guild | DEVOPS-HELM-09-001 | Helm/Compose environment profiles (dev/staging/airgap) with deterministic digests. | | Sprint 9 | DevOps Foundations | ops/devops/TASKS.md | DONE (2025-10-19) | DevOps Guild | DEVOPS-HELM-09-001 | Helm/Compose environment profiles (dev/staging/airgap) with deterministic digests. |
| Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-19) | Docs Guild, DevEx | DOCS-ADR-09-001 | Establish ADR process and template. | | Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-19) | Docs Guild, DevEx | DOCS-ADR-09-001 | Establish ADR process and template. |
| Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-19) | Docs Guild, Platform Events | DOCS-EVENTS-09-002 | Publish event schema catalog (`docs/events/`) for critical envelopes. | | Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-19) | Docs Guild, Platform Events | DOCS-EVENTS-09-002 | Publish event schema catalog (`docs/events/`) for critical envelopes. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-301 | Mongo catalog schemas/indexes for images, layers, artifacts, jobs, lifecycle rules plus migrations. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-301 | Mongo catalog schemas/indexes for images, layers, artifacts, jobs, lifecycle rules plus migrations. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-302 | MinIO layout, immutability policies, client abstraction, and configuration binding. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-302 | MinIO layout, immutability policies, client abstraction, and configuration binding. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-303 | Repositories/services with dual-write feature flag, deterministic digests, TTL enforcement tests. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-303 | Repositories/services with dual-write feature flag, deterministic digests, TTL enforcement tests. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-401 | Queue abstraction + Redis Streams adapter with ack/claim APIs and idempotency tokens. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-401 | Queue abstraction + Redis Streams adapter with ack/claim APIs and idempotency tokens. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-402 | Pluggable backend support (Redis, NATS) with configuration binding, health probes, failover docs. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-402 | Pluggable backend support (Redis, NATS) with configuration binding, health probes, failover docs. |
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-403 | Retry + dead-letter strategy with structured logs/metrics for offline deployments. | | Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-403 | Retry + dead-letter strategy with structured logs/metrics for offline deployments. |
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `src/Tools/FixtureUpdater` updates across connectors.<br>Progress 2025-10-20: Coordination matrix + rollout dashboard refreshed; upcoming deadlines tracked (Cccs/Cisco 2025-10-21, CertBund 2025-10-22, ICS-CISA 2025-10-23, KISA 2025-10-24) with escalation path documented in FEEDMERGE-COORD-02-900.| | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `src/Tools/FixtureUpdater` updates across connectors.<br>Progress 2025-10-20: Coordination matrix + rollout dashboard refreshed; upcoming deadlines tracked (Cccs/Cisco 2025-10-21, CertBund 2025-10-22, ICS-CISA 2025-10-23, KISA 2025-10-24) with escalation path documented in FEEDMERGE-COORD-02-900.|
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs now point at `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`; defaults/docs/tests updated to reflect the new layout. | | Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs now point at `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`; defaults/docs/tests updated to reflect the new layout. |
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-02-001 | Statement events & scoring signals – immutable VEX statements store, consensus signal fields, and migration `20251019-consensus-signals-statements` with tests (`dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`, `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`). | | Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-02-001 | Statement events & scoring signals – immutable VEX statements store, consensus signal fields, and migration `20251019-consensus-signals-statements` with tests (`dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`, `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`). |
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-19) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. | | Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-19) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. |
| Sprint 7 | Contextual Truth Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Concelier WebService Guild | FEEDWEB-EVENTS-07-001 | Advisory event replay API – expose `/concelier/advisories/{key}/replay` with `asOf` filter, hex hashes, and conflict data. | | Sprint 7 | Contextual Truth Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Concelier WebService Guild | FEEDWEB-EVENTS-07-001 | Advisory event replay API – expose `/concelier/advisories/{key}/replay` with `asOf` filter, hex hashes, and conflict data. |
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-20) | BE-Merge | FEEDMERGE-ENGINE-07-001 | Conflict sets & explainers – persist conflict materialization and replay hashes for merge decisions. | | Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-20) | BE-Merge | FEEDMERGE-ENGINE-07-001 | Conflict sets & explainers – persist conflict materialization and replay hashes for merge decisions. |
| Sprint 8 | Mongo strengthening | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Normalization & Storage Backbone | FEEDSTORAGE-MONGO-08-001 | Causal-consistent Concelier storage sessions<br>Scoped session facilitator registered, repositories accept optional session handles, and replica-set failover tests verify read-your-write + monotonic reads. | | Sprint 8 | Mongo strengthening | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Normalization & Storage Backbone | FEEDSTORAGE-MONGO-08-001 | Causal-consistent Concelier storage sessions<br>Scoped session facilitator registered, repositories accept optional session handles, and replica-set failover tests verify read-your-write + monotonic reads. |
| Sprint 8 | Mongo strengthening | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-19) | Authority Core & Storage Guild | AUTHSTORAGE-MONGO-08-001 | Harden Authority Mongo usage<br>Scoped Mongo sessions with majority read/write concerns wired through stores and GraphQL/HTTP pipelines; replica-set election regression validated. | | Sprint 8 | Mongo strengthening | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-19) | Authority Core & Storage Guild | AUTHSTORAGE-MONGO-08-001 | Harden Authority Mongo usage<br>Scoped Mongo sessions with majority read/write concerns wired through stores and GraphQL/HTTP pipelines; replica-set election regression validated. |
| Sprint 8 | Mongo strengthening | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-MONGO-08-001 | Causal consistency for Excititor repositories<br>Session-scoped repositories shipped with new Mongo records, orchestrators/workers now share scoped sessions, and replica-set failover coverage added via `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`. | | Sprint 8 | Mongo strengthening | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-MONGO-08-001 | Causal consistency for Excititor repositories<br>Session-scoped repositories shipped with new Mongo records, orchestrators/workers now share scoped sessions, and replica-set failover coverage added via `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`. |
| Sprint 8 | Platform Maintenance | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-03-001 | Statement backfill tooling – shipped admin backfill endpoint, CLI hook (`stellaops excititor backfill-statements`), integration tests, and operator runbook (`docs/dev/EXCITITOR_STATEMENT_BACKFILL.md`). | | Sprint 8 | Platform Maintenance | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-03-001 | Statement backfill tooling – shipped admin backfill endpoint, CLI hook (`stellaops excititor backfill-statements`), integration tests, and operator runbook (`docs/dev/EXCITITOR_STATEMENT_BACKFILL.md`). |
| Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json/TASKS.md | DONE (2025-10-19) | Concelier Export Guild | CONCELIER-EXPORT-08-201 | Mirror bundle + domain manifest – produce signed JSON aggregates for `*.stella-ops.org` mirrors. | | Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json/TASKS.md | DONE (2025-10-19) | Concelier Export Guild | CONCELIER-EXPORT-08-201 | Mirror bundle + domain manifest – produce signed JSON aggregates for `*.stella-ops.org` mirrors. |
| Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md | DONE (2025-10-19) | Concelier Export Guild | CONCELIER-EXPORT-08-202 | Mirror-ready Trivy DB bundles – mirror options emit per-domain manifests/metadata/db archives with deterministic digests for downstream sync. | | Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md | DONE (2025-10-19) | Concelier Export Guild | CONCELIER-EXPORT-08-202 | Mirror-ready Trivy DB bundles – mirror options emit per-domain manifests/metadata/db archives with deterministic digests for downstream sync. |
| Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-20) | Concelier WebService Guild | CONCELIER-WEB-08-201 | Mirror distribution endpoints – expose domain-scoped index/download APIs with auth/quota. | | Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-20) | Concelier WebService Guild | CONCELIER-WEB-08-201 | Mirror distribution endpoints – expose domain-scoped index/download APIs with auth/quota. |
| Sprint 8 | Mirror Distribution | ops/devops/TASKS.md | DONE (2025-10-19) | DevOps Guild | DEVOPS-MIRROR-08-001 | Managed mirror deployments for `*.stella-ops.org` – Helm/Compose overlays, CDN, runbooks. | | Sprint 8 | Mirror Distribution | ops/devops/TASKS.md | DONE (2025-10-19) | DevOps Guild | DEVOPS-MIRROR-08-001 | Managed mirror deployments for `*.stella-ops.org` – Helm/Compose overlays, CDN, runbooks. |
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-003 | Refactor Authority identity-provider registry to resolve scoped plugin services on-demand.<br>Introduce factory pattern aligned with scoped lifetimes decided in coordination workshop. | | Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-003 | Refactor Authority identity-provider registry to resolve scoped plugin services on-demand.<br>Introduce factory pattern aligned with scoped lifetimes decided in coordination workshop. |
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-004 | Update Authority plugin loader to activate registrars with DI support and scoped service awareness.<br>Add two-phase initialization allowing scoped dependencies post-container build. | | Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-004 | Update Authority plugin loader to activate registrars with DI support and scoped service awareness.<br>Add two-phase initialization allowing scoped dependencies post-container build. |
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-005 | Provide scoped-safe bootstrap execution for Authority plugins.<br>Implement scope-per-run pattern for hosted bootstrap tasks and document migration guidance. | | Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-005 | Provide scoped-safe bootstrap execution for Authority plugins.<br>Implement scope-per-run pattern for hosted bootstrap tasks and document migration guidance. |
| Sprint 10 | DevOps Security | ops/devops/TASKS.md | DONE (2025-10-20) | DevOps Guild | DEVOPS-SEC-10-301 | Address NU1902/NU1903 advisories for `MongoDB.Driver` 2.12.0 and `SharpCompress` 0.23.0; Wave 0A prerequisites confirmed complete before remediation work. | | Sprint 10 | DevOps Security | ops/devops/TASKS.md | DONE (2025-10-20) | DevOps Guild | DEVOPS-SEC-10-301 | Address NU1902/NU1903 advisories for `MongoDB.Driver` 2.12.0 and `SharpCompress` 0.23.0; Wave 0A prerequisites confirmed complete before remediation work. |
| Sprint 11 | Signing Chain Bring-up | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-20) | Authority Core & Security Guild | AUTH-DPOP-11-001 | Implement DPoP proof validation + nonce handling for high-value audiences per architecture. | | Sprint 11 | Signing Chain Bring-up | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-20) | Authority Core & Security Guild | AUTH-DPOP-11-001 | Implement DPoP proof validation + nonce handling for high-value audiences per architecture. |
| Sprint 15 | Notify Foundations | src/Notify/StellaOps.Notify.WebService/TASKS.md | DONE (2025-10-19) | Notify WebService Guild | NOTIFY-WEB-15-103 | Delivery history & test-send endpoints. | | Sprint 15 | Notify Foundations | src/Notify/StellaOps.Notify.WebService/TASKS.md | DONE (2025-10-19) | Notify WebService Guild | NOTIFY-WEB-15-103 | Delivery history & test-send endpoints. |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Slack/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-SLACK-15-502 | Slack health/test-send support. | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Slack/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-SLACK-15-502 | Slack health/test-send support. |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-602 | Teams health/test-send support. | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-602 | Teams health/test-send support. |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-604 | Teams health endpoint metadata alignment. | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-604 | Teams health endpoint metadata alignment. |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Slack/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-SLACK-15-503 | Package Slack connector as restart-time plug-in (manifest + host registration). | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Slack/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-SLACK-15-503 | Package Slack connector as restart-time plug-in (manifest + host registration). |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-603 | Package Teams connector as restart-time plug-in (manifest + host registration). | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-603 | Package Teams connector as restart-time plug-in (manifest + host registration). |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Email/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-EMAIL-15-703 | Package Email connector as restart-time plug-in (manifest + host registration). | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Email/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-EMAIL-15-703 | Package Email connector as restart-time plug-in (manifest + host registration). |
| Sprint 15 | Notify Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-20) | Scanner WebService Guild | SCANNER-EVENTS-15-201 | Emit `scanner.report.ready` + `scanner.scan.completed` events. | | Sprint 15 | Notify Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-20) | Scanner WebService Guild | SCANNER-EVENTS-15-201 | Emit `scanner.report.ready` + `scanner.scan.completed` events. |
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Webhook/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-WEBHOOK-15-803 | Package Webhook connector as restart-time plug-in (manifest + host registration). | | Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Webhook/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-WEBHOOK-15-803 | Package Webhook connector as restart-time plug-in (manifest + host registration). |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-103 | Versioning/migration helpers for schedules/runs. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-103 | Versioning/migration helpers for schedules/runs. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Queue abstraction + Redis Streams adapter. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Queue abstraction + Redis Streams adapter. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-402 | NATS JetStream adapter with health probes. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-402 | NATS JetStream adapter with health probes. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-300 | **STUB** ImpactIndex ingest/query using fixtures (to be removed by SP16 completion). | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-300 | **STUB** ImpactIndex ingest/query using fixtures (to be removed by SP16 completion). |

168
docs/implplan/SPRINTS_PRIOR_20251027.md
View File

@@ -1,84 +1,84 @@
This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not). This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not).
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description | | Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
| --- | --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- | --- |
| Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-NUGET-13-002 | Ensure all solutions/projects prioritize `local-nuget` before public feeds and add restore-order validation. | | Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-NUGET-13-002 | Ensure all solutions/projects prioritize `local-nuget` before public feeds and add restore-order validation. |
| Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-003 | Upgrade `Microsoft.*` dependencies pinned to 8.* to their latest .NET 10 (or 9.x) releases and refresh guidance. | | Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-003 | Upgrade `Microsoft.*` dependencies pinned to 8.* to their latest .NET 10 (or 9.x) releases and refresh guidance. |
| Sprint 14 | Release & Offline Ops | ops/deployment/TASKS.md | DONE (2025-10-26) | Deployment Guild | DEVOPS-OPS-14-003 | Deployment/update/rollback automation and channel management documentation. | | Sprint 14 | Release & Offline Ops | ops/deployment/TASKS.md | DONE (2025-10-26) | Deployment Guild | DEVOPS-OPS-14-003 | Deployment/update/rollback automation and channel management documentation. |
| Sprint 14 | Release & Offline Ops | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-14-001 | Deterministic build/release pipeline with SBOM/provenance, signing, and manifest generation. | | Sprint 14 | Release & Offline Ops | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-14-001 | Deterministic build/release pipeline with SBOM/provenance, signing, and manifest generation. |
| Sprint 14 | Release & Offline Ops | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, Scanner Guild | DEVOPS-REL-14-004 | Extend release/offline smoke jobs to cover Python analyzer plug-ins (warm/cold, determinism, signing). | | Sprint 14 | Release & Offline Ops | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, Scanner Guild | DEVOPS-REL-14-004 | Extend release/offline smoke jobs to cover Python analyzer plug-ins (warm/cold, determinism, signing). |
| Sprint 14 | Release & Offline Ops | ops/licensing/TASKS.md | DONE (2025-10-26) | Licensing Guild | DEVOPS-LIC-14-004 | Registry token service tied to Authority, plan gating, revocation handling, monitoring. | | Sprint 14 | Release & Offline Ops | ops/licensing/TASKS.md | DONE (2025-10-26) | Licensing Guild | DEVOPS-LIC-14-004 | Registry token service tied to Authority, plan gating, revocation handling, monitoring. |
| Sprint 14 | Release & Offline Ops | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild | DEVOPS-OFFLINE-14-002 | Offline kit packaging workflow with integrity verification and documentation. | | Sprint 14 | Release & Offline Ops | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild | DEVOPS-OFFLINE-14-002 | Offline kit packaging workflow with integrity verification and documentation. |
| Sprint 15 | Benchmarks | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild, Notify Team | BENCH-NOTIFY-15-001 | Notify dispatch throughput bench with results CSV. | | Sprint 15 | Benchmarks | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild, Notify Team | BENCH-NOTIFY-15-001 | Notify dispatch throughput bench with results CSV. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Define Scheduler DTOs & validation. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Define Scheduler DTOs & validation. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-102 | Publish schema docs/sample payloads. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-102 | Publish schema docs/sample payloads. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Mongo schemas/indexes for Scheduler state. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Mongo schemas/indexes for Scheduler state. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-202 | Repositories with tenant scoping, TTL, causal consistency. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-202 | Repositories with tenant scoping, TTL, causal consistency. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-203 | Audit/run stats materialization for UI. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-203 | Audit/run stats materialization for UI. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-302 | Query APIs for ResolveByPurls/ResolveByVulns/ResolveAll. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-302 | Query APIs for ResolveByPurls/ResolveByVulns/ResolveAll. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Ingest BOM-Index into roaring bitmap store. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Ingest BOM-Index into roaring bitmap store. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-102 | Schedules CRUD (cron validation, pause/resume, audit). | | Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-102 | Schedules CRUD (cron validation, pause/resume, audit). |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-103 | Runs API (list/detail/cancel) + impact previews. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-103 | Runs API (list/detail/cancel) + impact previews. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-WEB-16-104 | Feedser/Vexer webhook handlers with security enforcement. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-WEB-16-104 | Feedser/Vexer webhook handlers with security enforcement. |
| Sprint 17 | Symbol Intelligence & Forensics | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-RUNTIME-17-004 | Document build-id workflows for SBOMs, runtime events, and debug-store usage. | | Sprint 17 | Symbol Intelligence & Forensics | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-RUNTIME-17-004 | Document build-id workflows for SBOMs, runtime events, and debug-store usage. |
| Sprint 17 | Symbol Intelligence & Forensics | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-17-002 | Ship stripped debug artifacts organised by build-id within release/offline kits. | | Sprint 17 | Symbol Intelligence & Forensics | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-17-002 | Ship stripped debug artifacts organised by build-id within release/offline kits. |
| Sprint 17 | Symbol Intelligence & Forensics | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild, DevOps Guild | DEVOPS-OFFLINE-17-003 | Mirror release debug-store artefacts into Offline Kit packaging and document validation. | | Sprint 17 | Symbol Intelligence & Forensics | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild, DevOps Guild | DEVOPS-OFFLINE-17-003 | Mirror release debug-store artefacts into Offline Kit packaging and document validation. |
| Sprint 17 | Symbol Intelligence & Forensics | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-26) | Emit Guild | SCANNER-EMIT-17-701 | Record GNU build-id for ELF components and surface it in SBOM/diff outputs. | | Sprint 17 | Symbol Intelligence & Forensics | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-26) | Emit Guild | SCANNER-EMIT-17-701 | Record GNU build-id for ELF components and surface it in SBOM/diff outputs. |
| Sprint 18 | Launch Readiness | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-LAUNCH-18-001 | Production launch cutover rehearsal and runbook publication. | | Sprint 18 | Launch Readiness | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-LAUNCH-18-001 | Production launch cutover rehearsal and runbook publication. |
| Sprint 18 | Launch Readiness | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-005 | Rebuild Offline Kit with Python analyzer artefacts and refreshed manifest/signature pair. | | Sprint 18 | Launch Readiness | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-005 | Rebuild Offline Kit with Python analyzer artefacts and refreshed manifest/signature pair. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-AOC-19-001 | Publish aggregation-only contract reference documentation. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-AOC-19-001 | Publish aggregation-only contract reference documentation. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-002 | Update architecture overview with AOC boundary diagrams. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-002 | Update architecture overview with AOC boundary diagrams. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-AOC-19-003 | Refresh policy engine doc with raw ingestion constraints. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-AOC-19-003 | Refresh policy engine doc with raw ingestion constraints. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-AOC-19-004 | Document console AOC dashboard and drill-down flow. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-AOC-19-004 | Document console AOC dashboard and drill-down flow. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-AOC-19-005 | Document CLI AOC commands and exit codes. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-AOC-19-005 | Document CLI AOC commands and exit codes. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-AOC-19-006 | Document new AOC metrics, traces, and logs. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-AOC-19-006 | Document new AOC metrics, traces, and logs. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-AOC-19-007 | Document new Authority scopes and tenancy enforcement. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-AOC-19-007 | Document new Authority scopes and tenancy enforcement. |
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, DevOps Guild | DOCS-AOC-19-008 | Update deployment guide with validator enablement and verify user guidance. | | Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, DevOps Guild | DOCS-AOC-19-008 | Update deployment guide with validator enablement and verify user guidance. |
| Sprint 19 | Aggregation-Only Contract Enforcement | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce new ingestion/auth scopes across Authority. | | Sprint 19 | Aggregation-Only Contract Enforcement | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce new ingestion/auth scopes across Authority. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-POLICY-20-001 | Publish `/docs/policy/overview.md` with compliance checklist. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-POLICY-20-001 | Publish `/docs/policy/overview.md` with compliance checklist. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-POLICY-20-002 | Document DSL grammar + examples in `/docs/policy/dsl.md`. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-POLICY-20-002 | Document DSL grammar + examples in `/docs/policy/dsl.md`. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-POLICY-20-003 | Write `/docs/policy/lifecycle.md` covering workflow + roles. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-POLICY-20-003 | Write `/docs/policy/lifecycle.md` covering workflow + roles. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Scheduler Guild | DOCS-POLICY-20-004 | Document policy run modes + cursors in `/docs/policy/runs.md`. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Scheduler Guild | DOCS-POLICY-20-004 | Document policy run modes + cursors in `/docs/policy/runs.md`. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Platform Guild | DOCS-POLICY-20-005 | Produce `/docs/api/policy.md` with endpoint schemas + errors. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Platform Guild | DOCS-POLICY-20-005 | Produce `/docs/api/policy.md` with endpoint schemas + errors. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-POLICY-20-006 | Author `/docs/modules/cli/guides/policy.md` with commands, exit codes, JSON output. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-POLICY-20-006 | Author `/docs/modules/cli/guides/policy.md` with commands, exit codes, JSON output. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-POLICY-20-007 | Create `/docs/ui/policy-editor.md` covering editor, simulation, approvals. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-POLICY-20-007 | Create `/docs/ui/policy-editor.md` covering editor, simulation, approvals. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-POLICY-20-008 | Publish `/docs/modules/policy/architecture.md` with sequence diagrams. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-POLICY-20-008 | Publish `/docs/modules/policy/architecture.md` with sequence diagrams. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-POLICY-20-009 | Document metrics/traces/logs in `/docs/observability/policy.md`. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-POLICY-20-009 | Document metrics/traces/logs in `/docs/observability/policy.md`. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Security Guild | DOCS-POLICY-20-010 | Publish `/docs/security/policy-governance.md` for scopes + approvals. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Security Guild | DOCS-POLICY-20-010 | Publish `/docs/security/policy-governance.md` for scopes + approvals. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-POLICY-20-011 | Add example policies under `/docs/examples/policies/` with commentary. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-POLICY-20-011 | Add example policies under `/docs/examples/policies/` with commentary. |
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Support Guild | DOCS-POLICY-20-012 | Draft `/docs/faq/policy-faq.md` covering conflicts, determinism, pitfalls. | | Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Support Guild | DOCS-POLICY-20-012 | Draft `/docs/faq/policy-faq.md` covering conflicts, determinism, pitfalls. |
| Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-POLICY-20-001 | Add DSL lint + compile checks to CI pipelines. | | Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-POLICY-20-001 | Add DSL lint + compile checks to CI pipelines. |
| Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, QA Guild | DEVOPS-POLICY-20-003 | Add determinism CI job diffing repeated policy runs. | | Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, QA Guild | DEVOPS-POLICY-20-003 | Add determinism CI job diffing repeated policy runs. |
| Sprint 20 | Policy Engine v2 | samples/TASKS.md | DONE (2025-10-26) | Samples Guild, Policy Guild | SAMPLES-POLICY-20-001 | Commit baseline/serverless/internal-only policy samples + fixtures. | | Sprint 20 | Policy Engine v2 | samples/TASKS.md | DONE (2025-10-26) | Samples Guild, Policy Guild | SAMPLES-POLICY-20-001 | Commit baseline/serverless/internal-only policy samples + fixtures. |
| Sprint 20 | Policy Engine v2 | samples/TASKS.md | DONE (2025-10-26) | Samples Guild, UI Guild | SAMPLES-POLICY-20-002 | Produce simulation diff fixtures for UI/CLI tests. | | Sprint 20 | Policy Engine v2 | samples/TASKS.md | DONE (2025-10-26) | Samples Guild, UI Guild | SAMPLES-POLICY-20-002 | Produce simulation diff fixtures for UI/CLI tests. |
| Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Add new policy scopes (`policy:*`, `findings:read`, `effective:write`). | | Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Add new policy scopes (`policy:*`, `findings:read`, `effective:write`). |
| Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-002 | Enforce Policy Engine service identity and scope checks at gateway. | | Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-002 | Enforce Policy Engine service identity and scope checks at gateway. |
| Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-POLICY-20-003 | Update Authority docs/config samples for policy scopes + workflows. | | Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-POLICY-20-003 | Update Authority docs/config samples for policy scopes + workflows. |
| Sprint 20 | Policy Engine v2 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild, Policy Guild | BENCH-POLICY-20-001 | Create policy evaluation benchmark suite + baseline metrics. | | Sprint 20 | Policy Engine v2 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild, Policy Guild | BENCH-POLICY-20-001 | Create policy evaluation benchmark suite + baseline metrics. |
| Sprint 20 | Policy Engine v2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-26) | Policy Guild, Platform Guild | POLICY-ENGINE-20-000 | Spin up new Policy Engine service host with DI bootstrap and Authority wiring. | | Sprint 20 | Policy Engine v2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-26) | Policy Guild, Platform Guild | POLICY-ENGINE-20-000 | Spin up new Policy Engine service host with DI bootstrap and Authority wiring. |
| Sprint 20 | Policy Engine v2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-26) | Policy Guild | POLICY-ENGINE-20-001 | Deliver `stella-dsl@1` parser + IR compiler with diagnostics and checksums. | | Sprint 20 | Policy Engine v2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-26) | Policy Guild | POLICY-ENGINE-20-001 | Deliver `stella-dsl@1` parser + IR compiler with diagnostics and checksums. |
| Sprint 20 | Policy Engine v2 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-20-001 | Define policy run/diff DTOs + validation helpers. | | Sprint 20 | Policy Engine v2 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-20-001 | Define policy run/diff DTOs + validation helpers. |
| Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core Guild | AUTH-GRAPH-21-001 | Introduce graph scopes (`graph:*`) with configuration binding and defaults. | | Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core Guild | AUTH-GRAPH-21-001 | Introduce graph scopes (`graph:*`) with configuration binding and defaults. |
| Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core Guild | AUTH-GRAPH-21-002 | Enforce graph scopes/identities at gateway with tenant propagation. | | Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core Guild | AUTH-GRAPH-21-002 | Enforce graph scopes/identities at gateway with tenant propagation. |
| Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-GRAPH-21-003 | Update security docs/config samples for graph access and least privilege. | | Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-GRAPH-21-003 | Update security docs/config samples for graph access and least privilege. |
| Sprint 21 | Graph Explorer v1 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums; document in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`. | | Sprint 21 | Graph Explorer v1 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums; document in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`. |
| Sprint 21 | Graph Explorer v1 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-002 | Publish schema docs/sample payloads for graph job lifecycle. | | Sprint 21 | Graph Explorer v1 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-002 | Publish schema docs/sample payloads for graph job lifecycle. |
| Sprint 22 | Link-Not-Merge v1 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild | BENCH-LNM-22-001 | Benchmark advisory observation ingest/correlation throughput. | | Sprint 22 | Link-Not-Merge v1 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild | BENCH-LNM-22-001 | Benchmark advisory observation ingest/correlation throughput. |
| Sprint 22 | Link-Not-Merge v1 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild | BENCH-LNM-22-002 | Benchmark VEX ingest/correlation latency and event emission. | | Sprint 22 | Link-Not-Merge v1 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild | BENCH-LNM-22-002 | Benchmark VEX ingest/correlation latency and event emission. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-001 | Publish `/docs/ui/console-overview.md` (IA, tenant model, filters, AOC alignment). | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-001 | Publish `/docs/ui/console-overview.md` (IA, tenant model, filters, AOC alignment). |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-002 | Author `/docs/ui/navigation.md` with route map, filters, keyboard shortcuts, deep links. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-002 | Author `/docs/ui/navigation.md` with route map, filters, keyboard shortcuts, deep links. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-003 | Document `/docs/ui/sbom-explorer.md` covering catalog, graph, overlays, exports. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-003 | Document `/docs/ui/sbom-explorer.md` covering catalog, graph, overlays, exports. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-004 | Produce `/docs/ui/advisories-and-vex.md` detailing aggregation-not-merge UX. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-004 | Produce `/docs/ui/advisories-and-vex.md` detailing aggregation-not-merge UX. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-005 | Write `/docs/ui/findings.md` with filters, explain, exports, CLI parity notes. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-005 | Write `/docs/ui/findings.md` with filters, explain, exports, CLI parity notes. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-006 | Publish `/docs/ui/policies.md` (editor, simulation, approvals, RBAC). | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-006 | Publish `/docs/ui/policies.md` (editor, simulation, approvals, RBAC). |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-007 | Document `/docs/ui/runs.md` with SSE monitoring, diff, retries, evidence downloads. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-007 | Document `/docs/ui/runs.md` with SSE monitoring, diff, retries, evidence downloads. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-008 | Draft `/docs/ui/admin.md` covering tenants, roles, tokens, integrations, fresh-auth. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-008 | Draft `/docs/ui/admin.md` covering tenants, roles, tokens, integrations, fresh-auth. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-27) | Docs Guild | DOCS-CONSOLE-23-009 | Publish `/docs/ui/downloads.md` aligning manifest with commands and offline flow. | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-27) | Docs Guild | DOCS-CONSOLE-23-009 | Publish `/docs/ui/downloads.md` aligning manifest with commands and offline flow. |
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DOCS-CONSOLE-23-010 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, env vars, health checks). | | Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DOCS-CONSOLE-23-010 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, env vars, health checks). |
| Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-001 | Provide graph build/overlay job APIs; see `docs/SCHED-WEB-21-001-GRAPH-APIS.md`. | | Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-001 | Provide graph build/overlay job APIs; see `docs/SCHED-WEB-21-001-GRAPH-APIS.md`. |
| Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-002 | Provide overlay lag metrics endpoint/webhook; see `docs/SCHED-WEB-21-001-GRAPH-APIS.md`. | | Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-002 | Provide overlay lag metrics endpoint/webhook; see `docs/SCHED-WEB-21-001-GRAPH-APIS.md`. |
| Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild, Authority Core Guild | SCHED-WEB-21-003 | Replace header auth with Authority scopes using `StellaOpsScopes`; dev fallback only when `Scheduler:Authority:Enabled=false`. | | Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild, Authority Core Guild | SCHED-WEB-21-003 | Replace header auth with Authority scopes using `StellaOpsScopes`; dev fallback only when `Scheduler:Authority:Enabled=false`. |
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-001 | Deploy default OpenTelemetry collector manifests with secure OTLP pipeline. | | Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-001 | Deploy default OpenTelemetry collector manifests with secure OTLP pipeline. |
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-003 | Package telemetry stack configs for offline/air-gapped installs with signatures. | | Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-003 | Package telemetry stack configs for offline/air-gapped installs with signatures. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-WEB-16-101 | Minimal API host with Authority enforcement. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-WEB-16-101 | Minimal API host with Authority enforcement. |
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | ImpactIndex targeting and shard planning. | | Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | ImpactIndex targeting and shard planning. |

360
docs/ingestion/aggregation-only-contract.md
View File

@@ -1,181 +1,181 @@
# Aggregation-Only Contract Reference # Aggregation-Only Contract Reference
> The Aggregation-Only Contract (AOC) is the governing rule set that keeps StellaOps ingestion services deterministic, policy-neutral, and auditable. It applies to Concelier, Excititor, and any future collectors that write raw advisory or VEX documents. > The Aggregation-Only Contract (AOC) is the governing rule set that keeps StellaOps ingestion services deterministic, policy-neutral, and auditable. It applies to Concelier, Excititor, and any future collectors that write raw advisory or VEX documents.
## 1. Purpose and Scope ## 1. Purpose and Scope
- Defines the canonical behaviour for `advisory_raw` and `vex_raw` collections and the linkset hints they may emit. - Defines the canonical behaviour for `advisory_raw` and `vex_raw` collections and the linkset hints they may emit.
- Applies to every ingestion runtime (`StellaOps.Concelier.*`, `StellaOps.Excititor.*`), the Authority scopes that guard them, and the DevOps/QA surfaces that verify compliance. - Applies to every ingestion runtime (`StellaOps.Concelier.*`, `StellaOps.Excititor.*`), the Authority scopes that guard them, and the DevOps/QA surfaces that verify compliance.
- Complements the high-level architecture in [Concelier](../modules/concelier/architecture.md) and Authority enforcement documented in [Authority Architecture](../modules/authority/architecture.md). - Complements the high-level architecture in [Concelier](../modules/concelier/architecture.md) and Authority enforcement documented in [Authority Architecture](../modules/authority/architecture.md).
- Paired guidance: see the guard-rail checkpoints in [AOC Guardrails](../aoc/aoc-guardrails.md), the implementation reference in [AOC Guard Library](../aoc/guard-library.md), and CLI usage that will land in `/docs/modules/cli/guides/` as part of Sprint 19 follow-up. - Paired guidance: see the guard-rail checkpoints in [AOC Guardrails](../aoc/aoc-guardrails.md), the implementation reference in [AOC Guard Library](../aoc/guard-library.md), and CLI usage that will land in `/docs/modules/cli/guides/` as part of Sprint 19 follow-up.
## 2. Philosophy and Goals ## 2. Philosophy and Goals
- Preserve upstream truth: ingestion only captures immutable raw facts plus provenance, never derived severity or policy decisions. - Preserve upstream truth: ingestion only captures immutable raw facts plus provenance, never derived severity or policy decisions.
- Defer interpretation: Policy Engine and downstream overlays remain the sole writers of materialised findings, severity, consensus, or risk scores. - Defer interpretation: Policy Engine and downstream overlays remain the sole writers of materialised findings, severity, consensus, or risk scores.
- Make every write explainable: provenance, signatures, and content hashes are required so operators can prove where each fact originated. - Make every write explainable: provenance, signatures, and content hashes are required so operators can prove where each fact originated.
- Keep outputs reproducible: identical inputs must yield identical documents, hashes, and linksets across replays and air-gapped installs. - Keep outputs reproducible: identical inputs must yield identical documents, hashes, and linksets across replays and air-gapped installs.
## 3. Contract Invariants ## 3. Contract Invariants
| # | Invariant | What it forbids or requires | Enforcement surfaces | | # | Invariant | What it forbids or requires | Enforcement surfaces |
|---|-----------|-----------------------------|----------------------| |---|-----------|-----------------------------|----------------------|
| 1 | No derived severity at ingest | Reject top-level keys such as `severity`, `cvss`, `effective_status`, `consensus_provider`, `risk_score`. Raw upstream CVSS remains inside `content.raw`. | Mongo schema validator, `AOCWriteGuard`, Roslyn analyzer, `stella aoc verify`. | | 1 | No derived severity at ingest | Reject top-level keys such as `severity`, `cvss`, `effective_status`, `consensus_provider`, `risk_score`. Raw upstream CVSS remains inside `content.raw`. | Mongo schema validator, `AOCWriteGuard`, Roslyn analyzer, `stella aoc verify`. |
| 2 | No merges or opinionated dedupe | Each upstream document persists on its own; ingestion never collapses multiple vendors into one document. | Repository interceptors, unit/fixture suites. | | 2 | No merges or opinionated dedupe | Each upstream document persists on its own; ingestion never collapses multiple vendors into one document. | Repository interceptors, unit/fixture suites. |
| 3 | Provenance is mandatory | `source.*`, `upstream.*`, and `signature` metadata must be present; missing provenance triggers `ERR_AOC_004`. | Schema validator, guard, CLI verifier. | | 3 | Provenance is mandatory | `source.*`, `upstream.*`, and `signature` metadata must be present; missing provenance triggers `ERR_AOC_004`. | Schema validator, guard, CLI verifier. |
| 4 | Idempotent upserts | Writes keyed by `(vendor, upstream_id, content_hash)` either no-op or insert a new revision with `supersedes`. Duplicate hashes map to the same document. | Repository guard, storage unique index, CI smoke tests. | | 4 | Idempotent upserts | Writes keyed by `(vendor, upstream_id, content_hash)` either no-op or insert a new revision with `supersedes`. Duplicate hashes map to the same document. | Repository guard, storage unique index, CI smoke tests. |
| 5 | Append-only revisions | Updates create a new document with `supersedes` pointer; no in-place mutation of content. | Mongo schema (`supersedes` format), guard, data migration scripts. | | 5 | Append-only revisions | Updates create a new document with `supersedes` pointer; no in-place mutation of content. | Mongo schema (`supersedes` format), guard, data migration scripts. |
| 6 | Linkset only | Ingestion may compute link hints (`purls`, `cpes`, IDs) to accelerate joins, but must not transform or infer severity or policy. | Linkset builders reviewed via fixtures and analyzers. | | 6 | Linkset only | Ingestion may compute link hints (`purls`, `cpes`, IDs) to accelerate joins, but must not transform or infer severity or policy. Observations now persist both canonical linksets (for indexed queries) and raw linksets (preserving upstream order/duplicates) so downstream policy can decide how to normalise. | Linkset builders reviewed via fixtures/analyzers; raw-vs-canonical parity covered by observation fixtures. |
| 7 | Policy-only effective findings | Only Policy Engine identities can write `effective_finding_*`; ingestion callers receive `ERR_AOC_006` if they attempt it. | Authority scopes, Policy Engine guard. | | 7 | Policy-only effective findings | Only Policy Engine identities can write `effective_finding_*`; ingestion callers receive `ERR_AOC_006` if they attempt it. | Authority scopes, Policy Engine guard. |
| 8 | Schema safety | Unknown top-level keys reject with `ERR_AOC_007`; timestamps use ISO 8601 UTC strings; tenant is required. | Mongo validator, JSON schema tests. | | 8 | Schema safety | Unknown top-level keys reject with `ERR_AOC_007`; timestamps use ISO 8601 UTC strings; tenant is required. | Mongo validator, JSON schema tests. |
| 9 | Clock discipline | Collectors stamp `fetched_at` and `received_at` monotonically per batch to support reproducibility windows. | Collector contracts, QA fixtures. | | 9 | Clock discipline | Collectors stamp `fetched_at` and `received_at` monotonically per batch to support reproducibility windows. | Collector contracts, QA fixtures. |
## 4. Raw Schemas ## 4. Raw Schemas
### 4.1 `advisory_raw` ### 4.1 `advisory_raw`
| Field | Type | Notes | | Field | Type | Notes |
|-------|------|-------| |-------|------|-------|
| `_id` | string | `advisory_raw:{source}:{upstream_id}:{revision}`; deterministic and tenant-scoped. | | `_id` | string | `advisory_raw:{source}:{upstream_id}:{revision}`; deterministic and tenant-scoped. |
| `tenant` | string | Required; injected by Authority middleware and asserted by schema validator. | | `tenant` | string | Required; injected by Authority middleware and asserted by schema validator. |
| `source.vendor` | string | Provider identifier (e.g., `redhat`, `osv`, `ghsa`). | | `source.vendor` | string | Provider identifier (e.g., `redhat`, `osv`, `ghsa`). |
| `source.stream` | string | Connector stream name (`csaf`, `osv`, etc.). | | `source.stream` | string | Connector stream name (`csaf`, `osv`, etc.). |
| `source.api` | string | Absolute URI of upstream document; stored for traceability. | | `source.api` | string | Absolute URI of upstream document; stored for traceability. |
| `source.collector_version` | string | Semantic version of the collector. | | `source.collector_version` | string | Semantic version of the collector. |
| `upstream.upstream_id` | string | Vendor- or ecosystem-provided identifier (CVE, GHSA, vendor ID). | | `upstream.upstream_id` | string | Vendor- or ecosystem-provided identifier (CVE, GHSA, vendor ID). |
| `upstream.document_version` | string | Upstream issued timestamp or revision string. | | `upstream.document_version` | string | Upstream issued timestamp or revision string. |
| `upstream.fetched_at` / `received_at` | string | ISO 8601 UTC timestamps recorded by the collector. | | `upstream.fetched_at` / `received_at` | string | ISO 8601 UTC timestamps recorded by the collector. |
| `upstream.content_hash` | string | `sha256:` digest of the raw payload used for idempotency. | | `upstream.content_hash` | string | `sha256:` digest of the raw payload used for idempotency. |
| `upstream.signature` | object | Required structure storing `present`, `format`, `key_id`, `sig`; even unsigned payloads set `present: false`. | | `upstream.signature` | object | Required structure storing `present`, `format`, `key_id`, `sig`; even unsigned payloads set `present: false`. |
| `content.format` | string | Source format (`CSAF`, `OSV`, etc.). | | `content.format` | string | Source format (`CSAF`, `OSV`, etc.). |
| `content.spec_version` | string | Upstream spec version when known. | | `content.spec_version` | string | Upstream spec version when known. |
| `content.raw` | object | Full upstream payload, untouched except for transport normalisation. | | `content.raw` | object | Full upstream payload, untouched except for transport normalisation. |
| `identifiers` | object | Upstream identifiers (`cve`, `ghsa`, `aliases`, etc.) captured as provided (trimmed, order preserved, duplicates allowed). | | `identifiers` | object | Upstream identifiers (`cve`, `ghsa`, `aliases`, etc.) captured as provided (trimmed, order preserved, duplicates allowed). |
| `linkset` | object | Join hints (see section 4.3). | | `linkset` | object | Join hints (see section 4.3). |
| `supersedes` | string or null | Points to previous revision of same upstream doc when content hash changes. | | `supersedes` | string or null | Points to previous revision of same upstream doc when content hash changes. |
### 4.2 `vex_raw` ### 4.2 `vex_raw`
| Field | Type | Notes | | Field | Type | Notes |
|-------|------|-------| |-------|------|-------|
| `_id` | string | `vex_raw:{source}:{upstream_id}:{revision}`. | | `_id` | string | `vex_raw:{source}:{upstream_id}:{revision}`. |
| `tenant` | string | Required; matches advisory collection requirements. | | `tenant` | string | Required; matches advisory collection requirements. |
| `source.*` | object | Same shape and requirements as `advisory_raw`. | | `source.*` | object | Same shape and requirements as `advisory_raw`. |
| `upstream.*` | object | Includes `document_version`, timestamps, `content_hash`, and `signature`. | | `upstream.*` | object | Includes `document_version`, timestamps, `content_hash`, and `signature`. |
| `content.format` | string | Typically `CycloneDX-VEX` or `CSAF-VEX`. | | `content.format` | string | Typically `CycloneDX-VEX` or `CSAF-VEX`. |
| `content.raw` | object | Entire upstream VEX payload. | | `content.raw` | object | Entire upstream VEX payload. |
| `identifiers.statements` | array | Normalised statement summaries (IDs, PURLs, status, justification) to accelerate policy joins. | | `identifiers.statements` | array | Normalised statement summaries (IDs, PURLs, status, justification) to accelerate policy joins. |
| `linkset` | object | CVEs, GHSA IDs, and PURLs referenced in the document. | | `linkset` | object | CVEs, GHSA IDs, and PURLs referenced in the document. |
| `supersedes` | string or null | Same convention as advisory documents. | | `supersedes` | string or null | Same convention as advisory documents. |
### 4.3 Linkset Fields ### 4.3 Linkset Fields
- `purls`: fully qualified Package URLs extracted from raw ranges or product nodes. - `purls`: fully qualified Package URLs extracted from raw ranges or product nodes.
- `cpes`: Common Platform Enumerations when upstream docs provide them. - `cpes`: Common Platform Enumerations when upstream docs provide them.
- `aliases`: Any alternate advisory identifiers present in the payload. - `aliases`: Any alternate advisory identifiers present in the payload.
- `references`: Array of `{ type, url }` pairs pointing back to vendor advisories, patches, or exploits. - `references`: Array of `{ type, url }` pairs pointing back to vendor advisories, patches, or exploits.
- `reconciled_from`: Provenance of linkset entries (JSON Pointer or field origin) to make automated checks auditable. - `reconciled_from`: Provenance of linkset entries (JSON Pointer or field origin) to make automated checks auditable.
Canonicalisation rules: Canonicalisation rules:
- Package URLs are rendered in canonical form without qualifiers/subpaths (`pkg:type/namespace/name@version`). - Package URLs are rendered in canonical form without qualifiers/subpaths (`pkg:type/namespace/name@version`).
- CPE values are normalised to the 2.3 binding (`cpe:2.3:part:vendor:product:version:*:*:*:*:*:*:*`). - CPE values are normalised to the 2.3 binding (`cpe:2.3:part:vendor:product:version:*:*:*:*:*:*:*`).
- Connector mapping stages are responsible for the canonical form; ingestion trims whitespace but otherwise preserves the original order and duplicate entries so downstream policy can reason about upstream intent. - Connector mapping stages are responsible for the canonical form; ingestion trims whitespace but otherwise preserves the original order and duplicate entries so downstream policy can reason about upstream intent.
### 4.4 `advisory_observations` ### 4.4 `advisory_observations`
`advisory_observations` is an immutable projection of the validated raw document used by Link‑Not‑Merge overlays. Fields mirror the JSON contract surfaced by `StellaOps.Concelier.Models.Observations.AdvisoryObservation`. `advisory_observations` is an immutable projection of the validated raw document used by Link‑Not‑Merge overlays. Fields mirror the JSON contract surfaced by `StellaOps.Concelier.Models.Observations.AdvisoryObservation`.
| Field | Type | Notes | | Field | Type | Notes |
|-------|------|-------| |-------|------|-------|
| `_id` | string | Deterministic observation id — `{tenant}:{source.vendor}:{upstreamId}:{revision}`. | | `_id` | string | Deterministic observation id — `{tenant}:{source.vendor}:{upstreamId}:{revision}`. |
| `tenant` | string | Lower-case tenant identifier. | | `tenant` | string | Lower-case tenant identifier. |
| `source.vendor` / `source.stream` | string | Connector identity (e.g., `vendor/redhat`, `ecosystem/osv`). | | `source.vendor` / `source.stream` | string | Connector identity (e.g., `vendor/redhat`, `ecosystem/osv`). |
| `source.api` | string | Absolute URI the connector fetched from. | | `source.api` | string | Absolute URI the connector fetched from. |
| `source.collectorVersion` | string | Optional semantic version of the connector build. | | `source.collectorVersion` | string | Optional semantic version of the connector build. |
| `upstream.upstream_id` | string | Advisory identifier as issued by the provider (CVE, vendor ID, etc.). | | `upstream.upstream_id` | string | Advisory identifier as issued by the provider (CVE, vendor ID, etc.). |
| `upstream.document_version` | string | Upstream revision/version string. | | `upstream.document_version` | string | Upstream revision/version string. |
| `upstream.fetchedAt` / `upstream.receivedAt` | datetime | UTC timestamps recorded by the connector. | | `upstream.fetchedAt` / `upstream.receivedAt` | datetime | UTC timestamps recorded by the connector. |
| `upstream.contentHash` | string | `sha256:` digest used for idempotency. | | `upstream.contentHash` | string | `sha256:` digest used for idempotency. |
| `upstream.signature` | object | `{present, format?, keyId?, signature?}` describing upstream signature material. | | `upstream.signature` | object | `{present, format?, keyId?, signature?}` describing upstream signature material. |
| `content.format` / `content.specVersion` | string | Raw payload format metadata (CSAF, OSV, JSON, etc.). | | `content.format` / `content.specVersion` | string | Raw payload format metadata (CSAF, OSV, JSON, etc.). |
| `content.raw` | object | Full upstream document stored losslessly (Relaxed Extended JSON). | | `content.raw` | object | Full upstream document stored losslessly (Relaxed Extended JSON). |
| `content.metadata` | object | Optional connector-specific metadata (batch ids, hints). | | `content.metadata` | object | Optional connector-specific metadata (batch ids, hints). |
| `linkset.aliases` | array | Connector-supplied aliases (trimmed, order preserved, duplicates allowed). | | `linkset.aliases` | array | Connector-supplied aliases (trimmed, order preserved, duplicates allowed). |
| `linkset.purls` | array | Connector-supplied PURLs (ingestion preserves order and duplicates). | | `linkset.purls` | array | Connector-supplied PURLs (ingestion preserves order and duplicates). |
| `linkset.cpes` | array | Connector-supplied CPE URIs (trimmed, order preserved). | | `linkset.cpes` | array | Connector-supplied CPE URIs (trimmed, order preserved). |
| `linkset.references` | array | `{ type, url }` pairs (trimmed; ingestion preserves order). | | `linkset.references` | array | `{ type, url }` pairs (trimmed; ingestion preserves order). |
| `createdAt` | datetime | Timestamp when Concelier persisted the observation. | | `createdAt` | datetime | Timestamp when Concelier persisted the observation. |
| `attributes` | object | Optional provenance attributes keyed by connector. | | `attributes` | object | Optional provenance attributes keyed by connector. |
## 5. Error Model ## 5. Error Model
| Code | Description | HTTP status | Surfaces | | Code | Description | HTTP status | Surfaces |
|------|-------------|-------------|----------| |------|-------------|-------------|----------|
| `ERR_AOC_001` | Forbidden field detected (severity, cvss, effective data). | 400 | Ingestion APIs, CLI verifier, CI guard. | | `ERR_AOC_001` | Forbidden field detected (severity, cvss, effective data). | 400 | Ingestion APIs, CLI verifier, CI guard. |
| `ERR_AOC_002` | Merge attempt detected (multiple upstream sources fused into one document). | 400 | Ingestion APIs, CLI verifier. | | `ERR_AOC_002` | Merge attempt detected (multiple upstream sources fused into one document). | 400 | Ingestion APIs, CLI verifier. |
| `ERR_AOC_003` | Idempotency violation (duplicate without supersedes pointer). | 409 | Repository guard, Mongo unique index, CLI verifier. | | `ERR_AOC_003` | Idempotency violation (duplicate without supersedes pointer). | 409 | Repository guard, Mongo unique index, CLI verifier. |
| `ERR_AOC_004` | Missing provenance metadata (`source`, `upstream`, `signature`). | 422 | Schema validator, ingestion endpoints. | | `ERR_AOC_004` | Missing provenance metadata (`source`, `upstream`, `signature`). | 422 | Schema validator, ingestion endpoints. |
| `ERR_AOC_005` | Signature or checksum mismatch. | 422 | Collector validation, CLI verifier. | | `ERR_AOC_005` | Signature or checksum mismatch. | 422 | Collector validation, CLI verifier. |
| `ERR_AOC_006` | Attempt to persist derived findings from ingestion context. | 403 | Policy engine guard, Authority scopes. | | `ERR_AOC_006` | Attempt to persist derived findings from ingestion context. | 403 | Policy engine guard, Authority scopes. |
| `ERR_AOC_007` | Unknown top-level fields (schema violation). | 400 | Mongo validator, CLI verifier. | | `ERR_AOC_007` | Unknown top-level fields (schema violation). | 400 | Mongo validator, CLI verifier. |
Consumers should map these codes to CLI exit codes and structured log events so automation can fail fast and produce actionable guidance. Consumers should map these codes to CLI exit codes and structured log events so automation can fail fast and produce actionable guidance.
## 6. API and Tooling Interfaces ## 6. API and Tooling Interfaces
- **Concelier ingestion** (`StellaOps.Concelier.WebService`) - **Concelier ingestion** (`StellaOps.Concelier.WebService`)
- `POST /ingest/advisory`: accepts upstream payload metadata; server-side guard constructs and persists raw document. - `POST /ingest/advisory`: accepts upstream payload metadata; server-side guard constructs and persists raw document.
- `GET /advisories/raw/{id}` and filterable list endpoints expose raw documents for debugging and offline analysis. - `GET /advisories/raw/{id}` and filterable list endpoints expose raw documents for debugging and offline analysis.
- `POST /aoc/verify`: runs guard checks over recent documents and returns summary totals plus first violations. - `POST /aoc/verify`: runs guard checks over recent documents and returns summary totals plus first violations.
- **Excititor ingestion** (`StellaOps.Excititor.WebService`) mirrors the same surface for VEX documents. - **Excititor ingestion** (`StellaOps.Excititor.WebService`) mirrors the same surface for VEX documents.
- **CLI workflows** (`stella aoc verify`, `stella sources ingest --dry-run`) surface pre-flight verification; documentation will live in `/docs/modules/cli/guides/` alongside Sprint 19 CLI updates. - **CLI workflows** (`stella aoc verify`, `stella sources ingest --dry-run`) surface pre-flight verification; documentation will live in `/docs/modules/cli/guides/` alongside Sprint 19 CLI updates.
- **Authority scopes**: new `advisory:ingest`, `advisory:read`, `vex:ingest`, and `vex:read` scopes enforce least privilege; see [Authority Architecture](../modules/authority/architecture.md) for scope grammar. - **Authority scopes**: new `advisory:ingest`, `advisory:read`, `vex:ingest`, and `vex:read` scopes enforce least privilege; see [Authority Architecture](../modules/authority/architecture.md) for scope grammar.
## 7. Idempotency and Supersedes Rules ## 7. Idempotency and Supersedes Rules
1. Compute `content_hash` before any transformation; use it with `(source.vendor, upstream.upstream_id)` to detect duplicates. 1. Compute `content_hash` before any transformation; use it with `(source.vendor, upstream.upstream_id)` to detect duplicates.
2. If a document with the same hash already exists, skip the write and log a no-op. 2. If a document with the same hash already exists, skip the write and log a no-op.
3. When a new hash arrives for an existing upstream document, insert a new record and set `supersedes` to the previous `_id`. 3. When a new hash arrives for an existing upstream document, insert a new record and set `supersedes` to the previous `_id`.
4. Keep supersedes chains acyclic; collectors must resolve conflicts by rewinding before they insert. 4. Keep supersedes chains acyclic; collectors must resolve conflicts by rewinding before they insert.
5. Expose idempotency counters via metrics (`ingestion_write_total{result=ok|noop}`) to catch regressions early. 5. Expose idempotency counters via metrics (`ingestion_write_total{result=ok|noop}`) to catch regressions early.
## 8. Migration Playbook ## 8. Migration Playbook
1. Freeze ingestion writes except for raw pass-through paths while deploying schema validators. 1. Freeze ingestion writes except for raw pass-through paths while deploying schema validators.
2. Snapshot existing collections to `_backup_*` for rollback safety. 2. Snapshot existing collections to `_backup_*` for rollback safety.
3. Strip forbidden fields from historical documents into a temporary `advisory_view_legacy` used only during transition. 3. Strip forbidden fields from historical documents into a temporary `advisory_view_legacy` used only during transition.
4. Enable Mongo JSON schema validators for `advisory_raw` and `vex_raw`. 4. Enable Mongo JSON schema validators for `advisory_raw` and `vex_raw`.
5. Run collectors in `--dry-run` to confirm only allowed keys appear; fix violations before lifting the freeze. 5. Run collectors in `--dry-run` to confirm only allowed keys appear; fix violations before lifting the freeze.
6. Point Policy Engine to consume exclusively from raw collections and compute derived outputs downstream. 6. Point Policy Engine to consume exclusively from raw collections and compute derived outputs downstream.
7. Delete legacy normalisation paths from ingestion code and enable runtime guards plus CI linting. 7. Delete legacy normalisation paths from ingestion code and enable runtime guards plus CI linting.
8. Roll forward CLI, Console, and dashboards so operators can monitor AOC status end-to-end. 8. Roll forward CLI, Console, and dashboards so operators can monitor AOC status end-to-end.
## 9. Observability and Diagnostics ## 9. Observability and Diagnostics
- **Metrics**: `ingestion_write_total{result=ok|reject}`, `aoc_violation_total{code}`, `ingestion_signature_verified_total{result}`, `ingestion_latency_seconds`, `advisory_revision_count`. - **Metrics**: `ingestion_write_total{result=ok|reject}`, `aoc_violation_total{code}`, `ingestion_signature_verified_total{result}`, `ingestion_latency_seconds`, `advisory_revision_count`.
- **Traces**: spans `ingest.fetch`, `ingest.transform`, `ingest.write`, and `aoc.guard` with correlation IDs shared across workers. - **Traces**: spans `ingest.fetch`, `ingest.transform`, `ingest.write`, and `aoc.guard` with correlation IDs shared across workers.
- **Logs**: structured entries must include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, and `violation_code` when applicable. - **Logs**: structured entries must include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, and `violation_code` when applicable.
- **Dashboards**: DevOps should add panels for violation counts, signature failures, supersedes growth, and CLI verifier outcomes for each tenant. - **Dashboards**: DevOps should add panels for violation counts, signature failures, supersedes growth, and CLI verifier outcomes for each tenant.
## 10. Security and Tenancy Checklist ## 10. Security and Tenancy Checklist
- Enforce Authority scopes (`advisory:ingest`, `vex:ingest`, `advisory:read`, `vex:read`) and require tenant claims on every request. - Enforce Authority scopes (`advisory:ingest`, `vex:ingest`, `advisory:read`, `vex:read`) and require tenant claims on every request.
- Maintain pinned trust stores for signature verification; capture verification result in metrics and logs. - Maintain pinned trust stores for signature verification; capture verification result in metrics and logs.
- Ensure collectors never log secrets or raw authentication headers; redact tokens before persistence. - Ensure collectors never log secrets or raw authentication headers; redact tokens before persistence.
- Validate that Policy Engine remains the only identity with permission to write `effective_finding_*` documents. - Validate that Policy Engine remains the only identity with permission to write `effective_finding_*` documents.
- Verify offline bundles include the raw collections, guard configuration, and verifier binaries so air-gapped installs can audit parity. - Verify offline bundles include the raw collections, guard configuration, and verifier binaries so air-gapped installs can audit parity.
- Document operator steps for recovering from violations, including rollback to superseded revisions and re-running policy evaluation. - Document operator steps for recovering from violations, including rollback to superseded revisions and re-running policy evaluation.
## 11. Compliance Checklist ## 11. Compliance Checklist
- [ ] Deterministic guard enabled in Concelier and Excititor repositories. - [ ] Deterministic guard enabled in Concelier and Excititor repositories.
- [ ] Mongo validators deployed for `advisory_raw` and `vex_raw`. - [ ] Mongo validators deployed for `advisory_raw` and `vex_raw`.
- [ ] Authority scopes and tenant enforcement verified via integration tests. - [ ] Authority scopes and tenant enforcement verified via integration tests.
- [ ] CLI and CI pipelines run `stella aoc verify` against seeded snapshots. - [ ] CLI and CI pipelines run `stella aoc verify` against seeded snapshots.
- [ ] Observability feeds (metrics, logs, traces) wired into dashboards with alerts. - [ ] Observability feeds (metrics, logs, traces) wired into dashboards with alerts.
- [ ] Offline kit instructions updated to bundle validators and verifier tooling. - [ ] Offline kit instructions updated to bundle validators and verifier tooling.
- [ ] Security review recorded covering ingestion, tenancy, and rollback procedures. - [ ] Security review recorded covering ingestion, tenancy, and rollback procedures.
--- ---
*Last updated: 2025-10-27 (Sprint 19).* *Last updated: 2025-10-27 (Sprint 19).*

860
docs/moat.md
View File

@@ -1,430 +1,430 @@
# StellaOps Moat Track — Spec Outline v0.3 # StellaOps Moat Track — Spec Outline v0.3
**Scope of this doc:** **Scope of this doc:**
(1) Deterministic Replayable Scans (SRM), (2) Policy Engine & Lattice UI, (3) Sovereign Readiness (CryptoProfile + RootPack), (4) Attestation Observability Graph (AOG), (5) Procurement‑Grade Trust Statement, (6) Third‑Party Proof Channel, (7) Zastava differential SBOM + AI scheduler. (1) Deterministic Replayable Scans (SRM), (2) Policy Engine & Lattice UI, (3) Sovereign Readiness (CryptoProfile + RootPack), (4) Attestation Observability Graph (AOG), (5) Procurement‑Grade Trust Statement, (6) Third‑Party Proof Channel, (7) Zastava differential SBOM + AI scheduler.
Cross‑cutting principles: offline‑first, cryptographic determinism, evidence‑bound decisions, regional crypto compliance, minimal operational friction. Cross‑cutting principles: offline‑first, cryptographic determinism, evidence‑bound decisions, regional crypto compliance, minimal operational friction.
--- ---
## 0) Shared Concepts (applies to all 7) ## 0) Shared Concepts (applies to all 7)
* **Artifact identity:** digest-first (OCI image digest, file sha256). * **Artifact identity:** digest-first (OCI image digest, file sha256).
* **Canonicalization:** all structured payloads (SBOM, SRM, Trust Statement JSON, VEX) are normalized via Canonical JSON (RFC‑8785‑like) prior to hashing/signing. * **Canonicalization:** all structured payloads (SBOM, SRM, Trust Statement JSON, VEX) are normalized via Canonical JSON (RFC‑8785‑like) prior to hashing/signing.
* **Signatures:** DSSE envelopes; **dual‑signing** supported (e.g., FIPS ECDSA + GOST R 34.10; or ECDSA + SM2). * **Signatures:** DSSE envelopes; **dual‑signing** supported (e.g., FIPS ECDSA + GOST R 34.10; or ECDSA + SM2).
* **Attestation chain:** each decision (scan, VEX merge, policy evaluation) yields a signed, replayable record. * **Attestation chain:** each decision (scan, VEX merge, policy evaluation) yields a signed, replayable record.
* **Profiles & Packs:** **CryptoProfile** (algorithm + root policy) and **RootPack** (trust anchors + OCSP/CRL/TSA mirrors) are versioned and importable. * **Profiles & Packs:** **CryptoProfile** (algorithm + root policy) and **RootPack** (trust anchors + OCSP/CRL/TSA mirrors) are versioned and importable.
* **Policy Unit Tests:** any policy/lattice bundle ships with fixtures expected to pass during CI. * **Policy Unit Tests:** any policy/lattice bundle ships with fixtures expected to pass during CI.
--- ---
## 1) Deterministic Replayable Scans — SRM ## 1) Deterministic Replayable Scans — SRM
### Objective ### Objective
Make every scan a **provable, re‑executable fact**. Auditors can replay; results are bit‑for‑bit reproducible. Make every scan a **provable, re‑executable fact**. Auditors can replay; results are bit‑for‑bit reproducible.
### Deliverables ### Deliverables
* **SRM v0.1** schema (YAML/JSON) * **SRM v0.1** schema (YAML/JSON)
* Deterministic executor + `stella replay` * Deterministic executor + `stella replay`
* Replay diffing and result hashing * Replay diffing and result hashing
### SRM (Stella Replay Manifest) — schema (abridged) ### SRM (Stella Replay Manifest) — schema (abridged)
```yaml ```yaml
apiVersion: srm.stellaops.dev/v0.1 apiVersion: srm.stellaops.dev/v0.1
scan: scan:
id: uuid id: uuid
timestamp: ISO8601 timestamp: ISO8601
engine: { name: "stella-scan", version: "1.7.3", build_sha: "<sha>" } engine: { name: "stella-scan", version: "1.7.3", build_sha: "<sha>" }
environment: environment:
os_image: <digest> os_image: <digest>
kernel: { uname: "...", cgroups: "v2" } kernel: { uname: "...", cgroups: "v2" }
cpu_features: [avx2, sse4.2] cpu_features: [avx2, sse4.2]
inputs: inputs:
image: { name: "reg/app:1.9.2", digest: "<sha>", layers: ["<sha>", ...] } image: { name: "reg/app:1.9.2", digest: "<sha>", layers: ["<sha>", ...] }
sbom: { type: "cyclonedx@1.5", digest: "<sha>" } sbom: { type: "cyclonedx@1.5", digest: "<sha>" }
vex_set: [{ type: "openvex", digest: "<sha>" }] vex_set: [{ type: "openvex", digest: "<sha>" }]
lattice_policy: { id: "corp-policy@2025-08-15", digest: "<sha>" } lattice_policy: { id: "corp-policy@2025-08-15", digest: "<sha>" }
rules_and_feeds: rules_and_feeds:
rulesets: [{ name: "vuln-core", version: "2025.08.30", digest: "<sha>" }] rulesets: [{ name: "vuln-core", version: "2025.08.30", digest: "<sha>" }]
feeds: feeds:
- { name: "nvd", snapshot_date: "2025-08-30", archive_digest: "<sha>" } - { name: "nvd", snapshot_date: "2025-08-30", archive_digest: "<sha>" }
execution: execution:
mode: deterministic mode: deterministic
random_seed: 314159 random_seed: 314159
ordering: lexical ordering: lexical
heuristics: { binary_scan: true, secrets: false } heuristics: { binary_scan: true, secrets: false }
evidence: evidence:
files_hashed: 12873 files_hashed: 12873
samples: [{ path: "/usr/lib/libssl.so.3", sha256: "<sha>" }] samples: [{ path: "/usr/lib/libssl.so.3", sha256: "<sha>" }]
outputs: outputs:
report_digest: "<sha>" # canonical JSON root hash report_digest: "<sha>" # canonical JSON root hash
artifacts: artifacts:
- { name: "findings.json", sha256: "<sha>" } - { name: "findings.json", sha256: "<sha>" }
signatures: signatures:
- { scheme: "DSSE", CryptoProfile: "FIPS-140-3", signer: "build-ca@corp" } - { scheme: "DSSE", CryptoProfile: "FIPS-140-3", signer: "build-ca@corp" }
- { scheme: "DSSE", CryptoProfile: "GOST-2012", signer: "ru-ca@corp" } - { scheme: "DSSE", CryptoProfile: "GOST-2012", signer: "ru-ca@corp" }
rekor: { entries: ["<uuid>", ...] } # optional (offline allowed) rekor: { entries: ["<uuid>", ...] } # optional (offline allowed)
``` ```
### CLI & API ### CLI & API
* `stella scan --image reg/app@sha256:... --srm-out srm.yaml --findings findings.json` * `stella scan --image reg/app@sha256:... --srm-out srm.yaml --findings findings.json`
* `stella replay srm.yaml --out replay.json --assert-digest <sha>` * `stella replay srm.yaml --out replay.json --assert-digest <sha>`
* `POST /v1/srm/replay` → returns `ok`, `replay_report_digest`, `diff` (if any). * `POST /v1/srm/replay` → returns `ok`, `replay_report_digest`, `diff` (if any).
### Determinism Rules ### Determinism Rules
* Single thread or ordered parallel with stable scheduling; sorted inputs; fixed random seed; pinned rules/feeds/policies from SRM; identical canonicalization routines. * Single thread or ordered parallel with stable scheduling; sorted inputs; fixed random seed; pinned rules/feeds/policies from SRM; identical canonicalization routines.
### Acceptance Criteria ### Acceptance Criteria
* Replaying SRM on a different host returns identical `report_digest`. * Replaying SRM on a different host returns identical `report_digest`.
* If any feed archive differs by 1 bit, replay fails with a precise diff. * If any feed archive differs by 1 bit, replay fails with a precise diff.
* SRM size ≤ 25 MB for a typical microservice image (excludes large feed archives, which may be referenced by digest and bundled as a side‑car tar). * SRM size ≤ 25 MB for a typical microservice image (excludes large feed archives, which may be referenced by digest and bundled as a side‑car tar).
--- ---
## 2) Policy Engine & Lattice UI ## 2) Policy Engine & Lattice UI
### Objective ### Objective
Turn VEX merging and severity logic into **programmable, testable algebra** with explainability. Turn VEX merging and severity logic into **programmable, testable algebra** with explainability.
### Model ### Model
* **Domain:** partial order over vulnerability states: * **Domain:** partial order over vulnerability states:
`unknown < under_investigation < affected || not_affected < fixed`. `unknown < under_investigation < affected || not_affected < fixed`.
Cross‑product with *scope*: `{runtime_path, build_path, optional_path}` and *confidence*: `{low, med, high}`. Cross‑product with *scope*: `{runtime_path, build_path, optional_path}` and *confidence*: `{low, med, high}`.
* **Merge semantics:** monotonic lattice joins; conflict resolution rules prioritized by signed source trust and policy precedence. * **Merge semantics:** monotonic lattice joins; conflict resolution rules prioritized by signed source trust and policy precedence.
### DSL (sketch) ### DSL (sketch)
```hocon ```hocon
policy "corp-runtime" version "2025.08.15" { policy "corp-runtime" version "2025.08.15" {
sources { sources {
trust_order = ["vendor:redhat", "internal:appsec", "public:nvd"] trust_order = ["vendor:redhat", "internal:appsec", "public:nvd"]
require_signatures = true require_signatures = true
} }
rules { rules {
when vex.statement == "not_affected" when vex.statement == "not_affected"
and evidence.entrypoint_exposes == false and evidence.entrypoint_exposes == false
then state := not_affected with confidence := high; then state := not_affected with confidence := high;
when package.is_dev_dependency == true when package.is_dev_dependency == true
then scope := optional_path; then scope := optional_path;
when cvss >= 9.0 and reachable == true when cvss >= 9.0 and reachable == true
then priority := "block"; then priority := "block";
} }
guards { guards {
forbid unsigned_sources; forbid unsigned_sources;
forbid downgrade_of_state_below previous_state; forbid downgrade_of_state_below previous_state;
} }
} }
``` ```
### UI (“Trust Algebra Studio”) ### UI (“Trust Algebra Studio”)
* Drag‑and‑drop rule blocks, precedence editor, **simulation mode** on sample SBOM/VEX; **policy unit tests**. * Drag‑and‑drop rule blocks, precedence editor, **simulation mode** on sample SBOM/VEX; **policy unit tests**.
* Export **signed** `.lattice.json`; importable into CI. * Export **signed** `.lattice.json`; importable into CI.
### CLI & API ### CLI & API
* `stella policy lint corp-runtime.lattice.json` * `stella policy lint corp-runtime.lattice.json`
* `stella policy test --fixtures fixtures/` * `stella policy test --fixtures fixtures/`
* `POST /v1/policy/evaluate` → normalized decision + proof trail. * `POST /v1/policy/evaluate` → normalized decision + proof trail.
### Acceptance Criteria ### Acceptance Criteria
* Given same inputs, policy evaluation yields identical decision + proof trail hash. * Given same inputs, policy evaluation yields identical decision + proof trail hash.
* UI can round‑trip DSL ⇄ JSON with no semantic drift. * UI can round‑trip DSL ⇄ JSON with no semantic drift.
* Policy pack signature required to run in “enforced” mode. * Policy pack signature required to run in “enforced” mode.
--- ---
## 3) Sovereign Readiness — CryptoProfile + RootPack ## 3) Sovereign Readiness — CryptoProfile + RootPack
### Objective ### Objective
**Drop‑in regional cryptography** (Russia/China/EU/US) with offline operation. **Drop‑in regional cryptography** (Russia/China/EU/US) with offline operation.
### CryptoProfile (attached to every signature/attestation) ### CryptoProfile (attached to every signature/attestation)
```json ```json
{ {
"id": "GOST-2012@v1", "id": "GOST-2012@v1",
"algorithms": {"sign":"GOST R 34.10-2012","hash":"GOST R 34.11-2012","cipher":"GOST 34.12-2015"}, "algorithms": {"sign":"GOST R 34.10-2012","hash":"GOST R 34.11-2012","cipher":"GOST 34.12-2015"},
"key_policy": {"curve":"id-tc26-gost-3410-2012-256","hsm_required": true}, "key_policy": {"curve":"id-tc26-gost-3410-2012-256","hsm_required": true},
"time_stamping": {"tsa": "rootpack://ru/tsa1"}, "time_stamping": {"tsa": "rootpack://ru/tsa1"},
"roots": ["rootpack://ru/trustanchors/gost-ca1"] "roots": ["rootpack://ru/trustanchors/gost-ca1"]
} }
``` ```
### RootPack ### RootPack
* Tarball containing: trust anchors, intermediate CAs, OCSP/CRL snapshots, TSA profiles, policy constraints (e.g., no cross‑sign with foreign roots), and region metadata. * Tarball containing: trust anchors, intermediate CAs, OCSP/CRL snapshots, TSA profiles, policy constraints (e.g., no cross‑sign with foreign roots), and region metadata.
* Installed via: `stella rootpack import rootpack_ru_v1.tar.gz`. * Installed via: `stella rootpack import rootpack_ru_v1.tar.gz`.
### Dual‑Signing & Guardrails ### Dual‑Signing & Guardrails
* Policy flags like `allow_dual_signing FIPS+GOST`, `forbid_sm2_for_us_exports` (example only). * Policy flags like `allow_dual_signing FIPS+GOST`, `forbid_sm2_for_us_exports` (example only).
* Enforcement happens at signing time and verification time. * Enforcement happens at signing time and verification time.
### Acceptance Criteria ### Acceptance Criteria
* Offline verification using RootPack succeeds; online OCSP disabled per policy. * Offline verification using RootPack succeeds; online OCSP disabled per policy.
* Mis‑profiled signatures are rejected with explicit reason codes. * Mis‑profiled signatures are rejected with explicit reason codes.
* Dual‑signed DSSE verifies under both profiles when allowed. * Dual‑signed DSSE verifies under both profiles when allowed.
--- ---
## 4) Attestation Observability Graph (AOG) ## 4) Attestation Observability Graph (AOG)
### Objective ### Objective
Make trust **observable**. Expose SLIs/SLOs for cryptographic posture & policy compliance. Make trust **observable**. Expose SLIs/SLOs for cryptographic posture & policy compliance.
### Data Model ### Data Model
* Nodes: `{artifact, sbom, policy, vex, srm, signature, rootpack, runtime_instance}` * Nodes: `{artifact, sbom, policy, vex, srm, signature, rootpack, runtime_instance}`
* Edges: `derived_from`, `signed_by`, `evaluated_by`, `replayed_by`, `runs_as` * Edges: `derived_from`, `signed_by`, `evaluated_by`, `replayed_by`, `runs_as`
### Metrics (OpenTelemetry/Prometheus) ### Metrics (OpenTelemetry/Prometheus)
* `stella_trust_sli{service,env}` = fraction of running pods whose image has a valid SRM‑backed attestation chain under the active policy. * `stella_trust_sli{service,env}` = fraction of running pods whose image has a valid SRM‑backed attestation chain under the active policy.
* `stella_attestation_latency_seconds` (P50/P95) from build to verified‑ready. * `stella_attestation_latency_seconds` (P50/P95) from build to verified‑ready.
* `stella_policy_drift_events_total` (increment when running policy != signed policy). * `stella_policy_drift_events_total` (increment when running policy != signed policy).
* `stella_exception_without_proof_total` (must be 0). * `stella_exception_without_proof_total` (must be 0).
* `stella_replay_success_ratio` (per week). * `stella_replay_success_ratio` (per week).
**SLO Example** **SLO Example**
* **Trust SLO ≥ 99.9%** measured hourly; error budget resets monthly. * **Trust SLO ≥ 99.9%** measured hourly; error budget resets monthly.
### Interfaces ### Interfaces
* `stella aog export --format otlp` * `stella aog export --format otlp`
* `GET /v1/aog/graph?artifactDigest=...` → subgraph JSON (with signed edge proofs). * `GET /v1/aog/graph?artifactDigest=...` → subgraph JSON (with signed edge proofs).
* Grafana dashboards (packaged). * Grafana dashboards (packaged).
### Acceptance Criteria ### Acceptance Criteria
* AOG can reconstruct the full trust lineage for any running pod in ≤ 2s (p95) on a 1k‑service cluster. * AOG can reconstruct the full trust lineage for any running pod in ≤ 2s (p95) on a 1k‑service cluster.
* Metrics cardinality bounded (service/env/policy only). * Metrics cardinality bounded (service/env/policy only).
--- ---
## 5) Procurement‑Grade “Trust Statement” ## 5) Procurement‑Grade “Trust Statement”
### Objective ### Objective
One **board‑ready** artifact that unifies security posture across vendors; machine‑readable twin for ERP/GRC. One **board‑ready** artifact that unifies security posture across vendors; machine‑readable twin for ERP/GRC.
### Outputs ### Outputs
* **PDF** (human): signed, watermark, controlled fields for vendor name/version/date, summary graphs, SLOs, exceptions with PCE (proof‑carrying exceptions). * **PDF** (human): signed, watermark, controlled fields for vendor name/version/date, summary graphs, SLOs, exceptions with PCE (proof‑carrying exceptions).
* **JSON** (machine): normalized schema below; DSSE‑signed; includes SRM and policy references. * **JSON** (machine): normalized schema below; DSSE‑signed; includes SRM and policy references.
```json ```json
{ {
"schema": "trust-statement.stellaops.dev/v1", "schema": "trust-statement.stellaops.dev/v1",
"vendor": {"name": "Acme","product":"Payments API","version":"1.9.2"}, "vendor": {"name": "Acme","product":"Payments API","version":"1.9.2"},
"build": {"image_digest":"sha256:...","srm_digest":"sha256:..."}, "build": {"image_digest":"sha256:...","srm_digest":"sha256:..."},
"policy": {"id":"corp-runtime@2025-08-15","digest":"sha256:..."}, "policy": {"id":"corp-runtime@2025-08-15","digest":"sha256:..."},
"summary": { "summary": {
"trust_sli": 0.9992, "trust_sli": 0.9992,
"exceptions": 1, "exceptions": 1,
"open_findings": {"critical":0,"high":2,"medium":4,"low":12} "open_findings": {"critical":0,"high":2,"medium":4,"low":12}
}, },
"exceptions": [{ "exceptions": [{
"id":"EXC-2025-0912", "id":"EXC-2025-0912",
"reason":"not_affected-via-vex", "reason":"not_affected-via-vex",
"proof_digest":"sha256:...", "proof_digest":"sha256:...",
"expiry":"2026-01-15" "expiry":"2026-01-15"
}], }],
"signatures": [{ "CryptoProfile":"FIPS-140-3" }] "signatures": [{ "CryptoProfile":"FIPS-140-3" }]
} }
``` ```
### Integrations ### Integrations
* Push connectors: **SAP Ariba, ServiceNow, Archer, Jira** (webhooks or SFTP in offline flows). * Push connectors: **SAP Ariba, ServiceNow, Archer, Jira** (webhooks or SFTP in offline flows).
* CLI: `stella trust-statement generate --srm srm.yaml --policy corp-runtime.lattice.json --out acme-1.9.2.trust.json --pdf`. * CLI: `stella trust-statement generate --srm srm.yaml --policy corp-runtime.lattice.json --out acme-1.9.2.trust.json --pdf`.
### Acceptance Criteria ### Acceptance Criteria
* JSON validates against schema; PDF and JSON hashes match the DSSE statement. * JSON validates against schema; PDF and JSON hashes match the DSSE statement.
* ERP ingest POC: Ariba/ServiceNow field mapping validated. * ERP ingest POC: Ariba/ServiceNow field mapping validated.
--- ---
## 6) Third‑Party Proof Channel ## 6) Third‑Party Proof Channel
### Objective ### Objective
Create a **publisher ecosystem** for upstream proofs: SBOM, VEX, and **VDR (Vulnerability Derivation Reason)**. Create a **publisher ecosystem** for upstream proofs: SBOM, VEX, and **VDR (Vulnerability Derivation Reason)**.
### Publisher Model ### Publisher Model
* **Identity:** publishers obtain a **Publisher Certificate** (could be verified via RootPack‑anchored CA or cross‑signed). * **Identity:** publishers obtain a **Publisher Certificate** (could be verified via RootPack‑anchored CA or cross‑signed).
* **Submission:** `stella ledger publish --type {sbom|vex|vdr} --artifact <digest> --file proof.json --sign`. * **Submission:** `stella ledger publish --type {sbom|vex|vdr} --artifact <digest> --file proof.json --sign`.
* **Moderation & Revocation:** CRL‑like **Proof Revocation List (PRL)** with signed reasons. * **Moderation & Revocation:** CRL‑like **Proof Revocation List (PRL)** with signed reasons.
### VDR (schema sketch) ### VDR (schema sketch)
```json ```json
{ {
"schema":"vdr.stellaops.dev/v1", "schema":"vdr.stellaops.dev/v1",
"artifact":"sha256:...", "artifact":"sha256:...",
"cve":"CVE-2025-12345", "cve":"CVE-2025-12345",
"claim":"not_affected", "claim":"not_affected",
"method":"entrypoint_unreachable|abi_mismatch|dead_code", "method":"entrypoint_unreachable|abi_mismatch|dead_code",
"evidence_refs":[{"type":"symbol_map","digest":"sha256:..."}], "evidence_refs":[{"type":"symbol_map","digest":"sha256:..."}],
"publisher":"redhat://rhel", "publisher":"redhat://rhel",
"signatures":[...] "signatures":[...]
} }
``` ```
### Consumption ### Consumption
* Policies can **prioritize** publisher channels by trust level. * Policies can **prioritize** publisher channels by trust level.
* AOG shows which proofs originated from which publishers. * AOG shows which proofs originated from which publishers.
### Acceptance Criteria ### Acceptance Criteria
* At least one upstream (e.g., base image vendor) can publish and your policy can consume & rank it. * At least one upstream (e.g., base image vendor) can publish and your policy can consume & rank it.
* PRL revokes a proof and AOG reflects the change within 5 minutes. * PRL revokes a proof and AOG reflects the change within 5 minutes.
--- ---
## 7) Zastava — differential SBOM + AI enrichment scheduler ## 7) Zastava — differential SBOM + AI enrichment scheduler
### Objective ### Objective
Produce **entrypoint‑aware differential SBOMs** and continually **re‑enrich** new/old SBOMs with AI context and exposure‑aware prioritization. Produce **entrypoint‑aware differential SBOMs** and continually **re‑enrich** new/old SBOMs with AI context and exposure‑aware prioritization.
### Concepts ### Concepts
* **dSBOM:** SBOM that reflects effective dependency set for a specific `ENTRYPOINT/CMD` and runtime flags (e.g., `--server.urls`, `DOTNET_...`). * **dSBOM:** SBOM that reflects effective dependency set for a specific `ENTRYPOINT/CMD` and runtime flags (e.g., `--server.urls`, `DOTNET_...`).
* **Scheduler:** rescans **old SBOMs** when: (a) new CVE feeds arrive, (b) new VEX/VDR appear, (c) policy changes, or (d) AI models learn new exploitability signals. * **Scheduler:** rescans **old SBOMs** when: (a) new CVE feeds arrive, (b) new VEX/VDR appear, (c) policy changes, or (d) AI models learn new exploitability signals.
### Pipeline ### Pipeline
1. **Static slice:** infer reachable packages from entrypoint (e.g., `.NET Kestrel` vs CLI tool). 1. **Static slice:** infer reachable packages from entrypoint (e.g., `.NET Kestrel` vs CLI tool).
2. **Runtime slice (optional):** collect process tree, open sockets, and imported modules at startup in a **shadow run** or mirrored traffic. 2. **Runtime slice (optional):** collect process tree, open sockets, and imported modules at startup in a **shadow run** or mirrored traffic.
3. **Diff:** `dSBOM = SBOM ∩ (static_reachable ∪ runtime_observed)`. 3. **Diff:** `dSBOM = SBOM ∩ (static_reachable ∪ runtime_observed)`.
4. **AI Enrichment:** Zastava annotates each finding with *context weights* (exposed/not exposed, network scope, RBAC, secrets proximity). 4. **AI Enrichment:** Zastava annotates each finding with *context weights* (exposed/not exposed, network scope, RBAC, secrets proximity).
5. **Plan:** produce PRs (Dockerfile base bump, package pin, k8s Service change). 5. **Plan:** produce PRs (Dockerfile base bump, package pin, k8s Service change).
6. **Scheduler:** 6. **Scheduler:**
* `stella zastava schedule --query 'service=payments AND env=prod' --interval 6h` * `stella zastava schedule --query 'service=payments AND env=prod' --interval 6h`
* Triggers re‑evaluation and emits updated SRM + Trust deltas. * Triggers re‑evaluation and emits updated SRM + Trust deltas.
### dSBOM format (addon) ### dSBOM format (addon)
```json ```json
{ {
"schema":"cyclonedx+stella-diff@1.0", "schema":"cyclonedx+stella-diff@1.0",
"base_sbom":"sha256:...", "base_sbom":"sha256:...",
"entrypoint": ["/app/bin/Release/net8.0/app.dll"], "entrypoint": ["/app/bin/Release/net8.0/app.dll"],
"cmd": ["--urls","http://127.0.0.1:8080"], "cmd": ["--urls","http://127.0.0.1:8080"],
"static_reachable": ["pkg:nuget/Kestrel@*", "pkg:nuget/System.Data@*"], "static_reachable": ["pkg:nuget/Kestrel@*", "pkg:nuget/System.Data@*"],
"runtime_observed": ["pkg:rpm/openssl@3.0.9"], "runtime_observed": ["pkg:rpm/openssl@3.0.9"],
"excluded_paths": ["/usr/share/docs/**"], "excluded_paths": ["/usr/share/docs/**"],
"digest":"sha256:..." "digest":"sha256:..."
} }
``` ```
### Kestrel example (priority logic) ### Kestrel example (priority logic)
* If Kestrel present but Service is `ClusterIP` and no Ingress, **deprioritize**; if `LoadBalancer`/`NodePort` with 0‑RTT TLS disabled, **prioritize**. * If Kestrel present but Service is `ClusterIP` and no Ingress, **deprioritize**; if `LoadBalancer`/`NodePort` with 0‑RTT TLS disabled, **prioritize**.
* Zastava produces an **explainable card**: “Priority lowered due to non‑exposed runtime path; evidence: `kubectl get svc`, `netstat`, policy rule #42.” * Zastava produces an **explainable card**: “Priority lowered due to non‑exposed runtime path; evidence: `kubectl get svc`, `netstat`, policy rule #42.”
### Acceptance Criteria ### Acceptance Criteria
* Changing `ENTRYPOINT` produces a different, signed dSBOM and updated priorities. * Changing `ENTRYPOINT` produces a different, signed dSBOM and updated priorities.
* Scheduler updates stale SBOMs and issues signed deltas without network access (when RootPacks + feed mirrors are available). * Scheduler updates stale SBOMs and issues signed deltas without network access (when RootPacks + feed mirrors are available).
--- ---
## Cross‑Cutting Security & Compliance ## Cross‑Cutting Security & Compliance
* **Sanctions & Guardrails:** per‑profile constraints (e.g., forbid dual‑signing across certain jurisdictions). Policy‑enforced at sign/verify time. * **Sanctions & Guardrails:** per‑profile constraints (e.g., forbid dual‑signing across certain jurisdictions). Policy‑enforced at sign/verify time.
* **HSM Integrations:** PKCS#11 providers for each profile (FIPS, GOST, SM2). * **HSM Integrations:** PKCS#11 providers for each profile (FIPS, GOST, SM2).
* **Data handling:** SRMs and Trust Statements may contain paths and hashes; optional redaction profiles for vendor sharing. * **Data handling:** SRMs and Trust Statements may contain paths and hashes; optional redaction profiles for vendor sharing.
--- ---
## Interfaces Summary (CLI) ## Interfaces Summary (CLI)
```bash ```bash
# Scans / Replay # Scans / Replay
stella scan --image <digest> --srm-out srm.yaml stella scan --image <digest> --srm-out srm.yaml
stella replay srm.yaml --assert-digest <sha> stella replay srm.yaml --assert-digest <sha>
# Policy & Lattice # Policy & Lattice
stella policy lint corp.lattice.json stella policy lint corp.lattice.json
stella policy test --fixtures fixtures/ stella policy test --fixtures fixtures/
stella policy sign --profile FIPS-140-3 stella policy sign --profile FIPS-140-3
# Crypto Sovereign # Crypto Sovereign
stella rootpack import rootpack_ru_v1.tar.gz stella rootpack import rootpack_ru_v1.tar.gz
stella attest sign --profile GOST-2012 --in srm.yaml stella attest sign --profile GOST-2012 --in srm.yaml
stella attest verify --profiles GOST-2012,FIPS-140-3 stella attest verify --profiles GOST-2012,FIPS-140-3
# AOG # AOG
stella aog export --format otlp stella aog export --format otlp
stella aog graph --artifact <digest> stella aog graph --artifact <digest>
# Trust Statement # Trust Statement
stella trust-statement generate --srm srm.yaml --policy corp.lattice.json --pdf out.pdf --json out.json stella trust-statement generate --srm srm.yaml --policy corp.lattice.json --pdf out.pdf --json out.json
# Third-Party Proof Channel # Third-Party Proof Channel
stella ledger publish --type vdr --artifact <digest> --file vdr.json --sign stella ledger publish --type vdr --artifact <digest> --file vdr.json --sign
stella ledger revoke --id <proof-id> --reason "superseded" stella ledger revoke --id <proof-id> --reason "superseded"
# Zastava # Zastava
stella zastava diff-sbom --image <digest> --entrypoint "<cmd>" --out dsbom.json stella zastava diff-sbom --image <digest> --entrypoint "<cmd>" --out dsbom.json
stella zastava enrich --sbom dsbom.json --findings findings.json stella zastava enrich --sbom dsbom.json --findings findings.json
stella zastava schedule --query 'env=prod' --interval 6h stella zastava schedule --query 'env=prod' --interval 6h
``` ```
--- ---
## Success Metrics (per pillar) ## Success Metrics (per pillar)
* **SRM:** ≥ 99% of production images have SRM attached; **Replay Success Ratio** ≥ 0.99 weekly. * **SRM:** ≥ 99% of production images have SRM attached; **Replay Success Ratio** ≥ 0.99 weekly.
* **Policy/Lattice:** 100% of exceptions carry proof; policy test coverage ≥ 90% on top CVE classes. * **Policy/Lattice:** 100% of exceptions carry proof; policy test coverage ≥ 90% on top CVE classes.
* **Sovereign:** RootPacks for RU/CN/EU/US verified; dual‑signing working in CI; offline verification median < 200 ms. * **Sovereign:** RootPacks for RU/CN/EU/US verified; dual‑signing working in CI; offline verification median < 200 ms.
* **AOG:** Trust SLO ≥ 99.9%; lineage lookup p95 ≤ 2s. * **AOG:** Trust SLO ≥ 99.9%; lineage lookup p95 ≤ 2s.
* **Trust Statement:** Accepted by at least one ERP (pilot); generation time ≤ 15s. * **Trust Statement:** Accepted by at least one ERP (pilot); generation time ≤ 15s.
* **Third‑Party Channel:** ≥ 3 upstream publishers integrated; PRL revocations flow to AOG within 5 minutes. * **Third‑Party Channel:** ≥ 3 upstream publishers integrated; PRL revocations flow to AOG within 5 minutes.
* **Zastava:** dSBOM reduces non‑reachable high/critical findings by ≥ 35% without raising exposure incidents. * **Zastava:** dSBOM reduces non‑reachable high/critical findings by ≥ 35% without raising exposure incidents.
--- ---
## Risks & Mitigations ## Risks & Mitigations
* **Crypto complexity:** profile misuse → strict guardrails + default safe profiles, strong linting. * **Crypto complexity:** profile misuse → strict guardrails + default safe profiles, strong linting.
* **Cardinality/telemetry blow‑ups:** AOG label hygiene + sampling. * **Cardinality/telemetry blow‑ups:** AOG label hygiene + sampling.
* **Vendor adoption (Proof Channel):** seed with your own base images and internal frameworks; provide SDKs and reference publishers. * **Vendor adoption (Proof Channel):** seed with your own base images and internal frameworks; provide SDKs and reference publishers.
* **Determinism regressions:** CI runs replay tests on golden SRMs; any drift fails the build. * **Determinism regressions:** CI runs replay tests on golden SRMs; any drift fails the build.
--- ---
## 90‑Day Moat‑First Milestones ## 90‑Day Moat‑First Milestones
1. **SRM v0.1**: schema, deterministic executor, CLI replay, golden tests. 1. **SRM v0.1**: schema, deterministic executor, CLI replay, golden tests.
2. **Policy Engine MVP**: DSL + evaluator + UI simulation; policy unit tests; signed policy packs. 2. **Policy Engine MVP**: DSL + evaluator + UI simulation; policy unit tests; signed policy packs.
3. **CryptoProfile/RootPack MVP**: FIPS + GOST working; dual‑signing; offline verify. 3. **CryptoProfile/RootPack MVP**: FIPS + GOST working; dual‑signing; offline verify.
4. **AOG MVP**: lineage service + OTLP exporter + Grafana pack; Trust SLI. 4. **AOG MVP**: lineage service + OTLP exporter + Grafana pack; Trust SLI.
5. **Trust Statement MVP**: JSON + PDF; ServiceNow ingest POC. 5. **Trust Statement MVP**: JSON + PDF; ServiceNow ingest POC.
6. **Proof Channel alpha**: publisher identity + VDR schema + local ledger; PRL. 6. **Proof Channel alpha**: publisher identity + VDR schema + local ledger; PRL.
7. **Zastava α**: `diff-sbom` + exposure heuristics for `.NET Kestrel`; scheduler with offline mirrors. 7. **Zastava α**: `diff-sbom` + exposure heuristics for `.NET Kestrel`; scheduler with offline mirrors.
--- ---

44
docs/modules/advisory-ai/AGENTS.md
View File

@@ -1,22 +1,22 @@
# Advisory AI agent guide # Advisory AI agent guide
## Mission ## Mission
Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance. Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Review ./architecture.md for retrieval pipeline, guardrails, and profile support. 1. Review ./architecture.md for retrieval pipeline, guardrails, and profile support.
2. Open ../../implplan/SPRINTS.md and locate stories for this component. 2. Open ../../implplan/SPRINTS.md and locate stories for this component.
3. Check ./TASKS.md and update status before/after work. 3. Check ./TASKS.md and update status before/after work.
4. Read README/architecture for design context and update as the implementation evolves. 4. Read README/architecture for design context and update as the implementation evolves.
## Guardrails ## Guardrails
- Uphold Aggregation-Only Contract boundaries when consuming ingestion data. - Uphold Aggregation-Only Contract boundaries when consuming ingestion data.
- Preserve determinism and provenance in all derived outputs. - Preserve determinism and provenance in all derived outputs.
- Document offline/air-gap pathways for any new feature. - Document offline/air-gap pathways for any new feature.
- Update telemetry/observability assets alongside feature work. - Update telemetry/observability assets alongside feature work.

58
docs/modules/advisory-ai/README.md
View File

@@ -1,29 +1,29 @@
# StellaOps Advisory AI # StellaOps Advisory AI
Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance. Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance.
## Responsibilities ## Responsibilities
- Generate policy-aware advisory summaries with citations back to Conseiller and Excititor evidence. - Generate policy-aware advisory summaries with citations back to Conseiller and Excititor evidence.
- Explain conflicting advisories/VEX statements using weights from VEX Lens and Policy Engine. - Explain conflicting advisories/VEX statements using weights from VEX Lens and Policy Engine.
- Propose remediation hints aligned with Offline Kit staging and export bundles. - Propose remediation hints aligned with Offline Kit staging and export bundles.
- Expose API/UI surfaces with guardrails on model prompts, outputs, and retention. - Expose API/UI surfaces with guardrails on model prompts, outputs, and retention.
## Key components ## Key components
- RAG pipeline drawing from Conseiller, Excititor, VEX Lens, Policy Engine, and SBOM Service data. - RAG pipeline drawing from Conseiller, Excititor, VEX Lens, Policy Engine, and SBOM Service data.
- Prompt templates and guard models enforcing provenance and redaction policies. - Prompt templates and guard models enforcing provenance and redaction policies.
- Vercel/offline inference workers with deterministic caching of generated artefacts. - Vercel/offline inference workers with deterministic caching of generated artefacts.
## Integrations & dependencies ## Integrations & dependencies
- Authority for tenant-aware access control. - Authority for tenant-aware access control.
- Policy Engine for context-specific decisions and explain traces. - Policy Engine for context-specific decisions and explain traces.
- Console/CLI for interaction surfaces. - Console/CLI for interaction surfaces.
- Export Center/Vuln Explorer for embedding generated briefs. - Export Center/Vuln Explorer for embedding generated briefs.
## Operational notes ## Operational notes
- Model cache management and offline bundle packaging per Epic 8 requirements. - Model cache management and offline bundle packaging per Epic 8 requirements.
- Usage/latency dashboards for prompt/response monitoring. - Usage/latency dashboards for prompt/response monitoring.
- Redaction policies validated against security/LLM guardrail tests. - Redaction policies validated against security/LLM guardrail tests.
## Epic alignment ## Epic alignment
- Epic 8: Advisory AI Assistant. - Epic 8: Advisory AI Assistant.
- DOCS-AI stories to be tracked in ../../TASKS.md. - DOCS-AI stories to be tracked in ../../TASKS.md.

18
docs/modules/advisory-ai/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — Advisory AI # Task board — Advisory AI
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| ADVISORY-AI-DOCS-0001 | TODO | Docs Guild | Ensure ./README.md reflects the latest epic deliverables. | Align with ./AGENTS.md | | ADVISORY-AI-DOCS-0001 | TODO | Docs Guild | Ensure ./README.md reflects the latest epic deliverables. | Align with ./AGENTS.md |
| ADVISORY-AI-ENG-0001 | TODO | Module Team | Break down epic milestones into actionable stories. | Sync into ../../TASKS.md | | ADVISORY-AI-ENG-0001 | TODO | Module Team | Break down epic milestones into actionable stories. | Sync into ../../TASKS.md |
| ADVISORY-AI-OPS-0001 | TODO | Ops Guild | Prepare runbooks/observability assets once MVP lands. | Document outputs in ./README.md | | ADVISORY-AI-OPS-0001 | TODO | Ops Guild | Prepare runbooks/observability assets once MVP lands. | Document outputs in ./README.md |

200
docs/modules/advisory-ai/architecture.md
View File

@@ -1,100 +1,100 @@
# Advisory AI architecture # Advisory AI architecture
> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides. > Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
## 1) Goals ## 1) Goals
- Summarise advisories/VEX evidence into operator-ready briefs with citations. - Summarise advisories/VEX evidence into operator-ready briefs with citations.
- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data). - Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups. - Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
- Operate deterministically where possible; cache generated artefacts with digests for audit. - Operate deterministically where possible; cache generated artefacts with digests for audit.
## 2) Pipeline overview ## 2) Pipeline overview
``` ```
+---------------------+ +---------------------+
Concelier/VEX Lens | Evidence Retriever | Concelier/VEX Lens | Evidence Retriever |
Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON) Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON)
Zastava runtime +---------------------+ Zastava runtime +---------------------+
| |
v v
+-------------+ +-------------+
| Prompt | | Prompt |
| Assembler | | Assembler |
+-------------+ +-------------+
| |
v v
+-------------+ +-------------+
| Guarded LLM | | Guarded LLM |
| (local/host)| | (local/host)|
+-------------+ +-------------+
| |
v v
+-----------------+ +-----------------+
| Citation & | | Citation & |
| Validation | | Validation |
+-----------------+ +-----------------+
| |
v v
+----------------+ +----------------+
| Output cache | | Output cache |
| (hash, bundle) | | (hash, bundle) |
+----------------+ +----------------+
``` ```
## 3) Retrieval & context ## 3) Retrieval & context
- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs. - Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
- Context packs include: - Context packs include:
- Advisory raw excerpts with highlighted sections and source URLs. - Advisory raw excerpts with highlighted sections and source URLs.
- VEX statements (normalized tuples + trust metadata). - VEX statements (normalized tuples + trust metadata).
- Policy explain traces for the affected finding. - Policy explain traces for the affected finding.
- Runtime/impact hints from Zastava (exposure, entrypoints). - Runtime/impact hints from Zastava (exposure, entrypoints).
- Export-ready remediation data (fixed versions, patches). - Export-ready remediation data (fixed versions, patches).
All context references include `content_hash` and `source_id` enabling verifiable citations. All context references include `content_hash` and `source_id` enabling verifiable citations.
## 4) Guardrails ## 4) Guardrails
- Prompt templates enforce structure: summary, conflicts, remediation, references. - Prompt templates enforce structure: summary, conflicts, remediation, references.
- Response validator ensures: - Response validator ensures:
- No hallucinated advisories (every fact must map to input context). - No hallucinated advisories (every fact must map to input context).
- Citations follow `[n]` indexing referencing actual sources. - Citations follow `[n]` indexing referencing actual sources.
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes). - Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged. - Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
## 5) Output persistence ## 5) Output persistence
- Cached artefacts stored in `advisory_ai_outputs` with fields: - Cached artefacts stored in `advisory_ai_outputs` with fields:
- `output_hash` (sha256 of JSON response). - `output_hash` (sha256 of JSON response).
- `input_digest` (hash of context pack). - `input_digest` (hash of context pack).
- `summary`, `conflicts`, `remediation`, `citations`. - `summary`, `conflicts`, `remediation`, `citations`.
- `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.). - `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
- `signatures` (optional DSSE if run in deterministic mode). - `signatures` (optional DSSE if run in deterministic mode).
- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`. - Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
## 6) Profiles & sovereignty ## 6) Profiles & sovereignty
- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints. - **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements. - **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
## 7) APIs ## 7) APIs
- `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`. - `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`.
- `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking. - `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking.
- `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps. - `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps.
- `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center). - `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center).
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification. All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
## 8) Observability ## 8) Observability
- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`. - Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged. - Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write. - Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
## 9) Operational controls ## 9) Operational controls
- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`). - Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage. - Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests. - Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.

38
docs/modules/advisory-ai/implementation_plan.md
View File

@@ -1,19 +1,19 @@
# Implementation plan — Advisory AI # Implementation plan — Advisory AI
## Current objectives ## Current objectives
- Deliver Epic milestones summarised below while maintaining determinism and offline parity. - Deliver Epic milestones summarised below while maintaining determinism and offline parity.
- Keep documentation, telemetry, and runbooks aligned with sprint outcomes. - Keep documentation, telemetry, and runbooks aligned with sprint outcomes.
## Workstreams ## Workstreams
- Roadmap: reconcile open stories in ../../TASKS.md with module backlog. - Roadmap: reconcile open stories in ../../TASKS.md with module backlog.
- Delivery: ship features outlined in the epic while preserving AOC guardrails. - Delivery: ship features outlined in the epic while preserving AOC guardrails.
- Validation: extend tests/fixtures to guarantee reproducibility and provenance. - Validation: extend tests/fixtures to guarantee reproducibility and provenance.
## Epic milestones ## Epic milestones
- Epic 8: Advisory AI Assistant. - Epic 8: Advisory AI Assistant.
- DOCS-AI stories to be tracked in ../../TASKS.md. - DOCS-AI stories to be tracked in ../../TASKS.md.
## Coordination ## Coordination
- Review ./AGENTS.md before picking up work. - Review ./AGENTS.md before picking up work.
- Sync with owners listed in docs/implplan/SPRINTS.md. - Sync with owners listed in docs/implplan/SPRINTS.md.
- Update this plan whenever scope, dependencies, or guardrails change. - Update this plan whenever scope, dependencies, or guardrails change.

44
docs/modules/attestor/AGENTS.md
View File

@@ -1,22 +1,22 @@
# Attestor agent guide # Attestor agent guide
## Mission ## Mission
Attestor moves signed evidence through the trust chain by accepting DSSE bundles from Signer, registering them with Rekor v2, and serving deterministic verification payloads to other services. Attestor moves signed evidence through the trust chain by accepting DSSE bundles from Signer, registering them with Rekor v2, and serving deterministic verification payloads to other services.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

108
docs/modules/attestor/README.md
View File

@@ -1,54 +1,54 @@
# StellaOps Attestor # StellaOps Attestor
Attestor converts signed DSSE evidence from the Signer into transparency-log proofs and verifiable reports for every downstream surface (Policy Engine, Export Center, CLI, Console, Scheduler). It is the trust backbone that proves SBOM, scan, VEX, and policy artefacts were signed, witnessed, and preserved without tampering. Attestor converts signed DSSE evidence from the Signer into transparency-log proofs and verifiable reports for every downstream surface (Policy Engine, Export Center, CLI, Console, Scheduler). It is the trust backbone that proves SBOM, scan, VEX, and policy artefacts were signed, witnessed, and preserved without tampering.
## Why it exists ## Why it exists
- **Evidence first:** organisations need portable, verifiable attestations that prove build provenance, SBOM availability, policy verdicts, and VEX statements. - **Evidence first:** organisations need portable, verifiable attestations that prove build provenance, SBOM availability, policy verdicts, and VEX statements.
- **Policy enforcement:** verification policies ensure only approved issuers, key types, witnesses, and freshness windows are accepted. - **Policy enforcement:** verification policies ensure only approved issuers, key types, witnesses, and freshness windows are accepted.
- **Sovereign/offline-ready:** Attestor archives envelopes, signatures, and proofs so air-gapped deployments can replay verification without contacting external services. - **Sovereign/offline-ready:** Attestor archives envelopes, signatures, and proofs so air-gapped deployments can replay verification without contacting external services.
## Roles & surfaces ## Roles & surfaces
- **Subjects:** immutable digests for container images, SBOMs, reports, and policy bundles. - **Subjects:** immutable digests for container images, SBOMs, reports, and policy bundles.
- **Issuers:** builders, scanners, policy engines, or operators signing DSSE envelopes using keyless (Fulcio), KMS/HSM, or FIDO2 keys. - **Issuers:** builders, scanners, policy engines, or operators signing DSSE envelopes using keyless (Fulcio), KMS/HSM, or FIDO2 keys.
- **Consumers:** CLI/SDK, Console, Export Center, Scanner, Policy Engine, and Notify retrieving verification bundles or triggering policy checks. - **Consumers:** CLI/SDK, Console, Export Center, Scanner, Policy Engine, and Notify retrieving verification bundles or triggering policy checks.
- **Scopes:** Authority issues `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for issuer/key management; every call is bound with mTLS + DPoP. - **Scopes:** Authority issues `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for issuer/key management; every call is bound with mTLS + DPoP.
## Supported payloads ## Supported payloads
- `StellaOps.BuildProvenance@1`, `StellaOps.SBOMAttestation@1` - `StellaOps.BuildProvenance@1`, `StellaOps.SBOMAttestation@1`
- `StellaOps.ScanResults@1`, `StellaOps.VEXAttestation@1` - `StellaOps.ScanResults@1`, `StellaOps.VEXAttestation@1`
- `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1` - `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1`
All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`. All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`.
## Trust & envelope model ## Trust & envelope model
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof. - DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope. - Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
- Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts. - Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts.
- CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors. - CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors.
## UI, CLI, and SDK workflows ## UI, CLI, and SDK workflows
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows. - **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts. - **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact. - **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
## Storage, offline & air-gap posture ## Storage, offline & air-gap posture
- MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles. - MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
- Export Center packages attestation bundles (`stella export attestation-bundle`) for Offline Kit delivery. - Export Center packages attestation bundles (`stella export attestation-bundle`) for Offline Kit delivery.
- Transparency logs can be mirrored; offline mode records gaps and provides compensating controls. - Transparency logs can be mirrored; offline mode records gaps and provides compensating controls.
## Observability & performance ## Observability & performance
- Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_cache_hit_ratio`, `attestor_rekor_latency_seconds`. - Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_cache_hit_ratio`, `attestor_rekor_latency_seconds`.
- Logs capture tenant, issuer, subject digests, Rekor UUID, proof status, and policy verdict. - Logs capture tenant, issuer, subject digests, Rekor UUID, proof status, and policy verdict.
- Performance target: ≥1 000 envelopes/minute per worker with cached verification, batched operations, and concurrency controls. - Performance target: ≥1 000 envelopes/minute per worker with cached verification, batched operations, and concurrency controls.
## Key integrations ## Key integrations
- Signer (DSSE source), Authority (scopes & tenancy), Export Center (attestation bundles), Policy Engine (verification policies), Scanner/Excititor (subject evidence), Notify (key rotation & verification alerts), Observability stack (dashboards/alerts). - Signer (DSSE source), Authority (scopes & tenancy), Export Center (attestation bundles), Policy Engine (verification policies), Scanner/Excititor (subject evidence), Notify (key rotation & verification alerts), Observability stack (dashboards/alerts).
## Backlog references ## Backlog references
- DOCS-ATTEST-73-001 … DOCS-ATTEST-75-002 (Attestor console, key management, air-gap bundles) in ../../TASKS.md. - DOCS-ATTEST-73-001 … DOCS-ATTEST-75-002 (Attestor console, key management, air-gap bundles) in ../../TASKS.md.
- EXPORT-ATTEST-75-002 (Export Center attestation packaging) in ../export-center/TASKS.md. - EXPORT-ATTEST-75-002 (Export Center attestation packaging) in ../export-center/TASKS.md.
## Epic alignment ## Epic alignment
- **Epic 19 – Attestor Console:** console experience, verification APIs, issuer/key governance, transparency integration, and offline bundles. - **Epic 19 – Attestor Console:** console experience, verification APIs, issuer/key governance, transparency integration, and offline bundles.
- **Epic 10 – Export Center:** provenance alignment so exports carry signed manifests and attestation bundles. - **Epic 10 – Export Center:** provenance alignment so exports carry signed manifests and attestation bundles.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. > **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

18
docs/modules/attestor/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — Attestor # Task board — Attestor
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| ATTESTOR-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | ATTESTOR-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| ATTESTOR-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | ATTESTOR-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| ATTESTOR-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | ATTESTOR-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

864
docs/modules/attestor/architecture.md
View File

@@ -1,432 +1,432 @@
# component_architecture_attestor.md — **Stella Ops Attestor** (2025Q4) # component_architecture_attestor.md — **Stella Ops Attestor** (2025Q4)
> Derived from Epic 19 – Attestor Console with provenance hooks aligned to the Export Center bundle workflows scoped in Epic 10. > Derived from Epic 19 – Attestor Console with provenance hooks aligned to the Export Center bundle workflows scoped in Epic 10.
> **Scope.** Implementation‑ready architecture for the **Attestor**: the service that **submits** DSSE envelopes to **Rekor v2**, retrieves/validates inclusion proofs, caches results, and exposes verification APIs. It accepts DSSE **only** from the **Signer** over mTLS, enforces chain‑of‑trust to Stella Ops roots, and returns `{uuid, index, proof, logURL}` to calling services (Scanner.WebService for SBOMs; backend for final reports; Excititor exports when configured). > **Scope.** Implementation‑ready architecture for the **Attestor**: the service that **submits** DSSE envelopes to **Rekor v2**, retrieves/validates inclusion proofs, caches results, and exposes verification APIs. It accepts DSSE **only** from the **Signer** over mTLS, enforces chain‑of‑trust to Stella Ops roots, and returns `{uuid, index, proof, logURL}` to calling services (Scanner.WebService for SBOMs; backend for final reports; Excititor exports when configured).
--- ---
## 0) Mission & boundaries ## 0) Mission & boundaries
**Mission.** Turn a signed DSSE envelope from the Signer into a **transparency‑logged, verifiable fact** with a durable, replayable proof (Merkle inclusion + (optional) checkpoint anchoring). Provide **fast verification** for downstream consumers and a stable retrieval interface for UI/CLI. **Mission.** Turn a signed DSSE envelope from the Signer into a **transparency‑logged, verifiable fact** with a durable, replayable proof (Merkle inclusion + (optional) checkpoint anchoring). Provide **fast verification** for downstream consumers and a stable retrieval interface for UI/CLI.
**Boundaries.** **Boundaries.**
* Attestor **does not sign**; it **must not** accept unsigned or third‑party‑signed bundles. * Attestor **does not sign**; it **must not** accept unsigned or third‑party‑signed bundles.
* Attestor **does not decide PASS/FAIL**; it logs attestations for SBOMs, reports, and export artifacts. * Attestor **does not decide PASS/FAIL**; it logs attestations for SBOMs, reports, and export artifacts.
* Rekor v2 backends may be **local** (self‑hosted) or **remote**; Attestor handles both with retries, backoff, and idempotency. * Rekor v2 backends may be **local** (self‑hosted) or **remote**; Attestor handles both with retries, backoff, and idempotency.
--- ---
## 1) Topology & dependencies ## 1) Topology & dependencies
**Process shape:** single stateless service `stellaops/attestor` behind mTLS. **Process shape:** single stateless service `stellaops/attestor` behind mTLS.
**Dependencies:** **Dependencies:**
* **Signer** (caller) — authenticated via **mTLS** and **Authority** OpToks. * **Signer** (caller) — authenticated via **mTLS** and **Authority** OpToks.
* **Rekor v2** — tile‑backed transparency log endpoint(s). * **Rekor v2** — tile‑backed transparency log endpoint(s).
* **MinIO (S3)** — optional archive store for DSSE envelopes & verification bundles. * **MinIO (S3)** — optional archive store for DSSE envelopes & verification bundles.
* **MongoDB** — local cache of `{uuid, index, proof, artifactSha256, bundleSha256}`; job state; audit. * **MongoDB** — local cache of `{uuid, index, proof, artifactSha256, bundleSha256}`; job state; audit.
* **Redis** — dedupe/idempotency keys and short‑lived rate‑limit buckets. * **Redis** — dedupe/idempotency keys and short‑lived rate‑limit buckets.
* **Licensing Service (optional)** — “endorse” call for cross‑log publishing when customer opts‑in. * **Licensing Service (optional)** — “endorse” call for cross‑log publishing when customer opts‑in.
Trust boundary: **Only the Signer** is allowed to call submission endpoints; enforced by **mTLS peer cert allowlist** + `aud=attestor` OpTok. Trust boundary: **Only the Signer** is allowed to call submission endpoints; enforced by **mTLS peer cert allowlist** + `aud=attestor` OpTok.
--- ---
### Roles, identities & scopes ### Roles, identities & scopes
- **Subjects** — immutable digests for artifacts (container images, SBOMs, reports) referenced in DSSE envelopes. - **Subjects** — immutable digests for artifacts (container images, SBOMs, reports) referenced in DSSE envelopes.
- **Issuers** — authenticated builders/scanners/policy engines signing evidence; tracked with mode (`keyless`, `kms`, `hsm`, `fido2`) and tenant scope. - **Issuers** — authenticated builders/scanners/policy engines signing evidence; tracked with mode (`keyless`, `kms`, `hsm`, `fido2`) and tenant scope.
- **Consumers** — Scanner, Export Center, CLI, Console, Policy Engine that verify proofs using Attestor APIs. - **Consumers** — Scanner, Export Center, CLI, Console, Policy Engine that verify proofs using Attestor APIs.
- **Authority scopes** — `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for key management; all calls mTLS/DPoP-bound. - **Authority scopes** — `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for key management; all calls mTLS/DPoP-bound.
### Supported predicate types ### Supported predicate types
- `StellaOps.BuildProvenance@1` - `StellaOps.BuildProvenance@1`
- `StellaOps.SBOMAttestation@1` - `StellaOps.SBOMAttestation@1`
- `StellaOps.ScanResults@1` - `StellaOps.ScanResults@1`
- `StellaOps.PolicyEvaluation@1` - `StellaOps.PolicyEvaluation@1`
- `StellaOps.VEXAttestation@1` - `StellaOps.VEXAttestation@1`
- `StellaOps.RiskProfileEvidence@1` - `StellaOps.RiskProfileEvidence@1`
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`. Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
### Envelope & signature model ### Envelope & signature model
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing. - DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed. - Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements. - Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
- Archive CAS retains original envelope plus metadata for offline verification. - Archive CAS retains original envelope plus metadata for offline verification.
### Verification pipeline overview ### Verification pipeline overview
1. Fetch envelope (from request, cache, or storage) and validate DSSE structure. 1. Fetch envelope (from request, cache, or storage) and validate DSSE structure.
2. Verify signature(s) against configured trust roots; evaluate issuer policy. 2. Verify signature(s) against configured trust roots; evaluate issuer policy.
3. Retrieve or acquire inclusion proof from Rekor (primary + optional mirror). 3. Retrieve or acquire inclusion proof from Rekor (primary + optional mirror).
4. Validate Merkle proof against checkpoint; optionally verify witness endorsement. 4. Validate Merkle proof against checkpoint; optionally verify witness endorsement.
5. Return cached verification bundle including policy verdict and timestamps. 5. Return cached verification bundle including policy verdict and timestamps.
### UI & CLI touchpoints ### UI & CLI touchpoints
- Console: Evidence browser, verification report, chain-of-custody graph, issuer/key management, attestation workbench, bulk verification views. - Console: Evidence browser, verification report, chain-of-custody graph, issuer/key management, attestation workbench, bulk verification views.
- CLI: `stella attest sign|verify|list|fetch|key` with offline verification and export bundle support. - CLI: `stella attest sign|verify|list|fetch|key` with offline verification and export bundle support.
- SDKs expose sign/verify primitives for build pipelines. - SDKs expose sign/verify primitives for build pipelines.
### Performance & observability targets ### Performance & observability targets
- Throughput goal: ≥1 000 envelopes/minute per worker with cached verification. - Throughput goal: ≥1 000 envelopes/minute per worker with cached verification.
- Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_rekor_latency_seconds`, `attestor_cache_hit_ratio`. - Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_rekor_latency_seconds`, `attestor_cache_hit_ratio`.
- Logs include `tenant`, `issuer`, `subjectDigest`, `rekorUuid`, `proofStatus`; traces cover submission → Rekor → cache → response path. - Logs include `tenant`, `issuer`, `subjectDigest`, `rekorUuid`, `proofStatus`; traces cover submission → Rekor → cache → response path.
--- ---
## 2) Data model (Mongo) ## 2) Data model (Mongo)
Database: `attestor` Database: `attestor`
**Collections & schemas** **Collections & schemas**
* `entries` * `entries`
``` ```
{ _id: "<rekor-uuid>", { _id: "<rekor-uuid>",
artifact: { sha256: "<sha256>", kind: "sbom|report|vex-export", imageDigest?, subjectUri? }, artifact: { sha256: "<sha256>", kind: "sbom|report|vex-export", imageDigest?, subjectUri? },
bundleSha256: "<sha256>", // canonicalized DSSE bundleSha256: "<sha256>", // canonicalized DSSE
index: <int>, // log index/sequence if provided by backend index: <int>, // log index/sequence if provided by backend
proof: { // inclusion proof proof: { // inclusion proof
checkpoint: { origin, size, rootHash, timestamp }, checkpoint: { origin, size, rootHash, timestamp },
inclusion: { leafHash, path[] } // Merkle path (tiles) inclusion: { leafHash, path[] } // Merkle path (tiles)
}, },
log: { url, logId? }, log: { url, logId? },
createdAt, status: "included|pending|failed", createdAt, status: "included|pending|failed",
signerIdentity: { mode: "keyless|kms", issuer, san?, kid? } signerIdentity: { mode: "keyless|kms", issuer, san?, kid? }
} }
``` ```
* `dedupe` * `dedupe`
``` ```
{ key: "bundle:<sha256>", rekorUuid, createdAt, ttlAt } // idempotency key { key: "bundle:<sha256>", rekorUuid, createdAt, ttlAt } // idempotency key
``` ```
* `audit` * `audit`
``` ```
{ _id, ts, caller: { cn, mTLSThumbprint, sub, aud }, // from mTLS + OpTok { _id, ts, caller: { cn, mTLSThumbprint, sub, aud }, // from mTLS + OpTok
action: "submit|verify|fetch", action: "submit|verify|fetch",
artifactSha256, bundleSha256, rekorUuid?, index?, result, latencyMs, backend } artifactSha256, bundleSha256, rekorUuid?, index?, result, latencyMs, backend }
``` ```
Indexes: Indexes:
* `entries` on `artifact.sha256`, `bundleSha256`, `createdAt`, and `{status:1, createdAt:-1}`. * `entries` on `artifact.sha256`, `bundleSha256`, `createdAt`, and `{status:1, createdAt:-1}`.
* `dedupe.key` unique (TTL 24–48h). * `dedupe.key` unique (TTL 24–48h).
* `audit.ts` for time‑range queries. * `audit.ts` for time‑range queries.
--- ---
## 3) Input contract (from Signer) ## 3) Input contract (from Signer)
**Attestor accepts only** DSSE envelopes that satisfy all of: **Attestor accepts only** DSSE envelopes that satisfy all of:
1. **mTLS** peer certificate maps to `signer` service (CA‑pinned). 1. **mTLS** peer certificate maps to `signer` service (CA‑pinned).
2. **Authority** OpTok with `aud=attestor`, `scope=attestor.write`, DPoP or mTLS bound. 2. **Authority** OpTok with `aud=attestor`, `scope=attestor.write`, DPoP or mTLS bound.
3. DSSE envelope is **signed by the Signer’s key** (or includes a **Fulcio‑issued** cert chain) and **chains to configured roots** (Fulcio/KMS). 3. DSSE envelope is **signed by the Signer’s key** (or includes a **Fulcio‑issued** cert chain) and **chains to configured roots** (Fulcio/KMS).
4. **Predicate type** is one of Stella Ops types (sbom/report/vex‑export) with valid schema. 4. **Predicate type** is one of Stella Ops types (sbom/report/vex‑export) with valid schema.
5. `subject[*].digest.sha256` is present and canonicalized. 5. `subject[*].digest.sha256` is present and canonicalized.
**Wire shape (JSON):** **Wire shape (JSON):**
```json ```json
{ {
"bundle": { "dsse": { "payloadType": "application/vnd.in-toto+json", "payload": "<b64>", "signatures": [ ... ] }, "bundle": { "dsse": { "payloadType": "application/vnd.in-toto+json", "payload": "<b64>", "signatures": [ ... ] },
"certificateChain": [ "-----BEGIN CERTIFICATE-----..." ], "certificateChain": [ "-----BEGIN CERTIFICATE-----..." ],
"mode": "keyless" }, "mode": "keyless" },
"meta": { "meta": {
"artifact": { "sha256": "<subject sha256>", "kind": "sbom|report|vex-export", "imageDigest": "sha256:..." }, "artifact": { "sha256": "<subject sha256>", "kind": "sbom|report|vex-export", "imageDigest": "sha256:..." },
"bundleSha256": "<sha256 of canonical dsse>", "bundleSha256": "<sha256 of canonical dsse>",
"logPreference": "primary", // "primary" | "mirror" | "both" "logPreference": "primary", // "primary" | "mirror" | "both"
"archive": true // whether Attestor should archive bundle to S3 "archive": true // whether Attestor should archive bundle to S3
} }
} }
``` ```
--- ---
## 4) APIs ## 4) APIs
### 4.1 Submission ### 4.1 Submission
`POST /api/v1/rekor/entries` *(mTLS + OpTok required)* `POST /api/v1/rekor/entries` *(mTLS + OpTok required)*
* **Body**: as above. * **Body**: as above.
* **Behavior**: * **Behavior**:
* Verify caller (mTLS + OpTok). * Verify caller (mTLS + OpTok).
* Validate DSSE bundle (signature, cert chain to Fulcio/KMS; DSSE structure; payloadType allowed). * Validate DSSE bundle (signature, cert chain to Fulcio/KMS; DSSE structure; payloadType allowed).
* Idempotency: compute `bundleSha256`; check `dedupe`. If present, return existing `rekorUuid`. * Idempotency: compute `bundleSha256`; check `dedupe`. If present, return existing `rekorUuid`.
* Submit canonicalized bundle to Rekor v2 (primary or mirror according to `logPreference`). * Submit canonicalized bundle to Rekor v2 (primary or mirror according to `logPreference`).
* Retrieve **inclusion proof** (blocking until inclusion or up to `proofTimeoutMs`); if backend returns promise only, return `status=pending` and retry asynchronously. * Retrieve **inclusion proof** (blocking until inclusion or up to `proofTimeoutMs`); if backend returns promise only, return `status=pending` and retry asynchronously.
* Persist `entries` record; archive DSSE to S3 if `archive=true`. * Persist `entries` record; archive DSSE to S3 if `archive=true`.
* **Response 200**: * **Response 200**:
```json ```json
{ {
"uuid": "…", "uuid": "…",
"index": 123456, "index": 123456,
"proof": { "proof": {
"checkpoint": { "origin": "rekor@site", "size": 987654, "rootHash": "…", "timestamp": "…" }, "checkpoint": { "origin": "rekor@site", "size": 987654, "rootHash": "…", "timestamp": "…" },
"inclusion": { "leafHash": "…", "path": ["…","…"] } "inclusion": { "leafHash": "…", "path": ["…","…"] }
}, },
"logURL": "https://rekor…/api/v2/log/…/entries/…", "logURL": "https://rekor…/api/v2/log/…/entries/…",
"status": "included" "status": "included"
} }
``` ```
* **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`. * **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`.
### 4.2 Proof retrieval ### 4.2 Proof retrieval
`GET /api/v1/rekor/entries/{uuid}` `GET /api/v1/rekor/entries/{uuid}`
* Returns `entries` row (refreshes proof from Rekor if stale/missing). * Returns `entries` row (refreshes proof from Rekor if stale/missing).
* Accepts `?refresh=true` to force backend query. * Accepts `?refresh=true` to force backend query.
### 4.3 Verification (third‑party or internal) ### 4.3 Verification (third‑party or internal)
`POST /api/v1/rekor/verify` `POST /api/v1/rekor/verify`
* **Body** (one of): * **Body** (one of):
* `{ "uuid": "…" }` * `{ "uuid": "…" }`
* `{ "bundle": { …DSSE… } }` * `{ "bundle": { …DSSE… } }`
* `{ "artifactSha256": "…" }` *(looks up most recent entry)* * `{ "artifactSha256": "…" }` *(looks up most recent entry)*
* **Checks**: * **Checks**:
1. **Bundle signature** → cert chain to Fulcio/KMS roots configured. 1. **Bundle signature** → cert chain to Fulcio/KMS roots configured.
2. **Inclusion proof** → recompute leaf hash; verify Merkle path against checkpoint root. 2. **Inclusion proof** → recompute leaf hash; verify Merkle path against checkpoint root.
3. Optionally verify **checkpoint** against local trust anchors (if Rekor signs checkpoints). 3. Optionally verify **checkpoint** against local trust anchors (if Rekor signs checkpoints).
4. Confirm **subject.digest** matches caller‑provided hash (when given). 4. Confirm **subject.digest** matches caller‑provided hash (when given).
* **Response**: * **Response**:
```json ```json
{ "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" } { "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" }
``` ```
### 4.4 Batch submission (optional) ### 4.4 Batch submission (optional)
`POST /api/v1/rekor/batch` accepts an array of submission objects; processes with per‑item results. `POST /api/v1/rekor/batch` accepts an array of submission objects; processes with per‑item results.
--- ---
## 5) Rekor v2 driver (backend) ## 5) Rekor v2 driver (backend)
* **Canonicalization**: DSSE envelopes are **normalized** (stable JSON ordering, no insignificant whitespace) before hashing and submission. * **Canonicalization**: DSSE envelopes are **normalized** (stable JSON ordering, no insignificant whitespace) before hashing and submission.
* **Transport**: HTTP/2 with retries (exponential backoff, jitter), budgeted timeouts. * **Transport**: HTTP/2 with retries (exponential backoff, jitter), budgeted timeouts.
* **Idempotency**: if backend returns “already exists,” map to existing `uuid`. * **Idempotency**: if backend returns “already exists,” map to existing `uuid`.
* **Proof acquisition**: * **Proof acquisition**:
* In synchronous mode, poll the log for inclusion up to `proofTimeoutMs`. * In synchronous mode, poll the log for inclusion up to `proofTimeoutMs`.
* In asynchronous mode, return `pending` and schedule a **proof fetcher** job (Mongo job doc + backoff). * In asynchronous mode, return `pending` and schedule a **proof fetcher** job (Mongo job doc + backoff).
* **Mirrors/dual logs**: * **Mirrors/dual logs**:
* When `logPreference="both"`, submit to primary and mirror; store **both** UUIDs (primary canonical). * When `logPreference="both"`, submit to primary and mirror; store **both** UUIDs (primary canonical).
* Optional **cloud endorsement**: POST to the Stella Ops cloud `/attest/endorse` with `{uuid, artifactSha256}`; store returned endorsement id. * Optional **cloud endorsement**: POST to the Stella Ops cloud `/attest/endorse` with `{uuid, artifactSha256}`; store returned endorsement id.
--- ---
## 6) Security model ## 6) Security model
* **mTLS required** for submission from **Signer** (CA‑pinned). * **mTLS required** for submission from **Signer** (CA‑pinned).
* **Authority token** with `aud=attestor` and DPoP/mTLS binding must be presented; Attestor verifies both. * **Authority token** with `aud=attestor` and DPoP/mTLS binding must be presented; Attestor verifies both.
* **Bundle acceptance policy**: * **Bundle acceptance policy**:
* DSSE signature must chain to the configured **Fulcio** (keyless) or **KMS/HSM** roots. * DSSE signature must chain to the configured **Fulcio** (keyless) or **KMS/HSM** roots.
* SAN (Subject Alternative Name) must match **Signer identity** policy (e.g., `urn:stellaops:signer` or pinned OIDC issuer). * SAN (Subject Alternative Name) must match **Signer identity** policy (e.g., `urn:stellaops:signer` or pinned OIDC issuer).
* Predicate `predicateType` must be on allowlist (sbom/report/vex-export). * Predicate `predicateType` must be on allowlist (sbom/report/vex-export).
* `subject.digest.sha256` values must be present and well‑formed (hex). * `subject.digest.sha256` values must be present and well‑formed (hex).
* **No public submission** path. **Never** accept bundles from untrusted clients. * **No public submission** path. **Never** accept bundles from untrusted clients.
* **Client certificate allowlists**: optional `security.mtls.allowedSubjects` / `allowedThumbprints` tighten peer identity checks beyond CA pinning. * **Client certificate allowlists**: optional `security.mtls.allowedSubjects` / `allowedThumbprints` tighten peer identity checks beyond CA pinning.
* **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded. * **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded.
* **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor. * **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor.
--- ---
## 7) Storage & archival ## 7) Storage & archival
* **Entries** in Mongo provide a local ledger keyed by `rekorUuid` and **artifact sha256** for quick reverse lookups. * **Entries** in Mongo provide a local ledger keyed by `rekorUuid` and **artifact sha256** for quick reverse lookups.
* **S3 archival** (if enabled): * **S3 archival** (if enabled):
``` ```
s3://stellaops/attest/ s3://stellaops/attest/
dsse/<bundleSha256>.json dsse/<bundleSha256>.json
proof/<rekorUuid>.json proof/<rekorUuid>.json
bundle/<artifactSha256>.zip # optional verification bundle bundle/<artifactSha256>.zip # optional verification bundle
``` ```
* **Verification bundles** (zip): * **Verification bundles** (zip):
* DSSE (`*.dsse.json`), proof (`*.proof.json`), `chain.pem` (certs), `README.txt` with verification steps & hashes. * DSSE (`*.dsse.json`), proof (`*.proof.json`), `chain.pem` (certs), `README.txt` with verification steps & hashes.
--- ---
## 8) Observability & audit ## 8) Observability & audit
**Metrics** (Prometheus): **Metrics** (Prometheus):
* `attestor.submit_total{result,backend}` * `attestor.submit_total{result,backend}`
* `attestor.submit_latency_seconds{backend}` * `attestor.submit_latency_seconds{backend}`
* `attestor.proof_fetch_total{result}` * `attestor.proof_fetch_total{result}`
* `attestor.verify_total{result}` * `attestor.verify_total{result}`
* `attestor.dedupe_hits_total` * `attestor.dedupe_hits_total`
* `attestor.errors_total{type}` * `attestor.errors_total{type}`
**Correlation**: **Correlation**:
* HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing. * HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing.
**Tracing**: **Tracing**:
* Spans: `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `verify`. * Spans: `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `verify`.
**Audit**: **Audit**:
* Immutable `audit` rows (ts, caller, action, hashes, uuid, index, backend, result, latency). * Immutable `audit` rows (ts, caller, action, hashes, uuid, index, backend, result, latency).
--- ---
## 9) Configuration (YAML) ## 9) Configuration (YAML)
```yaml ```yaml
attestor: attestor:
listen: "https://0.0.0.0:8444" listen: "https://0.0.0.0:8444"
security: security:
mtls: mtls:
caBundle: /etc/ssl/signer-ca.pem caBundle: /etc/ssl/signer-ca.pem
requireClientCert: true requireClientCert: true
authority: authority:
issuer: "https://authority.internal" issuer: "https://authority.internal"
jwksUrl: "https://authority.internal/jwks" jwksUrl: "https://authority.internal/jwks"
requireSenderConstraint: "dpop" # or "mtls" requireSenderConstraint: "dpop" # or "mtls"
signerIdentity: signerIdentity:
mode: ["keyless","kms"] mode: ["keyless","kms"]
fulcioRoots: ["/etc/fulcio/root.pem"] fulcioRoots: ["/etc/fulcio/root.pem"]
allowedSANs: ["urn:stellaops:signer"] allowedSANs: ["urn:stellaops:signer"]
kmsKeys: ["kms://cluster-kms/stellaops-signer"] kmsKeys: ["kms://cluster-kms/stellaops-signer"]
rekor: rekor:
primary: primary:
url: "https://rekor-v2.internal" url: "https://rekor-v2.internal"
proofTimeoutMs: 15000 proofTimeoutMs: 15000
pollIntervalMs: 250 pollIntervalMs: 250
maxAttempts: 60 maxAttempts: 60
mirror: mirror:
enabled: false enabled: false
url: "https://rekor-v2.mirror" url: "https://rekor-v2.mirror"
mongo: mongo:
uri: "mongodb://mongo/attestor" uri: "mongodb://mongo/attestor"
s3: s3:
enabled: true enabled: true
endpoint: "http://minio:9000" endpoint: "http://minio:9000"
bucket: "stellaops" bucket: "stellaops"
prefix: "attest/" prefix: "attest/"
objectLock: "governance" objectLock: "governance"
redis: redis:
url: "redis://redis:6379/2" url: "redis://redis:6379/2"
quotas: quotas:
perCaller: perCaller:
qps: 50 qps: 50
burst: 100 burst: 100
``` ```
--- ---
## 10) End‑to‑end sequences ## 10) End‑to‑end sequences
**A) Submit & include (happy path)** **A) Submit & include (happy path)**
```mermaid ```mermaid
sequenceDiagram sequenceDiagram
autonumber autonumber
participant SW as Scanner.WebService participant SW as Scanner.WebService
participant SG as Signer participant SG as Signer
participant AT as Attestor participant AT as Attestor
participant RK as Rekor v2 participant RK as Rekor v2
SW->>SG: POST /sign/dsse (OpTok+PoE) SW->>SG: POST /sign/dsse (OpTok+PoE)
SG-->>SW: DSSE bundle (+certs) SG-->>SW: DSSE bundle (+certs)
SW->>AT: POST /rekor/entries (mTLS + OpTok) SW->>AT: POST /rekor/entries (mTLS + OpTok)
AT->>AT: Validate DSSE (chain to Fulcio/KMS; signer identity) AT->>AT: Validate DSSE (chain to Fulcio/KMS; signer identity)
AT->>RK: submit(bundle) AT->>RK: submit(bundle)
RK-->>AT: {uuid, index?} RK-->>AT: {uuid, index?}
AT->>RK: poll inclusion until proof or timeout AT->>RK: poll inclusion until proof or timeout
RK-->>AT: inclusion proof (checkpoint + path) RK-->>AT: inclusion proof (checkpoint + path)
AT-->>SW: {uuid, index, proof, logURL} AT-->>SW: {uuid, index, proof, logURL}
``` ```
**B) Verify by artifact digest (CLI)** **B) Verify by artifact digest (CLI)**
```mermaid ```mermaid
sequenceDiagram sequenceDiagram
autonumber autonumber
participant CLI as stellaops verify participant CLI as stellaops verify
participant SW as Scanner.WebService participant SW as Scanner.WebService
participant AT as Attestor participant AT as Attestor
CLI->>SW: GET /catalog/artifacts/{id} CLI->>SW: GET /catalog/artifacts/{id}
SW-->>CLI: {artifactSha256, rekor: {uuid}} SW-->>CLI: {artifactSha256, rekor: {uuid}}
CLI->>AT: POST /rekor/verify { uuid } CLI->>AT: POST /rekor/verify { uuid }
AT-->>CLI: { ok: true, index, logURL } AT-->>CLI: { ok: true, index, logURL }
``` ```
--- ---
## 11) Failure modes & responses ## 11) Failure modes & responses
| Condition | Return | Details | | | | Condition | Return | Details | | |
| ------------------------------------- | ----------------------- | --------------------------------------------------------- | -------- | ------------ | | ------------------------------------- | ----------------------- | --------------------------------------------------------- | -------- | ------------ |
| mTLS/OpTok invalid | `401 invalid_token` | Include `WWW-Authenticate` DPoP challenge when applicable | | | | mTLS/OpTok invalid | `401 invalid_token` | Include `WWW-Authenticate` DPoP challenge when applicable | | |
| Bundle not signed by trusted identity | `403 chain_untrusted` | DSSE accepted only from Signer identities | | | | Bundle not signed by trusted identity | `403 chain_untrusted` | DSSE accepted only from Signer identities | | |
| Duplicate bundle | `409 duplicate_bundle` | Return existing `uuid` (idempotent) | | | | Duplicate bundle | `409 duplicate_bundle` | Return existing `uuid` (idempotent) | | |
| Rekor unreachable/timeout | `502 rekor_unavailable` | Retry with backoff; surface `Retry-After` | | | | Rekor unreachable/timeout | `502 rekor_unavailable` | Retry with backoff; surface `Retry-After` | | |
| Inclusion proof timeout | `202 accepted` | `status=pending`, background job continues to fetch proof | | | | Inclusion proof timeout | `202 accepted` | `status=pending`, background job continues to fetch proof | | |
| Archive failure | `207 multi-status` | Entry recorded; archive will retry asynchronously | | | | Archive failure | `207 multi-status` | Entry recorded; archive will retry asynchronously | | |
| Verification mismatch | `400 verify_failed` | Include reason: chain | leafHash | rootMismatch | | Verification mismatch | `400 verify_failed` | Include reason: chain | leafHash | rootMismatch |
--- ---
## 12) Performance & scale ## 12) Performance & scale
* Stateless; scale horizontally. * Stateless; scale horizontally.
* **Targets**: * **Targets**:
* Submit+proof P95 ≤ **300 ms** (warm log; local Rekor). * Submit+proof P95 ≤ **300 ms** (warm log; local Rekor).
* Verify P95 ≤ **30 ms** from cache; ≤ **120 ms** with live proof fetch. * Verify P95 ≤ **30 ms** from cache; ≤ **120 ms** with live proof fetch.
* 1k submissions/minute per replica sustained. * 1k submissions/minute per replica sustained.
* **Hot caches**: `dedupe` (bundle hash → uuid), recent `entries` by artifact sha256. * **Hot caches**: `dedupe` (bundle hash → uuid), recent `entries` by artifact sha256.
--- ---
## 13) Testing matrix ## 13) Testing matrix
* **Happy path**: valid DSSE, inclusion within timeout. * **Happy path**: valid DSSE, inclusion within timeout.
* **Idempotency**: resubmit same `bundleSha256` → same `uuid`. * **Idempotency**: resubmit same `bundleSha256` → same `uuid`.
* **Security**: reject non‑Signer mTLS, wrong `aud`, DPoP replay, untrusted cert chain, forbidden predicateType. * **Security**: reject non‑Signer mTLS, wrong `aud`, DPoP replay, untrusted cert chain, forbidden predicateType.
* **Rekor variants**: promise‑then‑proof, proof delayed, mirror dual‑submit, mirror failure. * **Rekor variants**: promise‑then‑proof, proof delayed, mirror dual‑submit, mirror failure.
* **Verification**: corrupt leaf path, wrong root, tampered bundle. * **Verification**: corrupt leaf path, wrong root, tampered bundle.
* **Throughput**: soak test with 10k submissions; latency SLOs, zero drops. * **Throughput**: soak test with 10k submissions; latency SLOs, zero drops.
--- ---
## 14) Implementation notes ## 14) Implementation notes
* Language: **.NET 10** minimal API; `HttpClient` with **sockets handler** tuned for HTTP/2. * Language: **.NET 10** minimal API; `HttpClient` with **sockets handler** tuned for HTTP/2.
* JSON: **canonical writer** for DSSE payload hashing. * JSON: **canonical writer** for DSSE payload hashing.
* Crypto: use **BouncyCastle**/**System.Security.Cryptography**; PEM parsing for cert chains. * Crypto: use **BouncyCastle**/**System.Security.Cryptography**; PEM parsing for cert chains.
* Rekor client: pluggable driver; treat backend errors as retryable/non‑retryable with granular mapping. * Rekor client: pluggable driver; treat backend errors as retryable/non‑retryable with granular mapping.
* Safety: size caps on bundles; decompress bombs guarded; strict UTF‑8. * Safety: size caps on bundles; decompress bombs guarded; strict UTF‑8.
* CLI integration: `stellaops verify attestation <uuid|bundle|artifact>` calls `/rekor/verify`. * CLI integration: `stellaops verify attestation <uuid|bundle|artifact>` calls `/rekor/verify`.
--- ---
## 15) Optional features ## 15) Optional features
* **Dual‑log** write (primary + mirror) and **cross‑log proof** packaging. * **Dual‑log** write (primary + mirror) and **cross‑log proof** packaging.
* **Cloud endorsement**: send `{uuid, artifactSha256}` to Stella Ops cloud; store returned endorsement id for marketing/chain‑of‑custody. * **Cloud endorsement**: send `{uuid, artifactSha256}` to Stella Ops cloud; store returned endorsement id for marketing/chain‑of‑custody.
* **Checkpoint pinning**: periodically pin latest Rekor checkpoints to an external audit store for independent monitoring. * **Checkpoint pinning**: periodically pin latest Rekor checkpoints to an external audit store for independent monitoring.

148
docs/modules/attestor/implementation_plan.md
View File

@@ -1,74 +1,74 @@
# Implementation plan — Attestor # Implementation plan — Attestor
## Delivery phases ## Delivery phases
- **Phase 1 – Foundations** - **Phase 1 – Foundations**
Build the Attestor service skeleton, DSSE bundle ingestion, mTLS/OpTok enforcement, Rekor v2 client, and cache the `{uuid,index,proof}` tuple. Publish base API (`POST /rekor/entries`, `GET /entries/{uuid}`) and Mongo schemas. Build the Attestor service skeleton, DSSE bundle ingestion, mTLS/OpTok enforcement, Rekor v2 client, and cache the `{uuid,index,proof}` tuple. Publish base API (`POST /rekor/entries`, `GET /entries/{uuid}`) and Mongo schemas.
- **Phase 2 – Policies & UI** - **Phase 2 – Policies & UI**
Deliver verification policy authoring (Policy Studio integration), console views (evidence browser, verification reports, issuer management), and CLI verbs (`stella attest sign|verify|list|fetch`). Deliver verification policy authoring (Policy Studio integration), console views (evidence browser, verification reports, issuer management), and CLI verbs (`stella attest sign|verify|list|fetch`).
- **Phase 3 – Scan & VEX support** - **Phase 3 – Scan & VEX support**
Accept SBOM, ScanResults, VEX, and PolicyEvaluation predicates; integrate with Scanner, Export Center, Excititor, and Policy Engine pipelines. Ensure AOC invariants on ingestion. Accept SBOM, ScanResults, VEX, and PolicyEvaluation predicates; integrate with Scanner, Export Center, Excititor, and Policy Engine pipelines. Ensure AOC invariants on ingestion.
- **Phase 4 – Transparency & keys** - **Phase 4 – Transparency & keys**
Add multi-log submission (primary + mirror), witness endorsements, KMS/HSM/FIDO2 drivers, key rotation/revocation workflows, and audit trails. Add multi-log submission (primary + mirror), witness endorsements, KMS/HSM/FIDO2 drivers, key rotation/revocation workflows, and audit trails.
- **Phase 5 – Bulk & air gap** - **Phase 5 – Bulk & air gap**
Implement batch submission/verification, DSSE archival to CAS/object storage, export/import bundles for Offline Kit, and mirror transparency log snapshots. Implement batch submission/verification, DSSE archival to CAS/object storage, export/import bundles for Offline Kit, and mirror transparency log snapshots.
- **Phase 6 – Performance & hardening** - **Phase 6 – Performance & hardening**
Optimise cache usage, parallel verification (target ≥1 k envelopes/minute per worker), extend observability (metrics/logs/traces), fuzz parsers, and finalise incident playbooks. Optimise cache usage, parallel verification (target ≥1 k envelopes/minute per worker), extend observability (metrics/logs/traces), fuzz parsers, and finalise incident playbooks.
## Work breakdown ## Work breakdown
- **Attestor service & libraries** - **Attestor service & libraries**
- DSSE validation pipeline (payload whitelist, signature verification, trust roots). - DSSE validation pipeline (payload whitelist, signature verification, trust roots).
- Rekor client with inclusion-proof acquisition, retry/backoff, mirroring controls. - Rekor client with inclusion-proof acquisition, retry/backoff, mirroring controls.
- Mongo repositories for entries, dedupe, audit; CAS storage for DSSE envelopes. - Mongo repositories for entries, dedupe, audit; CAS storage for DSSE envelopes.
- Batch submission/verification APIs, verification cache, deterministic serialization. - Batch submission/verification APIs, verification cache, deterministic serialization.
- Observability hooks: metrics (`attestor_submission_total`, `attestor_verify_seconds`), structured logs, OpenTelemetry traces. - Observability hooks: metrics (`attestor_submission_total`, `attestor_verify_seconds`), structured logs, OpenTelemetry traces.
- **Signer & Authority integration** - **Signer & Authority integration**
- Enforce mTLS peer validation, Authority scope mapping (`attestor.write`, `attestor.verify`), and DPoP binding. - Enforce mTLS peer validation, Authority scope mapping (`attestor.write`, `attestor.verify`), and DPoP binding.
- Provide signer identity attestation metadata consumed by Attestor. - Provide signer identity attestation metadata consumed by Attestor.
- **Policy & Console** - **Policy & Console**
- Extend Policy Studio with `VerificationPolicy` authoring, approvals, and simulated results. - Extend Policy Studio with `VerificationPolicy` authoring, approvals, and simulated results.
- Console workflows: Evidence browser, verification reports, chain-of-custody graph, key management UI, bulk verification screens. - Console workflows: Evidence browser, verification reports, chain-of-custody graph, key management UI, bulk verification screens.
- **CLI & SDK** - **CLI & SDK**
- `stella attest` command group (sign/verify/list/fetch/key management) with DSSE canonicalisation and cosign interoperability. - `stella attest` command group (sign/verify/list/fetch/key management) with DSSE canonicalisation and cosign interoperability.
- SDK helpers for DSSE envelope creation, verification, and proof inspection. - SDK helpers for DSSE envelope creation, verification, and proof inspection.
- **Export Center & Offline Kit** - **Export Center & Offline Kit**
- Export Center adapters for attestation bundles; CLI/Console flows to export & import evidence in air-gapped environments. - Export Center adapters for attestation bundles; CLI/Console flows to export & import evidence in air-gapped environments.
- Offline Kit scripts for replaying verification, mirroring transparency logs, and reporting gaps. - Offline Kit scripts for replaying verification, mirroring transparency logs, and reporting gaps.
- **Security & key management** - **Security & key management**
- KMS/HSM/FIDO2 driver abstraction, key rotation and revocation runbooks, witness endorsements, and revocation telemetry. - KMS/HSM/FIDO2 driver abstraction, key rotation and revocation runbooks, witness endorsements, and revocation telemetry.
- **Docs & training** - **Docs & training**
- Update module dossier (overview, architecture, implementation plan), key management guides, transparency reference, CLI/Console documentation, and air-gap runbooks. - Update module dossier (overview, architecture, implementation plan), key management guides, transparency reference, CLI/Console documentation, and air-gap runbooks.
## Cross-module dependencies ## Cross-module dependencies
- **Policy Studio / Policy Engine:** verification policy artefacts, explain integration, remediation hints. - **Policy Studio / Policy Engine:** verification policy artefacts, explain integration, remediation hints.
- **Export Center:** attestation bundle export/import, provenance linking. - **Export Center:** attestation bundle export/import, provenance linking.
- **Authority & Tenancy:** scopes, identity attestations, tenant-aware issuer catalogues. - **Authority & Tenancy:** scopes, identity attestations, tenant-aware issuer catalogues.
- **Notifications:** attestation success/failure events, key rotation alerts. - **Notifications:** attestation success/failure events, key rotation alerts.
- **Observability:** dashboards and alerting for signing/verification pipelines. - **Observability:** dashboards and alerting for signing/verification pipelines.
## Acceptance criteria ## Acceptance criteria
- Service ingests DSSE envelopes for all supported predicate types, logs them to configured transparency logs, and returns proofs with deterministic hashes. - Service ingests DSSE envelopes for all supported predicate types, logs them to configured transparency logs, and returns proofs with deterministic hashes.
- Verification APIs/CLI/UI validate signatures, inclusion proofs, and policy compliance; cached verification accelerates repeated checks. - Verification APIs/CLI/UI validate signatures, inclusion proofs, and policy compliance; cached verification accelerates repeated checks.
- Verification policies gate attestation usage, enforcing issuer, freshness, signature count, and witness requirements. - Verification policies gate attestation usage, enforcing issuer, freshness, signature count, and witness requirements.
- Export Center and Offline Kit workflows bundle attestations and replay verification offline. - Export Center and Offline Kit workflows bundle attestations and replay verification offline.
- Observability coverage includes metrics, traces, logs, audit events, and alert triggers for key compromise, log outages, and verification failure spikes. - Observability coverage includes metrics, traces, logs, audit events, and alert triggers for key compromise, log outages, and verification failure spikes.
- Performance target met (≥1 k envelopes/minute per worker) with horizontal scaling. - Performance target met (≥1 k envelopes/minute per worker) with horizontal scaling.
## Risks & mitigations ## Risks & mitigations
- **Key compromise or leakage:** enforce hardware-backed keys, rotation procedures, revocation checks, and incident runbooks. - **Key compromise or leakage:** enforce hardware-backed keys, rotation procedures, revocation checks, and incident runbooks.
- **Parser bugs / malformed DSSE:** fuzz DSSE and predicate schemas, strict schema validation, fail closed. - **Parser bugs / malformed DSSE:** fuzz DSSE and predicate schemas, strict schema validation, fail closed.
- **Transparency outage:** mirror logs, support witness endorsements, queue submissions for retry with exponential backoff. - **Transparency outage:** mirror logs, support witness endorsements, queue submissions for retry with exponential backoff.
- **Policy complexity:** ship curated starter policies, provide simulation tooling, and document common scenarios. - **Policy complexity:** ship curated starter policies, provide simulation tooling, and document common scenarios.
- **Offline gaps:** archive bundles and proof material, surface gaps to operators, and document compensating controls. - **Offline gaps:** archive bundles and proof material, surface gaps to operators, and document compensating controls.
## Test strategy ## Test strategy
- **Unit:** DSSE validation, Rekor client, dedupe logic, key drivers, policy enforcement. - **Unit:** DSSE validation, Rekor client, dedupe logic, key drivers, policy enforcement.
- **Integration:** submit/verify flows across predicate types, multi-log publishing, batch operations, CLI/UI end-to-end exercises. - **Integration:** submit/verify flows across predicate types, multi-log publishing, batch operations, CLI/UI end-to-end exercises.
- **Security:** tenant isolation, scope enforcement, key rotation regression, tamper detection. - **Security:** tenant isolation, scope enforcement, key rotation regression, tamper detection.
- **Performance:** throughput benchmarks, cache hit-rate monitoring, large batch verification. - **Performance:** throughput benchmarks, cache hit-rate monitoring, large batch verification.
- **Chaos:** inject Rekor outages, network failures, corrupt bundles; ensure graceful degradation and auditable alerts. - **Chaos:** inject Rekor outages, network failures, corrupt bundles; ensure graceful degradation and auditable alerts.
## Definition of done ## Definition of done
- Phased milestones delivered with telemetry, documentation, and runbooks in place. - Phased milestones delivered with telemetry, documentation, and runbooks in place.
- CLI/Console parity verified; Offline Kit procedures validated in sealed environment. - CLI/Console parity verified; Offline Kit procedures validated in sealed environment.
- Cross-module dependencies acknowledged in ./TASKS.md and ../../TASKS.md. - Cross-module dependencies acknowledged in ./TASKS.md and ../../TASKS.md.
- Documentation set refreshed (overview, architecture, key management, transparency, CLI/UI) with imposed rule statement. - Documentation set refreshed (overview, architecture, key management, transparency, CLI/UI) with imposed rule statement.

42
docs/modules/authority/AGENTS.md
View File

@@ -1,22 +1,22 @@
# Authority agent guide # Authority agent guide
## Mission ## Mission
Authority is the platform OIDC/OAuth2 control plane that mints short-lived, sender-constrained operational tokens (OpToks) for every StellaOps service and tool. Authority is the platform OIDC/OAuth2 control plane that mints short-lived, sender-constrained operational tokens (OpToks) for every StellaOps service and tool.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

80
docs/modules/authority/README.md
View File

@@ -1,40 +1,40 @@
# StellaOps Authority # StellaOps Authority
Authority is the platform OIDC/OAuth2 control plane that mints short-lived, sender-constrained operational tokens (OpToks) for every StellaOps service and tool. Authority is the platform OIDC/OAuth2 control plane that mints short-lived, sender-constrained operational tokens (OpToks) for every StellaOps service and tool.
## Responsibilities ## Responsibilities
- Expose device-code, auth-code, and client-credential flows with DPoP or mTLS binding. - Expose device-code, auth-code, and client-credential flows with DPoP or mTLS binding.
- Manage signing keys, JWKS rotation, and PoE integration for plan enforcement. - Manage signing keys, JWKS rotation, and PoE integration for plan enforcement.
- Emit structured audit events and enforce tenant-aware scope policies. - Emit structured audit events and enforce tenant-aware scope policies.
- Provide plugin surface for custom identity providers and credential validators. - Provide plugin surface for custom identity providers and credential validators.
## Key components ## Key components
- `StellaOps.Authority` web host. - `StellaOps.Authority` web host.
- `StellaOps.Authority.Plugin.*` extensions for secret stores, identity bridges, and OpTok validation. - `StellaOps.Authority.Plugin.*` extensions for secret stores, identity bridges, and OpTok validation.
- Telemetry and audit pipeline feeding Security/Observability stacks. - Telemetry and audit pipeline feeding Security/Observability stacks.
## Integrations & dependencies ## Integrations & dependencies
- Signer/Attestor for PoE and OpTok introspection. - Signer/Attestor for PoE and OpTok introspection.
- CLI/UI for login flows and token management. - CLI/UI for login flows and token management.
- Scheduler/Scanner for machine-to-machine scope enforcement. - Scheduler/Scanner for machine-to-machine scope enforcement.
## Operational notes ## Operational notes
- MongoDB for tenant, client, and token state. - MongoDB for tenant, client, and token state.
- Key material in KMS/HSM with rotation runbooks (see ./operations/key-rotation.md). - Key material in KMS/HSM with rotation runbooks (see ./operations/key-rotation.md).
- Grafana/Prometheus dashboards for auth latency/issuance. - Grafana/Prometheus dashboards for auth latency/issuance.
## Related resources ## Related resources
- ./operations/backup-restore.md - ./operations/backup-restore.md
- ./operations/key-rotation.md - ./operations/key-rotation.md
- ./operations/monitoring.md - ./operations/monitoring.md
- ./operations/grafana-dashboard.json - ./operations/grafana-dashboard.json
## Backlog references ## Backlog references
- DOCS-SEC-62-001 (scope hardening doc) in ../../TASKS.md. - DOCS-SEC-62-001 (scope hardening doc) in ../../TASKS.md.
- AUTH-POLICY-20-001/002 follow-ups in src/Authority/StellaOps.Authority/TASKS.md. - AUTH-POLICY-20-001/002 follow-ups in src/Authority/StellaOps.Authority/TASKS.md.
## Epic alignment ## Epic alignment
- **Epic 1 – AOC enforcement:** enforce OpTok scopes and guardrails supporting raw ingestion boundaries. - **Epic 1 – AOC enforcement:** enforce OpTok scopes and guardrails supporting raw ingestion boundaries.
- **Epic 2 – Policy Engine & Editor:** supply policy evaluation/principal scopes and short-lived tokens for evaluator workflows. - **Epic 2 – Policy Engine & Editor:** supply policy evaluation/principal scopes and short-lived tokens for evaluator workflows.
- **Epic 4 – Policy Studio:** integrate approval/promotion signatures and policy registry access controls. - **Epic 4 – Policy Studio:** integrate approval/promotion signatures and policy registry access controls.
- **Epic 14 – Identity & Tenancy:** deliver tenant isolation, RBAC hierarchies, and governance tooling for authentication. - **Epic 14 – Identity & Tenancy:** deliver tenant isolation, RBAC hierarchies, and governance tooling for authentication.

18
docs/modules/authority/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — Authority # Task board — Authority
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| AUTHORITY-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | AUTHORITY-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| AUTHORITY-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | AUTHORITY-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| AUTHORITY-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | AUTHORITY-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

44
docs/modules/authority/implementation_plan.md
View File

@@ -1,22 +1,22 @@
# Implementation plan — Authority # Implementation plan — Authority
## Current objectives ## Current objectives
- Maintain deterministic behaviour and offline parity across releases. - Maintain deterministic behaviour and offline parity across releases.
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes. - Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
## Workstreams ## Workstreams
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap. - Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs. - Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
- Validation: extend tests/fixtures to preserve determinism and provenance requirements. - Validation: extend tests/fixtures to preserve determinism and provenance requirements.
## Epic milestones ## Epic milestones
- **Epic 1 – AOC enforcement:** deliver OpTok scopes, guardrails, and AOC verifier hooks for ingestion services. - **Epic 1 – AOC enforcement:** deliver OpTok scopes, guardrails, and AOC verifier hooks for ingestion services.
- **Epic 2 – Policy Engine & Editor:** support policy evaluator flows (device-code, client credentials, scope sandboxing). - **Epic 2 – Policy Engine & Editor:** support policy evaluator flows (device-code, client credentials, scope sandboxing).
- **Epic 4 – Policy Studio:** provide registry/promotion signing, approvals, and fresh-auth prompts. - **Epic 4 – Policy Studio:** provide registry/promotion signing, approvals, and fresh-auth prompts.
- **Epic 14 – Identity & Tenancy:** implement tenant isolation, RBAC hierarchies, audit trails, and PoE integration. - **Epic 14 – Identity & Tenancy:** implement tenant isolation, RBAC hierarchies, audit trails, and PoE integration.
- Track additional work (DOCS-SEC-62-001, AUTH-POLICY-20-001/002) in ../../TASKS.md and src/Authority/**/TASKS.md. - Track additional work (DOCS-SEC-62-001, AUTH-POLICY-20-001/002) in ../../TASKS.md and src/Authority/**/TASKS.md.
## Coordination ## Coordination
- Review ./AGENTS.md before picking up new work. - Review ./AGENTS.md before picking up new work.
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md. - Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
- Update this plan whenever scope, dependencies, or guardrails change. - Update this plan whenever scope, dependencies, or guardrails change.

190
docs/modules/authority/operations/backup-restore.md
View File

@@ -1,97 +1,97 @@
# Authority Backup & Restore Runbook # Authority Backup & Restore Runbook
## Scope ## Scope
- **Applies to:** StellaOps Authority deployments running the official `ops/authority/docker-compose.authority.yaml` stack or equivalent Kubernetes packaging. - **Applies to:** StellaOps Authority deployments running the official `ops/authority/docker-compose.authority.yaml` stack or equivalent Kubernetes packaging.
- **Artifacts covered:** MongoDB (`stellaops-authority` database), Authority configuration (`etc/authority.yaml`), plugin manifests under `etc/authority.plugins/`, and signing key material stored in the `authority-keys` volume (defaults to `/app/keys` inside the container). - **Artifacts covered:** MongoDB (`stellaops-authority` database), Authority configuration (`etc/authority.yaml`), plugin manifests under `etc/authority.plugins/`, and signing key material stored in the `authority-keys` volume (defaults to `/app/keys` inside the container).
- **Frequency:** Run the full procedure prior to upgrades, before rotating keys, and at least once per 24 h in production. Store snapshots in an encrypted, access-controlled vault. - **Frequency:** Run the full procedure prior to upgrades, before rotating keys, and at least once per 24 h in production. Store snapshots in an encrypted, access-controlled vault.
## Inventory Checklist ## Inventory Checklist
| Component | Location (compose default) | Notes | | Component | Location (compose default) | Notes |
| --- | --- | --- | | --- | --- | --- |
| Mongo data | `mongo-data` volume (`/var/lib/docker/volumes/.../mongo-data`) | Contains all Authority collections (`AuthorityUser`, `AuthorityClient`, `AuthorityToken`, etc.). | | Mongo data | `mongo-data` volume (`/var/lib/docker/volumes/.../mongo-data`) | Contains all Authority collections (`AuthorityUser`, `AuthorityClient`, `AuthorityToken`, etc.). |
| Configuration | `etc/authority.yaml` | Mounted read-only into the container at `/etc/authority.yaml`. | | Configuration | `etc/authority.yaml` | Mounted read-only into the container at `/etc/authority.yaml`. |
| Plugin manifests | `etc/authority.plugins/*.yaml` | Includes `standard.yaml` with `tokenSigning.keyDirectory`. | | Plugin manifests | `etc/authority.plugins/*.yaml` | Includes `standard.yaml` with `tokenSigning.keyDirectory`. |
| Signing keys | `authority-keys` volume -> `/app/keys` | Path is derived from `tokenSigning.keyDirectory` (defaults to `../keys` relative to the manifest). | | Signing keys | `authority-keys` volume -> `/app/keys` | Path is derived from `tokenSigning.keyDirectory` (defaults to `../keys` relative to the manifest). |
> **TIP:** Confirm the deployed key directory via `tokenSigning.keyDirectory` in `etc/authority.plugins/standard.yaml`; some installations relocate keys to `/var/lib/stellaops/authority/keys`. > **TIP:** Confirm the deployed key directory via `tokenSigning.keyDirectory` in `etc/authority.plugins/standard.yaml`; some installations relocate keys to `/var/lib/stellaops/authority/keys`.
## Hot Backup (no downtime) ## Hot Backup (no downtime)
1. **Create output directory:** `mkdir -p backup/$(date +%Y-%m-%d)` on the host. 1. **Create output directory:** `mkdir -p backup/$(date +%Y-%m-%d)` on the host.
2. **Dump Mongo:** 2. **Dump Mongo:**
```bash ```bash
docker compose -f ops/authority/docker-compose.authority.yaml exec mongo \ docker compose -f ops/authority/docker-compose.authority.yaml exec mongo \
mongodump --archive=/dump/authority-$(date +%Y%m%dT%H%M%SZ).gz \ mongodump --archive=/dump/authority-$(date +%Y%m%dT%H%M%SZ).gz \
--gzip --db stellaops-authority --gzip --db stellaops-authority
docker compose -f ops/authority/docker-compose.authority.yaml cp \ docker compose -f ops/authority/docker-compose.authority.yaml cp \
mongo:/dump/authority-$(date +%Y%m%dT%H%M%SZ).gz backup/ mongo:/dump/authority-$(date +%Y%m%dT%H%M%SZ).gz backup/
``` ```
The `mongodump` archive preserves indexes and can be restored with `mongorestore --archive --gzip`. The `mongodump` archive preserves indexes and can be restored with `mongorestore --archive --gzip`.
3. **Capture configuration + manifests:** 3. **Capture configuration + manifests:**
```bash ```bash
cp etc/authority.yaml backup/ cp etc/authority.yaml backup/
rsync -a etc/authority.plugins/ backup/authority.plugins/ rsync -a etc/authority.plugins/ backup/authority.plugins/
``` ```
4. **Export signing keys:** the compose file maps `authority-keys` to a local Docker volume. Snapshot it without stopping the service: 4. **Export signing keys:** the compose file maps `authority-keys` to a local Docker volume. Snapshot it without stopping the service:
```bash ```bash
docker run --rm \ docker run --rm \
-v authority-keys:/keys \ -v authority-keys:/keys \
-v "$(pwd)/backup:/backup" \ -v "$(pwd)/backup:/backup" \
busybox tar czf /backup/authority-keys-$(date +%Y%m%dT%H%M%SZ).tar.gz -C /keys . busybox tar czf /backup/authority-keys-$(date +%Y%m%dT%H%M%SZ).tar.gz -C /keys .
``` ```
5. **Checksum:** generate SHA-256 digests for every file and store them alongside the artefacts. 5. **Checksum:** generate SHA-256 digests for every file and store them alongside the artefacts.
6. **Encrypt & upload:** wrap the backup folder using your secrets management standard (e.g., age, GPG) and upload to the designated offline vault. 6. **Encrypt & upload:** wrap the backup folder using your secrets management standard (e.g., age, GPG) and upload to the designated offline vault.
## Cold Backup (planned downtime) ## Cold Backup (planned downtime)
1. Notify stakeholders and drain traffic (CLI clients should refresh tokens afterwards). 1. Notify stakeholders and drain traffic (CLI clients should refresh tokens afterwards).
2. Stop services: 2. Stop services:
```bash ```bash
docker compose -f ops/authority/docker-compose.authority.yaml down docker compose -f ops/authority/docker-compose.authority.yaml down
``` ```
3. Back up volumes directly using `tar`: 3. Back up volumes directly using `tar`:
```bash ```bash
docker run --rm -v mongo-data:/data -v "$(pwd)/backup:/backup" \ docker run --rm -v mongo-data:/data -v "$(pwd)/backup:/backup" \
busybox tar czf /backup/mongo-data-$(date +%Y%m%d).tar.gz -C /data . busybox tar czf /backup/mongo-data-$(date +%Y%m%d).tar.gz -C /data .
docker run --rm -v authority-keys:/keys -v "$(pwd)/backup:/backup" \ docker run --rm -v authority-keys:/keys -v "$(pwd)/backup:/backup" \
busybox tar czf /backup/authority-keys-$(date +%Y%m%d).tar.gz -C /keys . busybox tar czf /backup/authority-keys-$(date +%Y%m%d).tar.gz -C /keys .
``` ```
4. Copy configuration + manifests as in the hot backup (steps 3–6). 4. Copy configuration + manifests as in the hot backup (steps 3–6).
5. Restart services and verify health: 5. Restart services and verify health:
```bash ```bash
docker compose -f ops/authority/docker-compose.authority.yaml up -d docker compose -f ops/authority/docker-compose.authority.yaml up -d
curl -fsS http://localhost:8080/ready curl -fsS http://localhost:8080/ready
``` ```
## Restore Procedure ## Restore Procedure
1. **Provision clean volumes:** remove existing volumes if you’re rebuilding a node (`docker volume rm mongo-data authority-keys`), then recreate the compose stack so empty volumes exist. 1. **Provision clean volumes:** remove existing volumes if you’re rebuilding a node (`docker volume rm mongo-data authority-keys`), then recreate the compose stack so empty volumes exist.
2. **Restore Mongo:** 2. **Restore Mongo:**
```bash ```bash
docker compose exec -T mongo mongorestore --archive --gzip --drop < backup/authority-YYYYMMDDTHHMMSSZ.gz docker compose exec -T mongo mongorestore --archive --gzip --drop < backup/authority-YYYYMMDDTHHMMSSZ.gz
``` ```
Use `--drop` to replace collections; omit if doing a partial restore. Use `--drop` to replace collections; omit if doing a partial restore.
3. **Restore configuration/manifests:** copy `authority.yaml` and `authority.plugins/*` into place before starting the Authority container. 3. **Restore configuration/manifests:** copy `authority.yaml` and `authority.plugins/*` into place before starting the Authority container.
4. **Restore signing keys:** untar into the mounted volume: 4. **Restore signing keys:** untar into the mounted volume:
```bash ```bash
docker run --rm -v authority-keys:/keys -v "$(pwd)/backup:/backup" \ docker run --rm -v authority-keys:/keys -v "$(pwd)/backup:/backup" \
busybox tar xzf /backup/authority-keys-YYYYMMDD.tar.gz -C /keys busybox tar xzf /backup/authority-keys-YYYYMMDD.tar.gz -C /keys
``` ```
Ensure file permissions remain `600` for private keys (`chmod -R 600`). Ensure file permissions remain `600` for private keys (`chmod -R 600`).
5. **Start services & validate:** 5. **Start services & validate:**
```bash ```bash
docker compose up -d docker compose up -d
curl -fsS http://localhost:8080/health curl -fsS http://localhost:8080/health
``` ```
6. **Validate JWKS and tokens:** call `/jwks` and issue a short-lived token via the CLI to confirm key material matches expectations. If the restored environment requires a fresh signing key, follow the rotation SOP in [`docs/11_AUTHORITY.md`](../../../11_AUTHORITY.md) using `ops/authority/key-rotation.sh` to invoke `/internal/signing/rotate`. 6. **Validate JWKS and tokens:** call `/jwks` and issue a short-lived token via the CLI to confirm key material matches expectations. If the restored environment requires a fresh signing key, follow the rotation SOP in [`docs/11_AUTHORITY.md`](../../../11_AUTHORITY.md) using `ops/authority/key-rotation.sh` to invoke `/internal/signing/rotate`.
## Disaster Recovery Notes ## Disaster Recovery Notes
- **Air-gapped replication:** replicate archives via the Offline Update Kit transport channels; never attach USB devices without scanning. - **Air-gapped replication:** replicate archives via the Offline Update Kit transport channels; never attach USB devices without scanning.
- **Retention:** maintain 30 daily snapshots + 12 monthly archival copies. Rotate encryption keys annually. - **Retention:** maintain 30 daily snapshots + 12 monthly archival copies. Rotate encryption keys annually.
- **Key compromise:** if signing keys are suspected compromised, restore from the latest clean backup, rotate via OPS3 (see `ops/authority/key-rotation.sh` and [`docs/11_AUTHORITY.md`](../../../11_AUTHORITY.md)), and publish a revocation notice. - **Key compromise:** if signing keys are suspected compromised, restore from the latest clean backup, rotate via OPS3 (see `ops/authority/key-rotation.sh` and [`docs/11_AUTHORITY.md`](../../../11_AUTHORITY.md)), and publish a revocation notice.
- **Mongo version:** keep dump/restore images pinned to the deployment version (compose uses `mongo:7`). Driver 3.5.0 requires MongoDB **4.2+**—clusters still on 4.0 must be upgraded before restore, and future driver releases will drop 4.0 entirely. citeturn1open1 - **Mongo version:** keep dump/restore images pinned to the deployment version (compose uses `mongo:7`). Driver 3.5.0 requires MongoDB **4.2+**—clusters still on 4.0 must be upgraded before restore, and future driver releases will drop 4.0 entirely. citeturn1open1
## Verification Checklist ## Verification Checklist
- [ ] `/ready` reports all identity providers ready. - [ ] `/ready` reports all identity providers ready.
- [ ] OAuth flows issue tokens signed by the restored keys. - [ ] OAuth flows issue tokens signed by the restored keys.
- [ ] `PluginRegistrationSummary` logs expected providers on startup. - [ ] `PluginRegistrationSummary` logs expected providers on startup.
- [ ] Revocation manifest export (`dotnet run --project src/Authority/StellaOps.Authority`) succeeds. - [ ] Revocation manifest export (`dotnet run --project src/Authority/StellaOps.Authority`) succeeds.
- [ ] Monitoring dashboards show metrics resuming (see OPS5 deliverables). - [ ] Monitoring dashboards show metrics resuming (see OPS5 deliverables).

348
docs/modules/authority/operations/grafana-dashboard.json
View File

@@ -1,174 +1,174 @@
{ {
"title": "StellaOps Authority - Token & Access Monitoring", "title": "StellaOps Authority - Token & Access Monitoring",
"uid": "authority-token-monitoring", "uid": "authority-token-monitoring",
"schemaVersion": 38, "schemaVersion": 38,
"version": 1, "version": 1,
"editable": true, "editable": true,
"timezone": "", "timezone": "",
"graphTooltip": 0, "graphTooltip": 0,
"time": { "time": {
"from": "now-6h", "from": "now-6h",
"to": "now" "to": "now"
}, },
"templating": { "templating": {
"list": [ "list": [
{ {
"name": "datasource", "name": "datasource",
"type": "datasource", "type": "datasource",
"query": "prometheus", "query": "prometheus",
"refresh": 1, "refresh": 1,
"hide": 0, "hide": 0,
"current": {} "current": {}
} }
] ]
}, },
"panels": [ "panels": [
{ {
"id": 1, "id": 1,
"title": "Token Requests – Success vs Failure", "title": "Token Requests – Success vs Failure",
"type": "timeseries", "type": "timeseries",
"datasource": { "datasource": {
"type": "prometheus", "type": "prometheus",
"uid": "${datasource}" "uid": "${datasource}"
}, },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
"unit": "req/s", "unit": "req/s",
"displayName": "{{grant_type}} ({{status}})" "displayName": "{{grant_type}} ({{status}})"
}, },
"overrides": [] "overrides": []
}, },
"targets": [ "targets": [
{ {
"refId": "A", "refId": "A",
"expr": "sum by (grant_type, status) (rate(http_server_duration_seconds_count{service_name=\"stellaops-authority\", http_route=\"/token\"}[5m]))", "expr": "sum by (grant_type, status) (rate(http_server_duration_seconds_count{service_name=\"stellaops-authority\", http_route=\"/token\"}[5m]))",
"legendFormat": "{{grant_type}} {{status}}" "legendFormat": "{{grant_type}} {{status}}"
} }
], ],
"options": { "options": {
"legend": { "legend": {
"displayMode": "table", "displayMode": "table",
"placement": "bottom" "placement": "bottom"
}, },
"tooltip": { "tooltip": {
"mode": "multi" "mode": "multi"
} }
} }
}, },
{ {
"id": 2, "id": 2,
"title": "Rate Limiter Rejections", "title": "Rate Limiter Rejections",
"type": "timeseries", "type": "timeseries",
"datasource": { "datasource": {
"type": "prometheus", "type": "prometheus",
"uid": "${datasource}" "uid": "${datasource}"
}, },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
"unit": "req/s", "unit": "req/s",
"displayName": "{{limiter}}" "displayName": "{{limiter}}"
}, },
"overrides": [] "overrides": []
}, },
"targets": [ "targets": [
{ {
"refId": "A", "refId": "A",
"expr": "sum by (limiter) (rate(aspnetcore_rate_limiting_rejections_total{service_name=\"stellaops-authority\"}[5m]))", "expr": "sum by (limiter) (rate(aspnetcore_rate_limiting_rejections_total{service_name=\"stellaops-authority\"}[5m]))",
"legendFormat": "{{limiter}}" "legendFormat": "{{limiter}}"
} }
] ]
}, },
{ {
"id": 3, "id": 3,
"title": "Bypass Events (5m)", "title": "Bypass Events (5m)",
"type": "stat", "type": "stat",
"datasource": { "datasource": {
"type": "prometheus", "type": "prometheus",
"uid": "${datasource}" "uid": "${datasource}"
}, },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
"unit": "short", "unit": "short",
"color": { "color": {
"mode": "thresholds" "mode": "thresholds"
}, },
"thresholds": { "thresholds": {
"mode": "absolute", "mode": "absolute",
"steps": [ "steps": [
{ "color": "green", "value": null }, { "color": "green", "value": null },
{ "color": "orange", "value": 1 }, { "color": "orange", "value": 1 },
{ "color": "red", "value": 5 } { "color": "red", "value": 5 }
] ]
} }
}, },
"overrides": [] "overrides": []
}, },
"targets": [ "targets": [
{ {
"refId": "A", "refId": "A",
"expr": "sum(rate(log_messages_total{message_template=\"Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}.\"}[5m]))" "expr": "sum(rate(log_messages_total{message_template=\"Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}.\"}[5m]))"
} }
], ],
"options": { "options": {
"reduceOptions": { "reduceOptions": {
"calcs": ["last"], "calcs": ["last"],
"fields": "", "fields": "",
"values": false "values": false
}, },
"orientation": "horizontal", "orientation": "horizontal",
"textMode": "auto" "textMode": "auto"
} }
}, },
{ {
"id": 4, "id": 4,
"title": "Lockout Events (15m)", "title": "Lockout Events (15m)",
"type": "stat", "type": "stat",
"datasource": { "datasource": {
"type": "prometheus", "type": "prometheus",
"uid": "${datasource}" "uid": "${datasource}"
}, },
"fieldConfig": { "fieldConfig": {
"defaults": { "defaults": {
"unit": "short", "unit": "short",
"color": { "color": {
"mode": "thresholds" "mode": "thresholds"
}, },
"thresholds": { "thresholds": {
"mode": "absolute", "mode": "absolute",
"steps": [ "steps": [
{ "color": "green", "value": null }, { "color": "green", "value": null },
{ "color": "orange", "value": 5 }, { "color": "orange", "value": 5 },
{ "color": "red", "value": 10 } { "color": "red", "value": 10 }
] ]
} }
}, },
"overrides": [] "overrides": []
}, },
"targets": [ "targets": [
{ {
"refId": "A", "refId": "A",
"expr": "sum(rate(log_messages_total{message_template=\"Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter}).\"}[15m]))" "expr": "sum(rate(log_messages_total{message_template=\"Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter}).\"}[15m]))"
} }
], ],
"options": { "options": {
"reduceOptions": { "reduceOptions": {
"calcs": ["last"], "calcs": ["last"],
"fields": "", "fields": "",
"values": false "values": false
}, },
"orientation": "horizontal", "orientation": "horizontal",
"textMode": "auto" "textMode": "auto"
} }
}, },
{ {
"id": 5, "id": 5,
"title": "Trace Explorer Shortcut", "title": "Trace Explorer Shortcut",
"type": "text", "type": "text",
"options": { "options": {
"mode": "markdown", "mode": "markdown",
"content": "[Open Trace Explorer](#/explore?left={\"datasource\":\"tempo\",\"queries\":[{\"query\":\"{service.name=\\\"stellaops-authority\\\", span_name=~\\\"authority.token.*\\\"}\",\"refId\":\"A\"}]})" "content": "[Open Trace Explorer](#/explore?left={\"datasource\":\"tempo\",\"queries\":[{\"query\":\"{service.name=\\\"stellaops-authority\\\", span_name=~\\\"authority.token.*\\\"}\",\"refId\":\"A\"}]})"
} }
} }
], ],
"links": [] "links": []
} }

188
docs/modules/authority/operations/key-rotation.md
View File

@@ -1,94 +1,94 @@
# Authority Signing Key Rotation Playbook # Authority Signing Key Rotation Playbook
> **Status:** Authored 2025-10-12 as part of OPS3.KEY-ROTATION rollout. > **Status:** Authored 2025-10-12 as part of OPS3.KEY-ROTATION rollout.
> Use together with `docs/11_AUTHORITY.md` (Authority service guide) and the automation shipped under `ops/authority/`. > Use together with `docs/11_AUTHORITY.md` (Authority service guide) and the automation shipped under `ops/authority/`.
## 1. Overview ## 1. Overview
Authority publishes JWKS and revocation bundles signed with ES256 keys. To rotate those keys without downtime we now provide: Authority publishes JWKS and revocation bundles signed with ES256 keys. To rotate those keys without downtime we now provide:
- **Automation script:** `ops/authority/key-rotation.sh` - **Automation script:** `ops/authority/key-rotation.sh`
Shell helper that POSTS to `/internal/signing/rotate`, supports metadata, dry-run, and confirms JWKS afterwards. Shell helper that POSTS to `/internal/signing/rotate`, supports metadata, dry-run, and confirms JWKS afterwards.
- **CI workflow:** `.gitea/workflows/authority-key-rotation.yml` - **CI workflow:** `.gitea/workflows/authority-key-rotation.yml`
Manual dispatch workflow that pulls environment-specific secrets, runs the script, and records the result. Works across staging/production by passing the `environment` input. Manual dispatch workflow that pulls environment-specific secrets, runs the script, and records the result. Works across staging/production by passing the `environment` input.
This playbook documents the repeatable sequence for all environments. This playbook documents the repeatable sequence for all environments.
## 2. Pre-requisites ## 2. Pre-requisites
1. **Generate a new PEM key (per environment)** 1. **Generate a new PEM key (per environment)**
```bash ```bash
openssl ecparam -name prime256v1 -genkey -noout \ openssl ecparam -name prime256v1 -genkey -noout \
-out certificates/authority-signing-<env>-<year>.pem -out certificates/authority-signing-<env>-<year>.pem
chmod 600 certificates/authority-signing-<env>-<year>.pem chmod 600 certificates/authority-signing-<env>-<year>.pem
``` ```
2. **Stash the previous key** under the same volume so it can be referenced in `signing.additionalKeys` after rotation. 2. **Stash the previous key** under the same volume so it can be referenced in `signing.additionalKeys` after rotation.
3. **Ensure secrets/vars exist in Gitea** 3. **Ensure secrets/vars exist in Gitea**
- `<ENV>_AUTHORITY_BOOTSTRAP_KEY` - `<ENV>_AUTHORITY_BOOTSTRAP_KEY`
- `<ENV>_AUTHORITY_URL` - `<ENV>_AUTHORITY_URL`
- Optional shared defaults `AUTHORITY_BOOTSTRAP_KEY`, `AUTHORITY_URL`. - Optional shared defaults `AUTHORITY_BOOTSTRAP_KEY`, `AUTHORITY_URL`.
## 3. Executing the rotation ## 3. Executing the rotation
### Option A – via CI workflow (recommended) ### Option A – via CI workflow (recommended)
1. Navigate to **Actions → Authority Key Rotation**. 1. Navigate to **Actions → Authority Key Rotation**.
2. Provide inputs: 2. Provide inputs:
- `environment`: `staging`, `production`, etc. - `environment`: `staging`, `production`, etc.
- `key_id`: new `kid` (e.g. `authority-signing-2025-dev`). - `key_id`: new `kid` (e.g. `authority-signing-2025-dev`).
- `key_path`: path as seen by the Authority service (e.g. `../certificates/authority-signing-2025-dev.pem`). - `key_path`: path as seen by the Authority service (e.g. `../certificates/authority-signing-2025-dev.pem`).
- Optional `metadata`: comma-separated `key=value` pairs (for audit trails). - Optional `metadata`: comma-separated `key=value` pairs (for audit trails).
3. Trigger. The workflow: 3. Trigger. The workflow:
- Reads the bootstrap key/URL from secrets. - Reads the bootstrap key/URL from secrets.
- Runs `ops/authority/key-rotation.sh`. - Runs `ops/authority/key-rotation.sh`.
- Prints the JWKS response for verification. - Prints the JWKS response for verification.
### Option B – manual shell invocation ### Option B – manual shell invocation
```bash ```bash
AUTHORITY_BOOTSTRAP_KEY=$(cat /secure/authority-bootstrap.key) \ AUTHORITY_BOOTSTRAP_KEY=$(cat /secure/authority-bootstrap.key) \
./ops/authority/key-rotation.sh \ ./ops/authority/key-rotation.sh \
--authority-url https://authority.example.com \ --authority-url https://authority.example.com \
--key-id authority-signing-2025-dev \ --key-id authority-signing-2025-dev \
--key-path ../certificates/authority-signing-2025-dev.pem \ --key-path ../certificates/authority-signing-2025-dev.pem \
--meta rotatedBy=ops --meta changeTicket=OPS-1234 --meta rotatedBy=ops --meta changeTicket=OPS-1234
``` ```
Use `--dry-run` to inspect the payload before execution. Use `--dry-run` to inspect the payload before execution.
## 4. Post-rotation checklist ## 4. Post-rotation checklist
1. Update `authority.yaml` (or environment-specific overrides): 1. Update `authority.yaml` (or environment-specific overrides):
- Set `signing.activeKeyId` to the new key. - Set `signing.activeKeyId` to the new key.
- Set `signing.keyPath` to the new PEM. - Set `signing.keyPath` to the new PEM.
- Append the previous key into `signing.additionalKeys`. - Append the previous key into `signing.additionalKeys`.
- Ensure `keySource`/`provider` match the values passed to the script. - Ensure `keySource`/`provider` match the values passed to the script.
2. Run `stellaops-cli auth revoke export` so revocation bundles are re-signed with the new key. 2. Run `stellaops-cli auth revoke export` so revocation bundles are re-signed with the new key.
3. Confirm `/jwks` lists the new `kid` with `status: "active"` and the previous one as `retired`. 3. Confirm `/jwks` lists the new `kid` with `status: "active"` and the previous one as `retired`.
4. Archive the old key securely; keep it available until all tokens/bundles signed with it have expired. 4. Archive the old key securely; keep it available until all tokens/bundles signed with it have expired.
## 5. Development key state ## 5. Development key state
For the sample configuration (`etc/authority.yaml.sample`) we minted a placeholder dev key: For the sample configuration (`etc/authority.yaml.sample`) we minted a placeholder dev key:
- Active: `authority-signing-2025-dev` (`certificates/authority-signing-2025-dev.pem`) - Active: `authority-signing-2025-dev` (`certificates/authority-signing-2025-dev.pem`)
- Retired: `authority-signing-dev` - Retired: `authority-signing-dev`
Treat these as examples; real environments must maintain their own PEM material. Treat these as examples; real environments must maintain their own PEM material.
## 6. References ## 6. References
- `docs/11_AUTHORITY.md` – Architecture and rotation SOP (Section 5). - `docs/11_AUTHORITY.md` – Architecture and rotation SOP (Section 5).
- `docs/modules/authority/operations/backup-restore.md` – Recovery flow referencing this playbook. - `docs/modules/authority/operations/backup-restore.md` – Recovery flow referencing this playbook.
- `ops/authority/README.md` – CLI usage and examples. - `ops/authority/README.md` – CLI usage and examples.
- `scripts/rotate-policy-cli-secret.sh` – Helper to mint new `policy-cli` shared secrets when policy scope bundles change. - `scripts/rotate-policy-cli-secret.sh` – Helper to mint new `policy-cli` shared secrets when policy scope bundles change.
## 7. Appendix — Policy CLI secret rotation ## 7. Appendix — Policy CLI secret rotation
Scope migrations such as AUTH-POLICY-23-004 require issuing fresh credentials for the `policy-cli` client. Use the helper script committed with the repo to keep secrets deterministic across environments. Scope migrations such as AUTH-POLICY-23-004 require issuing fresh credentials for the `policy-cli` client. Use the helper script committed with the repo to keep secrets deterministic across environments.
```bash ```bash
./scripts/rotate-policy-cli-secret.sh --output etc/secrets/policy-cli.secret ./scripts/rotate-policy-cli-secret.sh --output etc/secrets/policy-cli.secret
``` ```
The script writes a timestamped header and a random secret into the target file. Use `--dry-run` when generating material for external secret stores. After updating secrets in staging/production, recycle the Authority pods and confirm the new client credentials work before the next release freeze. The script writes a timestamped header and a random secret into the target file. Use `--dry-run` when generating material for external secret stores. After updating secrets in staging/production, recycle the Authority pods and confirm the new client credentials work before the next release freeze.

166
docs/modules/authority/operations/monitoring.md
View File

@@ -1,83 +1,83 @@
# Authority Monitoring & Alerting Playbook # Authority Monitoring & Alerting Playbook
## Telemetry Sources ## Telemetry Sources
- **Traces:** Activity source `StellaOps.Authority` emits spans for every token flow (`authority.token.validate_*`, `authority.token.handle_*`, `authority.token.validate_access`). Key tags include `authority.endpoint`, `authority.grant_type`, `authority.username`, `authority.client_id`, and `authority.identity_provider`. - **Traces:** Activity source `StellaOps.Authority` emits spans for every token flow (`authority.token.validate_*`, `authority.token.handle_*`, `authority.token.validate_access`). Key tags include `authority.endpoint`, `authority.grant_type`, `authority.username`, `authority.client_id`, and `authority.identity_provider`.
- **Metrics:** OpenTelemetry instrumentation (`AddAspNetCoreInstrumentation`, `AddHttpClientInstrumentation`, custom meter `StellaOps.Authority`) exports: - **Metrics:** OpenTelemetry instrumentation (`AddAspNetCoreInstrumentation`, `AddHttpClientInstrumentation`, custom meter `StellaOps.Authority`) exports:
- `http.server.request.duration` histogram (`http_route`, `http_status_code`, `authority.endpoint` tag via `aspnetcore` enrichment). - `http.server.request.duration` histogram (`http_route`, `http_status_code`, `authority.endpoint` tag via `aspnetcore` enrichment).
- `process.runtime.gc.*`, `process.runtime.dotnet.*` (from `AddRuntimeInstrumentation`). - `process.runtime.gc.*`, `process.runtime.dotnet.*` (from `AddRuntimeInstrumentation`).
- **Logs:** Serilog writes structured events to stdout. Notable templates: - **Logs:** Serilog writes structured events to stdout. Notable templates:
- `"Password grant verification failed ..."` and `"Plugin {PluginName} denied access ... due to lockout"` (lockout spike detector). - `"Password grant verification failed ..."` and `"Plugin {PluginName} denied access ... due to lockout"` (lockout spike detector).
- `"Password grant validation failed for {Username}: provider '{Provider}' does not support MFA required for exception approvals."` (identifies users attempting `exceptions:approve` without MFA support; tie to fresh-auth errors). - `"Password grant validation failed for {Username}: provider '{Provider}' does not support MFA required for exception approvals."` (identifies users attempting `exceptions:approve` without MFA support; tie to fresh-auth errors).
- `"Client credentials validation failed for {ClientId}: exception scopes require tenant assignment."` (signals misconfigured exception service identities). - `"Client credentials validation failed for {ClientId}: exception scopes require tenant assignment."` (signals misconfigured exception service identities).
- `"Granting StellaOps bypass for remote {RemoteIp}"` (bypass usage). - `"Granting StellaOps bypass for remote {RemoteIp}"` (bypass usage).
- `"Rate limit exceeded for path {Path} from {RemoteIp}"` (limiter alerts). - `"Rate limit exceeded for path {Path} from {RemoteIp}"` (limiter alerts).
## Prometheus Metrics to Collect ## Prometheus Metrics to Collect
| Metric | Query | Purpose | | Metric | Query | Purpose |
| --- | --- | --- | | --- | --- | --- |
| `token_requests_total` | `sum by (grant_type, status) (rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Token issuance volume per grant type (`grant_type` comes via `authority.grant_type` span attribute → Exemplars in Grafana). | | `token_requests_total` | `sum by (grant_type, status) (rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Token issuance volume per grant type (`grant_type` comes via `authority.grant_type` span attribute → Exemplars in Grafana). |
| `token_failure_ratio` | `sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token", http_status_code=~"4..|5.."}[5m])) / sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Alert when > 5 % for 10 min. | | `token_failure_ratio` | `sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token", http_status_code=~"4..|5.."}[5m])) / sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Alert when > 5 % for 10 min. |
| `authorize_rate_limit_hits` | `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority", limiter="authority-token"}[5m]))` | Detect rate limiting saturations (requires OTEL ASP.NET rate limiter exporter). | | `authorize_rate_limit_hits` | `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority", limiter="authority-token"}[5m]))` | Detect rate limiting saturations (requires OTEL ASP.NET rate limiter exporter). |
| `lockout_events` | `sum by (plugin) (rate(log_messages_total{app="stellaops-authority", level="Warning", message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[5m]))` | Derived from Loki/Promtail log counter. | | `lockout_events` | `sum by (plugin) (rate(log_messages_total{app="stellaops-authority", level="Warning", message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[5m]))` | Derived from Loki/Promtail log counter. |
| `bypass_usage_total` | `sum(rate(log_messages_total{app="stellaops-authority", level="Information", message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m]))` | Track trusted bypass invocations. | | `bypass_usage_total` | `sum(rate(log_messages_total{app="stellaops-authority", level="Information", message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m]))` | Track trusted bypass invocations. |
> **Exporter note:** Enable `aspnetcore` meters (`dotnet-counters` name `Microsoft.AspNetCore.Hosting`), or configure the OpenTelemetry Collector `metrics` pipeline with `metric_statements` to remap histogram counts into the shown series. > **Exporter note:** Enable `aspnetcore` meters (`dotnet-counters` name `Microsoft.AspNetCore.Hosting`), or configure the OpenTelemetry Collector `metrics` pipeline with `metric_statements` to remap histogram counts into the shown series.
## Alert Rules ## Alert Rules
1. **Token Failure Surge** 1. **Token Failure Surge**
- _Expression_: `token_failure_ratio > 0.05` - _Expression_: `token_failure_ratio > 0.05`
- _For_: `10m` - _For_: `10m`
- _Labels_: `severity="critical"` - _Labels_: `severity="critical"`
- _Annotations_: Include `topk(5, sum by (authority_identity_provider) (increase(authority_token_rejections_total[10m])))` as diagnostic hint (requires span → metric transformation). - _Annotations_: Include `topk(5, sum by (authority_identity_provider) (increase(authority_token_rejections_total[10m])))` as diagnostic hint (requires span → metric transformation).
2. **Lockout Spike** 2. **Lockout Spike**
- _Expression_: `sum(rate(log_messages_total{message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[15m])) > 10` - _Expression_: `sum(rate(log_messages_total{message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[15m])) > 10`
- _For_: `15m` - _For_: `15m`
- Investigate credential stuffing; consider temporarily tightening `RateLimiting.Token`. - Investigate credential stuffing; consider temporarily tightening `RateLimiting.Token`.
3. **Bypass Threshold** 3. **Bypass Threshold**
- _Expression_: `sum(rate(log_messages_total{message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m])) > 1` - _Expression_: `sum(rate(log_messages_total{message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m])) > 1`
- _For_: `5m` - _For_: `5m`
- Alert severity `warning` — verify the calling host list. - Alert severity `warning` — verify the calling host list.
4. **Rate Limiter Saturation** 4. **Rate Limiter Saturation**
- _Expression_: `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority"}[5m])) > 0` - _Expression_: `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority"}[5m])) > 0`
- Escalate if sustained for 5 min; confirm trusted clients aren’t misconfigured. - Escalate if sustained for 5 min; confirm trusted clients aren’t misconfigured.
## Grafana Dashboard ## Grafana Dashboard
- Import `docs/modules/authority/operations/grafana-dashboard.json` to provision baseline panels: - Import `docs/modules/authority/operations/grafana-dashboard.json` to provision baseline panels:
- **Token Success vs Failure** – stacked rate visualization split by grant type. - **Token Success vs Failure** – stacked rate visualization split by grant type.
- **Rate Limiter Hits** – bar chart showing `authority-token` and `authority-authorize`. - **Rate Limiter Hits** – bar chart showing `authority-token` and `authority-authorize`.
- **Bypass & Lockout Events** – dual-stat panel using Loki-derived counters. - **Bypass & Lockout Events** – dual-stat panel using Loki-derived counters.
- **Trace Explorer Link** – panel links to `StellaOps.Authority` span search pre-filtered by `authority.grant_type`. - **Trace Explorer Link** – panel links to `StellaOps.Authority` span search pre-filtered by `authority.grant_type`.
## Collector Configuration Snippets ## Collector Configuration Snippets
```yaml ```yaml
receivers: receivers:
otlp: otlp:
protocols: protocols:
http: http:
exporters: exporters:
prometheus: prometheus:
endpoint: "0.0.0.0:9464" endpoint: "0.0.0.0:9464"
processors: processors:
batch: batch:
attributes/token_grant: attributes/token_grant:
actions: actions:
- key: grant_type - key: grant_type
action: upsert action: upsert
from_attribute: authority.grant_type from_attribute: authority.grant_type
service: service:
pipelines: pipelines:
metrics: metrics:
receivers: [otlp] receivers: [otlp]
processors: [attributes/token_grant, batch] processors: [attributes/token_grant, batch]
exporters: [prometheus] exporters: [prometheus]
logs: logs:
receivers: [otlp] receivers: [otlp]
processors: [batch] processors: [batch]
exporters: [loki] exporters: [loki]
``` ```
## Operational Checklist ## Operational Checklist
- [ ] Confirm `STELLAOPS_AUTHORITY__OBSERVABILITY__EXPORTERS` enables OTLP in production builds. - [ ] Confirm `STELLAOPS_AUTHORITY__OBSERVABILITY__EXPORTERS` enables OTLP in production builds.
- [ ] Ensure Promtail captures container stdout with Serilog structured formatting. - [ ] Ensure Promtail captures container stdout with Serilog structured formatting.
- [ ] Periodically validate alert noise by running load tests that trigger the rate limiter. - [ ] Periodically validate alert noise by running load tests that trigger the rate limiter.
- [ ] Include dashboard JSON in Offline Kit for air-gapped clusters; update version header when metrics change. - [ ] Include dashboard JSON in Offline Kit for air-gapped clusters; update version header when metrics change.

42
docs/modules/ci/AGENTS.md
View File

@@ -1,22 +1,22 @@
# CI Recipes agent guide # CI Recipes agent guide
## Mission ## Mission
CI module collects reproducible pipeline recipes for builds, tests, and release promotion across supported platforms. CI module collects reproducible pipeline recipes for builds, tests, and release promotion across supported platforms.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

58
docs/modules/ci/README.md
View File

@@ -1,29 +1,29 @@
# StellaOps CI Recipes # StellaOps CI Recipes
CI module collects reproducible pipeline recipes for builds, tests, and release promotion across supported platforms. CI module collects reproducible pipeline recipes for builds, tests, and release promotion across supported platforms.
## Responsibilities ## Responsibilities
- Provide ready-to-use pipeline snippets for ingestion, scanning, policy evaluation, and exports. - Provide ready-to-use pipeline snippets for ingestion, scanning, policy evaluation, and exports.
- Document required secrets/scopes and deterministic build knobs. - Document required secrets/scopes and deterministic build knobs.
- Highlight offline-compatible workflows and cache strategies. - Highlight offline-compatible workflows and cache strategies.
## Key components ## Key components
- Recipe catalogue in ./recipes.md. - Recipe catalogue in ./recipes.md.
## Integrations & dependencies ## Integrations & dependencies
- DevOps release workflows. - DevOps release workflows.
- Module-specific test suites referenced in recipes. - Module-specific test suites referenced in recipes.
## Operational notes ## Operational notes
- Encourage reuse through templated YAML/JSON fragments. - Encourage reuse through templated YAML/JSON fragments.
## Related resources ## Related resources
- ./recipes.md - ./recipes.md
## Backlog references ## Backlog references
- CI recipes refresh tracked in ../../TASKS.md under DOCS-CI stories. - CI recipes refresh tracked in ../../TASKS.md under DOCS-CI stories.
## Epic alignment ## Epic alignment
- **Epic 1 – AOC enforcement:** bake ingestion/verifier guardrails into CI recipes. - **Epic 1 – AOC enforcement:** bake ingestion/verifier guardrails into CI recipes.
- **Epic 10 – Export Center:** provide pipeline snippets for export packaging, signing, and Offline Kit publication. - **Epic 10 – Export Center:** provide pipeline snippets for export packaging, signing, and Offline Kit publication.
- **Epic 11 – Notifications Studio:** offer CI hooks for notification previews/tests where relevant. - **Epic 11 – Notifications Studio:** offer CI hooks for notification previews/tests where relevant.

18
docs/modules/ci/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — CI Recipes # Task board — CI Recipes
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| CI RECIPES-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | CI RECIPES-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| CI RECIPES-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | CI RECIPES-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| CI RECIPES-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | CI RECIPES-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

14
docs/modules/ci/architecture.md
View File

@@ -1,7 +1,7 @@
# CI Recipes architecture # CI Recipes architecture
> Reference the AOC guardrails, export workflows, and notification patterns documented in the Authority, Export Center, and Notify module guides when designing CI templates. > Reference the AOC guardrails, export workflows, and notification patterns documented in the Authority, Export Center, and Notify module guides when designing CI templates.
This placeholder summarises the planned architecture for CI Recipes. Consolidate design details from implementation plans and upcoming epics before coding. This placeholder summarises the planned architecture for CI Recipes. Consolidate design details from implementation plans and upcoming epics before coding.
Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised. Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised.

42
docs/modules/ci/implementation_plan.md
View File

@@ -1,21 +1,21 @@
# Implementation plan — CI Recipes # Implementation plan — CI Recipes
## Current objectives ## Current objectives
- Maintain deterministic behaviour and offline parity across releases. - Maintain deterministic behaviour and offline parity across releases.
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes. - Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
## Workstreams ## Workstreams
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap. - Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs. - Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
- Validation: extend tests/fixtures to preserve determinism and provenance requirements. - Validation: extend tests/fixtures to preserve determinism and provenance requirements.
## Epic milestones ## Epic milestones
- **Epic 1 – AOC enforcement:** ensure pipelines enforce schemas, provenance, and verifier jobs. - **Epic 1 – AOC enforcement:** ensure pipelines enforce schemas, provenance, and verifier jobs.
- **Epic 10 – Export Center:** add export/signing/Offline Kit automation templates. - **Epic 10 – Export Center:** add export/signing/Offline Kit automation templates.
- **Epic 11 – Notifications Studio:** document CI hooks for notification previews/tests. - **Epic 11 – Notifications Studio:** document CI hooks for notification previews/tests.
- Track DOCS-CI stories in ../../TASKS.md. - Track DOCS-CI stories in ../../TASKS.md.
## Coordination ## Coordination
- Review ./AGENTS.md before picking up new work. - Review ./AGENTS.md before picking up new work.
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md. - Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
- Update this plan whenever scope, dependencies, or guardrails change. - Update this plan whenever scope, dependencies, or guardrails change.

706
docs/modules/ci/recipes.md
View File

@@ -1,353 +1,353 @@
# Stella Ops CI Recipes — (2025‑08‑04) # Stella Ops CI Recipes — (2025‑08‑04)
## 0 · Key variables (export these once) ## 0 · Key variables (export these once)
| Variable | Meaning | Typical value | | Variable | Meaning | Typical value |
| ------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | | ------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| `STELLA_URL` | Host that: ① stores the **CLI** & **SBOM‑builder** images under `/registry` **and** ② receives API calls at `https://$STELLA_URL` | `stella-ops.ci.acme.example` | | `STELLA_URL` | Host that: ① stores the **CLI** & **SBOM‑builder** images under `/registry` **and** ② receives API calls at `https://$STELLA_URL` | `stella-ops.ci.acme.example` |
| `DOCKER_HOST` | How containers reach your Docker daemon (because we no longer mount `/var/run/docker.sock`) | `tcp://docker:2375` | | `DOCKER_HOST` | How containers reach your Docker daemon (because we no longer mount `/var/run/docker.sock`) | `tcp://docker:2375` |
| `WORKSPACE` | Directory where the pipeline stores artefacts (SBOM file) | `$(pwd)` | | `WORKSPACE` | Directory where the pipeline stores artefacts (SBOM file) | `$(pwd)` |
| `IMAGE` | The image you are building & scanning | `acme/backend:sha-${COMMIT_SHA}` | | `IMAGE` | The image you are building & scanning | `acme/backend:sha-${COMMIT_SHA}` |
| `SBOM_FILE` | Immutable SBOM name – `<image-ref>‑YYYYMMDDThhmmssZ.sbom.json` | `acme_backend_sha‑abc123‑20250804T153050Z.sbom.json` | | `SBOM_FILE` | Immutable SBOM name – `<image-ref>‑YYYYMMDDThhmmssZ.sbom.json` | `acme_backend_sha‑abc123‑20250804T153050Z.sbom.json` |
> **Authority graph scopes note (2025-10-27):** CI stages that spin up the Authority compose profile now rely on the checked-in `etc/authority.yaml`. Before running integration smoke jobs, inject real secrets for every `etc/secrets/*.secret` file (Cartographer, Graph API, Policy Engine, Concelier, Excititor). The repository defaults contain `*-change-me` placeholders and Authority will reject tokens if those secrets are not overridden. Reissue CI tokens that previously used `policy:write`/`policy:submit`/`policy:edit` scopes—new bundles must request `policy:read`, `policy:author`, `policy:review`, `policy:simulate`, and (`policy:approve`/`policy:operate`/`policy:activate` when pipelines promote policies). > **Authority graph scopes note (2025-10-27):** CI stages that spin up the Authority compose profile now rely on the checked-in `etc/authority.yaml`. Before running integration smoke jobs, inject real secrets for every `etc/secrets/*.secret` file (Cartographer, Graph API, Policy Engine, Concelier, Excititor). The repository defaults contain `*-change-me` placeholders and Authority will reject tokens if those secrets are not overridden. Reissue CI tokens that previously used `policy:write`/`policy:submit`/`policy:edit` scopes—new bundles must request `policy:read`, `policy:author`, `policy:review`, `policy:simulate`, and (`policy:approve`/`policy:operate`/`policy:activate` when pipelines promote policies).
```bash ```bash
export STELLA_URL="stella-ops.ci.acme.example" export STELLA_URL="stella-ops.ci.acme.example"
export DOCKER_HOST="tcp://docker:2375" # Jenkins/Circle often expose it like this export DOCKER_HOST="tcp://docker:2375" # Jenkins/Circle often expose it like this
export WORKSPACE="$(pwd)" export WORKSPACE="$(pwd)"
export IMAGE="acme/backend:sha-${COMMIT_SHA}" export IMAGE="acme/backend:sha-${COMMIT_SHA}"
export SBOM_FILE="$(echo "${IMAGE}" | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json" export SBOM_FILE="$(echo "${IMAGE}" | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"
``` ```
--- ---
## 1 · SBOM creation strategies ## 1 · SBOM creation strategies
### Option A – **Buildx attested SBOM** (preferred if you can use BuildKit) ### Option A – **Buildx attested SBOM** (preferred if you can use BuildKit)
You pass **two build args** so the Dockerfile can run the builder and copy the result out of the build context. You pass **two build args** so the Dockerfile can run the builder and copy the result out of the build context.
```bash ```bash
docker buildx build \ docker buildx build \
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \ --build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
--provenance=true --sbom=true \ --provenance=true --sbom=true \
--build-arg SBOM_FILE="$SBOM_FILE" \ --build-arg SBOM_FILE="$SBOM_FILE" \
-t "$IMAGE" . -t "$IMAGE" .
``` ```
**If you **cannot** use Buildx, use Option B below.** The older “run a builder stage inside the Dockerfile” pattern is unreliable for producing an SBOM of the final image. **If you **cannot** use Buildx, use Option B below.** The older “run a builder stage inside the Dockerfile” pattern is unreliable for producing an SBOM of the final image.
```Dockerfile ```Dockerfile
ARG STELLA_SBOM_BUILDER ARG STELLA_SBOM_BUILDER
ARG SBOM_FILE ARG SBOM_FILE
FROM $STELLA_SBOM_BUILDER as sbom FROM $STELLA_SBOM_BUILDER as sbom
ARG IMAGE ARG IMAGE
ARG SBOM_FILE ARG SBOM_FILE
RUN $STELLA_SBOM_BUILDER build --image $IMAGE --output /out/$SBOM_FILE RUN $STELLA_SBOM_BUILDER build --image $IMAGE --output /out/$SBOM_FILE
# ---- actual build stages … ---- # ---- actual build stages … ----
FROM alpine:3.20 FROM alpine:3.20
COPY --from=sbom /out/$SBOM_FILE / # (optional) keep or discard COPY --from=sbom /out/$SBOM_FILE / # (optional) keep or discard
# (rest of your Dockerfile) # (rest of your Dockerfile)
``` ```
### Option B – **External builder step** (works everywhere; recommended baseline if Buildx isn’t available) ### Option B – **External builder step** (works everywhere; recommended baseline if Buildx isn’t available)
*(keep this block if your pipeline already has an image‑build step that you can’t modify)* *(keep this block if your pipeline already has an image‑build step that you can’t modify)*
```bash ```bash
docker run --rm \ docker run --rm \
-e DOCKER_HOST="$DOCKER_HOST" \ # let builder reach the daemon remotely -e DOCKER_HOST="$DOCKER_HOST" \ # let builder reach the daemon remotely
-v "$WORKSPACE:/workspace" \ # place SBOM beside the source code -v "$WORKSPACE:/workspace" \ # place SBOM beside the source code
"$STELLA_URL/registry/stella-sbom-builder:latest" \ "$STELLA_URL/registry/stella-sbom-builder:latest" \
build --image "$IMAGE" --output "/workspace/${SBOM_FILE}" build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
``` ```
--- ---
## 2 · Scan the image & upload results ## 2 · Scan the image & upload results
```bash ```bash
docker run --rm \ docker run --rm \
-e DOCKER_HOST="$DOCKER_HOST" \ # remote‑daemon pointer -e DOCKER_HOST="$DOCKER_HOST" \ # remote‑daemon pointer
-v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \ # mount SBOM under same name at container root -v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \ # mount SBOM under same name at container root
-e STELLA_OPS_URL="https://${STELLA_URL}" \ # where the CLI posts findings -e STELLA_OPS_URL="https://${STELLA_URL}" \ # where the CLI posts findings
"$STELLA_URL/registry/stella-cli:latest" \ "$STELLA_URL/registry/stella-cli:latest" \
scan --sbom "/${SBOM_FILE}" "$IMAGE" scan --sbom "/${SBOM_FILE}" "$IMAGE"
``` ```
The CLI returns **exit 0** if policies pass, **>0** if blocked — perfect for failing the job. The CLI returns **exit 0** if policies pass, **>0** if blocked — perfect for failing the job.
--- ---
## 3 · CI templates ## 3 · CI templates
Below are minimal, cut‑and‑paste snippets. Below are minimal, cut‑and‑paste snippets.
**Feel free to delete Option B** if you adopt Option A. **Feel free to delete Option B** if you adopt Option A.
### 3.1 Jenkins (Declarative Pipeline) ### 3.1 Jenkins (Declarative Pipeline)
```groovy ```groovy
pipeline { pipeline {
agent { docker { image 'docker:25' args '--privileged' } } // gives us /usr/bin/docker agent { docker { image 'docker:25' args '--privileged' } } // gives us /usr/bin/docker
environment { environment {
STELLA_URL = 'stella-ops.ci.acme.example' STELLA_URL = 'stella-ops.ci.acme.example'
DOCKER_HOST = 'tcp://docker:2375' DOCKER_HOST = 'tcp://docker:2375'
IMAGE = "acme/backend:${env.BUILD_NUMBER}" IMAGE = "acme/backend:${env.BUILD_NUMBER}"
SBOM_FILE = "acme_backend_${env.BUILD_NUMBER}-${new Date().format('yyyyMMdd\'T\'HHmmss\'Z\'', TimeZone.getTimeZone('UTC'))}.sbom.json" SBOM_FILE = "acme_backend_${env.BUILD_NUMBER}-${new Date().format('yyyyMMdd\'T\'HHmmss\'Z\'', TimeZone.getTimeZone('UTC'))}.sbom.json"
} }
stages { stages {
stage('Build image + SBOM (Option A)') { stage('Build image + SBOM (Option A)') {
steps { steps {
sh ''' sh '''
docker build \ docker build \
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \ --build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
--build-arg SBOM_FILE="$SBOM_FILE" \ --build-arg SBOM_FILE="$SBOM_FILE" \
-t "$IMAGE" . -t "$IMAGE" .
''' '''
} }
} }
/* ---------- Option B fallback (when you must keep the existing build step as‑is) ---------- /* ---------- Option B fallback (when you must keep the existing build step as‑is) ----------
stage('SBOM builder (Option B)') { stage('SBOM builder (Option B)') {
steps { steps {
sh ''' sh '''
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \ docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
-v "$WORKSPACE:/workspace" \ -v "$WORKSPACE:/workspace" \
"$STELLA_URL/registry/stella-sbom-builder:latest" \ "$STELLA_URL/registry/stella-sbom-builder:latest" \
build --image "$IMAGE" --output "/workspace/${SBOM_FILE}" build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
''' '''
} }
} }
------------------------------------------------------------------------------------------ */ ------------------------------------------------------------------------------------------ */
stage('Scan & upload') { stage('Scan & upload') {
steps { steps {
sh ''' sh '''
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \ docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
-v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \ -v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \
-e STELLA_OPS_URL="https://$STELLA_URL" \ -e STELLA_OPS_URL="https://$STELLA_URL" \
"$STELLA_URL/registry/stella-cli:latest" \ "$STELLA_URL/registry/stella-cli:latest" \
scan --sbom "/${SBOM_FILE}" "$IMAGE" scan --sbom "/${SBOM_FILE}" "$IMAGE"
''' '''
} }
} }
} }
} }
``` ```
--- ---
### 3.2 CircleCI `.circleci/config.yml` ### 3.2 CircleCI `.circleci/config.yml`
```yaml ```yaml
version: 2.1 version: 2.1
jobs: jobs:
stella_scan: stella_scan:
docker: docker:
- image: cimg/base:stable # baremetal image with Docker CLI - image: cimg/base:stable # baremetal image with Docker CLI
environment: environment:
STELLA_URL: stella-ops.ci.acme.example STELLA_URL: stella-ops.ci.acme.example
DOCKER_HOST: tcp://docker:2375 # Circle’s “remote Docker” socket DOCKER_HOST: tcp://docker:2375 # Circle’s “remote Docker” socket
steps: steps:
- checkout - checkout
- run: - run:
name: Compute vars name: Compute vars
command: | command: |
echo 'export IMAGE="acme/backend:${CIRCLE_SHA1}"' >> $BASH_ENV echo 'export IMAGE="acme/backend:${CIRCLE_SHA1}"' >> $BASH_ENV
echo 'export SBOM_FILE="$(echo acme/backend:${CIRCLE_SHA1} | tr "/:+" "__")-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"' >> $BASH_ENV echo 'export SBOM_FILE="$(echo acme/backend:${CIRCLE_SHA1} | tr "/:+" "__")-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"' >> $BASH_ENV
- run: - run:
name: Build image + SBOM (Option A) name: Build image + SBOM (Option A)
command: | command: |
docker build \ docker build \
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \ --build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
--build-arg SBOM_FILE="$SBOM_FILE" \ --build-arg SBOM_FILE="$SBOM_FILE" \
-t "$IMAGE" . -t "$IMAGE" .
# --- Option B fallback (when you must keep the existing build step as‑is) --- # --- Option B fallback (when you must keep the existing build step as‑is) ---
#- run: #- run:
# name: SBOM builder (Option B) # name: SBOM builder (Option B)
# command: | # command: |
# docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \ # docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
# -v "$PWD:/workspace" \ # -v "$PWD:/workspace" \
# "$STELLA_URL/registry/stella-sbom-builder:latest" \ # "$STELLA_URL/registry/stella-sbom-builder:latest" \
# build --image "$IMAGE" --output "/workspace/${SBOM_FILE}" # build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
- run: - run:
name: Scan name: Scan
command: | command: |
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \ docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
-v "$PWD/${SBOM_FILE}:/${SBOM_FILE}:ro" \ -v "$PWD/${SBOM_FILE}:/${SBOM_FILE}:ro" \
-e STELLA_OPS_URL="https://$STELLA_URL" \ -e STELLA_OPS_URL="https://$STELLA_URL" \
"$STELLA_URL/registry/stella-cli:latest" \ "$STELLA_URL/registry/stella-cli:latest" \
scan --sbom "/${SBOM_FILE}" "$IMAGE" scan --sbom "/${SBOM_FILE}" "$IMAGE"
workflows: workflows:
stella: stella:
jobs: [stella_scan] jobs: [stella_scan]
``` ```
--- ---
### 3.3 Gitea Actions `.gitea/workflows/stella.yml` ### 3.3 Gitea Actions `.gitea/workflows/stella.yml`
*(Gitea 1.22+ ships native Actions compatible with GitHub syntax)* *(Gitea 1.22+ ships native Actions compatible with GitHub syntax)*
```yaml ```yaml
name: Stella Scan name: Stella Scan
on: [push] on: [push]
jobs: jobs:
stella: stella:
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
STELLA_URL: ${{ secrets.STELLA_URL }} STELLA_URL: ${{ secrets.STELLA_URL }}
DOCKER_HOST: tcp://docker:2375 # provided by the docker:dind service DOCKER_HOST: tcp://docker:2375 # provided by the docker:dind service
services: services:
docker: docker:
image: docker:dind image: docker:dind
options: >- options: >-
--privileged --privileged
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Compute vars - name: Compute vars
id: vars id: vars
run: | run: |
echo "IMAGE=ghcr.io/${{ gitea.repository }}:${{ gitea.sha }}" >> $GITEA_OUTPUT echo "IMAGE=ghcr.io/${{ gitea.repository }}:${{ gitea.sha }}" >> $GITEA_OUTPUT
echo "SBOM_FILE=$(echo ghcr.io/${{ gitea.repository }}:${{ gitea.sha }} | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json" >> $GITEA_OUTPUT echo "SBOM_FILE=$(echo ghcr.io/${{ gitea.repository }}:${{ gitea.sha }} | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json" >> $GITEA_OUTPUT
- name: Build image + SBOM (Option A) - name: Build image + SBOM (Option A)
run: | run: |
docker build \ docker build \
--build-arg STELLA_SBOM_BUILDER="${STELLA_URL}/registry/stella-sbom-builder:latest" \ --build-arg STELLA_SBOM_BUILDER="${STELLA_URL}/registry/stella-sbom-builder:latest" \
--build-arg SBOM_FILE="${{ steps.vars.outputs.SBOM_FILE }}" \ --build-arg SBOM_FILE="${{ steps.vars.outputs.SBOM_FILE }}" \
-t "${{ steps.vars.outputs.IMAGE }}" . -t "${{ steps.vars.outputs.IMAGE }}" .
# --- Option B fallback (when you must keep the existing build step as‑is) --- # --- Option B fallback (when you must keep the existing build step as‑is) ---
#- name: SBOM builder (Option B) #- name: SBOM builder (Option B)
# run: | # run: |
# docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \ # docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
# -v "$(pwd):/workspace" \ # -v "$(pwd):/workspace" \
# "${STELLA_URL}/registry/stella-sbom-builder:latest" \ # "${STELLA_URL}/registry/stella-sbom-builder:latest" \
# build --image "${{ steps.vars.outputs.IMAGE }}" --output "/workspace/${{ steps.vars.outputs.SBOM_FILE }}" # build --image "${{ steps.vars.outputs.IMAGE }}" --output "/workspace/${{ steps.vars.outputs.SBOM_FILE }}"
- name: Scan - name: Scan
run: | run: |
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \ docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
-v "$(pwd)/${{ steps.vars.outputs.SBOM_FILE }}:/${{ steps.vars.outputs.SBOM_FILE }}:ro" \ -v "$(pwd)/${{ steps.vars.outputs.SBOM_FILE }}:/${{ steps.vars.outputs.SBOM_FILE }}:ro" \
-e STELLA_OPS_URL="https://${STELLA_URL}" \ -e STELLA_OPS_URL="https://${STELLA_URL}" \
"${STELLA_URL}/registry/stella-cli:latest" \ "${STELLA_URL}/registry/stella-cli:latest" \
scan --sbom "/${{ steps.vars.outputs.SBOM_FILE }}" "${{ steps.vars.outputs.IMAGE }}" scan --sbom "/${{ steps.vars.outputs.SBOM_FILE }}" "${{ steps.vars.outputs.IMAGE }}"
``` ```
--- ---
## 4 · Docs CI (Gitea Actions & Offline Mirror) ## 4 · Docs CI (Gitea Actions & Offline Mirror)
StellaOps ships a dedicated Docs workflow at `.gitea/workflows/docs.yml`. When mirroring the pipeline offline or running it locally, install the same toolchain so markdown linting, schema validation, and HTML preview stay deterministic. StellaOps ships a dedicated Docs workflow at `.gitea/workflows/docs.yml`. When mirroring the pipeline offline or running it locally, install the same toolchain so markdown linting, schema validation, and HTML preview stay deterministic.
### 4.1 Toolchain bootstrap ### 4.1 Toolchain bootstrap
```bash ```bash
# Node.js 20.x is required; install once per runner # Node.js 20.x is required; install once per runner
npm install --no-save \ npm install --no-save \
markdown-link-check \ markdown-link-check \
remark-cli \ remark-cli \
remark-preset-lint-recommended \ remark-preset-lint-recommended \
ajv \ ajv \
ajv-cli \ ajv-cli \
ajv-formats ajv-formats
# Python 3.11+ powers the preview renderer # Python 3.11+ powers the preview renderer
python -m pip install --upgrade pip python -m pip install --upgrade pip
python -m pip install markdown pygments python -m pip install markdown pygments
``` ```
> **No `pip` available?** Some hardened Python builds (including the repo’s `tmp/docenv` > **No `pip` available?** Some hardened Python builds (including the repo’s `tmp/docenv`
> interpreter) ship without `pip`/`ensurepip`. In that case download the pure‑Python > interpreter) ship without `pip`/`ensurepip`. In that case download the pure‑Python
> sdists (e.g. `Markdown-3.x.tar.gz`, `pygments-2.x.tar.gz`) and extract their > sdists (e.g. `Markdown-3.x.tar.gz`, `pygments-2.x.tar.gz`) and extract their
> packages directly into the virtualenv’s `lib/python*/site-packages/` folder. > packages directly into the virtualenv’s `lib/python*/site-packages/` folder.
> This keeps the renderer working even when package managers are disabled. > This keeps the renderer working even when package managers are disabled.
**Offline tip.** Add the packages above to your artifact mirror (for example `ops/devops/offline-kit.json`) so runners can install them via `npm --offline` / `pip --no-index`. **Offline tip.** Add the packages above to your artifact mirror (for example `ops/devops/offline-kit.json`) so runners can install them via `npm --offline` / `pip --no-index`.
### 4.2 Schema validation step ### 4.2 Schema validation step
Ajv compiles every event schema to guard against syntax or format regressions. The workflow uses `ajv-formats` for UUID/date-time support. Ajv compiles every event schema to guard against syntax or format regressions. The workflow uses `ajv-formats` for UUID/date-time support.
```bash ```bash
for schema in docs/events/*.json; do for schema in docs/events/*.json; do
npx ajv compile -c ajv-formats -s "$schema" npx ajv compile -c ajv-formats -s "$schema"
done done
``` ```
Run this loop before committing schema changes. For new references, append `-r additional-file.json` so CI and local runs stay aligned. Run this loop before committing schema changes. For new references, append `-r additional-file.json` so CI and local runs stay aligned.
### 4.3 Preview build ### 4.3 Preview build
```bash ```bash
python scripts/render_docs.py --source docs --output artifacts/docs-preview --clean python scripts/render_docs.py --source docs --output artifacts/docs-preview --clean
``` ```
Host the resulting bundle via any static file server for review (for example `python -m http.server`). Host the resulting bundle via any static file server for review (for example `python -m http.server`).
### 4.4 Publishing checklist ### 4.4 Publishing checklist
- [ ] Toolchain installs succeed without hitting the public internet (mirror or cached tarballs). - [ ] Toolchain installs succeed without hitting the public internet (mirror or cached tarballs).
- [ ] Ajv validation passes for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. - [ ] Ajv validation passes for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`.
- [ ] Markdown link check (`npx markdown-link-check`) reports no broken references. - [ ] Markdown link check (`npx markdown-link-check`) reports no broken references.
- [ ] Preview bundle archived (or attached) for stakeholders. - [ ] Preview bundle archived (or attached) for stakeholders.
### 4.5 Policy DSL lint stage ### 4.5 Policy DSL lint stage
Policy Engine v2 pipelines now fail fast if policy documents are malformed. After checkout and dotnet restore, run: Policy Engine v2 pipelines now fail fast if policy documents are malformed. After checkout and dotnet restore, run:
```bash ```bash
dotnet run \ dotnet run \
--project src/Tools/PolicyDslValidator/PolicyDslValidator.csproj \ --project src/Tools/PolicyDslValidator/PolicyDslValidator.csproj \
-- \ -- \
--strict docs/examples/policies/*.yaml --strict docs/examples/policies/*.yaml
``` ```
- `--strict` treats warnings as errors so missing metadata doesn’t slip through. - `--strict` treats warnings as errors so missing metadata doesn’t slip through.
- The validator accepts globs, so you can point it at tenant policy directories later (`policies/**/*.yaml`). - The validator accepts globs, so you can point it at tenant policy directories later (`policies/**/*.yaml`).
- Exit codes follow UNIX conventions: `0` success, `1` parse/errors, `2` warnings when `--strict` is set, `64` usage mistakes. - Exit codes follow UNIX conventions: `0` success, `1` parse/errors, `2` warnings when `--strict` is set, `64` usage mistakes.
Capture the validator output as part of your build logs; Support uses it when triaging policy rollout issues. Capture the validator output as part of your build logs; Support uses it when triaging policy rollout issues.
### 4.6 Policy simulation smoke ### 4.6 Policy simulation smoke
Catch unexpected policy regressions by exercising a small set of golden SBOM findings via the simulation smoke tool: Catch unexpected policy regressions by exercising a small set of golden SBOM findings via the simulation smoke tool:
```bash ```bash
dotnet run \ dotnet run \
--project src/Tools/PolicySimulationSmoke/PolicySimulationSmoke.csproj \ --project src/Tools/PolicySimulationSmoke/PolicySimulationSmoke.csproj \
-- \ -- \
--scenario-root samples/policy/simulations \ --scenario-root samples/policy/simulations \
--output artifacts/policy-simulations --output artifacts/policy-simulations
``` ```
- The tool loads each `scenario.json` under `samples/policy/simulations`, evaluates the referenced policy, and fails the build if projected verdicts change. - The tool loads each `scenario.json` under `samples/policy/simulations`, evaluates the referenced policy, and fails the build if projected verdicts change.
- In CI the command runs twice (to `run1/` and `run2/`) and `diff -u` compares the summaries—any mismatch signals a determinism regression. - In CI the command runs twice (to `run1/` and `run2/`) and `diff -u` compares the summaries—any mismatch signals a determinism regression.
- Artifacts land in `artifacts/policy-simulations/policy-simulation-summary.json`; upload them for later inspection (see CI workflow). - Artifacts land in `artifacts/policy-simulations/policy-simulation-summary.json`; upload them for later inspection (see CI workflow).
- Expand scenarios by copying real-world findings into the samples directory—ensure expected statuses are recorded so regressions trip the pipeline. - Expand scenarios by copying real-world findings into the samples directory—ensure expected statuses are recorded so regressions trip the pipeline.
--- ---
## 5 · Troubleshooting cheat‑sheet ## 5 · Troubleshooting cheat‑sheet
| Symptom | Root cause | First things to try | | Symptom | Root cause | First things to try |
| ------------------------------------- | --------------------------- | --------------------------------------------------------------- | | ------------------------------------- | --------------------------- | --------------------------------------------------------------- |
| `no such host $STELLA_URL` | DNS typo or VPN outage | `ping $STELLA_URL` from runner | | `no such host $STELLA_URL` | DNS typo or VPN outage | `ping $STELLA_URL` from runner |
| `connection refused` when CLI uploads | Port 443 blocked | open firewall / check ingress | | `connection refused` when CLI uploads | Port 443 blocked | open firewall / check ingress |
| `failed to stat /<sbom>.json` | SBOM wasn’t produced | Did Option A actually run builder? If not, enable Option B | | `failed to stat /<sbom>.json` | SBOM wasn’t produced | Did Option A actually run builder? If not, enable Option B |
| `registry unauthorized` | Runner lacks registry creds | `docker login $STELLA_URL/registry` (store creds in CI secrets) | | `registry unauthorized` | Runner lacks registry creds | `docker login $STELLA_URL/registry` (store creds in CI secrets) |
| Non‑zero scan exit | Blocking vuln/licence | Open project in Ops UI → triage or waive | | Non‑zero scan exit | Blocking vuln/licence | Open project in Ops UI → triage or waive |
--- ---
### Change log ### Change log
* **2025‑10‑18** – Documented Docs CI toolchain (Ajv validation, static preview) and offline checklist. * **2025‑10‑18** – Documented Docs CI toolchain (Ajv validation, static preview) and offline checklist.
* **2025‑08‑04** – Variable clean‑up, removed Docker‑socket & cache mounts, added Jenkins / CircleCI / Gitea examples, clarified Option B comment. * **2025‑08‑04** – Variable clean‑up, removed Docker‑socket & cache mounts, added Jenkins / CircleCI / Gitea examples, clarified Option B comment.

42
docs/modules/cli/AGENTS.md
View File

@@ -1,22 +1,22 @@
# CLI agent guide # CLI agent guide
## Mission ## Mission
The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting. The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

80
docs/modules/cli/README.md
View File

@@ -1,40 +1,40 @@
# StellaOps CLI # StellaOps CLI
The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting. The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting.
## Responsibilities ## Responsibilities
- Deliver deterministic verbs for scan, diff, export, policy, and observability operations. - Deliver deterministic verbs for scan, diff, export, policy, and observability operations.
- Handle interactive and non-interactive authentication via Authority (device code, client credentials). - Handle interactive and non-interactive authentication via Authority (device code, client credentials).
- Support offline kit workflows including bundle verification and seed installation. - Support offline kit workflows including bundle verification and seed installation.
- Expose JSON outputs suitable for CI parity and golden tests. - Expose JSON outputs suitable for CI parity and golden tests.
## Key components ## Key components
- `StellaOps.Cli` native AOT host. - `StellaOps.Cli` native AOT host.
- Shared helpers in `StellaOps.Cli.Core`. - Shared helpers in `StellaOps.Cli.Core`.
- Restart-time plug-ins under `StellaOps.Cli.Plugins.*`. - Restart-time plug-ins under `StellaOps.Cli.Plugins.*`.
## Integrations & dependencies ## Integrations & dependencies
- Authority for token exchange. - Authority for token exchange.
- Backend APIs (Scanner, Policy, Export Center, Notify). - Backend APIs (Scanner, Policy, Export Center, Notify).
- Offline kit bundles and local keychain/DPoP storage. - Offline kit bundles and local keychain/DPoP storage.
## Operational notes ## Operational notes
- Deterministic output fixtures under `src/Cli/StellaOps.Cli.Tests`. - Deterministic output fixtures under `src/Cli/StellaOps.Cli.Tests`.
- Versioned command docs in `docs/modules/cli/guides`. - Versioned command docs in `docs/modules/cli/guides`.
- Plugin catalogue in `plugins/cli/**` (restart-only). - Plugin catalogue in `plugins/cli/**` (restart-only).
## Related resources ## Related resources
- ./guides/20_REFERENCE.md - ./guides/20_REFERENCE.md
- ./guides/cli-reference.md - ./guides/cli-reference.md
- ./guides/policy.md - ./guides/policy.md
## Backlog references ## Backlog references
- DOCS-CLI-OBS-52-001 / DOCS-CLI-FORENSICS-53-001 in ../../TASKS.md. - DOCS-CLI-OBS-52-001 / DOCS-CLI-FORENSICS-53-001 in ../../TASKS.md.
- CLI-CORE-41-001 epic in `src/Cli/StellaOps.Cli/TASKS.md`. - CLI-CORE-41-001 epic in `src/Cli/StellaOps.Cli/TASKS.md`.
## Epic alignment ## Epic alignment
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy authoring, simulation, and explain verbs. - **Epic 2 – Policy Engine & Editor:** deliver deterministic policy authoring, simulation, and explain verbs.
- **Epic 4 – Policy Studio:** integrate registry/promotion workflows, approvals, and lint tooling. - **Epic 4 – Policy Studio:** integrate registry/promotion workflows, approvals, and lint tooling.
- **Epic 6 – Vulnerability Explorer:** surface triage and ledger operations. - **Epic 6 – Vulnerability Explorer:** surface triage and ledger operations.
- **Epic 10 – Export Center:** orchestrate export requests, verification, and Offline Kit automation. - **Epic 10 – Export Center:** orchestrate export requests, verification, and Offline Kit automation.
- **Epic 11 – Notifications Studio:** manage notification authoring/previews from the command line. - **Epic 11 – Notifications Studio:** manage notification authoring/previews from the command line.

18
docs/modules/cli/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — CLI # Task board — CLI
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| CLI-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | CLI-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| CLI-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | CLI-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| CLI-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | CLI-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

16
docs/modules/cli/guides/20_REFERENCE.md
View File

@@ -1,8 +1,8 @@
# CLI Reference (`stella --help`) # CLI Reference (`stella --help`)
> **Auto‑generated file — do not edit manually.** > **Auto‑generated file — do not edit manually.**
> On every tagged release the CI pipeline runs > On every tagged release the CI pipeline runs
> `stella --help --markdown > docs/modules/cli/guides/20_REFERENCE.md` > `stella --help --markdown > docs/modules/cli/guides/20_REFERENCE.md`
> ensuring this document always matches the shipped binary. > ensuring this document always matches the shipped binary.
*(The reference will appear after the first public α release.)* *(The reference will appear after the first public α release.)*

626
docs/modules/cli/guides/cli-reference.md
View File

@@ -1,316 +1,316 @@
# CLI AOC Commands Reference # CLI AOC Commands Reference
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows. > **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19. > **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19.
Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../../ingestion/aggregation-only-contract.md) and the [architecture overview](../architecture.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores. Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../../ingestion/aggregation-only-contract.md) and the [architecture overview](../architecture.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores.
--- ---
## 1 · Prerequisites ## 1 · Prerequisites
- CLI version: `stella` ≥ 0.19.0 (AOC feature gate enabled). - CLI version: `stella` ≥ 0.19.0 (AOC feature gate enabled).
- Required scopes (DPoP-bound): - Required scopes (DPoP-bound):
- `advisory:read` for Concelier sources. - `advisory:read` for Concelier sources.
- `vex:read` for Excititor sources (optional but required for VEX checks). - `vex:read` for Excititor sources (optional but required for VEX checks).
- `aoc:verify` to invoke guard verification endpoints. - `aoc:verify` to invoke guard verification endpoints.
- `tenant:select` if your deployment uses tenant switching. - `tenant:select` if your deployment uses tenant switching.
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see § 4). - Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see § 4).
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache. - Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
--- ---
## 2 · `stella sources ingest --dry-run` ## 2 · `stella sources ingest --dry-run`
### 2.1 Synopsis ### 2.1 Synopsis
```bash ```bash
stella sources ingest --dry-run \ stella sources ingest --dry-run \
--source <source-key> \ --source <source-key> \
--input <path-or-uri> \ --input <path-or-uri> \
[--tenant <tenant-id>] \ [--tenant <tenant-id>] \
[--format json|table] \ [--format json|table] \
[--no-color] \ [--no-color] \
[--output <file>] [--output <file>]
``` ```
### 2.2 Description ### 2.2 Description
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents. Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
### 2.3 Options ### 2.3 Options
| Option | Description | | Option | Description |
|--------|-------------| |--------|-------------|
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. | | `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. | | `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. | | `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). | | `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
| `--no-color` | Disables ANSI colour output for CI logs. | | `--no-color` | Disables ANSI colour output for CI logs. |
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. | | `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
### 2.4 Output schema (JSON) ### 2.4 Output schema (JSON)
```json ```json
{ {
"source": "redhat", "source": "redhat",
"tenant": "default", "tenant": "default",
"guardVersion": "1.0.0", "guardVersion": "1.0.0",
"status": "ok", "status": "ok",
"document": { "document": {
"contentHash": "sha256:…", "contentHash": "sha256:…",
"supersedes": null, "supersedes": null,
"provenance": { "provenance": {
"signature": { "format": "pgp", "present": true } "signature": { "format": "pgp", "present": true }
} }
}, },
"violations": [] "violations": []
} }
``` ```
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields. When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
### 2.5 Exit codes ### 2.5 Exit codes
| Exit code | Meaning | | Exit code | Meaning |
|-----------|---------| |-----------|---------|
| `0` | Guard passed; would-write payload is AOC compliant. | | `0` | Guard passed; would-write payload is AOC compliant. |
| `11` | `ERR_AOC_001` – Forbidden field (`severity`, `cvss`, etc.) detected. | | `11` | `ERR_AOC_001` – Forbidden field (`severity`, `cvss`, etc.) detected. |
| `12` | `ERR_AOC_002` – Merge attempt (multiple upstream sources fused). | | `12` | `ERR_AOC_002` – Merge attempt (multiple upstream sources fused). |
| `13` | `ERR_AOC_003` – Idempotency violation (duplicate without supersedes). | | `13` | `ERR_AOC_003` – Idempotency violation (duplicate without supersedes). |
| `14` | `ERR_AOC_004` – Missing provenance fields. | | `14` | `ERR_AOC_004` – Missing provenance fields. |
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. | | `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
| `16` | `ERR_AOC_006` – Effective findings present (Policy-only data). | | `16` | `ERR_AOC_006` – Effective findings present (Policy-only data). |
| `17` | `ERR_AOC_007` – Unknown top-level fields / schema violation. | | `17` | `ERR_AOC_007` – Unknown top-level fields / schema violation. |
| `70` | Transport error (network, auth, malformed input). | | `70` | Transport error (network, auth, malformed input). |
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14). > Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
### 2.6 Examples ### 2.6 Examples
Dry-run a local CSAF file: Dry-run a local CSAF file:
```bash ```bash
stella sources ingest --dry-run \ stella sources ingest --dry-run \
--source redhat \ --source redhat \
--input ./fixtures/redhat/RHSA-2025-1234.json --input ./fixtures/redhat/RHSA-2025-1234.json
``` ```
Stream from HTTPS and emit JSON for CI: Stream from HTTPS and emit JSON for CI:
```bash ```bash
stella sources ingest --dry-run \ stella sources ingest --dry-run \
--source osv \ --source osv \
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \ --input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
--format json \ --format json \
--output artifacts/osv-dry-run.json --output artifacts/osv-dry-run.json
cat artifacts/osv-dry-run.json | jq '.violations' cat artifacts/osv-dry-run.json | jq '.violations'
``` ```
### 2.7 Offline notes ### 2.7 Offline notes
When operating in sealed/offline mode: When operating in sealed/offline mode:
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`). - Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants. - Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
- The command does not attempt network access when given a file path. - The command does not attempt network access when given a file path.
- Store reports with `--output` to include in transfer packages for policy review. - Store reports with `--output` to include in transfer packages for policy review.
--- ---
## 3 · `stella aoc verify` ## 3 · `stella aoc verify`
### 3.1 Synopsis ### 3.1 Synopsis
```bash ```bash
stella aoc verify \ stella aoc verify \
[--since <iso8601|duration>] \ [--since <iso8601|duration>] \
[--limit <count>] \ [--limit <count>] \
[--sources <list>] \ [--sources <list>] \
[--codes <ERR_AOC_00x,...>] \ [--codes <ERR_AOC_00x,...>] \
[--format table|json] \ [--format table|json] \
[--export <file>] \ [--export <file>] \
[--tenant <tenant-id>] \ [--tenant <tenant-id>] \
[--no-color] [--no-color]
``` ```
### 3.2 Description ### 3.2 Description
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24 hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response. Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24 hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
### 3.3 Options ### 3.3 Options
| Option | Description | | Option | Description |
|--------|-------------| |--------|-------------|
| `--since <value>` | Verification window. Accepts ISO 8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. | | `--since <value>` | Verification window. Accepts ISO 8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. | | `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. | | `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. | | `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. | | `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). | | `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. | | `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
| `--no-color` | Disables ANSI colours. | | `--no-color` | Disables ANSI colours. |
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path. `table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
### 3.4 Report structure (JSON) ### 3.4 Report structure (JSON)
```json ```json
{ {
"tenant": "default", "tenant": "default",
"window": { "window": {
"from": "2025-10-25T12:00:00Z", "from": "2025-10-25T12:00:00Z",
"to": "2025-10-26T12:00:00Z" "to": "2025-10-26T12:00:00Z"
}, },
"checked": { "checked": {
"advisories": 482, "advisories": 482,
"vex": 75 "vex": 75
}, },
"violations": [ "violations": [
{ {
"code": "ERR_AOC_001", "code": "ERR_AOC_001",
"count": 2, "count": 2,
"examples": [ "examples": [
{ {
"source": "redhat", "source": "redhat",
"documentId": "advisory_raw:redhat:RHSA-2025:1", "documentId": "advisory_raw:redhat:RHSA-2025:1",
"contentHash": "sha256:…", "contentHash": "sha256:…",
"path": "/content/raw/cvss" "path": "/content/raw/cvss"
} }
] ]
} }
], ],
"metrics": { "metrics": {
"ingestion_write_total": 557, "ingestion_write_total": 557,
"aoc_violation_total": 2 "aoc_violation_total": 2
}, },
"truncated": false "truncated": false
} }
``` ```
### 3.5 Exit codes ### 3.5 Exit codes
| Exit code | Meaning | | Exit code | Meaning |
|-----------|---------| |-----------|---------|
| `0` | Verification succeeded with zero violations. | | `0` | Verification succeeded with zero violations. |
| `11…17` | Same mapping as § 2.5 when violations are detected. Highest-priority code returned. | | `11…17` | Same mapping as § 2.5 when violations are detected. Highest-priority code returned. |
| `18` | Verification ran but results truncated (limit reached) – treat as warning; rerun with higher `--limit`. | | `18` | Verification ran but results truncated (limit reached) – treat as warning; rerun with higher `--limit`. |
| `70` | Transport/authentication error. | | `70` | Transport/authentication error. |
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). | | `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
### 3.6 Examples ### 3.6 Examples
Daily verification across all sources: Daily verification across all sources:
```bash ```bash
stella aoc verify --since 24h --format table stella aoc verify --since 24h --format table
``` ```
CI pipeline focusing on errant sources and exporting evidence: CI pipeline focusing on errant sources and exporting evidence:
```bash ```bash
stella aoc verify \ stella aoc verify \
--sources redhat,ubuntu \ --sources redhat,ubuntu \
--codes ERR_AOC_001,ERR_AOC_004 \ --codes ERR_AOC_001,ERR_AOC_004 \
--format json \ --format json \
--limit 100 \ --limit 100 \
--export artifacts/aoc-verify.json --export artifacts/aoc-verify.json
jq '.violations[] | {code, count}' artifacts/aoc-verify.json jq '.violations[] | {code, count}' artifacts/aoc-verify.json
``` ```
Air-gapped verification using Offline Kit snapshot (example script): Air-gapped verification using Offline Kit snapshot (example script):
```bash ```bash
stella aoc verify \ stella aoc verify \
--since 7d \ --since 7d \
--format json \ --format json \
--export /mnt/offline/aoc-verify-$(date +%F).json --export /mnt/offline/aoc-verify-$(date +%F).json
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
``` ```
### 3.7 Automation tips ### 3.7 Automation tips
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥ 11. - Schedule with `cron` or platform scheduler and fail the job when exit code ≥ 11.
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source. - Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
- Push JSON exports to observability pipelines for historical tracking of violation counts. - Push JSON exports to observability pipelines for historical tracking of violation counts.
### 3.8 Offline notes ### 3.8 Offline notes
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle. - Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above). - When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
- Include verification output in compliance packages alongside Offline Kit manifests. - Include verification output in compliance packages alongside Offline Kit manifests.
--- ---
## 4 · Global exit-code reference ## 4 · Global exit-code reference
| Code | Summary | | Code | Summary |
|------|---------| |------|---------|
| `0` | Success / no violations. | | `0` | Success / no violations. |
| `11` | `ERR_AOC_001` – Forbidden field present. | | `11` | `ERR_AOC_001` – Forbidden field present. |
| `12` | `ERR_AOC_002` – Merge attempt detected. | | `12` | `ERR_AOC_002` – Merge attempt detected. |
| `13` | `ERR_AOC_003` – Idempotency violation. | | `13` | `ERR_AOC_003` – Idempotency violation. |
| `14` | `ERR_AOC_004` – Missing provenance/signature metadata. | | `14` | `ERR_AOC_004` – Missing provenance/signature metadata. |
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. | | `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
| `16` | `ERR_AOC_006` – Effective findings in ingestion payload. | | `16` | `ERR_AOC_006` – Effective findings in ingestion payload. |
| `17` | `ERR_AOC_007` – Schema violation / unknown fields. | | `17` | `ERR_AOC_007` – Schema violation / unknown fields. |
| `18` | Partial verification (limit reached). | | `18` | Partial verification (limit reached). |
| `70` | Transport or HTTP failure. | | `70` | Transport or HTTP failure. |
| `71` | CLI usage error (invalid arguments, missing tenant). | | `71` | CLI usage error (invalid arguments, missing tenant). |
Use these codes in CI to map outcomes to build statuses or alert severities. Use these codes in CI to map outcomes to build statuses or alert severities.
--- ---
## 4 · `stella vuln observations` (Overlay paging) ## 4 · `stella vuln observations` (Overlay paging)
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically. `stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
| Option | Description | | Option | Description |
|--------|-------------| |--------|-------------|
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. | | `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. | | `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
Additional notes: Additional notes:
- Table mode prints a hint when `hasMore` is `true`: - Table mode prints a hint when `hasMore` is `true`:
`[yellow]More observations available. Continue with --cursor <token>[/]`. `[yellow]More observations available. Continue with --cursor <token>[/]`.
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`. - JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration. - Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
--- ---
## 5 · Related references ## 5 · Related references
- [Aggregation-Only Contract reference](../../../ingestion/aggregation-only-contract.md) - [Aggregation-Only Contract reference](../../../ingestion/aggregation-only-contract.md)
- [Architecture overview](../../platform/architecture-overview.md) - [Architecture overview](../../platform/architecture-overview.md)
- [Console AOC dashboard](../../../ui/console.md) - [Console AOC dashboard](../../../ui/console.md)
- [Authority scopes](../../authority/architecture.md) - [Authority scopes](../../authority/architecture.md)
--- ---
## 6 · Compliance checklist ## 6 · Compliance checklist
- [ ] Usage documented for both table and JSON formats. - [ ] Usage documented for both table and JSON formats.
- [ ] Exit-code mapping matches `ERR_AOC_00x` definitions and automation guidance. - [ ] Exit-code mapping matches `ERR_AOC_00x` definitions and automation guidance.
- [ ] Offline/air-gap workflow captured for both commands. - [ ] Offline/air-gap workflow captured for both commands.
- [ ] References to AOC architecture and console docs included. - [ ] References to AOC architecture and console docs included.
- [ ] Examples validated against current CLI syntax (update post-implementation). - [ ] Examples validated against current CLI syntax (update post-implementation).
- [ ] Docs guild screenshot/narrative placeholder logged for release notes (pending CLI team capture). - [ ] Docs guild screenshot/narrative placeholder logged for release notes (pending CLI team capture).
--- ---
*Last updated: 2025-10-29 (Sprint 24).* *Last updated: 2025-10-29 (Sprint 24).*
## 13. Authority configuration quick reference ## 13. Authority configuration quick reference
| Setting | Purpose | How to set | | Setting | Purpose | How to set |
|---------|---------|------------| |---------|---------|------------|
| `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. | | `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. |
| `StellaOps:Authority:OperatorTicket` | Change/incident ticket reference paired with orchestrator control actions. | CLI flag `--Authority:OperatorTicket=...` or env `STELLAOPS_ORCH_TICKET`. | | `StellaOps:Authority:OperatorTicket` | Change/incident ticket reference paired with orchestrator control actions. | CLI flag `--Authority:OperatorTicket=...` or env `STELLAOPS_ORCH_TICKET`. |
> Tokens requesting `orch:operate` will fail with `invalid_request` unless both values are present. Choose concise strings (≤256 chars for reason, ≤128 chars for ticket) and avoid sensitive data. > Tokens requesting `orch:operate` will fail with `invalid_request` unless both values are present. Choose concise strings (≤256 chars for reason, ≤128 chars for ticket) and avoid sensitive data.

636
docs/modules/cli/guides/policy.md
View File

@@ -1,318 +1,318 @@
# Stella CLI — Policy Commands # Stella CLI — Policy Commands
> **Audience:** Policy authors, reviewers, operators, and CI engineers using the `stella` CLI to interact with Policy Engine. > **Audience:** Policy authors, reviewers, operators, and CI engineers using the `stella` CLI to interact with Policy Engine.
> **Supported from:** `stella` CLI ≥ 0.20.0 (Policy Engine v2 sprint line). > **Supported from:** `stella` CLI ≥ 0.20.0 (Policy Engine v2 sprint line).
> **Prerequisites:** Authority-issued bearer token with the scopes noted per command (export `STELLA_TOKEN` or pass `--token`). > **Prerequisites:** Authority-issued bearer token with the scopes noted per command (export `STELLA_TOKEN` or pass `--token`).
> **2025-10-27 scope update:** CLI/CI tokens issued prior to Sprint 23 (AUTH-POLICY-23-001) must drop `policy:write`/`policy:submit`/`policy:edit` and instead request `policy:read`, `policy:author`, `policy:review`, and `policy:simulate` (plus `policy:approve`/`policy:operate`/`policy:activate` for promotion pipelines). > **2025-10-27 scope update:** CLI/CI tokens issued prior to Sprint 23 (AUTH-POLICY-23-001) must drop `policy:write`/`policy:submit`/`policy:edit` and instead request `policy:read`, `policy:author`, `policy:review`, and `policy:simulate` (plus `policy:approve`/`policy:operate`/`policy:activate` for promotion pipelines).
--- ---
## 1 · Global Options & Output Modes ## 1 · Global Options & Output Modes
All `stella policy *` commands honour the common CLI options: All `stella policy *` commands honour the common CLI options:
| Flag | Default | Description | | Flag | Default | Description |
|------|---------|-------------| |------|---------|-------------|
| `--server <url>` | `https://stella.local` | Policy Engine gateway root. | | `--server <url>` | `https://stella.local` | Policy Engine gateway root. |
| `--tenant <id>` | token default | Override tenant for multi-tenant installs. | | `--tenant <id>` | token default | Override tenant for multi-tenant installs. |
| `--format <table\|json\|yaml>` | `table` for TTY, `json` otherwise | Output format for listings/diffs. | | `--format <table\|json\|yaml>` | `table` for TTY, `json` otherwise | Output format for listings/diffs. |
| `--output <file>` | stdout | Write full JSON payload to file. | | `--output <file>` | stdout | Write full JSON payload to file. |
| `--sealed` | false | Force sealed-mode behaviour (no outbound fetch). | | `--sealed` | false | Force sealed-mode behaviour (no outbound fetch). |
| `--trace` | false | Emit verbose timing/log correlation info. | | `--trace` | false | Emit verbose timing/log correlation info. |
> **Tip:** Set `STELLA_PROFILE=policy` in CI to load saved defaults from `~/.stella/profiles/policy.toml`. > **Tip:** Set `STELLA_PROFILE=policy` in CI to load saved defaults from `~/.stella/profiles/policy.toml`.
--- ---
## 2 · Authoring & Drafting Commands ## 2 · Authoring & Drafting Commands
### 2.1 `stella policy new` ### 2.1 `stella policy new`
Create a draft policy from a template or scratch. Create a draft policy from a template or scratch.
``` ```
stella policy new --policy-id P-7 --name "Default Org Policy" \ stella policy new --policy-id P-7 --name "Default Org Policy" \
--template baseline --output-path policies/P-7.stella --template baseline --output-path policies/P-7.stella
``` ```
Options: Options:
| Flag | Description | | Flag | Description |
|------|-------------| |------|-------------|
| `--policy-id` *(required)* | Stable identifier (e.g., `P-7`). | | `--policy-id` *(required)* | Stable identifier (e.g., `P-7`). |
| `--name` | Friendly display name. | | `--name` | Friendly display name. |
| `--template` | `baseline`, `serverless`, `blank`. | | `--template` | `baseline`, `serverless`, `blank`. |
| `--from` | Start from existing version (`policyId@version`). | | `--from` | Start from existing version (`policyId@version`). |
| `--open` | Launches `$EDITOR` after creation. | | `--open` | Launches `$EDITOR` after creation. |
Writes DSL to local file and registers draft version (`status=draft`). Requires `policy:write`. Writes DSL to local file and registers draft version (`status=draft`). Requires `policy:write`.
### 2.2 `stella policy edit` ### 2.2 `stella policy edit`
Open an existing draft in the local editor. Open an existing draft in the local editor.
``` ```
stella policy edit P-7 --version 4 stella policy edit P-7 --version 4
``` ```
- Auto-checks out latest draft if `--version` omitted. - Auto-checks out latest draft if `--version` omitted.
- Saves to temp file, uploads on editor exit (unless `--no-upload`). - Saves to temp file, uploads on editor exit (unless `--no-upload`).
- Use `--watch` to keep command alive and re-upload on every save. - Use `--watch` to keep command alive and re-upload on every save.
### 2.3 `stella policy lint` ### 2.3 `stella policy lint`
Static validation without submitting. Static validation without submitting.
``` ```
stella policy lint policies/P-7.stella --format json stella policy lint policies/P-7.stella --format json
``` ```
Outputs diagnostics (line/column, code, message). Exit codes: Outputs diagnostics (line/column, code, message). Exit codes:
| Code | Meaning | | Code | Meaning |
|------|---------| |------|---------|
| `0` | No lint errors. | | `0` | No lint errors. |
| `10` | Syntax/compile errors (`ERR_POL_001`). | | `10` | Syntax/compile errors (`ERR_POL_001`). |
| `11` | Unsupported syntax version. | | `11` | Unsupported syntax version. |
### 2.4 `stella policy compile` ### 2.4 `stella policy compile`
Emits IR digest and rule summary. Emits IR digest and rule summary.
``` ```
stella policy compile P-7 --version 4 stella policy compile P-7 --version 4
``` ```
Returns JSON with `digest`, `rules.count`, action counts. Exit `0` success, `10` on compile errors. Returns JSON with `digest`, `rules.count`, action counts. Exit `0` success, `10` on compile errors.
--- ---
## 3 · Lifecycle Workflow ## 3 · Lifecycle Workflow
### 3.1 Submit ### 3.1 Submit
``` ```
stella policy submit P-7 --version 4 \ stella policy submit P-7 --version 4 \
--reviewer user:kay --reviewer group:sec-reviewers \ --reviewer user:kay --reviewer group:sec-reviewers \
--note "Simulated against golden SBOM set" \ --note "Simulated against golden SBOM set" \
--attach sims/P-7-v4-vs-v3.json --attach sims/P-7-v4-vs-v3.json
``` ```
Requires `policy:submit`. CLI validates that lint/compile run within 24 h and bundle attachments exist. Requires `policy:submit`. CLI validates that lint/compile run within 24 h and bundle attachments exist.
### 3.2 Review ### 3.2 Review
``` ```
stella policy review P-7 --version 4 --approve \ stella policy review P-7 --version 4 --approve \
--note "Looks good; ensure incident playbook updated." --note "Looks good; ensure incident playbook updated."
``` ```
- `--approve`, `--request-changes`, or `--comment`. - `--approve`, `--request-changes`, or `--comment`.
- Provide `--blocking` to mark comment as blocking. - Provide `--blocking` to mark comment as blocking.
- Requires `policy:review`. - Requires `policy:review`.
### 3.3 Approve ### 3.3 Approve
``` ```
stella policy approve P-7 --version 4 \ stella policy approve P-7 --version 4 \
--note "Determinism CI green; simulation diff attached." \ --note "Determinism CI green; simulation diff attached." \
--attach sims/P-7-v4-vs-v3.json --attach sims/P-7-v4-vs-v3.json
``` ```
Prompts for confirmation; refuses if approver == submitter. Requires `policy:approve`. Prompts for confirmation; refuses if approver == submitter. Requires `policy:approve`.
### 3.4 Activate ### 3.4 Activate
``` ```
stella policy activate P-7 --version 4 --run-now --priority high stella policy activate P-7 --version 4 --run-now --priority high
``` ```
- Optional `--scheduled-at 2025-10-27T02:00:00Z`. - Optional `--scheduled-at 2025-10-27T02:00:00Z`.
- Requires `policy:activate` and `policy:run`. - Requires `policy:activate` and `policy:run`.
**Options** **Options**
- `--version <number>` (required) – target revision to promote. - `--version <number>` (required) – target revision to promote.
- `--note <text>` – record an activation note alongside the approval. - `--note <text>` – record an activation note alongside the approval.
- `--run-now` – enqueue an immediate full run after activation. - `--run-now` – enqueue an immediate full run after activation.
- `--scheduled-at <timestamp>` – schedule activation for a specific UTC time (ISO-8601 format). - `--scheduled-at <timestamp>` – schedule activation for a specific UTC time (ISO-8601 format).
- `--priority <label>` – optional scheduling priority hint (`low`, `standard`, `high`). - `--priority <label>` – optional scheduling priority hint (`low`, `standard`, `high`).
- `--rollback` – mark the activation as a rollback of a previously active version. - `--rollback` – mark the activation as a rollback of a previously active version.
- `--incident <id>` – associate the activation with an incident identifier. - `--incident <id>` – associate the activation with an incident identifier.
**Exit codes** **Exit codes**
| Code | Meaning | | Code | Meaning |
|------|---------| |------|---------|
| `0` | Activation completed (or policy already active). | | `0` | Activation completed (or policy already active). |
| `75` | Activation recorded but awaiting a second approver. | | `75` | Activation recorded but awaiting a second approver. |
### 3.5 Archive / Rollback ### 3.5 Archive / Rollback
``` ```
stella policy archive P-7 --version 3 --reason "Superseded by v4" stella policy archive P-7 --version 3 --reason "Superseded by v4"
stella policy activate P-7 --version 3 --rollback --incident INC-2025-104 stella policy activate P-7 --version 3 --rollback --incident INC-2025-104
``` ```
--- ---
## 4 · Simulation & Runs ## 4 · Simulation & Runs
### 4.1 Simulate ### 4.1 Simulate
``` ```
stella policy simulate P-7 \ stella policy simulate P-7 \
--base 3 --candidate 4 \ --base 3 --candidate 4 \
--sbom sbom:S-42 --sbom sbom:S-318 \ --sbom sbom:S-42 --sbom sbom:S-318 \
--env exposure=internet --env sealed=false \ --env exposure=internet --env sealed=false \
--format json --output sims/P-7-v4-vs-v3.json --format json --output sims/P-7-v4-vs-v3.json
``` ```
Output fields (JSON): Output fields (JSON):
```json ```json
{ {
"diff": { "diff": {
"added": 12, "added": 12,
"removed": 8, "removed": 8,
"unchanged": 657, "unchanged": 657,
"bySeverity": { "bySeverity": {
"Critical": {"up": 1, "down": 0}, "Critical": {"up": 1, "down": 0},
"High": {"up": 3, "down": 4} "High": {"up": 3, "down": 4}
} }
}, },
"explainUri": "blob://policy/P-7/simulations/2025-10-26.json" "explainUri": "blob://policy/P-7/simulations/2025-10-26.json"
} }
``` ```
> Schema reminder: CLI commands surface objects defined in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-20-001-POLICY-RUNS.md`; use the samples in `samples/api/scheduler/` for contract validation when extending output parsing. > Schema reminder: CLI commands surface objects defined in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-20-001-POLICY-RUNS.md`; use the samples in `samples/api/scheduler/` for contract validation when extending output parsing.
Exit codes: Exit codes:
| Code | Meaning | | Code | Meaning |
|------|---------| |------|---------|
| `0` | Simulation succeeded; diffs informational. | | `0` | Simulation succeeded; diffs informational. |
| `20` | Blocking delta (`--fail-on-diff` triggered). | | `20` | Blocking delta (`--fail-on-diff` triggered). |
| `21` | Simulation input missing (`ERR_POL_003`). | | `21` | Simulation input missing (`ERR_POL_003`). |
| `22` | Determinism guard (`ERR_POL_004`). | | `22` | Determinism guard (`ERR_POL_004`). |
| `23` | API/permission error (`ERR_POL_002`, `ERR_POL_005`). | | `23` | API/permission error (`ERR_POL_002`, `ERR_POL_005`). |
### 4.2 Run ### 4.2 Run
``` ```
stella policy run P-7 --mode full \ stella policy run P-7 --mode full \
--sbom sbom:S-42 --env exposure=internal-only \ --sbom sbom:S-42 --env exposure=internal-only \
--wait --watch --wait --watch
``` ```
Options: Options:
| Flag | Description | | Flag | Description |
|------|-------------| |------|-------------|
| `--mode` | `full` or `incremental` (default incremental). | | `--mode` | `full` or `incremental` (default incremental). |
| `--sbom` | Explicit SBOM IDs (optional). | | `--sbom` | Explicit SBOM IDs (optional). |
| `--priority` | `normal`, `high`, `emergency`. | | `--priority` | `normal`, `high`, `emergency`. |
| `--wait` | Poll run status until completion. | | `--wait` | Poll run status until completion. |
| `--watch` | Stream progress events (requires TTY). | | `--watch` | Stream progress events (requires TTY). |
`stella policy run status <runId>` retrieves run metadata. `stella policy run status <runId>` retrieves run metadata.
`stella policy run list --status failed --limit 20` returns recent runs. `stella policy run list --status failed --limit 20` returns recent runs.
### 4.3 Replay & Cancel ### 4.3 Replay & Cancel
``` ```
stella policy run replay run:P-7:2025-10-26:auto --output bundles/replay.tgz stella policy run replay run:P-7:2025-10-26:auto --output bundles/replay.tgz
stella policy run cancel run:P-7:2025-10-26:auto stella policy run cancel run:P-7:2025-10-26:auto
``` ```
Replay downloads sealed bundle for deterministic verification. Replay downloads sealed bundle for deterministic verification.
### 4.4 Schema artefacts for CLI validation ### 4.4 Schema artefacts for CLI validation
- CI publishes canonical JSON Schema exports for `PolicyRunRequest`, `PolicyRunStatus`, `PolicyDiffSummary`, and `PolicyExplainTrace` as the `policy-schema-exports` artifact (see `.gitea/workflows/build-test-deploy.yml`). - CI publishes canonical JSON Schema exports for `PolicyRunRequest`, `PolicyRunStatus`, `PolicyDiffSummary`, and `PolicyExplainTrace` as the `policy-schema-exports` artifact (see `.gitea/workflows/build-test-deploy.yml`).
- Each run writes the files to `artifacts/policy-schemas/<commit>/` and stores a unified diff (`policy-schema-diff.patch`) comparing them with the tracked baseline in `docs/schemas/`. - Each run writes the files to `artifacts/policy-schemas/<commit>/` and stores a unified diff (`policy-schema-diff.patch`) comparing them with the tracked baseline in `docs/schemas/`.
- Schema changes trigger an alert in Slack `#policy-engine` via the `POLICY_ENGINE_SCHEMA_WEBHOOK` secret so CLI maintainers know to refresh fixtures or validation rules. - Schema changes trigger an alert in Slack `#policy-engine` via the `POLICY_ENGINE_SCHEMA_WEBHOOK` secret so CLI maintainers know to refresh fixtures or validation rules.
- Consume these artefacts in CLI tests to keep payload validation aligned without committing generated files into the repo. - Consume these artefacts in CLI tests to keep payload validation aligned without committing generated files into the repo.
--- ---
## 5 · Findings & Explainability ## 5 · Findings & Explainability
### 5.1 List Findings ### 5.1 List Findings
``` ```
stella findings ls --policy P-7 \ stella findings ls --policy P-7 \
--sbom sbom:S-42 \ --sbom sbom:S-42 \
--status affected --severity High,Critical \ --status affected --severity High,Critical \
--since 2025-10-01T00:00:00Z \ --since 2025-10-01T00:00:00Z \
--page 2 --page-size 100 \ --page 2 --page-size 100 \
--format table --format table
``` ```
Common flags: Common flags:
| Flag | Description | | Flag | Description |
|------|-------------| |------|-------------|
| `--sbom` | Repeatable filter for SBOM identifiers. | | `--sbom` | Repeatable filter for SBOM identifiers. |
| `--status` | Repeatable filter (`affected`, `quieted`, `mitigated`, `not_affected`, etc.). | | `--status` | Repeatable filter (`affected`, `quieted`, `mitigated`, `not_affected`, etc.). |
| `--severity` | Repeatable filter using normalized labels (`Critical`, `High`, `Medium`, `Low`, `Unknown`). | | `--severity` | Repeatable filter using normalized labels (`Critical`, `High`, `Medium`, `Low`, `Unknown`). |
| `--since` | Return findings updated on/after the ISO-8601 timestamp. | | `--since` | Return findings updated on/after the ISO-8601 timestamp. |
| `--cursor` | Resume listing using the opaque token from a prior page. | | `--cursor` | Resume listing using the opaque token from a prior page. |
| `--page`, `--page-size` | Page-based pagination (page >=1, size <=500; falls back to backend defaults). | | `--page`, `--page-size` | Page-based pagination (page >=1, size <=500; falls back to backend defaults). |
| `--output` | Persist JSON payload to disk (implied JSON rendering). | | `--output` | Persist JSON payload to disk (implied JSON rendering). |
| `--format` | `table` (default for TTY) or `json`. | | `--format` | `table` (default for TTY) or `json`. |
### 5.2 Fetch Explain ### 5.2 Fetch Explain
``` ```
stella findings explain --policy P-7 \ stella findings explain --policy P-7 \
P-7:S-42:pkg:npm/lodash@4.17.21:CVE-2021-23337 \ P-7:S-42:pkg:npm/lodash@4.17.21:CVE-2021-23337 \
--mode verbose \ --mode verbose \
--format json --output explains/lodash.json --format json --output explains/lodash.json
``` ```
Outputs ordered rule hits, inputs, evidence snapshots, and sealed-mode hints. Supported `--mode` values mirror API contracts (for example `summary`, `verbose`); omit to use backend default. Outputs ordered rule hits, inputs, evidence snapshots, and sealed-mode hints. Supported `--mode` values mirror API contracts (for example `summary`, `verbose`); omit to use backend default.
--- ---
## 6 · Exit Codes Summary ## 6 · Exit Codes Summary
| Exit code | Description | Typical ERR codes | | Exit code | Description | Typical ERR codes |
|-----------|-------------|-------------------| |-----------|-------------|-------------------|
| `0` | Success (command completed, warnings only). | — | | `0` | Success (command completed, warnings only). | — |
| `10` | DSL syntax/compile failure. | `ERR_POL_001` | | `10` | DSL syntax/compile failure. | `ERR_POL_001` |
| `11` | Unsupported DSL version / schema mismatch. | `ERR_POL_001` | | `11` | Unsupported DSL version / schema mismatch. | `ERR_POL_001` |
| `12` | Approval/rbac failure. | `ERR_POL_002`, `ERR_POL_005` | | `12` | Approval/rbac failure. | `ERR_POL_002`, `ERR_POL_005` |
| `20` | Simulation diff exceeded thresholds (`--fail-on-diff`). | — | | `20` | Simulation diff exceeded thresholds (`--fail-on-diff`). | — |
| `21` | Required inputs missing (SBOM/advisory/VEX). | `ERR_POL_003` | | `21` | Required inputs missing (SBOM/advisory/VEX). | `ERR_POL_003` |
| `22` | Determinism guard triggered. | `ERR_POL_004` | | `22` | Determinism guard triggered. | `ERR_POL_004` |
| `23` | Run canceled or timed out. | `ERR_POL_006` | | `23` | Run canceled or timed out. | `ERR_POL_006` |
| `30` | Network/transport error (non-HTTP success). | — | | `30` | Network/transport error (non-HTTP success). | — |
| `64` | CLI usage error (invalid flag/argument). | — | | `64` | CLI usage error (invalid flag/argument). | — |
All non-zero exits emit structured error envelope on stderr when `--format json` or `STELLA_JSON_ERRORS=1`. All non-zero exits emit structured error envelope on stderr when `--format json` or `STELLA_JSON_ERRORS=1`.
--- ---
## 7 · Offline & Air-Gap Usage ## 7 · Offline & Air-Gap Usage
- Use `--sealed` to ensure commands avoid outbound calls; required for sealed enclaves. - Use `--sealed` to ensure commands avoid outbound calls; required for sealed enclaves.
- `stella policy bundle export --policy P-7 --version 4 --output bundles/policy-P-7-v4.bundle` pairs with Offline Kit import. - `stella policy bundle export --policy P-7 --version 4 --output bundles/policy-P-7-v4.bundle` pairs with Offline Kit import.
- Replay bundles (`run replay`) are DSSE-signed; verify with `stella offline verify`. - Replay bundles (`run replay`) are DSSE-signed; verify with `stella offline verify`.
- Store credentials in `~/.stella/offline.toml` for non-interactive air-gapped pipelines. - Store credentials in `~/.stella/offline.toml` for non-interactive air-gapped pipelines.
--- ---
## 8 · Compliance Checklist ## 8 · Compliance Checklist
- [ ] **Help text synced:** `stella policy --help` matches documented flags/examples (update during release pipeline). - [ ] **Help text synced:** `stella policy --help` matches documented flags/examples (update during release pipeline).
- [ ] **Exit codes mapped:** Table above reflects CLI implementation and CI asserts mapping for `ERR_POL_*`. - [ ] **Exit codes mapped:** Table above reflects CLI implementation and CI asserts mapping for `ERR_POL_*`.
- [ ] **JSON schemas verified:** Example payloads validated against OpenAPI/SDK contracts before publishing. (_CI now exports canonical schemas as `policy-schema-exports`; wire tests to consume them._) - [ ] **JSON schemas verified:** Example payloads validated against OpenAPI/SDK contracts before publishing. (_CI now exports canonical schemas as `policy-schema-exports`; wire tests to consume them._)
- [ ] **Scope guidance present:** Each command lists required Authority scopes. - [ ] **Scope guidance present:** Each command lists required Authority scopes.
- [ ] **Offline guidance included:** Sealed-mode steps and bundle workflows documented. - [ ] **Offline guidance included:** Sealed-mode steps and bundle workflows documented.
- [ ] **Cross-links tested:** Links to DSL, lifecycle, runs, and API docs resolve locally (`yarn docs:lint`). - [ ] **Cross-links tested:** Links to DSL, lifecycle, runs, and API docs resolve locally (`yarn docs:lint`).
- [ ] **Examples no-op safe:** Command examples either read-only or use placeholders (no destructive defaults). - [ ] **Examples no-op safe:** Command examples either read-only or use placeholders (no destructive defaults).
--- ---
*Last updated: 2025-10-27 (Sprint 20).* *Last updated: 2025-10-27 (Sprint 20).*

46
docs/modules/cli/implementation_plan.md
View File

@@ -1,23 +1,23 @@
# Implementation plan — CLI # Implementation plan — CLI
## Current objectives ## Current objectives
- Maintain deterministic behaviour and offline parity across releases. - Maintain deterministic behaviour and offline parity across releases.
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes. - Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
## Workstreams ## Workstreams
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap. - Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs. - Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
- Validation: extend tests/fixtures to preserve determinism and provenance requirements. - Validation: extend tests/fixtures to preserve determinism and provenance requirements.
## Epic milestones ## Epic milestones
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy verbs, simulation, and explain outputs. - **Epic 2 – Policy Engine & Editor:** deliver deterministic policy verbs, simulation, and explain outputs.
- **Epic 4 – Policy Studio:** add registry/promotion workflows, lint tooling, and approvals UX. - **Epic 4 – Policy Studio:** add registry/promotion workflows, lint tooling, and approvals UX.
- **Epic 6 – Vulnerability Explorer:** integrate ledger/triage operations. - **Epic 6 – Vulnerability Explorer:** integrate ledger/triage operations.
- **Epic 10 – Export Center:** automate export verification and Offline Kit flows. - **Epic 10 – Export Center:** automate export verification and Offline Kit flows.
- **Epic 11 – Notifications Studio:** manage rule/channel authoring and previews via CLI. - **Epic 11 – Notifications Studio:** manage rule/channel authoring and previews via CLI.
- Track CLI-specific work (e.g., CLI-CORE-41-001, DOCS-CLI-OBS-52-001) in ../../TASKS.md and src/Cli/**/TASKS.md. - Track CLI-specific work (e.g., CLI-CORE-41-001, DOCS-CLI-OBS-52-001) in ../../TASKS.md and src/Cli/**/TASKS.md.
## Coordination ## Coordination
- Review ./AGENTS.md before picking up new work. - Review ./AGENTS.md before picking up new work.
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md. - Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
- Update this plan whenever scope, dependencies, or guardrails change. - Update this plan whenever scope, dependencies, or guardrails change.

268
docs/modules/cli/operations/release-and-packaging.md
View File

@@ -1,134 +1,134 @@
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. > **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
# CLI Release & Packaging Runbook # CLI Release & Packaging Runbook
This runbook describes how to build, sign, package, and distribute the StellaOps CLI with Task Pack support. It covers connected and air-gapped workflows, SBOM generation, parity gating, and distribution artifacts required by Sprint 43 (`DEVOPS-CLI-43-001`, `DEPLOY-PACKS-43-001`). This runbook describes how to build, sign, package, and distribute the StellaOps CLI with Task Pack support. It covers connected and air-gapped workflows, SBOM generation, parity gating, and distribution artifacts required by Sprint 43 (`DEVOPS-CLI-43-001`, `DEPLOY-PACKS-43-001`).
--- ---
## 1 · Release Artifacts ## 1 · Release Artifacts
| Artifact | Description | Notes | | Artifact | Description | Notes |
|----------|-------------|-------| |----------|-------------|-------|
| `stella-<version>-linux-x64.tar.gz` | Linux binary + completions | Includes man pages, localization files. | | `stella-<version>-linux-x64.tar.gz` | Linux binary + completions | Includes man pages, localization files. |
| `stella-<version>-macos-universal.tar.gz` | macOS universal binary | Signed/notarized where applicable. | | `stella-<version>-macos-universal.tar.gz` | macOS universal binary | Signed/notarized where applicable. |
| `stella-<version>-windows-x64.zip` | Windows binary + PowerShell modules | Code-signed. | | `stella-<version>-windows-x64.zip` | Windows binary + PowerShell modules | Code-signed. |
| `stella-cli-container:<version>` | OCI image with CLI + pack runtime | Deterministic rootfs (scratch/distroless). | | `stella-cli-container:<version>` | OCI image with CLI + pack runtime | Deterministic rootfs (scratch/distroless). |
| SBOM (`.cdx.json`) | CycloneDX SBOM per artifact | Generated via `stella sbom generate` or `syft`. | | SBOM (`.cdx.json`) | CycloneDX SBOM per artifact | Generated via `stella sbom generate` or `syft`. |
| Checksums (`SHA256SUMS`) | Aggregated digest list | Signed with cosign. | | Checksums (`SHA256SUMS`) | Aggregated digest list | Signed with cosign. |
| Provenance (`.intoto.jsonl`) | DSSE attestation (SLSA L2) | Contains build metadata. | | Provenance (`.intoto.jsonl`) | DSSE attestation (SLSA L2) | Contains build metadata. |
| Release notes | Markdown summary | Links to task packs docs, parity matrix. | | Release notes | Markdown summary | Links to task packs docs, parity matrix. |
--- ---
## 2 · Build Pipeline ## 2 · Build Pipeline
1. **Source checkout** – pinned commit, reproducible environment (Docker). 1. **Source checkout** – pinned commit, reproducible environment (Docker).
2. **Dependency lock** – `dotnet restore`, `npm ci` (for CLI frontends), ensure deterministic build flags. 2. **Dependency lock** – `dotnet restore`, `npm ci` (for CLI frontends), ensure deterministic build flags.
3. **Build binaries** – cross-platform targets with reproducible timestamps. 3. **Build binaries** – cross-platform targets with reproducible timestamps.
4. **Run tests** – unit + integration; include `stella pack` commands (plan/run/verify) in CI. 4. **Run tests** – unit + integration; include `stella pack` commands (plan/run/verify) in CI.
5. **Generate SBOM** – `syft packages dist/stella-linux-x64 --output cyclonedx-json`. 5. **Generate SBOM** – `syft packages dist/stella-linux-x64 --output cyclonedx-json`.
6. **Bundle** – compress artifacts, include completions (`bash`, `zsh`, `fish`, PowerShell). 6. **Bundle** – compress artifacts, include completions (`bash`, `zsh`, `fish`, PowerShell).
7. **Sign** – cosign signatures for binaries, checksums, container image. 7. **Sign** – cosign signatures for binaries, checksums, container image.
8. **Publish** – upload to `downloads.stella-ops.org`, container registry, Packs Registry (for CLI container). 8. **Publish** – upload to `downloads.stella-ops.org`, container registry, Packs Registry (for CLI container).
9. **Parity gating** – run CLI parity matrix tests vs Console features (automation in `DEVOPS-CLI-43-001`). 9. **Parity gating** – run CLI parity matrix tests vs Console features (automation in `DEVOPS-CLI-43-001`).
CI must run in isolated environment (no network beyond allowlist). Cache dependencies for offline bundling. CI must run in isolated environment (no network beyond allowlist). Cache dependencies for offline bundling.
--- ---
## 3 · Versioning & Channels ## 3 · Versioning & Channels
- Semantic versioning (`YYYY.MM.patch`), e.g., `2025.10.0`. - Semantic versioning (`YYYY.MM.patch`), e.g., `2025.10.0`.
- Channels: - Channels:
- `edge` – nightly builds, limited support. - `edge` – nightly builds, limited support.
- `beta` – pre-release candidates. - `beta` – pre-release candidates.
- `stable` – production-ready, after parity gating. - `stable` – production-ready, after parity gating.
- Release promotions mirror Task Pack channels; update downloads manifest (`deploy/downloads/manifest.json`). - Release promotions mirror Task Pack channels; update downloads manifest (`deploy/downloads/manifest.json`).
--- ---
## 4 · Signing & Verification ## 4 · Signing & Verification
- Binaries signed with cosign (`cosign sign-blob`). - Binaries signed with cosign (`cosign sign-blob`).
- Container image signed (`cosign sign stella-cli-container:<version>`). - Container image signed (`cosign sign stella-cli-container:<version>`).
- DSSE provenance includes: - DSSE provenance includes:
- Build pipeline ID. - Build pipeline ID.
- Source commit and repo. - Source commit and repo.
- Dependencies SBOM digest. - Dependencies SBOM digest.
- Test results summary. - Test results summary.
- Verification command for operators: - Verification command for operators:
```bash ```bash
cosign verify-blob \ cosign verify-blob \
--certificate-identity https://ci.stella-ops.org \ --certificate-identity https://ci.stella-ops.org \
--certificate-oidc-issuer https://fulcio.sigstore.dev \ --certificate-oidc-issuer https://fulcio.sigstore.dev \
--signature stella-2025.10.0-linux-x64.sig \ --signature stella-2025.10.0-linux-x64.sig \
stella-2025.10.0-linux-x64.tar.gz stella-2025.10.0-linux-x64.tar.gz
``` ```
--- ---
## 5 · Distribution ## 5 · Distribution
### 5.1 Online ### 5.1 Online
- Publish artifacts to Downloads service; update manifest with digests, SBOM URLs, attestations. - Publish artifacts to Downloads service; update manifest with digests, SBOM URLs, attestations.
- Update CLI parity docs (`docs/cli-vs-ui-parity.md`) and release notes. - Update CLI parity docs (`docs/cli-vs-ui-parity.md`) and release notes.
- Push container image to registry with SBOM + attestations referenced as OCI referrers. - Push container image to registry with SBOM + attestations referenced as OCI referrers.
- Notify stakeholders via `#release-cli` channel and release mailing list. - Notify stakeholders via `#release-cli` channel and release mailing list.
### 5.2 Offline / Air-Gap ### 5.2 Offline / Air-Gap
- Bundle CLI artifacts, Task Pack samples, and registry mirror: - Bundle CLI artifacts, Task Pack samples, and registry mirror:
```bash ```bash
stella pack bundle export \ stella pack bundle export \
--packs "sbom-remediation:1.3.0" \ --packs "sbom-remediation:1.3.0" \
--output offline/packs-bundle-2025.10.0.tgz --output offline/packs-bundle-2025.10.0.tgz
stella cli bundle export \ stella cli bundle export \
--output offline/cli-2025.10.0.tgz \ --output offline/cli-2025.10.0.tgz \
--include-container \ --include-container \
--include-sbom --include-sbom
``` ```
- Update Offline Kit manifest with new CLI version and pack bundle entries. - Update Offline Kit manifest with new CLI version and pack bundle entries.
- Provide import scripts (`ouk import`) for sealed sites. - Provide import scripts (`ouk import`) for sealed sites.
--- ---
## 6 · Parity Gating ## 6 · Parity Gating
- `stella cli parity check` compares CLI commands vs parity matrix. - `stella cli parity check` compares CLI commands vs parity matrix.
- CI fails release if any required command flagged `🟥` or `🟡` with severity > threshold. - CI fails release if any required command flagged `🟥` or `🟡` with severity > threshold.
- Parity report uploaded to Downloads workspace and linked in docs. - Parity report uploaded to Downloads workspace and linked in docs.
- Manual review required for new commands (ensure `man` pages and help text localized). - Manual review required for new commands (ensure `man` pages and help text localized).
--- ---
## 7 · Localization & Documentation ## 7 · Localization & Documentation
- CLI includes localization bundles; ensure `i18n.txz` packaged. - CLI includes localization bundles; ensure `i18n.txz` packaged.
- Update man pages (`man/stella-pack.1`) and HTML docs. - Update man pages (`man/stella-pack.1`) and HTML docs.
- Sync docs: `docs/modules/cli/guides/overview.md`, pack authoring guide, release notes. - Sync docs: `docs/modules/cli/guides/overview.md`, pack authoring guide, release notes.
- Document new flags/commands in `docs/modules/cli/guides/commands/pack.md` (tracked in Sprint 42 tasks). - Document new flags/commands in `docs/modules/cli/guides/commands/pack.md` (tracked in Sprint 42 tasks).
--- ---
## 8 · Release Checklist ## 8 · Release Checklist
- [ ] All binaries built reproducibly (CI logs archived). - [ ] All binaries built reproducibly (CI logs archived).
- [ ] Tests + parity matrix passing. - [ ] Tests + parity matrix passing.
- [ ] SBOM + provenance generated and published. - [ ] SBOM + provenance generated and published.
- [ ] Cosign signatures created and verified. - [ ] Cosign signatures created and verified.
- [ ] Downloads manifest updated (edge/beta/stable). - [ ] Downloads manifest updated (edge/beta/stable).
- [ ] Offline bundle exported and validated. - [ ] Offline bundle exported and validated.
- [ ] Release notes + documentation updates merged. - [ ] Release notes + documentation updates merged.
- [ ] Notifications sent (chat/email). - [ ] Notifications sent (chat/email).
- [ ] Imposed rule reminder present at top of document. - [ ] Imposed rule reminder present at top of document.
--- ---
*Last updated: 2025-10-27 (Sprint 43).* *Last updated: 2025-10-27 (Sprint 43).*

42
docs/modules/concelier/AGENTS.md
View File

@@ -1,22 +1,22 @@
# Concelier agent guide # Concelier agent guide
## Mission ## Mission
Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC). Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC).
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

72
docs/modules/concelier/README.md
View File

@@ -1,36 +1,36 @@
# StellaOps Concelier # StellaOps Concelier
Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC). Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC).
## Responsibilities ## Responsibilities
- Fetch and normalise vulnerability advisories via restart-time connectors. - Fetch and normalise vulnerability advisories via restart-time connectors.
- Persist observations and correlation linksets without precedence decisions. - Persist observations and correlation linksets without precedence decisions.
- Emit deterministic exports (JSON, Trivy DB) for downstream policy evaluation. - Emit deterministic exports (JSON, Trivy DB) for downstream policy evaluation.
- Coordinate offline/air-gap updates via Offline Kit bundles. - Coordinate offline/air-gap updates via Offline Kit bundles.
## Key components ## Key components
- `StellaOps.Concelier.WebService` orchestration host. - `StellaOps.Concelier.WebService` orchestration host.
- Connector libraries under `StellaOps.Concelier.Connector.*`. - Connector libraries under `StellaOps.Concelier.Connector.*`.
- Exporter packages (`StellaOps.Concelier.Exporter.*`). - Exporter packages (`StellaOps.Concelier.Exporter.*`).
## Integrations & dependencies ## Integrations & dependencies
- MongoDB for canonical observations and schedules. - MongoDB for canonical observations and schedules.
- Policy Engine / Export Center / CLI for evidence consumption. - Policy Engine / Export Center / CLI for evidence consumption.
- Notify and UI for advisory deltas. - Notify and UI for advisory deltas.
## Operational notes ## Operational notes
- Connector runbooks in ./operations/connectors/. - Connector runbooks in ./operations/connectors/.
- Mirror operations for Offline Kit parity. - Mirror operations for Offline Kit parity.
- Grafana dashboards for connector health. - Grafana dashboards for connector health.
## Related resources ## Related resources
- ./operations/conflict-resolution.md - ./operations/conflict-resolution.md
- ./operations/mirror.md - ./operations/mirror.md
## Backlog references ## Backlog references
- DOCS-LNM-22-001, DOCS-LNM-22-007 in ../../TASKS.md. - DOCS-LNM-22-001, DOCS-LNM-22-007 in ../../TASKS.md.
- Connector-specific TODOs in `src/Concelier/**/TASKS.md`. - Connector-specific TODOs in `src/Concelier/**/TASKS.md`.
## Epic alignment ## Epic alignment
- **Epic 1 – AOC enforcement:** uphold raw observation invariants, provenance requirements, linkset-only enrichment, and AOC verifier guardrails across every connector. - **Epic 1 – AOC enforcement:** uphold raw observation invariants, provenance requirements, linkset-only enrichment, and AOC verifier guardrails across every connector.
- **Epic 10 – Export Center:** expose deterministic advisory exports and metadata required by JSON/Trivy/mirror bundles. - **Epic 10 – Export Center:** expose deterministic advisory exports and metadata required by JSON/Trivy/mirror bundles.

18
docs/modules/concelier/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — Concelier # Task board — Concelier
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| CONCELIER-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | CONCELIER-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| CONCELIER-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | CONCELIER-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| CONCELIER-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | CONCELIER-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

1185
docs/modules/concelier/architecture.md
View File

File diff suppressed because it is too large Load Diff

134
docs/modules/concelier/implementation_plan.md
View File

@@ -1,67 +1,67 @@
# Implementation plan — Concelier # Implementation plan — Concelier
## Delivery timeline ## Delivery timeline
- **Phase 1 — Guardrails & schema** - **Phase 1 — Guardrails & schema**
Stand up Mongo JSON validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema. Stand up Mongo JSON validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
- **Phase 2 — API & observability** - **Phase 2 — API & observability**
Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps. Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps.
- **Phase 3 — Experience polish** - **Phase 3 — Experience polish**
Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures). Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures).
## Work breakdown by component ## Work breakdown by component
- **Concelier WebService & worker** - **Concelier WebService & worker**
- Add Mongo validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`. - Add Mongo validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
- Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts. - Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts.
- Deterministically compute linksets and persist canonical JSON payloads. - Deterministically compute linksets and persist canonical JSON payloads.
- Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes. - Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes.
- Emit guard metrics/traces and surface supersedes/violation audit logs. - Emit guard metrics/traces and surface supersedes/violation audit logs.
- **Excititor (shared ingestion contract)** - **Excititor (shared ingestion contract)**
- Mirror Concelier guard and schema changes for `vex_raw`. - Mirror Concelier guard and schema changes for `vex_raw`.
- Maintain restart-time plug-in determinism and linkset extraction parity. - Maintain restart-time plug-in determinism and linkset extraction parity.
- **Shared libraries** - **Shared libraries**
- Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification). - Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification).
- Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities. - Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities.
- **Policy Engine integration** - **Policy Engine integration**
- Enforce `effective_finding_*` write exclusivity. - Enforce `effective_finding_*` write exclusivity.
- Consume only raw documents + linksets, removing any implicit normalisation. - Consume only raw documents + linksets, removing any implicit normalisation.
- **Authority scopes** - **Authority scopes**
- Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services. - Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services.
- **CLI & Console** - **CLI & Console**
- Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`). - Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`).
- Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console. - Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console.
- **CI/CD** - **CI/CD**
- Add Roslyn analyzer / AST lint to block forbidden writes. - Add Roslyn analyzer / AST lint to block forbidden writes.
- Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating. - Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating.
## Documentation deliverables ## Documentation deliverables
- Update `docs/ingestion/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance. - Update `docs/ingestion/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
- Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards. - Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards.
- Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes. - Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes.
- Ensure Offline Kit documentation captures validator bootstrap and verify workflows. - Ensure Offline Kit documentation captures validator bootstrap and verify workflows.
## Acceptance criteria ## Acceptance criteria
- Mongo validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes. - Mongo validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
- Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents. - Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents.
- CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status. - CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status.
- Export Center consumes advisory datasets without relying on legacy normalised fields. - Export Center consumes advisory datasets without relying on legacy normalised fields.
- CI fails if lint rules detect forbidden writes or if seeded guard tests regress. - CI fails if lint rules detect forbidden writes or if seeded guard tests regress.
## Risks & mitigations ## Risks & mitigations
- **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes. - **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes.
- **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity. - **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity.
- **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets. - **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets.
- **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers. - **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers.
## Test strategy ## Test strategy
- **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism. - **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism.
- **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge. - **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge.
- **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes. - **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes.
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets. - **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting. - **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.
## Definition of done ## Definition of done
- Validators deployed and verified in staging/offline environments. - Validators deployed and verified in staging/offline environments.
- Runtime guards, CLI/Console workflows, and CI linting all active. - Runtime guards, CLI/Console workflows, and CI linting all active.
- Observability dashboards and runbooks updated; metrics visible. - Observability dashboards and runbooks updated; metrics visible.
- Documentation updates merged; Offline Kit instructions published. - Documentation updates merged; Offline Kit instructions published.
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md. - ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.

318
docs/modules/concelier/operations/authority-audit-runbook.md
View File

@@ -1,159 +1,159 @@
# Concelier Authority Audit Runbook # Concelier Authority Audit Runbook
_Last updated: 2025-10-22_ _Last updated: 2025-10-22_
This runbook helps operators verify and monitor the StellaOps Concelier ⇆ Authority integration. It focuses on the `/jobs*` surface, which now requires StellaOps Authority tokens, and the corresponding audit/metric signals that expose authentication and bypass activity. This runbook helps operators verify and monitor the StellaOps Concelier ⇆ Authority integration. It focuses on the `/jobs*` surface, which now requires StellaOps Authority tokens, and the corresponding audit/metric signals that expose authentication and bypass activity.
## 1. Prerequisites ## 1. Prerequisites
- Authority integration is enabled in `concelier.yaml` (or via `CONCELIER_AUTHORITY__*` environment variables) with a valid `clientId`, secret, audience, and required scopes. - Authority integration is enabled in `concelier.yaml` (or via `CONCELIER_AUTHORITY__*` environment variables) with a valid `clientId`, secret, audience, and required scopes.
- OTLP metrics/log exporters are configured (`concelier.telemetry.*`) or container stdout is shipped to your SIEM. - OTLP metrics/log exporters are configured (`concelier.telemetry.*`) or container stdout is shipped to your SIEM.
- Operators have access to the Concelier job trigger endpoints via CLI or REST for smoke tests. - Operators have access to the Concelier job trigger endpoints via CLI or REST for smoke tests.
- The rollout table in `docs/10_CONCELIER_CLI_QUICKSTART.md` has been reviewed so stakeholders align on the staged → enforced toggle timeline. - The rollout table in `docs/10_CONCELIER_CLI_QUICKSTART.md` has been reviewed so stakeholders align on the staged → enforced toggle timeline.
### Configuration snippet ### Configuration snippet
```yaml ```yaml
concelier: concelier:
authority: authority:
enabled: true enabled: true
allowAnonymousFallback: false # keep true only during initial rollout allowAnonymousFallback: false # keep true only during initial rollout
issuer: "https://authority.internal" issuer: "https://authority.internal"
audiences: audiences:
- "api://concelier" - "api://concelier"
requiredScopes: requiredScopes:
- "concelier.jobs.trigger" - "concelier.jobs.trigger"
- "advisory:read" - "advisory:read"
- "advisory:ingest" - "advisory:ingest"
requiredTenants: requiredTenants:
- "tenant-default" - "tenant-default"
bypassNetworks: bypassNetworks:
- "127.0.0.1/32" - "127.0.0.1/32"
- "::1/128" - "::1/128"
clientId: "concelier-jobs" clientId: "concelier-jobs"
clientSecretFile: "/run/secrets/concelier_authority_client" clientSecretFile: "/run/secrets/concelier_authority_client"
tokenClockSkewSeconds: 60 tokenClockSkewSeconds: 60
resilience: resilience:
enableRetries: true enableRetries: true
retryDelays: retryDelays:
- "00:00:01" - "00:00:01"
- "00:00:02" - "00:00:02"
- "00:00:05" - "00:00:05"
allowOfflineCacheFallback: true allowOfflineCacheFallback: true
offlineCacheTolerance: "00:10:00" offlineCacheTolerance: "00:10:00"
``` ```
> Store secrets outside source control. Concelier reads `clientSecretFile` on startup; rotate by updating the mounted file and restarting the service. > Store secrets outside source control. Concelier reads `clientSecretFile` on startup; rotate by updating the mounted file and restarting the service.
### Resilience tuning ### Resilience tuning
- **Connected sites:** keep the default 1 s / 2 s / 5 s retry ladder so Concelier retries transient Authority hiccups but still surfaces outages quickly. Leave `allowOfflineCacheFallback=true` so cached discovery/JWKS data can bridge short Pathfinder restarts. - **Connected sites:** keep the default 1 s / 2 s / 5 s retry ladder so Concelier retries transient Authority hiccups but still surfaces outages quickly. Leave `allowOfflineCacheFallback=true` so cached discovery/JWKS data can bridge short Pathfinder restarts.
- **Air-gapped/Offline Kit installs:** extend `offlineCacheTolerance` (15–30 minutes) to keep the cached metadata valid between manual synchronisations. You can also disable retries (`enableRetries=false`) if infrastructure teams prefer to handle exponential backoff at the network layer; Concelier will fail fast but keep deterministic logs. - **Air-gapped/Offline Kit installs:** extend `offlineCacheTolerance` (15–30 minutes) to keep the cached metadata valid between manual synchronisations. You can also disable retries (`enableRetries=false`) if infrastructure teams prefer to handle exponential backoff at the network layer; Concelier will fail fast but keep deterministic logs.
- Concelier resolves these knobs through `IOptionsMonitor<StellaOpsAuthClientOptions>`. Edits to `concelier.yaml` are applied on configuration reload; restart the container if you change environment variables or do not have file-watch reloads enabled. - Concelier resolves these knobs through `IOptionsMonitor<StellaOpsAuthClientOptions>`. Edits to `concelier.yaml` are applied on configuration reload; restart the container if you change environment variables or do not have file-watch reloads enabled.
## 2. Key Signals ## 2. Key Signals
### 2.1 Audit log channel ### 2.1 Audit log channel
Concelier emits structured audit entries via the `Concelier.Authorization.Audit` logger for every `/jobs*` request once Authority enforcement is active. Concelier emits structured audit entries via the `Concelier.Authorization.Audit` logger for every `/jobs*` request once Authority enforcement is active.
``` ```
Concelier authorization audit route=/jobs/definitions status=200 subject=ops@example.com clientId=concelier-cli scopes=concelier.jobs.trigger advisory:ingest bypass=False remote=10.1.4.7 Concelier authorization audit route=/jobs/definitions status=200 subject=ops@example.com clientId=concelier-cli scopes=concelier.jobs.trigger advisory:ingest bypass=False remote=10.1.4.7
``` ```
| Field | Sample value | Meaning | | Field | Sample value | Meaning |
|--------------|-------------------------|------------------------------------------------------------------------------------------| |--------------|-------------------------|------------------------------------------------------------------------------------------|
| `route` | `/jobs/definitions` | Endpoint that processed the request. | | `route` | `/jobs/definitions` | Endpoint that processed the request. |
| `status` | `200` / `401` / `409` | Final HTTP status code returned to the caller. | | `status` | `200` / `401` / `409` | Final HTTP status code returned to the caller. |
| `subject` | `ops@example.com` | User or service principal subject (falls back to `(anonymous)` when unauthenticated). | | `subject` | `ops@example.com` | User or service principal subject (falls back to `(anonymous)` when unauthenticated). |
| `clientId` | `concelier-cli` | OAuth client ID provided by Authority (`(none)` if the token lacked the claim). | | `clientId` | `concelier-cli` | OAuth client ID provided by Authority (`(none)` if the token lacked the claim). |
| `scopes` | `concelier.jobs.trigger advisory:ingest advisory:read` | Normalised scope list extracted from token claims; `(none)` if the token carried none. | | `scopes` | `concelier.jobs.trigger advisory:ingest advisory:read` | Normalised scope list extracted from token claims; `(none)` if the token carried none. |
| `tenant` | `tenant-default` | Tenant claim extracted from the Authority token (`(none)` when the token lacked it). | | `tenant` | `tenant-default` | Tenant claim extracted from the Authority token (`(none)` when the token lacked it). |
| `bypass` | `True` / `False` | Indicates whether the request succeeded because its source IP matched a bypass CIDR. | | `bypass` | `True` / `False` | Indicates whether the request succeeded because its source IP matched a bypass CIDR. |
| `remote` | `10.1.4.7` | Remote IP recorded from the connection / forwarded header test hooks. | | `remote` | `10.1.4.7` | Remote IP recorded from the connection / forwarded header test hooks. |
Use your logging backend (e.g., Loki) to index the logger name and filter for suspicious combinations: Use your logging backend (e.g., Loki) to index the logger name and filter for suspicious combinations:
- `status=401 AND bypass=True` – bypass network accepted an unauthenticated call (should be temporary during rollout). - `status=401 AND bypass=True` – bypass network accepted an unauthenticated call (should be temporary during rollout).
- `status=202 AND scopes="(none)"` – a token without scopes triggered a job; tighten client configuration. - `status=202 AND scopes="(none)"` – a token without scopes triggered a job; tighten client configuration.
- `status=202 AND NOT contains(scopes,"advisory:ingest")` – ingestion attempted without the new AOC scopes; confirm the Authority client registration matches the sample above. - `status=202 AND NOT contains(scopes,"advisory:ingest")` – ingestion attempted without the new AOC scopes; confirm the Authority client registration matches the sample above.
- `tenant!=(tenant-default)` – indicates a cross-tenant token was accepted. Ensure Concelier `requiredTenants` is aligned with Authority client registration. - `tenant!=(tenant-default)` – indicates a cross-tenant token was accepted. Ensure Concelier `requiredTenants` is aligned with Authority client registration.
- Spike in `clientId="(none)"` – indicates upstream Authority is not issuing `client_id` claims or the CLI is outdated. - Spike in `clientId="(none)"` – indicates upstream Authority is not issuing `client_id` claims or the CLI is outdated.
### 2.2 Metrics ### 2.2 Metrics
Concelier publishes counters under the OTEL meter `StellaOps.Concelier.WebService.Jobs`. Tags: `job.kind`, `job.trigger`, `job.outcome`. Concelier publishes counters under the OTEL meter `StellaOps.Concelier.WebService.Jobs`. Tags: `job.kind`, `job.trigger`, `job.outcome`.
| Metric name | Description | PromQL example | | Metric name | Description | PromQL example |
|-------------------------------|----------------------------------------------------|----------------| |-------------------------------|----------------------------------------------------|----------------|
| `web.jobs.triggered` | Accepted job trigger requests. | `sum by (job_kind) (rate(web_jobs_triggered_total[5m]))` | | `web.jobs.triggered` | Accepted job trigger requests. | `sum by (job_kind) (rate(web_jobs_triggered_total[5m]))` |
| `web.jobs.trigger.conflict` | Rejected triggers (already running, disabled…). | `sum(rate(web_jobs_trigger_conflict_total[5m]))` | | `web.jobs.trigger.conflict` | Rejected triggers (already running, disabled…). | `sum(rate(web_jobs_trigger_conflict_total[5m]))` |
| `web.jobs.trigger.failed` | Server-side job failures. | `sum(rate(web_jobs_trigger_failed_total[5m]))` | | `web.jobs.trigger.failed` | Server-side job failures. | `sum(rate(web_jobs_trigger_failed_total[5m]))` |
> Prometheus/OTEL collectors typically surface counters with `_total` suffix. Adjust queries to match your pipeline’s generated metric names. > Prometheus/OTEL collectors typically surface counters with `_total` suffix. Adjust queries to match your pipeline’s generated metric names.
Correlate audit logs with the following global meter exported via `Concelier.SourceDiagnostics`: Correlate audit logs with the following global meter exported via `Concelier.SourceDiagnostics`:
- `concelier.source.http.requests_total{concelier_source="jobs-run"}` – ensures REST/manual triggers route through Authority. - `concelier.source.http.requests_total{concelier_source="jobs-run"}` – ensures REST/manual triggers route through Authority.
- If Grafana dashboards are deployed, extend the “Concelier Jobs” board with the above counters plus a table of recent audit log entries. - If Grafana dashboards are deployed, extend the “Concelier Jobs” board with the above counters plus a table of recent audit log entries.
## 3. Alerting Guidance ## 3. Alerting Guidance
1. **Unauthorized bypass attempt** 1. **Unauthorized bypass attempt**
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", status="401", bypass="True"}[5m])) > 0` - Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", status="401", bypass="True"}[5m])) > 0`
- Action: verify `bypassNetworks` list; confirm expected maintenance windows; rotate credentials if suspicious. - Action: verify `bypassNetworks` list; confirm expected maintenance windows; rotate credentials if suspicious.
2. **Missing scopes** 2. **Missing scopes**
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", scopes="(none)", status="200"}[5m])) > 0` - Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", scopes="(none)", status="200"}[5m])) > 0`
- Action: audit Authority client registration; ensure `requiredScopes` includes `concelier.jobs.trigger`, `advisory:ingest`, and `advisory:read`. - Action: audit Authority client registration; ensure `requiredScopes` includes `concelier.jobs.trigger`, `advisory:ingest`, and `advisory:read`.
3. **Trigger failure surge** 3. **Trigger failure surge**
- Query: `sum(rate(web_jobs_trigger_failed_total[10m])) > 0` with severity `warning` if sustained for 10 minutes. - Query: `sum(rate(web_jobs_trigger_failed_total[10m])) > 0` with severity `warning` if sustained for 10 minutes.
- Action: inspect correlated audit entries and `Concelier.Telemetry` traces for job execution errors. - Action: inspect correlated audit entries and `Concelier.Telemetry` traces for job execution errors.
4. **Conflict spike** 4. **Conflict spike**
- Query: `sum(rate(web_jobs_trigger_conflict_total[10m])) > 5` (tune threshold). - Query: `sum(rate(web_jobs_trigger_conflict_total[10m])) > 5` (tune threshold).
- Action: downstream scheduling may be firing repetitive triggers; ensure precedence is configured properly. - Action: downstream scheduling may be firing repetitive triggers; ensure precedence is configured properly.
5. **Authority offline** 5. **Authority offline**
- Watch `Concelier.Authorization.Audit` logs for `status=503` or `status=500` along with `clientId="(none)"`. Investigate Authority availability before re-enabling anonymous fallback. - Watch `Concelier.Authorization.Audit` logs for `status=503` or `status=500` along with `clientId="(none)"`. Investigate Authority availability before re-enabling anonymous fallback.
## 4. Rollout & Verification Procedure ## 4. Rollout & Verification Procedure
1. **Pre-checks** 1. **Pre-checks**
- Align with the rollout phases documented in `docs/10_CONCELIER_CLI_QUICKSTART.md` (validation → rehearsal → enforced) and record the target dates in your change request. - Align with the rollout phases documented in `docs/10_CONCELIER_CLI_QUICKSTART.md` (validation → rehearsal → enforced) and record the target dates in your change request.
- Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation. - Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation.
- Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host). - Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host).
2. **Smoke test with valid token** 2. **Smoke test with valid token**
- Obtain a token via CLI: `stella auth login --scope "concelier.jobs.trigger advisory:ingest" --scope advisory:read`. - Obtain a token via CLI: `stella auth login --scope "concelier.jobs.trigger advisory:ingest" --scope advisory:read`.
- Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://concelier.internal/jobs/definitions`. - Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://concelier.internal/jobs/definitions`.
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger advisory:ingest advisory:read`, and `tenant=tenant-default`. - Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger advisory:ingest advisory:read`, and `tenant=tenant-default`.
3. **Negative test without token** 3. **Negative test without token**
- Call the same endpoint without a token. Expect HTTP 401, `bypass=False`. - Call the same endpoint without a token. Expect HTTP 401, `bypass=False`.
- If the request succeeds, double-check `bypassNetworks` and ensure fallback is disabled. - If the request succeeds, double-check `bypassNetworks` and ensure fallback is disabled.
4. **Bypass check (if applicable)** 4. **Bypass check (if applicable)**
- From an allowed maintenance IP, call `/jobs/definitions` without a token. Confirm the audit log shows `bypass=True`. Review business justification and expiry date for such entries. - From an allowed maintenance IP, call `/jobs/definitions` without a token. Confirm the audit log shows `bypass=True`. Review business justification and expiry date for such entries.
5. **Metrics validation** 5. **Metrics validation**
- Ensure `web.jobs.triggered` counter increments during accepted runs. - Ensure `web.jobs.triggered` counter increments during accepted runs.
- Exporters should show corresponding spans (`concelier.job.trigger`) if tracing is enabled. - Exporters should show corresponding spans (`concelier.job.trigger`) if tracing is enabled.
## 5. Troubleshooting ## 5. Troubleshooting
| Symptom | Probable cause | Remediation | | Symptom | Probable cause | Remediation |
|---------|----------------|-------------| |---------|----------------|-------------|
| Audit log shows `clientId=(none)` for all requests | Authority not issuing `client_id` claim or CLI outdated | Update StellaOps Authority configuration (`StellaOpsAuthorityOptions.Token.Claims.ClientId`), or upgrade the CLI token acquisition flow. | | Audit log shows `clientId=(none)` for all requests | Authority not issuing `client_id` claim or CLI outdated | Update StellaOps Authority configuration (`StellaOpsAuthorityOptions.Token.Claims.ClientId`), or upgrade the CLI token acquisition flow. |
| Requests succeed with `bypass=True` unexpectedly | Local network added to `bypassNetworks` or fallback still enabled | Remove/adjust the CIDR list, disable anonymous fallback, restart Concelier. | | Requests succeed with `bypass=True` unexpectedly | Local network added to `bypassNetworks` or fallback still enabled | Remove/adjust the CIDR list, disable anonymous fallback, restart Concelier. |
| HTTP 401 with valid token | `requiredScopes` missing from client registration or token audience mismatch | Verify Authority client scopes (`concelier.jobs.trigger`) and ensure the token audience matches `audiences` config. | | HTTP 401 with valid token | `requiredScopes` missing from client registration or token audience mismatch | Verify Authority client scopes (`concelier.jobs.trigger`) and ensure the token audience matches `audiences` config. |
| Metrics missing from Prometheus | Telemetry exporters disabled or filter missing OTEL meter | Set `concelier.telemetry.enableMetrics=true`, ensure collector includes `StellaOps.Concelier.WebService.Jobs` meter. | | Metrics missing from Prometheus | Telemetry exporters disabled or filter missing OTEL meter | Set `concelier.telemetry.enableMetrics=true`, ensure collector includes `StellaOps.Concelier.WebService.Jobs` meter. |
| Sudden spike in `web.jobs.trigger.failed` | Downstream job failure or Authority timeout mid-request | Inspect Concelier job logs, re-run with tracing enabled, validate Authority latency. | | Sudden spike in `web.jobs.trigger.failed` | Downstream job failure or Authority timeout mid-request | Inspect Concelier job logs, re-run with tracing enabled, validate Authority latency. |
## 6. References ## 6. References
- `docs/21_INSTALL_GUIDE.md` – Authority configuration quick start. - `docs/21_INSTALL_GUIDE.md` – Authority configuration quick start.
- `docs/17_SECURITY_HARDENING_GUIDE.md` – Security guardrails and enforcement deadlines. - `docs/17_SECURITY_HARDENING_GUIDE.md` – Security guardrails and enforcement deadlines.
- `docs/modules/authority/operations/monitoring.md` – Authority-side monitoring and alerting playbook. - `docs/modules/authority/operations/monitoring.md` – Authority-side monitoring and alerting playbook.
- `StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields. - `StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields.

320
docs/modules/concelier/operations/conflict-resolution.md
View File

@@ -1,160 +1,160 @@
# Concelier Conflict Resolution Runbook (Sprint 3) # Concelier Conflict Resolution Runbook (Sprint 3)
This runbook equips Concelier operators to detect, triage, and resolve advisory conflicts now that the Sprint 3 merge engine landed (`AdvisoryPrecedenceMerger`, merge-event hashing, and telemetry counters). It builds on the canonical rules defined in `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md` and the metrics/logging instrumentation delivered this sprint. This runbook equips Concelier operators to detect, triage, and resolve advisory conflicts now that the Sprint 3 merge engine landed (`AdvisoryPrecedenceMerger`, merge-event hashing, and telemetry counters). It builds on the canonical rules defined in `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md` and the metrics/logging instrumentation delivered this sprint.
--- ---
## 1. Precedence Model (recap) ## 1. Precedence Model (recap)
- **Default ranking:** `GHSA -> NVD -> OSV`, with distro/vendor PSIRTs outranking ecosystem feeds (`AdvisoryPrecedenceDefaults`). Use `concelier:merge:precedence:ranks` to override per source when incident response requires it. - **Default ranking:** `GHSA -> NVD -> OSV`, with distro/vendor PSIRTs outranking ecosystem feeds (`AdvisoryPrecedenceDefaults`). Use `concelier:merge:precedence:ranks` to override per source when incident response requires it.
- **Freshness override:** if a lower-ranked source is >= 48 hours newer for a freshness-sensitive field (title, summary, affected ranges, references, credits), it wins. Every override stamps `provenance[].decisionReason = freshness`. - **Freshness override:** if a lower-ranked source is >= 48 hours newer for a freshness-sensitive field (title, summary, affected ranges, references, credits), it wins. Every override stamps `provenance[].decisionReason = freshness`.
- **Tie-breakers:** when precedence and freshness tie, the engine falls back to (1) primary source order, (2) shortest normalized text, (3) lowest stable hash. Merge-generated provenance records set `decisionReason = tie-breaker`. - **Tie-breakers:** when precedence and freshness tie, the engine falls back to (1) primary source order, (2) shortest normalized text, (3) lowest stable hash. Merge-generated provenance records set `decisionReason = tie-breaker`.
- **Audit trail:** each merged advisory receives a `merge` provenance entry listing the participating sources plus a `merge_event` record with canonical before/after SHA-256 hashes. - **Audit trail:** each merged advisory receives a `merge` provenance entry listing the participating sources plus a `merge_event` record with canonical before/after SHA-256 hashes.
--- ---
## 2. Telemetry Shipped This Sprint ## 2. Telemetry Shipped This Sprint
| Instrument | Type | Key Tags | Purpose | | Instrument | Type | Key Tags | Purpose |
|------------|------|----------|---------| |------------|------|----------|---------|
| `concelier.merge.operations` | Counter | `inputs` | Total precedence merges executed. | | `concelier.merge.operations` | Counter | `inputs` | Total precedence merges executed. |
| `concelier.merge.overrides` | Counter | `primary_source`, `suppressed_source`, `primary_rank`, `suppressed_rank` | Field-level overrides chosen by precedence. | | `concelier.merge.overrides` | Counter | `primary_source`, `suppressed_source`, `primary_rank`, `suppressed_rank` | Field-level overrides chosen by precedence. |
| `concelier.merge.range_overrides` | Counter | `advisory_key`, `package_type`, `primary_source`, `suppressed_source`, `primary_range_count`, `suppressed_range_count` | Package range overrides emitted by `AffectedPackagePrecedenceResolver`. | | `concelier.merge.range_overrides` | Counter | `advisory_key`, `package_type`, `primary_source`, `suppressed_source`, `primary_range_count`, `suppressed_range_count` | Package range overrides emitted by `AffectedPackagePrecedenceResolver`. |
| `concelier.merge.conflicts` | Counter | `type` (`severity`, `precedence_tie`), `reason` (`mismatch`, `primary_missing`, `equal_rank`) | Conflicts requiring operator review. | | `concelier.merge.conflicts` | Counter | `type` (`severity`, `precedence_tie`), `reason` (`mismatch`, `primary_missing`, `equal_rank`) | Conflicts requiring operator review. |
| `concelier.merge.identity_conflicts` | Counter | `scheme`, `alias_value`, `advisory_count` | Alias collisions surfaced by the identity graph. | | `concelier.merge.identity_conflicts` | Counter | `scheme`, `alias_value`, `advisory_count` | Alias collisions surfaced by the identity graph. |
### Structured logs ### Structured logs
- `AdvisoryOverride` (EventId 1000) - logs merge suppressions with alias/provenance counts. - `AdvisoryOverride` (EventId 1000) - logs merge suppressions with alias/provenance counts.
- `PackageRangeOverride` (EventId 1001) - logs package-level precedence decisions. - `PackageRangeOverride` (EventId 1001) - logs package-level precedence decisions.
- `PrecedenceConflict` (EventId 1002) - logs mismatched severity or equal-rank scenarios. - `PrecedenceConflict` (EventId 1002) - logs mismatched severity or equal-rank scenarios.
- `Alias collision ...` (no EventId) - emitted when `concelier.merge.identity_conflicts` increments. - `Alias collision ...` (no EventId) - emitted when `concelier.merge.identity_conflicts` increments.
Expect all logs at `Information`. Ensure OTEL exporters include the scope `StellaOps.Concelier.Merge`. Expect all logs at `Information`. Ensure OTEL exporters include the scope `StellaOps.Concelier.Merge`.
--- ---
## 3. Detection & Alerting ## 3. Detection & Alerting
1. **Dashboard panels** 1. **Dashboard panels**
- `concelier.merge.conflicts` - table grouped by `type/reason`. Alert when > 0 in a 15 minute window. - `concelier.merge.conflicts` - table grouped by `type/reason`. Alert when > 0 in a 15 minute window.
- `concelier.merge.range_overrides` - stacked bar by `package_type`. Spikes highlight vendor PSIRT overrides over registry data. - `concelier.merge.range_overrides` - stacked bar by `package_type`. Spikes highlight vendor PSIRT overrides over registry data.
- `concelier.merge.overrides` with `primary_source|suppressed_source` - catches unexpected precedence flips (e.g., OSV overtaking GHSA). - `concelier.merge.overrides` with `primary_source|suppressed_source` - catches unexpected precedence flips (e.g., OSV overtaking GHSA).
- `concelier.merge.identity_conflicts` - single-stat; alert when alias collisions occur more than once per day. - `concelier.merge.identity_conflicts` - single-stat; alert when alias collisions occur more than once per day.
2. **Log based alerts** 2. **Log based alerts**
- `eventId=1002` with `reason="equal_rank"` - indicates precedence table gaps; page merge owners. - `eventId=1002` with `reason="equal_rank"` - indicates precedence table gaps; page merge owners.
- `eventId=1002` with `reason="mismatch"` - severity disagreement; open connector bug if sustained. - `eventId=1002` with `reason="mismatch"` - severity disagreement; open connector bug if sustained.
3. **Job health** 3. **Job health**
- `stellaops-cli db merge` exit code `1` signifies unresolved conflicts. Pipe to automation that captures logs and notifies #concelier-ops. - `stellaops-cli db merge` exit code `1` signifies unresolved conflicts. Pipe to automation that captures logs and notifies #concelier-ops.
### Threshold updates (2025-10-12) ### Threshold updates (2025-10-12)
- `concelier.merge.conflicts` – Page only when ≥ 2 events fire within 30 minutes; the synthetic conflict fixture run produces 0 conflicts, so the first event now routes to Slack for manual review instead of paging. - `concelier.merge.conflicts` – Page only when ≥ 2 events fire within 30 minutes; the synthetic conflict fixture run produces 0 conflicts, so the first event now routes to Slack for manual review instead of paging.
- `concelier.merge.overrides` – Raise a warning when the 30-minute sum exceeds 10 (canonical triple yields exactly 1 summary override with `primary_source=osv`, `suppressed_source=ghsa`). - `concelier.merge.overrides` – Raise a warning when the 30-minute sum exceeds 10 (canonical triple yields exactly 1 summary override with `primary_source=osv`, `suppressed_source=ghsa`).
- `concelier.merge.range_overrides` – Maintain the 15-minute alert at ≥ 3 but annotate dashboards that the regression triple emits a single `package_type=semver` override so ops can spot unexpected spikes. - `concelier.merge.range_overrides` – Maintain the 15-minute alert at ≥ 3 but annotate dashboards that the regression triple emits a single `package_type=semver` override so ops can spot unexpected spikes.
--- ---
## 4. Triage Workflow ## 4. Triage Workflow
1. **Confirm job context** 1. **Confirm job context**
- `stellaops-cli db merge` (CLI) or `POST /jobs/merge:reconcile` (API) to rehydrate the merge job. Use `--verbose` to stream structured logs during triage. - `stellaops-cli db merge` (CLI) or `POST /jobs/merge:reconcile` (API) to rehydrate the merge job. Use `--verbose` to stream structured logs during triage.
2. **Inspect metrics** 2. **Inspect metrics**
- Correlate spikes in `concelier.merge.conflicts` with `primary_source`/`suppressed_source` tags from `concelier.merge.overrides`. - Correlate spikes in `concelier.merge.conflicts` with `primary_source`/`suppressed_source` tags from `concelier.merge.overrides`.
3. **Pull structured logs** 3. **Pull structured logs**
- Example (vector output): - Example (vector output):
``` ```
jq 'select(.EventId.Name=="PrecedenceConflict") | {advisory: .State[0].Value, type: .ConflictType, reason: .Reason, primary: .PrimarySources, suppressed: .SuppressedSources}' stellaops-concelier.log jq 'select(.EventId.Name=="PrecedenceConflict") | {advisory: .State[0].Value, type: .ConflictType, reason: .Reason, primary: .PrimarySources, suppressed: .SuppressedSources}' stellaops-concelier.log
``` ```
4. **Review merge events** 4. **Review merge events**
- `mongosh`: - `mongosh`:
```javascript ```javascript
use concelier; use concelier;
db.merge_event.find({ advisoryKey: "CVE-2025-1234" }).sort({ mergedAt: -1 }).limit(5); db.merge_event.find({ advisoryKey: "CVE-2025-1234" }).sort({ mergedAt: -1 }).limit(5);
``` ```
- Compare `beforeHash` vs `afterHash` to confirm the merge actually changed canonical output. - Compare `beforeHash` vs `afterHash` to confirm the merge actually changed canonical output.
5. **Interrogate provenance** 5. **Interrogate provenance**
- `db.advisories.findOne({ advisoryKey: "CVE-2025-1234" }, { title: 1, severity: 1, provenance: 1, "affectedPackages.provenance": 1 })` - `db.advisories.findOne({ advisoryKey: "CVE-2025-1234" }, { title: 1, severity: 1, provenance: 1, "affectedPackages.provenance": 1 })`
- Check `provenance[].decisionReason` values (`precedence`, `freshness`, `tie-breaker`) to understand why the winning field was chosen. - Check `provenance[].decisionReason` values (`precedence`, `freshness`, `tie-breaker`) to understand why the winning field was chosen.
--- ---
## 5. Conflict Classification Matrix ## 5. Conflict Classification Matrix
| Signal | Likely Cause | Immediate Action | | Signal | Likely Cause | Immediate Action |
|--------|--------------|------------------| |--------|--------------|------------------|
| `reason="mismatch"` with `type="severity"` | Upstream feeds disagree on CVSS vector/severity. | Verify which feed is freshest; if correctness is known, adjust connector mapping or precedence override. | | `reason="mismatch"` with `type="severity"` | Upstream feeds disagree on CVSS vector/severity. | Verify which feed is freshest; if correctness is known, adjust connector mapping or precedence override. |
| `reason="primary_missing"` | Higher-ranked source lacks the field entirely. | Backfill connector data or temporarily allow lower-ranked source via precedence override. | | `reason="primary_missing"` | Higher-ranked source lacks the field entirely. | Backfill connector data or temporarily allow lower-ranked source via precedence override. |
| `reason="equal_rank"` | Two feeds share the same precedence rank (custom config or missing entry). | Update `concelier:merge:precedence:ranks` to break the tie; restart merge job. | | `reason="equal_rank"` | Two feeds share the same precedence rank (custom config or missing entry). | Update `concelier:merge:precedence:ranks` to break the tie; restart merge job. |
| Rising `concelier.merge.range_overrides` for a package type | Vendor PSIRT now supplies richer ranges. | Validate connectors emit `decisionReason="precedence"` and update dashboards to treat registry ranges as fallback. | | Rising `concelier.merge.range_overrides` for a package type | Vendor PSIRT now supplies richer ranges. | Validate connectors emit `decisionReason="precedence"` and update dashboards to treat registry ranges as fallback. |
| `concelier.merge.identity_conflicts` > 0 | Alias scheme mapping produced collisions (duplicate CVE <-> advisory pairs). | Inspect `Alias collision` log payload; reconcile the alias graph by adjusting connector alias output. | | `concelier.merge.identity_conflicts` > 0 | Alias scheme mapping produced collisions (duplicate CVE <-> advisory pairs). | Inspect `Alias collision` log payload; reconcile the alias graph by adjusting connector alias output. |
--- ---
## 6. Resolution Playbook ## 6. Resolution Playbook
1. **Connector data fix** 1. **Connector data fix**
- Re-run the offending connector stages (`stellaops-cli db fetch --source ghsa --stage map` etc.). - Re-run the offending connector stages (`stellaops-cli db fetch --source ghsa --stage map` etc.).
- Once fixed, rerun merge and verify `decisionReason` reflects `freshness` or `precedence` as expected. - Once fixed, rerun merge and verify `decisionReason` reflects `freshness` or `precedence` as expected.
2. **Temporary precedence override** 2. **Temporary precedence override**
- Edit `etc/concelier.yaml`: - Edit `etc/concelier.yaml`:
```yaml ```yaml
concelier: concelier:
merge: merge:
precedence: precedence:
ranks: ranks:
osv: 1 osv: 1
ghsa: 0 ghsa: 0
``` ```
- Restart Concelier workers; confirm tags in `concelier.merge.overrides` show the new ranks. - Restart Concelier workers; confirm tags in `concelier.merge.overrides` show the new ranks.
- Document the override with expiry in the change log. - Document the override with expiry in the change log.
3. **Alias remediation** 3. **Alias remediation**
- Update connector mapping rules to weed out duplicate aliases (e.g., skip GHSA aliases that mirror CVE IDs). - Update connector mapping rules to weed out duplicate aliases (e.g., skip GHSA aliases that mirror CVE IDs).
- Flush cached alias graphs if necessary (`db.alias_graph.drop()` is destructive-coordinate with Storage before issuing). - Flush cached alias graphs if necessary (`db.alias_graph.drop()` is destructive-coordinate with Storage before issuing).
4. **Escalation** 4. **Escalation**
- If override metrics spike due to upstream regression, open an incident with Security Guild, referencing merge logs and `merge_event` IDs. - If override metrics spike due to upstream regression, open an incident with Security Guild, referencing merge logs and `merge_event` IDs.
--- ---
## 7. Validation Checklist ## 7. Validation Checklist
- [ ] Merge job rerun returns exit code `0`. - [ ] Merge job rerun returns exit code `0`.
- [ ] `concelier.merge.conflicts` baseline returns to zero after corrective action. - [ ] `concelier.merge.conflicts` baseline returns to zero after corrective action.
- [ ] Latest `merge_event` entry shows expected hash delta. - [ ] Latest `merge_event` entry shows expected hash delta.
- [ ] Affected advisory document shows updated `provenance[].decisionReason`. - [ ] Affected advisory document shows updated `provenance[].decisionReason`.
- [ ] Ops change log updated with incident summary, config overrides, and rollback plan. - [ ] Ops change log updated with incident summary, config overrides, and rollback plan.
--- ---
## 8. Reference Material ## 8. Reference Material
- Canonical conflict rules: `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md`. - Canonical conflict rules: `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md`.
- Merge engine internals: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/AdvisoryPrecedenceMerger.cs`. - Merge engine internals: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/AdvisoryPrecedenceMerger.cs`.
- Metrics definitions: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/AdvisoryMergeService.cs` (identity conflicts) and `AdvisoryPrecedenceMerger`. - Metrics definitions: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/AdvisoryMergeService.cs` (identity conflicts) and `AdvisoryPrecedenceMerger`.
- Storage audit trail: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/MergeEventWriter.cs`, `src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/MergeEvents`. - Storage audit trail: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/MergeEventWriter.cs`, `src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/MergeEvents`.
Keep this runbook synchronized with future sprint notes and update alert thresholds as baseline volumes change. Keep this runbook synchronized with future sprint notes and update alert thresholds as baseline volumes change.
--- ---
## 9. Synthetic Regression Fixtures ## 9. Synthetic Regression Fixtures
- **Locations** – Canonical conflict snapshots now live at `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/Fixtures/conflict-ghsa.canonical.json`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/Nvd/Fixtures/conflict-nvd.canonical.json`, and `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/Fixtures/conflict-osv.canonical.json`. - **Locations** – Canonical conflict snapshots now live at `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/Fixtures/conflict-ghsa.canonical.json`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/Nvd/Fixtures/conflict-nvd.canonical.json`, and `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/Fixtures/conflict-osv.canonical.json`.
- **Validation commands** – To regenerate and verify the fixtures offline, run: - **Validation commands** – To regenerate and verify the fixtures offline, run:
```bash ```bash
dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests
dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/StellaOps.Concelier.Connector.Nvd.Tests.csproj --filter NvdConflictFixtureTests dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/StellaOps.Concelier.Connector.Nvd.Tests.csproj --filter NvdConflictFixtureTests
dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj --filter OsvConflictFixtureTests dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj --filter OsvConflictFixtureTests
dotnet test src/Concelier/__Tests/StellaOps.Concelier.Merge.Tests/StellaOps.Concelier.Merge.Tests.csproj --filter MergeAsync_AppliesCanonicalRulesAndPersistsDecisions dotnet test src/Concelier/__Tests/StellaOps.Concelier.Merge.Tests/StellaOps.Concelier.Merge.Tests.csproj --filter MergeAsync_AppliesCanonicalRulesAndPersistsDecisions
``` ```
- **Expected signals** – The triple produces one freshness-driven summary override (`primary_source=osv`, `suppressed_source=ghsa`) and one range override for the npm SemVer package while leaving `concelier.merge.conflicts` at zero. Use these values as the baseline when tuning dashboards or load-testing alert pipelines. - **Expected signals** – The triple produces one freshness-driven summary override (`primary_source=osv`, `suppressed_source=ghsa`) and one range override for the npm SemVer package while leaving `concelier.merge.conflicts` at zero. Use these values as the baseline when tuning dashboards or load-testing alert pipelines.
--- ---
## 10. Change Log ## 10. Change Log
| Date (UTC) | Change | Notes | | Date (UTC) | Change | Notes |
|------------|--------|-------| |------------|--------|-------|
| 2025-10-16 | Ops review signed off after connector expansion (CCCS, CERT-Bund, KISA, ICS CISA, MSRC) landed. Alert thresholds from §3 reaffirmed; dashboards updated to watch attachment signals emitted by ICS CISA connector. | Ops sign-off recorded by Concelier Ops Guild; no additional overrides required. | | 2025-10-16 | Ops review signed off after connector expansion (CCCS, CERT-Bund, KISA, ICS CISA, MSRC) landed. Alert thresholds from §3 reaffirmed; dashboards updated to watch attachment signals emitted by ICS CISA connector. | Ops sign-off recorded by Concelier Ops Guild; no additional overrides required. |

154
docs/modules/concelier/operations/connectors/apple.md
View File

@@ -1,77 +1,77 @@
# Concelier Apple Security Update Connector Operations # Concelier Apple Security Update Connector Operations
This runbook covers staging and production rollout for the Apple security updates connector (`source:vndr-apple:*`), including observability checks and fixture maintenance. This runbook covers staging and production rollout for the Apple security updates connector (`source:vndr-apple:*`), including observability checks and fixture maintenance.
## 1. Prerequisites ## 1. Prerequisites
- Network egress (or mirrored cache) for `https://gdmf.apple.com/v2/pmv` and the Apple Support domain (`https://support.apple.com/`). - Network egress (or mirrored cache) for `https://gdmf.apple.com/v2/pmv` and the Apple Support domain (`https://support.apple.com/`).
- Optional: corporate proxy exclusions for the Apple hosts if outbound traffic is normally filtered. - Optional: corporate proxy exclusions for the Apple hosts if outbound traffic is normally filtered.
- Updated configuration (environment variables or `concelier.yaml`) with an `apple` section. Example baseline: - Updated configuration (environment variables or `concelier.yaml`) with an `apple` section. Example baseline:
```yaml ```yaml
concelier: concelier:
sources: sources:
apple: apple:
softwareLookupUri: "https://gdmf.apple.com/v2/pmv" softwareLookupUri: "https://gdmf.apple.com/v2/pmv"
advisoryBaseUri: "https://support.apple.com/" advisoryBaseUri: "https://support.apple.com/"
localeSegment: "en-us" localeSegment: "en-us"
maxAdvisoriesPerFetch: 25 maxAdvisoriesPerFetch: 25
initialBackfill: "120.00:00:00" initialBackfill: "120.00:00:00"
modifiedTolerance: "02:00:00" modifiedTolerance: "02:00:00"
failureBackoff: "00:05:00" failureBackoff: "00:05:00"
``` ```
> ℹ️ `softwareLookupUri` and `advisoryBaseUri` must stay absolute and aligned with the HTTP allow-list; Concelier automatically adds both hosts to the connector HttpClient. > ℹ️ `softwareLookupUri` and `advisoryBaseUri` must stay absolute and aligned with the HTTP allow-list; Concelier automatically adds both hosts to the connector HttpClient.
## 2. Staging Smoke Test ## 2. Staging Smoke Test
1. Deploy the configuration and restart the Concelier workers to ensure the Apple connector options are bound. 1. Deploy the configuration and restart the Concelier workers to ensure the Apple connector options are bound.
2. Trigger a full connector cycle: 2. Trigger a full connector cycle:
- CLI: `stella db jobs run source:vndr-apple:fetch --and-then source:vndr-apple:parse --and-then source:vndr-apple:map` - CLI: `stella db jobs run source:vndr-apple:fetch --and-then source:vndr-apple:parse --and-then source:vndr-apple:map`
- REST: `POST /jobs/run { "kind": "source:vndr-apple:fetch", "chain": ["source:vndr-apple:parse", "source:vndr-apple:map"] }` - REST: `POST /jobs/run { "kind": "source:vndr-apple:fetch", "chain": ["source:vndr-apple:parse", "source:vndr-apple:map"] }`
3. Validate metrics exported under meter `StellaOps.Concelier.Connector.Vndr.Apple`: 3. Validate metrics exported under meter `StellaOps.Concelier.Connector.Vndr.Apple`:
- `apple.fetch.items` (documents fetched) - `apple.fetch.items` (documents fetched)
- `apple.fetch.failures` - `apple.fetch.failures`
- `apple.fetch.unchanged` - `apple.fetch.unchanged`
- `apple.parse.failures` - `apple.parse.failures`
- `apple.map.affected.count` (histogram of affected package counts) - `apple.map.affected.count` (histogram of affected package counts)
4. Cross-check the shared HTTP counters: 4. Cross-check the shared HTTP counters:
- `concelier.source.http.requests_total{concelier_source="vndr-apple"}` should increase for both index and detail phases. - `concelier.source.http.requests_total{concelier_source="vndr-apple"}` should increase for both index and detail phases.
- `concelier.source.http.failures_total{concelier_source="vndr-apple"}` should remain flat (0) during a healthy run. - `concelier.source.http.failures_total{concelier_source="vndr-apple"}` should remain flat (0) during a healthy run.
5. Inspect the info logs: 5. Inspect the info logs:
- `Apple software index fetch … processed=X newDocuments=Y` - `Apple software index fetch … processed=X newDocuments=Y`
- `Apple advisory parse complete … aliases=… affected=…` - `Apple advisory parse complete … aliases=… affected=…`
- `Mapped Apple advisory … pendingMappings=0` - `Mapped Apple advisory … pendingMappings=0`
6. Confirm MongoDB state: 6. Confirm MongoDB state:
- `raw_documents` store contains the HT article HTML with metadata (`apple.articleId`, `apple.postingDate`). - `raw_documents` store contains the HT article HTML with metadata (`apple.articleId`, `apple.postingDate`).
- `dtos` store has `schemaVersion="apple.security.update.v1"`. - `dtos` store has `schemaVersion="apple.security.update.v1"`.
- `advisories` collection includes keys `HTxxxxxx` with normalized SemVer rules. - `advisories` collection includes keys `HTxxxxxx` with normalized SemVer rules.
- `source_states` entry for `apple` shows a recent `cursor.lastPosted`. - `source_states` entry for `apple` shows a recent `cursor.lastPosted`.
## 3. Production Monitoring ## 3. Production Monitoring
- **Dashboards** – Add the following expressions to your Concelier Grafana board (OTLP/Prometheus naming assumed): - **Dashboards** – Add the following expressions to your Concelier Grafana board (OTLP/Prometheus naming assumed):
- `rate(apple_fetch_items_total[15m])` vs `rate(concelier_source_http_requests_total{concelier_source="vndr-apple"}[15m])` - `rate(apple_fetch_items_total[15m])` vs `rate(concelier_source_http_requests_total{concelier_source="vndr-apple"}[15m])`
- `rate(apple_fetch_failures_total[5m])` for error spikes (`severity=warning` at `>0`) - `rate(apple_fetch_failures_total[5m])` for error spikes (`severity=warning` at `>0`)
- `histogram_quantile(0.95, rate(apple_map_affected_count_bucket[1h]))` to watch affected-package fan-out - `histogram_quantile(0.95, rate(apple_map_affected_count_bucket[1h]))` to watch affected-package fan-out
- `increase(apple_parse_failures_total[6h])` to catch parser drift (alerts at `>0`) - `increase(apple_parse_failures_total[6h])` to catch parser drift (alerts at `>0`)
- **Alerts** – Page if `rate(apple_fetch_items_total[2h]) == 0` during business hours while other connectors are active. This often indicates lookup feed failures or misconfigured allow-lists. - **Alerts** – Page if `rate(apple_fetch_items_total[2h]) == 0` during business hours while other connectors are active. This often indicates lookup feed failures or misconfigured allow-lists.
- **Logs** – Surface warnings `Apple document {DocumentId} missing GridFS payload` or `Apple parse failed`—repeated hits imply storage issues or HTML regressions. - **Logs** – Surface warnings `Apple document {DocumentId} missing GridFS payload` or `Apple parse failed`—repeated hits imply storage issues or HTML regressions.
- **Telemetry pipeline** – `StellaOps.Concelier.WebService` now exports `StellaOps.Concelier.Connector.Vndr.Apple` alongside existing Concelier meters; ensure your OTEL collector or Prometheus scraper includes it. - **Telemetry pipeline** – `StellaOps.Concelier.WebService` now exports `StellaOps.Concelier.Connector.Vndr.Apple` alongside existing Concelier meters; ensure your OTEL collector or Prometheus scraper includes it.
## 4. Fixture Maintenance ## 4. Fixture Maintenance
Regression fixtures live under `src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/Apple/Fixtures`. Refresh them whenever Apple reshapes the HT layout or when new platforms appear. Regression fixtures live under `src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/Apple/Fixtures`. Refresh them whenever Apple reshapes the HT layout or when new platforms appear.
1. Run the helper script matching your platform: 1. Run the helper script matching your platform:
- Bash: `./scripts/update-apple-fixtures.sh` - Bash: `./scripts/update-apple-fixtures.sh`
- PowerShell: `./scripts/update-apple-fixtures.ps1` - PowerShell: `./scripts/update-apple-fixtures.ps1`
2. Each script exports `UPDATE_APPLE_FIXTURES=1`, updates the `WSLENV` passthrough, and touches `.update-apple-fixtures` so WSL+VS Code test runs observe the flag. The subsequent test execution fetches the live HT articles listed in `AppleFixtureManager`, sanitises the HTML, and rewrites the `.expected.json` DTO snapshots. 2. Each script exports `UPDATE_APPLE_FIXTURES=1`, updates the `WSLENV` passthrough, and touches `.update-apple-fixtures` so WSL+VS Code test runs observe the flag. The subsequent test execution fetches the live HT articles listed in `AppleFixtureManager`, sanitises the HTML, and rewrites the `.expected.json` DTO snapshots.
3. Review the diff for localisation or nav noise. Once satisfied, re-run the tests without the env var (`dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests.csproj`) to verify determinism. 3. Review the diff for localisation or nav noise. Once satisfied, re-run the tests without the env var (`dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests/StellaOps.Concelier.Connector.Vndr.Apple.Tests.csproj`) to verify determinism.
4. Commit fixture updates together with any parser/mapping changes that motivated them. 4. Commit fixture updates together with any parser/mapping changes that motivated them.
## 5. Known Issues & Follow-up Tasks ## 5. Known Issues & Follow-up Tasks
- Apple occasionally throttles anonymous requests after bursts. The connector backs off automatically, but persistent `apple.fetch.failures` spikes might require mirroring the HT content or scheduling wider fetch windows. - Apple occasionally throttles anonymous requests after bursts. The connector backs off automatically, but persistent `apple.fetch.failures` spikes might require mirroring the HT content or scheduling wider fetch windows.
- Rapid Security Responses may appear before the general patch notes surface in the lookup JSON. When that happens, the fetch run will log `detailFailures>0`. Collect sample HTML and refresh fixtures to confirm parser coverage. - Rapid Security Responses may appear before the general patch notes surface in the lookup JSON. When that happens, the fetch run will log `detailFailures>0`. Collect sample HTML and refresh fixtures to confirm parser coverage.
- Multi-locale content is still under regression sweep (`src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md`). Capture non-`en-us` snapshots once the fixture tooling stabilises. - Multi-locale content is still under regression sweep (`src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md`). Capture non-`en-us` snapshots once the fixture tooling stabilises.

144
docs/modules/concelier/operations/connectors/cccs.md
View File

@@ -1,72 +1,72 @@
# Concelier CCCS Connector Operations # Concelier CCCS Connector Operations
This runbook covers day‑to‑day operation of the Canadian Centre for Cyber Security (`source:cccs:*`) connector, including configuration, telemetry, and historical backfill guidance for English/French advisories. This runbook covers day‑to‑day operation of the Canadian Centre for Cyber Security (`source:cccs:*`) connector, including configuration, telemetry, and historical backfill guidance for English/French advisories.
## 1. Configuration Checklist ## 1. Configuration Checklist
- Network egress (or mirrored cache) for `https://www.cyber.gc.ca/` and the JSON API endpoints under `/api/cccs/`. - Network egress (or mirrored cache) for `https://www.cyber.gc.ca/` and the JSON API endpoints under `/api/cccs/`.
- Set the Concelier options before restarting workers. Example `concelier.yaml` snippet: - Set the Concelier options before restarting workers. Example `concelier.yaml` snippet:
```yaml ```yaml
concelier: concelier:
sources: sources:
cccs: cccs:
feeds: feeds:
- language: "en" - language: "en"
uri: "https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=en&content_type=cccs_threat" uri: "https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=en&content_type=cccs_threat"
- language: "fr" - language: "fr"
uri: "https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=fr&content_type=cccs_threat" uri: "https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=fr&content_type=cccs_threat"
maxEntriesPerFetch: 80 # increase temporarily for backfill runs maxEntriesPerFetch: 80 # increase temporarily for backfill runs
maxKnownEntries: 512 maxKnownEntries: 512
requestTimeout: "00:00:30" requestTimeout: "00:00:30"
requestDelay: "00:00:00.250" requestDelay: "00:00:00.250"
failureBackoff: "00:05:00" failureBackoff: "00:05:00"
``` ```
> ℹ️ The `/api/cccs/threats/v1/get` endpoint returns thousands of records per language (≈5 100 rows each as of 2025‑10‑14). The connector honours `maxEntriesPerFetch`, so leave it low for steady‑state and raise it for planned backfills. > ℹ️ The `/api/cccs/threats/v1/get` endpoint returns thousands of records per language (≈5 100 rows each as of 2025‑10‑14). The connector honours `maxEntriesPerFetch`, so leave it low for steady‑state and raise it for planned backfills.
## 2. Telemetry & Logging ## 2. Telemetry & Logging
- **Metrics (Meter `StellaOps.Concelier.Connector.Cccs`):** - **Metrics (Meter `StellaOps.Concelier.Connector.Cccs`):**
- `cccs.fetch.attempts`, `cccs.fetch.success`, `cccs.fetch.failures` - `cccs.fetch.attempts`, `cccs.fetch.success`, `cccs.fetch.failures`
- `cccs.fetch.documents`, `cccs.fetch.unchanged` - `cccs.fetch.documents`, `cccs.fetch.unchanged`
- `cccs.parse.success`, `cccs.parse.failures`, `cccs.parse.quarantine` - `cccs.parse.success`, `cccs.parse.failures`, `cccs.parse.quarantine`
- `cccs.map.success`, `cccs.map.failures` - `cccs.map.success`, `cccs.map.failures`
- **Shared HTTP metrics** via `SourceDiagnostics`: - **Shared HTTP metrics** via `SourceDiagnostics`:
- `concelier.source.http.requests{concelier.source="cccs"}` - `concelier.source.http.requests{concelier.source="cccs"}`
- `concelier.source.http.failures{concelier.source="cccs"}` - `concelier.source.http.failures{concelier.source="cccs"}`
- `concelier.source.http.duration{concelier.source="cccs"}` - `concelier.source.http.duration{concelier.source="cccs"}`
- **Structured logs** - **Structured logs**
- `CCCS fetch completed feeds=… items=… newDocuments=… pendingDocuments=…` - `CCCS fetch completed feeds=… items=… newDocuments=… pendingDocuments=…`
- `CCCS parse completed parsed=… failures=…` - `CCCS parse completed parsed=… failures=…`
- `CCCS map completed mapped=… failures=…` - `CCCS map completed mapped=… failures=…`
- Warnings fire when GridFS payloads/DTOs go missing or parser sanitisation fails. - Warnings fire when GridFS payloads/DTOs go missing or parser sanitisation fails.
Suggested Grafana alerts: Suggested Grafana alerts:
- `increase(cccs.fetch.failures_total[15m]) > 0` - `increase(cccs.fetch.failures_total[15m]) > 0`
- `rate(cccs.map.success_total[1h]) == 0` while other connectors are active - `rate(cccs.map.success_total[1h]) == 0` while other connectors are active
- `histogram_quantile(0.95, rate(concelier_source_http_duration_bucket{concelier_source="cccs"}[1h])) > 5s` - `histogram_quantile(0.95, rate(concelier_source_http_duration_bucket{concelier_source="cccs"}[1h])) > 5s`
## 3. Historical Backfill Plan ## 3. Historical Backfill Plan
1. **Snapshot the source** – the API accepts `page=<n>` and `lang=<en|fr>` query parameters. `page=0` returns the full dataset (observed earliest `date_created`: 2018‑06‑08 for EN, 2018‑06‑08 for FR). Mirror those responses into Offline Kit storage when operating air‑gapped. 1. **Snapshot the source** – the API accepts `page=<n>` and `lang=<en|fr>` query parameters. `page=0` returns the full dataset (observed earliest `date_created`: 2018‑06‑08 for EN, 2018‑06‑08 for FR). Mirror those responses into Offline Kit storage when operating air‑gapped.
2. **Stage ingestion**: 2. **Stage ingestion**:
- Temporarily raise `maxEntriesPerFetch` (e.g. 500) and restart Concelier workers. - Temporarily raise `maxEntriesPerFetch` (e.g. 500) and restart Concelier workers.
- Run chained jobs until `pendingDocuments` drains: - Run chained jobs until `pendingDocuments` drains:
`stella db jobs run source:cccs:fetch --and-then source:cccs:parse --and-then source:cccs:map` `stella db jobs run source:cccs:fetch --and-then source:cccs:parse --and-then source:cccs:map`
- Monitor `cccs.fetch.unchanged` growth; once it approaches dataset size the backfill is complete. - Monitor `cccs.fetch.unchanged` growth; once it approaches dataset size the backfill is complete.
3. **Optional pagination sweep** – for incremental mirrors, iterate `page=<n>` (0…N) while `response.Count == 50`, persisting JSON to disk. Store alongside metadata (`language`, `page`, SHA256) so repeated runs detect drift. 3. **Optional pagination sweep** – for incremental mirrors, iterate `page=<n>` (0…N) while `response.Count == 50`, persisting JSON to disk. Store alongside metadata (`language`, `page`, SHA256) so repeated runs detect drift.
4. **Language split** – keep EN/FR payloads separate to preserve canonical language fields. The connector emits `Language` directly from the feed entry, so mixed ingestion simply produces parallel advisories keyed by the same serial number. 4. **Language split** – keep EN/FR payloads separate to preserve canonical language fields. The connector emits `Language` directly from the feed entry, so mixed ingestion simply produces parallel advisories keyed by the same serial number.
5. **Throttle planning** – schedule backfills during maintenance windows; the API tolerates burst downloads but respect the 250 ms request delay or raise it if mirrored traffic is not available. 5. **Throttle planning** – schedule backfills during maintenance windows; the API tolerates burst downloads but respect the 250 ms request delay or raise it if mirrored traffic is not available.
## 4. Selector & Sanitiser Notes ## 4. Selector & Sanitiser Notes
- `CccsHtmlParser` now parses the **unsanitised DOM** (via AngleSharp) and only sanitises when persisting `ContentHtml`. - `CccsHtmlParser` now parses the **unsanitised DOM** (via AngleSharp) and only sanitises when persisting `ContentHtml`.
- Product extraction walks headings (`Affected Products`, `Produits touchés`, `Mesures recommandées`) and consumes nested lists within `div/section/article` containers. - Product extraction walks headings (`Affected Products`, `Produits touchés`, `Mesures recommandées`) and consumes nested lists within `div/section/article` containers.
- `HtmlContentSanitizer` allows `<h1>…<h6>` and `<section>` so stored HTML keeps headings for UI rendering and downstream summarisation. - `HtmlContentSanitizer` allows `<h1>…<h6>` and `<section>` so stored HTML keeps headings for UI rendering and downstream summarisation.
## 5. Fixture Maintenance ## 5. Fixture Maintenance
- Regression fixtures live in `src/Concelier/__Tests/StellaOps.Concelier.Connector.Cccs.Tests/Fixtures`. - Regression fixtures live in `src/Concelier/__Tests/StellaOps.Concelier.Connector.Cccs.Tests/Fixtures`.
- Refresh via `UPDATE_CCCS_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Cccs.Tests/StellaOps.Concelier.Connector.Cccs.Tests.csproj`. - Refresh via `UPDATE_CCCS_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Cccs.Tests/StellaOps.Concelier.Connector.Cccs.Tests.csproj`.
- Fixtures capture both EN/FR advisories with nested lists to guard against sanitiser regressions; review diffs for heading/list changes before committing. - Fixtures capture both EN/FR advisories with nested lists to guard against sanitiser regressions; review diffs for heading/list changes before committing.

286
docs/modules/concelier/operations/connectors/cve-kev.md
View File

@@ -1,143 +1,143 @@
# Concelier CVE & KEV Connector Operations # Concelier CVE & KEV Connector Operations
This playbook equips operators with the steps required to roll out and monitor the CVE Services and CISA KEV connectors across environments. This playbook equips operators with the steps required to roll out and monitor the CVE Services and CISA KEV connectors across environments.
## 1. CVE Services Connector (`source:cve:*`) ## 1. CVE Services Connector (`source:cve:*`)
### 1.1 Prerequisites ### 1.1 Prerequisites
- CVE Services API credentials (organisation ID, user ID, API key) with access to the JSON 5 API. - CVE Services API credentials (organisation ID, user ID, API key) with access to the JSON 5 API.
- Network egress to `https://cveawg.mitre.org` (or a mirrored endpoint) from the Concelier workers. - Network egress to `https://cveawg.mitre.org` (or a mirrored endpoint) from the Concelier workers.
- Updated `concelier.yaml` (or the matching environment variables) with the following section: - Updated `concelier.yaml` (or the matching environment variables) with the following section:
```yaml ```yaml
concelier: concelier:
sources: sources:
cve: cve:
baseEndpoint: "https://cveawg.mitre.org/api/" baseEndpoint: "https://cveawg.mitre.org/api/"
apiOrg: "ORG123" apiOrg: "ORG123"
apiUser: "user@example.org" apiUser: "user@example.org"
apiKeyFile: "/var/run/secrets/concelier/cve-api-key" apiKeyFile: "/var/run/secrets/concelier/cve-api-key"
seedDirectory: "./seed-data/cve" seedDirectory: "./seed-data/cve"
pageSize: 200 pageSize: 200
maxPagesPerFetch: 5 maxPagesPerFetch: 5
initialBackfill: "30.00:00:00" initialBackfill: "30.00:00:00"
requestDelay: "00:00:00.250" requestDelay: "00:00:00.250"
failureBackoff: "00:10:00" failureBackoff: "00:10:00"
``` ```
> ℹ️ Store the API key outside source control. When using `apiKeyFile`, mount the secret file into the container/host; alternatively supply `apiKey` via `CONCELIER_SOURCES__CVE__APIKEY`. > ℹ️ Store the API key outside source control. When using `apiKeyFile`, mount the secret file into the container/host; alternatively supply `apiKey` via `CONCELIER_SOURCES__CVE__APIKEY`.
> 🪙 When credentials are not yet available, configure `seedDirectory` to point at mirrored CVE JSON (for example, the repo’s `seed-data/cve/` bundle). The connector will ingest those records and log a warning instead of failing the job; live fetching resumes automatically once `apiOrg` / `apiUser` / `apiKey` are supplied. > 🪙 When credentials are not yet available, configure `seedDirectory` to point at mirrored CVE JSON (for example, the repo’s `seed-data/cve/` bundle). The connector will ingest those records and log a warning instead of failing the job; live fetching resumes automatically once `apiOrg` / `apiUser` / `apiKey` are supplied.
### 1.2 Smoke Test (staging) ### 1.2 Smoke Test (staging)
1. Deploy the updated configuration and restart the Concelier service so the connector picks up the credentials. 1. Deploy the updated configuration and restart the Concelier service so the connector picks up the credentials.
2. Trigger one end-to-end cycle: 2. Trigger one end-to-end cycle:
- Concelier CLI: `stella db jobs run source:cve:fetch --and-then source:cve:parse --and-then source:cve:map` - Concelier CLI: `stella db jobs run source:cve:fetch --and-then source:cve:parse --and-then source:cve:map`
- REST fallback: `POST /jobs/run { "kind": "source:cve:fetch", "chain": ["source:cve:parse", "source:cve:map"] }` - REST fallback: `POST /jobs/run { "kind": "source:cve:fetch", "chain": ["source:cve:parse", "source:cve:map"] }`
3. Observe the following metrics (exported via OTEL meter `StellaOps.Concelier.Connector.Cve`): 3. Observe the following metrics (exported via OTEL meter `StellaOps.Concelier.Connector.Cve`):
- `cve.fetch.attempts`, `cve.fetch.success`, `cve.fetch.documents`, `cve.fetch.failures`, `cve.fetch.unchanged` - `cve.fetch.attempts`, `cve.fetch.success`, `cve.fetch.documents`, `cve.fetch.failures`, `cve.fetch.unchanged`
- `cve.parse.success`, `cve.parse.failures`, `cve.parse.quarantine` - `cve.parse.success`, `cve.parse.failures`, `cve.parse.quarantine`
- `cve.map.success` - `cve.map.success`
4. Verify Prometheus shows matching `concelier.source.http.requests_total{concelier_source="cve"}` deltas (list vs detail phases) while `concelier.source.http.failures_total{concelier_source="cve"}` stays flat. 4. Verify Prometheus shows matching `concelier.source.http.requests_total{concelier_source="cve"}` deltas (list vs detail phases) while `concelier.source.http.failures_total{concelier_source="cve"}` stays flat.
5. Confirm the info-level summary log `CVEs fetch window … pages=X detailDocuments=Y detailFailures=Z` appears once per fetch run and shows `detailFailures=0`. 5. Confirm the info-level summary log `CVEs fetch window … pages=X detailDocuments=Y detailFailures=Z` appears once per fetch run and shows `detailFailures=0`.
6. Verify the MongoDB advisory store contains fresh CVE advisories (`advisoryKey` prefix `cve/`) and that the source cursor (`source_states` collection) advanced. 6. Verify the MongoDB advisory store contains fresh CVE advisories (`advisoryKey` prefix `cve/`) and that the source cursor (`source_states` collection) advanced.
### 1.3 Production Monitoring ### 1.3 Production Monitoring
- **Dashboards** – Plot `rate(cve_fetch_success_total[5m])`, `rate(cve_fetch_failures_total[5m])`, and `rate(cve_fetch_documents_total[5m])` alongside `concelier_source_http_requests_total{concelier_source="cve"}` to confirm HTTP and connector counters stay aligned. Keep `concelier.range.primitives{scheme=~"semver|vendor"}` on the same board for range coverage. Example alerts: - **Dashboards** – Plot `rate(cve_fetch_success_total[5m])`, `rate(cve_fetch_failures_total[5m])`, and `rate(cve_fetch_documents_total[5m])` alongside `concelier_source_http_requests_total{concelier_source="cve"}` to confirm HTTP and connector counters stay aligned. Keep `concelier.range.primitives{scheme=~"semver|vendor"}` on the same board for range coverage. Example alerts:
- `rate(cve_fetch_failures_total[5m]) > 0` for 10 minutes (`severity=warning`) - `rate(cve_fetch_failures_total[5m]) > 0` for 10 minutes (`severity=warning`)
- `rate(cve_map_success_total[15m]) == 0` while `rate(cve_fetch_success_total[15m]) > 0` (`severity=critical`) - `rate(cve_map_success_total[15m]) == 0` while `rate(cve_fetch_success_total[15m]) > 0` (`severity=critical`)
- `sum_over_time(cve_parse_quarantine_total[1h]) > 0` to catch schema anomalies - `sum_over_time(cve_parse_quarantine_total[1h]) > 0` to catch schema anomalies
- **Logs** – Monitor warnings such as `Failed fetching CVE record {CveId}` and `Malformed CVE JSON`, and surface the summary info log `CVEs fetch window … detailFailures=0 detailUnchanged=0` on dashboards. A non-zero `detailFailures` usually indicates rate-limit or auth issues on detail requests. - **Logs** – Monitor warnings such as `Failed fetching CVE record {CveId}` and `Malformed CVE JSON`, and surface the summary info log `CVEs fetch window … detailFailures=0 detailUnchanged=0` on dashboards. A non-zero `detailFailures` usually indicates rate-limit or auth issues on detail requests.
- **Grafana pack** – Import `docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json` and filter by panel legend (`CVE`, `KEV`) to reuse the canned layout. - **Grafana pack** – Import `docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json` and filter by panel legend (`CVE`, `KEV`) to reuse the canned layout.
- **Backfill window** – Operators can tighten or widen `initialBackfill` / `maxPagesPerFetch` after validating throughput. Update config and restart Concelier to apply changes. - **Backfill window** – Operators can tighten or widen `initialBackfill` / `maxPagesPerFetch` after validating throughput. Update config and restart Concelier to apply changes.
### 1.4 Staging smoke log (2025-10-15) ### 1.4 Staging smoke log (2025-10-15)
While Ops finalises long-lived CVE Services credentials, we validated the connector end-to-end against the recorded CVE-2024-0001 payloads used in regression tests: While Ops finalises long-lived CVE Services credentials, we validated the connector end-to-end against the recorded CVE-2024-0001 payloads used in regression tests:
- Command: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve.Tests/StellaOps.Concelier.Connector.Cve.Tests.csproj -l "console;verbosity=detailed"` - Command: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve.Tests/StellaOps.Concelier.Connector.Cve.Tests.csproj -l "console;verbosity=detailed"`
- Summary log emitted by the connector: - Summary log emitted by the connector:
``` ```
CVEs fetch window 2024-09-01T00:00:00Z->2024-10-01T00:00:00Z pages=1 listSuccess=1 detailDocuments=1 detailFailures=0 detailUnchanged=0 pendingDocuments=0->1 pendingMappings=0->1 hasMorePages=False nextWindowStart=2024-09-15T12:00:00Z nextWindowEnd=(none) nextPage=1 CVEs fetch window 2024-09-01T00:00:00Z->2024-10-01T00:00:00Z pages=1 listSuccess=1 detailDocuments=1 detailFailures=0 detailUnchanged=0 pendingDocuments=0->1 pendingMappings=0->1 hasMorePages=False nextWindowStart=2024-09-15T12:00:00Z nextWindowEnd=(none) nextPage=1
``` ```
- Telemetry captured by `Meter` `StellaOps.Concelier.Connector.Cve`: - Telemetry captured by `Meter` `StellaOps.Concelier.Connector.Cve`:
| Metric | Value | | Metric | Value |
|--------|-------| |--------|-------|
| `cve.fetch.attempts` | 1 | | `cve.fetch.attempts` | 1 |
| `cve.fetch.success` | 1 | | `cve.fetch.success` | 1 |
| `cve.fetch.documents` | 1 | | `cve.fetch.documents` | 1 |
| `cve.parse.success` | 1 | | `cve.parse.success` | 1 |
| `cve.map.success` | 1 | | `cve.map.success` | 1 |
The Grafana pack `docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json` has been imported into staging so the panels referenced above render against these counters once the live API keys are in place. The Grafana pack `docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json` has been imported into staging so the panels referenced above render against these counters once the live API keys are in place.
## 2. CISA KEV Connector (`source:kev:*`) ## 2. CISA KEV Connector (`source:kev:*`)
### 2.1 Prerequisites ### 2.1 Prerequisites
- Network egress (or mirrored content) for `https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json`. - Network egress (or mirrored content) for `https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json`.
- No credentials are required, but the HTTP allow-list must include `www.cisa.gov`. - No credentials are required, but the HTTP allow-list must include `www.cisa.gov`.
- Confirm the following snippet in `concelier.yaml` (defaults shown; tune as needed): - Confirm the following snippet in `concelier.yaml` (defaults shown; tune as needed):
```yaml ```yaml
concelier: concelier:
sources: sources:
kev: kev:
feedUri: "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json" feedUri: "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"
requestTimeout: "00:01:00" requestTimeout: "00:01:00"
failureBackoff: "00:05:00" failureBackoff: "00:05:00"
``` ```
### 2.2 Schema validation & anomaly handling ### 2.2 Schema validation & anomaly handling
The connector validates each catalog against `Schemas/kev-catalog.schema.json`. Failures increment `kev.parse.failures_total{reason="schema"}` and the document is quarantined (status `Failed`). Additional failure reasons include `download`, `invalidJson`, `deserialize`, `missingPayload`, and `emptyCatalog`. Entry-level anomalies are surfaced through `kev.parse.anomalies_total` with reasons: The connector validates each catalog against `Schemas/kev-catalog.schema.json`. Failures increment `kev.parse.failures_total{reason="schema"}` and the document is quarantined (status `Failed`). Additional failure reasons include `download`, `invalidJson`, `deserialize`, `missingPayload`, and `emptyCatalog`. Entry-level anomalies are surfaced through `kev.parse.anomalies_total` with reasons:
| Reason | Meaning | | Reason | Meaning |
| --- | --- | | --- | --- |
| `missingCveId` | Catalog entry omitted `cveID`; the entry is skipped. | | `missingCveId` | Catalog entry omitted `cveID`; the entry is skipped. |
| `countMismatch` | Catalog `count` field disagreed with the actual entry total. | | `countMismatch` | Catalog `count` field disagreed with the actual entry total. |
| `nullEntry` | Upstream emitted a `null` entry object (rare upstream defect). | | `nullEntry` | Upstream emitted a `null` entry object (rare upstream defect). |
Treat repeated schema failures or growing anomaly counts as an upstream regression and coordinate with CISA or mirror maintainers. Treat repeated schema failures or growing anomaly counts as an upstream regression and coordinate with CISA or mirror maintainers.
### 2.3 Smoke Test (staging) ### 2.3 Smoke Test (staging)
1. Deploy the configuration and restart Concelier. 1. Deploy the configuration and restart Concelier.
2. Trigger a pipeline run: 2. Trigger a pipeline run:
- CLI: `stella db jobs run source:kev:fetch --and-then source:kev:parse --and-then source:kev:map` - CLI: `stella db jobs run source:kev:fetch --and-then source:kev:parse --and-then source:kev:map`
- REST: `POST /jobs/run { "kind": "source:kev:fetch", "chain": ["source:kev:parse", "source:kev:map"] }` - REST: `POST /jobs/run { "kind": "source:kev:fetch", "chain": ["source:kev:parse", "source:kev:map"] }`
3. Verify the metrics exposed by meter `StellaOps.Concelier.Connector.Kev`: 3. Verify the metrics exposed by meter `StellaOps.Concelier.Connector.Kev`:
- `kev.fetch.attempts`, `kev.fetch.success`, `kev.fetch.unchanged`, `kev.fetch.failures` - `kev.fetch.attempts`, `kev.fetch.success`, `kev.fetch.unchanged`, `kev.fetch.failures`
- `kev.parse.entries` (tag `catalogVersion`), `kev.parse.failures`, `kev.parse.anomalies` (tag `reason`) - `kev.parse.entries` (tag `catalogVersion`), `kev.parse.failures`, `kev.parse.anomalies` (tag `reason`)
- `kev.map.advisories` (tag `catalogVersion`) - `kev.map.advisories` (tag `catalogVersion`)
4. Confirm `concelier.source.http.requests_total{concelier_source="kev"}` increments once per fetch and that the paired `concelier.source.http.failures_total` stays flat (zero increase). 4. Confirm `concelier.source.http.requests_total{concelier_source="kev"}` increments once per fetch and that the paired `concelier.source.http.failures_total` stays flat (zero increase).
5. Inspect the info logs `Fetched KEV catalog document … pendingDocuments=…` and `Parsed KEV catalog document … entries=…`—they should appear exactly once per run and `Mapped X/Y… skipped=0` should match the `kev.map.advisories` delta. 5. Inspect the info logs `Fetched KEV catalog document … pendingDocuments=…` and `Parsed KEV catalog document … entries=…`—they should appear exactly once per run and `Mapped X/Y… skipped=0` should match the `kev.map.advisories` delta.
6. Confirm MongoDB documents exist for the catalog JSON (`raw_documents` & `dtos`) and that advisories with prefix `kev/` are written. 6. Confirm MongoDB documents exist for the catalog JSON (`raw_documents` & `dtos`) and that advisories with prefix `kev/` are written.
### 2.4 Production Monitoring ### 2.4 Production Monitoring
- Alert when `rate(kev_fetch_success_total[8h]) == 0` during working hours (daily cadence breach) and when `increase(kev_fetch_failures_total[1h]) > 0`. - Alert when `rate(kev_fetch_success_total[8h]) == 0` during working hours (daily cadence breach) and when `increase(kev_fetch_failures_total[1h]) > 0`.
- Page the on-call if `increase(kev_parse_failures_total{reason="schema"}[6h]) > 0`—this usually signals an upstream payload change. Treat repeated `reason="download"` spikes as networking issues to the mirror. - Page the on-call if `increase(kev_parse_failures_total{reason="schema"}[6h]) > 0`—this usually signals an upstream payload change. Treat repeated `reason="download"` spikes as networking issues to the mirror.
- Track anomaly spikes through `sum_over_time(kev_parse_anomalies_total{reason="missingCveId"}[24h])`. Rising `countMismatch` trends point to catalog publishing bugs. - Track anomaly spikes through `sum_over_time(kev_parse_anomalies_total{reason="missingCveId"}[24h])`. Rising `countMismatch` trends point to catalog publishing bugs.
- Surface the fetch/mapping info logs (`Fetched KEV catalog document …` and `Mapped X/Y KEV advisories … skipped=S`) on dashboards; absence of those logs while metrics show success typically means schema validation short-circuited the run. - Surface the fetch/mapping info logs (`Fetched KEV catalog document …` and `Mapped X/Y KEV advisories … skipped=S`) on dashboards; absence of those logs while metrics show success typically means schema validation short-circuited the run.
### 2.5 Known good dashboard tiles ### 2.5 Known good dashboard tiles
Add the following panels to the Concelier observability board: Add the following panels to the Concelier observability board:
| Metric | Recommended visualisation | | Metric | Recommended visualisation |
|--------|---------------------------| |--------|---------------------------|
| `rate(kev_fetch_success_total[30m])` | Single-stat (last 24 h) with warning threshold `>0` | | `rate(kev_fetch_success_total[30m])` | Single-stat (last 24 h) with warning threshold `>0` |
| `rate(kev_parse_entries_total[1h])` by `catalogVersion` | Stacked area – highlights daily release size | | `rate(kev_parse_entries_total[1h])` by `catalogVersion` | Stacked area – highlights daily release size |
| `sum_over_time(kev_parse_anomalies_total[1d])` by `reason` | Table – anomaly breakdown (matches dashboard panel) | | `sum_over_time(kev_parse_anomalies_total[1d])` by `reason` | Table – anomaly breakdown (matches dashboard panel) |
| `rate(cve_map_success_total[15m])` vs `rate(kev_map_advisories_total[24h])` | Comparative timeseries for advisories emitted | | `rate(cve_map_success_total[15m])` vs `rate(kev_map_advisories_total[24h])` | Comparative timeseries for advisories emitted |
## 3. Runbook updates ## 3. Runbook updates
- Record staging/production smoke test results (date, catalog version, advisory counts) in your team’s change log. - Record staging/production smoke test results (date, catalog version, advisory counts) in your team’s change log.
- Add the CVE/KEV job kinds to the standard maintenance checklist so operators can manually trigger them after planned downtime. - Add the CVE/KEV job kinds to the standard maintenance checklist so operators can manually trigger them after planned downtime.
- Keep this document in sync with future connector changes (for example, new anomaly reasons or additional metrics). - Keep this document in sync with future connector changes (for example, new anomaly reasons or additional metrics).
- Version-control dashboard tweaks alongside `docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json` so operations can re-import the observability pack during restores. - Version-control dashboard tweaks alongside `docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json` so operations can re-import the observability pack during restores.

148
docs/modules/concelier/operations/connectors/kisa.md
View File

@@ -1,74 +1,74 @@
# Concelier KISA Connector Operations # Concelier KISA Connector Operations
Operational guidance for the Korea Internet & Security Agency (KISA / KNVD) connector (`source:kisa:*`). Pair this with the engineering brief in `docs/dev/kisa_connector_notes.md`. Operational guidance for the Korea Internet & Security Agency (KISA / KNVD) connector (`source:kisa:*`). Pair this with the engineering brief in `docs/dev/kisa_connector_notes.md`.
## 1. Prerequisites ## 1. Prerequisites
- Outbound HTTPS (or mirrored cache) for `https://knvd.krcert.or.kr/`. - Outbound HTTPS (or mirrored cache) for `https://knvd.krcert.or.kr/`.
- Connector options defined under `concelier:sources:kisa`: - Connector options defined under `concelier:sources:kisa`:
```yaml ```yaml
concelier: concelier:
sources: sources:
kisa: kisa:
feedUri: "https://knvd.krcert.or.kr/rss/securityInfo.do" feedUri: "https://knvd.krcert.or.kr/rss/securityInfo.do"
detailApiUri: "https://knvd.krcert.or.kr/rssDetailData.do" detailApiUri: "https://knvd.krcert.or.kr/rssDetailData.do"
detailPageUri: "https://knvd.krcert.or.kr/detailDos.do" detailPageUri: "https://knvd.krcert.or.kr/detailDos.do"
maxAdvisoriesPerFetch: 10 maxAdvisoriesPerFetch: 10
requestDelay: "00:00:01" requestDelay: "00:00:01"
failureBackoff: "00:05:00" failureBackoff: "00:05:00"
``` ```
> Ensure the URIs stay absolute—Concelier adds the `feedUri`/`detailApiUri` hosts to the HttpClient allow-list automatically. > Ensure the URIs stay absolute—Concelier adds the `feedUri`/`detailApiUri` hosts to the HttpClient allow-list automatically.
## 2. Staging Smoke Test ## 2. Staging Smoke Test
1. Restart the Concelier workers so the KISA options bind. 1. Restart the Concelier workers so the KISA options bind.
2. Run a full connector cycle: 2. Run a full connector cycle:
- CLI: `stella db jobs run source:kisa:fetch --and-then source:kisa:parse --and-then source:kisa:map` - CLI: `stella db jobs run source:kisa:fetch --and-then source:kisa:parse --and-then source:kisa:map`
- REST: `POST /jobs/run { "kind": "source:kisa:fetch", "chain": ["source:kisa:parse", "source:kisa:map"] }` - REST: `POST /jobs/run { "kind": "source:kisa:fetch", "chain": ["source:kisa:parse", "source:kisa:map"] }`
3. Confirm telemetry (Meter `StellaOps.Concelier.Connector.Kisa`): 3. Confirm telemetry (Meter `StellaOps.Concelier.Connector.Kisa`):
- `kisa.feed.success`, `kisa.feed.items` - `kisa.feed.success`, `kisa.feed.items`
- `kisa.detail.success` / `.failures` - `kisa.detail.success` / `.failures`
- `kisa.parse.success` / `.failures` - `kisa.parse.success` / `.failures`
- `kisa.map.success` / `.failures` - `kisa.map.success` / `.failures`
- `kisa.cursor.updates` - `kisa.cursor.updates`
4. Inspect logs for structured entries: 4. Inspect logs for structured entries:
- `KISA feed returned {ItemCount}` - `KISA feed returned {ItemCount}`
- `KISA fetched detail for {Idx} … category={Category}` - `KISA fetched detail for {Idx} … category={Category}`
- `KISA mapped advisory {AdvisoryId} (severity={Severity})` - `KISA mapped advisory {AdvisoryId} (severity={Severity})`
- Absence of warnings such as `document missing GridFS payload`. - Absence of warnings such as `document missing GridFS payload`.
5. Validate MongoDB state: 5. Validate MongoDB state:
- `raw_documents.metadata` has `kisa.idx`, `kisa.category`, `kisa.title`. - `raw_documents.metadata` has `kisa.idx`, `kisa.category`, `kisa.title`.
- DTO store contains `schemaVersion="kisa.detail.v1"`. - DTO store contains `schemaVersion="kisa.detail.v1"`.
- Advisories include aliases (`IDX`, CVE) and `language="ko"`. - Advisories include aliases (`IDX`, CVE) and `language="ko"`.
- `source_states` entry for `kisa` shows recent `cursor.lastFetchAt`. - `source_states` entry for `kisa` shows recent `cursor.lastFetchAt`.
## 3. Production Monitoring ## 3. Production Monitoring
- **Dashboards** – Add the following Prometheus/OTEL expressions: - **Dashboards** – Add the following Prometheus/OTEL expressions:
- `rate(kisa_feed_items_total[15m])` versus `rate(concelier_source_http_requests_total{concelier_source="kisa"}[15m])` - `rate(kisa_feed_items_total[15m])` versus `rate(concelier_source_http_requests_total{concelier_source="kisa"}[15m])`
- `increase(kisa_detail_failures_total{reason!="empty-document"}[1h])` alert at `>0` - `increase(kisa_detail_failures_total{reason!="empty-document"}[1h])` alert at `>0`
- `increase(kisa_parse_failures_total[1h])` for storage/JSON issues - `increase(kisa_parse_failures_total[1h])` for storage/JSON issues
- `increase(kisa_map_failures_total[1h])` to flag schema drift - `increase(kisa_map_failures_total[1h])` to flag schema drift
- `increase(kisa_cursor_updates_total[6h]) == 0` during active windows → warn - `increase(kisa_cursor_updates_total[6h]) == 0` during active windows → warn
- **Alerts** – Page when `rate(kisa_feed_success_total[2h]) == 0` while other connectors are active; back off for maintenance windows announced on `https://knvd.krcert.or.kr/`. - **Alerts** – Page when `rate(kisa_feed_success_total[2h]) == 0` while other connectors are active; back off for maintenance windows announced on `https://knvd.krcert.or.kr/`.
- **Logs** – Watch for repeated warnings (`document missing`, `DTO missing`) or errors with reason tags `HttpRequestException`, `download`, `parse`, `map`. - **Logs** – Watch for repeated warnings (`document missing`, `DTO missing`) or errors with reason tags `HttpRequestException`, `download`, `parse`, `map`.
## 4. Localisation Handling ## 4. Localisation Handling
- Hangul categories (for example `취약점정보`) flow into telemetry tags (`category=…`) and logs. Dashboards must render UTF‑8 and avoid transliteration. - Hangul categories (for example `취약점정보`) flow into telemetry tags (`category=…`) and logs. Dashboards must render UTF‑8 and avoid transliteration.
- HTML content is sanitised before storage; translation teams can consume the `ContentHtml` field safely. - HTML content is sanitised before storage; translation teams can consume the `ContentHtml` field safely.
- Advisory severity remains as provided by KISA (`High`, `Medium`, etc.). Map-level failures include the severity tag for filtering. - Advisory severity remains as provided by KISA (`High`, `Medium`, etc.). Map-level failures include the severity tag for filtering.
## 5. Fixture & Regression Maintenance ## 5. Fixture & Regression Maintenance
- Regression fixtures: `src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/Fixtures/kisa-feed.xml` and `kisa-detail.json`. - Regression fixtures: `src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/Fixtures/kisa-feed.xml` and `kisa-detail.json`.
- Refresh via `UPDATE_KISA_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/StellaOps.Concelier.Connector.Kisa.Tests.csproj`. - Refresh via `UPDATE_KISA_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/StellaOps.Concelier.Connector.Kisa.Tests.csproj`.
- The telemetry regression (`KisaConnectorTests.Telemetry_RecordsMetrics`) will fail if counters/log wiring drifts—treat failures as gating. - The telemetry regression (`KisaConnectorTests.Telemetry_RecordsMetrics`) will fail if counters/log wiring drifts—treat failures as gating.
## 6. Known Issues ## 6. Known Issues
- RSS feeds only expose the latest 10 advisories; long outages require replay via archived feeds or manual IDX seeds. - RSS feeds only expose the latest 10 advisories; long outages require replay via archived feeds or manual IDX seeds.
- Detail endpoint occasionally throttles; the connector honours `requestDelay` and reports failures with reason `HttpRequestException`. Consider increasing delay for weekend backfills. - Detail endpoint occasionally throttles; the connector honours `requestDelay` and reports failures with reason `HttpRequestException`. Consider increasing delay for weekend backfills.
- If `kisa.category` tags suddenly appear as `unknown`, verify KISA has not renamed RSS elements; update the parser fixtures before production rollout. - If `kisa.category` tags suddenly appear as `unknown`, verify KISA has not renamed RSS elements; update the parser fixtures before production rollout.

96
docs/modules/concelier/operations/connectors/nkcki.md
View File

@@ -1,48 +1,48 @@
# NKCKI Connector Operations Guide # NKCKI Connector Operations Guide
## Overview ## Overview
The NKCKI connector ingests JSON bulletin archives from cert.gov.ru, expanding each `*.json.zip` attachment into per-vulnerability DTOs before canonical mapping. The fetch pipeline now supports cache-backed recovery, deterministic pagination, and telemetry suitable for production monitoring. The NKCKI connector ingests JSON bulletin archives from cert.gov.ru, expanding each `*.json.zip` attachment into per-vulnerability DTOs before canonical mapping. The fetch pipeline now supports cache-backed recovery, deterministic pagination, and telemetry suitable for production monitoring.
## Configuration ## Configuration
Key options exposed through `concelier:sources:ru-nkcki:http`: Key options exposed through `concelier:sources:ru-nkcki:http`:
- `maxBulletinsPerFetch` – limits new bulletin downloads in a single run (default `5`). - `maxBulletinsPerFetch` – limits new bulletin downloads in a single run (default `5`).
- `maxListingPagesPerFetch` – maximum listing pages visited during pagination (default `3`). - `maxListingPagesPerFetch` – maximum listing pages visited during pagination (default `3`).
- `listingCacheDuration` – minimum interval between listing fetches before falling back to cached artefacts (default `00:10:00`). - `listingCacheDuration` – minimum interval between listing fetches before falling back to cached artefacts (default `00:10:00`).
- `cacheDirectory` – optional path for persisted bulletin archives used during offline or failure scenarios. - `cacheDirectory` – optional path for persisted bulletin archives used during offline or failure scenarios.
- `requestDelay` – delay inserted between bulletin downloads to respect upstream politeness. - `requestDelay` – delay inserted between bulletin downloads to respect upstream politeness.
When operating in offline-first mode, set `cacheDirectory` to a writable path (e.g. `/var/lib/concelier/cache/ru-nkcki`) and pre-populate bulletin archives via the offline kit. When operating in offline-first mode, set `cacheDirectory` to a writable path (e.g. `/var/lib/concelier/cache/ru-nkcki`) and pre-populate bulletin archives via the offline kit.
## Telemetry ## Telemetry
`RuNkckiDiagnostics` emits the following metrics under meter `StellaOps.Concelier.Connector.Ru.Nkcki`: `RuNkckiDiagnostics` emits the following metrics under meter `StellaOps.Concelier.Connector.Ru.Nkcki`:
- `nkcki.listing.fetch.attempts` / `nkcki.listing.fetch.success` / `nkcki.listing.fetch.failures` - `nkcki.listing.fetch.attempts` / `nkcki.listing.fetch.success` / `nkcki.listing.fetch.failures`
- `nkcki.listing.pages.visited` (histogram, `pages`) - `nkcki.listing.pages.visited` (histogram, `pages`)
- `nkcki.listing.attachments.discovered` / `nkcki.listing.attachments.new` - `nkcki.listing.attachments.discovered` / `nkcki.listing.attachments.new`
- `nkcki.bulletin.fetch.success` / `nkcki.bulletin.fetch.cached` / `nkcki.bulletin.fetch.failures` - `nkcki.bulletin.fetch.success` / `nkcki.bulletin.fetch.cached` / `nkcki.bulletin.fetch.failures`
- `nkcki.entries.processed` (histogram, `entries`) - `nkcki.entries.processed` (histogram, `entries`)
Integrate these counters into standard Concelier observability dashboards to track crawl coverage and cache hit rates. Integrate these counters into standard Concelier observability dashboards to track crawl coverage and cache hit rates.
## Archive Backfill Strategy ## Archive Backfill Strategy
Bitrix pagination surfaces archives via `?PAGEN_1=n`. The connector now walks up to `maxListingPagesPerFetch` pages, deduplicating bulletin IDs and maintaining a rolling `knownBulletins` window. Backfill strategy: Bitrix pagination surfaces archives via `?PAGEN_1=n`. The connector now walks up to `maxListingPagesPerFetch` pages, deduplicating bulletin IDs and maintaining a rolling `knownBulletins` window. Backfill strategy:
1. Enumerate pages from newest to oldest, respecting `maxListingPagesPerFetch` and `listingCacheDuration` to avoid refetch storms. 1. Enumerate pages from newest to oldest, respecting `maxListingPagesPerFetch` and `listingCacheDuration` to avoid refetch storms.
2. Persist every `*.json.zip` attachment to the configured cache directory. This enables replay when listing access is temporarily blocked. 2. Persist every `*.json.zip` attachment to the configured cache directory. This enables replay when listing access is temporarily blocked.
3. During archive replay, `ProcessCachedBulletinsAsync` enqueues missing documents while respecting `maxVulnerabilitiesPerFetch`. 3. During archive replay, `ProcessCachedBulletinsAsync` enqueues missing documents while respecting `maxVulnerabilitiesPerFetch`.
4. For historical HTML-only advisories, collect page URLs and metadata while offline (future work: HTML and PDF extraction pipeline documented in `docs/concelier-connector-research-20251011.md`). 4. For historical HTML-only advisories, collect page URLs and metadata while offline (future work: HTML and PDF extraction pipeline documented in `docs/concelier-connector-research-20251011.md`).
For large migrations, seed caches with archived zip bundles, then run fetch/parse/map cycles in chronological order to maintain deterministic outputs. For large migrations, seed caches with archived zip bundles, then run fetch/parse/map cycles in chronological order to maintain deterministic outputs.
## Failure Handling ## Failure Handling
- Listing failures mark the source state with exponential backoff while attempting cache replay. - Listing failures mark the source state with exponential backoff while attempting cache replay.
- Bulletin fetches fall back to cached copies before surfacing an error. - Bulletin fetches fall back to cached copies before surfacing an error.
- Mongo integration tests rely on bundled OpenSSL 1.1 libraries (`src/Tools/openssl/linux-x64`) to keep `Mongo2Go` operational on modern distros. - Mongo integration tests rely on bundled OpenSSL 1.1 libraries (`src/Tools/openssl/linux-x64`) to keep `Mongo2Go` operational on modern distros.
Refer to `ru-nkcki` entries in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md` for outstanding items. Refer to `ru-nkcki` entries in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md` for outstanding items.

48
docs/modules/concelier/operations/connectors/osv.md
View File

@@ -1,24 +1,24 @@
# Concelier OSV Connector – Operations Notes # Concelier OSV Connector – Operations Notes
_Last updated: 2025-10-16_ _Last updated: 2025-10-16_
The OSV connector ingests advisories from OSV.dev across OSS ecosystems. This note highlights the additional merge/export expectations introduced with the canonical metric fallback work in Sprint 4. The OSV connector ingests advisories from OSV.dev across OSS ecosystems. This note highlights the additional merge/export expectations introduced with the canonical metric fallback work in Sprint 4.
## 1. Canonical metric fallbacks ## 1. Canonical metric fallbacks
- When OSV omits CVSS vectors (common for CVSS v4-only payloads) the mapper now emits a deterministic canonical metric id in the form `osv:severity/<level>` and normalises the advisory severity to the same `<level>`. - When OSV omits CVSS vectors (common for CVSS v4-only payloads) the mapper now emits a deterministic canonical metric id in the form `osv:severity/<level>` and normalises the advisory severity to the same `<level>`.
- Metric: `osv.map.canonical_metric_fallbacks` (counter) with tags `severity`, `canonical_metric_id`, `ecosystem`, `reason=no_cvss`. Watch this alongside merge parity dashboards to catch spikes where OSV publishes severity-only advisories. - Metric: `osv.map.canonical_metric_fallbacks` (counter) with tags `severity`, `canonical_metric_id`, `ecosystem`, `reason=no_cvss`. Watch this alongside merge parity dashboards to catch spikes where OSV publishes severity-only advisories.
- Merge precedence still prefers GHSA over OSV; the shared severity-based canonical id keeps Merge/export parity deterministic even when only OSV supplies severity data. - Merge precedence still prefers GHSA over OSV; the shared severity-based canonical id keeps Merge/export parity deterministic even when only OSV supplies severity data.
## 2. CWE provenance ## 2. CWE provenance
- `database_specific.cwe_ids` now populates provenance decision reasons for every mapped weakness. Expect `decisionReason="database_specific.cwe_ids"` on OSV weakness provenance and confirm exporters preserve the value. - `database_specific.cwe_ids` now populates provenance decision reasons for every mapped weakness. Expect `decisionReason="database_specific.cwe_ids"` on OSV weakness provenance and confirm exporters preserve the value.
- If OSV ever attaches `database_specific.cwe_notes`, the connector will surface the joined note string in `decisionReason` instead of the default marker. - If OSV ever attaches `database_specific.cwe_notes`, the connector will surface the joined note string in `decisionReason` instead of the default marker.
## 3. Dashboards & alerts ## 3. Dashboards & alerts
- Extend existing merge dashboards with the new counter: - Extend existing merge dashboards with the new counter:
- Overlay `sum(osv.map.canonical_metric_fallbacks{ecosystem=~".+"})` with Merge severity overrides to confirm fallback advisories are reconciling cleanly. - Overlay `sum(osv.map.canonical_metric_fallbacks{ecosystem=~".+"})` with Merge severity overrides to confirm fallback advisories are reconciling cleanly.
- Alert when the 1-hour sum exceeds 50 for any ecosystem; baseline volume is currently <5 per day (mostly GHSA mirrors emitting CVSS v4 only). - Alert when the 1-hour sum exceeds 50 for any ecosystem; baseline volume is currently <5 per day (mostly GHSA mirrors emitting CVSS v4 only).
- Exporters already surface `canonicalMetricId`; no schema change is required, but ORAS/Trivy bundles should be spot-checked after deploying the connector update. - Exporters already surface `canonicalMetricId`; no schema change is required, but ORAS/Trivy bundles should be spot-checked after deploying the connector update.
## 4. Runbook updates ## 4. Runbook updates
- Fixture parity suites (`osv-ghsa.*`) now assert the fallback id and provenance notes. Regenerate via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`. - Fixture parity suites (`osv-ghsa.*`) now assert the fallback id and provenance notes. Regenerate via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`.
- When investigating merge severity conflicts, include the fallback counter and confirm OSV advisories carry the expected `osv:severity/<level>` id before raising connector bugs. - When investigating merge severity conflicts, include the fallback counter and confirm OSV advisories carry the expected `osv:severity/<level>` id before raising connector bugs.

476
docs/modules/concelier/operations/mirror.md
View File

@@ -1,238 +1,238 @@
# Concelier & Excititor Mirror Operations # Concelier & Excititor Mirror Operations
This runbook describes how Stella Ops operates the managed mirrors under `*.stella-ops.org`. This runbook describes how Stella Ops operates the managed mirrors under `*.stella-ops.org`.
It covers Docker Compose and Helm deployment overlays, secret handling for multi-tenant It covers Docker Compose and Helm deployment overlays, secret handling for multi-tenant
authn, CDN fronting, and the recurring sync pipeline that keeps mirror bundles current. authn, CDN fronting, and the recurring sync pipeline that keeps mirror bundles current.
## 1. Prerequisites ## 1. Prerequisites
- **Authority access** – client credentials (`client_id` + secret) authorised for - **Authority access** – client credentials (`client_id` + secret) authorised for
`concelier.mirror.read` and `excititor.mirror.read` scopes. Secrets live outside git. `concelier.mirror.read` and `excititor.mirror.read` scopes. Secrets live outside git.
- **Signed TLS certificates** – wildcard or per-domain (`mirror-primary`, `mirror-community`). - **Signed TLS certificates** – wildcard or per-domain (`mirror-primary`, `mirror-community`).
Store them under `deploy/compose/mirror-gateway/tls/` or in Kubernetes secrets. Store them under `deploy/compose/mirror-gateway/tls/` or in Kubernetes secrets.
- **Mirror gateway credentials** – Basic Auth htpasswd files per domain. Generate with - **Mirror gateway credentials** – Basic Auth htpasswd files per domain. Generate with
`htpasswd -B`. Operators distribute credentials to downstream consumers. `htpasswd -B`. Operators distribute credentials to downstream consumers.
- **Export artifact source** – read access to the canonical S3 buckets (or rsync share) - **Export artifact source** – read access to the canonical S3 buckets (or rsync share)
that hold `concelier` JSON bundles and `excititor` VEX exports. that hold `concelier` JSON bundles and `excititor` VEX exports.
- **Persistent volumes** – storage for Concelier job metadata and mirror export trees. - **Persistent volumes** – storage for Concelier job metadata and mirror export trees.
For Helm, provision PVCs (`concelier-mirror-jobs`, `concelier-mirror-exports`, For Helm, provision PVCs (`concelier-mirror-jobs`, `concelier-mirror-exports`,
`excititor-mirror-exports`, `mirror-mongo-data`, `mirror-minio-data`) before rollout. `excititor-mirror-exports`, `mirror-mongo-data`, `mirror-minio-data`) before rollout.
### 1.1 Service configuration quick reference ### 1.1 Service configuration quick reference
Concelier.WebService exposes the mirror HTTP endpoints once `CONCELIER__MIRROR__ENABLED=true`. Concelier.WebService exposes the mirror HTTP endpoints once `CONCELIER__MIRROR__ENABLED=true`.
Key knobs: Key knobs:
- `CONCELIER__MIRROR__EXPORTROOT` – root folder containing export snapshots (`<exportId>/mirror/*`). - `CONCELIER__MIRROR__EXPORTROOT` – root folder containing export snapshots (`<exportId>/mirror/*`).
- `CONCELIER__MIRROR__ACTIVEEXPORTID` – optional explicit export id; otherwise the service auto-falls back to the `latest/` symlink or newest directory. - `CONCELIER__MIRROR__ACTIVEEXPORTID` – optional explicit export id; otherwise the service auto-falls back to the `latest/` symlink or newest directory.
- `CONCELIER__MIRROR__REQUIREAUTHENTICATION` – default auth requirement; override per domain with `CONCELIER__MIRROR__DOMAINS__{n}__REQUIREAUTHENTICATION`. - `CONCELIER__MIRROR__REQUIREAUTHENTICATION` – default auth requirement; override per domain with `CONCELIER__MIRROR__DOMAINS__{n}__REQUIREAUTHENTICATION`.
- `CONCELIER__MIRROR__MAXINDEXREQUESTSPERHOUR` – budget for `/concelier/exports/index.json`. Domains inherit this value unless they define `__MAXDOWNLOADREQUESTSPERHOUR`. - `CONCELIER__MIRROR__MAXINDEXREQUESTSPERHOUR` – budget for `/concelier/exports/index.json`. Domains inherit this value unless they define `__MAXDOWNLOADREQUESTSPERHOUR`.
- `CONCELIER__MIRROR__DOMAINS__{n}__ID` – domain identifier matching the exporter manifest; additional keys configure display name and rate budgets. - `CONCELIER__MIRROR__DOMAINS__{n}__ID` – domain identifier matching the exporter manifest; additional keys configure display name and rate budgets.
> The service honours Stella Ops Authority when `CONCELIER__AUTHORITY__ENABLED=true` and `ALLOWANONYMOUSFALLBACK=false`. Use the bypass CIDR list (`CONCELIER__AUTHORITY__BYPASSNETWORKS__*`) for in-cluster ingress gateways that terminate Basic Auth. Unauthorized requests emit `WWW-Authenticate: Bearer` so downstream automation can detect token failures. > The service honours Stella Ops Authority when `CONCELIER__AUTHORITY__ENABLED=true` and `ALLOWANONYMOUSFALLBACK=false`. Use the bypass CIDR list (`CONCELIER__AUTHORITY__BYPASSNETWORKS__*`) for in-cluster ingress gateways that terminate Basic Auth. Unauthorized requests emit `WWW-Authenticate: Bearer` so downstream automation can detect token failures.
Mirror responses carry deterministic cache headers: `/index.json` returns `Cache-Control: public, max-age=60`, while per-domain manifests/bundles include `Cache-Control: public, max-age=300, immutable`. Rate limiting surfaces `Retry-After` when quotas are exceeded. Mirror responses carry deterministic cache headers: `/index.json` returns `Cache-Control: public, max-age=60`, while per-domain manifests/bundles include `Cache-Control: public, max-age=300, immutable`. Rate limiting surfaces `Retry-After` when quotas are exceeded.
### 1.2 Mirror connector configuration ### 1.2 Mirror connector configuration
Downstream Concelier instances ingest published bundles using the `StellaOpsMirrorConnector`. Operators running the connector in air‑gapped or limited connectivity environments can tune the following options (environment prefix `CONCELIER__SOURCES__STELLAOPSMIRROR__`): Downstream Concelier instances ingest published bundles using the `StellaOpsMirrorConnector`. Operators running the connector in air‑gapped or limited connectivity environments can tune the following options (environment prefix `CONCELIER__SOURCES__STELLAOPSMIRROR__`):
- `BASEADDRESS` – absolute mirror root (e.g., `https://mirror-primary.stella-ops.org`). - `BASEADDRESS` – absolute mirror root (e.g., `https://mirror-primary.stella-ops.org`).
- `INDEXPATH` – relative path to the mirror index (`/concelier/exports/index.json` by default). - `INDEXPATH` – relative path to the mirror index (`/concelier/exports/index.json` by default).
- `DOMAINID` – mirror domain identifier from the index (`primary`, `community`, etc.). - `DOMAINID` – mirror domain identifier from the index (`primary`, `community`, etc.).
- `HTTPTIMEOUT` – request timeout; raise when mirrors sit behind slow WAN links. - `HTTPTIMEOUT` – request timeout; raise when mirrors sit behind slow WAN links.
- `SIGNATURE__ENABLED` – require detached JWS verification for `bundle.json`. - `SIGNATURE__ENABLED` – require detached JWS verification for `bundle.json`.
- `SIGNATURE__KEYID` / `SIGNATURE__PROVIDER` – expected signing key metadata. - `SIGNATURE__KEYID` / `SIGNATURE__PROVIDER` – expected signing key metadata.
- `SIGNATURE__PUBLICKEYPATH` – PEM fallback used when the mirror key registry is offline. - `SIGNATURE__PUBLICKEYPATH` – PEM fallback used when the mirror key registry is offline.
The connector keeps a per-export fingerprint (bundle digest + generated-at timestamp) and tracks outstanding document IDs. If a scan is interrupted, the next run resumes parse/map work using the stored fingerprint and pending document lists—no network requests are reissued unless the upstream digest changes. The connector keeps a per-export fingerprint (bundle digest + generated-at timestamp) and tracks outstanding document IDs. If a scan is interrupted, the next run resumes parse/map work using the stored fingerprint and pending document lists—no network requests are reissued unless the upstream digest changes.
## 2. Secret & certificate layout ## 2. Secret & certificate layout
### Docker Compose (`deploy/compose/docker-compose.mirror.yaml`) ### Docker Compose (`deploy/compose/docker-compose.mirror.yaml`)
- `deploy/compose/env/mirror.env.example` – copy to `.env` and adjust quotas or domain IDs. - `deploy/compose/env/mirror.env.example` – copy to `.env` and adjust quotas or domain IDs.
- `deploy/compose/mirror-secrets/` – mount read-only into `/run/secrets`. Place: - `deploy/compose/mirror-secrets/` – mount read-only into `/run/secrets`. Place:
- `concelier-authority-client` – Authority client secret. - `concelier-authority-client` – Authority client secret.
- `excititor-authority-client` (optional) – reserve for future authn. - `excititor-authority-client` (optional) – reserve for future authn.
- `deploy/compose/mirror-gateway/tls/` – PEM-encoded cert/key pairs: - `deploy/compose/mirror-gateway/tls/` – PEM-encoded cert/key pairs:
- `mirror-primary.crt`, `mirror-primary.key` - `mirror-primary.crt`, `mirror-primary.key`
- `mirror-community.crt`, `mirror-community.key` - `mirror-community.crt`, `mirror-community.key`
- `deploy/compose/mirror-gateway/secrets/` – htpasswd files: - `deploy/compose/mirror-gateway/secrets/` – htpasswd files:
- `mirror-primary.htpasswd` - `mirror-primary.htpasswd`
- `mirror-community.htpasswd` - `mirror-community.htpasswd`
### Helm (`deploy/helm/stellaops/values-mirror.yaml`) ### Helm (`deploy/helm/stellaops/values-mirror.yaml`)
Create secrets in the target namespace: Create secrets in the target namespace:
```bash ```bash
kubectl create secret generic concelier-mirror-auth \ kubectl create secret generic concelier-mirror-auth \
--from-file=concelier-authority-client=concelier-authority-client --from-file=concelier-authority-client=concelier-authority-client
kubectl create secret generic excititor-mirror-auth \ kubectl create secret generic excititor-mirror-auth \
--from-file=excititor-authority-client=excititor-authority-client --from-file=excititor-authority-client=excititor-authority-client
kubectl create secret tls mirror-gateway-tls \ kubectl create secret tls mirror-gateway-tls \
--cert=mirror-primary.crt --key=mirror-primary.key --cert=mirror-primary.crt --key=mirror-primary.key
kubectl create secret generic mirror-gateway-htpasswd \ kubectl create secret generic mirror-gateway-htpasswd \
--from-file=mirror-primary.htpasswd --from-file=mirror-community.htpasswd --from-file=mirror-primary.htpasswd --from-file=mirror-community.htpasswd
``` ```
> Keep Basic Auth lists short-lived (rotate quarterly) and document credential recipients. > Keep Basic Auth lists short-lived (rotate quarterly) and document credential recipients.
## 3. Deployment ## 3. Deployment
### 3.1 Docker Compose (edge mirrors, lab validation) ### 3.1 Docker Compose (edge mirrors, lab validation)
1. `cp deploy/compose/env/mirror.env.example deploy/compose/env/mirror.env` 1. `cp deploy/compose/env/mirror.env.example deploy/compose/env/mirror.env`
2. Populate secrets/tls directories as described above. 2. Populate secrets/tls directories as described above.
3. Sync mirror bundles (see §4) into `deploy/compose/mirror-data/…` and ensure they are mounted 3. Sync mirror bundles (see §4) into `deploy/compose/mirror-data/…` and ensure they are mounted
on the host path backing the `concelier-exports` and `excititor-exports` volumes. on the host path backing the `concelier-exports` and `excititor-exports` volumes.
4. Run the profile validator: `deploy/tools/validate-profiles.sh`. 4. Run the profile validator: `deploy/tools/validate-profiles.sh`.
5. Launch: `docker compose --env-file env/mirror.env -f docker-compose.mirror.yaml up -d`. 5. Launch: `docker compose --env-file env/mirror.env -f docker-compose.mirror.yaml up -d`.
### 3.2 Helm (production mirrors) ### 3.2 Helm (production mirrors)
1. Provision PVCs sized for mirror bundles (baseline: 20 GiB per domain). 1. Provision PVCs sized for mirror bundles (baseline: 20 GiB per domain).
2. Create secrets/tls config maps (§2). 2. Create secrets/tls config maps (§2).
3. `helm upgrade --install mirror deploy/helm/stellaops -f deploy/helm/stellaops/values-mirror.yaml`. 3. `helm upgrade --install mirror deploy/helm/stellaops -f deploy/helm/stellaops/values-mirror.yaml`.
4. Annotate the `stellaops-mirror-gateway` service with ingress/LoadBalancer metadata required by 4. Annotate the `stellaops-mirror-gateway` service with ingress/LoadBalancer metadata required by
your CDN (e.g., AWS load balancer scheme internal + NLB idle timeout). your CDN (e.g., AWS load balancer scheme internal + NLB idle timeout).
## 4. Artifact sync workflow ## 4. Artifact sync workflow
Mirrors never generate exports—they ingest signed bundles produced by the Concelier and Excititor Mirrors never generate exports—they ingest signed bundles produced by the Concelier and Excititor
export jobs. Recommended sync pattern: export jobs. Recommended sync pattern:
### 4.1 Compose host (systemd timer) ### 4.1 Compose host (systemd timer)
`/usr/local/bin/mirror-sync.sh`: `/usr/local/bin/mirror-sync.sh`:
```bash ```bash
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
export AWS_ACCESS_KEY_ID=… export AWS_ACCESS_KEY_ID=…
export AWS_SECRET_ACCESS_KEY=… export AWS_SECRET_ACCESS_KEY=…
aws s3 sync s3://mirror-stellaops/concelier/latest \ aws s3 sync s3://mirror-stellaops/concelier/latest \
/opt/stellaops/mirror-data/concelier --delete --size-only /opt/stellaops/mirror-data/concelier --delete --size-only
aws s3 sync s3://mirror-stellaops/excititor/latest \ aws s3 sync s3://mirror-stellaops/excititor/latest \
/opt/stellaops/mirror-data/excititor --delete --size-only /opt/stellaops/mirror-data/excititor --delete --size-only
``` ```
Schedule with a systemd timer every 5 minutes. The Compose volumes mount `/opt/stellaops/mirror-data/*` Schedule with a systemd timer every 5 minutes. The Compose volumes mount `/opt/stellaops/mirror-data/*`
into the containers read-only, matching `CONCELIER__MIRROR__EXPORTROOT=/exports/json` and into the containers read-only, matching `CONCELIER__MIRROR__EXPORTROOT=/exports/json` and
`EXCITITOR__ARTIFACTS__FILESYSTEM__ROOT=/exports`. `EXCITITOR__ARTIFACTS__FILESYSTEM__ROOT=/exports`.
### 4.2 Kubernetes (CronJob) ### 4.2 Kubernetes (CronJob)
Create a CronJob running the AWS CLI (or rclone) in the same namespace, writing into the PVCs: Create a CronJob running the AWS CLI (or rclone) in the same namespace, writing into the PVCs:
```yaml ```yaml
apiVersion: batch/v1 apiVersion: batch/v1
kind: CronJob kind: CronJob
metadata: metadata:
name: mirror-sync name: mirror-sync
spec: spec:
schedule: "*/5 * * * *" schedule: "*/5 * * * *"
jobTemplate: jobTemplate:
spec: spec:
template: template:
spec: spec:
containers: containers:
- name: sync - name: sync
image: public.ecr.aws/aws-cli/aws-cli@sha256:5df5f52c29f5e3ba46d0ad9e0e3afc98701c4a0f879400b4c5f80d943b5fadea image: public.ecr.aws/aws-cli/aws-cli@sha256:5df5f52c29f5e3ba46d0ad9e0e3afc98701c4a0f879400b4c5f80d943b5fadea
command: command:
- /bin/sh - /bin/sh
- -c - -c
- > - >
aws s3 sync s3://mirror-stellaops/concelier/latest /exports/concelier --delete --size-only && aws s3 sync s3://mirror-stellaops/concelier/latest /exports/concelier --delete --size-only &&
aws s3 sync s3://mirror-stellaops/excititor/latest /exports/excititor --delete --size-only aws s3 sync s3://mirror-stellaops/excititor/latest /exports/excititor --delete --size-only
volumeMounts: volumeMounts:
- name: concelier-exports - name: concelier-exports
mountPath: /exports/concelier mountPath: /exports/concelier
- name: excititor-exports - name: excititor-exports
mountPath: /exports/excititor mountPath: /exports/excititor
envFrom: envFrom:
- secretRef: - secretRef:
name: mirror-sync-aws name: mirror-sync-aws
restartPolicy: OnFailure restartPolicy: OnFailure
volumes: volumes:
- name: concelier-exports - name: concelier-exports
persistentVolumeClaim: persistentVolumeClaim:
claimName: concelier-mirror-exports claimName: concelier-mirror-exports
- name: excititor-exports - name: excititor-exports
persistentVolumeClaim: persistentVolumeClaim:
claimName: excititor-mirror-exports claimName: excititor-mirror-exports
``` ```
## 5. CDN integration ## 5. CDN integration
1. Point the CDN origin at the mirror gateway (Compose host or Kubernetes LoadBalancer). 1. Point the CDN origin at the mirror gateway (Compose host or Kubernetes LoadBalancer).
2. Honour the response headers emitted by the gateway and Concelier/Excititor: 2. Honour the response headers emitted by the gateway and Concelier/Excititor:
`Cache-Control: public, max-age=300, immutable` for mirror payloads. `Cache-Control: public, max-age=300, immutable` for mirror payloads.
3. Configure origin shields in the CDN to prevent cache stampedes. Recommended TTLs: 3. Configure origin shields in the CDN to prevent cache stampedes. Recommended TTLs:
- Index (`/concelier/exports/index.json`, `/excititor/mirror/*/index`) → 60 s. - Index (`/concelier/exports/index.json`, `/excititor/mirror/*/index`) → 60 s.
- Bundle/manifest payloads → 300 s. - Bundle/manifest payloads → 300 s.
4. Forward the `Authorization` header—Basic Auth terminates at the gateway. 4. Forward the `Authorization` header—Basic Auth terminates at the gateway.
5. Enforce per-domain rate limits at the CDN (matching gateway budgets) and enable logging 5. Enforce per-domain rate limits at the CDN (matching gateway budgets) and enable logging
to SIEM for anomaly detection. to SIEM for anomaly detection.
## 6. Smoke tests ## 6. Smoke tests
After each deployment or sync cycle (temporarily set low budgets if you need to observe 429 responses): After each deployment or sync cycle (temporarily set low budgets if you need to observe 429 responses):
```bash ```bash
# Index with Basic Auth # Index with Basic Auth
curl -u $PRIMARY_CREDS https://mirror-primary.stella-ops.org/concelier/exports/index.json | jq 'keys' curl -u $PRIMARY_CREDS https://mirror-primary.stella-ops.org/concelier/exports/index.json | jq 'keys'
# Mirror manifest signature and cache headers # Mirror manifest signature and cache headers
curl -u $PRIMARY_CREDS -I https://mirror-primary.stella-ops.org/concelier/exports/mirror/primary/manifest.json \ curl -u $PRIMARY_CREDS -I https://mirror-primary.stella-ops.org/concelier/exports/mirror/primary/manifest.json \
| tee /tmp/manifest-headers.txt | tee /tmp/manifest-headers.txt
grep -E '^Cache-Control: ' /tmp/manifest-headers.txt # expect public, max-age=300, immutable grep -E '^Cache-Control: ' /tmp/manifest-headers.txt # expect public, max-age=300, immutable
# Excititor consensus bundle metadata # Excititor consensus bundle metadata
curl -u $COMMUNITY_CREDS https://mirror-community.stella-ops.org/excititor/mirror/community/index \ curl -u $COMMUNITY_CREDS https://mirror-community.stella-ops.org/excititor/mirror/community/index \
| jq '.exports[].exportKey' | jq '.exports[].exportKey'
# Signed bundle + detached JWS (spot check digests) # Signed bundle + detached JWS (spot check digests)
curl -u $PRIMARY_CREDS https://mirror-primary.stella-ops.org/concelier/exports/mirror/primary/bundle.json.jws \ curl -u $PRIMARY_CREDS https://mirror-primary.stella-ops.org/concelier/exports/mirror/primary/bundle.json.jws \
-o bundle.json.jws -o bundle.json.jws
cosign verify-blob --signature bundle.json.jws --key mirror-key.pub bundle.json cosign verify-blob --signature bundle.json.jws --key mirror-key.pub bundle.json
# Service-level auth check (inside cluster – no gateway credentials) # Service-level auth check (inside cluster – no gateway credentials)
kubectl exec deploy/stellaops-concelier -- curl -si http://localhost:8443/concelier/exports/mirror/primary/manifest.json \ kubectl exec deploy/stellaops-concelier -- curl -si http://localhost:8443/concelier/exports/mirror/primary/manifest.json \
| head -n 5 # expect HTTP/1.1 401 with WWW-Authenticate: Bearer | head -n 5 # expect HTTP/1.1 401 with WWW-Authenticate: Bearer
# Rate limit smoke (repeat quickly; second call should return 429 + Retry-After) # Rate limit smoke (repeat quickly; second call should return 429 + Retry-After)
for i in 1 2; do for i in 1 2; do
curl -s -o /dev/null -D - https://mirror-primary.stella-ops.org/concelier/exports/index.json \ curl -s -o /dev/null -D - https://mirror-primary.stella-ops.org/concelier/exports/index.json \
-u $PRIMARY_CREDS | grep -E '^(HTTP/|Retry-After:)' -u $PRIMARY_CREDS | grep -E '^(HTTP/|Retry-After:)'
sleep 1 sleep 1
done done
``` ```
Watch the gateway metrics (`nginx_vts` or access logs) for cache hits. In Kubernetes, `kubectl logs deploy/stellaops-mirror-gateway` Watch the gateway metrics (`nginx_vts` or access logs) for cache hits. In Kubernetes, `kubectl logs deploy/stellaops-mirror-gateway`
should show `X-Cache-Status: HIT/MISS`. should show `X-Cache-Status: HIT/MISS`.
## 7. Maintenance & rotation ## 7. Maintenance & rotation
- **Bundle freshness** – alert if sync job lag exceeds 15 minutes or if `concelier` logs - **Bundle freshness** – alert if sync job lag exceeds 15 minutes or if `concelier` logs
`Mirror export root is not configured`. `Mirror export root is not configured`.
- **Secret rotation** – change Authority client secrets and Basic Auth credentials quarterly. - **Secret rotation** – change Authority client secrets and Basic Auth credentials quarterly.
Update the mounted secrets and restart deployments (`docker compose restart concelier` or Update the mounted secrets and restart deployments (`docker compose restart concelier` or
`kubectl rollout restart deploy/stellaops-concelier`). `kubectl rollout restart deploy/stellaops-concelier`).
- **TLS renewal** – reissue certificates, place new files, and reload gateway (`docker compose exec mirror-gateway nginx -s reload`). - **TLS renewal** – reissue certificates, place new files, and reload gateway (`docker compose exec mirror-gateway nginx -s reload`).
- **Quota tuning** – adjust per-domain `MAXDOWNLOADREQUESTSPERHOUR` in `.env` or values file. - **Quota tuning** – adjust per-domain `MAXDOWNLOADREQUESTSPERHOUR` in `.env` or values file.
Align CDN rate limits and inform downstreams. Align CDN rate limits and inform downstreams.
## 8. References ## 8. References
- Deployment profiles: `deploy/compose/docker-compose.mirror.yaml`, - Deployment profiles: `deploy/compose/docker-compose.mirror.yaml`,
`deploy/helm/stellaops/values-mirror.yaml` `deploy/helm/stellaops/values-mirror.yaml`
- Mirror architecture dossiers: `docs/modules/concelier/architecture.md`, - Mirror architecture dossiers: `docs/modules/concelier/architecture.md`,
`docs/modules/excititor/mirrors.md` `docs/modules/excititor/mirrors.md`
- Export bundling: `docs/modules/devops/architecture.md` §3, `docs/modules/excititor/architecture.md` §7 - Export bundling: `docs/modules/devops/architecture.md` §3, `docs/modules/excititor/architecture.md` §7

42
docs/modules/devops/AGENTS.md
View File

@@ -1,22 +1,22 @@
# DevOps agent guide # DevOps agent guide
## Mission ## Mission
The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments. The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

82
docs/modules/devops/README.md
View File

@@ -1,41 +1,41 @@
# StellaOps DevOps # StellaOps DevOps
The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments. The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments.
## Responsibilities ## Responsibilities
- Maintain CI pipelines, signing workflows, and release packaging steps. - Maintain CI pipelines, signing workflows, and release packaging steps.
- Operate shared runbooks for launch readiness, upgrades, and NuGet previews. - Operate shared runbooks for launch readiness, upgrades, and NuGet previews.
- Provide offline kit assembly instructions and tooling integration. - Provide offline kit assembly instructions and tooling integration.
- Wrap observability/telemetry bootstrap flows for platform teams. - Wrap observability/telemetry bootstrap flows for platform teams.
## Key components ## Key components
- Runbooks under ./runbooks/ (launch, deployment, nuget). - Runbooks under ./runbooks/ (launch, deployment, nuget).
- Migration guidance under ./migrations/. - Migration guidance under ./migrations/.
- Architecture overview bridging CI/CD & infrastructure concerns. - Architecture overview bridging CI/CD & infrastructure concerns.
## Integrations & dependencies ## Integrations & dependencies
- Ops pipelines (Gitea, GitHub Actions) and artifact registries. - Ops pipelines (Gitea, GitHub Actions) and artifact registries.
- Authority/Signer for supply chain signing. - Authority/Signer for supply chain signing.
- Telemetry stack bootstrap scripts. - Telemetry stack bootstrap scripts.
## Operational notes ## Operational notes
- Offline bundle packaging guidance in docs/modules/export-center/operations/runbook.md. - Offline bundle packaging guidance in docs/modules/export-center/operations/runbook.md.
- Dashboards for launch cutover rehearsals. - Dashboards for launch cutover rehearsals.
- Coordination with Security for enforced guardrails. - Coordination with Security for enforced guardrails.
## Related resources ## Related resources
- ./runbooks/launch-readiness.md - ./runbooks/launch-readiness.md
- ./runbooks/launch-cutover.md - ./runbooks/launch-cutover.md
- ./runbooks/deployment-upgrade.md - ./runbooks/deployment-upgrade.md
- ./runbooks/nuget-preview-bootstrap.md - ./runbooks/nuget-preview-bootstrap.md
- ./migrations/semver-style.md - ./migrations/semver-style.md
## Backlog references ## Backlog references
- DEVOPS-LAUNCH-18-001 / 18-900 runbooks in ../../TASKS.md. - DEVOPS-LAUNCH-18-001 / 18-900 runbooks in ../../TASKS.md.
- Telemetry bootstrap automation tracked in `ops/devops/TASKS.md`. - Telemetry bootstrap automation tracked in `ops/devops/TASKS.md`.
## Epic alignment ## Epic alignment
- **Epic 1 – AOC enforcement:** bake AOC verifier steps, CI guards, and schema validation into pipelines. - **Epic 1 – AOC enforcement:** bake AOC verifier steps, CI guards, and schema validation into pipelines.
- **Epic 9 – Orchestrator Dashboard:** support operational dashboards, job recovery runbooks, and rate-limit governance. - **Epic 9 – Orchestrator Dashboard:** support operational dashboards, job recovery runbooks, and rate-limit governance.
- **Epic 10 – Export Center:** manage signing workflows, Offline Kit packaging, and release promotion for exports. - **Epic 10 – Export Center:** manage signing workflows, Offline Kit packaging, and release promotion for exports.
- **Epic 15 – Observability & Forensics:** coordinate telemetry deployment, evidence retention, and forensic automation. - **Epic 15 – Observability & Forensics:** coordinate telemetry deployment, evidence retention, and forensic automation.

18
docs/modules/devops/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — DevOps # Task board — DevOps
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| DEVOPS-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | DEVOPS-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| DEVOPS-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | DEVOPS-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| DEVOPS-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | DEVOPS-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

976
docs/modules/devops/architecture.md
View File

@@ -1,488 +1,488 @@
# component_architecture_devops.md — **Stella Ops Release & Operations** (2025Q4) # component_architecture_devops.md — **Stella Ops Release & Operations** (2025Q4)
> Draws from the AOC guardrails, Orchestrator, Export Center, and Observability module plans to describe how Stella Ops is built, signed, distributed, and operated. > Draws from the AOC guardrails, Orchestrator, Export Center, and Observability module plans to describe how Stella Ops is built, signed, distributed, and operated.
> **Scope.** Implementation‑ready blueprint for **how Stella Ops is built, versioned, signed, distributed, upgraded, licensed (PoE)**, and operated in customer environments (online and air‑gapped). Covers reproducible builds, supply‑chain attestations, registries, offline kits, migration/rollback, artifact lifecycle (RustFS default + Mongo, S3 fallback), monitoring SLOs, and customer activation. > **Scope.** Implementation‑ready blueprint for **how Stella Ops is built, versioned, signed, distributed, upgraded, licensed (PoE)**, and operated in customer environments (online and air‑gapped). Covers reproducible builds, supply‑chain attestations, registries, offline kits, migration/rollback, artifact lifecycle (RustFS default + Mongo, S3 fallback), monitoring SLOs, and customer activation.
--- ---
## 0) Product vision (operations lens) ## 0) Product vision (operations lens)
Stella Ops must be **trustable at a glance** and **boringly operable**: Stella Ops must be **trustable at a glance** and **boringly operable**:
* Every release ships with **first‑party SBOMs, provenance, and signatures**; services verify **each other’s** integrity at runtime. * Every release ships with **first‑party SBOMs, provenance, and signatures**; services verify **each other’s** integrity at runtime.
* Customers can deploy by **digest** and stay aligned with **LTS/stable/edge** channels. * Customers can deploy by **digest** and stay aligned with **LTS/stable/edge** channels.
* Paid customers receive **attestation authority** (Signer accepts their PoE) while the core platform remains **free to run**. * Paid customers receive **attestation authority** (Signer accepts their PoE) while the core platform remains **free to run**.
* Air‑gapped customers receive **offline kits** with verifiable digests and deterministic import. * Air‑gapped customers receive **offline kits** with verifiable digests and deterministic import.
* Artifacts expire predictably; operators know what’s kept, for how long, and why. * Artifacts expire predictably; operators know what’s kept, for how long, and why.
--- ---
## 1) Release trains & versioning ## 1) Release trains & versioning
### 1.1 Channels ### 1.1 Channels
* **LTS** (12‑month support window): quarterly cadence (Q1/Q2/Q3/Q4). * **LTS** (12‑month support window): quarterly cadence (Q1/Q2/Q3/Q4).
* **Stable** (default): monthly rollup (bug fixes + compatible features). * **Stable** (default): monthly rollup (bug fixes + compatible features).
* **Edge**: weekly; for early adopters, no guarantees. * **Edge**: weekly; for early adopters, no guarantees.
### 1.2 Version strings ### 1.2 Version strings
Semantic core + calendar tag: Semantic core + calendar tag:
``` ```
<MAJOR>.<MINOR>.<PATCH> (<YYYY>.<MM>) e.g., 2.4.1 (2027.06) <MAJOR>.<MINOR>.<PATCH> (<YYYY>.<MM>) e.g., 2.4.1 (2027.06)
``` ```
* **MAJOR**: breaking API/DB changes (rare). * **MAJOR**: breaking API/DB changes (rare).
* **MINOR**: new features, compatible schema migrations (expand/contract pattern). * **MINOR**: new features, compatible schema migrations (expand/contract pattern).
* **PATCH**: bug fixes, perf and security updates. * **PATCH**: bug fixes, perf and security updates.
* **Calendar tag** exposes **release year** used by Signer for **PoE window checks**. * **Calendar tag** exposes **release year** used by Signer for **PoE window checks**.
### 1.3 Component alignment ### 1.3 Component alignment
A release is a **bundle** of image digests + charts + manifests. All services in a bundle are **wire‑compatible**. Mixed minor versions are allowed within a bounded skew: A release is a **bundle** of image digests + charts + manifests. All services in a bundle are **wire‑compatible**. Mixed minor versions are allowed within a bounded skew:
* **Web UI ↔ backend**: `±1 minor`. * **Web UI ↔ backend**: `±1 minor`.
* **Scanner ↔ Policy/Excititor/Concelier**: `±1 minor`. * **Scanner ↔ Policy/Excititor/Concelier**: `±1 minor`.
* **Authority/Signer/Attestor triangle**: **must** be same minor (crypto and DPoP/mTLS binding rules). * **Authority/Signer/Attestor triangle**: **must** be same minor (crypto and DPoP/mTLS binding rules).
At startup, services **self‑advertise** their semver & channel; the UI surfaces **mismatch warnings**. At startup, services **self‑advertise** their semver & channel; the UI surfaces **mismatch warnings**.
--- ---
## 2) Supply‑chain pipeline (how a release is built) ## 2) Supply‑chain pipeline (how a release is built)
### 2.1 Deterministic builds ### 2.1 Deterministic builds
* **Builders**: isolated **BuildKit** workers with pinned base images (digest only). * **Builders**: isolated **BuildKit** workers with pinned base images (digest only).
* **Pinning**: lock files or `go.mod`, `package-lock.json`, `global.json`, `Directory.Packages.props` are **frozen** at tag. * **Pinning**: lock files or `go.mod`, `package-lock.json`, `global.json`, `Directory.Packages.props` are **frozen** at tag.
* **Reproducibility**: timestamps normalized; source date epoch; deterministic zips/tars. * **Reproducibility**: timestamps normalized; source date epoch; deterministic zips/tars.
* **Multi‑arch**: linux/amd64 + linux/arm64 (Windows images track M2 roadmap). * **Multi‑arch**: linux/amd64 + linux/arm64 (Windows images track M2 roadmap).
### 2.2 First‑party SBOMs & provenance ### 2.2 First‑party SBOMs & provenance
* Each image gets **CycloneDX (JSON+Protobuf) SBOM** and **SLSA‑style provenance** attached as **OCI referrers**. * Each image gets **CycloneDX (JSON+Protobuf) SBOM** and **SLSA‑style provenance** attached as **OCI referrers**.
* Scanner’s **Buildx generator** is used to produce SBOMs *during* build; a separate post‑build scan verifies parity (red flag if drift). * Scanner’s **Buildx generator** is used to produce SBOMs *during* build; a separate post‑build scan verifies parity (red flag if drift).
* **Release manifest** (see §6.1) lists all digests and SBOM/attestation refs. * **Release manifest** (see §6.1) lists all digests and SBOM/attestation refs.
### 2.3 Signing & transparency ### 2.3 Signing & transparency
* Images are **cosign‑signed** (keyless) with a Stella Ops release identity; inclusion in a **transparency log** (Rekor) is required. * Images are **cosign‑signed** (keyless) with a Stella Ops release identity; inclusion in a **transparency log** (Rekor) is required.
* SBOM and provenance attestations are **DSSE** and also transparency‑logged. * SBOM and provenance attestations are **DSSE** and also transparency‑logged.
* Release keys (Fulcio roots or public keys) are embedded in **Signer** policy (for **scanner‑release validation** at customer side). * Release keys (Fulcio roots or public keys) are embedded in **Signer** policy (for **scanner‑release validation** at customer side).
### 2.4 Gates & tests ### 2.4 Gates & tests
* **Static**: linters, codegen checks, protobuf API freeze (backward‑compat tests). * **Static**: linters, codegen checks, protobuf API freeze (backward‑compat tests).
* **Unit/integration**: per‑component, plus **end‑to‑end** flows (scan→vex→policy→sign→attest). * **Unit/integration**: per‑component, plus **end‑to‑end** flows (scan→vex→policy→sign→attest).
* **Perf SLOs**: hot paths (SBOM compose, diff, export) measured against budgets. * **Perf SLOs**: hot paths (SBOM compose, diff, export) measured against budgets.
* **Security**: dependency audit vs Concelier export; container hardening tests; minimal caps. * **Security**: dependency audit vs Concelier export; container hardening tests; minimal caps.
* **Analyzer smoke**: restart-time language plug-ins (currently Python) verified via `dotnet run --project src/Tools/LanguageAnalyzerSmoke` to ensure manifest integrity plus cold vs warm determinism (< 30 s / < 5 s budgets); the harness logs deviations from repository goldens for follow-up. * **Analyzer smoke**: restart-time language plug-ins (currently Python) verified via `dotnet run --project src/Tools/LanguageAnalyzerSmoke` to ensure manifest integrity plus cold vs warm determinism (< 30 s / < 5 s budgets); the harness logs deviations from repository goldens for follow-up.
* **Canary cohort**: internal staging + selected customers; one week on **edge** before **stable** tag. * **Canary cohort**: internal staging + selected customers; one week on **edge** before **stable** tag.
### 2.5 Debug-store artefacts ### 2.5 Debug-store artefacts
* Every release exports stripped debug information for ELF binaries discovered in service images. Debug files follow the GNU build-id layout (`debug/.build-id/<aa>/<rest>.debug`) and are generated via `objcopy --only-keep-debug`. * Every release exports stripped debug information for ELF binaries discovered in service images. Debug files follow the GNU build-id layout (`debug/.build-id/<aa>/<rest>.debug`) and are generated via `objcopy --only-keep-debug`.
* `debug/debug-manifest.json` captures build-id → component/image/source mappings with SHA-256 checksums so operators can mirror the directory into debuginfod or offline symbol stores. The manifest (and its `.sha256` companion) ships with every release bundle and Offline Kit. * `debug/debug-manifest.json` captures build-id → component/image/source mappings with SHA-256 checksums so operators can mirror the directory into debuginfod or offline symbol stores. The manifest (and its `.sha256` companion) ships with every release bundle and Offline Kit.
--- ---
## 3) Distribution & activation ## 3) Distribution & activation
### 3.1 Registries ### 3.1 Registries
* **Primary**: `registry.stella-ops.org` (OCI v2, supports Referrers API). * **Primary**: `registry.stella-ops.org` (OCI v2, supports Referrers API).
* **Mirrors**: GHCR (read‑only), regional mirrors for latency. * **Mirrors**: GHCR (read‑only), regional mirrors for latency.
* Operational runbook: see `docs/modules/concelier/operations/mirror.md` for deployment profiles, CDN guidance, and sync automation. * Operational runbook: see `docs/modules/concelier/operations/mirror.md` for deployment profiles, CDN guidance, and sync automation.
* **Pull by digest only** in Kubernetes/Compose manifests. * **Pull by digest only** in Kubernetes/Compose manifests.
**Gating policy**: **Gating policy**:
* **Core images** (Authority, Scanner, Concelier, Excititor, Attestor, UI): public **read**. * **Core images** (Authority, Scanner, Concelier, Excititor, Attestor, UI): public **read**.
* **Enterprise add‑ons** (if any) and **pre‑release**: private repos via the **Registry Token Service** (`src/Registry/StellaOps.Registry.TokenService`) which exchanges Authority-issued OpToks for short-lived Docker registry bearer tokens. * **Enterprise add‑ons** (if any) and **pre‑release**: private repos via the **Registry Token Service** (`src/Registry/StellaOps.Registry.TokenService`) which exchanges Authority-issued OpToks for short-lived Docker registry bearer tokens.
> Monetization lever is **signing** (PoE gate), not image pulls, so the core remains simple to consume. > Monetization lever is **signing** (PoE gate), not image pulls, so the core remains simple to consume.
### 3.2 OAuth2 token service (for private repos) ### 3.2 OAuth2 token service (for private repos)
* Docker Registry’s token flow backed by **Authority**: * Docker Registry’s token flow backed by **Authority**:
1. Client hits registry (`401` with `WWW-Authenticate: Bearer realm=…`). 1. Client hits registry (`401` with `WWW-Authenticate: Bearer realm=…`).
2. Client gets an **access token** from the token service (validated by Authority) with `scope=repository:…:pull`. 2. Client gets an **access token** from the token service (validated by Authority) with `scope=repository:…:pull`.
3. Registry allows pull for the requested repo. 3. Registry allows pull for the requested repo.
* Tokens are **short‑lived** (60–300 s) and **DPoP‑bound**. * Tokens are **short‑lived** (60–300 s) and **DPoP‑bound**.
The token service enforces plan gating via `registry-token.yaml` (see `docs/modules/registry/operations/token-service.md`) and exposes Prometheus metrics (`registry_token_issued_total`, `registry_token_rejected_total`). Revoked licence identifiers halt issuance even when scope requirements are met. The token service enforces plan gating via `registry-token.yaml` (see `docs/modules/registry/operations/token-service.md`) and exposes Prometheus metrics (`registry_token_issued_total`, `registry_token_rejected_total`). Revoked licence identifiers halt issuance even when scope requirements are met.
### 3.3 Offline kits (air‑gapped) ### 3.3 Offline kits (air‑gapped)
* Tarball per release channel: * Tarball per release channel:
``` ```
stellaops-kit-<ver>-<channel>.tar.zst stellaops-kit-<ver>-<channel>.tar.zst
/images/ OCI layout with all first-party images (multi-arch) /images/ OCI layout with all first-party images (multi-arch)
/sboms/ CycloneDX JSON+PB for each image /sboms/ CycloneDX JSON+PB for each image
/attest/ DSSE bundles + Rekor proofs /attest/ DSSE bundles + Rekor proofs
/charts/ Helm charts + values templates /charts/ Helm charts + values templates
/compose/ docker-compose.yml + .env template /compose/ docker-compose.yml + .env template
/plugins/ Concelier/Excititor connectors (restart-time) /plugins/ Concelier/Excititor connectors (restart-time)
/policy/ example policies /policy/ example policies
/manifest/ release.yaml (see §6.1) /manifest/ release.yaml (see §6.1)
``` ```
* Import via CLI `offline kit import`; checks digests and signatures before load. * Import via CLI `offline kit import`; checks digests and signatures before load.
--- ---
## 4) Licensing (PoE) & monetization ## 4) Licensing (PoE) & monetization
**Principle**: **Only paid Stella Ops issues valid signed attestations.** Running the stack is free; signing requires PoE. **Principle**: **Only paid Stella Ops issues valid signed attestations.** Running the stack is free; signing requires PoE.
### 4.1 PoE issuance ### 4.1 PoE issuance
* Customers purchase a plan and obtain a **PoE artifact** from `www.stella-ops.org`: * Customers purchase a plan and obtain a **PoE artifact** from `www.stella-ops.org`:
* **PoE‑JWT** (DPoP/mTLS‑bound) **or** **PoE mTLS client certificate**. * **PoE‑JWT** (DPoP/mTLS‑bound) **or** **PoE mTLS client certificate**.
* Contains: `license_id`, `plan`, `valid_release_year`, `max_version`, `exp`, optional `tenant/customer` IDs. * Contains: `license_id`, `plan`, `valid_release_year`, `max_version`, `exp`, optional `tenant/customer` IDs.
### 4.2 Online enforcement ### 4.2 Online enforcement
* **Signer** calls **Licensing /license/introspect** on every signing request (see signer doc). * **Signer** calls **Licensing /license/introspect** on every signing request (see signer doc).
* If **revoked/expired/out‑of‑window** → deny with machine‑readable reason. * If **revoked/expired/out‑of‑window** → deny with machine‑readable reason.
* All **valid** bundles are DSSE‑signed and **Attestor** logs them; Rekor UUID returned. * All **valid** bundles are DSSE‑signed and **Attestor** logs them; Rekor UUID returned.
* UI badges: “**Verified by Stella Ops**” with link to the public log. * UI badges: “**Verified by Stella Ops**” with link to the public log.
### 4.3 Air‑gapped / offline ### 4.3 Air‑gapped / offline
* Customers obtain a **time‑boxed PoE lease** (signed JSON, 7–30 days). * Customers obtain a **time‑boxed PoE lease** (signed JSON, 7–30 days).
* Signer accepts the lease and emits **provisional** attestations (clearly labeled). * Signer accepts the lease and emits **provisional** attestations (clearly labeled).
* When connectivity returns, a background job **endorses** the provisional entries with the cloud service, updating their status to **verified**. * When connectivity returns, a background job **endorses** the provisional entries with the cloud service, updating their status to **verified**.
* Operators can export a **verification bundle** for auditors even before endorsement (contains DSSE + local Rekor proof + lease snapshot). * Operators can export a **verification bundle** for auditors even before endorsement (contains DSSE + local Rekor proof + lease snapshot).
### 4.4 Stolen/abused PoE ### 4.4 Stolen/abused PoE
* Customers report theft; **Licensing** flags `license_id` as **revoked**. * Customers report theft; **Licensing** flags `license_id` as **revoked**.
* Subsequent Signer requests **deny**; previous attestations remain but can be marked **contested** (UI shows badge, optional re‑sign path upon new PoE). * Subsequent Signer requests **deny**; previous attestations remain but can be marked **contested** (UI shows badge, optional re‑sign path upon new PoE).
--- ---
## 5) Deployment path (customer side) ## 5) Deployment path (customer side)
### 5.1 First install ### 5.1 First install
* **Helm** (Kubernetes) or **Compose** (VMs). Example (K8s): * **Helm** (Kubernetes) or **Compose** (VMs). Example (K8s):
```bash ```bash
helm repo add stellaops https://charts.stella-ops.org helm repo add stellaops https://charts.stella-ops.org
helm install stella stellaops/platform \ helm install stella stellaops/platform \
--version 2.4.0 \ --version 2.4.0 \
--set global.channel=stable \ --set global.channel=stable \
--set authority.issuer=https://authority.stella.local \ --set authority.issuer=https://authority.stella.local \
--set scanner.minio.endpoint=http://minio.stella.local:9000 \ --set scanner.minio.endpoint=http://minio.stella.local:9000 \
--set scanner.mongo.uri=mongodb://mongo/scanner \ --set scanner.mongo.uri=mongodb://mongo/scanner \
--set concelier.mongo.uri=mongodb://mongo/concelier \ --set concelier.mongo.uri=mongodb://mongo/concelier \
--set excititor.mongo.uri=mongodb://mongo/excititor --set excititor.mongo.uri=mongodb://mongo/excititor
``` ```
* Post‑install job registers **Authority clients** (Scanner, Signer, Attestor, UI) and prints **bootstrap** URLs and client credentials (sealed secrets). * Post‑install job registers **Authority clients** (Scanner, Signer, Attestor, UI) and prints **bootstrap** URLs and client credentials (sealed secrets).
* UI banner shows **release bundle** and verification state (cosign OK? Rekor OK?). * UI banner shows **release bundle** and verification state (cosign OK? Rekor OK?).
### 5.2 Updates ### 5.2 Updates
* **Blue/green**: pull new bundle by **digest**; deploy side‑by‑side; cut traffic. * **Blue/green**: pull new bundle by **digest**; deploy side‑by‑side; cut traffic.
* **Rolling**: upgrade stateful components in safe order: * **Rolling**: upgrade stateful components in safe order:
1. Authority (stateless, dual‑key rotation ready) 1. Authority (stateless, dual‑key rotation ready)
2. Signer/Attestor (same minor) 2. Signer/Attestor (same minor)
3. Scanner WebService & Workers 3. Scanner WebService & Workers
4. Concelier, then Excititor (schema migrations are expand/contract) 4. Concelier, then Excititor (schema migrations are expand/contract)
5. UI last 5. UI last
* **DB migrations** are **expand/contract**: * **DB migrations** are **expand/contract**:
* Phase A (release N): **add** new fields/indexes, write old+new. * Phase A (release N): **add** new fields/indexes, write old+new.
* Phase B (N+1): **read** new fields; **drop** old. * Phase B (N+1): **read** new fields; **drop** old.
* Rollback is a matter of redeploying previous images and keeping both schemas valid. * Rollback is a matter of redeploying previous images and keeping both schemas valid.
### 5.3 Rollback ### 5.3 Rollback
* Images referenced by **digest**; keep previous release manifest `K` versions back. * Images referenced by **digest**; keep previous release manifest `K` versions back.
* `helm rollback` or compose `docker compose -f release-K.yml up -d`. * `helm rollback` or compose `docker compose -f release-K.yml up -d`.
* Mongo migrations are additive; **no destructive changes** within a single minor. * Mongo migrations are additive; **no destructive changes** within a single minor.
--- ---
## 6) Release payloads & manifests ## 6) Release payloads & manifests
### 6.1 Release manifest (`release.yaml`) ### 6.1 Release manifest (`release.yaml`)
```yaml ```yaml
release: release:
version: "2.4.1" version: "2.4.1"
channel: "stable" channel: "stable"
date: "2027-06-20T12:00:00Z" date: "2027-06-20T12:00:00Z"
calendar: "2027.06" calendar: "2027.06"
components: components:
- name: scanner-webservice - name: scanner-webservice
image: registry.stella-ops.org/stellaops/scanner-web@sha256:aa..bb image: registry.stella-ops.org/stellaops/scanner-web@sha256:aa..bb
sbom: oci://.../referrers/cdx-json@sha256:11..22 sbom: oci://.../referrers/cdx-json@sha256:11..22
provenance: oci://.../attest/provenance@sha256:33..44 provenance: oci://.../attest/provenance@sha256:33..44
signature: { rekorUUID: "…" } signature: { rekorUUID: "…" }
- name: signer - name: signer
image: registry.stella-ops.org/stellaops/signer@sha256:cc..dd image: registry.stella-ops.org/stellaops/signer@sha256:cc..dd
signature: { rekorUUID: "…" } signature: { rekorUUID: "…" }
charts: charts:
- name: platform - name: platform
version: "2.4.1" version: "2.4.1"
digest: "sha256:ee..ff" digest: "sha256:ee..ff"
compose: compose:
file: "docker-compose.yml" file: "docker-compose.yml"
digest: "sha256:77..88" digest: "sha256:77..88"
checksums: checksums:
sha256: "… digest of this release.yaml …" sha256: "… digest of this release.yaml …"
``` ```
The manifest is **cosign‑signed**; UI/CLI can verify a bundle without talking to registries. The manifest is **cosign‑signed**; UI/CLI can verify a bundle without talking to registries.
> Deployment guardrails – The repository keeps channel-aligned Compose bundles > Deployment guardrails – The repository keeps channel-aligned Compose bundles
> in `deploy/compose/` and Helm overlays in `deploy/helm/stellaops/`. Both sets > in `deploy/compose/` and Helm overlays in `deploy/helm/stellaops/`. Both sets
> pull their digests from `deploy/releases/` and are validated by > pull their digests from `deploy/releases/` and are validated by
> `deploy/tools/validate-profiles.sh` to guarantee lint/dry-run cleanliness. > `deploy/tools/validate-profiles.sh` to guarantee lint/dry-run cleanliness.
### 6.2 Image labels (release metadata) ### 6.2 Image labels (release metadata)
Each image sets OCI labels: Each image sets OCI labels:
``` ```
org.opencontainers.image.version = "2.4.1" org.opencontainers.image.version = "2.4.1"
org.opencontainers.image.revision = "<git sha>" org.opencontainers.image.revision = "<git sha>"
org.opencontainers.image.created = "2027-06-20T12:00:00Z" org.opencontainers.image.created = "2027-06-20T12:00:00Z"
org.stellaops.release.calendar = "2027.06" org.stellaops.release.calendar = "2027.06"
org.stellaops.release.channel = "stable" org.stellaops.release.channel = "stable"
org.stellaops.build.slsaProvenance = "oci://…" org.stellaops.build.slsaProvenance = "oci://…"
``` ```
Signer validates **scanner** image’s cosign identity + calendar tag for **release window** checks. Signer validates **scanner** image’s cosign identity + calendar tag for **release window** checks.
--- ---
## 7) Artifact lifecycle & storage (RustFS/Mongo) ## 7) Artifact lifecycle & storage (RustFS/Mongo)
### 7.1 Buckets & prefixes (RustFS) ### 7.1 Buckets & prefixes (RustFS)
``` ```
rustfs://stellaops/ rustfs://stellaops/
scanner/ scanner/
layers/<sha256>/sbom.cdx.json.zst layers/<sha256>/sbom.cdx.json.zst
images/<imgDigest>/inventory.cdx.pb images/<imgDigest>/inventory.cdx.pb
images/<imgDigest>/usage.cdx.pb images/<imgDigest>/usage.cdx.pb
diffs/<old>_<new>/diff.json.zst diffs/<old>_<new>/diff.json.zst
attest/<artifactSha256>.dsse.json attest/<artifactSha256>.dsse.json
concelier/ concelier/
json/<exportId>/... json/<exportId>/...
trivy/<exportId>/... trivy/<exportId>/...
excititor/ excititor/
exports/<exportId>/... exports/<exportId>/...
attestor/ attestor/
dsse/<bundleSha256>.json dsse/<bundleSha256>.json
proof/<rekorUuid>.json proof/<rekorUuid>.json
``` ```
### 7.2 ILM classes ### 7.2 ILM classes
* **`short`**: working artifacts (diffs, queues) — TTL 7–14 days. * **`short`**: working artifacts (diffs, queues) — TTL 7–14 days.
* **`default`**: SBOMs & indexes — TTL 90–180 days (configurable). * **`default`**: SBOMs & indexes — TTL 90–180 days (configurable).
* **`compliance`**: signed reports & attested exports — retention enforced via RustFS hold or S3 Object Lock (governance/compliance) 1–7 years. * **`compliance`**: signed reports & attested exports — retention enforced via RustFS hold or S3 Object Lock (governance/compliance) 1–7 years.
### 7.3 Artifact Lifecycle Controller (ALC) ### 7.3 Artifact Lifecycle Controller (ALC)
* A background worker (part of Scanner.WebService) enforces **TTL** and **reference counting**: * A background worker (part of Scanner.WebService) enforces **TTL** and **reference counting**:
* Artifacts referenced by **reports** or **tickets** are pinned. * Artifacts referenced by **reports** or **tickets** are pinned.
* ILM actions logged; UI shows per‑class usage & upcoming purges. * ILM actions logged; UI shows per‑class usage & upcoming purges.
> **Migration note.** Follow `docs/modules/scanner/operations/rustfs-migration.md` when transitioning existing > **Migration note.** Follow `docs/modules/scanner/operations/rustfs-migration.md` when transitioning existing
> MinIO buckets to RustFS. The provided migrator is idempotent and safe to rerun per prefix. > MinIO buckets to RustFS. The provided migrator is idempotent and safe to rerun per prefix.
### 7.4 Mongo retention ### 7.4 Mongo retention
* **Scanner**: `runtime.events` use TTL (e.g., 30–90 days); **catalog** permanent. * **Scanner**: `runtime.events` use TTL (e.g., 30–90 days); **catalog** permanent.
* **Concelier/Excititor**: raw docs keep **last N windows**; canonical stores permanent. * **Concelier/Excititor**: raw docs keep **last N windows**; canonical stores permanent.
* **Attestor**: `entries` permanent; `dedupe` TTL 24–48h. * **Attestor**: `entries` permanent; `dedupe` TTL 24–48h.
### 7.5 Mongo server baseline ### 7.5 Mongo server baseline
* **Minimum supported server:** MongoDB **4.2+**. Driver 3.5.0 removes compatibility shims for 4.0; upstream has already announced 4.0 support will be dropped in upcoming C# driver releases. citeturn1open1 * **Minimum supported server:** MongoDB **4.2+**. Driver 3.5.0 removes compatibility shims for 4.0; upstream has already announced 4.0 support will be dropped in upcoming C# driver releases. citeturn1open1
* **Deploy images:** Compose/Helm defaults stay on `mongo:7.x`. For air-gapped installs, refresh Offline Kit bundles so the packaged `mongod` matches ≥4.2. * **Deploy images:** Compose/Helm defaults stay on `mongo:7.x`. For air-gapped installs, refresh Offline Kit bundles so the packaged `mongod` matches ≥4.2.
* **Upgrade guard:** During rollout, verify replica sets reach FCV `4.2` or above before swapping binaries; automation should hard-stop if FCV is <4.2. * **Upgrade guard:** During rollout, verify replica sets reach FCV `4.2` or above before swapping binaries; automation should hard-stop if FCV is <4.2.
--- ---
## 8) Observability & SLOs (operations) ## 8) Observability & SLOs (operations)
* **Uptime SLO**: 99.9% for Signer/Authority/Attestor; 99.5% for Scanner WebService; Excititor/Concelier 99.0%. * **Uptime SLO**: 99.9% for Signer/Authority/Attestor; 99.5% for Scanner WebService; Excititor/Concelier 99.0%.
* **Error budgets**: tracked per month; dashboards show burn rates. * **Error budgets**: tracked per month; dashboards show burn rates.
* **Golden signals**: * **Golden signals**:
* **Latency**: token issuance, sign→attest round‑trip, scan enqueue→emit, export build. * **Latency**: token issuance, sign→attest round‑trip, scan enqueue→emit, export build.
* **Saturation**: queue depth, Mongo write IOPS, RustFS throughput / queue depth (or S3 metrics when in fallback mode). * **Saturation**: queue depth, Mongo write IOPS, RustFS throughput / queue depth (or S3 metrics when in fallback mode).
* **Traffic**: scans/min, attestations/min, webhook admits/min. * **Traffic**: scans/min, attestations/min, webhook admits/min.
* **Errors**: 5xx rates, cosign verification failures, Rekor timeouts. * **Errors**: 5xx rates, cosign verification failures, Rekor timeouts.
Prometheus + OTLP; Grafana dashboards ship in the charts. Prometheus + OTLP; Grafana dashboards ship in the charts.
--- ---
## 9) Security & compliance operations ## 9) Security & compliance operations
* **Key rotation**: * **Key rotation**:
* Authority JWKS: 60‑day cadence, dual‑key overlap. * Authority JWKS: 60‑day cadence, dual‑key overlap.
* Release signing identities: rotate per minor or quarterly. * Release signing identities: rotate per minor or quarterly.
* Sigstore roots mirrored and pinned; alarms on drift. * Sigstore roots mirrored and pinned; alarms on drift.
* **FIPS mode** (Gov build): * **FIPS mode** (Gov build):
* Enforce `ES256` + KMS/HSM; disable Ed25519; MLS ciphers only. * Enforce `ES256` + KMS/HSM; disable Ed25519; MLS ciphers only.
* Local **Rekor v2** and **Fulcio** alternatives; **air‑gapped** CA. * Local **Rekor v2** and **Fulcio** alternatives; **air‑gapped** CA.
* **Vulnerability response**: * **Vulnerability response**:
* Concelier red-flag advisories trigger accelerated **stable** patch rollout; UI/CLI “security patch available” notice. * Concelier red-flag advisories trigger accelerated **stable** patch rollout; UI/CLI “security patch available” notice.
* 2025-10: Pinned `MongoDB.Driver` **3.5.0** and `SharpCompress` **0.41.0** across services (DEVOPS-SEC-10-301) to eliminate NU1902/NU1903 warnings surfaced during scanner cache/worker test runs; repacked the local `Mongo2Go` feed so test fixtures inherit the patched dependencies; future bumps follow the same central override pattern. * 2025-10: Pinned `MongoDB.Driver` **3.5.0** and `SharpCompress` **0.41.0** across services (DEVOPS-SEC-10-301) to eliminate NU1902/NU1903 warnings surfaced during scanner cache/worker test runs; repacked the local `Mongo2Go` feed so test fixtures inherit the patched dependencies; future bumps follow the same central override pattern.
* **Backups/DR**: * **Backups/DR**:
* Mongo nightly snapshots; MinIO versioning + replication (if configured). * Mongo nightly snapshots; MinIO versioning + replication (if configured).
* Restore runbooks tested quarterly with synthetic data. * Restore runbooks tested quarterly with synthetic data.
--- ---
## 10) Customer update flow (how versions are fetched & activated) ## 10) Customer update flow (how versions are fetched & activated)
### 10.1 Online clusters ### 10.1 Online clusters
* **UI** surfaces update banner with **release manifest** diff and risk notes. * **UI** surfaces update banner with **release manifest** diff and risk notes.
* Operator approves → **Controller** pulls new images by digest; health‑checks; moves traffic; deprecates old revision. * Operator approves → **Controller** pulls new images by digest; health‑checks; moves traffic; deprecates old revision.
* Post‑switch, **schema Phase B** migrations (if any) run automatically. * Post‑switch, **schema Phase B** migrations (if any) run automatically.
### 10.2 Air‑gapped clusters ### 10.2 Air‑gapped clusters
* Operator downloads **offline kit** from a mirror → `stellaops offline kit import`. * Operator downloads **offline kit** from a mirror → `stellaops offline kit import`.
* Controller validates bundle checksums and **cosign signatures**; applies charts/compose by digest. * Controller validates bundle checksums and **cosign signatures**; applies charts/compose by digest.
* After install, **verify** page shows green checks: image sigs, SBOMs attached, provenance logged. * After install, **verify** page shows green checks: image sigs, SBOMs attached, provenance logged.
### 10.3 CLI self‑update (optional) ### 10.3 CLI self‑update (optional)
* `stellaops self-update` pulls a **signed release manifest** and verifies the **CLI binary** with cosign before swapping (admin can disable). * `stellaops self-update` pulls a **signed release manifest** and verifies the **CLI binary** with cosign before swapping (admin can disable).
--- ---
## 11) Compatibility & deprecation policy ## 11) Compatibility & deprecation policy
* **APIs** are stable within a **major**; breaking changes imply **MAJOR++** and deprecation period of one minor. * **APIs** are stable within a **major**; breaking changes imply **MAJOR++** and deprecation period of one minor.
* **Storage**: expand/contract; “drop old fields” only after one minor grace. * **Storage**: expand/contract; “drop old fields” only after one minor grace.
* **Config**: feature flags (default off) for risky features (e.g., eBPF). * **Config**: feature flags (default off) for risky features (e.g., eBPF).
--- ---
## 12) Runbooks (selected) ## 12) Runbooks (selected)
### 12.1 Lost PoE ### 12.1 Lost PoE
1. Suspend **automatic attestation** jobs. 1. Suspend **automatic attestation** jobs.
2. Use CLI `stellaops signer status` to confirm `entitlement_denied`. 2. Use CLI `stellaops signer status` to confirm `entitlement_denied`.
3. Obtain new PoE from portal; verify on Signer `/poe/verify`. 3. Obtain new PoE from portal; verify on Signer `/poe/verify`.
4. Re‑enable; optionally **re‑sign** last N reports (UI button → batch). 4. Re‑enable; optionally **re‑sign** last N reports (UI button → batch).
### 12.2 Rekor outage (self‑hosted) ### 12.2 Rekor outage (self‑hosted)
* Attestor returns `202 (pending)` with queued proof fetch. * Attestor returns `202 (pending)` with queued proof fetch.
* Keep DSSE bundles locally; re‑submit on schedule; UI badge shows **Pending**. * Keep DSSE bundles locally; re‑submit on schedule; UI badge shows **Pending**.
* If outage > SLA, you can switch to a **mirror** log in config; Attestor writes to both when restored. * If outage > SLA, you can switch to a **mirror** log in config; Attestor writes to both when restored.
### 12.3 Emergency downgrade ### 12.3 Emergency downgrade
* Identify prior release manifest (UI → Admin → Releases). * Identify prior release manifest (UI → Admin → Releases).
* `helm rollback stella <revision>` (or compose apply previous file). * `helm rollback stella <revision>` (or compose apply previous file).
* Services tolerate skew per §1.3; ensure **Signer/Authority/Attestor** are rolled together. * Services tolerate skew per §1.3; ensure **Signer/Authority/Attestor** are rolled together.
--- ---
## 13) Example: cluster bootstrap (Compose) ## 13) Example: cluster bootstrap (Compose)
```yaml ```yaml
version: "3.9" version: "3.9"
services: services:
authority: authority:
image: registry.stella-ops.org/stellaops/authority@sha256:... image: registry.stella-ops.org/stellaops/authority@sha256:...
env_file: ./env/authority.env env_file: ./env/authority.env
ports: ["8440:8440"] ports: ["8440:8440"]
signer: signer:
image: registry.stella-ops.org/stellaops/signer@sha256:... image: registry.stella-ops.org/stellaops/signer@sha256:...
depends_on: [authority] depends_on: [authority]
environment: environment:
- SIGNER__POE__LICENSING__INTROSPECTURL=https://www.stella-ops.org/api/v1/license/introspect - SIGNER__POE__LICENSING__INTROSPECTURL=https://www.stella-ops.org/api/v1/license/introspect
attestor: attestor:
image: registry.stella-ops.org/stellaops/attestor@sha256:... image: registry.stella-ops.org/stellaops/attestor@sha256:...
depends_on: [signer] depends_on: [signer]
scanner-web: scanner-web:
image: registry.stella-ops.org/stellaops/scanner-web@sha256:... image: registry.stella-ops.org/stellaops/scanner-web@sha256:...
environment: environment:
- SCANNER__S3__ENDPOINT=http://minio:9000 - SCANNER__S3__ENDPOINT=http://minio:9000
scanner-worker: scanner-worker:
image: registry.stella-ops.org/stellaops/scanner-worker@sha256:... image: registry.stella-ops.org/stellaops/scanner-worker@sha256:...
deploy: { replicas: 4 } deploy: { replicas: 4 }
concelier: concelier:
image: registry.stella-ops.org/stellaops/concelier@sha256:... image: registry.stella-ops.org/stellaops/concelier@sha256:...
excititor: excititor:
image: registry.stella-ops.org/stellaops/excititor@sha256:... image: registry.stella-ops.org/stellaops/excititor@sha256:...
web-ui: web-ui:
image: registry.stella-ops.org/stellaops/web-ui@sha256:... image: registry.stella-ops.org/stellaops/web-ui@sha256:...
mongo: mongo:
image: mongo:7 image: mongo:7
minio: minio:
image: minio/minio:RELEASE.2025-07-10T00-00-00Z image: minio/minio:RELEASE.2025-07-10T00-00-00Z
``` ```
--- ---
## 14) Governance & keys (who owns the trust root) ## 14) Governance & keys (who owns the trust root)
* **Release key policy**: only the Release Engineering group can push signed releases; 4‑eyes approval; TUF‑style manifest possible in future. * **Release key policy**: only the Release Engineering group can push signed releases; 4‑eyes approval; TUF‑style manifest possible in future.
* **Signer acceptance policy**: embedded release identities are updated **only** via minor upgrade; emergency CRL supported. * **Signer acceptance policy**: embedded release identities are updated **only** via minor upgrade; emergency CRL supported.
* **Customer keys**: none needed for core use; enterprise add‑ons may require per‑customer registries and keys. * **Customer keys**: none needed for core use; enterprise add‑ons may require per‑customer registries and keys.
--- ---
## 15) Roadmap (Ops) ## 15) Roadmap (Ops)
* **Windows containers GA** (Scanner + Zastava). * **Windows containers GA** (Scanner + Zastava).
* **Key Transparency** for Signer certs. * **Key Transparency** for Signer certs.
* **Delta‑kit** (offline) for incremental updates. * **Delta‑kit** (offline) for incremental updates.
* **Operator CRDs** (K8s) to manage policy and ILM declaratively. * **Operator CRDs** (K8s) to manage policy and ILM declaratively.
* **SBOM **protobuf** as default transport at rest (smaller, faster). * **SBOM **protobuf** as default transport at rest (smaller, faster).
--- ---
### Appendix A — Minimal SLO monitors ### Appendix A — Minimal SLO monitors
* `authority.tokens_issued_total` slope ≈ normal. * `authority.tokens_issued_total` slope ≈ normal.
* `signer.requests_total{result="success"}/minute` > 0 (when scans occur). * `signer.requests_total{result="success"}/minute` > 0 (when scans occur).
* `attestor.submit_latency_seconds{quantile=0.95}` < 0.3. * `attestor.submit_latency_seconds{quantile=0.95}` < 0.3.
* `scanner.scan_latency_seconds{quantile=0.95}` < target per image size. * `scanner.scan_latency_seconds{quantile=0.95}` < target per image size.
* `concelier.export.duration_seconds` stable; `excititor.consensus.conflicts_total` not exploding after policy changes. * `concelier.export.duration_seconds` stable; `excititor.consensus.conflicts_total` not exploding after policy changes.
* RustFS request error rate near zero (or `s3_requests_errors_total` when operating against S3); Mongo `opcounters` hit expected baseline. * RustFS request error rate near zero (or `s3_requests_errors_total` when operating against S3); Mongo `opcounters` hit expected baseline.
### Appendix B — Upgrade safety checklist ### Appendix B — Upgrade safety checklist
* Verify **release manifest** signature. * Verify **release manifest** signature.
* Ensure **Signer/Authority/Attestor** are same minor. * Ensure **Signer/Authority/Attestor** are same minor.
* Verify **DB backups** < 24h old. * Verify **DB backups** < 24h old.
* Confirm **ILM** won’t purge compliance artifacts during upgrade window. * Confirm **ILM** won’t purge compliance artifacts during upgrade window.
* Roll **one component** at a time; watch SLOs; abort on regression. * Roll **one component** at a time; watch SLOs; abort on regression.
--- ---
**End — component_architecture_devops.md** **End — component_architecture_devops.md**

44
docs/modules/devops/implementation_plan.md
View File

@@ -1,22 +1,22 @@
# Implementation plan — DevOps # Implementation plan — DevOps
## Current objectives ## Current objectives
- Maintain deterministic behaviour and offline parity across releases. - Maintain deterministic behaviour and offline parity across releases.
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes. - Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
## Workstreams ## Workstreams
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap. - Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs. - Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
- Validation: extend tests/fixtures to preserve determinism and provenance requirements. - Validation: extend tests/fixtures to preserve determinism and provenance requirements.
## Epic milestones ## Epic milestones
- **Epic 1 – AOC enforcement:** ensure CI/CD guardrails, schema validation, and verifier pipelines are enforced. - **Epic 1 – AOC enforcement:** ensure CI/CD guardrails, schema validation, and verifier pipelines are enforced.
- **Epic 9 – Orchestrator Dashboard:** deliver dashboards, recovery runbooks, and rate-limit governance. - **Epic 9 – Orchestrator Dashboard:** deliver dashboards, recovery runbooks, and rate-limit governance.
- **Epic 10 – Export Center:** manage signing/promotions and Offline Kit bundle publishing. - **Epic 10 – Export Center:** manage signing/promotions and Offline Kit bundle publishing.
- **Epic 15 – Observability & Forensics:** coordinate telemetry deployments, evidence retention, and forensic automation. - **Epic 15 – Observability & Forensics:** coordinate telemetry deployments, evidence retention, and forensic automation.
- Track module runbooks (DEVOPS-LAUNCH-18-001/900) and telemetry automation via ../../TASKS.md and ops/devops/TASKS.md. - Track module runbooks (DEVOPS-LAUNCH-18-001/900) and telemetry automation via ../../TASKS.md and ops/devops/TASKS.md.
## Coordination ## Coordination
- Review ./AGENTS.md before picking up new work. - Review ./AGENTS.md before picking up new work.
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md. - Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
- Update this plan whenever scope, dependencies, or guardrails change. - Update this plan whenever scope, dependencies, or guardrails change.

302
docs/modules/devops/runbooks/deployment-upgrade.md
View File

@@ -1,151 +1,151 @@
# Stella Ops Deployment Upgrade & Rollback Runbook # Stella Ops Deployment Upgrade & Rollback Runbook
_Last updated: 2025-10-26 (Sprint 14 – DEVOPS-OPS-14-003)._ _Last updated: 2025-10-26 (Sprint 14 – DEVOPS-OPS-14-003)._
This runbook describes how to promote a new release across the supported deployment profiles (Helm and Docker Compose), how to roll back safely, and how to keep channels (`edge`, `stable`, `airgap`) aligned. All steps assume you are working from a clean checkout of the release branch/tag. This runbook describes how to promote a new release across the supported deployment profiles (Helm and Docker Compose), how to roll back safely, and how to keep channels (`edge`, `stable`, `airgap`) aligned. All steps assume you are working from a clean checkout of the release branch/tag.
--- ---
## 1. Channel overview ## 1. Channel overview
| Channel | Release manifest | Helm values | Compose profile | | Channel | Release manifest | Helm values | Compose profile |
|---------|------------------|-------------|-----------------| |---------|------------------|-------------|-----------------|
| `edge` | `deploy/releases/2025.10-edge.yaml` | `deploy/helm/stellaops/values-dev.yaml` | `deploy/compose/docker-compose.dev.yaml` | | `edge` | `deploy/releases/2025.10-edge.yaml` | `deploy/helm/stellaops/values-dev.yaml` | `deploy/compose/docker-compose.dev.yaml` |
| `stable` | `deploy/releases/2025.09-stable.yaml` | `deploy/helm/stellaops/values-stage.yaml`, `deploy/helm/stellaops/values-prod.yaml` | `deploy/compose/docker-compose.stage.yaml`, `deploy/compose/docker-compose.prod.yaml` | | `stable` | `deploy/releases/2025.09-stable.yaml` | `deploy/helm/stellaops/values-stage.yaml`, `deploy/helm/stellaops/values-prod.yaml` | `deploy/compose/docker-compose.stage.yaml`, `deploy/compose/docker-compose.prod.yaml` |
| `airgap` | `deploy/releases/2025.09-airgap.yaml` | `deploy/helm/stellaops/values-airgap.yaml` | `deploy/compose/docker-compose.airgap.yaml` | | `airgap` | `deploy/releases/2025.09-airgap.yaml` | `deploy/helm/stellaops/values-airgap.yaml` | `deploy/compose/docker-compose.airgap.yaml` |
Infrastructure components (MongoDB, MinIO, RustFS) are pinned in the release manifests and inherited by the deployment profiles. Supporting dependencies such as `nats` remain on upstream LTS tags; review `deploy/compose/*.yaml` for the authoritative set. Infrastructure components (MongoDB, MinIO, RustFS) are pinned in the release manifests and inherited by the deployment profiles. Supporting dependencies such as `nats` remain on upstream LTS tags; review `deploy/compose/*.yaml` for the authoritative set.
--- ---
## 2. Pre-flight checklist ## 2. Pre-flight checklist
1. **Refresh release manifest** 1. **Refresh release manifest**
Pull the latest manifest for the channel you are promoting (`deploy/releases/<version>-<channel>.yaml`). Pull the latest manifest for the channel you are promoting (`deploy/releases/<version>-<channel>.yaml`).
2. **Align deployment bundles with the manifest** 2. **Align deployment bundles with the manifest**
Run the alignment checker for every profile that should pick up the release. Pass `--ignore-repo nats` to skip auxiliary services. Run the alignment checker for every profile that should pick up the release. Pass `--ignore-repo nats` to skip auxiliary services.
```bash ```bash
./deploy/tools/check-channel-alignment.py \ ./deploy/tools/check-channel-alignment.py \
--release deploy/releases/2025.10-edge.yaml \ --release deploy/releases/2025.10-edge.yaml \
--target deploy/helm/stellaops/values-dev.yaml \ --target deploy/helm/stellaops/values-dev.yaml \
--target deploy/compose/docker-compose.dev.yaml \ --target deploy/compose/docker-compose.dev.yaml \
--ignore-repo nats --ignore-repo nats
``` ```
Repeat for other channels (`stable`, `airgap`), substituting the manifest and target files. Repeat for other channels (`stable`, `airgap`), substituting the manifest and target files.
3. **Lint and template profiles** 3. **Lint and template profiles**
```bash ```bash
./deploy/tools/validate-profiles.sh ./deploy/tools/validate-profiles.sh
``` ```
4. **Smoke the Offline Kit debug store (edge/stable only)** 4. **Smoke the Offline Kit debug store (edge/stable only)**
When the release pipeline has generated `out/release/debug/.build-id/**`, mirror the assets into the Offline Kit staging tree: When the release pipeline has generated `out/release/debug/.build-id/**`, mirror the assets into the Offline Kit staging tree:
```bash ```bash
./ops/offline-kit/mirror_debug_store.py \ ./ops/offline-kit/mirror_debug_store.py \
--release-dir out/release \ --release-dir out/release \
--offline-kit-dir out/offline-kit --offline-kit-dir out/offline-kit
``` ```
Archive the resulting `out/offline-kit/metadata/debug-store.json` alongside the kit bundle. Archive the resulting `out/offline-kit/metadata/debug-store.json` alongside the kit bundle.
5. **Review compatibility matrix** 5. **Review compatibility matrix**
Confirm MongoDB, MinIO, and RustFS versions in the release manifest match platform SLOs. The default targets are `mongo@sha256:c258…`, `minio@sha256:14ce…`, `rustfs:2025.10.0-edge`. Confirm MongoDB, MinIO, and RustFS versions in the release manifest match platform SLOs. The default targets are `mongo@sha256:c258…`, `minio@sha256:14ce…`, `rustfs:2025.10.0-edge`.
6. **Create a rollback bookmark** 6. **Create a rollback bookmark**
Record the current Helm revision (`helm history stellaops -n stellaops`) and compose tag (`git describe --tags`) before applying changes. Record the current Helm revision (`helm history stellaops -n stellaops`) and compose tag (`git describe --tags`) before applying changes.
--- ---
## 3. Helm upgrade procedure (staging → production) ## 3. Helm upgrade procedure (staging → production)
1. Switch to the deployment branch and ensure secrets/config maps are current. 1. Switch to the deployment branch and ensure secrets/config maps are current.
2. Apply the upgrade in the staging cluster: 2. Apply the upgrade in the staging cluster:
```bash ```bash
helm upgrade stellaops deploy/helm/stellaops \ helm upgrade stellaops deploy/helm/stellaops \
-f deploy/helm/stellaops/values-stage.yaml \ -f deploy/helm/stellaops/values-stage.yaml \
--namespace stellaops \ --namespace stellaops \
--atomic \ --atomic \
--timeout 15m --timeout 15m
``` ```
3. Run smoke tests (`scripts/smoke-tests.sh` or environment-specific checks). 3. Run smoke tests (`scripts/smoke-tests.sh` or environment-specific checks).
4. Promote to production using the prod values file and the same command. 4. Promote to production using the prod values file and the same command.
5. Record the new revision number and Git SHA in the change log. 5. Record the new revision number and Git SHA in the change log.
### Rollback (Helm) ### Rollback (Helm)
1. Identify the previous revision: `helm history stellaops -n stellaops`. 1. Identify the previous revision: `helm history stellaops -n stellaops`.
2. Execute: 2. Execute:
```bash ```bash
helm rollback stellaops <revision> \ helm rollback stellaops <revision> \
--namespace stellaops \ --namespace stellaops \
--wait \ --wait \
--timeout 10m --timeout 10m
``` ```
3. Verify `kubectl get pods` returns healthy workloads; rerun smoke tests. 3. Verify `kubectl get pods` returns healthy workloads; rerun smoke tests.
4. Update the incident/operations log with root cause and rollback details. 4. Update the incident/operations log with root cause and rollback details.
--- ---
## 4. Docker Compose upgrade procedure ## 4. Docker Compose upgrade procedure
1. Update environment files (`deploy/compose/env/*.env.example`) with any new settings and sync secrets to hosts. 1. Update environment files (`deploy/compose/env/*.env.example`) with any new settings and sync secrets to hosts.
2. Pull the tagged repository state corresponding to the release (e.g. `git checkout 2025.09.2` for stable). 2. Pull the tagged repository state corresponding to the release (e.g. `git checkout 2025.09.2` for stable).
3. Apply the upgrade: 3. Apply the upgrade:
```bash ```bash
docker compose \ docker compose \
--env-file deploy/compose/env/prod.env \ --env-file deploy/compose/env/prod.env \
-f deploy/compose/docker-compose.prod.yaml \ -f deploy/compose/docker-compose.prod.yaml \
pull pull
docker compose \ docker compose \
--env-file deploy/compose/env/prod.env \ --env-file deploy/compose/env/prod.env \
-f deploy/compose/docker-compose.prod.yaml \ -f deploy/compose/docker-compose.prod.yaml \
up -d up -d
``` ```
4. Tail logs for critical services (`docker compose logs -f authority concelier`). 4. Tail logs for critical services (`docker compose logs -f authority concelier`).
5. Update monitoring dashboards/alerts to confirm normal operation. 5. Update monitoring dashboards/alerts to confirm normal operation.
### Rollback (Compose) ### Rollback (Compose)
1. Check out the previous release tag (e.g. `git checkout 2025.09.1`). 1. Check out the previous release tag (e.g. `git checkout 2025.09.1`).
2. Re-run `docker compose pull` and `docker compose up -d` with that profile. Docker will restore the prior digests. 2. Re-run `docker compose pull` and `docker compose up -d` with that profile. Docker will restore the prior digests.
3. If reverting to a known-good snapshot is required, restore volume backups (see `docs/modules/authority/operations/backup-restore.md` and associated service guides). 3. If reverting to a known-good snapshot is required, restore volume backups (see `docs/modules/authority/operations/backup-restore.md` and associated service guides).
4. Log the rollback in the operations journal. 4. Log the rollback in the operations journal.
--- ---
## 5. Channel promotion workflow ## 5. Channel promotion workflow
1. Author or update the channel manifest under `deploy/releases/`. 1. Author or update the channel manifest under `deploy/releases/`.
2. Mirror the new digests into Helm/Compose values and run the alignment script for each profile. 2. Mirror the new digests into Helm/Compose values and run the alignment script for each profile.
3. Commit the changes with a message that references the release version and channel (e.g. `deploy: promote 2025.10.0-edge`). 3. Commit the changes with a message that references the release version and channel (e.g. `deploy: promote 2025.10.0-edge`).
4. Publish release notes and update `deploy/releases/README.md` (if applicable). 4. Publish release notes and update `deploy/releases/README.md` (if applicable).
5. Tag the repository when promoting stable or airgap builds. 5. Tag the repository when promoting stable or airgap builds.
--- ---
## 6. Upgrade rehearsal & rollback drill log ## 6. Upgrade rehearsal & rollback drill log
Maintain rehearsal notes in `docs/modules/devops/runbooks/launch-cutover.md` or the relevant sprint planning document. After each drill capture: Maintain rehearsal notes in `docs/modules/devops/runbooks/launch-cutover.md` or the relevant sprint planning document. After each drill capture:
- Release version tested - Release version tested
- Date/time - Date/time
- Participants - Participants
- Issues encountered & fixes - Issues encountered & fixes
- Rollback duration (if executed) - Rollback duration (if executed)
Attach the log to the sprint retro or operational wiki. Attach the log to the sprint retro or operational wiki.
| Date (UTC) | Channel | Outcome | Notes | | Date (UTC) | Channel | Outcome | Notes |
|------------|---------|---------|-------| |------------|---------|---------|-------|
| 2025-10-26 | Documentation dry-run | Planned | Runbook refreshed; next live drill scheduled for 2025-11 edge → stable promotion. | 2025-10-26 | Documentation dry-run | Planned | Runbook refreshed; next live drill scheduled for 2025-11 edge → stable promotion.
--- ---
## 7. References ## 7. References
- `deploy/README.md` – structure and validation workflow for deployment bundles. - `deploy/README.md` – structure and validation workflow for deployment bundles.
- `docs/13_RELEASE_ENGINEERING_PLAYBOOK.md` – release automation and signing pipeline. - `docs/13_RELEASE_ENGINEERING_PLAYBOOK.md` – release automation and signing pipeline.
- `docs/modules/devops/architecture.md` – high-level DevOps architecture, SLOs, and compliance requirements. - `docs/modules/devops/architecture.md` – high-level DevOps architecture, SLOs, and compliance requirements.
- `ops/offline-kit/mirror_debug_store.py` – debug-store mirroring helper. - `ops/offline-kit/mirror_debug_store.py` – debug-store mirroring helper.
- `deploy/tools/check-channel-alignment.py` – release vs deployment digest alignment checker. - `deploy/tools/check-channel-alignment.py` – release vs deployment digest alignment checker.

256
docs/modules/devops/runbooks/launch-cutover.md
View File

@@ -1,128 +1,128 @@
# Launch Cutover Runbook - Stella Ops # Launch Cutover Runbook - Stella Ops
_Document owner: DevOps Guild (2025-10-26)_ _Document owner: DevOps Guild (2025-10-26)_
_Scope:_ Full-platform launch from staging to production for release `2025.09.2`. _Scope:_ Full-platform launch from staging to production for release `2025.09.2`.
## 1. Roles and Communication ## 1. Roles and Communication
| Role | Primary | Backup | Contact | | Role | Primary | Backup | Contact |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Cutover lead | DevOps Guild (on-call engineer) | Platform Ops lead | `#launch-bridge` (Mattermost) | | Cutover lead | DevOps Guild (on-call engineer) | Platform Ops lead | `#launch-bridge` (Mattermost) |
| Authority stack | Authority Core guild rep | Security guild rep | `#authority` | | Authority stack | Authority Core guild rep | Security guild rep | `#authority` |
| Scanner / Queue | Scanner WebService guild rep | Runtime guild rep | `#scanner` | | Scanner / Queue | Scanner WebService guild rep | Runtime guild rep | `#scanner` |
| Storage | Mongo/MinIO operators | Backup DB admin | Pager escalation | | Storage | Mongo/MinIO operators | Backup DB admin | Pager escalation |
| Observability | Telemetry guild rep | SRE on-call | `#telemetry` | | Observability | Telemetry guild rep | SRE on-call | `#telemetry` |
| Approvals | Product owner + CTO | DevOps lead | Approval recorded in change ticket | | Approvals | Product owner + CTO | DevOps lead | Approval recorded in change ticket |
Set up a bridge call 30 minutes before start and keep `#launch-bridge` updated every 10 minutes. Set up a bridge call 30 minutes before start and keep `#launch-bridge` updated every 10 minutes.
## 2. Timeline Overview (UTC) ## 2. Timeline Overview (UTC)
| Time | Activity | Owner | | Time | Activity | Owner |
| --- | --- | --- | | --- | --- | --- |
| T-24h | Change ticket approved, prod secrets verified, offline kit build status checked (`DEVOPS-OFFLINE-18-005`). | DevOps lead | | T-24h | Change ticket approved, prod secrets verified, offline kit build status checked (`DEVOPS-OFFLINE-18-005`). | DevOps lead |
| T-12h | Run `deploy/tools/validate-profiles.sh`; capture logs in ticket. | DevOps engineer | | T-12h | Run `deploy/tools/validate-profiles.sh`; capture logs in ticket. | DevOps engineer |
| T-6h | Freeze non-launch deployments; notify guild leads. | Product owner | | T-6h | Freeze non-launch deployments; notify guild leads. | Product owner |
| T-2h | Execute rehearsal in staging (Section 3) using `values-stage.yaml` to verify scripts. | DevOps + module reps | | T-2h | Execute rehearsal in staging (Section 3) using `values-stage.yaml` to verify scripts. | DevOps + module reps |
| T-30m | Final go/no-go with guild leads; confirm monitoring dashboards green. | Cutover lead | | T-30m | Final go/no-go with guild leads; confirm monitoring dashboards green. | Cutover lead |
| T0 | Execute production cutover steps (Section 4). | Cutover team | | T0 | Execute production cutover steps (Section 4). | Cutover team |
| T+45m | Smoke tests complete (Section 5); announce success or trigger rollback. | Cutover lead | | T+45m | Smoke tests complete (Section 5); announce success or trigger rollback. | Cutover lead |
| T+4h | Post-cutover metrics review, notify stakeholders, close ticket. | DevOps + product owner | | T+4h | Post-cutover metrics review, notify stakeholders, close ticket. | DevOps + product owner |
## 3. Rehearsal (Staging) Checklist ## 3. Rehearsal (Staging) Checklist
1. `docker network create stellaops_frontdoor || true` (if not present on staging jump host). 1. `docker network create stellaops_frontdoor || true` (if not present on staging jump host).
2. Run `deploy/tools/validate-profiles.sh` and archive output. 2. Run `deploy/tools/validate-profiles.sh` and archive output.
3. Apply staging secrets (`kubectl apply -f secrets/stage/*.yaml` or `helm secrets upgrade`) ensuring `stellaops-stage` credentials align with `values-stage.yaml`. 3. Apply staging secrets (`kubectl apply -f secrets/stage/*.yaml` or `helm secrets upgrade`) ensuring `stellaops-stage` credentials align with `values-stage.yaml`.
4. Perform `helm upgrade stellaops deploy/helm/stellaops -f deploy/helm/stellaops/values-stage.yaml` in staging cluster. 4. Perform `helm upgrade stellaops deploy/helm/stellaops -f deploy/helm/stellaops/values-stage.yaml` in staging cluster.
5. Verify health endpoints: `curl https://authority.stage.../healthz`, `curl https://scanner.stage.../healthz`. 5. Verify health endpoints: `curl https://authority.stage.../healthz`, `curl https://scanner.stage.../healthz`.
6. Execute smoke CLI: `stellaops-cli scan submit --profile staging --sbom samples/sbom/demo.json` and confirm report status in UI. 6. Execute smoke CLI: `stellaops-cli scan submit --profile staging --sbom samples/sbom/demo.json` and confirm report status in UI.
7. Document total wall time and any deviations in the rehearsal log. 7. Document total wall time and any deviations in the rehearsal log.
Rehearsal must complete without manual interventions before proceeding to production. Rehearsal must complete without manual interventions before proceeding to production.
## 4. Production Cutover Steps ## 4. Production Cutover Steps
### 4.1 Pre-flight ### 4.1 Pre-flight
- Confirm production secrets in the appropriate secret store (`stellaops-prod-core`, `stellaops-prod-mongo`, `stellaops-prod-minio`, `stellaops-prod-notify`) contain the keys referenced in `values-prod.yaml`. - Confirm production secrets in the appropriate secret store (`stellaops-prod-core`, `stellaops-prod-mongo`, `stellaops-prod-minio`, `stellaops-prod-notify`) contain the keys referenced in `values-prod.yaml`.
- Ensure the external reverse proxy network exists: `docker network create stellaops_frontdoor || true` on each compose host. - Ensure the external reverse proxy network exists: `docker network create stellaops_frontdoor || true` on each compose host.
- Back up current configuration and data: - Back up current configuration and data:
- Mongo snapshot: `mongodump --uri "$MONGO_BACKUP_URI" --out /backups/launch-$(date -Iseconds)`. - Mongo snapshot: `mongodump --uri "$MONGO_BACKUP_URI" --out /backups/launch-$(date -Iseconds)`.
- MinIO policy export: `mc mirror --overwrite minio/stellaops minio-backup/stellaops-$(date +%Y%m%d%H%M)`. - MinIO policy export: `mc mirror --overwrite minio/stellaops minio-backup/stellaops-$(date +%Y%m%d%H%M)`.
### 4.2 Apply Updates (Compose) ### 4.2 Apply Updates (Compose)
1. On each compose node, pull updated images for release `2025.09.2`: 1. On each compose node, pull updated images for release `2025.09.2`:
```bash ```bash
docker compose --env-file prod.env -f deploy/compose/docker-compose.prod.yaml pull docker compose --env-file prod.env -f deploy/compose/docker-compose.prod.yaml pull
``` ```
2. Deploy changes: 2. Deploy changes:
```bash ```bash
docker compose --env-file prod.env -f deploy/compose/docker-compose.prod.yaml up -d docker compose --env-file prod.env -f deploy/compose/docker-compose.prod.yaml up -d
``` ```
3. Confirm containers healthy via `docker compose ps` and `docker logs <service> --tail 50`. 3. Confirm containers healthy via `docker compose ps` and `docker logs <service> --tail 50`.
### 4.3 Apply Updates (Helm/Kubernetes) ### 4.3 Apply Updates (Helm/Kubernetes)
If using Kubernetes, perform: If using Kubernetes, perform:
```bash ```bash
helm upgrade stellaops deploy/helm/stellaops -f deploy/helm/stellaops/values-prod.yaml --atomic --timeout 15m helm upgrade stellaops deploy/helm/stellaops -f deploy/helm/stellaops/values-prod.yaml --atomic --timeout 15m
``` ```
Monitor rollout with `kubectl get pods -n stellaops --watch` and `kubectl rollout status deployment/<service>`. Monitor rollout with `kubectl get pods -n stellaops --watch` and `kubectl rollout status deployment/<service>`.
### 4.4 Configuration Validation ### 4.4 Configuration Validation
- Verify Authority issuer metadata: `curl https://authority.prod.../.well-known/openid-configuration`. - Verify Authority issuer metadata: `curl https://authority.prod.../.well-known/openid-configuration`.
- Validate Signer DSSE endpoint: `stellaops-cli signer verify --base-url https://signer.prod... --bundle samples/dsse/demo.json`. - Validate Signer DSSE endpoint: `stellaops-cli signer verify --base-url https://signer.prod... --bundle samples/dsse/demo.json`.
- Check Scanner queue connectivity: `docker exec stellaops-scanner-web dotnet StellaOps.Scanner.WebService.dll health queue` (returns success). - Check Scanner queue connectivity: `docker exec stellaops-scanner-web dotnet StellaOps.Scanner.WebService.dll health queue` (returns success).
- Ensure Notify (legacy) still accessible while Notifier migration pending. - Ensure Notify (legacy) still accessible while Notifier migration pending.
## 5. Smoke Tests ## 5. Smoke Tests
| Test | Command / Action | Expected Result | | Test | Command / Action | Expected Result |
| --- | --- | --- | | --- | --- | --- |
| API health | `curl https://scanner.prod.../healthz` | HTTP 200 with `status":"Healthy"` | | API health | `curl https://scanner.prod.../healthz` | HTTP 200 with `status":"Healthy"` |
| Scan submit | `stellaops-cli scan submit --profile prod --sbom samples/sbom/demo.json` | Scan completes < 5 minutes; report accessible with signed DSSE | | Scan submit | `stellaops-cli scan submit --profile prod --sbom samples/sbom/demo.json` | Scan completes < 5 minutes; report accessible with signed DSSE |
| Runtime event ingest | Post sample event from Zastava observer fixture | `/runtime/events` responds 202 Accepted; record visible in Mongo `runtime_events` | | Runtime event ingest | Post sample event from Zastava observer fixture | `/runtime/events` responds 202 Accepted; record visible in Mongo `runtime_events` |
| Signing | `stellaops-cli signer sign --bundle demo.json` | Returns DSSE with matching SHA256 and signer metadata | | Signing | `stellaops-cli signer sign --bundle demo.json` | Returns DSSE with matching SHA256 and signer metadata |
| Attestor verify | `stellaops-cli attestor verify --uuid <uuid>` | Verification result `ok=true` | | Attestor verify | `stellaops-cli attestor verify --uuid <uuid>` | Verification result `ok=true` |
| Web UI | Manual login, verify dashboards render and latency within budget | UI loads under 2 seconds; policy views consistent | | Web UI | Manual login, verify dashboards render and latency within budget | UI loads under 2 seconds; policy views consistent |
Log results in the change ticket with timestamps and screenshots where applicable. Log results in the change ticket with timestamps and screenshots where applicable.
## 6. Rollback Procedure ## 6. Rollback Procedure
1. Assess failure scope; if systemic, initiate rollback immediately while preserving logs/artifacts. 1. Assess failure scope; if systemic, initiate rollback immediately while preserving logs/artifacts.
2. For Compose: 2. For Compose:
```bash ```bash
docker compose --env-file prod.env -f deploy/compose/docker-compose.prod.yaml down docker compose --env-file prod.env -f deploy/compose/docker-compose.prod.yaml down
docker compose --env-file stage.env -f deploy/compose/docker-compose.stage.yaml up -d docker compose --env-file stage.env -f deploy/compose/docker-compose.stage.yaml up -d
``` ```
3. For Helm: 3. For Helm:
```bash ```bash
helm rollback stellaops <previous-release-number> --namespace stellaops helm rollback stellaops <previous-release-number> --namespace stellaops
``` ```
4. Restore Mongo snapshot if data inconsistency detected: `mongorestore --uri "$MONGO_BACKUP_URI" --drop /backups/launch-<timestamp>`. 4. Restore Mongo snapshot if data inconsistency detected: `mongorestore --uri "$MONGO_BACKUP_URI" --drop /backups/launch-<timestamp>`.
5. Restore MinIO mirror if required: `mc mirror minio-backup/stellaops-<timestamp> minio/stellaops`. 5. Restore MinIO mirror if required: `mc mirror minio-backup/stellaops-<timestamp> minio/stellaops`.
6. Notify stakeholders of rollback and capture root cause notes in incident ticket. 6. Notify stakeholders of rollback and capture root cause notes in incident ticket.
## 7. Post-cutover Actions ## 7. Post-cutover Actions
- Keep heightened monitoring for 4 hours post cutover; track latency, error rates, and queue depth. - Keep heightened monitoring for 4 hours post cutover; track latency, error rates, and queue depth.
- Confirm audit trails: Authority tokens issued, Scanner events recorded, Attestor submissions stored. - Confirm audit trails: Authority tokens issued, Scanner events recorded, Attestor submissions stored.
- Update `docs/modules/devops/runbooks/launch-readiness.md` if any new gaps or follow-ups discovered. - Update `docs/modules/devops/runbooks/launch-readiness.md` if any new gaps or follow-ups discovered.
- Schedule retrospective within 48 hours; include DevOps, module guilds, and product owner. - Schedule retrospective within 48 hours; include DevOps, module guilds, and product owner.
## 8. Approval Matrix ## 8. Approval Matrix
| Step | Required Approvers | Record Location | | Step | Required Approvers | Record Location |
| --- | --- | --- | | --- | --- | --- |
| Production deployment plan | CTO + DevOps lead | Change ticket comment | | Production deployment plan | CTO + DevOps lead | Change ticket comment |
| Cutover start (T0) | DevOps lead + module reps | `#launch-bridge` summary | | Cutover start (T0) | DevOps lead + module reps | `#launch-bridge` summary |
| Post-smoke success | DevOps lead + product owner | Change ticket closure | | Post-smoke success | DevOps lead + product owner | Change ticket closure |
| Rollback (if invoked) | DevOps lead + CTO | Incident ticket | | Rollback (if invoked) | DevOps lead + CTO | Incident ticket |
Retain all approvals and logs for audit. Update this runbook after each execution to record actual timings and lessons learned. Retain all approvals and logs for audit. Update this runbook after each execution to record actual timings and lessons learned.
## 9. Rehearsal Log ## 9. Rehearsal Log
| Date (UTC) | What We Exercised | Outcome | Follow-up | | Date (UTC) | What We Exercised | Outcome | Follow-up |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| 2025-10-26 | Dry-run of compose/Helm validation via `deploy/tools/validate-profiles.sh` (dev/stage/prod/airgap/mirror). Network creation simulated (`docker network create stellaops_frontdoor` planned) and stage CLI submission reviewed. | Validation script succeeded; all profiles templated cleanly. Stage deployment apply deferred because no staging cluster is accessible from the current environment. | Schedule full stage rehearsal once staging cluster credentials are available; reuse this log section to capture timings. | | 2025-10-26 | Dry-run of compose/Helm validation via `deploy/tools/validate-profiles.sh` (dev/stage/prod/airgap/mirror). Network creation simulated (`docker network create stellaops_frontdoor` planned) and stage CLI submission reviewed. | Validation script succeeded; all profiles templated cleanly. Stage deployment apply deferred because no staging cluster is accessible from the current environment. | Schedule full stage rehearsal once staging cluster credentials are available; reuse this log section to capture timings. |

98
docs/modules/devops/runbooks/launch-readiness.md
View File

@@ -1,49 +1,49 @@
# Launch Readiness Record - Stella Ops # Launch Readiness Record - Stella Ops
_Updated: 2025-10-26 (UTC)_ _Updated: 2025-10-26 (UTC)_
This document captures production launch sign-offs, deployment readiness checkpoints, and any open risks that must be tracked before GA cutover. This document captures production launch sign-offs, deployment readiness checkpoints, and any open risks that must be tracked before GA cutover.
## 1. Sign-off Summary ## 1. Sign-off Summary
| Module / Service | Guild / Point of Contact | Evidence (Task or Runbook) | Status | Timestamp (UTC) | Notes | | Module / Service | Guild / Point of Contact | Evidence (Task or Runbook) | Status | Timestamp (UTC) | Notes |
| --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- |
| Authority (Issuer) | Authority Core Guild | `AUTH-AOC-19-001` - scope issuance & configuration complete (DONE 2025-10-26) | READY | 2025-10-26T14:05Z | Tenant scope propagation follow-up (`AUTH-AOC-19-002`) tracked in gaps section. | | Authority (Issuer) | Authority Core Guild | `AUTH-AOC-19-001` - scope issuance & configuration complete (DONE 2025-10-26) | READY | 2025-10-26T14:05Z | Tenant scope propagation follow-up (`AUTH-AOC-19-002`) tracked in gaps section. |
| Signer | Signer Guild | `SIGNER-API-11-101` / `SIGNER-REF-11-102` / `SIGNER-QUOTA-11-103` (DONE 2025-10-21) | READY | 2025-10-26T14:07Z | DSSE signing, referrer verification, and quota enforcement validated in CI. | | Signer | Signer Guild | `SIGNER-API-11-101` / `SIGNER-REF-11-102` / `SIGNER-QUOTA-11-103` (DONE 2025-10-21) | READY | 2025-10-26T14:07Z | DSSE signing, referrer verification, and quota enforcement validated in CI. |
| Attestor | Attestor Guild | `ATTESTOR-API-11-201` / `ATTESTOR-VERIFY-11-202` / `ATTESTOR-OBS-11-203` (DONE 2025-10-19) | READY | 2025-10-26T14:10Z | Rekor submission/verification pipeline green; telemetry pack published. | | Attestor | Attestor Guild | `ATTESTOR-API-11-201` / `ATTESTOR-VERIFY-11-202` / `ATTESTOR-OBS-11-203` (DONE 2025-10-19) | READY | 2025-10-26T14:10Z | Rekor submission/verification pipeline green; telemetry pack published. |
| Scanner Web + Worker | Scanner WebService Guild | `SCANNER-WEB-09-10x`, `SCANNER-RUNTIME-12-30x` (DONE 2025-10-18 -> 2025-10-24) | READY* | 2025-10-26T14:20Z | Orchestrator envelope work (`SCANNER-EVENTS-16-301/302`) still open; see gaps. | | Scanner Web + Worker | Scanner WebService Guild | `SCANNER-WEB-09-10x`, `SCANNER-RUNTIME-12-30x` (DONE 2025-10-18 -> 2025-10-24) | READY* | 2025-10-26T14:20Z | Orchestrator envelope work (`SCANNER-EVENTS-16-301/302`) still open; see gaps. |
| Concelier Core & Connectors | Concelier Core / Ops Guild | Ops runbook sign-off in `docs/modules/concelier/operations/conflict-resolution.md` (2025-10-16) | READY | 2025-10-26T14:25Z | Conflict resolution & connector coverage accepted; Mongo schema hardening pending (see gaps). | | Concelier Core & Connectors | Concelier Core / Ops Guild | Ops runbook sign-off in `docs/modules/concelier/operations/conflict-resolution.md` (2025-10-16) | READY | 2025-10-26T14:25Z | Conflict resolution & connector coverage accepted; Mongo schema hardening pending (see gaps). |
| Excititor API | Excititor Core Guild | Wave 0 connector ingest sign-offs (EXECPLAN.Section Wave 0) | READY | 2025-10-26T14:28Z | VEX linkset publishing complete for launch datasets. | | Excititor API | Excititor Core Guild | Wave 0 connector ingest sign-offs (EXECPLAN.Section Wave 0) | READY | 2025-10-26T14:28Z | VEX linkset publishing complete for launch datasets. |
| Notify Web (legacy) | Notify Guild | Existing stack carried forward; Notifier program tracked separately (Sprint 38-40) | PENDING | 2025-10-26T14:32Z | Legacy notify web remains operational; migration to Notifier blocked on `SCANNER-EVENTS-16-301`. | | Notify Web (legacy) | Notify Guild | Existing stack carried forward; Notifier program tracked separately (Sprint 38-40) | PENDING | 2025-10-26T14:32Z | Legacy notify web remains operational; migration to Notifier blocked on `SCANNER-EVENTS-16-301`. |
| Web UI | UI Guild | Stable build `registry.stella-ops.org/.../web-ui@sha256:10d9248...` deployed in stage and smoke-tested | READY | 2025-10-26T14:35Z | Policy editor GA items (Sprint 20) outside launch scope. | | Web UI | UI Guild | Stable build `registry.stella-ops.org/.../web-ui@sha256:10d9248...` deployed in stage and smoke-tested | READY | 2025-10-26T14:35Z | Policy editor GA items (Sprint 20) outside launch scope. |
| DevOps / Release | DevOps Guild | `deploy/tools/validate-profiles.sh` run (2025-10-26) covering dev/stage/prod/airgap/mirror | READY | 2025-10-26T15:02Z | Compose/Helm lint + docker compose config validated; see Section 2 for details. | | DevOps / Release | DevOps Guild | `deploy/tools/validate-profiles.sh` run (2025-10-26) covering dev/stage/prod/airgap/mirror | READY | 2025-10-26T15:02Z | Compose/Helm lint + docker compose config validated; see Section 2 for details. |
| Offline Kit | Offline Kit Guild | `DEVOPS-OFFLINE-18-004` (Go analyzer) and `DEVOPS-OFFLINE-18-005` (Python analyzer) complete; debug-store mirror pending (`DEVOPS-OFFLINE-17-004`). | PENDING | 2025-10-26T15:05Z | Awaiting release debug artefacts to finalise `DEVOPS-OFFLINE-17-004`; tracked in Section 3. | | Offline Kit | Offline Kit Guild | `DEVOPS-OFFLINE-18-004` (Go analyzer) and `DEVOPS-OFFLINE-18-005` (Python analyzer) complete; debug-store mirror pending (`DEVOPS-OFFLINE-17-004`). | PENDING | 2025-10-26T15:05Z | Awaiting release debug artefacts to finalise `DEVOPS-OFFLINE-17-004`; tracked in Section 3. |
_\* READY with caveat - remaining work noted in Section 3._ _\* READY with caveat - remaining work noted in Section 3._
## 2. Deployment Readiness Checklist ## 2. Deployment Readiness Checklist
- **Production profiles committed:** `deploy/compose/docker-compose.prod.yaml` and `deploy/helm/stellaops/values-prod.yaml` added with front-door network hand-off and secret references for Mongo/MinIO/core services. - **Production profiles committed:** `deploy/compose/docker-compose.prod.yaml` and `deploy/helm/stellaops/values-prod.yaml` added with front-door network hand-off and secret references for Mongo/MinIO/core services.
- **Secrets placeholders documented:** `deploy/compose/env/prod.env.example` enumerates required credentials (`MONGO_INITDB_ROOT_PASSWORD`, `MINIO_ROOT_PASSWORD`, Redis/NATS endpoints, `FRONTDOOR_NETWORK`). Helm values reference Kubernetes secrets (`stellaops-prod-core`, `stellaops-prod-mongo`, `stellaops-prod-minio`, `stellaops-prod-notify`). - **Secrets placeholders documented:** `deploy/compose/env/prod.env.example` enumerates required credentials (`MONGO_INITDB_ROOT_PASSWORD`, `MINIO_ROOT_PASSWORD`, Redis/NATS endpoints, `FRONTDOOR_NETWORK`). Helm values reference Kubernetes secrets (`stellaops-prod-core`, `stellaops-prod-mongo`, `stellaops-prod-minio`, `stellaops-prod-notify`).
- **Static validation executed:** `deploy/tools/validate-profiles.sh` run on 2025-10-26 (docker compose config + helm lint/template) with all profiles passing. - **Static validation executed:** `deploy/tools/validate-profiles.sh` run on 2025-10-26 (docker compose config + helm lint/template) with all profiles passing.
- **Ingress model defined:** Production compose profile introduces external `frontdoor` network; README updated with creation instructions and scope of externally reachable services. - **Ingress model defined:** Production compose profile introduces external `frontdoor` network; README updated with creation instructions and scope of externally reachable services.
- **Observability hooks:** Authority/Signer/Attestor telemetry packs verified; scanner runtime build-id metrics landed (`SCANNER-RUNTIME-17-401`). Grafana dashboards referenced in component runbooks. - **Observability hooks:** Authority/Signer/Attestor telemetry packs verified; scanner runtime build-id metrics landed (`SCANNER-RUNTIME-17-401`). Grafana dashboards referenced in component runbooks.
- **Rollback assets:** Stage Compose profile remains aligned (`docker-compose.stage.yaml`), enabling rehearsals before prod cutover; release manifests (`deploy/releases/2025.09-stable.yaml`) map digests for reproducible rollback. - **Rollback assets:** Stage Compose profile remains aligned (`docker-compose.stage.yaml`), enabling rehearsals before prod cutover; release manifests (`deploy/releases/2025.09-stable.yaml`) map digests for reproducible rollback.
- **Rehearsal status:** 2025-10-26 validation dry-run executed (`deploy/tools/validate-profiles.sh` across dev/stage/prod/airgap/mirror). Full stage Helm rollout pending access to the managed staging cluster; target to complete once credentials are provisioned. - **Rehearsal status:** 2025-10-26 validation dry-run executed (`deploy/tools/validate-profiles.sh` across dev/stage/prod/airgap/mirror). Full stage Helm rollout pending access to the managed staging cluster; target to complete once credentials are provisioned.
## 3. Outstanding Gaps & Follow-ups ## 3. Outstanding Gaps & Follow-ups
| Item | Owner | Tracking Ref | Target / Next Step | Impact | | Item | Owner | Tracking Ref | Target / Next Step | Impact |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| Tenant scope propagation and audit coverage | Authority Core Guild | `AUTH-AOC-19-002` (DOING 2025-10-26) | Land enforcement + audit fixtures by Sprint 19 freeze | Medium - required for multi-tenant GA but does not block initial cutover if tenants scoped manually. | | Tenant scope propagation and audit coverage | Authority Core Guild | `AUTH-AOC-19-002` (DOING 2025-10-26) | Land enforcement + audit fixtures by Sprint 19 freeze | Medium - required for multi-tenant GA but does not block initial cutover if tenants scoped manually. |
| Orchestrator event envelopes + Notifier handshake | Scanner WebService Guild | `SCANNER-EVENTS-16-301` (BLOCKED), `SCANNER-EVENTS-16-302` (DOING) | Coordinate with Gateway/Notifier owners on preview package replacement or binding redirects; rerun `dotnet test` once patch lands and refresh schema docs. Share envelope samples in `docs/events/` after tests pass. | High — gating Notifier migration; legacy notify path remains functional meanwhile. | | Orchestrator event envelopes + Notifier handshake | Scanner WebService Guild | `SCANNER-EVENTS-16-301` (BLOCKED), `SCANNER-EVENTS-16-302` (DOING) | Coordinate with Gateway/Notifier owners on preview package replacement or binding redirects; rerun `dotnet test` once patch lands and refresh schema docs. Share envelope samples in `docs/events/` after tests pass. | High — gating Notifier migration; legacy notify path remains functional meanwhile. |
| Offline Kit Python analyzer bundle | Offline Kit Guild + Scanner Guild | `DEVOPS-OFFLINE-18-005` (DONE 2025-10-26) | Monitor for follow-up manifest updates and rerun smoke script when analyzers change. | Medium - ensures language analyzer coverage stays current for offline installs. | | Offline Kit Python analyzer bundle | Offline Kit Guild + Scanner Guild | `DEVOPS-OFFLINE-18-005` (DONE 2025-10-26) | Monitor for follow-up manifest updates and rerun smoke script when analyzers change. | Medium - ensures language analyzer coverage stays current for offline installs. |
| Offline Kit debug store mirror | Offline Kit Guild + DevOps Guild | `DEVOPS-OFFLINE-17-004` (BLOCKED 2025-10-26) | Release pipeline must publish `out/release/debug` artefacts; once available, run `mirror_debug_store.py` and commit `metadata/debug-store.json`. | Low - symbol lookup remains accessible from staging assets but required before next Offline Kit tag. | | Offline Kit debug store mirror | Offline Kit Guild + DevOps Guild | `DEVOPS-OFFLINE-17-004` (BLOCKED 2025-10-26) | Release pipeline must publish `out/release/debug` artefacts; once available, run `mirror_debug_store.py` and commit `metadata/debug-store.json`. | Low - symbol lookup remains accessible from staging assets but required before next Offline Kit tag. |
| Mongo schema validators for advisory ingestion | Concelier Storage Guild | `CONCELIER-STORE-AOC-19-001` (TODO) | Finalize JSON schema + migration toggles; coordinate with Ops for rollout window | Low - current validation handled in app layer; schema guard adds defense-in-depth. | | Mongo schema validators for advisory ingestion | Concelier Storage Guild | `CONCELIER-STORE-AOC-19-001` (TODO) | Finalize JSON schema + migration toggles; coordinate with Ops for rollout window | Low - current validation handled in app layer; schema guard adds defense-in-depth. |
| Authority plugin telemetry alignment | Security Guild | `SEC2.PLG`, `SEC3.PLG`, `SEC5.PLG` (BLOCKED pending AUTH DPoP/MTLS tasks) | Resume once upstream auth surfacing stabilises | Low - plugin remains optional; launch uses default Authority configuration. | | Authority plugin telemetry alignment | Security Guild | `SEC2.PLG`, `SEC3.PLG`, `SEC5.PLG` (BLOCKED pending AUTH DPoP/MTLS tasks) | Resume once upstream auth surfacing stabilises | Low - plugin remains optional; launch uses default Authority configuration. |
## 4. Approvals & Distribution ## 4. Approvals & Distribution
- Record shared in `#launch-readiness` (Mattermost) 2025-10-26 15:15 UTC with DevOps + Guild leads for acknowledgement. - Record shared in `#launch-readiness` (Mattermost) 2025-10-26 15:15 UTC with DevOps + Guild leads for acknowledgement.
- Updates to this document require dual sign-off from DevOps Guild (owner) and impacted module guild lead; retain change log via Git history. - Updates to this document require dual sign-off from DevOps Guild (owner) and impacted module guild lead; retain change log via Git history.
- Cutover rehearsal and rollback drills are tracked separately in `docs/modules/devops/runbooks/launch-cutover.md` (see associated Task `DEVOPS-LAUNCH-18-001`). *** End Patch - Cutover rehearsal and rollback drills are tracked separately in `docs/modules/devops/runbooks/launch-cutover.md` (see associated Task `DEVOPS-LAUNCH-18-001`). *** End Patch

128
docs/modules/devops/runbooks/nuget-preview-bootstrap.md
View File

@@ -1,64 +1,64 @@
# NuGet Preview Bootstrap (Offline-Friendly) # NuGet Preview Bootstrap (Offline-Friendly)
The StellaOps build relies on .NET 10 RC2 packages (Microsoft.Extensions.*, JwtBearer 10.0 RC). The StellaOps build relies on .NET 10 RC2 packages (Microsoft.Extensions.*, JwtBearer 10.0 RC).
`NuGet.config` now wires three sources: `NuGet.config` now wires three sources:
1. `local` → `./local-nuget` (preferred, air-gapped mirror) 1. `local` → `./local-nuget` (preferred, air-gapped mirror)
2. `dotnet-public` → `https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json` 2. `dotnet-public` → `https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json`
3. `nuget.org` → fallback for everything else 3. `nuget.org` → fallback for everything else
Follow the steps below whenever you refresh the repo or roll a new Offline Kit drop. Follow the steps below whenever you refresh the repo or roll a new Offline Kit drop.
## 1. Mirror the preview packages ## 1. Mirror the preview packages
```bash ```bash
./ops/devops/sync-preview-nuget.sh ./ops/devops/sync-preview-nuget.sh
``` ```
* Reads `ops/devops/nuget-preview-packages.csv`. Each line specifies the package, version, expected SHA-256 hash, and (optionally) the flat-container base URL (we pin to `dotnet-public`). * Reads `ops/devops/nuget-preview-packages.csv`. Each line specifies the package, version, expected SHA-256 hash, and (optionally) the flat-container base URL (we pin to `dotnet-public`).
* Downloads the `.nupkg` straight into `./local-nuget/` and re-verifies the checksum. Existing files are skipped when hashes already match. * Downloads the `.nupkg` straight into `./local-nuget/` and re-verifies the checksum. Existing files are skipped when hashes already match.
* Use `NUGET_V2_BASE` if you need to temporarily point at a different mirror. * Use `NUGET_V2_BASE` if you need to temporarily point at a different mirror.
💡 The script never mutates packages in place—if a checksum changes you will see a “SHA mismatch … refreshing” message. 💡 The script never mutates packages in place—if a checksum changes you will see a “SHA mismatch … refreshing” message.
## 2. Restore using the shared `NuGet.config` ## 2. Restore using the shared `NuGet.config`
From the repo root: From the repo root:
```bash ```bash
DOTNET_NOLOGO=1 dotnet restore src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/StellaOps.Excititor.Connectors.Abstractions.csproj \ DOTNET_NOLOGO=1 dotnet restore src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/StellaOps.Excititor.Connectors.Abstractions.csproj \
--configfile NuGet.config --configfile NuGet.config
``` ```
The `packageSourceMapping` section keeps `Microsoft.Extensions.*`, `Microsoft.AspNetCore.*`, and `Microsoft.Data.Sqlite` bound to `local`/`dotnet-public`, so `dotnet restore` never has to reach out to nuget.org when mirrors are populated. The `packageSourceMapping` section keeps `Microsoft.Extensions.*`, `Microsoft.AspNetCore.*`, and `Microsoft.Data.Sqlite` bound to `local`/`dotnet-public`, so `dotnet restore` never has to reach out to nuget.org when mirrors are populated.
Before committing changes (or when wiring up a new environment) run: Before committing changes (or when wiring up a new environment) run:
```bash ```bash
python3 ops/devops/validate_restore_sources.py python3 ops/devops/validate_restore_sources.py
``` ```
The validator asserts: The validator asserts:
- `NuGet.config` lists `local` → `dotnet-public` → `nuget.org` in that order. - `NuGet.config` lists `local` → `dotnet-public` → `nuget.org` in that order.
- `Directory.Build.props` pins `RestoreSources` so every project prioritises the local mirror. - `Directory.Build.props` pins `RestoreSources` so every project prioritises the local mirror.
- No stray `NuGet.config` files shadow the repo root configuration. - No stray `NuGet.config` files shadow the repo root configuration.
CI executes the validator in both the `build-test-deploy` and `release` workflows, CI executes the validator in both the `build-test-deploy` and `release` workflows,
so regressions trip before any restore/build begins. so regressions trip before any restore/build begins.
If you run fully air-gapped, remember to clear the cache between SDK upgrades: If you run fully air-gapped, remember to clear the cache between SDK upgrades:
```bash ```bash
dotnet nuget locals all --clear dotnet nuget locals all --clear
``` ```
## 3. Troubleshooting ## 3. Troubleshooting
| Symptom | Fix | | Symptom | Fix |
| --- | --- | | --- | --- |
| `dotnet restore` still hits nuget.org for preview packages | Re-run `sync-preview-nuget.sh` to ensure the `.nupkg` exists locally, then delete `~/.nuget/packages/microsoft.extensions.*` so the resolver picks up the mirrored copy. | | `dotnet restore` still hits nuget.org for preview packages | Re-run `sync-preview-nuget.sh` to ensure the `.nupkg` exists locally, then delete `~/.nuget/packages/microsoft.extensions.*` so the resolver picks up the mirrored copy. |
| SHA mismatch in the manifest | Update `ops/devops/nuget-preview-packages.csv` with the new version + checksum (from the feed) and re-run the sync script. | | SHA mismatch in the manifest | Update `ops/devops/nuget-preview-packages.csv` with the new version + checksum (from the feed) and re-run the sync script. |
| Azure DevOps feed throttling | Set `DOTNET_PUBLIC_FLAT_BASE` env var and point it at your own mirrored flat-container, then add the URL to the 4th column of the manifest. | | Azure DevOps feed throttling | Set `DOTNET_PUBLIC_FLAT_BASE` env var and point it at your own mirrored flat-container, then add the URL to the 4th column of the manifest. |
Keep this doc alongside Offline Kit instructions so air-gapped operators know exactly how to refresh the mirror and verify packages before restore. Keep this doc alongside Offline Kit instructions so air-gapped operators know exactly how to refresh the mirror and verify packages before restore.

42
docs/modules/excititor/AGENTS.md
View File

@@ -1,22 +1,22 @@
# Excititor agent guide # Excititor agent guide
## Mission ## Mission
Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract. Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

66
docs/modules/excititor/README.md
View File

@@ -1,33 +1,33 @@
# StellaOps Excititor # StellaOps Excititor
Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract. Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract.
## Responsibilities ## Responsibilities
- Fetch OpenVEX/CSAF/CycloneDX statements via restart-only connectors. - Fetch OpenVEX/CSAF/CycloneDX statements via restart-only connectors.
- Store immutable VEX observations with full provenance. - Store immutable VEX observations with full provenance.
- Publish linksets and events that drive policy suppression decisions. - Publish linksets and events that drive policy suppression decisions.
- Provide deterministic exports for Offline Kit and downstream tooling. - Provide deterministic exports for Offline Kit and downstream tooling.
## Key components ## Key components
- `StellaOps.Excititor.WebService` scheduler/API host. - `StellaOps.Excititor.WebService` scheduler/API host.
- Connector libraries under `StellaOps.Excititor.Connector.*`. - Connector libraries under `StellaOps.Excititor.Connector.*`.
- Normalization helpers and exporters in `StellaOps.Excititor.*`. - Normalization helpers and exporters in `StellaOps.Excititor.*`.
## Integrations & dependencies ## Integrations & dependencies
- Policy Engine for evidence queries. - Policy Engine for evidence queries.
- UI/CLI for conflict visibility and explanation. - UI/CLI for conflict visibility and explanation.
- Notify for VEX-driven alerts. - Notify for VEX-driven alerts.
## Operational notes ## Operational notes
- MongoDB for observation storage and job metadata. - MongoDB for observation storage and job metadata.
- Offline kit packaging aligned with Concelier merges. - Offline kit packaging aligned with Concelier merges.
- Connector-specific runbooks (see `docs/modules/concelier/operations/connectors`). - Connector-specific runbooks (see `docs/modules/concelier/operations/connectors`).
## Backlog references ## Backlog references
- DOCS-LNM-22-006 / DOCS-LNM-22-007 (shared with Concelier). - DOCS-LNM-22-006 / DOCS-LNM-22-007 (shared with Concelier).
- CLI-EXC-25-001..002 follow-up for CLI parity. - CLI-EXC-25-001..002 follow-up for CLI parity.
## Epic alignment ## Epic alignment
- **Epic 1 – AOC enforcement:** maintain immutable VEX observations, provenance, and AOC verifier coverage. - **Epic 1 – AOC enforcement:** maintain immutable VEX observations, provenance, and AOC verifier coverage.
- **Epic 7 – VEX Consensus Lens:** supply trustworthy raw inputs, trust metadata, and consensus hooks for the lens computations. - **Epic 7 – VEX Consensus Lens:** supply trustworthy raw inputs, trust metadata, and consensus hooks for the lens computations.
- **Epic 8 – Advisory AI:** expose citation-ready VEX payloads for the advisory assistant pipeline. - **Epic 8 – Advisory AI:** expose citation-ready VEX payloads for the advisory assistant pipeline.

18
docs/modules/excititor/TASKS.md
View File

@@ -1,9 +1,9 @@
# Task board — Excititor # Task board — Excititor
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable. > Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes | | ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------| |----|--------|----------|-------------|-------|
| EXCITITOR-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md | | EXCITITOR-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| EXCITITOR-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md | | EXCITITOR-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| EXCITITOR-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow | | EXCITITOR-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

1498
docs/modules/excititor/architecture.md
View File

File diff suppressed because it is too large Load Diff

42
docs/modules/excititor/implementation_plan.md
View File

@@ -1,21 +1,21 @@
# Implementation plan — Excititor # Implementation plan — Excititor
## Current objectives ## Current objectives
- Maintain deterministic behaviour and offline parity across releases. - Maintain deterministic behaviour and offline parity across releases.
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes. - Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
## Workstreams ## Workstreams
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap. - Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs. - Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
- Validation: extend tests/fixtures to preserve determinism and provenance requirements. - Validation: extend tests/fixtures to preserve determinism and provenance requirements.
## Epic milestones ## Epic milestones
- **Epic 1 – AOC enforcement:** enforce immutable VEX observation schema, provenance capture, and guardrails. - **Epic 1 – AOC enforcement:** enforce immutable VEX observation schema, provenance capture, and guardrails.
- **Epic 7 – VEX Consensus Lens:** provide lens-ready metadata (issuer trust, temporal scoping) and consensus APIs. - **Epic 7 – VEX Consensus Lens:** provide lens-ready metadata (issuer trust, temporal scoping) and consensus APIs.
- **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers. - **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers.
- Track DOCS-LNM-22-006/007 and CLI-EXC-25-001..002 in ../../TASKS.md. - Track DOCS-LNM-22-006/007 and CLI-EXC-25-001..002 in ../../TASKS.md.
## Coordination ## Coordination
- Review ./AGENTS.md before picking up new work. - Review ./AGENTS.md before picking up new work.
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md. - Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
- Update this plan whenever scope, dependencies, or guardrails change. - Update this plan whenever scope, dependencies, or guardrails change.

328
docs/modules/excititor/mirrors.md
View File

@@ -1,164 +1,164 @@
# architecture_excititor_mirrors.md — Excititor Mirror Distribution # architecture_excititor_mirrors.md — Excititor Mirror Distribution
> **Status:** Draft (Sprint 7). Complements `docs/modules/excititor/architecture.md` by describing the mirror export surface exposed by `Excititor.WebService` and the configuration hooks used by operators and downstream mirrors. > **Status:** Draft (Sprint 7). Complements `docs/modules/excititor/architecture.md` by describing the mirror export surface exposed by `Excititor.WebService` and the configuration hooks used by operators and downstream mirrors.
--- ---
## 0) Purpose ## 0) Purpose
Excititor publishes canonical VEX consensus data. Operators (or StellaOps-managed mirrors) need a deterministic way to sync those exports into downstream environments. Mirror distribution provides: Excititor publishes canonical VEX consensus data. Operators (or StellaOps-managed mirrors) need a deterministic way to sync those exports into downstream environments. Mirror distribution provides:
* A declarative map of export bundles (`json`, `jsonl`, `openvex`, `csaf`) reachable via signed HTTP endpoints under `/excititor/mirror`. * A declarative map of export bundles (`json`, `jsonl`, `openvex`, `csaf`) reachable via signed HTTP endpoints under `/excititor/mirror`.
* Thin quota/authentication controls on top of the existing export cache so mirrors cannot starve the web service. * Thin quota/authentication controls on top of the existing export cache so mirrors cannot starve the web service.
* Stable payload shapes that downstream automation can monitor (index → fetch updates → download artifact → verify signature). * Stable payload shapes that downstream automation can monitor (index → fetch updates → download artifact → verify signature).
Mirror endpoints are intentionally **read-only**. Write paths (export generation, attestation, cache) remain the responsibility of the export pipeline. Mirror endpoints are intentionally **read-only**. Write paths (export generation, attestation, cache) remain the responsibility of the export pipeline.
--- ---
## 1) Configuration model ## 1) Configuration model
The web service reads mirror configuration from `Excititor:Mirror` (YAML/JSON/appsettings). Each domain groups a set of exports that share rate limits and authentication rules. The web service reads mirror configuration from `Excititor:Mirror` (YAML/JSON/appsettings). Each domain groups a set of exports that share rate limits and authentication rules.
```yaml ```yaml
Excititor: Excititor:
Mirror: Mirror:
Domains: Domains:
- id: primary - id: primary
displayName: Primary Mirror displayName: Primary Mirror
requireAuthentication: false requireAuthentication: false
maxIndexRequestsPerHour: 600 maxIndexRequestsPerHour: 600
maxDownloadRequestsPerHour: 1200 maxDownloadRequestsPerHour: 1200
exports: exports:
- key: consensus - key: consensus
format: json format: json
filters: filters:
vulnId: CVE-2025-0001 vulnId: CVE-2025-0001
productKey: pkg:test/demo productKey: pkg:test/demo
sort: sort:
createdAt: false # descending createdAt: false # descending
limit: 1000 limit: 1000
- key: consensus-openvex - key: consensus-openvex
format: openvex format: openvex
filters: filters:
vulnId: CVE-2025-0001 vulnId: CVE-2025-0001
``` ```
### Root settings ### Root settings
| Field | Required | Description | | Field | Required | Description |
| --- | --- | --- | | --- | --- | --- |
| `outputRoot` | – | Filesystem root where mirror artefacts are written. Defaults to the Excititor file-system artifact store root when omitted. | | `outputRoot` | – | Filesystem root where mirror artefacts are written. Defaults to the Excititor file-system artifact store root when omitted. |
| `directoryName` | – | Optional subdirectory created under `outputRoot`; defaults to `mirror`. | | `directoryName` | – | Optional subdirectory created under `outputRoot`; defaults to `mirror`. |
| `targetRepository` | – | Hint propagated to manifests/index files indicating the operator-visible location (for example `s3://mirror/excititor`). | | `targetRepository` | – | Hint propagated to manifests/index files indicating the operator-visible location (for example `s3://mirror/excititor`). |
| `signing` | – | Bundle signing configuration. When enabled, the exporter emits a detached JWS (`bundle.json.jws`) alongside each domain bundle. | | `signing` | – | Bundle signing configuration. When enabled, the exporter emits a detached JWS (`bundle.json.jws`) alongside each domain bundle. |
`signing` supports the following fields: `signing` supports the following fields:
| Field | Required | Description | | Field | Required | Description |
| --- | --- | --- | | --- | --- | --- |
| `enabled` | – | Toggles detached signing for domain bundles. | | `enabled` | – | Toggles detached signing for domain bundles. |
| `algorithm` | – | Signing algorithm identifier (default `ES256`). | | `algorithm` | – | Signing algorithm identifier (default `ES256`). |
| `keyId` | ✅ (when `enabled`) | Signing key identifier resolved via the configured crypto provider registry. | | `keyId` | ✅ (when `enabled`) | Signing key identifier resolved via the configured crypto provider registry. |
| `provider` | – | Optional provider hint when multiple registries are available. | | `provider` | – | Optional provider hint when multiple registries are available. |
| `keyPath` | – | Optional PEM path used to seed the provider when the key is not already loaded. | | `keyPath` | – | Optional PEM path used to seed the provider when the key is not already loaded. |
### Domain field reference ### Domain field reference
| Field | Required | Description | | Field | Required | Description |
| --- | --- | --- | | --- | --- | --- |
| `id` | ✅ | Stable identifier. Appears in URLs (`/excititor/mirror/domains/{id}`) and download filenames. | | `id` | ✅ | Stable identifier. Appears in URLs (`/excititor/mirror/domains/{id}`) and download filenames. |
| `displayName` | – | Human-friendly label surfaced in the `/domains` listing. Falls back to `id`. | | `displayName` | – | Human-friendly label surfaced in the `/domains` listing. Falls back to `id`. |
| `requireAuthentication` | – | When `true` the service enforces that the caller is authenticated (Authority token). | | `requireAuthentication` | – | When `true` the service enforces that the caller is authenticated (Authority token). |
| `maxIndexRequestsPerHour` | – | Per-domain quota for index endpoints. `0`/negative disables the guard. | | `maxIndexRequestsPerHour` | – | Per-domain quota for index endpoints. `0`/negative disables the guard. |
| `maxDownloadRequestsPerHour` | – | Per-domain quota for artifact downloads. | | `maxDownloadRequestsPerHour` | – | Per-domain quota for artifact downloads. |
| `exports` | ✅ | Collection of export projections. | | `exports` | ✅ | Collection of export projections. |
Export-level fields: Export-level fields:
| Field | Required | Description | | Field | Required | Description |
| --- | --- | --- | | --- | --- | --- |
| `key` | ✅ | Unique key within the domain. Used in URLs (`/exports/{key}`) and filenames/bundle entries. | | `key` | ✅ | Unique key within the domain. Used in URLs (`/exports/{key}`) and filenames/bundle entries. |
| `format` | ✅ | One of `json`, `jsonl`, `openvex`, `csaf`. Maps to `VexExportFormat`. | | `format` | ✅ | One of `json`, `jsonl`, `openvex`, `csaf`. Maps to `VexExportFormat`. |
| `filters` | – | Key/value pairs executed via `VexQueryFilter`. Keys must match export data source columns (e.g., `vulnId`, `productKey`). | | `filters` | – | Key/value pairs executed via `VexQueryFilter`. Keys must match export data source columns (e.g., `vulnId`, `productKey`). |
| `sort` | – | Key/boolean map (false = descending). | | `sort` | – | Key/boolean map (false = descending). |
| `limit`, `offset`, `view` | – | Optional query bounds passed through to the export query. | | `limit`, `offset`, `view` | – | Optional query bounds passed through to the export query. |
⚠️ **Misconfiguration:** invalid formats or missing keys cause exports to be flagged with `status` in the index response; they are not exposed downstream. ⚠️ **Misconfiguration:** invalid formats or missing keys cause exports to be flagged with `status` in the index response; they are not exposed downstream.
--- ---
## 2) HTTP surface ## 2) HTTP surface
Routes are grouped under `/excititor/mirror`. Routes are grouped under `/excititor/mirror`.
| Method | Path | Description | | Method | Path | Description |
| --- | --- | --- | | --- | --- | --- |
| `GET` | `/domains` | Returns configured domains with quota metadata. | | `GET` | `/domains` | Returns configured domains with quota metadata. |
| `GET` | `/domains/{domainId}` | Domain detail (auth/quota + export keys). `404` for unknown domains. | | `GET` | `/domains/{domainId}` | Domain detail (auth/quota + export keys). `404` for unknown domains. |
| `GET` | `/domains/{domainId}/index` | Lists exports with exportId, query signature, format, artifact digest, attestation metadata, and size. Applies index quota. | | `GET` | `/domains/{domainId}/index` | Lists exports with exportId, query signature, format, artifact digest, attestation metadata, and size. Applies index quota. |
| `GET` | `/domains/{domainId}/exports/{exportKey}` | Returns manifest metadata (single export). `404` if unknown/missing. | | `GET` | `/domains/{domainId}/exports/{exportKey}` | Returns manifest metadata (single export). `404` if unknown/missing. |
| `GET` | `/domains/{domainId}/exports/{exportKey}/download` | Streams export content from the artifact store. Applies download quota. | | `GET` | `/domains/{domainId}/exports/{exportKey}/download` | Streams export content from the artifact store. Applies download quota. |
Responses are serialized via `VexCanonicalJsonSerializer` ensuring stable ordering. Download responses include a content-disposition header naming the file `<domain>-<export>.<ext>`. Responses are serialized via `VexCanonicalJsonSerializer` ensuring stable ordering. Download responses include a content-disposition header naming the file `<domain>-<export>.<ext>`.
### Error handling ### Error handling
* `401` – authentication required (`requireAuthentication=true`). * `401` – authentication required (`requireAuthentication=true`).
* `404` – domain/export not found or manifest not persisted. * `404` – domain/export not found or manifest not persisted.
* `429` – per-domain quota exceeded (`Retry-After` header set in seconds). * `429` – per-domain quota exceeded (`Retry-After` header set in seconds).
* `503` – export misconfiguration (invalid format/query). * `503` – export misconfiguration (invalid format/query).
--- ---
## 3) Rate limiting ## 3) Rate limiting
`MirrorRateLimiter` implements a simple rolling 1-hour window using `IMemoryCache`. Each domain has two quotas: `MirrorRateLimiter` implements a simple rolling 1-hour window using `IMemoryCache`. Each domain has two quotas:
* `index` scope → `maxIndexRequestsPerHour` * `index` scope → `maxIndexRequestsPerHour`
* `download` scope → `maxDownloadRequestsPerHour` * `download` scope → `maxDownloadRequestsPerHour`
`0` or negative limits disable enforcement. Quotas are best-effort (per-instance). For HA deployments, configure sticky routing at the ingress or replace the limiter with a distributed implementation. `0` or negative limits disable enforcement. Quotas are best-effort (per-instance). For HA deployments, configure sticky routing at the ingress or replace the limiter with a distributed implementation.
--- ---
## 4) Interaction with export pipeline ## 4) Interaction with export pipeline
Mirror endpoints consume manifests produced by the export engine (`MongoVexExportStore`). They do **not** trigger new exports. Operators must configure connectors/exporters to keep targeted exports fresh (see `EXCITITOR-EXPORT-01-005/006/007`). Mirror endpoints consume manifests produced by the export engine (`MongoVexExportStore`). They do **not** trigger new exports. Operators must configure connectors/exporters to keep targeted exports fresh (see `EXCITITOR-EXPORT-01-005/006/007`).
Recommended workflow: Recommended workflow:
1. Define export plans at the export layer (JSON/OpenVEX/CSAF). 1. Define export plans at the export layer (JSON/OpenVEX/CSAF).
2. Configure mirror domains mapping to those plans. 2. Configure mirror domains mapping to those plans.
3. Downstream mirror automation: 3. Downstream mirror automation:
* `GET /domains/{id}/index` * `GET /domains/{id}/index`
* Compare `exportId` / `consensusRevision` * Compare `exportId` / `consensusRevision`
* `GET /download` when new * `GET /download` when new
* Verify digest + attestation * Verify digest + attestation
When the export engine runs, it materializes the following artefacts under `outputRoot/<directoryName>`: When the export engine runs, it materializes the following artefacts under `outputRoot/<directoryName>`:
- `index.json` – canonical index listing each configured domain, manifest/bundle descriptors (with SHA-256 digests), and available export keys. - `index.json` – canonical index listing each configured domain, manifest/bundle descriptors (with SHA-256 digests), and available export keys.
- `<domain>/manifest.json` – per-domain summary with export metadata (query signature, consensus/score digests, source providers) and a descriptor pointing at the bundle. - `<domain>/manifest.json` – per-domain summary with export metadata (query signature, consensus/score digests, source providers) and a descriptor pointing at the bundle.
- `<domain>/bundle.json` – canonical payload containing serialized consensus, score envelopes, and normalized VEX claims for the matching export definitions. - `<domain>/bundle.json` – canonical payload containing serialized consensus, score envelopes, and normalized VEX claims for the matching export definitions.
- `<domain>/bundle.json.jws` – optional detached JWS when signing is enabled. - `<domain>/bundle.json.jws` – optional detached JWS when signing is enabled.
Downstream automation reads `manifest.json`/`bundle.json` directly, while `/excititor/mirror` endpoints stream the same artefacts through authenticated HTTP. Downstream automation reads `manifest.json`/`bundle.json` directly, while `/excititor/mirror` endpoints stream the same artefacts through authenticated HTTP.
--- ---
## 5) Operational guidance ## 5) Operational guidance
* Track quota utilisation via HTTP 429 metrics (configure structured logging or OTEL counters when rate limiting triggers). * Track quota utilisation via HTTP 429 metrics (configure structured logging or OTEL counters when rate limiting triggers).
* Mirror domains can be deployed per tenant (e.g., `tenant-a`, `tenant-b`) with different auth requirements. * Mirror domains can be deployed per tenant (e.g., `tenant-a`, `tenant-b`) with different auth requirements.
* Ensure the underlying artifact stores (`FileSystem`, `S3`, offline bundle) retain artefacts long enough for mirrors to sync. * Ensure the underlying artifact stores (`FileSystem`, `S3`, offline bundle) retain artefacts long enough for mirrors to sync.
* For air-gapped mirrors, combine mirror endpoints with the Offline Kit (see `docs/24_OFFLINE_KIT.md`). * For air-gapped mirrors, combine mirror endpoints with the Offline Kit (see `docs/24_OFFLINE_KIT.md`).
--- ---
## 6) Future alignment ## 6) Future alignment
* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships. * Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata. * Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively. * Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.

42
docs/modules/export-center/AGENTS.md
View File

@@ -1,22 +1,22 @@
# Export Center agent guide # Export Center agent guide
## Mission ## Mission
Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) with provenance metadata and optional signing for offline or mirrored deployments. Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) with provenance metadata and optional signing for offline or mirrored deployments.
## Key docs ## Key docs
- [Module README](./README.md) - [Module README](./README.md)
- [Architecture](./architecture.md) - [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md) - [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md) - [Task board](./TASKS.md)
## How to get started ## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module. 1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED). 2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs. 3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan. 4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails ## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md). - Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts. - Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature. - Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change. - Update runbooks/observability assets when operational characteristics change.

68
docs/modules/export-center/README.md
View File

@@ -1,34 +1,34 @@
# StellaOps Export Center # StellaOps Export Center
Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) with provenance metadata and optional signing for offline or mirrored deployments. Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) with provenance metadata and optional signing for offline or mirrored deployments.
## Responsibilities ## Responsibilities
- Coordinate export jobs based on profiles and scope selectors. - Coordinate export jobs based on profiles and scope selectors.
- Assemble manifests, provenance documents, and cosign signatures. - Assemble manifests, provenance documents, and cosign signatures.
- Stream bundles via HTTP/OCI and stage them for Offline Kit uses. - Stream bundles via HTTP/OCI and stage them for Offline Kit uses.
- Expose CLI/API surfaces for automation. - Expose CLI/API surfaces for automation.
## Key components ## Key components
- `StellaOps.ExportCenter.WebService` planner. - `StellaOps.ExportCenter.WebService` planner.
- `StellaOps.ExportCenter.Worker` bundle builder. - `StellaOps.ExportCenter.Worker` bundle builder.
- Adapters in `StellaOps.ExportCenter.*` for JSON/Trivy/mirror variants. - Adapters in `StellaOps.ExportCenter.*` for JSON/Trivy/mirror variants.
## Integrations & dependencies ## Integrations & dependencies
- Concelier/Excititor/Policy data stores for evidence. - Concelier/Excititor/Policy data stores for evidence.
- Signer/Attestor for provenance signing. - Signer/Attestor for provenance signing.
- CLI for operator-managed exports. - CLI for operator-managed exports.
## Operational notes ## Operational notes
- Runbooks in ./operations/ for deployment and monitoring. - Runbooks in ./operations/ for deployment and monitoring.
- Mirror bundle instructions and validation notes. - Mirror bundle instructions and validation notes.
- Telemetry dashboards for export latency and retry rates. - Telemetry dashboards for export latency and retry rates.
## Related resources ## Related resources
- ./operations/runbook.md - ./operations/runbook.md
## Backlog references ## Backlog references
- DOCS-EXPORT-35-001 … DOCS-EXPORT-37-002 in ../../TASKS.md. - DOCS-EXPORT-35-001 … DOCS-EXPORT-37-002 in ../../TASKS.md.
- EXPORT-ATTEST-75-002 cross-team deliverable. - EXPORT-ATTEST-75-002 cross-team deliverable.
## Epic alignment ## Epic alignment
- **Epic 10 – Export Center:** deliver canonical JSON, Trivy DB, and mirror bundle workflows with provenance, signatures, and offline parity. - **Epic 10 – Export Center:** deliver canonical JSON, Trivy DB, and mirror bundle workflows with provenance, signatures, and offline parity.

Some files were not shown because too many files have changed in this diff Show More