feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes. - Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes. - Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables. - Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
This commit is contained in:
@@ -616,7 +616,7 @@ See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-
|
||||
|---------|---------|-----------------------|-------|
|
||||
| `stellaops-cli scanner download` | Fetch and install scanner container | `--channel <stable\|beta\|nightly>` (default `stable`)<br>`--output <path>`<br>`--overwrite`<br>`--no-install` | Saves artefact under `ScannerCacheDirectory`, verifies digest/signature, and executes `docker load` unless `--no-install` is supplied. |
|
||||
| `stellaops-cli scan run` | Execute scanner container against a directory (auto-upload) | `--target <directory>` (required)<br>`--runner <docker\|dotnet\|self>` (default from config)<br>`--entry <image-or-entrypoint>`<br>`[scanner-args...]` | Runs the scanner, writes results into `ResultsDirectory`, emits a structured `scan-run-*.json` metadata file, and automatically uploads the artefact when the exit code is `0`. |
|
||||
| `stellaops-cli scan upload` | Re-upload existing scan artefact | `--file <path>` | Useful for retries when automatic upload fails or when operating offline. |
|
||||
| `stellaops-cli scan upload` | Re-upload existing scan artefact | `--file <path>` | Useful for retries when automatic upload fails or when operating offline. |
|
||||
| `stellaops-cli db fetch` | Trigger connector jobs | `--source <id>` (e.g. `redhat`, `osv`)<br>`--stage <fetch\|parse\|map>` (default `fetch`)<br>`--mode <resume|init|cursor>` | Translates to `POST /jobs/source:{source}:{stage}` with `trigger=cli` |
|
||||
| `stellaops-cli db merge` | Run canonical merge reconcile | — | Calls `POST /jobs/merge:reconcile`; exit code `0` on acceptance, `1` on failures/conflicts |
|
||||
| `stellaops-cli db export` | Kick JSON / Trivy exports | `--format <json\|trivy-db>` (default `json`)<br>`--delta`<br>`--publish-full/--publish-delta`<br>`--bundle-full/--bundle-delta` | Sets `{ delta = true }` parameter when requested and can override ORAS/bundle toggles per run |
|
||||
@@ -629,7 +629,9 @@ See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-
|
||||
| `stellaops-cli sources ingest --dry-run` | Dry-run guard validation for individual payloads | `--source <id>`<br>`--input <path\|uri>`<br>`--tenant <id>`<br>`--format table\|json`<br>`--output <file>` | Normalises gzip/base64 payloads, invokes `api/aoc/ingest/dry-run`, and maps guard failures to deterministic `ERR_AOC_00x` exit codes. |
|
||||
| `stellaops-cli aoc verify` | Replay AOC guardrails over stored documents | `--since <ISO8601\|duration>`<br>`--limit <count>`<br>`--sources <list>`<br>`--codes <ERR_AOC_00x,...>`<br>`--format table\|json`<br>`--export <file>` | Summarises checked counts/violations, supports JSON evidence exports, and returns `0`, `11…17`, `18`, `70`, or `71` depending on guard outcomes. |
|
||||
| `stellaops-cli config show` | Display resolved configuration | — | Masks secret values; helpful for air‑gapped installs |
|
||||
| `stellaops-cli runtime policy test` | Ask Scanner.WebService for runtime verdicts (Webhook parity) | `--image/-i <digest>` (repeatable, comma/space lists supported)<br>`--file/-f <path>`<br>`--namespace/--ns <name>`<br>`--label/-l key=value` (repeatable)<br>`--json` | Posts to `POST /api/v1/scanner/policy/runtime`, deduplicates image digests, and prints TTL/policy revision plus per-image columns for signed state, SBOM referrers, quieted-by metadata, confidence, Rekor attestation (uuid + verified flag), and recently observed build IDs (shortened for readability). Accepts newline/whitespace-delimited stdin when piped; `--json` emits the raw response without additional logging. |
|
||||
| `stellaops-cli runtime policy test` | Ask Scanner.WebService for runtime verdicts (Webhook parity) | `--image/-i <digest>` (repeatable, comma/space lists supported)<br>`--file/-f <path>`<br>`--namespace/--ns <name>`<br>`--label/-l key=value` (repeatable)<br>`--json` | Posts to `POST /api/v1/scanner/policy/runtime`, deduplicates image digests, and prints TTL/policy revision plus per-image columns for signed state, SBOM referrers, quieted-by metadata, confidence, Rekor attestation (uuid + verified flag), and recently observed build IDs (shortened for readability). Accepts newline/whitespace-delimited stdin when piped; `--json` emits the raw response without additional logging. |
|
||||
|
||||
> Need to debug how the scanner resolves entry points? See the [entry-point documentation index](modules/scanner/operations/entrypoint.md), which links to static/dynamic reducers, ShellFlow, and runtime-specific guides.
|
||||
|
||||
#### Example: Pivot from runtime verdicts to debug symbols
|
||||
|
||||
|
||||
@@ -351,4 +351,4 @@ a problem document.
|
||||
- **Rollout checklist.**
|
||||
1. Stage the integration with fallback enabled (`allowAnonymousFallback=true`) and confirm CLI/token issuance using `stella auth status`.
|
||||
2. Follow the rehearsal pattern (`allowAnonymousFallback=false`) while monitoring `Concelier.Authorization.Audit` and `web.jobs.triggered`/`web.jobs.trigger.failed` metrics.
|
||||
3. Lock in enforcement, review the audit runbook (`docs/ops/concelier-authority-audit-runbook.md`), and document the bypass CIDR approvals in your change log.
|
||||
3. Lock in enforcement, review the audit runbook (`docs/modules/concelier/operations/authority-audit-runbook.md`), and document the bypass CIDR approvals in your change log.
|
||||
|
||||
@@ -452,14 +452,14 @@ Planned for Q1‑2026 (kept here for early plug‑in authors).
|
||||
|
||||
## 6 Notify Foundations (Rule · Channel · Event)
|
||||
|
||||
*Sprint 15 target* – canonically describe the Notify data shapes that UI, workers, and storage consume. JSON Schemas live under `docs/notify/schemas/` and deterministic fixtures under `docs/notify/samples/`.
|
||||
*Sprint 15 target* – canonically describe the Notify data shapes that UI, workers, and storage consume. JSON Schemas live under `docs/modules/notify/resources/schemas/` and deterministic fixtures under `docs/modules/notify/resources/samples/`.
|
||||
|
||||
| Artifact | Schema | Sample |
|
||||
|----------|--------|--------|
|
||||
| **Rule** (catalogued routing logic) | `docs/notify/schemas/notify-rule@1.json` | `docs/notify/samples/notify-rule@1.sample.json` |
|
||||
| **Channel** (delivery endpoint definition) | `docs/notify/schemas/notify-channel@1.json` | `docs/notify/samples/notify-channel@1.sample.json` |
|
||||
| **Template** (rendering payload) | `docs/notify/schemas/notify-template@1.json` | `docs/notify/samples/notify-template@1.sample.json` |
|
||||
| **Event envelope** (Notify ingest surface) | `docs/notify/schemas/notify-event@1.json` | `docs/notify/samples/notify-event@1.sample.json` |
|
||||
| **Rule** (catalogued routing logic) | `docs/modules/notify/resources/schemas/notify-rule@1.json` | `docs/modules/notify/resources/samples/notify-rule@1.sample.json` |
|
||||
| **Channel** (delivery endpoint definition) | `docs/modules/notify/resources/schemas/notify-channel@1.json` | `docs/modules/notify/resources/samples/notify-channel@1.sample.json` |
|
||||
| **Template** (rendering payload) | `docs/modules/notify/resources/schemas/notify-template@1.json` | `docs/modules/notify/resources/samples/notify-template@1.sample.json` |
|
||||
| **Event envelope** (Notify ingest surface) | `docs/modules/notify/resources/schemas/notify-event@1.json` | `docs/modules/notify/resources/samples/notify-event@1.sample.json` |
|
||||
|
||||
### 6.1 Rule highlights (`notify-rule@1`)
|
||||
|
||||
@@ -498,12 +498,12 @@ Planned for Q1‑2026 (kept here for early plug‑in authors).
|
||||
|
||||
```bash
|
||||
# Validate Notify schemas and samples (matches Docs CI)
|
||||
for schema in docs/notify/schemas/*.json; do
|
||||
for schema in docs/modules/notify/resources/schemas/*.json; do
|
||||
npx ajv compile -c ajv-formats -s "$schema"
|
||||
done
|
||||
|
||||
for sample in docs/notify/samples/*.sample.json; do
|
||||
schema="docs/notify/schemas/$(basename "${sample%.sample.json}").json"
|
||||
for sample in docs/modules/notify/resources/samples/*.sample.json; do
|
||||
schema="docs/modules/notify/resources/schemas/$(basename "${sample%.sample.json}").json"
|
||||
npx ajv validate -c ajv-formats -s "$schema" -d "$sample"
|
||||
done
|
||||
```
|
||||
|
||||
@@ -145,7 +145,7 @@ P99 = 48 ms. Meets 50 ms gate.
|
||||
|
||||
|
||||
|
||||
> **Grafana/Alerting** – Import `docs/ops/scanner-analyzers-grafana-dashboard.json` and point it at the Prometheus datasource storing `scanner_analyzer_bench_*` metrics. Configure an alert on `scanner_analyzer_bench_regression_ratio` ≥ 1.20 (default limit); the bundled Stat panel surfaces breached scenarios (non-zero values). On-call runbook: `docs/ops/scanner-analyzers-operations.md`.
|
||||
> **Grafana/Alerting** – Import `docs/modules/scanner/operations/analyzers-grafana-dashboard.json` and point it at the Prometheus datasource storing `scanner_analyzer_bench_*` metrics. Configure an alert on `scanner_analyzer_bench_regression_ratio` ≥ 1.20 (default limit); the bundled Stat panel surfaces breached scenarios (non-zero values). On-call runbook: `docs/modules/scanner/operations/analyzers.md`.
|
||||
|
||||
_Plot generated weekly by `scripts/update‑trend.py`; shows last 12 weeks P95 per phase._
|
||||
|
||||
|
||||
@@ -1,62 +1,62 @@
|
||||
# 13 · Release Engineering Playbook — Stella Ops
|
||||
|
||||
|
||||
A concise, automation‑first guide describing **how source code on `main` becomes a verifiably signed, air‑gap‑friendly release**.
|
||||
It is opinionated for offline use‑cases and supply‑chain security (SLSA ≥ level 2 today, aiming for level 3).
|
||||
|
||||
---
|
||||
|
||||
## 0 Release Philosophy
|
||||
|
||||
* **Fast but fearless** – every commit on `main` must be releasable; broken builds break the build, not the team.
|
||||
* **Reproducible** – anyone can rebuild byte‑identical artefacts with a single `make release` offline.
|
||||
* **Secure by default** – every artefact ships with a SBOM, Cosign signature and (future) Rekor log entry.
|
||||
* **Offline‑first** – all dependencies are vendored or mirrored into the internal registry; no Internet required at runtime.
|
||||
|
||||
---
|
||||
|
||||
## 1 Versioning & Branching
|
||||
|
||||
| Branch | Purpose | Auto‑publish? |
|
||||
| ------------- | ------------------------------ | --------------------------------------- |
|
||||
| `main` | Always‑green development trunk | `nightly-*` images |
|
||||
| `release/X.Y` | Stabilise a minor line | `stella:X.Y-rcN` |
|
||||
| Tags | `X.Y.Z` = SemVer | `stella:X.Y.Z`, OUK tarball, Helm chart |
|
||||
|
||||
* **SemVer** – MAJOR for breaking API/CLI changes, MINOR for features, PATCH for fixes.
|
||||
* Release tags are **signed** (`git tag -s`) with the Stella Ops GPG key (`0x90C4…`).
|
||||
|
||||
---
|
||||
|
||||
## 2 CI/CD Overview (GitLab CI + GitLab Runner)
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[push / MR] --> Lint
|
||||
Lint --> Unit
|
||||
Unit --> Build
|
||||
Build --> Test-Container
|
||||
Test-Container --> SBOM
|
||||
SBOM --> Sign
|
||||
Sign --> Publish
|
||||
Publish --> E2E
|
||||
Publish --> Notify
|
||||
```
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
| Stage | Key tasks |
|
||||
| ------------------ | ------------------------------------------------------------------------------------------------ |
|
||||
| **Lint** | ESLint, golangci‑lint, hadolint, markdown‑lint. |
|
||||
| **Unit** | `dotnet test`, `go test`, Jest UI tests. |
|
||||
| **Quota unit‑tests 🏷** | Validate QuotaService logic: reset at UTC, 5 s vs 60 s waits, header correctness. |
|
||||
| **Build** | Multi‑arch container build (`linux/amd64`, `linux/arm64`) using **BuildKit** + `--provenance` 📌. |
|
||||
| **Test‑Container** | Spin up compose file, run smoke APIs. |
|
||||
| **SBOM** 📌 | Invoke **StellaOps.SBOMBuilder** to generate SPDX JSON + attach `.sbom` label to image. |
|
||||
| **Sign** | Sign image with **Cosign** (`cosign sign --key cosign.key`). |
|
||||
| **Publish** | Push to `registry.git.stella-ops.org`. |
|
||||
| **E2E** | Kind‑based Kubernetes test incl. Zastava DaemonSet; verify sub‑5 s scan SLA. |
|
||||
| **Notify** | Report to Mattermost & GitLab Slack app. |
|
||||
# 13 · Release Engineering Playbook — Stella Ops
|
||||
|
||||
|
||||
A concise, automation‑first guide describing **how source code on `main` becomes a verifiably signed, air‑gap‑friendly release**.
|
||||
It is opinionated for offline use‑cases and supply‑chain security (SLSA ≥ level 2 today, aiming for level 3).
|
||||
|
||||
---
|
||||
|
||||
## 0 Release Philosophy
|
||||
|
||||
* **Fast but fearless** – every commit on `main` must be releasable; broken builds break the build, not the team.
|
||||
* **Reproducible** – anyone can rebuild byte‑identical artefacts with a single `make release` offline.
|
||||
* **Secure by default** – every artefact ships with a SBOM, Cosign signature and (future) Rekor log entry.
|
||||
* **Offline‑first** – all dependencies are vendored or mirrored into the internal registry; no Internet required at runtime.
|
||||
|
||||
---
|
||||
|
||||
## 1 Versioning & Branching
|
||||
|
||||
| Branch | Purpose | Auto‑publish? |
|
||||
| ------------- | ------------------------------ | --------------------------------------- |
|
||||
| `main` | Always‑green development trunk | `nightly-*` images |
|
||||
| `release/X.Y` | Stabilise a minor line | `stella:X.Y-rcN` |
|
||||
| Tags | `X.Y.Z` = SemVer | `stella:X.Y.Z`, OUK tarball, Helm chart |
|
||||
|
||||
* **SemVer** – MAJOR for breaking API/CLI changes, MINOR for features, PATCH for fixes.
|
||||
* Release tags are **signed** (`git tag -s`) with the Stella Ops GPG key (`0x90C4…`).
|
||||
|
||||
---
|
||||
|
||||
## 2 CI/CD Overview (GitLab CI + GitLab Runner)
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[push / MR] --> Lint
|
||||
Lint --> Unit
|
||||
Unit --> Build
|
||||
Build --> Test-Container
|
||||
Test-Container --> SBOM
|
||||
SBOM --> Sign
|
||||
Sign --> Publish
|
||||
Publish --> E2E
|
||||
Publish --> Notify
|
||||
```
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
| Stage | Key tasks |
|
||||
| ------------------ | ------------------------------------------------------------------------------------------------ |
|
||||
| **Lint** | ESLint, golangci‑lint, hadolint, markdown‑lint. |
|
||||
| **Unit** | `dotnet test`, `go test`, Jest UI tests. |
|
||||
| **Quota unit‑tests 🏷** | Validate QuotaService logic: reset at UTC, 5 s vs 60 s waits, header correctness. |
|
||||
| **Build** | Multi‑arch container build (`linux/amd64`, `linux/arm64`) using **BuildKit** + `--provenance` 📌. |
|
||||
| **Test‑Container** | Spin up compose file, run smoke APIs. |
|
||||
| **SBOM** 📌 | Invoke **StellaOps.SBOMBuilder** to generate SPDX JSON + attach `.sbom` label to image. |
|
||||
| **Sign** | Sign image with **Cosign** (`cosign sign --key cosign.key`). |
|
||||
| **Publish** | Push to `registry.git.stella-ops.org`. |
|
||||
| **E2E** | Kind‑based Kubernetes test incl. Zastava DaemonSet; verify sub‑5 s scan SLA. |
|
||||
| **Notify** | Report to Mattermost & GitLab Slack app. |
|
||||
| **OfflineToken** | Call `JwtIssuer.Generate(exp=30d)` → store `client.jwt` artefact → attach to OUK build context |
|
||||
|
||||
*All stages run in parallel where possible; max wall‑time < 15 min.*
|
||||
@@ -70,63 +70,63 @@ The `build-test-deploy` workflow also runs
|
||||
regressions fail fast during every CI pass.
|
||||
|
||||
---
|
||||
|
||||
## 3 Container Image Strategy
|
||||
|
||||
| Image | Registry Tag | Contents |
|
||||
| ------------------------------ | --------------------------- | ---------------------------------------------------------------------- |
|
||||
| **backend** | `stella/backend:{ver}` | ASP.NET API, plugin loader. |
|
||||
| **ui** | `stella/ui:{ver}` | Pre‑built Angular SPA. |
|
||||
| **runner-trivy** | `stella/runner-trivy:{ver}` | Trivy CLI + SPDX/CycloneDX 🛠. |
|
||||
| **runner-grype** | `stella/runner-grype:{ver}` | Optional plug‑in scanner. |
|
||||
| **🏷️ StellaOps.Registry** 📌 | `stella/registry:{ver}` | Scratch image embedding Docker Registry v2 + Cosign policy controller. |
|
||||
| **🏷️ StellaOps.MutePolicies** 📌 | `stella/policies:{ver}` | Sidecar serving policy bundles. |
|
||||
| **🏷️ StellaOps.Attestor** 📌 | `stella/attestor:{ver}` | SLSA provenance & Rekor signer (future). |
|
||||
|
||||
*Images are **`--label org.opencontainers.image.source=git.stella-ops.ru`** and include SBOMs generated at build time.*
|
||||
|
||||
---
|
||||
|
||||
## 4 📌 Offline Update Kit (OUK) Build & Distribution
|
||||
|
||||
**Purpose** – deliver updated CVE feeds & Trivy DB to air‑gapped clusters.
|
||||
|
||||
### 4.1 CLI Tool
|
||||
|
||||
*Go binary `ouk` lives in `tools/ouk/`.*
|
||||
|
||||
```sh
|
||||
ouk fetch \
|
||||
--nvd --osv \
|
||||
--trivy-db --date $(date -I) \
|
||||
--output ouk-$(date +%Y%m%d).tar.gz \
|
||||
--sign cosign.key
|
||||
```
|
||||
|
||||
|
||||
## 3 Container Image Strategy
|
||||
|
||||
| Image | Registry Tag | Contents |
|
||||
| ------------------------------ | --------------------------- | ---------------------------------------------------------------------- |
|
||||
| **backend** | `stella/backend:{ver}` | ASP.NET API, plugin loader. |
|
||||
| **ui** | `stella/ui:{ver}` | Pre‑built Angular SPA. |
|
||||
| **runner-trivy** | `stella/runner-trivy:{ver}` | Trivy CLI + SPDX/CycloneDX 🛠. |
|
||||
| **runner-grype** | `stella/runner-grype:{ver}` | Optional plug‑in scanner. |
|
||||
| **🏷️ StellaOps.Registry** 📌 | `stella/registry:{ver}` | Scratch image embedding Docker Registry v2 + Cosign policy controller. |
|
||||
| **🏷️ StellaOps.MutePolicies** 📌 | `stella/policies:{ver}` | Sidecar serving policy bundles. |
|
||||
| **🏷️ StellaOps.Attestor** 📌 | `stella/attestor:{ver}` | SLSA provenance & Rekor signer (future). |
|
||||
|
||||
*Images are **`--label org.opencontainers.image.source=git.stella-ops.ru`** and include SBOMs generated at build time.*
|
||||
|
||||
---
|
||||
|
||||
## 4 📌 Offline Update Kit (OUK) Build & Distribution
|
||||
|
||||
**Purpose** – deliver updated CVE feeds & Trivy DB to air‑gapped clusters.
|
||||
|
||||
### 4.1 CLI Tool
|
||||
|
||||
*Go binary `ouk` lives in `src/Tools/ouk/`.*
|
||||
|
||||
```sh
|
||||
ouk fetch \
|
||||
--nvd --osv \
|
||||
--trivy-db --date $(date -I) \
|
||||
--output ouk-$(date +%Y%m%d).tar.gz \
|
||||
--sign cosign.key
|
||||
```
|
||||
|
||||
### 4.2 Pipeline Hook
|
||||
|
||||
* Runs on **first Friday** each month (cron).
|
||||
* Generates tarball, signs it, uploads to **GitLab Release asset**.
|
||||
* SHA‑256 + signature published alongside.
|
||||
* Release job must emit `out/release/debug/` with `debug-manifest.json` and `.sha256` so `ops/offline-kit/mirror_debug_store.py` can mirror symbols into the Offline Kit (see `DEVOPS-REL-17-004`).
|
||||
|
||||
### 4.3 Activation Flow (runtime)
|
||||
|
||||
1. Admin uploads `.tar.gz` via **UI → Settings → Offline Updates (OUK)**.
|
||||
2. Backend verifies Cosign signature & digest.
|
||||
3. Files extracted into `var/lib/stella/db`.
|
||||
4. Redis caches invalidated; Dashboard “Feed Age” ticks green.
|
||||
5. Audit event `ouk_update` stored.
|
||||
|
||||
### 4.4 Token Detail
|
||||
|
||||
client.jwt placed under /root/ inside the tarball.
|
||||
CI job fails if token expiry < 29 days (guard against stale caches).
|
||||
|
||||
---
|
||||
|
||||
## 5 Artifact Signing & Transparency
|
||||
|
||||
|
||||
### 4.3 Activation Flow (runtime)
|
||||
|
||||
1. Admin uploads `.tar.gz` via **UI → Settings → Offline Updates (OUK)**.
|
||||
2. Backend verifies Cosign signature & digest.
|
||||
3. Files extracted into `var/lib/stella/db`.
|
||||
4. Redis caches invalidated; Dashboard “Feed Age” ticks green.
|
||||
5. Audit event `ouk_update` stored.
|
||||
|
||||
### 4.4 Token Detail
|
||||
|
||||
client.jwt placed under /root/ inside the tarball.
|
||||
CI job fails if token expiry < 29 days (guard against stale caches).
|
||||
|
||||
---
|
||||
|
||||
## 5 Artifact Signing & Transparency
|
||||
|
||||
| Artefact | Signer | Tool/Notes |
|
||||
| ------------ | --------------- | ---------------------------------- |
|
||||
| Git tags | GPG (`0x90C4…`) | `git tag -s` |
|
||||
@@ -134,18 +134,18 @@ CI job fails if token expiry < 29 days (guard against stale caches).
|
||||
| Helm Charts | prov file | `helm package --sign` |
|
||||
| OUK tarballs | Cosign | `cosign sign-blob` |
|
||||
| Debug store | — | `debug/debug-manifest.json` hashed |
|
||||
|
||||
**Rekor** integration is **TODO** – once the internal Rekor mirror is online (`StellaOpsAttestor`) a post‑publish job will submit transparency log entries.
|
||||
|
||||
---
|
||||
|
||||
## 6 Release Checklist
|
||||
|
||||
1. CI pipeline green.
|
||||
2. Bump `VERSION` file.
|
||||
3. Tag `git tag -s X.Y.Z -m "Release X.Y.Z"` & push.
|
||||
4. GitLab CI auto‑publishes images & charts.
|
||||
5. Draft GitLab **Release Notes** using `tools/release-notes-gen`.
|
||||
|
||||
**Rekor** integration is **TODO** – once the internal Rekor mirror is online (`StellaOpsAttestor`) a post‑publish job will submit transparency log entries.
|
||||
|
||||
---
|
||||
|
||||
## 6 Release Checklist
|
||||
|
||||
1. CI pipeline green.
|
||||
2. Bump `VERSION` file.
|
||||
3. Tag `git tag -s X.Y.Z -m "Release X.Y.Z"` & push.
|
||||
4. GitLab CI auto‑publishes images & charts.
|
||||
5. Draft GitLab **Release Notes** using `src/Tools/release-notes-gen`.
|
||||
6. Verify SBOM attachment with `stella sbom verify stella/backend:X.Y.Z`.
|
||||
7. Run the release verifier locally if CI isn’t available (mirrors the workflow step):
|
||||
`python ops/devops/release/test_verify_release.py`
|
||||
@@ -160,71 +160,71 @@ CI job fails if token expiry < 29 days (guard against stale caches).
|
||||
Validate that the hash from `readelf` matches the `.build-id/<aa>/<rest>.debug` path created by the script.
|
||||
9. Smoke-test OUK tarball in offline lab.
|
||||
10. Announce in `#stella-release` Mattermost channel.
|
||||
|
||||
---
|
||||
|
||||
## 7 Hot‑fix Procedure
|
||||
|
||||
* Branch from latest tag → `hotfix/X.Y.Z+1-hf1`.
|
||||
* Apply minimal patch, add regression test.
|
||||
* CI pipeline (with reduced stages) must pass.
|
||||
* Tag `X.Y.Z+1`.
|
||||
* Publish only container + Helm chart; OUK not rebuilt.
|
||||
* Cherry‑pick back to `main`.
|
||||
|
||||
---
|
||||
|
||||
## 8 Deprecation & End‑of‑Life Policy
|
||||
|
||||
| Feature | Deprecation notice | Removal earliest |
|
||||
| ------------------------ | ------------------ | ---------------- |
|
||||
| Legacy CSV policy import | 2025‑10‑01 | 2026‑04‑01 |
|
||||
| Docker v1 Registry auth | 2025‑12‑01 | 2026‑06‑01 |
|
||||
| In‑image Trivy DB | 2025‑12‑15 | 2026‑03‑15 |
|
||||
|
||||
*At least 6 months notice; removal requires major version bump.*
|
||||
|
||||
---
|
||||
|
||||
## 9 📌 Non‑Commercial Usage Rules (English canonical)
|
||||
|
||||
1. **Free for internal security assessments** (company or personal).
|
||||
2. **SaaS resale / re‑hosting prohibited** without prior written consent (AGPL §13).
|
||||
3. If you distribute a fork with UI or backend modifications **you must**:
|
||||
* Publish the complete modified source code.
|
||||
* Retain the original Stella Ops attribution in UI footer and CLI `--version`.
|
||||
4. All third‑party dependencies remain under their respective licences (MIT, Apache‑2.0, ISC, BSD).
|
||||
5. Deployments in state‑regulated or classified environments must obey**applicable local regulations** governing cryptography and software distribution.
|
||||
|
||||
---
|
||||
|
||||
## 10 Best Practices Snapshot 📌
|
||||
|
||||
* **SBOM‑per‑image** → attach at build time; store as OCI artifact for supply‑chain introspection.
|
||||
* **Provenance flag** (`--provenance=true`) in BuildKit fulfils SLSA 2 requirement.
|
||||
* Use **multi‑arch, reproducible builds** (`SOURCE_DATE_EPOCH` pins timestamps).
|
||||
* All pipelines enforce **Signed‑off‑by (DCO)**; CI fails if trailer missing.
|
||||
* `cosign policy` ensures only images signed by the project key run in production.
|
||||
|
||||
---
|
||||
|
||||
## 11 Contributing to Release Engineering
|
||||
|
||||
* Fork & create MR to `infra/release-*`.
|
||||
* All infra changes require green **`integration-e2e-offline`** job.
|
||||
* Discuss larger infra migrations in `#sig-release` Mattermost; decisions recorded in `ADR/` folder.
|
||||
|
||||
---
|
||||
|
||||
## 12 Change Log (high‑level)
|
||||
|
||||
| Version | Date | Note |
|
||||
| ------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.1 | 2025‑07‑15 | Added OUK build/publish pipeline, internal registry image (`StellaOps.Registry`), non‑commercial usage rules extraction, SBOM stage, BuildKit provenance. |
|
||||
| v2.0 | 2025‑07‑12 | Initial open‑sourcing of Release Engineering guide. |
|
||||
| v1.1 | 2025‑07‑09 | Fixed inner fencing; added retention policy |
|
||||
| v1.0 | 2025‑07‑09 | Initial playbook |
|
||||
|
||||
---
|
||||
|
||||
*(End of Release Engineering Playbook v1.1)*
|
||||
|
||||
---
|
||||
|
||||
## 7 Hot‑fix Procedure
|
||||
|
||||
* Branch from latest tag → `hotfix/X.Y.Z+1-hf1`.
|
||||
* Apply minimal patch, add regression test.
|
||||
* CI pipeline (with reduced stages) must pass.
|
||||
* Tag `X.Y.Z+1`.
|
||||
* Publish only container + Helm chart; OUK not rebuilt.
|
||||
* Cherry‑pick back to `main`.
|
||||
|
||||
---
|
||||
|
||||
## 8 Deprecation & End‑of‑Life Policy
|
||||
|
||||
| Feature | Deprecation notice | Removal earliest |
|
||||
| ------------------------ | ------------------ | ---------------- |
|
||||
| Legacy CSV policy import | 2025‑10‑01 | 2026‑04‑01 |
|
||||
| Docker v1 Registry auth | 2025‑12‑01 | 2026‑06‑01 |
|
||||
| In‑image Trivy DB | 2025‑12‑15 | 2026‑03‑15 |
|
||||
|
||||
*At least 6 months notice; removal requires major version bump.*
|
||||
|
||||
---
|
||||
|
||||
## 9 📌 Non‑Commercial Usage Rules (English canonical)
|
||||
|
||||
1. **Free for internal security assessments** (company or personal).
|
||||
2. **SaaS resale / re‑hosting prohibited** without prior written consent (AGPL §13).
|
||||
3. If you distribute a fork with UI or backend modifications **you must**:
|
||||
* Publish the complete modified source code.
|
||||
* Retain the original Stella Ops attribution in UI footer and CLI `--version`.
|
||||
4. All third‑party dependencies remain under their respective licences (MIT, Apache‑2.0, ISC, BSD).
|
||||
5. Deployments in state‑regulated or classified environments must obey**applicable local regulations** governing cryptography and software distribution.
|
||||
|
||||
---
|
||||
|
||||
## 10 Best Practices Snapshot 📌
|
||||
|
||||
* **SBOM‑per‑image** → attach at build time; store as OCI artifact for supply‑chain introspection.
|
||||
* **Provenance flag** (`--provenance=true`) in BuildKit fulfils SLSA 2 requirement.
|
||||
* Use **multi‑arch, reproducible builds** (`SOURCE_DATE_EPOCH` pins timestamps).
|
||||
* All pipelines enforce **Signed‑off‑by (DCO)**; CI fails if trailer missing.
|
||||
* `cosign policy` ensures only images signed by the project key run in production.
|
||||
|
||||
---
|
||||
|
||||
## 11 Contributing to Release Engineering
|
||||
|
||||
* Fork & create MR to `infra/release-*`.
|
||||
* All infra changes require green **`integration-e2e-offline`** job.
|
||||
* Discuss larger infra migrations in `#sig-release` Mattermost; decisions recorded in `ADR/` folder.
|
||||
|
||||
---
|
||||
|
||||
## 12 Change Log (high‑level)
|
||||
|
||||
| Version | Date | Note |
|
||||
| ------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.1 | 2025‑07‑15 | Added OUK build/publish pipeline, internal registry image (`StellaOps.Registry`), non‑commercial usage rules extraction, SBOM stage, BuildKit provenance. |
|
||||
| v2.0 | 2025‑07‑12 | Initial open‑sourcing of Release Engineering guide. |
|
||||
| v1.1 | 2025‑07‑09 | Fixed inner fencing; added retention policy |
|
||||
| v1.0 | 2025‑07‑09 | Initial playbook |
|
||||
|
||||
---
|
||||
|
||||
*(End of Release Engineering Playbook v1.1)*
|
||||
|
||||
@@ -1,145 +1,145 @@
|
||||
# 17 · Security Hardening Guide — **Stella Ops**
|
||||
*(v2.0 — 12 Jul 2025)*
|
||||
|
||||
> **Audience** — Site‑reliability and platform teams deploying **the open‑source Core** in production or restricted networks.
|
||||
---
|
||||
|
||||
## 0 Table of Contents
|
||||
|
||||
1. Threat model (summary)
|
||||
2. Host‑OS baseline
|
||||
3. Container & runtime hardening
|
||||
4. Network‑plane guidance
|
||||
5. Secrets & key management
|
||||
6. Image, SBOM & plug‑in supply‑chain controls
|
||||
7. Logging, monitoring & audit
|
||||
8. Update & patch strategy
|
||||
9. Incident‑response workflow
|
||||
10. Pen‑testing & continuous assurance
|
||||
11. Contacts & vulnerability disclosure
|
||||
12. Change log
|
||||
|
||||
---
|
||||
|
||||
## 1 Threat model (summary)
|
||||
|
||||
| Asset | Threats | Mitigations |
|
||||
| -------------------- | --------------------- | ---------------------------------------------------------------------- |
|
||||
| SBOMs & scan results | Disclosure, tamper | TLS‑in‑transit, read‑only Redis volume, RBAC, Cosign‑verified plug‑ins |
|
||||
| Backend container | RCE, code‑injection | Distroless image, non‑root UID, read‑only FS, seccomp + `CAP_DROP:ALL` |
|
||||
| Update artefacts | Supply‑chain attack | Cosign‑signed images & SBOMs, enforced by admission controller |
|
||||
| Admin credentials | Phishing, brute force | OAuth 2.0 with 12‑h token TTL, optional mTLS |
|
||||
|
||||
---
|
||||
|
||||
## 2 Host‑OS baseline checklist
|
||||
|
||||
| Item | Recommended setting |
|
||||
| ------------- | --------------------------------------------------------- |
|
||||
| OS | Ubuntu 22.04 LTS (kernel ≥ 5.15) or Alma 9 |
|
||||
| Patches | `unattended‑upgrades` or vendor‑equivalent enabled |
|
||||
| Filesystem | `noexec,nosuid` on `/tmp`, `/var/tmp` |
|
||||
| Docker Engine | v24.*, API socket root‑owned (`0660`) |
|
||||
| Auditd | Watch `/etc/docker`, `/usr/bin/docker*` and Compose files |
|
||||
| Time sync | `chrony` or `systemd‑timesyncd` |
|
||||
|
||||
---
|
||||
|
||||
## 3 Container & runtime hardening
|
||||
|
||||
### 3.1 Docker Compose reference (`compose-core.yml`)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
backend:
|
||||
image: registry.stella-ops.org/stella-ops/stella-ops:<PINNED_TAG_OR_DIGEST>
|
||||
user: "101:101" # non‑root
|
||||
read_only: true
|
||||
security_opt:
|
||||
- "no-new-privileges:true"
|
||||
- "seccomp:./seccomp-backend.json"
|
||||
cap_drop: [ALL]
|
||||
tmpfs:
|
||||
- /tmp:size=64m,exec,nosymlink
|
||||
environment:
|
||||
- ASPNETCORE_URLS=https://+:8080
|
||||
- TLSPROVIDER=OpenSslGost
|
||||
depends_on: [redis]
|
||||
networks: [core-net]
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "https://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
redis:
|
||||
image: redis:7.2-alpine
|
||||
command: ["redis-server", "--requirepass", "${REDIS_PASS}", "--rename-command", "FLUSHALL", ""]
|
||||
user: "redis"
|
||||
read_only: true
|
||||
cap_drop: [ALL]
|
||||
tmpfs:
|
||||
- /data
|
||||
networks: [core-net]
|
||||
|
||||
networks:
|
||||
core-net:
|
||||
driver: bridge
|
||||
```
|
||||
|
||||
No dedicated “Redis” or “Mongo” sub‑nets are declared; the single bridge network suffices for the default stack.
|
||||
|
||||
### 3.2 Kubernetes deployment highlights
|
||||
|
||||
Use a separate NetworkPolicy that only allows egress from backend to Redis :6379.
|
||||
securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, drop all capabilities.
|
||||
PodDisruptionBudget of minAvailable: 1.
|
||||
Optionally add CosignVerified=true label enforced by an admission controller (e.g. Kyverno or Connaisseur).
|
||||
|
||||
## 4 Network‑plane guidance
|
||||
|
||||
| Plane | Recommendation |
|
||||
| ------------------ | -------------------------------------------------------------------------- |
|
||||
| North‑south | Terminate TLS 1.2+ (OpenSSL‑GOST default). Use LetsEncrypt or internal CA. |
|
||||
| East‑west | Compose bridge or K8s ClusterIP only; no public Redis/Mongo ports. |
|
||||
| Ingress controller | Limit methods to GET, POST, PATCH (no TRACE). |
|
||||
| Rate‑limits | 40 rps default; tune ScannerPool.Workers and ingress limit‑req to match. |
|
||||
|
||||
## 5 Secrets & key management
|
||||
|
||||
| Secret | Storage | Rotation |
|
||||
| --------------------------------- | ---------------------------------- | ----------------------------- |
|
||||
| **Client‑JWT (offline)** | `/var/lib/stella/tokens/client.jwt` (root : 600) | **30 days** – provided by each OUK |
|
||||
| REDIS_PASS | Docker/K8s secret | 90 days |
|
||||
| OAuth signing key | /keys/jwt.pem (read‑only mount) | 180 days |
|
||||
| Cosign public key | /keys/cosign.pub baked into image; | change on every major release |
|
||||
| Trivy DB mirror token (if remote) | Secret + read‑only | 30 days |
|
||||
|
||||
Never bake secrets into images; always inject at runtime.
|
||||
|
||||
> **Operational tip:** schedule a cron reminding ops 5 days before
|
||||
> `client.jwt` expiry. The backend also emits a Prometheus metric
|
||||
> `stella_quota_token_days_remaining`.
|
||||
|
||||
## 6 Image, SBOM & plug‑in supply‑chain controls
|
||||
|
||||
* Images — Pull by digest not latest; verify:
|
||||
|
||||
```bash
|
||||
cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
|
||||
--key https://stella-ops.org/keys/cosign.pub
|
||||
```
|
||||
|
||||
* SBOM — Each release ships an SPDX file; store alongside images for audit.
|
||||
* Third‑party plug‑ins — Place in /plugins/; backend will:
|
||||
* Validate Cosign signature.
|
||||
* Check [StellaPluginVersion("major.minor")].
|
||||
* Refuse to start if Security.DisablePluginUnsigned=false (default).
|
||||
|
||||
## 7 Logging, monitoring & audit
|
||||
|
||||
| Control | Implementation |
|
||||
| ------------ | ----------------------------------------------------------------- |
|
||||
# 17 · Security Hardening Guide — **Stella Ops**
|
||||
*(v2.0 — 12 Jul 2025)*
|
||||
|
||||
> **Audience** — Site‑reliability and platform teams deploying **the open‑source Core** in production or restricted networks.
|
||||
---
|
||||
|
||||
## 0 Table of Contents
|
||||
|
||||
1. Threat model (summary)
|
||||
2. Host‑OS baseline
|
||||
3. Container & runtime hardening
|
||||
4. Network‑plane guidance
|
||||
5. Secrets & key management
|
||||
6. Image, SBOM & plug‑in supply‑chain controls
|
||||
7. Logging, monitoring & audit
|
||||
8. Update & patch strategy
|
||||
9. Incident‑response workflow
|
||||
10. Pen‑testing & continuous assurance
|
||||
11. Contacts & vulnerability disclosure
|
||||
12. Change log
|
||||
|
||||
---
|
||||
|
||||
## 1 Threat model (summary)
|
||||
|
||||
| Asset | Threats | Mitigations |
|
||||
| -------------------- | --------------------- | ---------------------------------------------------------------------- |
|
||||
| SBOMs & scan results | Disclosure, tamper | TLS‑in‑transit, read‑only Redis volume, RBAC, Cosign‑verified plug‑ins |
|
||||
| Backend container | RCE, code‑injection | Distroless image, non‑root UID, read‑only FS, seccomp + `CAP_DROP:ALL` |
|
||||
| Update artefacts | Supply‑chain attack | Cosign‑signed images & SBOMs, enforced by admission controller |
|
||||
| Admin credentials | Phishing, brute force | OAuth 2.0 with 12‑h token TTL, optional mTLS |
|
||||
|
||||
---
|
||||
|
||||
## 2 Host‑OS baseline checklist
|
||||
|
||||
| Item | Recommended setting |
|
||||
| ------------- | --------------------------------------------------------- |
|
||||
| OS | Ubuntu 22.04 LTS (kernel ≥ 5.15) or Alma 9 |
|
||||
| Patches | `unattended‑upgrades` or vendor‑equivalent enabled |
|
||||
| Filesystem | `noexec,nosuid` on `/tmp`, `/var/tmp` |
|
||||
| Docker Engine | v24.*, API socket root‑owned (`0660`) |
|
||||
| Auditd | Watch `/etc/docker`, `/usr/bin/docker*` and Compose files |
|
||||
| Time sync | `chrony` or `systemd‑timesyncd` |
|
||||
|
||||
---
|
||||
|
||||
## 3 Container & runtime hardening
|
||||
|
||||
### 3.1 Docker Compose reference (`compose-core.yml`)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
backend:
|
||||
image: registry.stella-ops.org/stella-ops/stella-ops:<PINNED_TAG_OR_DIGEST>
|
||||
user: "101:101" # non‑root
|
||||
read_only: true
|
||||
security_opt:
|
||||
- "no-new-privileges:true"
|
||||
- "seccomp:./seccomp-backend.json"
|
||||
cap_drop: [ALL]
|
||||
tmpfs:
|
||||
- /tmp:size=64m,exec,nosymlink
|
||||
environment:
|
||||
- ASPNETCORE_URLS=https://+:8080
|
||||
- TLSPROVIDER=OpenSslGost
|
||||
depends_on: [redis]
|
||||
networks: [core-net]
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "https://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
redis:
|
||||
image: redis:7.2-alpine
|
||||
command: ["redis-server", "--requirepass", "${REDIS_PASS}", "--rename-command", "FLUSHALL", ""]
|
||||
user: "redis"
|
||||
read_only: true
|
||||
cap_drop: [ALL]
|
||||
tmpfs:
|
||||
- /data
|
||||
networks: [core-net]
|
||||
|
||||
networks:
|
||||
core-net:
|
||||
driver: bridge
|
||||
```
|
||||
|
||||
No dedicated “Redis” or “Mongo” sub‑nets are declared; the single bridge network suffices for the default stack.
|
||||
|
||||
### 3.2 Kubernetes deployment highlights
|
||||
|
||||
Use a separate NetworkPolicy that only allows egress from backend to Redis :6379.
|
||||
securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, drop all capabilities.
|
||||
PodDisruptionBudget of minAvailable: 1.
|
||||
Optionally add CosignVerified=true label enforced by an admission controller (e.g. Kyverno or Connaisseur).
|
||||
|
||||
## 4 Network‑plane guidance
|
||||
|
||||
| Plane | Recommendation |
|
||||
| ------------------ | -------------------------------------------------------------------------- |
|
||||
| North‑south | Terminate TLS 1.2+ (OpenSSL‑GOST default). Use LetsEncrypt or internal CA. |
|
||||
| East‑west | Compose bridge or K8s ClusterIP only; no public Redis/Mongo ports. |
|
||||
| Ingress controller | Limit methods to GET, POST, PATCH (no TRACE). |
|
||||
| Rate‑limits | 40 rps default; tune ScannerPool.Workers and ingress limit‑req to match. |
|
||||
|
||||
## 5 Secrets & key management
|
||||
|
||||
| Secret | Storage | Rotation |
|
||||
| --------------------------------- | ---------------------------------- | ----------------------------- |
|
||||
| **Client‑JWT (offline)** | `/var/lib/stella/tokens/client.jwt` (root : 600) | **30 days** – provided by each OUK |
|
||||
| REDIS_PASS | Docker/K8s secret | 90 days |
|
||||
| OAuth signing key | /keys/jwt.pem (read‑only mount) | 180 days |
|
||||
| Cosign public key | /keys/cosign.pub baked into image; | change on every major release |
|
||||
| Trivy DB mirror token (if remote) | Secret + read‑only | 30 days |
|
||||
|
||||
Never bake secrets into images; always inject at runtime.
|
||||
|
||||
> **Operational tip:** schedule a cron reminding ops 5 days before
|
||||
> `client.jwt` expiry. The backend also emits a Prometheus metric
|
||||
> `stella_quota_token_days_remaining`.
|
||||
|
||||
## 6 Image, SBOM & plug‑in supply‑chain controls
|
||||
|
||||
* Images — Pull by digest not latest; verify:
|
||||
|
||||
```bash
|
||||
cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
|
||||
--key https://stella-ops.org/keys/cosign.pub
|
||||
```
|
||||
|
||||
* SBOM — Each release ships an SPDX file; store alongside images for audit.
|
||||
* Third‑party plug‑ins — Place in /plugins/; backend will:
|
||||
* Validate Cosign signature.
|
||||
* Check [StellaPluginVersion("major.minor")].
|
||||
* Refuse to start if Security.DisablePluginUnsigned=false (default).
|
||||
|
||||
## 7 Logging, monitoring & audit
|
||||
|
||||
| Control | Implementation |
|
||||
| ------------ | ----------------------------------------------------------------- |
|
||||
| Log format | Serilog JSON; ship via Fluent‑Bit to ELK or Loki |
|
||||
| Metrics | Prometheus /metrics endpoint; default Grafana dashboard in infra/ |
|
||||
| Audit events | Redis stream audit; export daily to SIEM |
|
||||
@@ -159,44 +159,44 @@ cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
|
||||
showing whether a network bypass CIDR allowed the request. Configure your SIEM
|
||||
to alert when unauthenticated requests (`status=401`) appear with
|
||||
`bypass=true`, or when unexpected scopes invoke job triggers.
|
||||
Detailed monitoring and response guidance lives in `docs/ops/concelier-authority-audit-runbook.md`.
|
||||
|
||||
## 8 Update & patch strategy
|
||||
|
||||
| Layer | Cadence | Method |
|
||||
| -------------------- | -------------------------------------------------------- | ------------------------------ |
|
||||
| Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d |
|
||||
Detailed monitoring and response guidance lives in `docs/modules/concelier/operations/authority-audit-runbook.md`.
|
||||
|
||||
## 8 Update & patch strategy
|
||||
|
||||
| Layer | Cadence | Method |
|
||||
| -------------------- | -------------------------------------------------------- | ------------------------------ |
|
||||
| Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d |
|
||||
| Trivy DB | 24 h scheduler via Concelier (vulnerability ingest/merge/export service) | configurable via Concelier scheduler options |
|
||||
| Docker Engine | vendor LTS | distro package manager |
|
||||
| Host OS | security repos enabled | unattended‑upgrades |
|
||||
|
||||
## 9 Incident‑response workflow
|
||||
|
||||
* Detect — PagerDuty alert from Prometheus or SIEM.
|
||||
* Contain — Stop affected Backend container; isolate Redis RDB snapshot.
|
||||
* Eradicate — Pull verified images, redeploy, rotate secrets.
|
||||
* Recover — Restore RDB, replay SBOMs if history lost.
|
||||
* Review — Post‑mortem within 72 h; create follow‑up issues.
|
||||
* Escalate P1 incidents to <security@stella‑ops.org> (24 × 7).
|
||||
|
||||
|
||||
## 10 Pen‑testing & continuous assurance
|
||||
|
||||
| Control | Frequency | Tool/Runner |
|
||||
|----------------------|-----------------------|-------------------------------------------|
|
||||
| OWASP ZAP baseline | Each merge to `main` | GitHub Action `zap-baseline-scan` |
|
||||
| Dependency scanning | Per pull request | Trivy FS + Dependabot |
|
||||
| External red‑team | Annual or pre‑GA | CREST‑accredited third‑party |
|
||||
|
||||
## 11 Vulnerability disclosure & contact
|
||||
|
||||
* Preferred channel: security@stella‑ops.org (GPG key on website).
|
||||
* Coordinated disclosure reward: public credit and swag (no monetary bounty at this time).
|
||||
|
||||
## 12 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.0 | 2025‑07‑12 | Full overhaul: host‑OS baseline, supply‑chain signing, removal of unnecessary sub‑nets, role‑based contact e‑mail, K8s guidance. |
|
||||
| v1.1 | 2025‑07‑09 | Minor fence fixes. |
|
||||
| v1.0 | 2025‑07‑09 | Original draft. |
|
||||
| Docker Engine | vendor LTS | distro package manager |
|
||||
| Host OS | security repos enabled | unattended‑upgrades |
|
||||
|
||||
## 9 Incident‑response workflow
|
||||
|
||||
* Detect — PagerDuty alert from Prometheus or SIEM.
|
||||
* Contain — Stop affected Backend container; isolate Redis RDB snapshot.
|
||||
* Eradicate — Pull verified images, redeploy, rotate secrets.
|
||||
* Recover — Restore RDB, replay SBOMs if history lost.
|
||||
* Review — Post‑mortem within 72 h; create follow‑up issues.
|
||||
* Escalate P1 incidents to <security@stella‑ops.org> (24 × 7).
|
||||
|
||||
|
||||
## 10 Pen‑testing & continuous assurance
|
||||
|
||||
| Control | Frequency | Tool/Runner |
|
||||
|----------------------|-----------------------|-------------------------------------------|
|
||||
| OWASP ZAP baseline | Each merge to `main` | GitHub Action `zap-baseline-scan` |
|
||||
| Dependency scanning | Per pull request | Trivy FS + Dependabot |
|
||||
| External red‑team | Annual or pre‑GA | CREST‑accredited third‑party |
|
||||
|
||||
## 11 Vulnerability disclosure & contact
|
||||
|
||||
* Preferred channel: security@stella‑ops.org (GPG key on website).
|
||||
* Coordinated disclosure reward: public credit and swag (no monetary bounty at this time).
|
||||
|
||||
## 12 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.0 | 2025‑07‑12 | Full overhaul: host‑OS baseline, supply‑chain signing, removal of unnecessary sub‑nets, role‑based contact e‑mail, K8s guidance. |
|
||||
| v1.1 | 2025‑07‑09 | Minor fence fixes. |
|
||||
| v1.0 | 2025‑07‑09 | Original draft. |
|
||||
|
||||
@@ -1,190 +1,190 @@
|
||||
# Stella Ops — Installation Guide (Docker & Air‑Gap)
|
||||
|
||||
<!--
|
||||
This file is processed by the Eleventy build.
|
||||
Do **not** hard‑code versions or quota numbers; inherit from
|
||||
docs/_includes/CONSTANTS.md instead.
|
||||
{{ dotnet }} → ".NET 10 LTS"
|
||||
{{ angular }} → "20"
|
||||
-->
|
||||
|
||||
> **Status — public α not yet published.**
|
||||
> The commands below will work as soon as the first image is tagged
|
||||
> `registry.stella-ops.org/stella-ops/stella-ops:0.1.0-alpha`
|
||||
> (target date: **late 2025**). Track progress on the
|
||||
> [road‑map](/roadmap/).
|
||||
|
||||
---
|
||||
|
||||
## 0 · Prerequisites
|
||||
|
||||
| Item | Minimum | Notes |
|
||||
|------|---------|-------|
|
||||
| Linux | Ubuntu 22.04 LTS / Alma 9 | x86‑64 or arm64 |
|
||||
| CPU / RAM | 2 vCPU / 2 GiB | Laptop baseline |
|
||||
| Disk | 10 GiB SSD | SBOM + vuln DB cache |
|
||||
| Docker | **Engine 25 + Compose v2** | `docker -v` |
|
||||
| TLS | OpenSSL 1.1 + | Self‑signed cert generated at first run |
|
||||
|
||||
---
|
||||
|
||||
## 1 · Connected‑host install (Docker Compose)
|
||||
|
||||
```bash
|
||||
# 1. Make a working directory
|
||||
mkdir stella && cd stella
|
||||
|
||||
# 2. Download the signed Compose bundle + example .env
|
||||
curl -LO https://get.stella-ops.org/releases/latest/.env.example
|
||||
curl -LO https://get.stella-ops.org/releases/latest/.env.example.sig
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml.sig
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml.sig
|
||||
|
||||
# 3. Verify provenance (Cosign public key is stable)
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature .env.example.sig \
|
||||
.env.example
|
||||
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature docker-compose.infrastructure.yml.sig \
|
||||
docker-compose.infrastructure.yml
|
||||
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature docker-compose.stella-ops.yml.sig \
|
||||
docker-compose.stella-ops.yml
|
||||
|
||||
# 4. Copy .env.example → .env and edit secrets
|
||||
cp .env.example .env
|
||||
$EDITOR .env
|
||||
|
||||
# 5. Launch databases (MongoDB + Redis)
|
||||
docker compose --env-file .env -f docker-compose.infrastructure.yml up -d
|
||||
|
||||
# 6. Launch Stella Ops (first run pulls ~50 MB merged vuln DB)
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d
|
||||
````
|
||||
|
||||
*Default login:* `admin / changeme`
|
||||
UI: [https://\<host\>:8443](https://<host>:8443) (self‑signed certificate)
|
||||
|
||||
> **Pinning best‑practice** – in production environments replace
|
||||
> `stella-ops:latest` with the immutable digest printed by
|
||||
> `docker images --digests`.
|
||||
|
||||
> **Repo bundles** – Development, staging, and air‑gapped Compose profiles live
|
||||
> under `deploy/compose/`, already tied to the release manifests in
|
||||
> `deploy/releases/`. Helm users can pull the same channel overlays from
|
||||
> `deploy/helm/stellaops/values-*.yaml` and validate everything with
|
||||
> `deploy/tools/validate-profiles.sh`.
|
||||
|
||||
### 1.1 · Concelier authority configuration
|
||||
|
||||
The Concelier container reads configuration from `etc/concelier.yaml` plus
|
||||
`CONCELIER_` environment variables. To enable the new Authority integration:
|
||||
|
||||
1. Add the following keys to `.env` (replace values for your environment):
|
||||
|
||||
```bash
|
||||
CONCELIER_AUTHORITY__ENABLED=true
|
||||
CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true # temporary rollout only
|
||||
CONCELIER_AUTHORITY__ISSUER="https://authority.internal"
|
||||
CONCELIER_AUTHORITY__AUDIENCES__0="api://concelier"
|
||||
CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger"
|
||||
CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read"
|
||||
CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest"
|
||||
CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default"
|
||||
CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
|
||||
CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger"
|
||||
CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read"
|
||||
CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest"
|
||||
CONCELIER_AUTHORITY__CLIENTSECRETFILE="/run/secrets/concelier_authority_client"
|
||||
CONCELIER_AUTHORITY__BYPASSNETWORKS__0="127.0.0.1/32"
|
||||
CONCELIER_AUTHORITY__BYPASSNETWORKS__1="::1/128"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES=true
|
||||
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0="00:00:01"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1="00:00:02"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__2="00:00:05"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK=true
|
||||
CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE="00:10:00"
|
||||
```
|
||||
|
||||
Store the client secret outside source control (Docker secrets, mounted file,
|
||||
or Kubernetes Secret). Concelier loads the secret during post-configuration, so
|
||||
the value never needs to appear in the YAML template.
|
||||
|
||||
Connected sites can keep the retry ladder short (1 s, 2 s, 5 s) so job triggers fail fast when Authority is down. For air‑gapped or intermittently connected deployments, extend `RESILIENCE__OFFLINECACHETOLERANCE` (e.g. `00:30:00`) so cached discovery/JWKS data remains valid while the Offline Kit synchronises upstream changes.
|
||||
|
||||
2. Redeploy Concelier:
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d concelier
|
||||
```
|
||||
|
||||
3. Tail the logs: `docker compose logs -f concelier`. Successful `/jobs*` calls now
|
||||
emit `Concelier.Authorization.Audit` entries with `route`, `status`, `subject`,
|
||||
`clientId`, `scopes`, `bypass`, and `remote` fields. 401 denials keep the same
|
||||
shape—watch for `bypass=True`, which indicates a bypass CIDR accepted an anonymous
|
||||
call. See `docs/ops/concelier-authority-audit-runbook.md` for a full audit/alerting checklist.
|
||||
|
||||
> **Enforcement deadline** – keep `CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true`
|
||||
> only while validating the rollout. Set it to `false` (and restart Concelier)
|
||||
> before **2025-12-31 UTC** to require tokens in production.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Optional: request a free quota token
|
||||
|
||||
Anonymous installs allow **{{ quota\_anon }} scans per UTC day**.
|
||||
Email `token@stella-ops.org` to receive a signed JWT that raises the limit to
|
||||
**{{ quota\_token }} scans/day**. Insert it into `.env`:
|
||||
|
||||
```bash
|
||||
STELLA_JWT="paste‑token‑here"
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml \
|
||||
exec stella-ops stella set-jwt "$STELLA_JWT"
|
||||
```
|
||||
|
||||
> The UI shows a reminder at 200 scans and throttles above the limit but will
|
||||
> **never block** your pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Air‑gapped install (Offline Update Kit)
|
||||
|
||||
When running on an isolated network use the **Offline Update Kit (OUK)**:
|
||||
|
||||
```bash
|
||||
# Download & verify on a connected host
|
||||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz
|
||||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz.sig
|
||||
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature stella-ops-offline-kit-v0.1a.tgz.sig \
|
||||
stella-ops-offline-kit-v0.1a.tgz
|
||||
|
||||
# Transfer → air‑gap → import
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml \
|
||||
exec stella admin import-offline-usage-kit stella-ops-offline-kit-v0.1a.tgz
|
||||
```
|
||||
|
||||
*Import is atomic; no service downtime.*
|
||||
|
||||
For details see the dedicated [Offline Kit guide](/offline/).
|
||||
|
||||
---
|
||||
|
||||
## 4 · Next steps
|
||||
|
||||
* **5‑min Quick‑Start:** `/quickstart/`
|
||||
* **CI recipes:** `docs/ci/20_CI_RECIPES.md`
|
||||
* **Plug‑in SDK:** `/plugins/`
|
||||
|
||||
---
|
||||
|
||||
*Generated {{ "now" | date: "%Y‑%m‑%d" }} — build tags inserted at render time.*
|
||||
# Stella Ops — Installation Guide (Docker & Air‑Gap)
|
||||
|
||||
<!--
|
||||
This file is processed by the Eleventy build.
|
||||
Do **not** hard‑code versions or quota numbers; inherit from
|
||||
docs/_includes/CONSTANTS.md instead.
|
||||
{{ dotnet }} → ".NET 10 LTS"
|
||||
{{ angular }} → "20"
|
||||
-->
|
||||
|
||||
> **Status — public α not yet published.**
|
||||
> The commands below will work as soon as the first image is tagged
|
||||
> `registry.stella-ops.org/stella-ops/stella-ops:0.1.0-alpha`
|
||||
> (target date: **late 2025**). Track progress on the
|
||||
> [road‑map](/roadmap/).
|
||||
|
||||
---
|
||||
|
||||
## 0 · Prerequisites
|
||||
|
||||
| Item | Minimum | Notes |
|
||||
|------|---------|-------|
|
||||
| Linux | Ubuntu 22.04 LTS / Alma 9 | x86‑64 or arm64 |
|
||||
| CPU / RAM | 2 vCPU / 2 GiB | Laptop baseline |
|
||||
| Disk | 10 GiB SSD | SBOM + vuln DB cache |
|
||||
| Docker | **Engine 25 + Compose v2** | `docker -v` |
|
||||
| TLS | OpenSSL 1.1 + | Self‑signed cert generated at first run |
|
||||
|
||||
---
|
||||
|
||||
## 1 · Connected‑host install (Docker Compose)
|
||||
|
||||
```bash
|
||||
# 1. Make a working directory
|
||||
mkdir stella && cd stella
|
||||
|
||||
# 2. Download the signed Compose bundle + example .env
|
||||
curl -LO https://get.stella-ops.org/releases/latest/.env.example
|
||||
curl -LO https://get.stella-ops.org/releases/latest/.env.example.sig
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.infrastructure.yml.sig
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml
|
||||
curl -LO https://get.stella-ops.org/releases/latest/docker-compose.stella-ops.yml.sig
|
||||
|
||||
# 3. Verify provenance (Cosign public key is stable)
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature .env.example.sig \
|
||||
.env.example
|
||||
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature docker-compose.infrastructure.yml.sig \
|
||||
docker-compose.infrastructure.yml
|
||||
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature docker-compose.stella-ops.yml.sig \
|
||||
docker-compose.stella-ops.yml
|
||||
|
||||
# 4. Copy .env.example → .env and edit secrets
|
||||
cp .env.example .env
|
||||
$EDITOR .env
|
||||
|
||||
# 5. Launch databases (MongoDB + Redis)
|
||||
docker compose --env-file .env -f docker-compose.infrastructure.yml up -d
|
||||
|
||||
# 6. Launch Stella Ops (first run pulls ~50 MB merged vuln DB)
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d
|
||||
````
|
||||
|
||||
*Default login:* `admin / changeme`
|
||||
UI: [https://\<host\>:8443](https://<host>:8443) (self‑signed certificate)
|
||||
|
||||
> **Pinning best‑practice** – in production environments replace
|
||||
> `stella-ops:latest` with the immutable digest printed by
|
||||
> `docker images --digests`.
|
||||
|
||||
> **Repo bundles** – Development, staging, and air‑gapped Compose profiles live
|
||||
> under `deploy/compose/`, already tied to the release manifests in
|
||||
> `deploy/releases/`. Helm users can pull the same channel overlays from
|
||||
> `deploy/helm/stellaops/values-*.yaml` and validate everything with
|
||||
> `deploy/tools/validate-profiles.sh`.
|
||||
|
||||
### 1.1 · Concelier authority configuration
|
||||
|
||||
The Concelier container reads configuration from `etc/concelier.yaml` plus
|
||||
`CONCELIER_` environment variables. To enable the new Authority integration:
|
||||
|
||||
1. Add the following keys to `.env` (replace values for your environment):
|
||||
|
||||
```bash
|
||||
CONCELIER_AUTHORITY__ENABLED=true
|
||||
CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true # temporary rollout only
|
||||
CONCELIER_AUTHORITY__ISSUER="https://authority.internal"
|
||||
CONCELIER_AUTHORITY__AUDIENCES__0="api://concelier"
|
||||
CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger"
|
||||
CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read"
|
||||
CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest"
|
||||
CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default"
|
||||
CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
|
||||
CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger"
|
||||
CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read"
|
||||
CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest"
|
||||
CONCELIER_AUTHORITY__CLIENTSECRETFILE="/run/secrets/concelier_authority_client"
|
||||
CONCELIER_AUTHORITY__BYPASSNETWORKS__0="127.0.0.1/32"
|
||||
CONCELIER_AUTHORITY__BYPASSNETWORKS__1="::1/128"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES=true
|
||||
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0="00:00:01"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1="00:00:02"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__2="00:00:05"
|
||||
CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK=true
|
||||
CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE="00:10:00"
|
||||
```
|
||||
|
||||
Store the client secret outside source control (Docker secrets, mounted file,
|
||||
or Kubernetes Secret). Concelier loads the secret during post-configuration, so
|
||||
the value never needs to appear in the YAML template.
|
||||
|
||||
Connected sites can keep the retry ladder short (1 s, 2 s, 5 s) so job triggers fail fast when Authority is down. For air‑gapped or intermittently connected deployments, extend `RESILIENCE__OFFLINECACHETOLERANCE` (e.g. `00:30:00`) so cached discovery/JWKS data remains valid while the Offline Kit synchronises upstream changes.
|
||||
|
||||
2. Redeploy Concelier:
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml up -d concelier
|
||||
```
|
||||
|
||||
3. Tail the logs: `docker compose logs -f concelier`. Successful `/jobs*` calls now
|
||||
emit `Concelier.Authorization.Audit` entries with `route`, `status`, `subject`,
|
||||
`clientId`, `scopes`, `bypass`, and `remote` fields. 401 denials keep the same
|
||||
shape—watch for `bypass=True`, which indicates a bypass CIDR accepted an anonymous
|
||||
call. See `docs/modules/concelier/operations/authority-audit-runbook.md` for a full audit/alerting checklist.
|
||||
|
||||
> **Enforcement deadline** – keep `CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=true`
|
||||
> only while validating the rollout. Set it to `false` (and restart Concelier)
|
||||
> before **2025-12-31 UTC** to require tokens in production.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Optional: request a free quota token
|
||||
|
||||
Anonymous installs allow **{{ quota\_anon }} scans per UTC day**.
|
||||
Email `token@stella-ops.org` to receive a signed JWT that raises the limit to
|
||||
**{{ quota\_token }} scans/day**. Insert it into `.env`:
|
||||
|
||||
```bash
|
||||
STELLA_JWT="paste‑token‑here"
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml \
|
||||
exec stella-ops stella set-jwt "$STELLA_JWT"
|
||||
```
|
||||
|
||||
> The UI shows a reminder at 200 scans and throttles above the limit but will
|
||||
> **never block** your pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Air‑gapped install (Offline Update Kit)
|
||||
|
||||
When running on an isolated network use the **Offline Update Kit (OUK)**:
|
||||
|
||||
```bash
|
||||
# Download & verify on a connected host
|
||||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz
|
||||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-v0.1a.tgz.sig
|
||||
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature stella-ops-offline-kit-v0.1a.tgz.sig \
|
||||
stella-ops-offline-kit-v0.1a.tgz
|
||||
|
||||
# Transfer → air‑gap → import
|
||||
docker compose --env-file .env -f docker-compose.stella-ops.yml \
|
||||
exec stella admin import-offline-usage-kit stella-ops-offline-kit-v0.1a.tgz
|
||||
```
|
||||
|
||||
*Import is atomic; no service downtime.*
|
||||
|
||||
For details see the dedicated [Offline Kit guide](/offline/).
|
||||
|
||||
---
|
||||
|
||||
## 4 · Next steps
|
||||
|
||||
* **5‑min Quick‑Start:** `/quickstart/`
|
||||
* **CI recipes:** `docs/ci/20_CI_RECIPES.md`
|
||||
* **Plug‑in SDK:** `/plugins/`
|
||||
|
||||
---
|
||||
|
||||
*Generated {{ "now" | date: "%Y‑%m‑%d" }} — build tags inserted at render time.*
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
# Offline Update Kit (OUK) — Air‑Gap Bundle
|
||||
|
||||
<!--
|
||||
Build‑time variable injection:
|
||||
{{ quota_anon }} = 33
|
||||
{{ quota_token }} = 333
|
||||
{{ dotnet }} = "10 LTS"
|
||||
-->
|
||||
|
||||
The **Offline Update Kit** packages everything Stella Ops needs to run on a
|
||||
completely isolated network:
|
||||
|
||||
# Offline Update Kit (OUK) — Air‑Gap Bundle
|
||||
|
||||
<!--
|
||||
Build‑time variable injection:
|
||||
{{ quota_anon }} = 33
|
||||
{{ quota_token }} = 333
|
||||
{{ dotnet }} = "10 LTS"
|
||||
-->
|
||||
|
||||
The **Offline Update Kit** packages everything Stella Ops needs to run on a
|
||||
completely isolated network:
|
||||
|
||||
| Component | Contents |
|
||||
|-----------|----------|
|
||||
| **Merged vulnerability feeds** | OSV, GHSA plus optional NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU |
|
||||
@@ -94,7 +94,7 @@ cosign verify-blob \
|
||||
````
|
||||
|
||||
**CLI shortcut.** `stellaops-cli offline kit pull --destination ./offline-kit` downloads the bundle, manifest, and detached signatures in one step, resumes partial transfers, and writes a `.metadata.json` summary for later import.
|
||||
|
||||
|
||||
Verification prints **OK** and the SHA‑256 digest; cross‑check against the
|
||||
[changelog](https://git.stella-ops.org/stella-ops/offline-kit/-/releases).
|
||||
|
||||
@@ -228,7 +228,7 @@ tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/Stell
|
||||
|
||||
The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted.
|
||||
|
||||
> **Release guardrail.** The automated release pipeline now publishes the Python plug-in from source and executes `dotnet run --project tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle.
|
||||
> **Release guardrail.** The automated release pipeline now publishes the Python plug-in from source and executes `dotnet run --project src/Tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle.
|
||||
|
||||
### Debug store mirror
|
||||
|
||||
@@ -246,41 +246,41 @@ The script mirrors the debug tree into the Offline Kit staging directory, verifi
|
||||
---
|
||||
|
||||
## 3 · Delta patch workflow
|
||||
|
||||
1. **Connected site** fetches `stella-ouk-YYYY‑MM‑DD.delta.tgz`.
|
||||
2. Transfer via any medium (USB, portable disk).
|
||||
3. `stella admin import-offline-usage-kit <delta>` applies only changed CVE rows & images.
|
||||
|
||||
Daily deltas are **< 30 MB**; weekly roll‑up produces a fresh full kit.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Quota behaviour offline
|
||||
|
||||
The scanner enforces the same fair‑use limits offline:
|
||||
|
||||
* **Anonymous:** {{ quota\_anon }} scans per UTC day
|
||||
* **Free JWT:** {{ quota\_token }} scans per UTC day
|
||||
|
||||
Soft reminder at 200 scans; throttle above the ceiling but **never block**.
|
||||
See the detailed rules in
|
||||
[`33_333_QUOTA_OVERVIEW.md`](33_333_QUOTA_OVERVIEW.md).
|
||||
|
||||
---
|
||||
|
||||
## 5 · Troubleshooting
|
||||
|
||||
| Symptom | Explanation | Fix |
|
||||
| -------------------------------------- | ---------------------------------------- | ------------------------------------- |
|
||||
| `could not verify SBOM hash` | Bundle corrupted in transit | Re‑download / re‑copy |
|
||||
| Import hangs at `Applying feeds…` | Low disk space in `/var/lib/stella` | Free ≥ 2 GiB before retry |
|
||||
| `quota exceeded` same day after import | Import resets counters at UTC 00:00 only | Wait until next UTC day or load a JWT |
|
||||
|
||||
---
|
||||
|
||||
## 6 · Related documentation
|
||||
|
||||
|
||||
1. **Connected site** fetches `stella-ouk-YYYY‑MM‑DD.delta.tgz`.
|
||||
2. Transfer via any medium (USB, portable disk).
|
||||
3. `stella admin import-offline-usage-kit <delta>` applies only changed CVE rows & images.
|
||||
|
||||
Daily deltas are **< 30 MB**; weekly roll‑up produces a fresh full kit.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Quota behaviour offline
|
||||
|
||||
The scanner enforces the same fair‑use limits offline:
|
||||
|
||||
* **Anonymous:** {{ quota\_anon }} scans per UTC day
|
||||
* **Free JWT:** {{ quota\_token }} scans per UTC day
|
||||
|
||||
Soft reminder at 200 scans; throttle above the ceiling but **never block**.
|
||||
See the detailed rules in
|
||||
[`33_333_QUOTA_OVERVIEW.md`](33_333_QUOTA_OVERVIEW.md).
|
||||
|
||||
---
|
||||
|
||||
## 5 · Troubleshooting
|
||||
|
||||
| Symptom | Explanation | Fix |
|
||||
| -------------------------------------- | ---------------------------------------- | ------------------------------------- |
|
||||
| `could not verify SBOM hash` | Bundle corrupted in transit | Re‑download / re‑copy |
|
||||
| Import hangs at `Applying feeds…` | Low disk space in `/var/lib/stella` | Free ≥ 2 GiB before retry |
|
||||
| `quota exceeded` same day after import | Import resets counters at UTC 00:00 only | Wait until next UTC day or load a JWT |
|
||||
|
||||
---
|
||||
|
||||
## 6 · Related documentation
|
||||
|
||||
* **Install guide:** `/install/#air-gapped`
|
||||
* **Sovereign mode rationale:** `/sovereign/`
|
||||
* **Security policy:** `/security/#reporting-a-vulnerability`
|
||||
* **CERT-Bund snapshots:** `python tools/certbund_offline_snapshot.py --help` (see `docs/ops/concelier-certbund-operations.md`)
|
||||
* **CERT-Bund snapshots:** `python src/Tools/certbund_offline_snapshot.py --help` (see `docs/modules/concelier/operations/connectors/certbund.md`)
|
||||
|
||||
@@ -1,20 +1,20 @@
|
||||
# Docs & Enablement Guild
|
||||
|
||||
## Mission
|
||||
Produce and maintain offline-friendly documentation for StellaOps modules, covering architecture, configuration, operator workflows, and developer onboarding.
|
||||
|
||||
## Scope Highlights
|
||||
- Authority docs (`docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, upcoming `docs/11_AUTHORITY.md`).
|
||||
- Concelier quickstarts, CLI guides, Offline Kit manuals.
|
||||
- Release notes and migration playbooks.
|
||||
|
||||
## Operating Principles
|
||||
- Keep guides deterministic and in sync with shipped configuration samples.
|
||||
- Prefer tables/checklists for operator steps; flag security-sensitive actions.
|
||||
- When work involves a specific `StellaOps.<Component>` project, consult both `docs/07_HIGH_LEVEL_ARCHITECTURE.md` and the matching dossier `docs/ARCHITECTURE_<COMPONENT>.md` before drafting or editing content.
|
||||
- Update `docs/TASKS.md` whenever work items change status (TODO/DOING/REVIEW/DONE/BLOCKED).
|
||||
|
||||
## Coordination
|
||||
- Authority Core & Plugin teams for auth-related changes.
|
||||
- Security Guild for threat-model outputs and mitigations.
|
||||
- DevEx for tooling diagrams and documentation pipeline.
|
||||
# Docs & Enablement Guild
|
||||
|
||||
## Mission
|
||||
Produce and maintain offline-friendly documentation for StellaOps modules, covering architecture, configuration, operator workflows, and developer onboarding.
|
||||
|
||||
## Scope Highlights
|
||||
- Authority docs (`docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, upcoming `docs/11_AUTHORITY.md`).
|
||||
- Concelier quickstarts, CLI guides, Offline Kit manuals.
|
||||
- Release notes and migration playbooks.
|
||||
|
||||
## Operating Principles
|
||||
- Keep guides deterministic and in sync with shipped configuration samples.
|
||||
- Prefer tables/checklists for operator steps; flag security-sensitive actions.
|
||||
- When work involves a specific `StellaOps.<Component>` project, consult both `docs/07_HIGH_LEVEL_ARCHITECTURE.md` and the matching dossier `docs/modules/<component>/architecture.md` before drafting or editing content.
|
||||
- Update `docs/TASKS.md` whenever work items change status (TODO/DOING/REVIEW/DONE/BLOCKED).
|
||||
|
||||
## Coordination
|
||||
- Authority Core & Plugin teams for auth-related changes.
|
||||
- Security Guild for threat-model outputs and mitigations.
|
||||
- DevEx for tooling diagrams and documentation pipeline.
|
||||
|
||||
@@ -36,21 +36,21 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
- **07 – [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)**
|
||||
- **08 – [Architecture Decision Records](adr/index.md)**
|
||||
- **08 – Module Architecture Dossiers**
|
||||
- [Architecture Overview](architecture/overview.md)
|
||||
- [Scanner](ARCHITECTURE_SCANNER.md)
|
||||
- [Concelier](ARCHITECTURE_CONCELIER.md)
|
||||
- [Excititor](ARCHITECTURE_EXCITITOR.md)
|
||||
- [Excititor Mirrors](ARCHITECTURE_EXCITITOR_MIRRORS.md)
|
||||
- [Signer](ARCHITECTURE_SIGNER.md)
|
||||
- [Attestor](ARCHITECTURE_ATTESTOR.md)
|
||||
- [Authority](ARCHITECTURE_AUTHORITY.md)
|
||||
- [Policy Engine](architecture/policy-engine.md)
|
||||
- [Notify](ARCHITECTURE_NOTIFY.md)
|
||||
- [Scheduler](ARCHITECTURE_SCHEDULER.md)
|
||||
- [CLI](ARCHITECTURE_CLI.md)
|
||||
- [Web UI](ARCHITECTURE_UI.md)
|
||||
- [Zastava Runtime](ARCHITECTURE_ZASTAVA.md)
|
||||
- [Release & Operations](ARCHITECTURE_DEVOPS.md)
|
||||
- [Architecture Overview](modules/platform/architecture-overview.md)
|
||||
- [Scanner](modules/scanner/architecture.md)
|
||||
- [Concelier](modules/concelier/architecture.md)
|
||||
- [Excititor](modules/excititor/architecture.md)
|
||||
- [Excititor Mirrors](modules/excititor/mirrors.md)
|
||||
- [Signer](modules/signer/architecture.md)
|
||||
- [Attestor](modules/attestor/architecture.md)
|
||||
- [Authority](modules/authority/architecture.md)
|
||||
- [Policy Engine](modules/policy/architecture.md)
|
||||
- [Notify](modules/notify/architecture.md)
|
||||
- [Scheduler](modules/scheduler/architecture.md)
|
||||
- [CLI](modules/cli/architecture.md)
|
||||
- [Web UI](modules/ui/architecture.md)
|
||||
- [Zastava Runtime](modules/zastava/architecture.md)
|
||||
- [Release & Operations](modules/devops/architecture.md)
|
||||
- **09 – [API & CLI Reference](09_API_CLI_REFERENCE.md)**
|
||||
- **10 – [Plug‑in SDK Guide](10_PLUGIN_SDK_GUIDE.md)**
|
||||
- **10 – [Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)**
|
||||
@@ -60,13 +60,14 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
- **31 – [Aggregation-Only Contract Reference](ingestion/aggregation-only-contract.md)**
|
||||
- **31 – [Advisory Observations & Linksets](advisories/aggregation.md)**
|
||||
- **31 – [VEX Observations & Linksets](vex/aggregation.md)**
|
||||
- **32 – [Entry-Point Detection Playbook](modules/scanner/operations/entrypoint.md)**
|
||||
- **30 – Developer Templates**
|
||||
- [Excititor Connector Skeleton](dev/templates/excititor-connector/)
|
||||
- **11 – [Authority Service](11_AUTHORITY.md)**
|
||||
- **11 – [Data Schemas](11_DATA_SCHEMAS.md)**
|
||||
- **12 – [Performance Workbook](12_PERFORMANCE_WORKBOOK.md)**
|
||||
- **13 – [Release‑Engineering Playbook](13_RELEASE_ENGINEERING_PLAYBOOK.md)**
|
||||
- **20 – [CLI AOC Commands Reference](cli/cli-reference.md)**
|
||||
- **20 – [CLI AOC Commands Reference](modules/cli/guides/cli-reference.md)**
|
||||
- **20 – [Console CLI Parity Matrix](cli-vs-ui-parity.md)**
|
||||
- **60 – [Policy Engine Overview](policy/overview.md)**
|
||||
- **61 – [Policy DSL Grammar](policy/dsl.md)**
|
||||
@@ -74,7 +75,7 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
- **63 – [Policy Runs & Orchestration](policy/runs.md)**
|
||||
- **64 – [Policy Exception Effects](policy/exception-effects.md)**
|
||||
- **65 – [Policy Engine REST API](api/policy.md)**
|
||||
- **66 – [Policy CLI Guide](cli/policy.md)**
|
||||
- **66 – [Policy CLI Guide](modules/cli/guides/policy.md)**
|
||||
- **67 – [Policy Editor Workspace](ui/policy-editor.md)**
|
||||
- **68 – [Policy Observability](observability/policy.md)**
|
||||
- **69 – [Console Observability](observability/ui-telemetry.md)**
|
||||
@@ -84,14 +85,14 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
- **72 – [Policy FAQ](faq/policy-faq.md)**
|
||||
- **73 – [Policy Run DTOs](../src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-20-001-POLICY-RUNS.md)**
|
||||
- **30 – [Fixture Maintenance](dev/fixtures.md)**
|
||||
- **74 – [Export Center Overview](export-center/overview.md)**
|
||||
- **75 – [Export Center Architecture](export-center/architecture.md)**
|
||||
- **76 – [Export Center Profiles](export-center/profiles.md)**
|
||||
- **77 – [Export Center API Reference](export-center/api.md)**
|
||||
- **78 – [Export Center CLI Guide](export-center/cli.md)**
|
||||
- **79 – [Export Center Trivy Adapters](export-center/trivy-adapter.md)**
|
||||
- **80 – [Export Center Mirror Bundles](export-center/mirror-bundles.md)**
|
||||
- **81 – [Export Center Provenance & Signing](export-center/provenance-and-signing.md)**
|
||||
- **74 – [Export Center Overview](modules/export-center/overview.md)**
|
||||
- **75 – [Export Center Architecture](modules/export-center/architecture.md)**
|
||||
- **76 – [Export Center Profiles](modules/export-center/profiles.md)**
|
||||
- **77 – [Export Center API Reference](modules/export-center/api.md)**
|
||||
- **78 – [Export Center CLI Guide](modules/export-center/cli.md)**
|
||||
- **79 – [Export Center Trivy Adapters](modules/export-center/trivy-adapter.md)**
|
||||
- **80 – [Export Center Mirror Bundles](modules/export-center/mirror-bundles.md)**
|
||||
- **81 – [Export Center Provenance & Signing](modules/export-center/provenance-and-signing.md)**
|
||||
|
||||
### User & operator guides
|
||||
- **14 – [Glossary](14_GLOSSARY_OF_TERMS.md)**
|
||||
@@ -148,7 +149,7 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
> Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
- **Aggregation-Only Contract (AOC).** Ingestion services aggregate and link facts only—derived precedence, severity, and safe-fix hints live in Policy overlays and dedicated explorers. Review [`implplan/AGENTS.md`](implplan/AGENTS.md) and the AOC guardrails in [`aoc/aoc-guardrails.md`](aoc/aoc-guardrails.md).
|
||||
- **Cartographer owns graphs.** SBOM Service emits projections/events; Cartographer (`CARTO-GRAPH-21-00x`) builds graph storage, overlays, and tiles. See `ARCHITECTURE_CONCELIER.md` (Cartographer handshake section) for handoff boundaries.
|
||||
- **Cartographer owns graphs.** SBOM Service emits projections/events; Cartographer (`CARTO-GRAPH-21-00x`) builds graph storage, overlays, and tiles. See `modules/concelier/architecture.md` (Cartographer handshake section) for handoff boundaries.
|
||||
- **Notifier replaces legacy Notify.** Sprint‑15 `StellaOps.Notify.*` tasks are frozen; use the Notifications Studio/Notifier backlogs (`NOTIFY-SVC-38..40`, `WEB-NOTIFY-3x-00x`, `CLI-NOTIFY-3x-00x`).
|
||||
- **Dedicated services for Vuln & Policy.** Vuln Explorer work flows through `src/VulnExplorer/StellaOps.VulnExplorer.Api`/Console/CLI (Sprint 29); gateway routes proxy only. Policy Engine remains the sole source for precedence/suppression overlays.
|
||||
- **Cleanup log.** The backlog consolidation summary lives in [`backlog/2025-10-cleanup.md`](backlog/2025-10-cleanup.md).
|
||||
|
||||
762
docs/TASKS.md
762
docs/TASKS.md
@@ -1,381 +1,381 @@
|
||||
# Docs Guild Task Board (UTC 2025-10-10)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOC7.README-INDEX | DONE (2025-10-17) | Docs Guild | — | Refresh index docs (docs/README.md + root README) after architecture dossier split and Offline Kit overhaul. | ✅ ToC reflects new component architecture docs; ✅ root README highlights updated doc set; ✅ Offline Kit guide linked correctly. |
|
||||
| DOC4.AUTH-PDG | DONE (2025-10-19) | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. |
|
||||
| DOC1.AUTH | DONE (2025-10-12) | Docs Guild, Authority Core | CORE5B.DOC | Draft `docs/11_AUTHORITY.md` covering architecture, configuration, bootstrap flows. | ✅ Architecture + config sections approved by Core; ✅ Samples reference latest options; ✅ Offline note added. |
|
||||
| DOC3.Concelier-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Concelier authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. |
|
||||
| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. |
|
||||
| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Concelier conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/ops/concelier-conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Concelier team. |
|
||||
| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/ops/concelier-authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. |
|
||||
| DOCS-ADR-09-001 | DONE (2025-10-19) | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted (`docs/updates/2025-10-18-docs-guild.md`). |
|
||||
| DOCS-EVENTS-09-002 | DONE (2025-10-19) | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated (Ajv CI hooked); docs/events/README summarises usage; Platform Events notified via `docs/updates/2025-10-18-docs-guild.md`. |
|
||||
| DOCS-EVENTS-09-003 | DONE (2025-10-19) | Docs Guild | DOCS-EVENTS-09-002 | Add human-readable envelope field references and canonical payload samples for published events, including offline validation workflow. | Tables explain common headers/payload segments; versioned sample payloads committed; README links to validation instructions and samples. |
|
||||
| DOCS-EVENTS-09-004 | DONE (2025-10-19) | Docs Guild, Scanner WebService | SCANNER-EVENTS-15-201 | Refresh scanner event docs to mirror DSSE-backed report fields, document `scanner.scan.completed`, and capture canonical sample validation. | Schemas updated for new payload shape; README references DSSE reuse and validation test; samples align with emitted events. |
|
||||
| PLATFORM-EVENTS-09-401 | DONE (2025-10-21) | Platform Events Guild | DOCS-EVENTS-09-003 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. | Notify models tests now run schema validation against `docs/events/*.json`, event schemas allow optional `attributes`, and docs capture the new validation workflow. |
|
||||
| RUNTIME-GUILD-09-402 | DONE (2025-10-19) | Runtime Guild | SCANNER-POLICY-09-107 | Confirm Scanner WebService surfaces `quietedFindingCount` and progress hints to runtime consumers; document readiness checklist. | Runtime verification run captures enriched payload; checklist/doc updates merged; stakeholders acknowledge availability. |
|
||||
| DOCS-CONCELIER-07-201 | DONE (2025-10-22) | Docs Guild, Concelier WebService | FEEDWEB-DOCS-01-001 | Final editorial review and publish pass for Concelier authority toggle documentation (Quickstart + operator guide). | Review feedback resolved, publish PR merged, release notes updated with documentation pointer. |
|
||||
| DOCS-RUNTIME-17-004 | DONE (2025-10-26) | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads (`process.buildId`), Scanner `/policy/runtime` response (`buildIds` list), debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections (Observer, Scanner, CLI), examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides + CLI help. |
|
||||
| DOCS-OBS-50-001 | BLOCKED (2025-10-26) | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Publish `/docs/observability/overview.md` introducing scope, imposed rule banner, architecture diagram, and tenant guarantees. | Doc merged with imposed rule banner; diagram committed; cross-links to telemetry stack + evidence locker docs. |
|
||||
> Blocked: waiting on telemetry core deliverable (TELEMETRY-OBS-50-001) to finalise architecture details and diagrams.
|
||||
| DOCS-OBS-50-002 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-50-002 | Author `/docs/observability/telemetry-standards.md` detailing common fields, scrubbing policy, sampling defaults, and redaction override procedure. | Doc merged; imposed rule banner present; examples validated with telemetry fixtures; security review sign-off captured. |
|
||||
| DOCS-OBS-50-003 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Create `/docs/observability/logging.md` covering structured log schema, dos/don'ts, tenant isolation, and copyable examples. | Doc merged with banner; sample logs redacted; lint passes; linked from coding standards. |
|
||||
| DOCS-OBS-50-004 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-002 | Draft `/docs/observability/tracing.md` explaining context propagation, async linking, CLI header usage, and sampling strategies. | Doc merged; imposed rule banner included; diagrams updated; references to CLI/Console features added. |
|
||||
| DOCS-OBS-51-001 | TODO | Docs Guild, DevOps Guild | WEB-OBS-51-001, DEVOPS-OBS-51-001 | Publish `/docs/observability/metrics-and-slos.md` cataloging metrics, SLO targets, burn rate policies, and alert runbooks. | Doc merged with banner; SLO tables verified; alert workflows linked to incident runbook. |
|
||||
| DOCS-SEC-OBS-50-001 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-51-002 | Update `/docs/security/redaction-and-privacy.md` to cover telemetry privacy controls, tenant opt-in debug, and imposed rule reminder. | Doc merged; redaction matrix updated; banner present; security sign-off recorded. |
|
||||
| DOCS-INSTALL-50-001 | TODO | Docs Guild, DevOps Guild | DEVOPS-OBS-50-003 | Add `/docs/install/telemetry-stack.md` with collector deployment, exporter options, offline kit notes, and imposed rule banner. | Doc merged; install steps verified on air-gapped profile; banner present; screenshots attached. |
|
||||
| DOCS-FORENSICS-53-001 | TODO | Docs Guild, Evidence Locker Guild | EVID-OBS-53-003 | Publish `/docs/forensics/evidence-locker.md` describing bundle formats, WORM options, retention, legal hold, and imposed rule banner. | Doc merged; manifest examples validated; banner present; legal hold steps aligned with API. |
|
||||
| DOCS-FORENSICS-53-002 | TODO | Docs Guild, Provenance Guild | PROV-OBS-54-001 | Release `/docs/forensics/provenance-attestation.md` covering DSSE schema, signing process, verification workflow, and imposed rule banner. | Doc merged; sample statements reference fixtures; banner included; verification steps tested. |
|
||||
| DOCS-FORENSICS-53-003 | TODO | Docs Guild, Timeline Indexer Guild | TIMELINE-OBS-52-003 | Publish `/docs/forensics/timeline.md` with schema, event kinds, filters, query examples, and imposed rule banner. | Doc merged; query examples validated; banner present; linked from Console/CLI docs. |
|
||||
| DOCS-CONSOLE-OBS-52-001 | TODO | Docs Guild, Console Guild | CONSOLE-OBS-51-001 | Document `/docs/console/observability.md` showcasing Observability Hub widgets, trace/log search, imposed rule banner, and accessibility tips. | Doc merged; screenshots updated; banner present; navigation steps verified. |
|
||||
| DOCS-CONSOLE-OBS-52-002 | TODO | Docs Guild, Console Guild | CONSOLE-OBS-52-002, CONSOLE-OBS-53-001 | Publish `/docs/console/forensics.md` covering timeline explorer, evidence viewer, attestation verifier, imposed rule banner, and troubleshooting. | Doc merged; banner included; workflows validated via Playwright capture; troubleshooting section populated. |
|
||||
| DOCS-CLI-OBS-52-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-OBS-52-001 | Create `/docs/cli/observability.md` detailing `stella obs` commands, examples, exit codes, imposed rule banner, and scripting tips. | Doc merged; examples tested; banner included; CLI parity matrix updated. |
|
||||
| DOCS-CLI-FORENSICS-53-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-FORENSICS-54-001 | Publish `/docs/cli/forensics.md` for snapshot/verify/attest commands with sample outputs, imposed rule banner, and offline workflows. | Doc merged; sample bundles verified; banner present; offline notes cross-linked. |
|
||||
| DOCS-RUNBOOK-55-001 | TODO | Docs Guild, Ops Guild | DEVOPS-OBS-55-001, WEB-OBS-55-001 | Author `/docs/runbooks/incidents.md` describing incident mode activation, escalation steps, retention impact, verification checklist, and imposed rule banner. | Doc merged; runbook rehearsed; banner included; linked from alerts. |
|
||||
| DOCS-AOC-19-001 | DONE (2025-10-26) | Docs Guild, Concelier Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Author `/docs/ingestion/aggregation-only-contract.md` covering philosophy, invariants, schemas, error codes, migration, observability, and security checklist. | New doc published with compliance checklist; cross-links from existing docs added. |
|
||||
| DOCS-AOC-19-002 | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-001 | Update `/docs/architecture/overview.md` to include AOC boundary, raw stores, and sequence diagram (fetch → guard → raw insert → policy evaluation). | Overview doc updated with diagrams/text; lint passes; stakeholders sign off. |
|
||||
| DOCS-AOC-19-003 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-AOC-19-003 | Refresh `/docs/architecture/policy-engine.md` clarifying ingestion boundary, raw inputs, and policy-only derived data. | Doc highlights raw-only ingestion contract, updated diagrams merge, compliance checklist added. |
|
||||
| DOCS-AOC-19-004 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-AOC-19-001 | Extend `/docs/ui/console.md` with Sources dashboard tiles, violation drill-down workflow, and verification action. | UI doc updated with screenshots/flow descriptions, compliance checklist appended. |
|
||||
> DOCS-AOC-19-004: Architecture overview & policy-engine updates landed 2025-10-26; incorporate the new AOC boundary diagrams and metrics references.
|
||||
| DOCS-AOC-19-005 | DONE (2025-10-26) | Docs Guild, CLI Guild | CLI-AOC-19-003 | Update `/docs/cli/cli-reference.md` with `stella sources ingest --dry-run` and `stella aoc verify` usage, exit codes, and offline notes. | CLI reference + quickstart sections updated; examples validated; compliance checklist added. |
|
||||
> DOCS-AOC-19-005: New ingestion reference + architecture overview published 2025-10-26; ensure CLI docs link to both and surface AOC exit codes mapping.
|
||||
| DOCS-AOC-19-006 | DONE (2025-10-26) | Docs Guild, Observability Guild | CONCELIER-WEB-AOC-19-002, EXCITITOR-WEB-AOC-19-002 | Document new metrics/traces/log keys in `/docs/observability/observability.md`. | Observability doc lists new metrics/traces/log fields; dashboards referenced; compliance checklist appended. |
|
||||
| DOCS-AOC-19-007 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update `/docs/security/authority-scopes.md` with new ingestion scopes and tenancy enforcement notes. | Doc reflects new scopes, sample policies updated, compliance checklist added. |
|
||||
| DOCS-AOC-19-008 | DONE (2025-10-26) | Docs Guild, DevOps Guild | DEVOPS-AOC-19-002 | Refresh `/docs/deploy/containers.md` to cover validator enablement, guard env flags, and read-only verify user. | Deploy doc updated; offline kit section mentions validator scripts; compliance checklist appended. |
|
||||
| DOCS-AOC-19-009 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update AOC docs/samples to reflect new `advisory:*`, `vex:*`, and `aoc:verify` scopes. | Docs reference new scopes, samples aligned, compliance checklist updated. |
|
||||
|
||||
## Air-Gapped Mode (Epic 16)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-AIRGAP-56-001 | TODO | Docs Guild, AirGap Controller Guild | AIRGAP-CTL-56-002 | Publish `/docs/airgap/overview.md` outlining modes, lifecycle, responsibilities, and imposed rule banner. | Doc merged; banner present; diagrams included. |
|
||||
| DOCS-AIRGAP-56-002 | TODO | Docs Guild, DevOps Guild | DEVOPS-AIRGAP-56-001 | Author `/docs/airgap/sealing-and-egress.md` covering network policies, EgressPolicy facade usage, and verification steps. | Doc merged; examples validated; banner included. |
|
||||
| DOCS-AIRGAP-56-003 | TODO | Docs Guild, Exporter Guild | EXPORT-AIRGAP-56-001 | Create `/docs/airgap/mirror-bundles.md` describing bundle format, DSSE/TUF/Merkle validation, creation/import workflows. | Doc merged; sample commands verified; banner present. |
|
||||
| DOCS-AIRGAP-56-004 | TODO | Docs Guild, Deployment Guild | DEVOPS-AIRGAP-56-003 | Publish `/docs/airgap/bootstrap.md` detailing Bootstrap Pack creation, validation, and install procedures. | Doc merged; checklist appended; screenshots verified. |
|
||||
| DOCS-AIRGAP-57-001 | TODO | Docs Guild, AirGap Time Guild | AIRGAP-TIME-58-001 | Write `/docs/airgap/staleness-and-time.md` explaining time anchors, drift policies, staleness budgets, and UI indicators. | Doc merged; math checked; banner included. |
|
||||
| DOCS-AIRGAP-57-002 | TODO | Docs Guild, Console Guild | CONSOLE-AIRGAP-57-001 | Publish `/docs/console/airgap.md` covering sealed badge, import wizard, staleness dashboards. | Doc merged; screenshots captured; banner present. |
|
||||
| DOCS-AIRGAP-57-003 | TODO | Docs Guild, CLI Guild | CLI-AIRGAP-57-001 | Publish `/docs/cli/airgap.md` documenting commands, examples, exit codes. | Doc merged; examples validated; banner present. |
|
||||
| DOCS-AIRGAP-57-004 | TODO | Docs Guild, Ops Guild | DEVOPS-AIRGAP-56-002 | Create `/docs/airgap/operations.md` with runbooks for imports, failure recovery, and auditing. | Doc merged; runbooks rehearsed; banner included. |
|
||||
| DOCS-AIRGAP-58-001 | TODO | Docs Guild, Product Guild | CONSOLE-AIRGAP-58-002 | Provide `/docs/airgap/degradation-matrix.md` enumerating feature availability, fallbacks, remediation. | Doc merged; matrix reviewed; banner included. |
|
||||
| DOCS-AIRGAP-58-002 | TODO | Docs Guild, Security Guild | PROV-OBS-54-001 | Update `/docs/security/trust-and-signing.md` with DSSE/TUF roots, rotation, and signed time tokens. | Doc merged; security sign-off recorded; banner present. |
|
||||
| DOCS-AIRGAP-58-003 | TODO | Docs Guild, DevEx Guild | AIRGAP-POL-56-001 | Publish `/docs/dev/airgap-contracts.md` describing EgressPolicy usage, sealed-mode tests, linting. | Doc merged; sample code validated; banner included. |
|
||||
| DOCS-AIRGAP-58-004 | TODO | Docs Guild, Evidence Locker Guild | EVID-OBS-55-001 | Document `/docs/airgap/portable-evidence.md` for exporting/importing portable evidence bundles across enclaves. | Doc merged; verification steps tested; banner present. |
|
||||
|
||||
## SDKs & OpenAPI (Epic 17)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-OAS-61-001 | TODO | Docs Guild, API Contracts Guild | OAS-61-002 | Publish `/docs/api/overview.md` covering auth, tenancy, pagination, idempotency, rate limits with banner. | Doc merged; examples validated; banner present. |
|
||||
| DOCS-OAS-61-002 | TODO | Docs Guild, API Governance Guild | APIGOV-61-001 | Author `/docs/api/conventions.md` capturing naming, errors, filters, sorting, examples. | Doc merged; lint passes; banner included. |
|
||||
| DOCS-OAS-61-003 | TODO | Docs Guild, API Governance Guild | APIGOV-63-001 | Publish `/docs/api/versioning.md` describing SemVer, deprecation headers, migration playbooks. | Doc merged; example headers validated; banner present. |
|
||||
| DOCS-OAS-62-001 | TODO | Docs Guild, Developer Portal Guild | DEVPORT-62-002 | Stand up `/docs/api/reference/` auto-generated site; integrate with portal nav. | Reference site builds; search works; banner included. |
|
||||
| DOCS-SDK-62-001 | TODO | Docs Guild, SDK Generator Guild | SDKGEN-63-001 | Publish `/docs/sdks/overview.md` plus language guides (`typescript.md`, `python.md`, `go.md`, `java.md`). | Docs merged; code samples pulled from tested examples; banner present. |
|
||||
| DOCS-DEVPORT-62-001 | TODO | Docs Guild, Developer Portal Guild | DEVPORT-62-001 | Document `/docs/devportal/publishing.md` for build pipeline, offline bundle steps. | Doc merged; cross-links validated; banner included. |
|
||||
| DOCS-CONTRIB-62-001 | TODO | Docs Guild, API Governance Guild | APIGOV-61-001 | Publish `/docs/contributing/api-contracts.md` detailing how to edit OAS, lint rules, compatibility checks. | Doc merged; banner present; examples validated. |
|
||||
| DOCS-TEST-62-001 | TODO | Docs Guild, Contract Testing Guild | CONTR-62-001 | Author `/docs/testing/contract-testing.md` covering mock server, replay tests, golden fixtures. | Doc merged; references to tooling validated; banner present. |
|
||||
| DOCS-SEC-62-001 | TODO | Docs Guild, Authority Core | AUTH-AIRGAP-56-001 | Update `/docs/security/auth-scopes.md` with OAuth2/PAT scopes, tenancy header usage. | Doc merged; scope tables verified; banner included. |
|
||||
| DOCS-AIRGAP-DEVPORT-64-001 | TODO | Docs Guild, DevPortal Offline Guild | DVOFF-64-001 | Create `/docs/airgap/devportal-offline.md` describing offline bundle usage and verification. | Doc merged; verification steps tested; banner present. |
|
||||
|
||||
## Risk Profiles (Epic 18)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-RISK-66-001 | TODO | Docs Guild, Risk Profile Schema Guild | POLICY-RISK-66-001 | Publish `/docs/risk/overview.md` covering concepts and glossary. | Doc merged with banner; terminology reviewed. |
|
||||
| DOCS-RISK-66-002 | TODO | Docs Guild, Policy Guild | POLICY-RISK-66-003 | Author `/docs/risk/profiles.md` (authoring, versioning, scope). | Doc merged; schema examples validated; banner present. |
|
||||
| DOCS-RISK-66-003 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Publish `/docs/risk/factors.md` cataloging signals, transforms, reducers, TTLs. | Document merged; tables verified; banner included. |
|
||||
| DOCS-RISK-66-004 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-66-002 | Create `/docs/risk/formulas.md` detailing math, normalization, gating, severity. | Doc merged; equations rendered; banner present. |
|
||||
| DOCS-RISK-67-001 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-68-001 | Publish `/docs/risk/explainability.md` showing artifact schema and UI screenshots. | Doc merged; CLI examples validated; banner included. |
|
||||
| DOCS-RISK-67-002 | TODO | Docs Guild, API Guild | POLICY-RISK-67-002 | Produce `/docs/risk/api.md` with endpoint reference/examples. | Doc merged; OAS examples synced; banner present. |
|
||||
| DOCS-RISK-67-003 | TODO | Docs Guild, Console Guild | CONSOLE-RISK-66-001 | Document `/docs/console/risk-ui.md` for authoring, simulation, dashboards. | Doc merged; screenshots updated; banner included. |
|
||||
| DOCS-RISK-67-004 | TODO | Docs Guild, CLI Guild | CLI-RISK-66-001 | Publish `/docs/cli/risk.md` covering CLI workflows. | Doc merged; command examples validated; banner present. |
|
||||
| DOCS-RISK-68-001 | TODO | Docs Guild, Export Guild | RISK-BUNDLE-69-001 | Add `/docs/airgap/risk-bundles.md` for offline factor bundles. | Doc merged; verification steps confirmed; banner included. |
|
||||
| DOCS-RISK-68-002 | TODO | Docs Guild, Security Guild | POLICY-RISK-66-003 | Update `/docs/security/aoc-invariants.md` with risk scoring provenance guarantees. | Doc merged; audit references updated; banner present. |
|
||||
|
||||
## Attestor Console (Epic 19)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-ATTEST-73-001 | TODO | Docs Guild, Attestor Service Guild | ATTEST-TYPES-73-001 | Publish `/docs/attestor/overview.md` with imposed rule banner. | Doc merged; terminology validated. |
|
||||
| DOCS-ATTEST-73-002 | TODO | Docs Guild, Attestation Payloads Guild | ATTEST-TYPES-73-002 | Write `/docs/attestor/payloads.md` with schemas/examples. | Doc merged; examples validated via tests. |
|
||||
| DOCS-ATTEST-73-003 | TODO | Docs Guild, Policy Guild | POLICY-ATTEST-73-002 | Publish `/docs/attestor/policies.md` covering verification policies. | Doc merged; policy examples validated. |
|
||||
| DOCS-ATTEST-73-004 | TODO | Docs Guild, Attestor Service Guild | ATTESTOR-73-002 | Add `/docs/attestor/workflows.md` detailing ingest, verify, bulk operations. | Doc merged; workflows tested. |
|
||||
| DOCS-ATTEST-74-001 | TODO | Docs Guild, KMS Guild | KMS-73-001 | Publish `/docs/attestor/keys-and-issuers.md`. | Doc merged; rotation guidance verified. |
|
||||
| DOCS-ATTEST-74-002 | TODO | Docs Guild, Transparency Guild | TRANSP-74-001 | Document `/docs/attestor/transparency.md` with witness usage/offline validation. | Doc merged; proofs validated. |
|
||||
| DOCS-ATTEST-74-003 | TODO | Docs Guild, Attestor Console Guild | CONSOLE-ATTEST-73-001 | Write `/docs/console/attestor-ui.md` with screenshots/workflows. | Doc merged; screenshots captured; banner present. |
|
||||
| DOCS-ATTEST-74-004 | TODO | Docs Guild, CLI Attestor Guild | CLI-ATTEST-73-001 | Publish `/docs/cli/attest.md` covering CLI usage. | Doc merged; commands validated. |
|
||||
| DOCS-ATTEST-75-001 | TODO | Docs Guild, Export Attestation Guild | EXPORT-ATTEST-75-002 | Add `/docs/attestor/airgap.md` for attestation bundles. | Doc merged; verification steps confirmed. |
|
||||
| DOCS-ATTEST-75-002 | TODO | Docs Guild, Security Guild | ATTESTOR-73-002 | Update `/docs/security/aoc-invariants.md` with attestation invariants. | Doc merged; invariants detailed. |
|
||||
## Policy Engine v2
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-POLICY-20-001 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-000 | Author `/docs/policy/overview.md` covering concepts, inputs/outputs, determinism, and compliance checklist. | Doc published with diagrams + glossary; lint passes; checklist included. |
|
||||
| DOCS-POLICY-20-002 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Write `/docs/policy/dsl.md` with grammar, built-ins, examples, anti-patterns. | DSL doc includes grammar tables, examples, compliance checklist; validated against parser tests. |
|
||||
| DOCS-POLICY-20-003 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-POLICY-20-001 | Publish `/docs/policy/lifecycle.md` describing draft→approve workflow, roles, audit, compliance list. | Lifecycle doc linked from UI/CLI help; approvals roles documented; checklist appended. |
|
||||
| DOCS-POLICY-20-004 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-MODELS-20-001 | Create `/docs/policy/runs.md` detailing run modes, incremental mechanics, cursors, replay. | Run doc includes sequence diagrams + compliance checklist; cross-links to scheduler docs. |
|
||||
| DOCS-POLICY-20-005 | DONE (2025-10-26) | Docs Guild, BE-Base Platform Guild | WEB-POLICY-20-001 | Draft `/docs/api/policy.md` describing endpoints, schemas, error codes. | API doc validated against OpenAPI; examples included; checklist appended. |
|
||||
| DOCS-POLICY-20-006 | DONE (2025-10-26) | Docs Guild, DevEx/CLI Guild | CLI-POLICY-20-002 | Produce `/docs/cli/policy.md` with command usage, exit codes, JSON output contracts. | CLI doc includes examples, exit codes, compliance checklist. |
|
||||
| DOCS-POLICY-20-007 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-POLICY-20-001 | Document `/docs/ui/policy-editor.md` covering editor, simulation, diff workflows, approvals. | UI doc includes screenshots/placeholders, accessibility notes, compliance checklist. |
|
||||
| DOCS-POLICY-20-008 | DONE (2025-10-26) | Docs Guild, Architecture Guild | POLICY-ENGINE-20-003 | Write `/docs/architecture/policy-engine.md` (new epic content) with sequence diagrams, selection strategy, schema. | Architecture doc merged with diagrams; compliance checklist appended; references updated. |
|
||||
| DOCS-POLICY-20-009 | DONE (2025-10-26) | Docs Guild, Observability Guild | POLICY-ENGINE-20-007 | Add `/docs/observability/policy.md` for metrics/traces/logs, sample dashboards. | Observability doc includes metrics tables, dashboard screenshots, checklist. |
|
||||
| DOCS-POLICY-20-010 | DONE (2025-10-26) | Docs Guild, Security Guild | AUTH-POLICY-20-002 | Publish `/docs/security/policy-governance.md` covering scopes, approvals, tenancy, least privilege. | Security doc merged; compliance checklist appended; reviewed by Security Guild. |
|
||||
| DOCS-POLICY-20-011 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Populate `/docs/examples/policies/` with baseline/serverless/internal-only samples and commentary. | Example policies committed with explanations; lint passes; compliance checklist per file. |
|
||||
| DOCS-POLICY-20-012 | DONE (2025-10-26) | Docs Guild, Support Guild | WEB-POLICY-20-003 | Draft `/docs/faq/policy-faq.md` addressing common pitfalls, VEX conflicts, determinism issues. | FAQ published with Q/A entries, cross-links, compliance checklist. |
|
||||
|
||||
## Graph Explorer v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
|
||||
## Link-Not-Merge v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-LNM-22-001 | BLOCKED (2025-10-27) | Docs Guild, Concelier Guild | CONCELIER-LNM-21-001..003 | Author `/docs/advisories/aggregation.md` covering observation vs linkset, conflict handling, AOC requirements, and reviewer checklist. | Draft doc merged with examples + checklist; final sign-off blocked until Concelier schema/API tasks land. |
|
||||
> Blocker (2025-10-27): `CONCELIER-LNM-21-001..003` still TODO; update doc + fixtures once schema/API implementations are available.
|
||||
| DOCS-LNM-22-002 | BLOCKED (2025-10-27) | Docs Guild, Excititor Guild | EXCITITOR-LNM-21-001..003 | Publish `/docs/vex/aggregation.md` describing VEX observation/linkset model, product matching, conflicts. | Draft doc merged with fixtures; final approval blocked until Excititor observation/linkset work ships. |
|
||||
> Blocker (2025-10-27): `EXCITITOR-LNM-21-001..003` remain TODO; refresh doc, fixtures, and examples post-implementation.
|
||||
| DOCS-LNM-22-003 | BLOCKED (2025-10-27) | Docs Guild, BE-Base Platform Guild | WEB-LNM-21-001..003 | Update `/docs/api/advisories.md` and `/docs/api/vex.md` for new endpoints, parameters, errors, exports. | Draft pending gateway/API delivery; unblock once endpoints + OpenAPI specs are available. |
|
||||
> Blocker (2025-10-27): `WEB-LNM-21-001..003` all TODO—no gateway endpoints/OpenAPI to document yet.
|
||||
| DOCS-LNM-22-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-40-001 | Create `/docs/policy/effective-severity.md` detailing severity selection strategies from multiple sources. | Doc merged with policy examples; checklist included. |
|
||||
| DOCS-LNM-22-005 | BLOCKED (2025-10-27) | Docs Guild, UI Guild | UI-LNM-22-001..003 | Document `/docs/ui/evidence-panel.md` with screenshots, conflict badges, accessibility guidance. | Awaiting UI implementation to capture screenshots + flows; unblock once Evidence panel ships. |
|
||||
> Blocker (2025-10-27): `UI-LNM-22-001..003` all TODO; documentation requires final UI states and accessibility audit artifacts.
|
||||
|
||||
## StellaOps Console (Sprint 23)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-CONSOLE-23-001 | DONE (2025-10-26) | Docs Guild, Console Guild | CONSOLE-CORE-23-004 | Publish `/docs/ui/console-overview.md` covering IA, tenant model, global filters, and AOC alignment with compliance checklist. | Doc merged with diagrams + overview tables; checklist appended; Console Guild sign-off. |
|
||||
| DOCS-CONSOLE-23-002 | DONE (2025-10-26) | Docs Guild, Console Guild | DOCS-CONSOLE-23-001 | Author `/docs/ui/navigation.md` detailing routes, breadcrumbs, keyboard shortcuts, deep links, and tenant context switching. | Navigation doc merged with shortcut tables and screenshots; accessibility checklist satisfied. |
|
||||
| DOCS-CONSOLE-23-003 | DONE (2025-10-26) | Docs Guild, SBOM Service Guild, Console Guild | SBOM-CONSOLE-23-001, CONSOLE-FEAT-23-102 | Document `/docs/ui/sbom-explorer.md` (catalog, detail, graph overlays, exports) including compliance checklist and performance tips. | Doc merged with annotated screenshots, export instructions, and overlay examples; checklist appended. |
|
||||
| DOCS-CONSOLE-23-004 | DONE (2025-10-26) | Docs Guild, Concelier Guild, Excititor Guild | CONCELIER-CONSOLE-23-001, EXCITITOR-CONSOLE-23-001 | Produce `/docs/ui/advisories-and-vex.md` explaining aggregation-not-merge, conflict indicators, raw viewers, and provenance banners. | Doc merged; raw JSON examples included; compliance checklist complete. |
|
||||
| DOCS-CONSOLE-23-005 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-CONSOLE-23-001, CONSOLE-FEAT-23-104 | Write `/docs/ui/findings.md` describing filters, saved views, explain drawer, exports, and CLI parity callouts. | Doc merged with filter matrix + explain walkthrough; checklist appended. |
|
||||
| DOCS-CONSOLE-23-006 | DONE (2025-10-26) | Docs Guild, Policy Guild, Product Ops | POLICY-CONSOLE-23-002, CONSOLE-FEAT-23-105 | Publish `/docs/ui/policies.md` with editor, simulation, approvals, compliance checklist, and RBAC mapping. | Doc merged; Monaco screenshots + simulation diff examples included; approval flow described; checklist appended. |
|
||||
| DOCS-CONSOLE-23-007 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-CONSOLE-23-001, CONSOLE-FEAT-23-106 | Document `/docs/ui/runs.md` covering queues, live progress, diffs, retries, evidence downloads, and troubleshooting. | Doc merged with SSE troubleshooting, metrics references, compliance checklist. |
|
||||
| DOCS-CONSOLE-23-008 | DONE (2025-10-26) | Docs Guild, Authority Guild | AUTH-CONSOLE-23-002, CONSOLE-FEAT-23-108 | Draft `/docs/ui/admin.md` describing users/roles, tenants, tokens, integrations, fresh-auth prompts, and RBAC mapping. | Doc merged with tables for scopes vs roles, screenshots, compliance checklist. |
|
||||
| DOCS-CONSOLE-23-009 | DONE (2025-10-27) | Docs Guild, DevOps Guild | DOWNLOADS-CONSOLE-23-001, CONSOLE-FEAT-23-109 | Publish `/docs/ui/downloads.md` listing product images, commands, offline instructions, parity with CLI, and compliance checklist. | Doc merged; manifest sample included; copy-to-clipboard guidance documented; checklist complete. |
|
||||
| DOCS-CONSOLE-23-010 | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DEVOPS-CONSOLE-23-002, CONSOLE-REL-23-301 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, CSP, env vars, health checks) with compliance checklist. | Deploy doc merged; templates validated; CSP guidance included; checklist appended. |
|
||||
| DOCS-CONSOLE-23-011 | DONE (2025-10-28) | Docs Guild, Deployment Guild | DOCS-CONSOLE-23-010 | Update `/docs/install/docker.md` to cover Console image, Compose/Helm usage, offline tarballs, parity with CLI. | Doc updated with new sections; commands validated; compliance checklist appended. |
|
||||
| DOCS-CONSOLE-23-012 | DONE (2025-10-28) | Docs Guild, Security Guild | AUTH-CONSOLE-23-003, WEB-CONSOLE-23-002 | Publish `/docs/security/console-security.md` detailing OIDC flows, scopes, CSP, fresh-auth, evidence handling, and compliance checklist. | Security doc merged; threat model notes included; checklist appended. |
|
||||
| DOCS-CONSOLE-23-013 | DONE (2025-10-28) | Docs Guild, Observability Guild | TELEMETRY-CONSOLE-23-001, CONSOLE-QA-23-403 | Write `/docs/observability/ui-telemetry.md` cataloguing metrics/logs/traces, dashboards, alerts, and feature flags. | Doc merged with instrumentation tables, dashboard screenshots, checklist appended. |
|
||||
| DOCS-CONSOLE-23-014 | DONE (2025-10-28) | Docs Guild, Console Guild, CLI Guild | CONSOLE-DOC-23-502 | Maintain `/docs/cli-vs-ui-parity.md` matrix and integrate CI check guidance. | Matrix published with parity status, CI workflow documented, compliance checklist appended. |
|
||||
> 2025-10-28: Install Docker guide references pending CLI commands (`stella downloads manifest`, `stella downloads mirror`, `stella console status`). Update once CLI parity lands.
|
||||
| DOCS-CONSOLE-23-015 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONSOLE-CORE-23-001, WEB-CONSOLE-23-001 | Produce `/docs/architecture/console.md` describing frontend packages, data flow diagrams, SSE design, performance budgets. | Architecture doc merged with diagrams + compliance checklist; reviewers approve. |
|
||||
| DOCS-CONSOLE-23-016 | DONE (2025-10-28) | Docs Guild, Accessibility Guild | CONSOLE-QA-23-402, CONSOLE-FEAT-23-102 | Refresh `/docs/accessibility.md` with Console-specific keyboard flows, color tokens, testing tools, and compliance checklist updates. | Accessibility doc updated; audits referenced; checklist appended. |
|
||||
> 2025-10-28: Added guide covering keyboard matrix, screen reader behaviour, colour/focus tokens, testing workflow, offline guidance, and compliance checklist.
|
||||
| DOCS-CONSOLE-23-017 | DONE (2025-10-27) | Docs Guild, Console Guild | CONSOLE-FEAT-23-101..109 | Create `/docs/examples/ui-tours.md` providing triage, audit, policy rollout walkthroughs with annotated screenshots and GIFs. | UI tours doc merged; capture instructions + asset placeholders committed; compliance checklist appended. |
|
||||
| DOCS-CONSOLE-23-018 | DONE (2025-10-27) | Docs Guild, Security Guild | DOCS-CONSOLE-23-012 | Execute console security compliance checklist and capture Security Guild sign-off in Sprint 23 log. | Checklist completed; findings addressed or tickets filed; sign-off noted in updates file. |
|
||||
| DOCS-LNM-22-006 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONCELIER-LNM-21-001..005, EXCITITOR-LNM-21-001..005 | Refresh `/docs/architecture/conseiller.md` and `/docs/architecture/excitator.md` describing observation/linkset pipelines and event contracts. | Architecture docs updated with observation/linkset flow + event tables; revisit once service implementations land. |
|
||||
> Follow-up: align diagrams/examples after `CONCELIER-LNM-21` & `EXCITITOR-LNM-21` work merges (currently TODO).
|
||||
| DOCS-LNM-22-007 | TODO | Docs Guild, Observability Guild | CONCELIER-LNM-21-005, EXCITITOR-LNM-21-005, DEVOPS-LNM-22-002 | Publish `/docs/observability/aggregation.md` with metrics/traces/logs/SLOs. | Observability doc merged; dashboards referenced; checklist appended. |
|
||||
| DOCS-LNM-22-008 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, CONCELIER-LNM-21-102 | Write `/docs/migration/no-merge.md` describing migration plan, backfill steps, rollback, feature flags. | Migration doc approved by stakeholders; checklist appended. |
|
||||
|
||||
## Policy Engine + Editor v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-POLICY-23-001 | TODO | Docs Guild, Policy Guild | POLICY-SPL-23-001..003 | Author `/docs/policy/overview.md` describing SPL philosophy, layering, and glossary with reviewer checklist. | Doc merged; lint passes; checklist appended. |
|
||||
| DOCS-POLICY-23-002 | TODO | Docs Guild, Policy Guild | POLICY-SPL-23-001 | Write `/docs/policy/spl-v1.md` (language reference, JSON Schema, examples). | Reference published with schema snippets; checklist completed. |
|
||||
| DOCS-POLICY-23-003 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-50-001..004 | Produce `/docs/policy/runtime.md` covering compiler, evaluator, caching, events, SLOs. | Runtime doc merged with diagrams; observability references included. |
|
||||
| DOCS-POLICY-23-004 | TODO | Docs Guild, UI Guild | UI-POLICY-23-001..006 | Document `/docs/policy/editor.md` (UI walkthrough, validation, simulation, approvals). | Editor doc merged with screenshots; accessibility checklist satisfied. |
|
||||
| DOCS-POLICY-23-005 | TODO | Docs Guild, Security Guild | AUTH-POLICY-23-001..002 | Publish `/docs/policy/governance.md` (roles, scopes, approvals, signing, exceptions). | Governance doc merged; checklist appended. |
|
||||
| DOCS-POLICY-23-006 | TODO | Docs Guild, BE-Base Platform Guild | WEB-POLICY-23-001..004 | Update `/docs/api/policy.md` with new endpoints, schemas, errors, pagination. | API doc aligns with OpenAPI; examples validated; checklist included. |
|
||||
| DOCS-POLICY-23-007 | TODO | Docs Guild, DevEx/CLI Guild | CLI-POLICY-23-004..006 | Update `/docs/cli/policy.md` for lint/simulate/activate/history commands, exit codes. | CLI doc updated; samples verified; checklist appended. |
|
||||
| DOCS-POLICY-23-008 | TODO | Docs Guild, Architecture Guild | POLICY-ENGINE-50-005..006 | Refresh `/docs/architecture/policy-engine.md` with data model, sequence diagrams, event flows. | Architecture doc merged with diagrams; checklist appended. |
|
||||
| DOCS-POLICY-23-009 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, DEVOPS-LNM-22-001 | Create `/docs/migration/policy-parity.md` covering dual-run parity plan and rollback. | Migration doc approved; checklist appended. |
|
||||
| DOCS-POLICY-23-010 | TODO | Docs Guild, UI Guild | UI-POLICY-23-006 | Write `/docs/ui/explainers.md` showing explain trees, evidence overlays, interpretation guidance. | Doc merged with annotated screenshots; checklist appended. |
|
||||
|
||||
## Graph & Vuln Explorer v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-GRAPH-24-001 | TODO | Docs Guild, UI Guild | UI-GRAPH-24-001..006 | Author `/docs/ui/sbom-graph-explorer.md` detailing overlays, filters, saved views, accessibility, and AOC visibility. | Doc merged; screenshots included; checklist appended. |
|
||||
| DOCS-GRAPH-24-002 | TODO | Docs Guild, UI Guild | UI-GRAPH-24-005 | Publish `/docs/ui/vulnerability-explorer.md` covering table usage, grouping, fix suggestions, Why drawer. | Doc merged with annotated images; accessibility checklist satisfied. |
|
||||
| DOCS-GRAPH-24-003 | TODO | Docs Guild, SBOM Service Guild | SBOM-GRAPH-24-001..003 | Create `/docs/architecture/graph-index.md` describing data model, ingestion pipeline, caches, events. | Architecture doc merged with diagrams; checklist appended. |
|
||||
| DOCS-GRAPH-24-004 | TODO | Docs Guild, BE-Base Platform Guild | WEB-GRAPH-24-001..003 | Document `/docs/api/graph.md` and `/docs/api/vuln.md` avec endpoints, parameters, errors, RBAC. | API docs aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-GRAPH-24-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-GRAPH-24-001..003 | Update `/docs/cli/graph-and-vuln.md` covering new CLI commands, exit codes, scripting. | CLI doc merged; examples tested; checklist appended. |
|
||||
| DOCS-GRAPH-24-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-60-001..002 | Write `/docs/policy/ui-integration.md` explaining overlays, cache usage, simulator contracts. | Doc merged; references cross-linked; checklist appended. |
|
||||
| DOCS-GRAPH-24-007 | TODO | Docs Guild, DevOps Guild | DEVOPS-GRAPH-24-001..003 | Produce `/docs/migration/graph-parity.md` with rollout plan, parity checks, fallback guidance. | Migration doc approved; checklist appended. |
|
||||
|
||||
## Exceptions v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-EXC-25-001 | TODO | Docs Guild, Governance Guild | WEB-EXC-25-001 | Author `/docs/governance/exceptions.md` covering lifecycle, scope patterns, examples, compliance checklist. | Doc merged; reviewers sign off; checklist included. |
|
||||
| DOCS-EXC-25-002 | TODO | Docs Guild, Authority Core | AUTH-EXC-25-001 | Publish `/docs/governance/approvals-and-routing.md` detailing roles, routing matrix, MFA rules, audit trails. | Doc merged; routing examples validated; checklist appended. |
|
||||
| DOCS-EXC-25-003 | TODO | Docs Guild, BE-Base Platform Guild | WEB-EXC-25-001..003 | Create `/docs/api/exceptions.md` with endpoints, payloads, errors, idempotency notes. | API doc aligned with OpenAPI; examples tested; checklist appended. |
|
||||
| DOCS-EXC-25-004 | DONE (2025-10-27) | Docs Guild, Policy Guild | POLICY-ENGINE-70-001 | Document `/docs/policy/exception-effects.md` explaining evaluation order, conflicts, simulation. | Doc merged; tests cross-referenced; checklist appended. |
|
||||
| DOCS-EXC-25-005 | TODO | Docs Guild, UI Guild | UI-EXC-25-001..004 | Write `/docs/ui/exception-center.md` with UI walkthrough, badges, accessibility, shortcuts. | Doc merged with screenshots; accessibility checklist completed. |
|
||||
| DOCS-EXC-25-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXC-25-001..002 | Update `/docs/cli/exceptions.md` covering command usage and exit codes. | CLI doc updated; examples validated; checklist appended. |
|
||||
| DOCS-EXC-25-007 | TODO | Docs Guild, DevOps Guild | SCHED-WORKER-25-101, DEVOPS-GRAPH-24-003 | Publish `/docs/migration/exception-governance.md` describing cutover from legacy suppressions, notifications, rollback. | Migration doc approved; checklist included. |
|
||||
|
||||
> Update statuses (TODO/DOING/REVIEW/DONE/BLOCKED) as progress changes. Keep guides in sync with configuration samples under `etc/`.
|
||||
|
||||
> Remark (2025-10-13, DOC4.AUTH-PDG): Rate limit guide published (`docs/security/rate-limits.md`) and handed to plugin docs team for diagram uplift once PLG6.DIAGRAM lands.
|
||||
|
||||
## Orchestrator Dashboard (Epic 9)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-ORCH-32-001 | TODO | Docs Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Author `/docs/orchestrator/overview.md` covering mission, roles, AOC alignment, governance, with imposed rule reminder. | Doc merged with diagrams; imposed rule statement included; entry linked from docs index. |
|
||||
| DOCS-ORCH-32-002 | TODO | Docs Guild | ORCH-SVC-32-002 | Author `/docs/orchestrator/architecture.md` detailing scheduler, DAGs, rate limits, data model, message bus, storage layout, restating imposed rule. | Architecture doc merged; diagrams reviewed; imposed rule noted. |
|
||||
| DOCS-ORCH-33-001 | TODO | Docs Guild | ORCH-SVC-33-001..004, WEB-ORCH-33-001 | Publish `/docs/orchestrator/api.md` (REST/WebSocket endpoints, payloads, error codes) with imposed rule note. | API doc merged; examples validated; imposed rule appended. |
|
||||
| DOCS-ORCH-33-002 | TODO | Docs Guild | CONSOLE-ORCH-32-002, CONSOLE-ORCH-33-001..002 | Publish `/docs/orchestrator/console.md` covering screens, a11y, live updates, control actions, reiterating imposed rule. | Console doc merged with screenshots; accessibility checklist done; imposed rule statement present. |
|
||||
| DOCS-ORCH-33-003 | TODO | Docs Guild | CLI-ORCH-33-001 | Publish `/docs/orchestrator/cli.md` documenting commands, options, exit codes, streaming output, offline usage, and imposed rule. | CLI doc merged; examples tested; imposed rule appended. |
|
||||
| DOCS-ORCH-34-001 | TODO | Docs Guild | ORCH-SVC-34-002, LEDGER-34-101 | Author `/docs/orchestrator/run-ledger.md` covering ledger schema, provenance chain, audit workflows, with imposed rule reminder. | Run-ledger doc merged; payload samples validated; imposed rule included; cross-links added. |
|
||||
| DOCS-ORCH-34-002 | TODO | Docs Guild | AUTH-ORCH-32-001, AUTH-ORCH-34-001 | Update `/docs/security/secrets-handling.md` for orchestrator KMS refs, redaction badges, operator hygiene, reiterating imposed rule. | Security doc merged; checklists updated; imposed rule restated; references from Console/CLI docs added. |
|
||||
| DOCS-ORCH-34-003 | TODO | Docs Guild | ORCH-SVC-33-003, ORCH-SVC-34-001, DEVOPS-ORCH-34-001 | Publish `/docs/operations/orchestrator-runbook.md` (incident playbook, backfill guide, circuit breakers, throttling) with imposed rule statement. | Runbook merged; steps validated with DevOps; imposed rule included; runbook linked from ops index. |
|
||||
| DOCS-ORCH-34-004 | TODO | Docs Guild | ORCH-SVC-32-005, WORKER-GO-33-001, WORKER-PY-33-001 | Document `/docs/schemas/artifacts.md` describing artifact kinds, schema versions, hashing, storage layout, restating imposed rule. | Schema doc merged; JSON schema provided; imposed rule included; sample payload validated. |
|
||||
| DOCS-ORCH-34-005 | TODO | Docs Guild | ORCH-SVC-34-001, DEVOPS-ORCH-34-001 | Author `/docs/slo/orchestrator-slo.md` defining SLOs, burn alerts, measurement, and reiterating imposed rule. | SLO doc merged; dashboard screenshots embedded; imposed rule appended; alerts documented. |
|
||||
|
||||
## Export Center (Epic 10)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-EXPORT-35-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-001..006 | Author `/docs/export-center/overview.md` covering purpose, profiles, security, AOC alignment, surfaces, ending with imposed rule statement. | Doc merged with diagrams/examples; imposed rule line present; index updated. |
|
||||
| DOCS-EXPORT-35-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-002..005 | Publish `/docs/export-center/architecture.md` describing planner, adapters, manifests, signing, distribution flows, restating imposed rule. | Architecture doc merged; sequence diagrams included; rule statement appended. |
|
||||
| DOCS-EXPORT-35-003 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-003..004 | Publish `/docs/export-center/profiles.md` detailing schema fields, examples, compatibility, and imposed rule reminder. | Profiles doc merged; JSON schemas linked; imposed rule noted. |
|
||||
| DOCS-EXPORT-36-004 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001..004, WEB-EXPORT-36-001 | Publish `/docs/export-center/api.md` covering endpoints, payloads, errors, and mention imposed rule. | API doc merged; examples validated; rule included. |
|
||||
| DOCS-EXPORT-36-005 | DONE (2025-10-29) | Docs Guild | CLI-EXPORT-35-001, CLI-EXPORT-36-001 | Publish `/docs/export-center/cli.md` with command reference, CI scripts, verification steps, restating imposed rule. | CLI doc merged; script snippets tested; rule appended. |
|
||||
| DOCS-EXPORT-36-006 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001, DEVOPS-EXPORT-36-001 | Publish `/docs/export-center/trivy-adapter.md` covering field mappings, compatibility matrix, and imposed rule reminder. | Doc merged; mapping tables validated; rule included. |
|
||||
| DOCS-EXPORT-37-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-37-001, DEVOPS-EXPORT-37-001 | Publish `/docs/export-center/mirror-bundles.md` describing filesystem/OCI layouts, delta/encryption, import guide, ending with imposed rule. | Doc merged; diagrams provided; verification steps tested; rule stated. |
|
||||
| DOCS-EXPORT-37-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-005, EXPORT-SVC-37-002 | Publish `/docs/export-center/provenance-and-signing.md` detailing manifests, attestation flow, verification, reiterating imposed rule. | Doc merged; signature examples validated; rule appended. |
|
||||
| DOCS-EXPORT-37-003 | DONE (2025-10-29) | Docs Guild | DEVOPS-EXPORT-37-001 | Publish `/docs/operations/export-runbook.md` covering failures, tuning, capacity planning, with imposed rule reminder. | Runbook merged; procedures validated; rule included. |
|
||||
| DOCS-EXPORT-37-004 | TODO | Docs Guild | AUTH-EXPORT-37-001, EXPORT-SVC-37-002 | Publish `/docs/security/export-hardening.md` outlining RBAC, tenancy, encryption, redaction, restating imposed rule. | Security doc merged; checklist updated; rule appended. |
|
||||
| DOCS-EXPORT-37-101 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXPORT-37-001 | Refresh CLI verification sections once `stella export verify` lands (flags, exit codes, samples). | `docs/export-center/cli.md` & `docs/export-center/provenance-and-signing.md` updated with final command syntax; examples tested; rule reminder retained. |
|
||||
| DOCS-EXPORT-37-102 | TODO | Docs Guild, DevOps Guild | DEVOPS-EXPORT-37-001 | Embed export dashboards/alerts references into provenance/runbook docs after Grafana work ships. | Docs updated with dashboard IDs/alert notes; update logged; rule reminder present. |
|
||||
| DOCS-EXPORT-37-005 | TODO | Docs Guild, Exporter Service Guild | EXPORT-SVC-35-006, DEVOPS-EXPORT-36-001 | Validate Export Center docs against live Trivy/mirror bundles once implementation lands; refresh examples and CLI snippets accordingly. | Real bundle examples recorded; docs updated; verification steps confirmed with production artefacts. |
|
||||
> Note (2025-10-29): Blocked until exporter API (`EXPORT-SVC-35-006`) and Trivy/mirror adapters (`EXPORT-SVC-36-001`, `EXPORT-SVC-37-001`) ship. Requires access to CI smoke outputs (`DEVOPS-EXPORT-36-001`) for verification artifacts.
|
||||
|
||||
## Reachability v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-SIG-26-001 | TODO | Docs Guild, Signals Guild | SIGNALS-24-004 | Write `/docs/signals/reachability.md` covering states, scores, provenance, retention. | Doc merged with diagrams/examples; checklist appended. |
|
||||
| DOCS-SIG-26-002 | TODO | Docs Guild, Signals Guild | SIGNALS-24-002 | Publish `/docs/signals/callgraph-formats.md` with schemas and validation errors. | Doc merged; examples tested; checklist included. |
|
||||
| DOCS-SIG-26-003 | TODO | Docs Guild, Runtime Guild | SIGNALS-24-003 | Create `/docs/signals/runtime-facts.md` detailing agent capabilities, privacy safeguards, opt-in flags. | Doc merged; privacy review done; checklist appended. |
|
||||
| DOCS-SIG-26-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-80-001 | Document `/docs/policy/signals-weighting.md` for SPL predicates and weighting strategies. | Doc merged; sample policies validated; checklist appended. |
|
||||
| DOCS-SIG-26-005 | TODO | Docs Guild, UI Guild | UI-SIG-26-001..003 | Draft `/docs/ui/reachability-overlays.md` with badges, timelines, shortcuts. | Doc merged with screenshots; accessibility checklist completed. |
|
||||
| DOCS-SIG-26-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-SIG-26-001..002 | Update `/docs/cli/reachability.md` for new commands and automation recipes. | Doc merged; examples verified; checklist appended. |
|
||||
| DOCS-SIG-26-007 | TODO | Docs Guild, BE-Base Platform Guild | WEB-SIG-26-001..003 | Publish `/docs/api/signals.md` covering endpoints, payloads, ETags, errors. | API doc aligned with OpenAPI; examples tested; checklist appended. |
|
||||
| DOCS-SIG-26-008 | TODO | Docs Guild, DevOps Guild | DEVOPS-SIG-26-001..002 | Write `/docs/migration/enable-reachability.md` guiding rollout, fallbacks, monitoring. | Migration doc approved; checklist appended. |
|
||||
|
||||
## Policy Studio (Sprint 27)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-POLICY-27-001 | BLOCKED (2025-10-27) | Docs Guild, Policy Guild | REGISTRY-API-27-001, POLICY-ENGINE-27-001 | Publish `/docs/policy/studio-overview.md` covering lifecycle, roles, glossary, and compliance checklist. | Doc merged with diagrams + lifecycle table; checklist appended; stakeholders sign off. |
|
||||
> Blocked by `REGISTRY-API-27-001` and `POLICY-ENGINE-27-001`; need spec + compile data.
|
||||
> Blocker: Registry OpenAPI (`REGISTRY-API-27-001`) and policy compile enrichments (`POLICY-ENGINE-27-001`) are still TODO; need final interfaces before drafting overview.
|
||||
| DOCS-POLICY-27-002 | BLOCKED (2025-10-27) | Docs Guild, Console Guild | CONSOLE-STUDIO-27-001 | Write `/docs/policy/authoring.md` detailing workspace templates, snippets, lint rules, IDE shortcuts, and best practices. | Authoring doc includes annotated screenshots, snippet catalog, compliance checklist. |
|
||||
> Blocked by `CONSOLE-STUDIO-27-001` Studio authoring UI pending.
|
||||
> Blocker: Console Studio authoring UI (`CONSOLE-STUDIO-27-001`) not implemented; awaiting UX to capture flows/snippets.
|
||||
| DOCS-POLICY-27-003 | BLOCKED (2025-10-27) | Docs Guild, Policy Registry Guild | REGISTRY-API-27-007 | Document `/docs/policy/versioning-and-publishing.md` (semver rules, attestations, rollback) with compliance checklist. | Doc merged with flow diagrams; attestation steps documented; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-007` publish/sign pipeline outstanding.
|
||||
> Blocker: Registry publish/sign workflow (`REGISTRY-API-27-007`) pending.
|
||||
| DOCS-POLICY-27-004 | BLOCKED (2025-10-27) | Docs Guild, Scheduler Guild | REGISTRY-API-27-005, SCHED-WORKER-27-301 | Write `/docs/policy/simulation.md` covering quick vs batch sim, thresholds, evidence bundles, CLI examples. | Simulation doc includes charts, sample manifests, checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-005`/`SCHED-WORKER-27-301` batch simulation not ready.
|
||||
> Blocker: Batch simulation APIs/workers (`REGISTRY-API-27-005`, `SCHED-WORKER-27-301`) still TODO.
|
||||
| DOCS-POLICY-27-005 | BLOCKED (2025-10-27) | Docs Guild, Product Ops | REGISTRY-API-27-006 | Publish `/docs/policy/review-and-approval.md` with approver requirements, comments, webhooks, audit trail guidance. | Doc merged with role matrix + webhook schema; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-006` review workflow not implemented.
|
||||
> Blocker: Review workflow (`REGISTRY-API-27-006`) not landed.
|
||||
| DOCS-POLICY-27-006 | BLOCKED (2025-10-27) | Docs Guild, Policy Guild | REGISTRY-API-27-008 | Author `/docs/policy/promotion.md` covering environments, canary, rollback, and monitoring steps. | Promotion doc includes examples + checklist; verified by Policy Ops. |
|
||||
> Blocked by `REGISTRY-API-27-008` promotion APIs pending.
|
||||
> Blocker: Promotion/canary APIs (`REGISTRY-API-27-008`) outstanding.
|
||||
| DOCS-POLICY-27-007 | BLOCKED (2025-10-27) | Docs Guild, DevEx/CLI Guild | CLI-POLICY-27-001..004 | Update `/docs/policy/cli.md` with new commands, JSON schemas, CI usage, and compliance checklist. | CLI doc merged with transcripts; schema references validated; checklist appended. |
|
||||
> Blocked by `CLI-POLICY-27-001..004` CLI commands missing.
|
||||
> Blocker: Policy CLI commands (`CLI-POLICY-27-001..004`) yet to implement.
|
||||
| DOCS-POLICY-27-008 | BLOCKED (2025-10-27) | Docs Guild, Policy Registry Guild | REGISTRY-API-27-001..008 | Publish `/docs/policy/api.md` describing Registry endpoints, request/response schemas, errors, and feature flags. | API doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-001..008` OpenAPI + endpoints incomplete.
|
||||
> Blocker: Registry OpenAPI/spec suite (`REGISTRY-API-27-001..008`) incomplete.
|
||||
| DOCS-POLICY-27-009 | BLOCKED (2025-10-27) | Docs Guild, Security Guild | AUTH-POLICY-27-002 | Create `/docs/security/policy-attestations.md` covering signing, verification, key rotation, and compliance checklist. | Security doc approved by Security Guild; verifier steps documented; checklist appended. |
|
||||
> Blocked by `AUTH-POLICY-27-002` signing enforcement pending.
|
||||
> Blocker: Authority signing enforcement (`AUTH-POLICY-27-002`) pending.
|
||||
| DOCS-POLICY-27-010 | BLOCKED (2025-10-27) | Docs Guild, Architecture Guild | REGISTRY-API-27-001, SCHED-WORKER-27-301 | Author `/docs/architecture/policy-registry.md` (service design, schemas, queues, failure modes) with diagrams and checklist. | Architecture doc merged; diagrams committed; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-001` & `SCHED-WORKER-27-301` need delivery.
|
||||
> Blocker: Policy Registry schema/workers not delivered (see `REGISTRY-API-27-001`, `SCHED-WORKER-27-301`).
|
||||
| DOCS-POLICY-27-011 | BLOCKED (2025-10-27) | Docs Guild, Observability Guild | DEVOPS-POLICY-27-004 | Publish `/docs/observability/policy-telemetry.md` with metrics/log tables, dashboards, alerts, and compliance checklist. | Observability doc merged; dashboards linked; checklist appended. |
|
||||
> Blocked by `DEVOPS-POLICY-27-004` observability dashboards outstanding.
|
||||
> Blocker: Observability dashboards (`DEVOPS-POLICY-27-004`) not built.
|
||||
| DOCS-POLICY-27-012 | BLOCKED (2025-10-27) | Docs Guild, Ops Guild | DEPLOY-POLICY-27-002 | Write `/docs/runbooks/policy-incident.md` detailing rollback, freeze, forensic steps, notifications. | Runbook merged; rehearsal recorded; checklist appended. |
|
||||
> Blocked by `DEPLOY-POLICY-27-002` incident runbook inputs pending.
|
||||
> Blocker: Ops runbook inputs (`DEPLOY-POLICY-27-002`) pending.
|
||||
| DOCS-POLICY-27-013 | BLOCKED (2025-10-27) | Docs Guild, Policy Guild | CONSOLE-STUDIO-27-001, REGISTRY-API-27-002 | Update `/docs/examples/policy-templates.md` with new templates, snippets, and sample policies. | Examples committed with commentary; lint passes; checklist appended. |
|
||||
> Blocked by `CONSOLE-STUDIO-27-001`/`REGISTRY-API-27-002` templates missing.
|
||||
> Blocker: Studio templates and registry storage (`CONSOLE-STUDIO-27-001`, `REGISTRY-API-27-002`) not available.
|
||||
| DOCS-POLICY-27-014 | BLOCKED (2025-10-27) | Docs Guild, Policy Registry Guild | REGISTRY-API-27-003, WEB-POLICY-27-001 | Refresh `/docs/aoc/aoc-guardrails.md` to include Studio-specific guardrails and validation scenarios. | Doc updated with Studio guardrails; compliance checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-003` & `WEB-POLICY-27-001` guardrails not implemented.
|
||||
> Blocker: Registry compile pipeline/web proxy (`REGISTRY-API-27-003`, `WEB-POLICY-27-001`) outstanding.
|
||||
|
||||
## Vulnerability Explorer (Sprint 29)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-VULN-29-001 | TODO | Docs Guild, Vuln Explorer Guild | VULN-API-29-001 | Publish `/docs/vuln/explorer-overview.md` covering domain model, identities, AOC guarantees, workflow summary. | Doc merged with diagrams/table; compliance checklist appended. |
|
||||
| DOCS-VULN-29-002 | TODO | Docs Guild, Console Guild | CONSOLE-VULN-29-001..006 | Write `/docs/vuln/explorer-using-console.md` with workflows, screenshots, keyboard shortcuts, saved views, deep links. | Doc merged; images stored; WCAG notes included; checklist appended. |
|
||||
| DOCS-VULN-29-003 | TODO | Docs Guild, Vuln Explorer API Guild | VULN-API-29-001..009 | Author `/docs/vuln/explorer-api.md` (endpoints, query schema, grouping, errors, rate limits). | Doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-VULN-29-004 | TODO | Docs Guild, DevEx/CLI Guild | CLI-VULN-29-001..005 | Publish `/docs/vuln/explorer-cli.md` with command reference, samples, exit codes, CI snippets. | CLI doc merged; transcripts/JSON outputs validated; checklist appended. |
|
||||
| DOCS-VULN-29-005 | TODO | Docs Guild, Findings Ledger Guild | LEDGER-29-001..009 | Write `/docs/vuln/findings-ledger.md` detailing event schema, hashing, Merkle roots, replay tooling. | Doc merged; compliance checklist appended; audit team sign-off. |
|
||||
| DOCS-VULN-29-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-29-001..003 | Update `/docs/policy/vuln-determinations.md` for new rationale, signals, simulation semantics. | Doc updated; examples validated; checklist appended. |
|
||||
| DOCS-VULN-29-007 | TODO | Docs Guild, Excititor Guild | EXCITITOR-VULN-29-001..004 | Publish `/docs/vex/explorer-integration.md` covering CSAF mapping, suppression precedence, status semantics. | Doc merged; compliance checklist appended. |
|
||||
| DOCS-VULN-29-008 | TODO | Docs Guild, Concelier Guild | CONCELIER-VULN-29-001..004 | Publish `/docs/advisories/explorer-integration.md` covering key normalization, withdrawn handling, provenance. | Doc merged; checklist appended. |
|
||||
| DOCS-VULN-29-009 | TODO | Docs Guild, SBOM Service Guild | SBOM-VULN-29-001..002 | Author `/docs/sbom/vuln-resolution.md` detailing version semantics, scope, paths, safe version hints. | Doc merged; ecosystem tables validated; checklist appended. |
|
||||
| DOCS-VULN-29-010 | TODO | Docs Guild, Observability Guild | VULN-API-29-009, DEVOPS-VULN-29-002 | Publish `/docs/observability/vuln-telemetry.md` (metrics, logs, tracing, dashboards, SLOs). | Doc merged; dashboards linked; checklist appended. |
|
||||
| DOCS-VULN-29-011 | TODO | Docs Guild, Security Guild | AUTH-VULN-29-001..003 | Create `/docs/security/vuln-rbac.md` for roles, ABAC policies, attachment encryption, CSRF. | Security doc approved; checklist appended. |
|
||||
| DOCS-VULN-29-012 | TODO | Docs Guild, Ops Guild | DEVOPS-VULN-29-002, SCHED-WORKER-29-003 | Write `/docs/runbooks/vuln-ops.md` (projector lag, resolver storms, export failures, policy activation). | Runbook merged; rehearsal recorded; checklist appended. |
|
||||
| DOCS-VULN-29-013 | TODO | Docs Guild, Deployment Guild | DEPLOY-VULN-29-001..002 | Update `/docs/install/containers.md` with Findings Ledger & Vuln Explorer API images, manifests, resource sizing, health checks. | Install doc updated; validation commands included; checklist appended. |
|
||||
|
||||
## VEX Lens (Sprint 30)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-VEX-30-001 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-005 | Publish `/docs/vex/consensus-overview.md` describing purpose, scope, AOC guarantees. | Doc merged with diagrams/terminology tables; compliance checklist appended. |
|
||||
| DOCS-VEX-30-002 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-005 | Author `/docs/vex/consensus-algorithm.md` covering normalization, weighting, thresholds, examples. | Doc merged; math reviewed by Policy; checklist appended. |
|
||||
| DOCS-VEX-30-003 | TODO | Docs Guild, Issuer Directory Guild | ISSUER-30-001..003 | Document `/docs/vex/issuer-directory.md` (issuer management, keys, trust overrides, audit). | Doc merged; security review done; checklist appended. |
|
||||
| DOCS-VEX-30-004 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-007 | Publish `/docs/vex/consensus-api.md` with endpoint specs, query params, rate limits. | API doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-VEX-30-005 | TODO | Docs Guild, Console Guild | CONSOLE-VEX-30-001 | Write `/docs/vex/consensus-console.md` covering UI workflows, filters, conflicts, accessibility. | Doc merged; screenshots added; checklist appended. |
|
||||
| DOCS-VEX-30-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-29-001, VEXLENS-30-004 | Add `/docs/policy/vex-trust-model.md` detailing policy knobs, thresholds, simulation. | Doc merged; policy review completed; checklist appended. |
|
||||
| DOCS-VEX-30-007 | TODO | Docs Guild, SBOM Service Guild | VEXLENS-30-002 | Publish `/docs/sbom/vex-mapping.md` (CPE→purl strategy, edge cases, overrides). | Doc merged; mapping tables validated; checklist appended. |
|
||||
| DOCS-VEX-30-008 | TODO | Docs Guild, Security Guild | ISSUER-30-002, VEXLENS-30-003 | Deliver `/docs/security/vex-signatures.md` (verification flow, key rotation, audit). | Doc approved by Security; checklist appended. |
|
||||
| DOCS-VEX-30-009 | TODO | Docs Guild, DevOps Guild | VEXLENS-30-009, DEVOPS-VEX-30-001 | Create `/docs/runbooks/vex-ops.md` for recompute storms, mapping failures, signature errors. | Runbook merged; rehearsal logged; checklist appended. |
|
||||
|
||||
## Advisory AI (Sprint 31)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-AIAI-31-001 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-006 | Publish `/docs/advisory-ai/overview.md` covering capabilities, guardrails, RBAC. | Doc merged with diagrams; compliance checklist appended. |
|
||||
| DOCS-AIAI-31-002 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-004 | Author `/docs/advisory-ai/architecture.md` detailing RAG pipeline, deterministics, caching, model options. | Doc merged; architecture review done; checklist appended. |
|
||||
| DOCS-AIAI-31-003 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-006 | Write `/docs/advisory-ai/api.md` describing endpoints, schemas, errors, rate limits. | API doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-AIAI-31-004 | TODO | Docs Guild, Console Guild | CONSOLE-VULN-29-001, CONSOLE-VEX-30-001 | Create `/docs/advisory-ai/console.md` with screenshots, a11y notes, copy-as-ticket instructions. | Doc merged; images stored; checklist appended. |
|
||||
| DOCS-AIAI-31-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-VULN-29-001, CLI-VEX-30-001 | Publish `/docs/advisory-ai/cli.md` covering commands, exit codes, scripting patterns. | Doc merged; examples tested; checklist appended. |
|
||||
| DOCS-AIAI-31-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-31-001 | Update `/docs/policy/assistant-parameters.md` covering temperature, token limits, ranking weights, TTLs. | Doc merged; policy review done; checklist appended. |
|
||||
| DOCS-AIAI-31-007 | TODO | Docs Guild, Security Guild | AIAI-31-005 | Write `/docs/security/assistant-guardrails.md` detailing redaction, injection defense, logging. | Doc approved by Security; checklist appended. |
|
||||
| DOCS-AIAI-31-008 | TODO | Docs Guild, SBOM Service Guild | SBOM-AIAI-31-001 | Publish `/docs/sbom/remediation-heuristics.md` (feasibility scoring, blast radius). | Doc merged; heuristics reviewed; checklist appended. |
|
||||
| DOCS-AIAI-31-009 | TODO | Docs Guild, DevOps Guild | DEVOPS-AIAI-31-001 | Create `/docs/runbooks/assistant-ops.md` for warmup, cache priming, model outages, scaling. | Runbook merged; rehearsal logged; checklist appended. |
|
||||
|
||||
## Notifications Studio
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-NOTIFY-38-001 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-38-001..004 | Publish `/docs/notifications/overview.md` and `/docs/notifications/architecture.md`, each ending with imposed rule reminder. | Docs merged; diagrams verified; imposed rule appended. |
|
||||
| DOCS-NOTIFY-39-002 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-39-001..004 | Publish `/docs/notifications/rules.md`, `/docs/notifications/templates.md`, `/docs/notifications/digests.md` with examples and imposed rule line. | Docs merged; examples validated; imposed rule appended. |
|
||||
| DOCS-NOTIFY-40-001 | TODO | Docs Guild, Security Guild | AUTH-NOTIFY-38-001, NOTIFY-SVC-40-001..004 | Publish `/docs/notifications/channels.md`, `/docs/notifications/escalations.md`, `/docs/notifications/api.md`, `/docs/operations/notifier-runbook.md`, `/docs/security/notifications-hardening.md`; each ends with imposed rule line. | Docs merged; accessibility checks passed; imposed rule appended. |
|
||||
|
||||
## CLI Parity & Task Packs
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-CLI-41-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-CORE-41-001 | Publish `/docs/cli/overview.md`, `/docs/cli/configuration.md`, `/docs/cli/output-and-exit-codes.md` with imposed rule statements. | Docs merged; examples verified; imposed rule appended. |
|
||||
| DOCS-CLI-42-001 | TODO | Docs Guild | DOCS-CLI-41-001, CLI-PARITY-41-001 | Publish `/docs/cli/parity-matrix.md` and command guides under `/docs/cli/commands/*.md` (policy, sbom, vuln, vex, advisory, export, orchestrator, notify, aoc, auth). | Guides merged; parity automation documented; imposed rule appended. |
|
||||
| DOCS-PACKS-43-001 | DONE (2025-10-27) | Docs Guild, Task Runner Guild | PACKS-REG-42-001, TASKRUN-42-001 | Publish `/docs/task-packs/spec.md`, `/docs/task-packs/authoring-guide.md`, `/docs/task-packs/registry.md`, `/docs/task-packs/runbook.md`, `/docs/security/pack-signing-and-rbac.md`, `/docs/operations/cli-release-and-packaging.md` with imposed rule statements. | Docs merged; tutorials validated; imposed rule appended; cross-links added. |
|
||||
|
||||
## Containerized Distribution (Epic 13)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-INSTALL-44-001 | TODO | Docs Guild, Deployment Guild | COMPOSE-44-001 | Publish `/docs/install/overview.md` and `/docs/install/compose-quickstart.md` with imposed rule line and copy-ready commands. | Docs merged; screenshots/commands verified; imposed rule appended. |
|
||||
| DOCS-INSTALL-45-001 | TODO | Docs Guild, Deployment Guild | HELM-45-001 | Publish `/docs/install/helm-prod.md` and `/docs/install/configuration-reference.md` with values tables and imposed rule reminder. | Docs merged; configuration matrix verified; imposed rule appended. |
|
||||
| DOCS-INSTALL-46-001 | TODO | Docs Guild, Security Guild | DEPLOY-PACKS-43-001, CLI-PACKS-43-001 | Publish `/docs/install/airgap.md`, `/docs/security/supply-chain.md`, `/docs/operations/health-and-readiness.md`, `/docs/release/image-catalog.md`, `/docs/console/onboarding.md` (each with imposed rule). | Docs merged; checksum/signature sections validated; imposed rule appended. |
|
||||
|
||||
## Authority-Backed Scopes & Tenancy (Epic 14)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-TEN-47-001 | TODO | Docs Guild, Authority Core | AUTH-TEN-47-001 | Publish `/docs/security/tenancy-overview.md` and `/docs/security/scopes-and-roles.md` outlining scope grammar, tenant model, imposed rule reminder. | Docs merged; diagrams included; imposed rule appended. |
|
||||
| DOCS-TEN-48-001 | TODO | Docs Guild, Platform Ops | WEB-TEN-48-001 | Publish `/docs/operations/multi-tenancy.md`, `/docs/operations/rls-and-data-isolation.md`, `/docs/console/admin-tenants.md`. | Docs merged; examples validated; imposed rule appended. |
|
||||
| DOCS-TEN-49-001 | TODO | Docs & DevEx Guilds | CLI-TEN-47-001, AUTH-TEN-49-001 | Publish `/docs/cli/authentication.md`, `/docs/api/authentication.md`, `/docs/policy/examples/abac-overlays.md`, update `/docs/install/configuration-reference.md` with new env vars, all ending with imposed rule line. | Docs merged; command examples verified; imposed rule appended. |
|
||||
# Docs Guild Task Board (UTC 2025-10-10)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOC7.README-INDEX | DONE (2025-10-17) | Docs Guild | — | Refresh index docs (docs/README.md + root README) after architecture dossier split and Offline Kit overhaul. | ✅ ToC reflects new component architecture docs; ✅ root README highlights updated doc set; ✅ Offline Kit guide linked correctly. |
|
||||
| DOC4.AUTH-PDG | DONE (2025-10-19) | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. |
|
||||
| DOC1.AUTH | DONE (2025-10-12) | Docs Guild, Authority Core | CORE5B.DOC | Draft `docs/11_AUTHORITY.md` covering architecture, configuration, bootstrap flows. | ✅ Architecture + config sections approved by Core; ✅ Samples reference latest options; ✅ Offline note added. |
|
||||
| DOC3.Concelier-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Concelier authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. |
|
||||
| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. |
|
||||
| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Concelier conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/modules/concelier/operations/conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Concelier team. |
|
||||
| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/modules/concelier/operations/authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. |
|
||||
| DOCS-ADR-09-001 | DONE (2025-10-19) | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted (`docs/updates/2025-10-18-docs-guild.md`). |
|
||||
| DOCS-EVENTS-09-002 | DONE (2025-10-19) | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated (Ajv CI hooked); docs/events/README summarises usage; Platform Events notified via `docs/updates/2025-10-18-docs-guild.md`. |
|
||||
| DOCS-EVENTS-09-003 | DONE (2025-10-19) | Docs Guild | DOCS-EVENTS-09-002 | Add human-readable envelope field references and canonical payload samples for published events, including offline validation workflow. | Tables explain common headers/payload segments; versioned sample payloads committed; README links to validation instructions and samples. |
|
||||
| DOCS-EVENTS-09-004 | DONE (2025-10-19) | Docs Guild, Scanner WebService | SCANNER-EVENTS-15-201 | Refresh scanner event docs to mirror DSSE-backed report fields, document `scanner.scan.completed`, and capture canonical sample validation. | Schemas updated for new payload shape; README references DSSE reuse and validation test; samples align with emitted events. |
|
||||
| PLATFORM-EVENTS-09-401 | DONE (2025-10-21) | Platform Events Guild | DOCS-EVENTS-09-003 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. | Notify models tests now run schema validation against `docs/events/*.json`, event schemas allow optional `attributes`, and docs capture the new validation workflow. |
|
||||
| RUNTIME-GUILD-09-402 | DONE (2025-10-19) | Runtime Guild | SCANNER-POLICY-09-107 | Confirm Scanner WebService surfaces `quietedFindingCount` and progress hints to runtime consumers; document readiness checklist. | Runtime verification run captures enriched payload; checklist/doc updates merged; stakeholders acknowledge availability. |
|
||||
| DOCS-CONCELIER-07-201 | DONE (2025-10-22) | Docs Guild, Concelier WebService | FEEDWEB-DOCS-01-001 | Final editorial review and publish pass for Concelier authority toggle documentation (Quickstart + operator guide). | Review feedback resolved, publish PR merged, release notes updated with documentation pointer. |
|
||||
| DOCS-RUNTIME-17-004 | DONE (2025-10-26) | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads (`process.buildId`), Scanner `/policy/runtime` response (`buildIds` list), debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections (Observer, Scanner, CLI), examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides + CLI help. |
|
||||
| DOCS-OBS-50-001 | BLOCKED (2025-10-26) | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Publish `/docs/observability/overview.md` introducing scope, imposed rule banner, architecture diagram, and tenant guarantees. | Doc merged with imposed rule banner; diagram committed; cross-links to telemetry stack + evidence locker docs. |
|
||||
> Blocked: waiting on telemetry core deliverable (TELEMETRY-OBS-50-001) to finalise architecture details and diagrams.
|
||||
| DOCS-OBS-50-002 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-50-002 | Author `/docs/observability/telemetry-standards.md` detailing common fields, scrubbing policy, sampling defaults, and redaction override procedure. | Doc merged; imposed rule banner present; examples validated with telemetry fixtures; security review sign-off captured. |
|
||||
| DOCS-OBS-50-003 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Create `/docs/observability/logging.md` covering structured log schema, dos/don'ts, tenant isolation, and copyable examples. | Doc merged with banner; sample logs redacted; lint passes; linked from coding standards. |
|
||||
| DOCS-OBS-50-004 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-002 | Draft `/docs/observability/tracing.md` explaining context propagation, async linking, CLI header usage, and sampling strategies. | Doc merged; imposed rule banner included; diagrams updated; references to CLI/Console features added. |
|
||||
| DOCS-OBS-51-001 | TODO | Docs Guild, DevOps Guild | WEB-OBS-51-001, DEVOPS-OBS-51-001 | Publish `/docs/observability/metrics-and-slos.md` cataloging metrics, SLO targets, burn rate policies, and alert runbooks. | Doc merged with banner; SLO tables verified; alert workflows linked to incident runbook. |
|
||||
| DOCS-SEC-OBS-50-001 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-51-002 | Update `/docs/security/redaction-and-privacy.md` to cover telemetry privacy controls, tenant opt-in debug, and imposed rule reminder. | Doc merged; redaction matrix updated; banner present; security sign-off recorded. |
|
||||
| DOCS-INSTALL-50-001 | TODO | Docs Guild, DevOps Guild | DEVOPS-OBS-50-003 | Add `/docs/install/telemetry-stack.md` with collector deployment, exporter options, offline kit notes, and imposed rule banner. | Doc merged; install steps verified on air-gapped profile; banner present; screenshots attached. |
|
||||
| DOCS-FORENSICS-53-001 | TODO | Docs Guild, Evidence Locker Guild | EVID-OBS-53-003 | Publish `/docs/forensics/evidence-locker.md` describing bundle formats, WORM options, retention, legal hold, and imposed rule banner. | Doc merged; manifest examples validated; banner present; legal hold steps aligned with API. |
|
||||
| DOCS-FORENSICS-53-002 | TODO | Docs Guild, Provenance Guild | PROV-OBS-54-001 | Release `/docs/forensics/provenance-attestation.md` covering DSSE schema, signing process, verification workflow, and imposed rule banner. | Doc merged; sample statements reference fixtures; banner included; verification steps tested. |
|
||||
| DOCS-FORENSICS-53-003 | TODO | Docs Guild, Timeline Indexer Guild | TIMELINE-OBS-52-003 | Publish `/docs/forensics/timeline.md` with schema, event kinds, filters, query examples, and imposed rule banner. | Doc merged; query examples validated; banner present; linked from Console/CLI docs. |
|
||||
| DOCS-CONSOLE-OBS-52-001 | TODO | Docs Guild, Console Guild | CONSOLE-OBS-51-001 | Document `/docs/console/observability.md` showcasing Observability Hub widgets, trace/log search, imposed rule banner, and accessibility tips. | Doc merged; screenshots updated; banner present; navigation steps verified. |
|
||||
| DOCS-CONSOLE-OBS-52-002 | TODO | Docs Guild, Console Guild | CONSOLE-OBS-52-002, CONSOLE-OBS-53-001 | Publish `/docs/console/forensics.md` covering timeline explorer, evidence viewer, attestation verifier, imposed rule banner, and troubleshooting. | Doc merged; banner included; workflows validated via Playwright capture; troubleshooting section populated. |
|
||||
| DOCS-CLI-OBS-52-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-OBS-52-001 | Create `/docs/modules/cli/guides/observability.md` detailing `stella obs` commands, examples, exit codes, imposed rule banner, and scripting tips. | Doc merged; examples tested; banner included; CLI parity matrix updated. |
|
||||
| DOCS-CLI-FORENSICS-53-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-FORENSICS-54-001 | Publish `/docs/modules/cli/guides/forensics.md` for snapshot/verify/attest commands with sample outputs, imposed rule banner, and offline workflows. | Doc merged; sample bundles verified; banner present; offline notes cross-linked. |
|
||||
| DOCS-RUNBOOK-55-001 | TODO | Docs Guild, Ops Guild | DEVOPS-OBS-55-001, WEB-OBS-55-001 | Author `/docs/runbooks/incidents.md` describing incident mode activation, escalation steps, retention impact, verification checklist, and imposed rule banner. | Doc merged; runbook rehearsed; banner included; linked from alerts. |
|
||||
| DOCS-AOC-19-001 | DONE (2025-10-26) | Docs Guild, Concelier Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Author `/docs/ingestion/aggregation-only-contract.md` covering philosophy, invariants, schemas, error codes, migration, observability, and security checklist. | New doc published with compliance checklist; cross-links from existing docs added. |
|
||||
| DOCS-AOC-19-002 | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-001 | Update `/docs/modules/platform/architecture-overview.md` to include AOC boundary, raw stores, and sequence diagram (fetch → guard → raw insert → policy evaluation). | Overview doc updated with diagrams/text; lint passes; stakeholders sign off. |
|
||||
| DOCS-AOC-19-003 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-AOC-19-003 | Refresh `/docs/modules/policy/architecture.md` clarifying ingestion boundary, raw inputs, and policy-only derived data. | Doc highlights raw-only ingestion contract, updated diagrams merge, compliance checklist added. |
|
||||
| DOCS-AOC-19-004 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-AOC-19-001 | Extend `/docs/ui/console.md` with Sources dashboard tiles, violation drill-down workflow, and verification action. | UI doc updated with screenshots/flow descriptions, compliance checklist appended. |
|
||||
> DOCS-AOC-19-004: Architecture overview & policy-engine updates landed 2025-10-26; incorporate the new AOC boundary diagrams and metrics references.
|
||||
| DOCS-AOC-19-005 | DONE (2025-10-26) | Docs Guild, CLI Guild | CLI-AOC-19-003 | Update `/docs/modules/cli/guides/cli-reference.md` with `stella sources ingest --dry-run` and `stella aoc verify` usage, exit codes, and offline notes. | CLI reference + quickstart sections updated; examples validated; compliance checklist added. |
|
||||
> DOCS-AOC-19-005: New ingestion reference + architecture overview published 2025-10-26; ensure CLI docs link to both and surface AOC exit codes mapping.
|
||||
| DOCS-AOC-19-006 | DONE (2025-10-26) | Docs Guild, Observability Guild | CONCELIER-WEB-AOC-19-002, EXCITITOR-WEB-AOC-19-002 | Document new metrics/traces/log keys in `/docs/observability/observability.md`. | Observability doc lists new metrics/traces/log fields; dashboards referenced; compliance checklist appended. |
|
||||
| DOCS-AOC-19-007 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update `/docs/security/authority-scopes.md` with new ingestion scopes and tenancy enforcement notes. | Doc reflects new scopes, sample policies updated, compliance checklist added. |
|
||||
| DOCS-AOC-19-008 | DONE (2025-10-26) | Docs Guild, DevOps Guild | DEVOPS-AOC-19-002 | Refresh `/docs/deploy/containers.md` to cover validator enablement, guard env flags, and read-only verify user. | Deploy doc updated; offline kit section mentions validator scripts; compliance checklist appended. |
|
||||
| DOCS-AOC-19-009 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update AOC docs/samples to reflect new `advisory:*`, `vex:*`, and `aoc:verify` scopes. | Docs reference new scopes, samples aligned, compliance checklist updated. |
|
||||
|
||||
## Air-Gapped Mode (Epic 16)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-AIRGAP-56-001 | TODO | Docs Guild, AirGap Controller Guild | AIRGAP-CTL-56-002 | Publish `/docs/airgap/overview.md` outlining modes, lifecycle, responsibilities, and imposed rule banner. | Doc merged; banner present; diagrams included. |
|
||||
| DOCS-AIRGAP-56-002 | TODO | Docs Guild, DevOps Guild | DEVOPS-AIRGAP-56-001 | Author `/docs/airgap/sealing-and-egress.md` covering network policies, EgressPolicy facade usage, and verification steps. | Doc merged; examples validated; banner included. |
|
||||
| DOCS-AIRGAP-56-003 | TODO | Docs Guild, Exporter Guild | EXPORT-AIRGAP-56-001 | Create `/docs/airgap/mirror-bundles.md` describing bundle format, DSSE/TUF/Merkle validation, creation/import workflows. | Doc merged; sample commands verified; banner present. |
|
||||
| DOCS-AIRGAP-56-004 | TODO | Docs Guild, Deployment Guild | DEVOPS-AIRGAP-56-003 | Publish `/docs/airgap/bootstrap.md` detailing Bootstrap Pack creation, validation, and install procedures. | Doc merged; checklist appended; screenshots verified. |
|
||||
| DOCS-AIRGAP-57-001 | TODO | Docs Guild, AirGap Time Guild | AIRGAP-TIME-58-001 | Write `/docs/airgap/staleness-and-time.md` explaining time anchors, drift policies, staleness budgets, and UI indicators. | Doc merged; math checked; banner included. |
|
||||
| DOCS-AIRGAP-57-002 | TODO | Docs Guild, Console Guild | CONSOLE-AIRGAP-57-001 | Publish `/docs/console/airgap.md` covering sealed badge, import wizard, staleness dashboards. | Doc merged; screenshots captured; banner present. |
|
||||
| DOCS-AIRGAP-57-003 | TODO | Docs Guild, CLI Guild | CLI-AIRGAP-57-001 | Publish `/docs/modules/cli/guides/airgap.md` documenting commands, examples, exit codes. | Doc merged; examples validated; banner present. |
|
||||
| DOCS-AIRGAP-57-004 | TODO | Docs Guild, Ops Guild | DEVOPS-AIRGAP-56-002 | Create `/docs/airgap/operations.md` with runbooks for imports, failure recovery, and auditing. | Doc merged; runbooks rehearsed; banner included. |
|
||||
| DOCS-AIRGAP-58-001 | TODO | Docs Guild, Product Guild | CONSOLE-AIRGAP-58-002 | Provide `/docs/airgap/degradation-matrix.md` enumerating feature availability, fallbacks, remediation. | Doc merged; matrix reviewed; banner included. |
|
||||
| DOCS-AIRGAP-58-002 | TODO | Docs Guild, Security Guild | PROV-OBS-54-001 | Update `/docs/security/trust-and-signing.md` with DSSE/TUF roots, rotation, and signed time tokens. | Doc merged; security sign-off recorded; banner present. |
|
||||
| DOCS-AIRGAP-58-003 | TODO | Docs Guild, DevEx Guild | AIRGAP-POL-56-001 | Publish `/docs/dev/airgap-contracts.md` describing EgressPolicy usage, sealed-mode tests, linting. | Doc merged; sample code validated; banner included. |
|
||||
| DOCS-AIRGAP-58-004 | TODO | Docs Guild, Evidence Locker Guild | EVID-OBS-55-001 | Document `/docs/airgap/portable-evidence.md` for exporting/importing portable evidence bundles across enclaves. | Doc merged; verification steps tested; banner present. |
|
||||
|
||||
## SDKs & OpenAPI (Epic 17)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-OAS-61-001 | TODO | Docs Guild, API Contracts Guild | OAS-61-002 | Publish `/docs/api/overview.md` covering auth, tenancy, pagination, idempotency, rate limits with banner. | Doc merged; examples validated; banner present. |
|
||||
| DOCS-OAS-61-002 | TODO | Docs Guild, API Governance Guild | APIGOV-61-001 | Author `/docs/api/conventions.md` capturing naming, errors, filters, sorting, examples. | Doc merged; lint passes; banner included. |
|
||||
| DOCS-OAS-61-003 | TODO | Docs Guild, API Governance Guild | APIGOV-63-001 | Publish `/docs/api/versioning.md` describing SemVer, deprecation headers, migration playbooks. | Doc merged; example headers validated; banner present. |
|
||||
| DOCS-OAS-62-001 | TODO | Docs Guild, Developer Portal Guild | DEVPORT-62-002 | Stand up `/docs/api/reference/` auto-generated site; integrate with portal nav. | Reference site builds; search works; banner included. |
|
||||
| DOCS-SDK-62-001 | TODO | Docs Guild, SDK Generator Guild | SDKGEN-63-001 | Publish `/docs/sdks/overview.md` plus language guides (`typescript.md`, `python.md`, `go.md`, `java.md`). | Docs merged; code samples pulled from tested examples; banner present. |
|
||||
| DOCS-DEVPORT-62-001 | TODO | Docs Guild, Developer Portal Guild | DEVPORT-62-001 | Document `/docs/devportal/publishing.md` for build pipeline, offline bundle steps. | Doc merged; cross-links validated; banner included. |
|
||||
| DOCS-CONTRIB-62-001 | TODO | Docs Guild, API Governance Guild | APIGOV-61-001 | Publish `/docs/contributing/api-contracts.md` detailing how to edit OAS, lint rules, compatibility checks. | Doc merged; banner present; examples validated. |
|
||||
| DOCS-TEST-62-001 | TODO | Docs Guild, Contract Testing Guild | CONTR-62-001 | Author `/docs/testing/contract-testing.md` covering mock server, replay tests, golden fixtures. | Doc merged; references to tooling validated; banner present. |
|
||||
| DOCS-SEC-62-001 | TODO | Docs Guild, Authority Core | AUTH-AIRGAP-56-001 | Update `/docs/security/auth-scopes.md` with OAuth2/PAT scopes, tenancy header usage. | Doc merged; scope tables verified; banner included. |
|
||||
| DOCS-AIRGAP-DEVPORT-64-001 | TODO | Docs Guild, DevPortal Offline Guild | DVOFF-64-001 | Create `/docs/airgap/devportal-offline.md` describing offline bundle usage and verification. | Doc merged; verification steps tested; banner present. |
|
||||
|
||||
## Risk Profiles (Epic 18)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-RISK-66-001 | TODO | Docs Guild, Risk Profile Schema Guild | POLICY-RISK-66-001 | Publish `/docs/risk/overview.md` covering concepts and glossary. | Doc merged with banner; terminology reviewed. |
|
||||
| DOCS-RISK-66-002 | TODO | Docs Guild, Policy Guild | POLICY-RISK-66-003 | Author `/docs/risk/profiles.md` (authoring, versioning, scope). | Doc merged; schema examples validated; banner present. |
|
||||
| DOCS-RISK-66-003 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Publish `/docs/risk/factors.md` cataloging signals, transforms, reducers, TTLs. | Document merged; tables verified; banner included. |
|
||||
| DOCS-RISK-66-004 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-66-002 | Create `/docs/risk/formulas.md` detailing math, normalization, gating, severity. | Doc merged; equations rendered; banner present. |
|
||||
| DOCS-RISK-67-001 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-68-001 | Publish `/docs/risk/explainability.md` showing artifact schema and UI screenshots. | Doc merged; CLI examples validated; banner included. |
|
||||
| DOCS-RISK-67-002 | TODO | Docs Guild, API Guild | POLICY-RISK-67-002 | Produce `/docs/risk/api.md` with endpoint reference/examples. | Doc merged; OAS examples synced; banner present. |
|
||||
| DOCS-RISK-67-003 | TODO | Docs Guild, Console Guild | CONSOLE-RISK-66-001 | Document `/docs/console/risk-ui.md` for authoring, simulation, dashboards. | Doc merged; screenshots updated; banner included. |
|
||||
| DOCS-RISK-67-004 | TODO | Docs Guild, CLI Guild | CLI-RISK-66-001 | Publish `/docs/modules/cli/guides/risk.md` covering CLI workflows. | Doc merged; command examples validated; banner present. |
|
||||
| DOCS-RISK-68-001 | TODO | Docs Guild, Export Guild | RISK-BUNDLE-69-001 | Add `/docs/airgap/risk-bundles.md` for offline factor bundles. | Doc merged; verification steps confirmed; banner included. |
|
||||
| DOCS-RISK-68-002 | TODO | Docs Guild, Security Guild | POLICY-RISK-66-003 | Update `/docs/security/aoc-invariants.md` with risk scoring provenance guarantees. | Doc merged; audit references updated; banner present. |
|
||||
|
||||
## Attestor Console (Epic 19)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-ATTEST-73-001 | TODO | Docs Guild, Attestor Service Guild | ATTEST-TYPES-73-001 | Publish `/docs/modules/attestor/overview.md` with imposed rule banner. | Doc merged; terminology validated. |
|
||||
| DOCS-ATTEST-73-002 | TODO | Docs Guild, Attestation Payloads Guild | ATTEST-TYPES-73-002 | Write `/docs/modules/attestor/payloads.md` with schemas/examples. | Doc merged; examples validated via tests. |
|
||||
| DOCS-ATTEST-73-003 | TODO | Docs Guild, Policy Guild | POLICY-ATTEST-73-002 | Publish `/docs/modules/attestor/policies.md` covering verification policies. | Doc merged; policy examples validated. |
|
||||
| DOCS-ATTEST-73-004 | TODO | Docs Guild, Attestor Service Guild | ATTESTOR-73-002 | Add `/docs/modules/attestor/workflows.md` detailing ingest, verify, bulk operations. | Doc merged; workflows tested. |
|
||||
| DOCS-ATTEST-74-001 | TODO | Docs Guild, KMS Guild | KMS-73-001 | Publish `/docs/modules/attestor/keys-and-issuers.md`. | Doc merged; rotation guidance verified. |
|
||||
| DOCS-ATTEST-74-002 | TODO | Docs Guild, Transparency Guild | TRANSP-74-001 | Document `/docs/modules/attestor/transparency.md` with witness usage/offline validation. | Doc merged; proofs validated. |
|
||||
| DOCS-ATTEST-74-003 | TODO | Docs Guild, Attestor Console Guild | CONSOLE-ATTEST-73-001 | Write `/docs/console/attestor-ui.md` with screenshots/workflows. | Doc merged; screenshots captured; banner present. |
|
||||
| DOCS-ATTEST-74-004 | TODO | Docs Guild, CLI Attestor Guild | CLI-ATTEST-73-001 | Publish `/docs/modules/cli/guides/attest.md` covering CLI usage. | Doc merged; commands validated. |
|
||||
| DOCS-ATTEST-75-001 | TODO | Docs Guild, Export Attestation Guild | EXPORT-ATTEST-75-002 | Add `/docs/modules/attestor/airgap.md` for attestation bundles. | Doc merged; verification steps confirmed. |
|
||||
| DOCS-ATTEST-75-002 | TODO | Docs Guild, Security Guild | ATTESTOR-73-002 | Update `/docs/security/aoc-invariants.md` with attestation invariants. | Doc merged; invariants detailed. |
|
||||
## Policy Engine v2
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-POLICY-20-001 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-000 | Author `/docs/policy/overview.md` covering concepts, inputs/outputs, determinism, and compliance checklist. | Doc published with diagrams + glossary; lint passes; checklist included. |
|
||||
| DOCS-POLICY-20-002 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Write `/docs/policy/dsl.md` with grammar, built-ins, examples, anti-patterns. | DSL doc includes grammar tables, examples, compliance checklist; validated against parser tests. |
|
||||
| DOCS-POLICY-20-003 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-POLICY-20-001 | Publish `/docs/policy/lifecycle.md` describing draft→approve workflow, roles, audit, compliance list. | Lifecycle doc linked from UI/CLI help; approvals roles documented; checklist appended. |
|
||||
| DOCS-POLICY-20-004 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-MODELS-20-001 | Create `/docs/policy/runs.md` detailing run modes, incremental mechanics, cursors, replay. | Run doc includes sequence diagrams + compliance checklist; cross-links to scheduler docs. |
|
||||
| DOCS-POLICY-20-005 | DONE (2025-10-26) | Docs Guild, BE-Base Platform Guild | WEB-POLICY-20-001 | Draft `/docs/api/policy.md` describing endpoints, schemas, error codes. | API doc validated against OpenAPI; examples included; checklist appended. |
|
||||
| DOCS-POLICY-20-006 | DONE (2025-10-26) | Docs Guild, DevEx/CLI Guild | CLI-POLICY-20-002 | Produce `/docs/modules/cli/guides/policy.md` with command usage, exit codes, JSON output contracts. | CLI doc includes examples, exit codes, compliance checklist. |
|
||||
| DOCS-POLICY-20-007 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-POLICY-20-001 | Document `/docs/ui/policy-editor.md` covering editor, simulation, diff workflows, approvals. | UI doc includes screenshots/placeholders, accessibility notes, compliance checklist. |
|
||||
| DOCS-POLICY-20-008 | DONE (2025-10-26) | Docs Guild, Architecture Guild | POLICY-ENGINE-20-003 | Write `/docs/modules/policy/architecture.md` (new epic content) with sequence diagrams, selection strategy, schema. | Architecture doc merged with diagrams; compliance checklist appended; references updated. |
|
||||
| DOCS-POLICY-20-009 | DONE (2025-10-26) | Docs Guild, Observability Guild | POLICY-ENGINE-20-007 | Add `/docs/observability/policy.md` for metrics/traces/logs, sample dashboards. | Observability doc includes metrics tables, dashboard screenshots, checklist. |
|
||||
| DOCS-POLICY-20-010 | DONE (2025-10-26) | Docs Guild, Security Guild | AUTH-POLICY-20-002 | Publish `/docs/security/policy-governance.md` covering scopes, approvals, tenancy, least privilege. | Security doc merged; compliance checklist appended; reviewed by Security Guild. |
|
||||
| DOCS-POLICY-20-011 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Populate `/docs/examples/policies/` with baseline/serverless/internal-only samples and commentary. | Example policies committed with explanations; lint passes; compliance checklist per file. |
|
||||
| DOCS-POLICY-20-012 | DONE (2025-10-26) | Docs Guild, Support Guild | WEB-POLICY-20-003 | Draft `/docs/faq/policy-faq.md` addressing common pitfalls, VEX conflicts, determinism issues. | FAQ published with Q/A entries, cross-links, compliance checklist. |
|
||||
|
||||
## Graph Explorer v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
|
||||
## Link-Not-Merge v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-LNM-22-001 | BLOCKED (2025-10-27) | Docs Guild, Concelier Guild | CONCELIER-LNM-21-001..003 | Author `/docs/advisories/aggregation.md` covering observation vs linkset, conflict handling, AOC requirements, and reviewer checklist. | Draft doc merged with examples + checklist; final sign-off blocked until Concelier schema/API tasks land. |
|
||||
> Blocker (2025-10-27): `CONCELIER-LNM-21-001..003` still TODO; update doc + fixtures once schema/API implementations are available.
|
||||
| DOCS-LNM-22-002 | BLOCKED (2025-10-27) | Docs Guild, Excititor Guild | EXCITITOR-LNM-21-001..003 | Publish `/docs/vex/aggregation.md` describing VEX observation/linkset model, product matching, conflicts. | Draft doc merged with fixtures; final approval blocked until Excititor observation/linkset work ships. |
|
||||
> Blocker (2025-10-27): `EXCITITOR-LNM-21-001..003` remain TODO; refresh doc, fixtures, and examples post-implementation.
|
||||
| DOCS-LNM-22-003 | BLOCKED (2025-10-27) | Docs Guild, BE-Base Platform Guild | WEB-LNM-21-001..003 | Update `/docs/api/advisories.md` and `/docs/api/vex.md` for new endpoints, parameters, errors, exports. | Draft pending gateway/API delivery; unblock once endpoints + OpenAPI specs are available. |
|
||||
> Blocker (2025-10-27): `WEB-LNM-21-001..003` all TODO—no gateway endpoints/OpenAPI to document yet.
|
||||
| DOCS-LNM-22-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-40-001 | Create `/docs/policy/effective-severity.md` detailing severity selection strategies from multiple sources. | Doc merged with policy examples; checklist included. |
|
||||
| DOCS-LNM-22-005 | BLOCKED (2025-10-27) | Docs Guild, UI Guild | UI-LNM-22-001..003 | Document `/docs/ui/evidence-panel.md` with screenshots, conflict badges, accessibility guidance. | Awaiting UI implementation to capture screenshots + flows; unblock once Evidence panel ships. |
|
||||
> Blocker (2025-10-27): `UI-LNM-22-001..003` all TODO; documentation requires final UI states and accessibility audit artifacts.
|
||||
|
||||
## StellaOps Console (Sprint 23)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-CONSOLE-23-001 | DONE (2025-10-26) | Docs Guild, Console Guild | CONSOLE-CORE-23-004 | Publish `/docs/ui/console-overview.md` covering IA, tenant model, global filters, and AOC alignment with compliance checklist. | Doc merged with diagrams + overview tables; checklist appended; Console Guild sign-off. |
|
||||
| DOCS-CONSOLE-23-002 | DONE (2025-10-26) | Docs Guild, Console Guild | DOCS-CONSOLE-23-001 | Author `/docs/ui/navigation.md` detailing routes, breadcrumbs, keyboard shortcuts, deep links, and tenant context switching. | Navigation doc merged with shortcut tables and screenshots; accessibility checklist satisfied. |
|
||||
| DOCS-CONSOLE-23-003 | DONE (2025-10-26) | Docs Guild, SBOM Service Guild, Console Guild | SBOM-CONSOLE-23-001, CONSOLE-FEAT-23-102 | Document `/docs/ui/sbom-explorer.md` (catalog, detail, graph overlays, exports) including compliance checklist and performance tips. | Doc merged with annotated screenshots, export instructions, and overlay examples; checklist appended. |
|
||||
| DOCS-CONSOLE-23-004 | DONE (2025-10-26) | Docs Guild, Concelier Guild, Excititor Guild | CONCELIER-CONSOLE-23-001, EXCITITOR-CONSOLE-23-001 | Produce `/docs/ui/advisories-and-vex.md` explaining aggregation-not-merge, conflict indicators, raw viewers, and provenance banners. | Doc merged; raw JSON examples included; compliance checklist complete. |
|
||||
| DOCS-CONSOLE-23-005 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-CONSOLE-23-001, CONSOLE-FEAT-23-104 | Write `/docs/ui/findings.md` describing filters, saved views, explain drawer, exports, and CLI parity callouts. | Doc merged with filter matrix + explain walkthrough; checklist appended. |
|
||||
| DOCS-CONSOLE-23-006 | DONE (2025-10-26) | Docs Guild, Policy Guild, Product Ops | POLICY-CONSOLE-23-002, CONSOLE-FEAT-23-105 | Publish `/docs/ui/policies.md` with editor, simulation, approvals, compliance checklist, and RBAC mapping. | Doc merged; Monaco screenshots + simulation diff examples included; approval flow described; checklist appended. |
|
||||
| DOCS-CONSOLE-23-007 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-CONSOLE-23-001, CONSOLE-FEAT-23-106 | Document `/docs/ui/runs.md` covering queues, live progress, diffs, retries, evidence downloads, and troubleshooting. | Doc merged with SSE troubleshooting, metrics references, compliance checklist. |
|
||||
| DOCS-CONSOLE-23-008 | DONE (2025-10-26) | Docs Guild, Authority Guild | AUTH-CONSOLE-23-002, CONSOLE-FEAT-23-108 | Draft `/docs/ui/admin.md` describing users/roles, tenants, tokens, integrations, fresh-auth prompts, and RBAC mapping. | Doc merged with tables for scopes vs roles, screenshots, compliance checklist. |
|
||||
| DOCS-CONSOLE-23-009 | DONE (2025-10-27) | Docs Guild, DevOps Guild | DOWNLOADS-CONSOLE-23-001, CONSOLE-FEAT-23-109 | Publish `/docs/ui/downloads.md` listing product images, commands, offline instructions, parity with CLI, and compliance checklist. | Doc merged; manifest sample included; copy-to-clipboard guidance documented; checklist complete. |
|
||||
| DOCS-CONSOLE-23-010 | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DEVOPS-CONSOLE-23-002, CONSOLE-REL-23-301 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, CSP, env vars, health checks) with compliance checklist. | Deploy doc merged; templates validated; CSP guidance included; checklist appended. |
|
||||
| DOCS-CONSOLE-23-011 | DONE (2025-10-28) | Docs Guild, Deployment Guild | DOCS-CONSOLE-23-010 | Update `/docs/install/docker.md` to cover Console image, Compose/Helm usage, offline tarballs, parity with CLI. | Doc updated with new sections; commands validated; compliance checklist appended. |
|
||||
| DOCS-CONSOLE-23-012 | DONE (2025-10-28) | Docs Guild, Security Guild | AUTH-CONSOLE-23-003, WEB-CONSOLE-23-002 | Publish `/docs/security/console-security.md` detailing OIDC flows, scopes, CSP, fresh-auth, evidence handling, and compliance checklist. | Security doc merged; threat model notes included; checklist appended. |
|
||||
| DOCS-CONSOLE-23-013 | DONE (2025-10-28) | Docs Guild, Observability Guild | TELEMETRY-CONSOLE-23-001, CONSOLE-QA-23-403 | Write `/docs/observability/ui-telemetry.md` cataloguing metrics/logs/traces, dashboards, alerts, and feature flags. | Doc merged with instrumentation tables, dashboard screenshots, checklist appended. |
|
||||
| DOCS-CONSOLE-23-014 | DONE (2025-10-28) | Docs Guild, Console Guild, CLI Guild | CONSOLE-DOC-23-502 | Maintain `/docs/cli-vs-ui-parity.md` matrix and integrate CI check guidance. | Matrix published with parity status, CI workflow documented, compliance checklist appended. |
|
||||
> 2025-10-28: Install Docker guide references pending CLI commands (`stella downloads manifest`, `stella downloads mirror`, `stella console status`). Update once CLI parity lands.
|
||||
| DOCS-CONSOLE-23-015 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONSOLE-CORE-23-001, WEB-CONSOLE-23-001 | Produce `/docs/modules/ui/console-architecture.md` describing frontend packages, data flow diagrams, SSE design, performance budgets. | Architecture doc merged with diagrams + compliance checklist; reviewers approve. |
|
||||
| DOCS-CONSOLE-23-016 | DONE (2025-10-28) | Docs Guild, Accessibility Guild | CONSOLE-QA-23-402, CONSOLE-FEAT-23-102 | Refresh `/docs/accessibility.md` with Console-specific keyboard flows, color tokens, testing tools, and compliance checklist updates. | Accessibility doc updated; audits referenced; checklist appended. |
|
||||
> 2025-10-28: Added guide covering keyboard matrix, screen reader behaviour, colour/focus tokens, testing workflow, offline guidance, and compliance checklist.
|
||||
| DOCS-CONSOLE-23-017 | DONE (2025-10-27) | Docs Guild, Console Guild | CONSOLE-FEAT-23-101..109 | Create `/docs/examples/ui-tours.md` providing triage, audit, policy rollout walkthroughs with annotated screenshots and GIFs. | UI tours doc merged; capture instructions + asset placeholders committed; compliance checklist appended. |
|
||||
| DOCS-CONSOLE-23-018 | DONE (2025-10-27) | Docs Guild, Security Guild | DOCS-CONSOLE-23-012 | Execute console security compliance checklist and capture Security Guild sign-off in Sprint 23 log. | Checklist completed; findings addressed or tickets filed; sign-off noted in updates file. |
|
||||
| DOCS-LNM-22-006 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONCELIER-LNM-21-001..005, EXCITITOR-LNM-21-001..005 | Refresh `/docs/modules/concelier/architecture.md` and `/docs/modules/excititor/architecture.md` describing observation/linkset pipelines and event contracts. | Architecture docs updated with observation/linkset flow + event tables; revisit once service implementations land. |
|
||||
> Follow-up: align diagrams/examples after `CONCELIER-LNM-21` & `EXCITITOR-LNM-21` work merges (currently TODO).
|
||||
| DOCS-LNM-22-007 | TODO | Docs Guild, Observability Guild | CONCELIER-LNM-21-005, EXCITITOR-LNM-21-005, DEVOPS-LNM-22-002 | Publish `/docs/observability/aggregation.md` with metrics/traces/logs/SLOs. | Observability doc merged; dashboards referenced; checklist appended. |
|
||||
| DOCS-LNM-22-008 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, CONCELIER-LNM-21-102 | Write `/docs/migration/no-merge.md` describing migration plan, backfill steps, rollback, feature flags. | Migration doc approved by stakeholders; checklist appended. |
|
||||
|
||||
## Policy Engine + Editor v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-POLICY-23-001 | TODO | Docs Guild, Policy Guild | POLICY-SPL-23-001..003 | Author `/docs/policy/overview.md` describing SPL philosophy, layering, and glossary with reviewer checklist. | Doc merged; lint passes; checklist appended. |
|
||||
| DOCS-POLICY-23-002 | TODO | Docs Guild, Policy Guild | POLICY-SPL-23-001 | Write `/docs/policy/spl-v1.md` (language reference, JSON Schema, examples). | Reference published with schema snippets; checklist completed. |
|
||||
| DOCS-POLICY-23-003 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-50-001..004 | Produce `/docs/policy/runtime.md` covering compiler, evaluator, caching, events, SLOs. | Runtime doc merged with diagrams; observability references included. |
|
||||
| DOCS-POLICY-23-004 | TODO | Docs Guild, UI Guild | UI-POLICY-23-001..006 | Document `/docs/policy/editor.md` (UI walkthrough, validation, simulation, approvals). | Editor doc merged with screenshots; accessibility checklist satisfied. |
|
||||
| DOCS-POLICY-23-005 | TODO | Docs Guild, Security Guild | AUTH-POLICY-23-001..002 | Publish `/docs/policy/governance.md` (roles, scopes, approvals, signing, exceptions). | Governance doc merged; checklist appended. |
|
||||
| DOCS-POLICY-23-006 | TODO | Docs Guild, BE-Base Platform Guild | WEB-POLICY-23-001..004 | Update `/docs/api/policy.md` with new endpoints, schemas, errors, pagination. | API doc aligns with OpenAPI; examples validated; checklist included. |
|
||||
| DOCS-POLICY-23-007 | TODO | Docs Guild, DevEx/CLI Guild | CLI-POLICY-23-004..006 | Update `/docs/modules/cli/guides/policy.md` for lint/simulate/activate/history commands, exit codes. | CLI doc updated; samples verified; checklist appended. |
|
||||
| DOCS-POLICY-23-008 | TODO | Docs Guild, Architecture Guild | POLICY-ENGINE-50-005..006 | Refresh `/docs/modules/policy/architecture.md` with data model, sequence diagrams, event flows. | Architecture doc merged with diagrams; checklist appended. |
|
||||
| DOCS-POLICY-23-009 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, DEVOPS-LNM-22-001 | Create `/docs/migration/policy-parity.md` covering dual-run parity plan and rollback. | Migration doc approved; checklist appended. |
|
||||
| DOCS-POLICY-23-010 | TODO | Docs Guild, UI Guild | UI-POLICY-23-006 | Write `/docs/ui/explainers.md` showing explain trees, evidence overlays, interpretation guidance. | Doc merged with annotated screenshots; checklist appended. |
|
||||
|
||||
## Graph & Vuln Explorer v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-GRAPH-24-001 | TODO | Docs Guild, UI Guild | UI-GRAPH-24-001..006 | Author `/docs/ui/sbom-graph-explorer.md` detailing overlays, filters, saved views, accessibility, and AOC visibility. | Doc merged; screenshots included; checklist appended. |
|
||||
| DOCS-GRAPH-24-002 | TODO | Docs Guild, UI Guild | UI-GRAPH-24-005 | Publish `/docs/ui/vulnerability-explorer.md` covering table usage, grouping, fix suggestions, Why drawer. | Doc merged with annotated images; accessibility checklist satisfied. |
|
||||
| DOCS-GRAPH-24-003 | TODO | Docs Guild, SBOM Service Guild | SBOM-GRAPH-24-001..003 | Create `/docs/modules/graph/architecture-index.md` describing data model, ingestion pipeline, caches, events. | Architecture doc merged with diagrams; checklist appended. |
|
||||
| DOCS-GRAPH-24-004 | TODO | Docs Guild, BE-Base Platform Guild | WEB-GRAPH-24-001..003 | Document `/docs/api/graph.md` and `/docs/api/vuln.md` avec endpoints, parameters, errors, RBAC. | API docs aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-GRAPH-24-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-GRAPH-24-001..003 | Update `/docs/modules/cli/guides/graph-and-vuln.md` covering new CLI commands, exit codes, scripting. | CLI doc merged; examples tested; checklist appended. |
|
||||
| DOCS-GRAPH-24-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-60-001..002 | Write `/docs/policy/ui-integration.md` explaining overlays, cache usage, simulator contracts. | Doc merged; references cross-linked; checklist appended. |
|
||||
| DOCS-GRAPH-24-007 | TODO | Docs Guild, DevOps Guild | DEVOPS-GRAPH-24-001..003 | Produce `/docs/migration/graph-parity.md` with rollout plan, parity checks, fallback guidance. | Migration doc approved; checklist appended. |
|
||||
|
||||
## Exceptions v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-EXC-25-001 | TODO | Docs Guild, Governance Guild | WEB-EXC-25-001 | Author `/docs/governance/exceptions.md` covering lifecycle, scope patterns, examples, compliance checklist. | Doc merged; reviewers sign off; checklist included. |
|
||||
| DOCS-EXC-25-002 | TODO | Docs Guild, Authority Core | AUTH-EXC-25-001 | Publish `/docs/governance/approvals-and-routing.md` detailing roles, routing matrix, MFA rules, audit trails. | Doc merged; routing examples validated; checklist appended. |
|
||||
| DOCS-EXC-25-003 | TODO | Docs Guild, BE-Base Platform Guild | WEB-EXC-25-001..003 | Create `/docs/api/exceptions.md` with endpoints, payloads, errors, idempotency notes. | API doc aligned with OpenAPI; examples tested; checklist appended. |
|
||||
| DOCS-EXC-25-004 | DONE (2025-10-27) | Docs Guild, Policy Guild | POLICY-ENGINE-70-001 | Document `/docs/policy/exception-effects.md` explaining evaluation order, conflicts, simulation. | Doc merged; tests cross-referenced; checklist appended. |
|
||||
| DOCS-EXC-25-005 | TODO | Docs Guild, UI Guild | UI-EXC-25-001..004 | Write `/docs/ui/exception-center.md` with UI walkthrough, badges, accessibility, shortcuts. | Doc merged with screenshots; accessibility checklist completed. |
|
||||
| DOCS-EXC-25-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXC-25-001..002 | Update `/docs/modules/cli/guides/exceptions.md` covering command usage and exit codes. | CLI doc updated; examples validated; checklist appended. |
|
||||
| DOCS-EXC-25-007 | TODO | Docs Guild, DevOps Guild | SCHED-WORKER-25-101, DEVOPS-GRAPH-24-003 | Publish `/docs/migration/exception-governance.md` describing cutover from legacy suppressions, notifications, rollback. | Migration doc approved; checklist included. |
|
||||
|
||||
> Update statuses (TODO/DOING/REVIEW/DONE/BLOCKED) as progress changes. Keep guides in sync with configuration samples under `etc/`.
|
||||
|
||||
> Remark (2025-10-13, DOC4.AUTH-PDG): Rate limit guide published (`docs/security/rate-limits.md`) and handed to plugin docs team for diagram uplift once PLG6.DIAGRAM lands.
|
||||
|
||||
## Orchestrator Dashboard (Epic 9)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-ORCH-32-001 | TODO | Docs Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Author `/docs/orchestrator/overview.md` covering mission, roles, AOC alignment, governance, with imposed rule reminder. | Doc merged with diagrams; imposed rule statement included; entry linked from docs index. |
|
||||
| DOCS-ORCH-32-002 | TODO | Docs Guild | ORCH-SVC-32-002 | Author `/docs/orchestrator/architecture.md` detailing scheduler, DAGs, rate limits, data model, message bus, storage layout, restating imposed rule. | Architecture doc merged; diagrams reviewed; imposed rule noted. |
|
||||
| DOCS-ORCH-33-001 | TODO | Docs Guild | ORCH-SVC-33-001..004, WEB-ORCH-33-001 | Publish `/docs/orchestrator/api.md` (REST/WebSocket endpoints, payloads, error codes) with imposed rule note. | API doc merged; examples validated; imposed rule appended. |
|
||||
| DOCS-ORCH-33-002 | TODO | Docs Guild | CONSOLE-ORCH-32-002, CONSOLE-ORCH-33-001..002 | Publish `/docs/orchestrator/console.md` covering screens, a11y, live updates, control actions, reiterating imposed rule. | Console doc merged with screenshots; accessibility checklist done; imposed rule statement present. |
|
||||
| DOCS-ORCH-33-003 | TODO | Docs Guild | CLI-ORCH-33-001 | Publish `/docs/orchestrator/cli.md` documenting commands, options, exit codes, streaming output, offline usage, and imposed rule. | CLI doc merged; examples tested; imposed rule appended. |
|
||||
| DOCS-ORCH-34-001 | TODO | Docs Guild | ORCH-SVC-34-002, LEDGER-34-101 | Author `/docs/orchestrator/run-ledger.md` covering ledger schema, provenance chain, audit workflows, with imposed rule reminder. | Run-ledger doc merged; payload samples validated; imposed rule included; cross-links added. |
|
||||
| DOCS-ORCH-34-002 | TODO | Docs Guild | AUTH-ORCH-32-001, AUTH-ORCH-34-001 | Update `/docs/security/secrets-handling.md` for orchestrator KMS refs, redaction badges, operator hygiene, reiterating imposed rule. | Security doc merged; checklists updated; imposed rule restated; references from Console/CLI docs added. |
|
||||
| DOCS-ORCH-34-003 | TODO | Docs Guild | ORCH-SVC-33-003, ORCH-SVC-34-001, DEVOPS-ORCH-34-001 | Publish `/docs/operations/orchestrator-runbook.md` (incident playbook, backfill guide, circuit breakers, throttling) with imposed rule statement. | Runbook merged; steps validated with DevOps; imposed rule included; runbook linked from ops index. |
|
||||
| DOCS-ORCH-34-004 | TODO | Docs Guild | ORCH-SVC-32-005, WORKER-GO-33-001, WORKER-PY-33-001 | Document `/docs/schemas/artifacts.md` describing artifact kinds, schema versions, hashing, storage layout, restating imposed rule. | Schema doc merged; JSON schema provided; imposed rule included; sample payload validated. |
|
||||
| DOCS-ORCH-34-005 | TODO | Docs Guild | ORCH-SVC-34-001, DEVOPS-ORCH-34-001 | Author `/docs/slo/orchestrator-slo.md` defining SLOs, burn alerts, measurement, and reiterating imposed rule. | SLO doc merged; dashboard screenshots embedded; imposed rule appended; alerts documented. |
|
||||
|
||||
## Export Center (Epic 10)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-EXPORT-35-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-001..006 | Author `/docs/modules/export-center/overview.md` covering purpose, profiles, security, AOC alignment, surfaces, ending with imposed rule statement. | Doc merged with diagrams/examples; imposed rule line present; index updated. |
|
||||
| DOCS-EXPORT-35-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-002..005 | Publish `/docs/modules/export-center/architecture.md` describing planner, adapters, manifests, signing, distribution flows, restating imposed rule. | Architecture doc merged; sequence diagrams included; rule statement appended. |
|
||||
| DOCS-EXPORT-35-003 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-003..004 | Publish `/docs/modules/export-center/profiles.md` detailing schema fields, examples, compatibility, and imposed rule reminder. | Profiles doc merged; JSON schemas linked; imposed rule noted. |
|
||||
| DOCS-EXPORT-36-004 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001..004, WEB-EXPORT-36-001 | Publish `/docs/modules/export-center/api.md` covering endpoints, payloads, errors, and mention imposed rule. | API doc merged; examples validated; rule included. |
|
||||
| DOCS-EXPORT-36-005 | DONE (2025-10-29) | Docs Guild | CLI-EXPORT-35-001, CLI-EXPORT-36-001 | Publish `/docs/modules/export-center/cli.md` with command reference, CI scripts, verification steps, restating imposed rule. | CLI doc merged; script snippets tested; rule appended. |
|
||||
| DOCS-EXPORT-36-006 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001, DEVOPS-EXPORT-36-001 | Publish `/docs/modules/export-center/trivy-adapter.md` covering field mappings, compatibility matrix, and imposed rule reminder. | Doc merged; mapping tables validated; rule included. |
|
||||
| DOCS-EXPORT-37-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-37-001, DEVOPS-EXPORT-37-001 | Publish `/docs/modules/export-center/mirror-bundles.md` describing filesystem/OCI layouts, delta/encryption, import guide, ending with imposed rule. | Doc merged; diagrams provided; verification steps tested; rule stated. |
|
||||
| DOCS-EXPORT-37-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-005, EXPORT-SVC-37-002 | Publish `/docs/modules/export-center/provenance-and-signing.md` detailing manifests, attestation flow, verification, reiterating imposed rule. | Doc merged; signature examples validated; rule appended. |
|
||||
| DOCS-EXPORT-37-003 | DONE (2025-10-29) | Docs Guild | DEVOPS-EXPORT-37-001 | Publish `/docs/operations/export-runbook.md` covering failures, tuning, capacity planning, with imposed rule reminder. | Runbook merged; procedures validated; rule included. |
|
||||
| DOCS-EXPORT-37-004 | TODO | Docs Guild | AUTH-EXPORT-37-001, EXPORT-SVC-37-002 | Publish `/docs/security/export-hardening.md` outlining RBAC, tenancy, encryption, redaction, restating imposed rule. | Security doc merged; checklist updated; rule appended. |
|
||||
| DOCS-EXPORT-37-101 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXPORT-37-001 | Refresh CLI verification sections once `stella export verify` lands (flags, exit codes, samples). | `docs/modules/export-center/cli.md` & `docs/modules/export-center/provenance-and-signing.md` updated with final command syntax; examples tested; rule reminder retained. |
|
||||
| DOCS-EXPORT-37-102 | TODO | Docs Guild, DevOps Guild | DEVOPS-EXPORT-37-001 | Embed export dashboards/alerts references into provenance/runbook docs after Grafana work ships. | Docs updated with dashboard IDs/alert notes; update logged; rule reminder present. |
|
||||
| DOCS-EXPORT-37-005 | TODO | Docs Guild, Exporter Service Guild | EXPORT-SVC-35-006, DEVOPS-EXPORT-36-001 | Validate Export Center docs against live Trivy/mirror bundles once implementation lands; refresh examples and CLI snippets accordingly. | Real bundle examples recorded; docs updated; verification steps confirmed with production artefacts. |
|
||||
> Note (2025-10-29): Blocked until exporter API (`EXPORT-SVC-35-006`) and Trivy/mirror adapters (`EXPORT-SVC-36-001`, `EXPORT-SVC-37-001`) ship. Requires access to CI smoke outputs (`DEVOPS-EXPORT-36-001`) for verification artifacts.
|
||||
|
||||
## Reachability v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-SIG-26-001 | TODO | Docs Guild, Signals Guild | SIGNALS-24-004 | Write `/docs/signals/reachability.md` covering states, scores, provenance, retention. | Doc merged with diagrams/examples; checklist appended. |
|
||||
| DOCS-SIG-26-002 | TODO | Docs Guild, Signals Guild | SIGNALS-24-002 | Publish `/docs/signals/callgraph-formats.md` with schemas and validation errors. | Doc merged; examples tested; checklist included. |
|
||||
| DOCS-SIG-26-003 | TODO | Docs Guild, Runtime Guild | SIGNALS-24-003 | Create `/docs/signals/runtime-facts.md` detailing agent capabilities, privacy safeguards, opt-in flags. | Doc merged; privacy review done; checklist appended. |
|
||||
| DOCS-SIG-26-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-80-001 | Document `/docs/policy/signals-weighting.md` for SPL predicates and weighting strategies. | Doc merged; sample policies validated; checklist appended. |
|
||||
| DOCS-SIG-26-005 | TODO | Docs Guild, UI Guild | UI-SIG-26-001..003 | Draft `/docs/ui/reachability-overlays.md` with badges, timelines, shortcuts. | Doc merged with screenshots; accessibility checklist completed. |
|
||||
| DOCS-SIG-26-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-SIG-26-001..002 | Update `/docs/modules/cli/guides/reachability.md` for new commands and automation recipes. | Doc merged; examples verified; checklist appended. |
|
||||
| DOCS-SIG-26-007 | TODO | Docs Guild, BE-Base Platform Guild | WEB-SIG-26-001..003 | Publish `/docs/api/signals.md` covering endpoints, payloads, ETags, errors. | API doc aligned with OpenAPI; examples tested; checklist appended. |
|
||||
| DOCS-SIG-26-008 | TODO | Docs Guild, DevOps Guild | DEVOPS-SIG-26-001..002 | Write `/docs/migration/enable-reachability.md` guiding rollout, fallbacks, monitoring. | Migration doc approved; checklist appended. |
|
||||
|
||||
## Policy Studio (Sprint 27)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-POLICY-27-001 | BLOCKED (2025-10-27) | Docs Guild, Policy Guild | REGISTRY-API-27-001, POLICY-ENGINE-27-001 | Publish `/docs/policy/studio-overview.md` covering lifecycle, roles, glossary, and compliance checklist. | Doc merged with diagrams + lifecycle table; checklist appended; stakeholders sign off. |
|
||||
> Blocked by `REGISTRY-API-27-001` and `POLICY-ENGINE-27-001`; need spec + compile data.
|
||||
> Blocker: Registry OpenAPI (`REGISTRY-API-27-001`) and policy compile enrichments (`POLICY-ENGINE-27-001`) are still TODO; need final interfaces before drafting overview.
|
||||
| DOCS-POLICY-27-002 | BLOCKED (2025-10-27) | Docs Guild, Console Guild | CONSOLE-STUDIO-27-001 | Write `/docs/policy/authoring.md` detailing workspace templates, snippets, lint rules, IDE shortcuts, and best practices. | Authoring doc includes annotated screenshots, snippet catalog, compliance checklist. |
|
||||
> Blocked by `CONSOLE-STUDIO-27-001` Studio authoring UI pending.
|
||||
> Blocker: Console Studio authoring UI (`CONSOLE-STUDIO-27-001`) not implemented; awaiting UX to capture flows/snippets.
|
||||
| DOCS-POLICY-27-003 | BLOCKED (2025-10-27) | Docs Guild, Policy Registry Guild | REGISTRY-API-27-007 | Document `/docs/policy/versioning-and-publishing.md` (semver rules, attestations, rollback) with compliance checklist. | Doc merged with flow diagrams; attestation steps documented; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-007` publish/sign pipeline outstanding.
|
||||
> Blocker: Registry publish/sign workflow (`REGISTRY-API-27-007`) pending.
|
||||
| DOCS-POLICY-27-004 | BLOCKED (2025-10-27) | Docs Guild, Scheduler Guild | REGISTRY-API-27-005, SCHED-WORKER-27-301 | Write `/docs/policy/simulation.md` covering quick vs batch sim, thresholds, evidence bundles, CLI examples. | Simulation doc includes charts, sample manifests, checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-005`/`SCHED-WORKER-27-301` batch simulation not ready.
|
||||
> Blocker: Batch simulation APIs/workers (`REGISTRY-API-27-005`, `SCHED-WORKER-27-301`) still TODO.
|
||||
| DOCS-POLICY-27-005 | BLOCKED (2025-10-27) | Docs Guild, Product Ops | REGISTRY-API-27-006 | Publish `/docs/policy/review-and-approval.md` with approver requirements, comments, webhooks, audit trail guidance. | Doc merged with role matrix + webhook schema; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-006` review workflow not implemented.
|
||||
> Blocker: Review workflow (`REGISTRY-API-27-006`) not landed.
|
||||
| DOCS-POLICY-27-006 | BLOCKED (2025-10-27) | Docs Guild, Policy Guild | REGISTRY-API-27-008 | Author `/docs/policy/promotion.md` covering environments, canary, rollback, and monitoring steps. | Promotion doc includes examples + checklist; verified by Policy Ops. |
|
||||
> Blocked by `REGISTRY-API-27-008` promotion APIs pending.
|
||||
> Blocker: Promotion/canary APIs (`REGISTRY-API-27-008`) outstanding.
|
||||
| DOCS-POLICY-27-007 | BLOCKED (2025-10-27) | Docs Guild, DevEx/CLI Guild | CLI-POLICY-27-001..004 | Update `/docs/policy/cli.md` with new commands, JSON schemas, CI usage, and compliance checklist. | CLI doc merged with transcripts; schema references validated; checklist appended. |
|
||||
> Blocked by `CLI-POLICY-27-001..004` CLI commands missing.
|
||||
> Blocker: Policy CLI commands (`CLI-POLICY-27-001..004`) yet to implement.
|
||||
| DOCS-POLICY-27-008 | BLOCKED (2025-10-27) | Docs Guild, Policy Registry Guild | REGISTRY-API-27-001..008 | Publish `/docs/policy/api.md` describing Registry endpoints, request/response schemas, errors, and feature flags. | API doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-001..008` OpenAPI + endpoints incomplete.
|
||||
> Blocker: Registry OpenAPI/spec suite (`REGISTRY-API-27-001..008`) incomplete.
|
||||
| DOCS-POLICY-27-009 | BLOCKED (2025-10-27) | Docs Guild, Security Guild | AUTH-POLICY-27-002 | Create `/docs/security/policy-attestations.md` covering signing, verification, key rotation, and compliance checklist. | Security doc approved by Security Guild; verifier steps documented; checklist appended. |
|
||||
> Blocked by `AUTH-POLICY-27-002` signing enforcement pending.
|
||||
> Blocker: Authority signing enforcement (`AUTH-POLICY-27-002`) pending.
|
||||
| DOCS-POLICY-27-010 | BLOCKED (2025-10-27) | Docs Guild, Architecture Guild | REGISTRY-API-27-001, SCHED-WORKER-27-301 | Author `/docs/modules/policy/registry-architecture.md` (service design, schemas, queues, failure modes) with diagrams and checklist. | Architecture doc merged; diagrams committed; checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-001` & `SCHED-WORKER-27-301` need delivery.
|
||||
> Blocker: Policy Registry schema/workers not delivered (see `REGISTRY-API-27-001`, `SCHED-WORKER-27-301`).
|
||||
| DOCS-POLICY-27-011 | BLOCKED (2025-10-27) | Docs Guild, Observability Guild | DEVOPS-POLICY-27-004 | Publish `/docs/observability/policy-telemetry.md` with metrics/log tables, dashboards, alerts, and compliance checklist. | Observability doc merged; dashboards linked; checklist appended. |
|
||||
> Blocked by `DEVOPS-POLICY-27-004` observability dashboards outstanding.
|
||||
> Blocker: Observability dashboards (`DEVOPS-POLICY-27-004`) not built.
|
||||
| DOCS-POLICY-27-012 | BLOCKED (2025-10-27) | Docs Guild, Ops Guild | DEPLOY-POLICY-27-002 | Write `/docs/runbooks/policy-incident.md` detailing rollback, freeze, forensic steps, notifications. | Runbook merged; rehearsal recorded; checklist appended. |
|
||||
> Blocked by `DEPLOY-POLICY-27-002` incident runbook inputs pending.
|
||||
> Blocker: Ops runbook inputs (`DEPLOY-POLICY-27-002`) pending.
|
||||
| DOCS-POLICY-27-013 | BLOCKED (2025-10-27) | Docs Guild, Policy Guild | CONSOLE-STUDIO-27-001, REGISTRY-API-27-002 | Update `/docs/examples/policy-templates.md` with new templates, snippets, and sample policies. | Examples committed with commentary; lint passes; checklist appended. |
|
||||
> Blocked by `CONSOLE-STUDIO-27-001`/`REGISTRY-API-27-002` templates missing.
|
||||
> Blocker: Studio templates and registry storage (`CONSOLE-STUDIO-27-001`, `REGISTRY-API-27-002`) not available.
|
||||
| DOCS-POLICY-27-014 | BLOCKED (2025-10-27) | Docs Guild, Policy Registry Guild | REGISTRY-API-27-003, WEB-POLICY-27-001 | Refresh `/docs/aoc/aoc-guardrails.md` to include Studio-specific guardrails and validation scenarios. | Doc updated with Studio guardrails; compliance checklist appended. |
|
||||
> Blocked by `REGISTRY-API-27-003` & `WEB-POLICY-27-001` guardrails not implemented.
|
||||
> Blocker: Registry compile pipeline/web proxy (`REGISTRY-API-27-003`, `WEB-POLICY-27-001`) outstanding.
|
||||
|
||||
## Vulnerability Explorer (Sprint 29)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-VULN-29-001 | TODO | Docs Guild, Vuln Explorer Guild | VULN-API-29-001 | Publish `/docs/vuln/explorer-overview.md` covering domain model, identities, AOC guarantees, workflow summary. | Doc merged with diagrams/table; compliance checklist appended. |
|
||||
| DOCS-VULN-29-002 | TODO | Docs Guild, Console Guild | CONSOLE-VULN-29-001..006 | Write `/docs/vuln/explorer-using-console.md` with workflows, screenshots, keyboard shortcuts, saved views, deep links. | Doc merged; images stored; WCAG notes included; checklist appended. |
|
||||
| DOCS-VULN-29-003 | TODO | Docs Guild, Vuln Explorer API Guild | VULN-API-29-001..009 | Author `/docs/vuln/explorer-api.md` (endpoints, query schema, grouping, errors, rate limits). | Doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-VULN-29-004 | TODO | Docs Guild, DevEx/CLI Guild | CLI-VULN-29-001..005 | Publish `/docs/vuln/explorer-cli.md` with command reference, samples, exit codes, CI snippets. | CLI doc merged; transcripts/JSON outputs validated; checklist appended. |
|
||||
| DOCS-VULN-29-005 | TODO | Docs Guild, Findings Ledger Guild | LEDGER-29-001..009 | Write `/docs/vuln/findings-ledger.md` detailing event schema, hashing, Merkle roots, replay tooling. | Doc merged; compliance checklist appended; audit team sign-off. |
|
||||
| DOCS-VULN-29-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-29-001..003 | Update `/docs/policy/vuln-determinations.md` for new rationale, signals, simulation semantics. | Doc updated; examples validated; checklist appended. |
|
||||
| DOCS-VULN-29-007 | TODO | Docs Guild, Excititor Guild | EXCITITOR-VULN-29-001..004 | Publish `/docs/vex/explorer-integration.md` covering CSAF mapping, suppression precedence, status semantics. | Doc merged; compliance checklist appended. |
|
||||
| DOCS-VULN-29-008 | TODO | Docs Guild, Concelier Guild | CONCELIER-VULN-29-001..004 | Publish `/docs/advisories/explorer-integration.md` covering key normalization, withdrawn handling, provenance. | Doc merged; checklist appended. |
|
||||
| DOCS-VULN-29-009 | TODO | Docs Guild, SBOM Service Guild | SBOM-VULN-29-001..002 | Author `/docs/sbom/vuln-resolution.md` detailing version semantics, scope, paths, safe version hints. | Doc merged; ecosystem tables validated; checklist appended. |
|
||||
| DOCS-VULN-29-010 | TODO | Docs Guild, Observability Guild | VULN-API-29-009, DEVOPS-VULN-29-002 | Publish `/docs/observability/vuln-telemetry.md` (metrics, logs, tracing, dashboards, SLOs). | Doc merged; dashboards linked; checklist appended. |
|
||||
| DOCS-VULN-29-011 | TODO | Docs Guild, Security Guild | AUTH-VULN-29-001..003 | Create `/docs/security/vuln-rbac.md` for roles, ABAC policies, attachment encryption, CSRF. | Security doc approved; checklist appended. |
|
||||
| DOCS-VULN-29-012 | TODO | Docs Guild, Ops Guild | DEVOPS-VULN-29-002, SCHED-WORKER-29-003 | Write `/docs/runbooks/vuln-ops.md` (projector lag, resolver storms, export failures, policy activation). | Runbook merged; rehearsal recorded; checklist appended. |
|
||||
| DOCS-VULN-29-013 | TODO | Docs Guild, Deployment Guild | DEPLOY-VULN-29-001..002 | Update `/docs/install/containers.md` with Findings Ledger & Vuln Explorer API images, manifests, resource sizing, health checks. | Install doc updated; validation commands included; checklist appended. |
|
||||
|
||||
## VEX Lens (Sprint 30)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-VEX-30-001 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-005 | Publish `/docs/vex/consensus-overview.md` describing purpose, scope, AOC guarantees. | Doc merged with diagrams/terminology tables; compliance checklist appended. |
|
||||
| DOCS-VEX-30-002 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-005 | Author `/docs/vex/consensus-algorithm.md` covering normalization, weighting, thresholds, examples. | Doc merged; math reviewed by Policy; checklist appended. |
|
||||
| DOCS-VEX-30-003 | TODO | Docs Guild, Issuer Directory Guild | ISSUER-30-001..003 | Document `/docs/vex/issuer-directory.md` (issuer management, keys, trust overrides, audit). | Doc merged; security review done; checklist appended. |
|
||||
| DOCS-VEX-30-004 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-007 | Publish `/docs/vex/consensus-api.md` with endpoint specs, query params, rate limits. | API doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-VEX-30-005 | TODO | Docs Guild, Console Guild | CONSOLE-VEX-30-001 | Write `/docs/vex/consensus-console.md` covering UI workflows, filters, conflicts, accessibility. | Doc merged; screenshots added; checklist appended. |
|
||||
| DOCS-VEX-30-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-29-001, VEXLENS-30-004 | Add `/docs/policy/vex-trust-model.md` detailing policy knobs, thresholds, simulation. | Doc merged; policy review completed; checklist appended. |
|
||||
| DOCS-VEX-30-007 | TODO | Docs Guild, SBOM Service Guild | VEXLENS-30-002 | Publish `/docs/sbom/vex-mapping.md` (CPE→purl strategy, edge cases, overrides). | Doc merged; mapping tables validated; checklist appended. |
|
||||
| DOCS-VEX-30-008 | TODO | Docs Guild, Security Guild | ISSUER-30-002, VEXLENS-30-003 | Deliver `/docs/security/vex-signatures.md` (verification flow, key rotation, audit). | Doc approved by Security; checklist appended. |
|
||||
| DOCS-VEX-30-009 | TODO | Docs Guild, DevOps Guild | VEXLENS-30-009, DEVOPS-VEX-30-001 | Create `/docs/runbooks/vex-ops.md` for recompute storms, mapping failures, signature errors. | Runbook merged; rehearsal logged; checklist appended. |
|
||||
|
||||
## Advisory AI (Sprint 31)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-AIAI-31-001 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-006 | Publish `/docs/advisory-ai/overview.md` covering capabilities, guardrails, RBAC. | Doc merged with diagrams; compliance checklist appended. |
|
||||
| DOCS-AIAI-31-002 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-004 | Author `/docs/advisory-ai/architecture.md` detailing RAG pipeline, deterministics, caching, model options. | Doc merged; architecture review done; checklist appended. |
|
||||
| DOCS-AIAI-31-003 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-006 | Write `/docs/advisory-ai/api.md` describing endpoints, schemas, errors, rate limits. | API doc aligned with OpenAPI; examples validated; checklist appended. |
|
||||
| DOCS-AIAI-31-004 | TODO | Docs Guild, Console Guild | CONSOLE-VULN-29-001, CONSOLE-VEX-30-001 | Create `/docs/advisory-ai/console.md` with screenshots, a11y notes, copy-as-ticket instructions. | Doc merged; images stored; checklist appended. |
|
||||
| DOCS-AIAI-31-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-VULN-29-001, CLI-VEX-30-001 | Publish `/docs/advisory-ai/cli.md` covering commands, exit codes, scripting patterns. | Doc merged; examples tested; checklist appended. |
|
||||
| DOCS-AIAI-31-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-31-001 | Update `/docs/policy/assistant-parameters.md` covering temperature, token limits, ranking weights, TTLs. | Doc merged; policy review done; checklist appended. |
|
||||
| DOCS-AIAI-31-007 | TODO | Docs Guild, Security Guild | AIAI-31-005 | Write `/docs/security/assistant-guardrails.md` detailing redaction, injection defense, logging. | Doc approved by Security; checklist appended. |
|
||||
| DOCS-AIAI-31-008 | TODO | Docs Guild, SBOM Service Guild | SBOM-AIAI-31-001 | Publish `/docs/sbom/remediation-heuristics.md` (feasibility scoring, blast radius). | Doc merged; heuristics reviewed; checklist appended. |
|
||||
| DOCS-AIAI-31-009 | TODO | Docs Guild, DevOps Guild | DEVOPS-AIAI-31-001 | Create `/docs/runbooks/assistant-ops.md` for warmup, cache priming, model outages, scaling. | Runbook merged; rehearsal logged; checklist appended. |
|
||||
|
||||
## Notifications Studio
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-NOTIFY-38-001 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-38-001..004 | Publish `/docs/notifications/overview.md` and `/docs/notifications/architecture.md`, each ending with imposed rule reminder. | Docs merged; diagrams verified; imposed rule appended. |
|
||||
| DOCS-NOTIFY-39-002 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-39-001..004 | Publish `/docs/notifications/rules.md`, `/docs/notifications/templates.md`, `/docs/notifications/digests.md` with examples and imposed rule line. | Docs merged; examples validated; imposed rule appended. |
|
||||
| DOCS-NOTIFY-40-001 | TODO | Docs Guild, Security Guild | AUTH-NOTIFY-38-001, NOTIFY-SVC-40-001..004 | Publish `/docs/notifications/channels.md`, `/docs/notifications/escalations.md`, `/docs/notifications/api.md`, `/docs/operations/notifier-runbook.md`, `/docs/security/notifications-hardening.md`; each ends with imposed rule line. | Docs merged; accessibility checks passed; imposed rule appended. |
|
||||
|
||||
## CLI Parity & Task Packs
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-CLI-41-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-CORE-41-001 | Publish `/docs/modules/cli/guides/overview.md`, `/docs/modules/cli/guides/configuration.md`, `/docs/modules/cli/guides/output-and-exit-codes.md` with imposed rule statements. | Docs merged; examples verified; imposed rule appended. |
|
||||
| DOCS-CLI-42-001 | TODO | Docs Guild | DOCS-CLI-41-001, CLI-PARITY-41-001 | Publish `/docs/modules/cli/guides/parity-matrix.md` and command guides under `/docs/modules/cli/guides/commands/*.md` (policy, sbom, vuln, vex, advisory, export, orchestrator, notify, aoc, auth). | Guides merged; parity automation documented; imposed rule appended. |
|
||||
| DOCS-PACKS-43-001 | DONE (2025-10-27) | Docs Guild, Task Runner Guild | PACKS-REG-42-001, TASKRUN-42-001 | Publish `/docs/task-packs/spec.md`, `/docs/task-packs/authoring-guide.md`, `/docs/task-packs/registry.md`, `/docs/task-packs/runbook.md`, `/docs/security/pack-signing-and-rbac.md`, `/docs/operations/cli-release-and-packaging.md` with imposed rule statements. | Docs merged; tutorials validated; imposed rule appended; cross-links added. |
|
||||
|
||||
## Containerized Distribution (Epic 13)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-INSTALL-44-001 | TODO | Docs Guild, Deployment Guild | COMPOSE-44-001 | Publish `/docs/install/overview.md` and `/docs/install/compose-quickstart.md` with imposed rule line and copy-ready commands. | Docs merged; screenshots/commands verified; imposed rule appended. |
|
||||
| DOCS-INSTALL-45-001 | TODO | Docs Guild, Deployment Guild | HELM-45-001 | Publish `/docs/install/helm-prod.md` and `/docs/install/configuration-reference.md` with values tables and imposed rule reminder. | Docs merged; configuration matrix verified; imposed rule appended. |
|
||||
| DOCS-INSTALL-46-001 | TODO | Docs Guild, Security Guild | DEPLOY-PACKS-43-001, CLI-PACKS-43-001 | Publish `/docs/install/airgap.md`, `/docs/security/supply-chain.md`, `/docs/operations/health-and-readiness.md`, `/docs/release/image-catalog.md`, `/docs/console/onboarding.md` (each with imposed rule). | Docs merged; checksum/signature sections validated; imposed rule appended. |
|
||||
|
||||
## Authority-Backed Scopes & Tenancy (Epic 14)
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| DOCS-TEN-47-001 | TODO | Docs Guild, Authority Core | AUTH-TEN-47-001 | Publish `/docs/security/tenancy-overview.md` and `/docs/security/scopes-and-roles.md` outlining scope grammar, tenant model, imposed rule reminder. | Docs merged; diagrams included; imposed rule appended. |
|
||||
| DOCS-TEN-48-001 | TODO | Docs Guild, Platform Ops | WEB-TEN-48-001 | Publish `/docs/operations/multi-tenancy.md`, `/docs/operations/rls-and-data-isolation.md`, `/docs/console/admin-tenants.md`. | Docs merged; examples validated; imposed rule appended. |
|
||||
| DOCS-TEN-49-001 | TODO | Docs & DevEx Guilds | CLI-TEN-47-001, AUTH-TEN-49-001 | Publish `/docs/modules/cli/guides/authentication.md`, `/docs/api/authentication.md`, `/docs/policy/examples/abac-overlays.md`, update `/docs/install/configuration-reference.md` with new env vars, all ending with imposed rule line. | Docs merged; command examples verified; imposed rule appended. |
|
||||
|
||||
@@ -1,131 +1,131 @@
|
||||
# StellaOps Console Accessibility Guide
|
||||
|
||||
> **Audience:** Accessibility Guild, Console Guild, Docs Guild, QA.
|
||||
> **Scope:** Keyboard interaction model, screen-reader behaviour, colour & focus tokens, testing workflows, offline considerations, and compliance checklist for the StellaOps Console (Sprint 23).
|
||||
|
||||
The console targets **WCAG 2.2 AA** across all supported browsers (Chromium, Firefox ESR) and honours StellaOps’ sovereign/offline constraints. Every build must keep keyboard-only users, screen-reader users, and high-contrast operators productive without relying on third-party services.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Accessibility Principles
|
||||
|
||||
1. **Deterministic navigation** – Focus order, shortcuts, and announcements remain stable across releases; URLs encode state for deep links.
|
||||
2. **Keyboard-first design** – Every actionable element is reachable via keyboard; shortcuts provide accelerators, and remapping is available via *Settings → Accessibility → Keyboard shortcuts*.
|
||||
3. **Assistive technology parity** – ARIA roles and live regions mirror visual affordances (status banners, SSE tickers, progress drawers). Screen readers receive polite/atomic updates to avoid chatter.
|
||||
4. **Colour & contrast tokens** – All palettes derive from design tokens that achieve ≥ 4.5:1 contrast (text) and ≥ 3:1 for graphical indicators; tokens pass automated contrast linting.
|
||||
5. **Offline equivalence** – Accessibility features (shortcuts, offline banners, focus restoration) behave the same in sealed environments, with guidance when actions require online authority.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Keyboard Interaction Map
|
||||
|
||||
### 2.1 Global shortcuts
|
||||
|
||||
| Action | Macs | Windows/Linux | Notes |
|
||||
|--------|------|---------------|-------|
|
||||
| Command palette | `⌘ K` | `Ctrl K` | Focuses palette search; respects tenant scope. |
|
||||
| Tenant picker | `⌘ T` | `Ctrl T` | Opens modal; `Enter` confirms, `Esc` cancels. |
|
||||
| Filter tray toggle | `⇧ F` | `Shift F` | Focus lands on first filter; `Tab` cycles filters before returning to page. |
|
||||
| Saved view presets | `⌘ 1-9` | `Ctrl 1-9` | Bound per tenant; missing preset triggers tooltip. |
|
||||
| Keyboard reference | `?` | `?` | Opens overlay listing context-specific shortcuts; `Esc` closes. |
|
||||
| Global search (context) | `/` | `/` | When the filter tray is closed, focuses inline search field. |
|
||||
|
||||
### 2.2 Module-specific shortcuts
|
||||
|
||||
| Module | Action | Macs | Windows/Linux | Notes |
|
||||
|--------|--------|------|---------------|-------|
|
||||
| Findings | Explain search | `⌘ /` | `Ctrl /` | Only when Explain drawer open; announces results via live region. |
|
||||
| SBOM Explorer | Toggle overlays | `⌘ G` | `Ctrl G` | Persists per session (see `/docs/ui/sbom-explorer.md`). |
|
||||
| Advisories & VEX | Provider filter | `⌘ ⌥ F` | `Ctrl Alt F` | Moves focus to provider chip row. |
|
||||
| Runs | Refresh snapshot | `⌘ R` | `Ctrl R` | Soft refresh of SSE state; no full page reload. |
|
||||
| Policies | Save draft | `⌘ S` | `Ctrl S` | Requires edit scope; exposes toast + status live update. |
|
||||
| Downloads | Copy CLI command | `⇧ D` | `Shift D` | Copies manifest or export command; toast announces scope hints. |
|
||||
|
||||
All shortcuts are remappable. Remaps persist in IndexedDB (per tenant) and export as part of profile bundles so operators can restore preferences offline.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Screen Reader & Focus Behaviour
|
||||
|
||||
- **Skip navigation** – Each route exposes a “Skip to content” link revealed on keyboard focus. Focus order: global header → page breadcrumb → action shelf → data grid/list → drawers/dialogs.
|
||||
- **Live regions** – Status ticker and SSE progress bars use `aria-live="polite"` with throttling to avoid flooding AT. Error toasts use `aria-live="assertive"` and auto-focus dismiss buttons.
|
||||
- **Drawers & modals** – Dialog components trap focus, support `Esc` to close, and restore focus to the launching control. Screen readers announce title + purpose.
|
||||
- **Tables & grids** – Large tables (Findings, SBOM inventory) switch to virtualised rows but retain ARIA grid semantics (`aria-rowcount`, `aria-colindex`). Column headers include sorting state via `aria-sort`.
|
||||
- **Tenancy context** – Tenant badge exposes `aria-describedby` linking to context summary (environment, offline snapshot). Switching tenant queues a polite announcement summarising new scope.
|
||||
- **Command palette** – Uses `role="dialog"` with search input labelled. Keyboard navigation within results uses `Up/Down`; screen readers announce result category + command.
|
||||
- **Offline banner** – When offline, a dismissible banner announces reason and includes instructions for CLI fallback. The banner has `role="status"` so it announces once without stealing focus.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Colour & Focus Tokens
|
||||
|
||||
Console consumes design tokens published by the Console Guild (tracked via CONSOLE-FEAT-23-102). Tokens live in the design system bundle (`ui/design/tokens/colors.json`, mirrored at build time). Key tokens:
|
||||
|
||||
| Token | Purpose | Contrast target |
|
||||
|-------|---------|-----------------|
|
||||
| `so-color-surface-base` | Primary surface/background | ≥ 4.5:1 against `so-color-text-primary`. |
|
||||
| `so-color-surface-raised` | Cards, drawers, modals | ≥ 3:1 against surrounding surfaces. |
|
||||
| `so-color-text-primary` | Default text colour | ≥ 4.5:1 against base surfaces. |
|
||||
| `so-color-text-inverted` | Text on accent buttons | ≥ 4.5:1 against accent fills. |
|
||||
| `so-color-accent-primary` | Action buttons, focus headings | ≥ 3:1 against surface. |
|
||||
| `so-color-status-critical` | Error toasts, violation chips | ≥ 4.5:1 for text; `critical-bg` provides >3:1 on neutral surface. |
|
||||
| `so-color-status-warning` | Warning banners | Meets 3:1 on surface and 4.5:1 for text overlays. |
|
||||
| `so-color-status-success` | Success toasts, pass badges | ≥ 3:1 for iconography; text uses `text-primary`. |
|
||||
| `so-focus-ring` | 2 px outline used across focusable elements | 3:1 against both light/dark surfaces. |
|
||||
|
||||
Colour tokens undergo automated linting (**axe-core contrast checks** + custom luminance script) during build. Any new token must include dark/light variants and pass the token contract tests.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Testing Workflow
|
||||
|
||||
| Layer | Tooling | Frequency | Notes |
|
||||
|-------|---------|-----------|-------|
|
||||
| Component a11y | Storybook + axe-core addon | On PR (story CI) | Fails when axe detects violations. |
|
||||
| Route regression | Playwright a11y sweep (`pnpm test:a11y`) | Nightly & release pipeline | Executes keyboard navigation, checks focus trap, runs Axe on key routes (Dashboard, Findings, SBOM, Admin). |
|
||||
| Colour contrast lint | Token validator (`tools/a11y/check-contrast.ts`) | On token change | Guards design token updates. |
|
||||
| CI parity | Pending `scripts/check-console-cli-parity.sh` (CONSOLE-DOC-23-502) | Release CI | Ensures CLI commands documented for parity features. |
|
||||
| Screen-reader spot checks | Manual NVDA + VoiceOver scripts | Pre-release checklist | Scenarios: tenant switch, explain drawer, downloads parity copy. |
|
||||
| Offline smoke | `stella offline kit import` + Playwright sealed-mode run | Prior to Offline Kit cut | Validates offline banners, disabled actions, keyboard flows without Authority. |
|
||||
|
||||
Accessibility QA (CONSOLE-QA-23-402) tracks failing scenarios via Playwright snapshots and publishes reports in the Downloads parity channel (`kind = "parity.report"` placeholder until CLI parity CI lands).
|
||||
|
||||
---
|
||||
|
||||
## 6 · Offline & Internationalisation Considerations
|
||||
|
||||
- Offline mode surfaces staleness badges and disables remote-only palette entries; keyboard focus skips disabled controls.
|
||||
- Saved shortcuts, presets, and remaps serialise into Offline Kit bundles so operators can restore preferences post-import.
|
||||
- Locale switching (future feature flag) will load translations at runtime; ensure ARIA labels use i18n tokens rather than hard-coded strings.
|
||||
- For sealed installs, guidance panels include CLI equivalents (`stella auth fresh-auth`, `stella runs export`) to unblock tasks when Authority is unavailable.
|
||||
|
||||
---
|
||||
|
||||
## 7 · Compliance Checklist
|
||||
|
||||
- [ ] Keyboard shortcut matrix validated (default + remapped) and documented.
|
||||
- [ ] Screen-reader pass recorded for tenant switch, Explain drawer, Downloads copy-to-clipboard.
|
||||
- [ ] Colour tokens audited; contrast reports stored with release artifacts.
|
||||
- [ ] Automated a11y pipelines (Storybook axe, Playwright a11y) green; failures feed the `#console-qa` channel.
|
||||
- [ ] Offline kit a11y smoke executed before publishing each bundle.
|
||||
- [ ] CLI parity gaps logged in `/docs/cli-vs-ui-parity.md`; UI callouts reference fallback commands until parity closes.
|
||||
- [ ] Accessibility Guild sign-off captured in sprint log and release notes reference this guide.
|
||||
- [ ] References cross-checked (`/docs/ui/navigation.md`, `/docs/ui/downloads.md`, `/docs/security/console-security.md`, `/docs/observability/ui-telemetry.md`).
|
||||
|
||||
---
|
||||
|
||||
## 8 · References
|
||||
|
||||
- `/docs/ui/navigation.md` – shortcut definitions, URL schema.
|
||||
- `/docs/ui/downloads.md` – CLI parity and offline copy workflows.
|
||||
- `/docs/ui/console-overview.md` – tenant model, filter behaviours.
|
||||
- `/docs/security/console-security.md` – security metrics and DPoP/fresh-auth requirements.
|
||||
- `/docs/observability/ui-telemetry.md` – telemetry metrics mapped to accessibility features.
|
||||
- `/docs/cli-vs-ui-parity.md` – parity status per console feature.
|
||||
- `CONSOLE-QA-23-402` – Accessibility QA backlog (Playwright + manual checks).
|
||||
- `CONSOLE-FEAT-23-102` – Design tokens & theming delivery.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-28 (Sprint 23).*
|
||||
|
||||
# StellaOps Console Accessibility Guide
|
||||
|
||||
> **Audience:** Accessibility Guild, Console Guild, Docs Guild, QA.
|
||||
> **Scope:** Keyboard interaction model, screen-reader behaviour, colour & focus tokens, testing workflows, offline considerations, and compliance checklist for the StellaOps Console (Sprint 23).
|
||||
|
||||
The console targets **WCAG 2.2 AA** across all supported browsers (Chromium, Firefox ESR) and honours StellaOps’ sovereign/offline constraints. Every build must keep keyboard-only users, screen-reader users, and high-contrast operators productive without relying on third-party services.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Accessibility Principles
|
||||
|
||||
1. **Deterministic navigation** – Focus order, shortcuts, and announcements remain stable across releases; URLs encode state for deep links.
|
||||
2. **Keyboard-first design** – Every actionable element is reachable via keyboard; shortcuts provide accelerators, and remapping is available via *Settings → Accessibility → Keyboard shortcuts*.
|
||||
3. **Assistive technology parity** – ARIA roles and live regions mirror visual affordances (status banners, SSE tickers, progress drawers). Screen readers receive polite/atomic updates to avoid chatter.
|
||||
4. **Colour & contrast tokens** – All palettes derive from design tokens that achieve ≥ 4.5:1 contrast (text) and ≥ 3:1 for graphical indicators; tokens pass automated contrast linting.
|
||||
5. **Offline equivalence** – Accessibility features (shortcuts, offline banners, focus restoration) behave the same in sealed environments, with guidance when actions require online authority.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Keyboard Interaction Map
|
||||
|
||||
### 2.1 Global shortcuts
|
||||
|
||||
| Action | Macs | Windows/Linux | Notes |
|
||||
|--------|------|---------------|-------|
|
||||
| Command palette | `⌘ K` | `Ctrl K` | Focuses palette search; respects tenant scope. |
|
||||
| Tenant picker | `⌘ T` | `Ctrl T` | Opens modal; `Enter` confirms, `Esc` cancels. |
|
||||
| Filter tray toggle | `⇧ F` | `Shift F` | Focus lands on first filter; `Tab` cycles filters before returning to page. |
|
||||
| Saved view presets | `⌘ 1-9` | `Ctrl 1-9` | Bound per tenant; missing preset triggers tooltip. |
|
||||
| Keyboard reference | `?` | `?` | Opens overlay listing context-specific shortcuts; `Esc` closes. |
|
||||
| Global search (context) | `/` | `/` | When the filter tray is closed, focuses inline search field. |
|
||||
|
||||
### 2.2 Module-specific shortcuts
|
||||
|
||||
| Module | Action | Macs | Windows/Linux | Notes |
|
||||
|--------|--------|------|---------------|-------|
|
||||
| Findings | Explain search | `⌘ /` | `Ctrl /` | Only when Explain drawer open; announces results via live region. |
|
||||
| SBOM Explorer | Toggle overlays | `⌘ G` | `Ctrl G` | Persists per session (see `/docs/ui/sbom-explorer.md`). |
|
||||
| Advisories & VEX | Provider filter | `⌘ ⌥ F` | `Ctrl Alt F` | Moves focus to provider chip row. |
|
||||
| Runs | Refresh snapshot | `⌘ R` | `Ctrl R` | Soft refresh of SSE state; no full page reload. |
|
||||
| Policies | Save draft | `⌘ S` | `Ctrl S` | Requires edit scope; exposes toast + status live update. |
|
||||
| Downloads | Copy CLI command | `⇧ D` | `Shift D` | Copies manifest or export command; toast announces scope hints. |
|
||||
|
||||
All shortcuts are remappable. Remaps persist in IndexedDB (per tenant) and export as part of profile bundles so operators can restore preferences offline.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Screen Reader & Focus Behaviour
|
||||
|
||||
- **Skip navigation** – Each route exposes a “Skip to content” link revealed on keyboard focus. Focus order: global header → page breadcrumb → action shelf → data grid/list → drawers/dialogs.
|
||||
- **Live regions** – Status ticker and SSE progress bars use `aria-live="polite"` with throttling to avoid flooding AT. Error toasts use `aria-live="assertive"` and auto-focus dismiss buttons.
|
||||
- **Drawers & modals** – Dialog components trap focus, support `Esc` to close, and restore focus to the launching control. Screen readers announce title + purpose.
|
||||
- **Tables & grids** – Large tables (Findings, SBOM inventory) switch to virtualised rows but retain ARIA grid semantics (`aria-rowcount`, `aria-colindex`). Column headers include sorting state via `aria-sort`.
|
||||
- **Tenancy context** – Tenant badge exposes `aria-describedby` linking to context summary (environment, offline snapshot). Switching tenant queues a polite announcement summarising new scope.
|
||||
- **Command palette** – Uses `role="dialog"` with search input labelled. Keyboard navigation within results uses `Up/Down`; screen readers announce result category + command.
|
||||
- **Offline banner** – When offline, a dismissible banner announces reason and includes instructions for CLI fallback. The banner has `role="status"` so it announces once without stealing focus.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Colour & Focus Tokens
|
||||
|
||||
Console consumes design tokens published by the Console Guild (tracked via CONSOLE-FEAT-23-102). Tokens live in the design system bundle (`ui/design/tokens/colors.json`, mirrored at build time). Key tokens:
|
||||
|
||||
| Token | Purpose | Contrast target |
|
||||
|-------|---------|-----------------|
|
||||
| `so-color-surface-base` | Primary surface/background | ≥ 4.5:1 against `so-color-text-primary`. |
|
||||
| `so-color-surface-raised` | Cards, drawers, modals | ≥ 3:1 against surrounding surfaces. |
|
||||
| `so-color-text-primary` | Default text colour | ≥ 4.5:1 against base surfaces. |
|
||||
| `so-color-text-inverted` | Text on accent buttons | ≥ 4.5:1 against accent fills. |
|
||||
| `so-color-accent-primary` | Action buttons, focus headings | ≥ 3:1 against surface. |
|
||||
| `so-color-status-critical` | Error toasts, violation chips | ≥ 4.5:1 for text; `critical-bg` provides >3:1 on neutral surface. |
|
||||
| `so-color-status-warning` | Warning banners | Meets 3:1 on surface and 4.5:1 for text overlays. |
|
||||
| `so-color-status-success` | Success toasts, pass badges | ≥ 3:1 for iconography; text uses `text-primary`. |
|
||||
| `so-focus-ring` | 2 px outline used across focusable elements | 3:1 against both light/dark surfaces. |
|
||||
|
||||
Colour tokens undergo automated linting (**axe-core contrast checks** + custom luminance script) during build. Any new token must include dark/light variants and pass the token contract tests.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Testing Workflow
|
||||
|
||||
| Layer | Tooling | Frequency | Notes |
|
||||
|-------|---------|-----------|-------|
|
||||
| Component a11y | Storybook + axe-core addon | On PR (story CI) | Fails when axe detects violations. |
|
||||
| Route regression | Playwright a11y sweep (`pnpm test:a11y`) | Nightly & release pipeline | Executes keyboard navigation, checks focus trap, runs Axe on key routes (Dashboard, Findings, SBOM, Admin). |
|
||||
| Colour contrast lint | Token validator (`src/Tools/a11y/check-contrast.ts`) | On token change | Guards design token updates. |
|
||||
| CI parity | Pending `scripts/check-console-cli-parity.sh` (CONSOLE-DOC-23-502) | Release CI | Ensures CLI commands documented for parity features. |
|
||||
| Screen-reader spot checks | Manual NVDA + VoiceOver scripts | Pre-release checklist | Scenarios: tenant switch, explain drawer, downloads parity copy. |
|
||||
| Offline smoke | `stella offline kit import` + Playwright sealed-mode run | Prior to Offline Kit cut | Validates offline banners, disabled actions, keyboard flows without Authority. |
|
||||
|
||||
Accessibility QA (CONSOLE-QA-23-402) tracks failing scenarios via Playwright snapshots and publishes reports in the Downloads parity channel (`kind = "parity.report"` placeholder until CLI parity CI lands).
|
||||
|
||||
---
|
||||
|
||||
## 6 · Offline & Internationalisation Considerations
|
||||
|
||||
- Offline mode surfaces staleness badges and disables remote-only palette entries; keyboard focus skips disabled controls.
|
||||
- Saved shortcuts, presets, and remaps serialise into Offline Kit bundles so operators can restore preferences post-import.
|
||||
- Locale switching (future feature flag) will load translations at runtime; ensure ARIA labels use i18n tokens rather than hard-coded strings.
|
||||
- For sealed installs, guidance panels include CLI equivalents (`stella auth fresh-auth`, `stella runs export`) to unblock tasks when Authority is unavailable.
|
||||
|
||||
---
|
||||
|
||||
## 7 · Compliance Checklist
|
||||
|
||||
- [ ] Keyboard shortcut matrix validated (default + remapped) and documented.
|
||||
- [ ] Screen-reader pass recorded for tenant switch, Explain drawer, Downloads copy-to-clipboard.
|
||||
- [ ] Colour tokens audited; contrast reports stored with release artifacts.
|
||||
- [ ] Automated a11y pipelines (Storybook axe, Playwright a11y) green; failures feed the `#console-qa` channel.
|
||||
- [ ] Offline kit a11y smoke executed before publishing each bundle.
|
||||
- [ ] CLI parity gaps logged in `/docs/cli-vs-ui-parity.md`; UI callouts reference fallback commands until parity closes.
|
||||
- [ ] Accessibility Guild sign-off captured in sprint log and release notes reference this guide.
|
||||
- [ ] References cross-checked (`/docs/ui/navigation.md`, `/docs/ui/downloads.md`, `/docs/security/console-security.md`, `/docs/observability/ui-telemetry.md`).
|
||||
|
||||
---
|
||||
|
||||
## 8 · References
|
||||
|
||||
- `/docs/ui/navigation.md` – shortcut definitions, URL schema.
|
||||
- `/docs/ui/downloads.md` – CLI parity and offline copy workflows.
|
||||
- `/docs/ui/console-overview.md` – tenant model, filter behaviours.
|
||||
- `/docs/security/console-security.md` – security metrics and DPoP/fresh-auth requirements.
|
||||
- `/docs/observability/ui-telemetry.md` – telemetry metrics mapped to accessibility features.
|
||||
- `/docs/cli-vs-ui-parity.md` – parity status per console feature.
|
||||
- `CONSOLE-QA-23-402` – Accessibility QA backlog (Playwright + manual checks).
|
||||
- `CONSOLE-FEAT-23-102` – Design tokens & theming delivery.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-28 (Sprint 23).*
|
||||
|
||||
|
||||
@@ -1,429 +0,0 @@
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 16: Air‑Gapped Mode
|
||||
|
||||
**Short name:** Air‑Gapped Mode
|
||||
**Primary components:** Web Services API, Console, CLI, Orchestrator, Task Runner, Conseiller (Feedser), Excitator (VEXer), Policy Engine, Findings Ledger, Export Center, Authority & Tenancy, Notifications, Observability & Forensics
|
||||
**Surfaces:** offline bootstrap, update ingestion via mirror bundles, sealed egress, deterministic jobs, offline advisories/VEX, offline policy packs, offline notifications, evidence exports
|
||||
**Dependencies:** Export Center, Containerized Distribution, Authority‑Backed Scopes & Tenancy, Observability & Forensics, Policy Studio
|
||||
|
||||
**AOC ground rule reminder:** Conseiller and Excitator aggregate and link advisories/VEX. They never merge or mutate source records. Air‑Gapped Mode must preserve this invariant even when mirroring and importing updates.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
A fully supported operating profile where StellaOps runs in a disconnected environment with:
|
||||
|
||||
* **Zero external egress** from platform services and jobs.
|
||||
* **Deterministic inputs** provided via signed, offline **Mirror Bundles** (advisories, VEX, policy packs, vendor feeds, Stella metadata, container images, dashboards).
|
||||
* **Offline bootstrap** for images and charts, plus reproducible configuration and cryptographically verifiable updates.
|
||||
* **Graceful feature degradation** with explicit UX: features that require external connectivity are either backed by local artifacts or clearly disabled with an explanation.
|
||||
* **Auditable import/export** including provenance attestations, evidence bundles, and chain‑of‑custody for all offline exchanges.
|
||||
|
||||
Air‑Gapped Mode is selectable at install time and enforceable at runtime. When enabled, all components operate under an “egress sealed” policy and only consume data from local stores.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
Many users operate in classified, regulated, or high‑sensitivity networks where egress is prohibited. They still need SBOM analysis, policy evaluation, advisory/VEX mapping, and reporting. Air‑Gapped Mode provides the same core outcomes with verifiable offline inputs and explicit operational guardrails.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work
|
||||
|
||||
### 3.1 Modes and lifecycle
|
||||
|
||||
* **Connected Mode:** normal operation; can create Mirror Bundles on a staging host.
|
||||
* **Sealed Air‑Gapped Mode:** platform enforces no egress. Only local resources are allowed.
|
||||
* **Transition flow:**
|
||||
|
||||
1. Prepare an offline **Bootstrap Pack** with all container images, Helm/compose charts, seed database, and initial Mirror Bundle.
|
||||
2. Install in the air‑gapped enclave and **seal** egress.
|
||||
3. Periodically import new **Mirror Bundles** via removable media.
|
||||
4. Export evidence/reports as needed.
|
||||
|
||||
### 3.2 Egress sealing
|
||||
|
||||
* **Static guardrails:**
|
||||
|
||||
* Platform flag `STELLA_AIRGAP=sealed` and database feature flag `env.mode='sealed'`.
|
||||
* NetworkPolicy/iptables/eBPF deny‑all egress for namespaces/pods except loopback and the internal object store.
|
||||
* Outbound DNS blocked.
|
||||
* HTTP clients in code use a single `EgressPolicy` facade. When sealed, it panics on direct network calls and returns a typed error with remediation (“import a Mirror Bundle”).
|
||||
* **Verification:** `GET /system/airgap/status` returns `sealed: true|false`, current policy hash, and last import timestamp. CLI prints warning if not sealed in declared air‑gapped install.
|
||||
|
||||
### 3.3 Trusted time
|
||||
|
||||
* Air‑gapped systems cannot NTP. Each Mirror Bundle includes a **signed time token** (Roughtime‑style or RFC 3161) from a trusted authority. On import, platform stores `time_anchor` for drift calculations and staleness checks.
|
||||
* If time drift exceeds policy threshold, UI shows “stale view” badges and some jobs are blocked until a new bundle provides a fresh anchor.
|
||||
|
||||
### 3.4 Mirror Bundles (offline updates)
|
||||
|
||||
* **Content types:**
|
||||
|
||||
* Public advisories (OSV, GHSA, vendor advisories), NVD mappings, CPE/Package metadata.
|
||||
* VEX statements from vendors/communities.
|
||||
* Policy packs (templates, baselines, versioned rule sets).
|
||||
* StellaOps engine metadata and schema migrations.
|
||||
* Optional: **OCI image set** for platform and recommended runners.
|
||||
* Optional: dashboards and alert rule packs.
|
||||
* **Format:** a TUF‑like layout:
|
||||
|
||||
```
|
||||
root.json, snapshot.json, timestamp.json, targets/
|
||||
advisories/*.jsonl.zst
|
||||
vex/*.jsonl.zst
|
||||
policy/*.tar.zst
|
||||
images/* (OCI layout or oci-archive)
|
||||
meta/engine/*.tgz
|
||||
meta/time-anchor.json (signed)
|
||||
```
|
||||
* **Integrity & trust:**
|
||||
|
||||
* DSSE‑signed target manifests.
|
||||
* Root of trust rotated via `root.json` within strict policy; rotation requires manual dual approval in sealed mode.
|
||||
* Each content artifact has a content digest and a **Merkle root** for the overall bundle.
|
||||
* **Creation:** in connected networks, `stella mirror create --content advisories,vex,policy,images --since 2025-01-01 --out bundle.tgz`.
|
||||
* **Import:** in air‑gap, `stella airgap import bundle.tgz`. The importer verifies DSSE, TUF metadata, Merkle root, then writes to local object store and updates catalog tables.
|
||||
* **Idempotence:** imports are content‑addressed; re‑imports deduplicate.
|
||||
|
||||
### 3.5 Deterministic jobs and sources
|
||||
|
||||
* **Allowed sources:** filesystem, internal object store, tenant private registry, and pre‑approved connectors that don’t require external egress.
|
||||
* **Disallowed in sealed mode:** remote package registries, web scrapers, outbound webhooks, cloud KMS unless on the enclave network.
|
||||
* **Runner policy:** the Task Runner verifies job descriptors contain no network calls unless marked `internal:` with allow‑listed destinations. Violations fail at plan time with an explainable error.
|
||||
|
||||
### 3.6 Conseiller and Excitator in air‑gap
|
||||
|
||||
* **Conseiller (Feedser):** ingests advisories only from imported bundles or tenant local feeds. It preserves source identities and never merges. Linkage uses bundle‑provided cross‑refs and local heuristics.
|
||||
* **Excitator (VEXer):** imports VEX records as‑is, links them to components and advisories, and records the origin bundle and statement digests. Consensus Lens (Epic 7) operates offline across the imported sources.
|
||||
|
||||
### 3.7 Policy Engine and Studio
|
||||
|
||||
* Policy packs are versioned and imported via bundles.
|
||||
* Simulation and authoring work locally. Exports of new or updated policies can be packaged as **Policy Sub‑Bundles** for transfer back to connected environments if needed.
|
||||
* Engine shows which rules depend on external evidence and how they degrade in sealed mode (e.g., “No external EPSS; using cached percentile from last bundle.”).
|
||||
|
||||
### 3.8 Notifications in sealed mode
|
||||
|
||||
* Default to **local delivery** only: SMTP relay inside enclave, syslog, file sink.
|
||||
* External webhooks are disabled.
|
||||
* Notification templates show “air‑gap compliant channel” tags to avoid misconfiguration.
|
||||
|
||||
### 3.9 Observability & Forensics
|
||||
|
||||
* Traces, logs, metrics remain local.
|
||||
* Evidence Locker supports **portable evidence packages** for cross‑domain transfer: `stella forensic snapshot create --portable`.
|
||||
* Importing an evidence bundle in another enclave verifies signatures and maintains chain‑of‑custody.
|
||||
|
||||
### 3.10 Console and CLI behavior
|
||||
|
||||
* Console shows a prominent **Air‑Gapped: Sealed** badge with last import time and staleness indicators for advisories, VEX, and policy packs.
|
||||
* CLI commands gain `--sealed` awareness: any operation that would egress prints a refusal with remediation suggesting the appropriate import.
|
||||
|
||||
### 3.11 Multi‑tenant and scope
|
||||
|
||||
* Tenancy works unchanged. Bundle imports can target:
|
||||
|
||||
* `--tenant-global`: shared catalogs (advisories, VEX, policy baselines).
|
||||
* `--tenant=<id>`: tenant‑specific content (e.g., private advisories).
|
||||
* Authority scopes gain `airgap:import`, `airgap:status:read`, `airgap:seal` (admin‑only).
|
||||
|
||||
### 3.12 Feature degradation matrix
|
||||
|
||||
* **AI Assistant:** offline variants use local models if installed; otherwise feature is disabled with a message.
|
||||
* **External reputation feeds (e.g., EPSS‑like):** replaced by cached values from the bundle.
|
||||
* **Container base image lookups:** rely on imported metadata or tenant private registry.
|
||||
|
||||
---
|
||||
|
||||
## 4) Architecture
|
||||
|
||||
### 4.1 New modules
|
||||
|
||||
* `airgap/controller`
|
||||
|
||||
* Sealing state machine; status API; guardrails wiring into HTTP clients and runner.
|
||||
* `airgap/importer`
|
||||
|
||||
* TUF/DSSE verification, Merkle validation, object store loader, catalog updater.
|
||||
* `mirror/creator`
|
||||
|
||||
* Connected‑side builder for bundles; content plug‑ins for advisories/VEX/policy/images.
|
||||
* `airgap/policy`
|
||||
|
||||
* Enforcement library exposing `EgressPolicy` facade and job plan validators.
|
||||
* `airgap/time`
|
||||
|
||||
* Time anchor parser, drift checks, staleness annotations.
|
||||
* `console/airgap`
|
||||
|
||||
* Sealed badge, import UI, staleness dashboards, degradation notices.
|
||||
* `cli/airgap`
|
||||
|
||||
* `stella airgap seal|status|import|verify` commands; `stella mirror create|verify`.
|
||||
|
||||
### 4.2 Data model additions
|
||||
|
||||
* `airgap_state(id, sealed BOOLEAN, policy_hash TEXT, last_import_at TIMESTAMP, time_anchor JSONB)`
|
||||
* `bundle_catalog(id, kind ENUM, merkle_root TEXT, dsse_signer TEXT, created_at TIMESTAMP, imported_at TIMESTAMP, scope ENUM('global','tenant'), tenant_id NULLABLE, labels JSONB)`
|
||||
* `bundle_items(bundle_id, path TEXT, sha256 TEXT, size BIGINT, type TEXT, meta JSONB)`
|
||||
* `import_audit(id, bundle_id, actor, tenant_scope, verify_result, trace_id, created_at)`
|
||||
|
||||
RLS: tenant‑scoped rows when `scope='tenant'`; global rows readable only with `stella:airgap:status:read`.
|
||||
|
||||
### 4.3 Storage layout
|
||||
|
||||
Object store paths:
|
||||
|
||||
```
|
||||
tenants/_global/mirror/<bundle_id>/targets/...
|
||||
tenants/<tenant>/mirror/<bundle_id>/targets/...
|
||||
tenants/_global/images/<digest>/...
|
||||
```
|
||||
|
||||
Evidence locker remains separate. Imported images use **OCI layout** for local registry sync.
|
||||
|
||||
### 4.4 Message topics
|
||||
|
||||
* `stella.<tenant>.airgap.imported` with bundle metadata.
|
||||
* `stella.<tenant>.airgap.staleness` periodic events emitted for UX.
|
||||
* `stella.<tenant>.policy.degraded` when rules fall back due to sealed mode.
|
||||
|
||||
---
|
||||
|
||||
## 5) APIs and contracts
|
||||
|
||||
### 5.1 Status and control
|
||||
|
||||
* `GET /system/airgap/status` → `{ sealed, policy_hash, last_import_at, time_anchor, drift_seconds, staleness: { advisories_days, vex_days, policy_days } }`
|
||||
* `POST /system/airgap/seal` → seals environment; requires `stella:airgap:seal#tenant/<id or global>`.
|
||||
* `POST /system/airgap/unseal` → only allowed if installed mode is not declared “permanently sealed” at bootstrap. Typically disabled.
|
||||
|
||||
### 5.2 Import & verify
|
||||
|
||||
* `POST /airgap/import` multipart or file reference → runs verify, writes catalog, returns bundle summary and warnings.
|
||||
* `POST /airgap/verify` dry‑run verification returning DSSE/TUF and Merkle results.
|
||||
* `GET /airgap/bundles` list imported bundles with filters.
|
||||
|
||||
### 5.3 Conseiller/Excitator sources
|
||||
|
||||
* `POST /feeds/register` supports `kind=mirror` with `bundle_id` and paths; disallowed to point to external URLs in sealed mode.
|
||||
* `GET /feeds/status` shows per‑source staleness and last artifact version.
|
||||
|
||||
### 5.4 Errors
|
||||
|
||||
Standardized sealed‑mode error:
|
||||
|
||||
```
|
||||
{
|
||||
"code": "AIRGAP_EGRESS_BLOCKED",
|
||||
"message": "Egress is sealed. Import a Mirror Bundle with advisories.",
|
||||
"remediation": "Run: stella airgap import bundle.tgz",
|
||||
"trace_id": "..."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) Documentation changes
|
||||
|
||||
Create or update:
|
||||
|
||||
1. `/docs/airgap/overview.md`
|
||||
|
||||
* Modes, lifecycle, responsibilities, threat model, what degrades.
|
||||
2. `/docs/airgap/bootstrap.md`
|
||||
|
||||
* Offline Bootstrap Pack creation, validation, install steps for Helm/compose, local registry seeding.
|
||||
3. `/docs/airgap/mirror-bundles.md`
|
||||
|
||||
* Bundle format, DSSE/TUF/Merkle, signed time, creation on connected host, import in sealed environment, rotation of roots.
|
||||
4. `/docs/airgap/sealing-and-egress.md`
|
||||
|
||||
* Network policies, EgressPolicy facade, runner validation, verifying sealed status.
|
||||
5. `/docs/airgap/staleness-and-time.md`
|
||||
|
||||
* Time anchor, drift, staleness budgets and UI behavior.
|
||||
6. `/docs/airgap/operations.md`
|
||||
|
||||
* Periodic update cadence, runbooks, failure scenarios, disaster recovery.
|
||||
7. `/docs/airgap/degradation-matrix.md`
|
||||
|
||||
* Feature map: available, degraded, disabled; with remediation.
|
||||
8. `/docs/console/airgap.md`
|
||||
|
||||
* Status badges, import wizard, staleness indicators.
|
||||
9. `/docs/cli/airgap.md`
|
||||
|
||||
* Commands, examples, exit codes.
|
||||
10. `/docs/security/trust-and-signing.md`
|
||||
|
||||
* Roots of trust, key rotation, DSSE, TUF model.
|
||||
|
||||
11. `/docs/dev/airgap-contracts.md`
|
||||
|
||||
* EgressPolicy usage, testing patterns, sealed‑mode CI gates.
|
||||
|
||||
Add the banner at the top of each page:
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Implementation plan
|
||||
|
||||
### Phase 1 — Foundations
|
||||
|
||||
* Add `airgap/controller` with sealed state and status API.
|
||||
* Integrate `EgressPolicy` facade in all outbound network call sites.
|
||||
* Provide default NetworkPolicy/iptables templates and Helm values to block egress.
|
||||
* Console shows sealed badge and status.
|
||||
|
||||
### Phase 2 — Mirror Bundles
|
||||
|
||||
* Implement `mirror/creator` in connected mode with content plug‑ins.
|
||||
* Implement `airgap/importer` with DSSE/TUF/Merkle verification and catalog updates.
|
||||
* Export Center gains **Mirror bundle** build and verify commands (connected side).
|
||||
|
||||
### Phase 3 — Deterministic jobs
|
||||
|
||||
* Add job plan validation in the Task Runner.
|
||||
* Restrict sources in sealed mode.
|
||||
* Conseiller/Excitator add “mirror source” adapters.
|
||||
|
||||
### Phase 4 — Staleness and time
|
||||
|
||||
* Parse time anchors; enforce staleness budgets; add UI indicators and task refusal when budgets exceeded.
|
||||
* Notifications for expiring anchors.
|
||||
|
||||
### Phase 5 — Degradation matrix and UX
|
||||
|
||||
* Wire feature flags and fallbacks in Console and APIs.
|
||||
* Improve error messages with remediation guidance.
|
||||
|
||||
### Phase 6 — Evidence portability
|
||||
|
||||
* Portable evidence packages: export/import with full verification.
|
||||
* Document cross‑domain workflows.
|
||||
|
||||
---
|
||||
|
||||
## 8) Engineering tasks
|
||||
|
||||
**Air‑gap controller and sealing**
|
||||
|
||||
* [ ] Implement `airgap/controller` with persistent state and RBAC.
|
||||
* [ ] Add `GET /system/airgap/status`, `POST /system/airgap/seal`.
|
||||
* [ ] Provide cluster egress templates for Kubernetes and for docker‑compose.
|
||||
* [ ] Instrument startup checks to refuse running in sealed mode if egress rules aren’t applied.
|
||||
|
||||
**EgressPolicy integration**
|
||||
|
||||
* [ ] Create `pkg/egress` facade and replace all direct HTTP client constructions in services.
|
||||
* [ ] Add linter rule and CI check forbidding raw `http.NewClient` in server code.
|
||||
* [ ] Add unit tests for sealed and unsealed behavior.
|
||||
|
||||
**Mirror bundles**
|
||||
|
||||
* [ ] Implement TUF/DSSE verifiers and Merkle root builder.
|
||||
* [ ] Build content plug‑ins: advisories, VEX, policy packs, images.
|
||||
* [ ] Write `bundle_catalog` and `bundle_items` tables with RLS.
|
||||
* [ ] CLI: `stella mirror create|verify`, `stella airgap import|verify`.
|
||||
|
||||
**Conseiller/Excitator**
|
||||
|
||||
* [ ] Add mirror adapters for read‑only ingestion from bundle paths.
|
||||
* [ ] Persist source digests and bundle IDs on each linked record.
|
||||
* [ ] Unit tests to ensure no merge behavior is introduced by bundle ingestion.
|
||||
|
||||
**Policy Engine & Studio**
|
||||
|
||||
* [ ] Accept policy packs from bundles; track `policy_version` and `bundle_id`.
|
||||
* [ ] Add degradation notices for rules requiring external reputation; provide cached fallbacks.
|
||||
|
||||
**Task Runner & Orchestrator**
|
||||
|
||||
* [ ] Plan‑time validation against network calls; add `internal:` allow‑list mapping.
|
||||
* [ ] Emit sealed‑mode violations to Timeline with remediation text.
|
||||
|
||||
**Console**
|
||||
|
||||
* [ ] Status panel: sealed badge, last import, staleness meters.
|
||||
* [ ] Import wizard with verify results and catalog diff preview.
|
||||
* [ ] Degradation matrix UI and contextual tooltips.
|
||||
|
||||
**Observability & Forensics**
|
||||
|
||||
* [ ] Mark sealed mode in telemetry attributes.
|
||||
* [ ] Add portable evidence package export/import; verify on read.
|
||||
|
||||
**Authority & Tenancy**
|
||||
|
||||
* [ ] New scopes: `airgap:seal`, `airgap:import`, `airgap:status:read`.
|
||||
* [ ] Audit import actions with actor and trace ID.
|
||||
|
||||
**Docs**
|
||||
|
||||
* [ ] Author all pages listed in section 6, include signed‑time workflow diagrams.
|
||||
* [ ] Insert banner statement in each page.
|
||||
|
||||
**Testing**
|
||||
|
||||
* [ ] Sealed‑mode e2e: attempt egress; ensure refusal and remediation.
|
||||
* [ ] Bundle import e2e: corrupt DSSE, wrong root, tampered artifact → rejected.
|
||||
* [ ] Performance: large advisory bundle import within target time (see Acceptance).
|
||||
* [ ] Time drift scenarios and staleness budget enforcement.
|
||||
* [ ] Regression: ensure AOC rules unchanged in sealed mode.
|
||||
|
||||
---
|
||||
|
||||
## 9) Feature changes required in other components
|
||||
|
||||
* **Export Center:** add mirror bundle export profile, signed‑time token inclusion, and portable evidence packages.
|
||||
* **Notifications:** remove external webhooks by default in sealed mode; add local SMTP/syslog sinks.
|
||||
* **CLI Parity:** ensure all admin and import operations are exposed; add sealed‑mode safety prompts.
|
||||
* **Containerized Distribution:** ship **Bootstrap Pack** that includes all images and charts in a single oci‑archive set with index manifest.
|
||||
* **Observability:** disable remote exporters; include local dashboards; mark sealed mode in UI.
|
||||
* **Policy Studio:** enable offline authoring and export of policy sub‑bundles.
|
||||
* **VEX Consensus Lens:** ensure it operates solely on imported VEX statements; highlight coverage vs. stale.
|
||||
|
||||
> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 10) Acceptance criteria
|
||||
|
||||
* Environment can be **sealed** and verified via API, CLI, and network policies.
|
||||
* Import of a valid Mirror Bundle succeeds; DSSE, TUF, and Merkle validations recorded in `import_audit`.
|
||||
* Conseiller and Excitator operate only on imported sources; linkage reflects original source identities.
|
||||
* Policy packs are importable and versioned; rules that depend on external evidence show clear degradation.
|
||||
* Large bundle (e.g., 8–12 GB with images) imports in under 20 minutes on SSD storage and indexes advisories in under 5 minutes on a 4‑core node.
|
||||
* Console displays sealed badge, last import, staleness, and degradation matrix.
|
||||
* Attempted egress in sealed mode fails with `AIRGAP_EGRESS_BLOCKED` and remediation.
|
||||
* Portable evidence packages export and verify across separate enclaves.
|
||||
* All changes documented with the banner statement.
|
||||
|
||||
---
|
||||
|
||||
## 11) Risks and mitigations
|
||||
|
||||
* **Key management complexity:** rotate TUF roots with dual‑control workflow and explicit docs; fail‑safe to previous root if rotation bundle absent.
|
||||
* **Staleness risk:** enforce budgets and block risk‑critical jobs when expired; provide monitoring and notifications for impending staleness.
|
||||
* **Operator error during import:** dry‑run verification, diff preview of catalog changes, and ability to roll back via content address.
|
||||
* **Hidden egress paths:** CI lints and runtime guardrails; network policies enforced at cluster layer.
|
||||
* **Bundle size bloat:** Zstandard compression, delta bundles, and selective content flags for creation.
|
||||
|
||||
---
|
||||
|
||||
## 12) Philosophy
|
||||
|
||||
* **Predictable over perfect:** deterministic, explainable results beat unknown “live” results in sensitive networks.
|
||||
* **Trust is earned:** every offline exchange is signed, verifiable, and auditable.
|
||||
* **Degrade transparently:** when features reduce capability, explain it and guide remediation.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
71
docs/airgap/airgap-mode.md
Normal file
71
docs/airgap/airgap-mode.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# Air-Gapped Mode Playbook
|
||||
|
||||
> Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
## Overview
|
||||
|
||||
Air-Gapped Mode is the supported operating profile for deployments with **zero external egress**. All inputs arrive via signed mirror bundles, and every surface (CLI, Console, APIs, schedulers, scanners) operates under sealed-network constraints while preserving Aggregation-Only Contract invariants.
|
||||
|
||||
- **Primary components:** Web Services API, Console, CLI, Orchestrator, Task Runner, Conseiller (Feedser), Excitator (VEXer), Policy Engine, Findings Ledger, Export Center, Authority & Tenancy, Notifications, Observability & Forensics.
|
||||
- **Surfaces:** offline bootstrap, mirror ingestion, deterministic jobs, offline advisories/VEX/policy packs/notifications, evidence exports.
|
||||
- **Dependencies:** Export Center, Containerized Distribution, Authority-backed scopes & tenancy, Observability & Forensics, Policy Studio.
|
||||
|
||||
## Guiding principles
|
||||
|
||||
1. **Zero egress:** all outbound network calls are disabled unless explicitly allowed. Any feature requiring online data must degrade gracefully with clear UX messaging.
|
||||
2. **Deterministic inputs:** the platform accepts only signed Mirror Bundles (advisories, VEX, policy packs, vendor feeds, images, dashboards). Bundles carry provenance attestations and chain-of-custody manifests.
|
||||
3. **Auditable exchange:** every import/export records provenance, signatures, and operator identity. Evidence bundles and reports remain verifiable offline.
|
||||
4. **Aggregation-Only Contract compliance:** Conseiller and Excitator continue to aggregate without mutating source records, even when ingesting mirrored feeds.
|
||||
5. **Operator ergonomics:** offline bootstrap, upgrade, and verification steps are reproducible and scripted.
|
||||
|
||||
## Lifecycle & modes
|
||||
|
||||
| Mode | Description | Tooling |
|
||||
| --- | --- | --- |
|
||||
| Connected | Standard deployment with online feeds. Operators use Export Center to build mirror bundles for offline environments. | `stella export bundle create --profile mirror:full` |
|
||||
| Staging mirror | Sealed host that fetches upstream feeds, runs validation, and signs mirror bundles. | Export Center, cosign, bundle validation scripts |
|
||||
| Air-gapped | Production cluster with egress sealed, consuming validated bundles, issuing provenance for inward/outward transfers. | Mirror import CLI, sealed-mode runtime flags |
|
||||
|
||||
### Installation & bootstrap
|
||||
|
||||
1. Prepare mirror bundles (images, charts, advisories/VEX, policy packs, dashboards, telemetry configs).
|
||||
2. Transfer bundles via approved media and validate signatures (`cosign verify`, bundle manifest hash).
|
||||
3. Deploy platform using offline artefacts (`helm install --set airgap.enabled=true`), referencing local registry/object storage.
|
||||
|
||||
### Updates
|
||||
|
||||
1. Staging host generates incremental bundles (mirror delta) with provenance.
|
||||
2. Offline site imports bundles via the CLI (`stella airgap import --bundle`) and records chain-of-custody.
|
||||
3. Scheduler triggers replay jobs with deterministic timelines; results remain reproducible across imports.
|
||||
|
||||
## Component responsibilities
|
||||
|
||||
| Component | Offline duties |
|
||||
| --- | --- |
|
||||
| Export Center | Produce full/delta mirror bundles, signed manifests, provenance attestations. |
|
||||
| Authority & Tenancy | Provide offline scope enforcement, short-lived tokens, revocation via local CRLs. |
|
||||
| Conseiller / Excitator | Ingest mirrored advisories/VEX, enforce AOC, versioned observations. |
|
||||
| Policy Engine & Findings Ledger | Replay evaluations using offline feeds, emit explain traces, support sealed-mode hints. |
|
||||
| Notifications | Deliver locally via approved channels (email relay, webhook proxies) or queue for manual export. |
|
||||
| Observability | Collect metrics/logs/traces locally, generate forensic bundles for external analysis. |
|
||||
|
||||
## Operational guardrails
|
||||
|
||||
- **Network policy:** enforce allowlists (`airgap.egressAllowlist=[]`). Any unexpected outbound request raises an alert.
|
||||
- **Bundle validation:** double-sign manifests (bundle signer + site-specific cosign key); reject on mismatch.
|
||||
- **Time synchronization:** rely on local NTP or manual clock audits; many signatures require monotonic time.
|
||||
- **Key rotation:** plan for offline key ceremonies; Export Center and Authority document rotation playbooks.
|
||||
- **Incident response:** maintain scripts for replaying imports, regenerating manifests, and exporting forensic data without egress.
|
||||
|
||||
## Testing & verification
|
||||
|
||||
- Integration tests mimic offline installs by running with `AIRGAP_ENABLED=true` in CI.
|
||||
- Mirror bundles include validation scripts to compare hash manifests across staging and production.
|
||||
- Sealed-mode smoke tests ensure services fail closed when attempting egress.
|
||||
|
||||
## References
|
||||
|
||||
- Export workflows: `docs/modules/export-center/overview.md`
|
||||
- Policy sealed-mode hints: `docs/policy/overview.md`
|
||||
- Observability forensic bundles: `docs/modules/telemetry/architecture.md`
|
||||
- Runtime posture enforcement: `docs/modules/zastava/operations/runtime.md`
|
||||
@@ -1,13 +1,13 @@
|
||||
# Aggregation-Only Contract (AOC) Guardrails
|
||||
|
||||
The Aggregation-Only Contract keeps ingestion services deterministic and policy-neutral. Use these checkpoints whenever you add or modify backlog items:
|
||||
|
||||
1. **Ingestion writes raw facts only.** Concelier and Excititor append immutable observations/linksets. No precedence, severity, suppression, or "safe fix" hints may be computed at ingest time.
|
||||
2. **Derived semantics live elsewhere.** Policy Engine overlays, Vuln Explorer composition, and downstream reporting layers attach severity, precedence, policy verdicts, and UI hints.
|
||||
3. **Provenance is mandatory.** Every ingestion write must include original source metadata, digests, and signing/provenance evidence when available. Reject writes lacking provenance.
|
||||
4. **Deterministic outputs.** Given the same inputs, ingestion must produce identical documents, hashes, and event payloads across reruns.
|
||||
5. **Guardrails everywhere.** Roslyn analyzers, schema validators, and CI smoke tests should fail builds that attempt forbidden writes.
|
||||
|
||||
For detailed roles and ownership boundaries, see `AGENTS.md` at the repo root and the module-specific `ARCHITECTURE_*.md` dossiers.
|
||||
|
||||
Need the full contract? Read the [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md) for schemas, error codes, and migration guidance.
|
||||
# Aggregation-Only Contract (AOC) Guardrails
|
||||
|
||||
The Aggregation-Only Contract keeps ingestion services deterministic and policy-neutral. Use these checkpoints whenever you add or modify backlog items:
|
||||
|
||||
1. **Ingestion writes raw facts only.** Concelier and Excititor append immutable observations/linksets. No precedence, severity, suppression, or "safe fix" hints may be computed at ingest time.
|
||||
2. **Derived semantics live elsewhere.** Policy Engine overlays, Vuln Explorer composition, and downstream reporting layers attach severity, precedence, policy verdicts, and UI hints.
|
||||
3. **Provenance is mandatory.** Every ingestion write must include original source metadata, digests, and signing/provenance evidence when available. Reject writes lacking provenance.
|
||||
4. **Deterministic outputs.** Given the same inputs, ingestion must produce identical documents, hashes, and event payloads across reruns.
|
||||
5. **Guardrails everywhere.** Roslyn analyzers, schema validators, and CI smoke tests should fail builds that attempt forbidden writes.
|
||||
|
||||
For detailed roles and ownership boundaries, see `AGENTS.md` at the repo root and the module-specific dossiers under `docs/modules/<module>/architecture.md`.
|
||||
|
||||
Need the full contract? Read the [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md) for schemas, error codes, and migration guidance.
|
||||
|
||||
@@ -1,357 +0,0 @@
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 17: SDKs and OpenAPI Docs
|
||||
|
||||
**Short name:** SDKs & OpenAPI
|
||||
**Primary components:** API Gateway, Web Services, Policy Engine, Conseiller (Feedser), Excitator (VEXer), Orchestrator, Findings Ledger, Export Center, Authority & Tenancy, Console, CLI
|
||||
**Surfaces:** OpenAPI 3.1 contracts, language SDKs (TS/Node, Python, Go, Java, C#), dev portal, examples, mock server, conformance tests, changelogs, versioning, deprecations
|
||||
**Dependencies:** Authority‑Backed Scopes & Tenancy, CLI Parity, Export Center, Notifications Studio, Air‑Gapped Mode, Observability
|
||||
|
||||
**AOC ground rule reminder:** Conseiller and Excitator aggregate and link advisories/VEX. They never merge or mutate source records. SDKs must preserve this invariant and expose source identity in all models.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
A contract‑first program that standardizes StellaOps’ APIs with OpenAPI 3.1 and ships official, versioned SDKs for popular languages. It includes:
|
||||
|
||||
* A single **source‑of‑truth OpenAPI** for each service and a canonical aggregate spec.
|
||||
* **Generated SDKs** with idiomatic ergonomics, retries, auth helpers, pagination cursors, streaming downloads, and typed error envelopes.
|
||||
* A **developer portal** with interactive reference, runnable examples, and “copy‑curl” snippets.
|
||||
* **Mock server & conformance tests** so changes are validated against the contract before code ships.
|
||||
* **Versioning & deprecation policy**, automated changelogs, and notification hooks.
|
||||
* **Air‑gapped bundles** of docs and SDKs for disconnected environments.
|
||||
|
||||
Net result: partners and internal teams integrate quickly without reverse‑engineering request bodies from error logs.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* Reduce friction and support load with a single, accurate contract.
|
||||
* Make the platform extensible: third parties can build automation, dashboards, and policy pipelines without trawling source code.
|
||||
* Enforce stability: contract linting and backwards‑compat checks prevent accidental breakage.
|
||||
* Bring CLI and Console parity to programmatic users through first‑class clients.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work
|
||||
|
||||
### 3.1 Source of truth and layout
|
||||
|
||||
* Each service owns a **module-scoped OAS** file: `src/Api/StellaOps.Api.OpenApi/<service>/openapi.yaml`.
|
||||
* Authority authentication/token surface now lives at `src/Api/StellaOps.Api.OpenApi/authority/openapi.yaml`, covering `/token`, `/introspect`, `/revoke`, and `/jwks` flows with examples and scope catalog metadata.
|
||||
* An aggregate spec `src/Api/StellaOps.Api.OpenApi/stella.yaml` is produced by build tooling that composes per-service specs, resolves `$ref`s, and validates cross-service schemas.
|
||||
* JSON Schema dialect: 2020‑12 (OpenAPI 3.1). No vendor‑specific features for core models.
|
||||
* Every response and error has at least one **validated example**.
|
||||
|
||||
### 3.2 API conventions (normative)
|
||||
|
||||
* **Paths:** `/v1/{resource}`, plural nouns. Subresources use `/v1/resources/{id}/subresources`.
|
||||
* **Identifiers:** `id` fields are ULID/UUIDv7 as strings.
|
||||
* **Pagination:** cursor‑based: `?cursor=<token>&limit=<n>`, response envelope includes `next_cursor`.
|
||||
* **Sorting/filtering:** `?sort=field:asc|desc`, `?filter[field]=op:value` with documented operators.
|
||||
* **Idempotency:** POST operations that create or mutate accept `Idempotency-Key`.
|
||||
* **Errors:** single envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": "STRING_CODE",
|
||||
"message": "human friendly",
|
||||
"details": { "field": "value" },
|
||||
"trace_id": "..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Standard codes include `AIRGAP_EGRESS_BLOCKED`, `POLICY_VIOLATION`, `NOT_FOUND`, `RATE_LIMITED`.
|
||||
* **Auth:** OAuth2 client credentials and PAT. Scopes are explicit (see 14: Authority‑Backed Scopes). Tenancy via claims; optional override header: `X-Stella-Scope: tenant/<id>` if the token permits delegation.
|
||||
* **Content negotiation:** JSON only for request/response unless endpoint is a stream or file download (`application/octet-stream`).
|
||||
* **Long‑running operations:** either webhooks (if enabled) or polling via `operation_id` resource.
|
||||
|
||||
### 3.3 Versioning and deprecation
|
||||
|
||||
* **SemVer** for the aggregate API: `v1`, `v2` in base path.
|
||||
* Backwards‑compatible changes allowed in minor versions (add fields, new optional params).
|
||||
* Breaking changes require new major version and coexistence for a **deprecation window** (min 12 months) with:
|
||||
|
||||
* Deprecation headers: `Deprecation: true`, `Sunset: <rfc1123 date>`, `Link: <doc rel="deprecation">`.
|
||||
* Portal banners and Notifications Studio broadcast.
|
||||
|
||||
### 3.4 Governance and linting
|
||||
|
||||
* Enforce naming, pagination, error envelope, and example requirements via an OAS linter.
|
||||
* CI gate: no PR merges if OAS validation fails or coverage < 100% for operation examples.
|
||||
* **Compatibility check:** diff new OAS vs previous release, fail on breaking changes unless explicitly flagged.
|
||||
|
||||
### 3.5 SDK generation
|
||||
|
||||
* Initial languages: **TypeScript/Node**, **Python**, **Go**, **Java**. C# and Rust are follow‑ups.
|
||||
* Generated via a stable, reproducible toolchain. Post‑generation patches are applied by templates, not hand edits.
|
||||
* **Capabilities:**
|
||||
|
||||
* Auth helpers: PAT and OAuth2.
|
||||
* Retries with decorrelated jitter and `Retry‑After` respect.
|
||||
* Pluggable HTTP transport for proxies and air‑gapped environments.
|
||||
* Binary download helpers and upload helpers for multipart endpoints.
|
||||
* Paginators that yield items and handle `next_cursor`.
|
||||
* Rich error types mapping `error.code` to language enums.
|
||||
* Telemetry hooks (before/after request callbacks).
|
||||
* **Packaging:**
|
||||
|
||||
* TS: npm package with ESM and CJS builds, types included.
|
||||
* Python: PyPI package, Pydantic‑friendly models, type hints.
|
||||
* Go: module with context‑aware methods and `io.Reader` streaming.
|
||||
* Java: Maven coordinates, builder pattern, OkHttp/HTTP client provider.
|
||||
* **Versioning:** SDK major matches API major. Minor/patch track generator changes only.
|
||||
|
||||
### 3.6 Dev portal and artifacts
|
||||
|
||||
* **Reference docs** auto‑built from the aggregate OAS with searchable nav, schema diagrams, and example blocks.
|
||||
* **Try‑it** panel wired to the sandbox environment (disabled in air‑gap).
|
||||
* **Download center:** links to SDKs, changelogs, and Postman/HTTP collection exports.
|
||||
* **.well‑known discovery:** `GET /.well-known/openapi` returns the canonical spec.
|
||||
|
||||
### 3.7 Conformance testing
|
||||
|
||||
* **Mock server** generated from OAS for contract tests.
|
||||
* **Replay tests**: real services are validated against the OAS via request/response capture; deviations fail CI.
|
||||
* **Golden examples**: every endpoint has recorded examples exercised in tests.
|
||||
|
||||
### 3.8 Air‑Gapped support
|
||||
|
||||
* Export Center can build a **Docs & SDKs bundle**: `stella export devportal --offline`, including HTML docs, specs, and packages.
|
||||
* SDKs avoid network discovery and accept explicit base URLs; no auto‑updates.
|
||||
|
||||
### 3.9 Domain‑specific notes
|
||||
|
||||
* **Conseiller/Excitator:** models expose `source_id`, `source_type`, `source_digest`. SDKs never hide source multiplicity.
|
||||
* **Policy Engine:** policy documents are versioned; SDK supports dry‑run/simulate endpoints with structured explanations.
|
||||
* **Findings Ledger:** paginated listing includes stable, filterable fields for evidence export.
|
||||
|
||||
---
|
||||
|
||||
## 4) Architecture
|
||||
|
||||
### 4.1 New modules
|
||||
|
||||
* `src/Api/StellaOps.Api.OpenApi/*` per service and aggregate composer
|
||||
* `src/Api/StellaOps.Api.Governance` OAS linter rules and compatibility checker
|
||||
* `src/Sdk/StellaOps.Sdk.Generator` codegen drivers, post‑processing templates, smoke tests
|
||||
* `src/Sdk/StellaOps.Sdk.Release` packaging, signing, publishing
|
||||
* `src/DevPortal/StellaOps.DevPortal.Site` static generator and assets
|
||||
* `test/contract` mock server config, golden examples
|
||||
* `src/ExportCenter/StellaOps.ExportCenter.DevPortalOffline` bundler
|
||||
|
||||
### 4.2 Build flow
|
||||
|
||||
1. Validate per‑service specs → compose aggregate → lint → compatibility diff.
|
||||
2. Generate SDKs → build → run language‑level tests → publish to internal registry.
|
||||
3. Build dev portal and publish.
|
||||
4. Optionally build offline bundle.
|
||||
|
||||
### 4.3 Runtime contracts
|
||||
|
||||
* `GET /.well-known/openapi` per service and at the gateway.
|
||||
* All services embed `x-stella-service` and `x-stella-version` extensions for traceability.
|
||||
|
||||
---
|
||||
|
||||
## 5) APIs and contracts (select)
|
||||
|
||||
* **Discovery**: `GET /.well-known/openapi` → JSON or YAML.
|
||||
* **Errors**: standard envelope (see 3.2).
|
||||
* **Rate limits**: expose `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`.
|
||||
* **Operations**: long‑running ops expose `operation_id` and `status` via `GET /v1/operations/{id}`.
|
||||
|
||||
---
|
||||
|
||||
## 6) Documentation changes
|
||||
|
||||
Create or update:
|
||||
|
||||
1. `/docs/api/overview.md`
|
||||
|
||||
* API surface, auth, tenancy, pagination, idempotency, rate limits.
|
||||
2. `/docs/api/conventions.md`
|
||||
|
||||
* Path, naming, errors, filters, sorting, examples.
|
||||
3. `/docs/api/versioning.md`
|
||||
|
||||
* SemVer policy, deprecation windows, headers, migration playbooks.
|
||||
4. `/docs/api/reference/`
|
||||
|
||||
* Auto‑generated OAS site; link into service pages.
|
||||
5. `/docs/sdks/overview.md`
|
||||
|
||||
* Supported languages, install, hello‑world, retry/auth patterns.
|
||||
6. `/docs/sdks/typescript.md`, `/python.md`, `/go.md`, `/java.md`
|
||||
|
||||
* Language‑specific guides, snippets, paginator usage, streaming.
|
||||
7. `/docs/devportal/publishing.md`
|
||||
|
||||
* Build pipeline, offline bundle steps.
|
||||
8. `/docs/contributing/api-contracts.md`
|
||||
|
||||
* How to edit OAS, lint rules, compatibility checks, examples.
|
||||
9. `/docs/testing/contract-testing.md`
|
||||
|
||||
* Mock server, golden examples, replay tests.
|
||||
10. `/docs/security/auth-scopes.md`
|
||||
|
||||
* OAuth2, PAT, scope mapping, tenancy header.
|
||||
11. `/docs/airgap/devportal-offline.md`
|
||||
|
||||
* Air‑gapped docs and SDK bundle usage.
|
||||
|
||||
Add the banner at the top of each page:
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Implementation plan
|
||||
|
||||
### Phase 1 — Foundations
|
||||
|
||||
* Establish per‑service OAS skeletons and the aggregate composer.
|
||||
* Introduce linting and compatibility checks in CI.
|
||||
* Define the standard error envelope and migrate services.
|
||||
|
||||
### Phase 2 — Reference & discovery
|
||||
|
||||
* Implement `/.well-known/openapi` for gateway and services.
|
||||
* Build the dev portal with search, schema diagrams, and examples.
|
||||
|
||||
### Phase 3 — SDKs (TS, Python, Go, Java)
|
||||
|
||||
* Implement generator drivers and templates.
|
||||
* Publish alpha packages internally; integrate in CLI and Console integration tests.
|
||||
* Add paginators, retries, auth helpers, and streaming.
|
||||
|
||||
### Phase 4 — Conformance & examples
|
||||
|
||||
* Wire mock server into PR CI.
|
||||
* Record golden example fixtures and replay tests against staging.
|
||||
* Automate example extraction into docs.
|
||||
|
||||
### Phase 5 — Release automation & deprecation
|
||||
|
||||
* Automate changelogs from OAS diffs.
|
||||
* Notifications Studio integration for API deprecations.
|
||||
* Offline dev portal bundle through Export Center.
|
||||
|
||||
### Phase 6 — Follow‑ups
|
||||
|
||||
* C# and Rust SDKs, Postman/HTTP collections, sample apps repo.
|
||||
|
||||
---
|
||||
|
||||
## 8) Engineering tasks
|
||||
|
||||
**OAS & governance**
|
||||
|
||||
* [ ] Create `src/Api/StellaOps.Api.OpenApi/<service>/openapi.yaml` for all services with minimal paths and shared components.
|
||||
* [ ] Implement aggregate composer and `$ref` resolver.
|
||||
* [ ] Add CI job: lint, validate, compatibility diff; block merges on failure.
|
||||
* [ ] Migrate all endpoints to standard error envelope and provide examples.
|
||||
|
||||
**Discovery & portal**
|
||||
|
||||
* [ ] Implement `GET /.well-known/openapi` at service and gateway.
|
||||
* [ ] Build dev portal: nav, search, schema viewer, try‑it (non‑prod), copy‑curl.
|
||||
* [ ] Add version selector for v1/v2 specs.
|
||||
|
||||
**SDKs**
|
||||
|
||||
* [ ] Generator driver with pinned templates; forbid manual edits in generated folders.
|
||||
* [ ] TS SDK: ESM/CJS build, tree‑shaking, paginator, middleware hooks.
|
||||
* [ ] Python SDK: async and sync clients, type hints, file upload/download helpers.
|
||||
* [ ] Go SDK: context‑first API, streaming, error type mapping.
|
||||
* [ ] Java SDK: builder pattern, HTTP client provider abstraction.
|
||||
* [ ] Common: retries, `Retry‑After` handling, idempotency key helper, auth helpers, telemetry hooks.
|
||||
* [ ] Language‑specific tests and smoke examples.
|
||||
|
||||
**Conformance**
|
||||
|
||||
* [ ] Mock server config with operation examples.
|
||||
* [ ] Replay tests against staging; fail on schema drift.
|
||||
* [ ] Golden example extraction pipeline.
|
||||
|
||||
**Air‑Gapped**
|
||||
|
||||
* [ ] Export Center job: `devportal --offline` producing HTML docs, specs, and package artifacts.
|
||||
* [ ] SDKs accept explicit base URLs; disable online discovery.
|
||||
|
||||
**Authority & Tenancy**
|
||||
|
||||
* [ ] Document scopes per endpoint in OAS (`securitySchemes` + `security` blocks).
|
||||
* [ ] Implement optional `X-Stella-Scope` override with validation.
|
||||
|
||||
**Release automation**
|
||||
|
||||
* [ ] Version bump tooling for OAS and SDKs; SemVer aligned.
|
||||
* [ ] Auto‑generate `CHANGELOG.md` from OAS diffs.
|
||||
* [ ] Publish to registries with signed artifacts and provenance.
|
||||
|
||||
**Docs**
|
||||
|
||||
* [ ] Author all pages listed in section 6; embed code snippets pulled from tested examples.
|
||||
* [ ] Insert banner statement in each page.
|
||||
|
||||
**Testing**
|
||||
|
||||
* [ ] Contract tests in PR CI; 100% operation coverage with at least one example.
|
||||
* [ ] Language SDK integration tests against mock server and staging.
|
||||
* [ ] Backwards‑compat test suite comparing last N releases.
|
||||
|
||||
---
|
||||
|
||||
## 9) Feature changes required in other components
|
||||
|
||||
* **Web Services:** unify on error envelope, pagination, idempotency handling, and deprecation headers.
|
||||
* **CLI:** consume the official TS or Go SDK instead of bespoke HTTP calls; this enforces parity.
|
||||
* **Console:** use SDKs for backend calls where appropriate; helps dogfood the clients.
|
||||
* **Export Center:** add `devportal --offline` and package signing.
|
||||
* **Observability:** include `x-stella-service` and API version attributes in spans; trace IDs mirrored in error responses.
|
||||
* **Notifications Studio:** templates for API deprecations and SDK updates.
|
||||
* **Air‑Gapped Mode:** ship offline dev portal and SDKs bundle; console disables try‑it.
|
||||
|
||||
> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 10) Acceptance criteria
|
||||
|
||||
* Aggregate OpenAPI validates, lints cleanly, and covers 100% of public endpoints with examples.
|
||||
* `/.well-known/openapi` available at gateway and service level.
|
||||
* Dev portal builds with search, example blocks, and version selector.
|
||||
* TS/Python/Go/Java SDKs publish successfully; each has paginators, retries, auth helpers, streaming, and typed errors.
|
||||
* CLI integrations pass using SDKs.
|
||||
* Contract tests run in PR CI; schema drift causes failures.
|
||||
* Deprecation headers and Notifications Studio flow proven in a staged deprecation.
|
||||
* Offline dev portal bundle exports and renders in a sealed environment.
|
||||
|
||||
---
|
||||
|
||||
## 11) Risks and mitigations
|
||||
|
||||
* **Spec drift vs code reality:** mock‑first development and replay tests keep services aligned with OAS.
|
||||
* **Generator churn:** pin generator and templates; only update via planned minor releases.
|
||||
* **Breaking changes under pressure:** enforce compatibility gate and documented exception process.
|
||||
* **SDK ergonomics mismatch:** run language‑native design reviews with maintainers before GA.
|
||||
* **Air‑gapped constraints:** prebuild full offline bundles; avoid dynamic CDN assets in docs.
|
||||
|
||||
---
|
||||
|
||||
## 12) Philosophy
|
||||
|
||||
* **Contract first, code second.** The spec is the product; servers and SDKs are implementations.
|
||||
* **Stability over cleverness.** Boring, predictable APIs beat “magical” behavior.
|
||||
* **Truth preservation.** Never hide or merge advisory/VEX sources; surface provenance everywhere.
|
||||
* **Automation everywhere.** Humans shouldn’t manually edit generated code or publish packages.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
51
docs/api/sdk-openapi-program.md
Normal file
51
docs/api/sdk-openapi-program.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# SDK & OpenAPI Program
|
||||
|
||||
> Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
## Overview
|
||||
|
||||
The SDK & OpenAPI program delivers canonical OpenAPI 3.1 contracts for every Stella Ops surface, plus officially supported SDKs (TypeScript/Node, Python, Go, Java, C#). It ensures backwards-compatible evolution, documentation, and offline availability.
|
||||
|
||||
- **Primary components:** API Gateway, Web Services, Policy Engine, Conseiller, Excitator, Orchestrator, Findings Ledger, Export Center, Authority & Tenancy, Console, CLI.
|
||||
- **Surfaces:** OpenAPI specs, language SDKs, developer portal, examples, mock server, conformance tests, changelog feeds, deprecation notices.
|
||||
- **Dependencies:** Authority scopes/tenancy, CLI parity, Export Center, Notifications, Air-Gapped Mode, Observability.
|
||||
|
||||
## Program pillars
|
||||
|
||||
1. **Contract-first:** treat OpenAPI specs as the source of truth. CI validates schemas, compatibility, and documentation generation.
|
||||
2. **SDK parity:** language SDKs cover the same surfaces with deterministic clients, pagination helpers, and typed models mirroring Aggregation-Only Contract semantics.
|
||||
3. **Version discipline:** semantic versioning for specs and SDKs, release notes, deprecation windows, and automated change alerts via Notifications.
|
||||
4. **Offline readiness:** specs and SDK bundles ship in Mirror Bundles for air-gapped environments; examples include smoke tests.
|
||||
5. **Observability:** telemetry around SDK usage, spec download metrics, and error reporting funnels back into product decisions.
|
||||
|
||||
## Deliverables
|
||||
|
||||
| Workstream | Deliverable |
|
||||
| --- | --- |
|
||||
| Spec authoring | Unified OpenAPI 3.1 documents per service plus aggregate spec; lint rules; schema registries. |
|
||||
| SDK generation | Language-specific clients with idiomatic ergonomics, retries, pagination, long-running operation helpers, unit + integration tests. |
|
||||
| Dev portal | Consolidated documentation, guides, changelog, copy/paste examples, quickstart scripts. |
|
||||
| Testing | Contract tests against staging, mock server for integration tests, compatibility verification per release. |
|
||||
| Release ops | Automated CI pipelines, version bump workflows, release notes, deprecation policies. |
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Aggregation-Only Contract compliance:** SDKs expose raw advisory/VEX objects without hidden merges; all derived fields require explicit Policy Engine calls.
|
||||
- **Security:** enforce scopes via SDK configuration; redact secrets; support DPoP/mTLS and offline token provisioning.
|
||||
- **Compatibility:** maintain backwards-compatible paths for at least two minor releases; log warnings on deprecated endpoints.
|
||||
- **Documentation:** publish examples for common workflows (scan, policy evaluate, export, attestation) with language parity.
|
||||
|
||||
## Roadmap checkpoints
|
||||
|
||||
1. Baseline OpenAPI specs extracted from gateway, validated, and published.
|
||||
2. TypeScript/Node SDK as pilot, followed by Python and Go.
|
||||
3. Developer portal launch with SDK docs, quickstarts, and mock server.
|
||||
4. Offline kit integration (mirror bundles include specs + SDK tarballs).
|
||||
5. Runtime alerting for breaking changes and dependency vulnerabilities.
|
||||
|
||||
## References
|
||||
|
||||
- API gateway integration: `docs/modules/platform/architecture-overview.md`
|
||||
- Policy/Findings models: `docs/modules/policy/architecture.md`, `docs/modules/vuln-explorer/architecture.md`
|
||||
- Export bundle distribution: `docs/modules/export-center/overview.md`
|
||||
- Offline workflows: `docs/airgap/airgap-mode.md`
|
||||
@@ -1,135 +0,0 @@
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 19: Attestor Console
|
||||
|
||||
**Short name:** Attestor Console
|
||||
**Primary components:** Attestation Service, Policy Studio, Authority & Tenancy, Export Center, Observability, CLI, StellaOps Console
|
||||
**Surfaces:** Web UI, REST APIs, CLI, SDKs, docs
|
||||
|
||||
**AOC ground rule reminder:** Conseiller and Excitator only aggregate and link. They never merge sources. Attestations must reference the exact subject digests and their source provenance. No collapsing of distinct advisories or VEX statements.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
Attestor Console centralizes creation, verification, browsing, and operationalization of software supply chain attestations within StellaOps. It covers build provenance, SBOM attestations, VEX statements, scan results, policy evaluations, risk profile evidence, and custom facts—all signed and verifiable end-to-end.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
Trustworthy pipelines need signed evidence, repeatable verification, explainability, and policy enforcement. Attestor Console weaves attestations into the rest of Stella (AOC, Policy Studio gates, Risk Profiles, Notifications, Export Center) to reduce blind spots and accelerate compliance.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work
|
||||
|
||||
### 3.1 Roles, identities, scopes
|
||||
|
||||
* Subjects: immutable digests for artifacts (images, packages, SBOMs, findings).
|
||||
* Issuers: identities that sign attestations (builders, scanners, policy engines).
|
||||
* Scopes: tenant/project/environment context enforced by Authority & Tenancy.
|
||||
|
||||
### 3.2 Payload types
|
||||
|
||||
Supports DSSE envelopes for BuildProvenance v1, SBOMAttestation v1, VEXAttestation v1, ScanResults v1, PolicyEvaluation v1, RiskProfileEvidence v1, and CustomEvidence v1. All payloads include subject, issuer, scope, materials, provenance, policy context, and versioned schemas.
|
||||
|
||||
### 3.3 Envelope & signature model
|
||||
|
||||
DSSE envelopes with multi-signature support, Ed25519/ECDSA keys, KMS/HSM/FIDO2 drivers, transparency log witnesses, and detached payload storage. Identity policies ensure least privilege and traceability.
|
||||
|
||||
### 3.4 Verification pipeline
|
||||
|
||||
Runs at ingest, policy gates, and interactively. Steps: resolve subject, fetch envelopes + witness proofs, validate DSSE structure/signatures, evaluate issuer trust and policies, produce cached verification reports.
|
||||
|
||||
### 3.5 Verification policies
|
||||
|
||||
Policy Studio authored rules covering required evidence types, allowed issuers, freshness, transparency requirements, signature counts, and waivers. Supports scoped overrides and defaults.
|
||||
|
||||
### 3.6 UI workflows
|
||||
|
||||
Evidence Browser, Verification Reports, Chain of Custody Graph, Key & Issuer Management, Attestation Workbench, and Bulk Verification views in Console.
|
||||
|
||||
### 3.7 CLI & SDK
|
||||
|
||||
Commands: `stella attest sign`, `verify`, `list`, `fetch`, `key create/import/rotate/revoke`. SDKs expose Sign/Verify/List/Fetch APIs.
|
||||
|
||||
### 3.8 Data model
|
||||
|
||||
Tables for attestations, issuers, verification reports, transparency index, key store. Indexed by subject digest, type, issuer, and timestamps.
|
||||
|
||||
### 3.9 Storage & air gap
|
||||
|
||||
Store envelopes in CAS object storage; optionally mirror transparency logs. `stella export attestation-bundle` enables offline transfer. Policies can relax witness requirements for sealed environments while logging the gap.
|
||||
|
||||
### 3.10 Observability & security
|
||||
|
||||
Spans, metrics, logs for signing and verification. Private keys never leave KMS/HSM. Revocation/rotation supported. Verification rejects mismatched subjects and ensures AOC invariants for scan/VEX evidence.
|
||||
|
||||
### 3.11 Performance
|
||||
|
||||
Use compressed JSON payloads, cached verification results, batched operations, and concurrency controls. P95 target: 1k envelopes/min on a single worker.
|
||||
|
||||
---
|
||||
|
||||
## 4) Architecture
|
||||
|
||||
New services (`src/Attestor/StellaOps.Attestor/`), libraries (`src/Attestor/StellaOps.Attestor.Envelope/`, `src/Attestor/StellaOps.Attestor.Types/`, `src/Attestor/StellaOps.Attestor.Verify/`), CLI (`src/Cli/StellaOps.Cli/`), export tooling (`src/ExportCenter/StellaOps.ExportCenter.AttestationBundles/`), and shared KMS providers (`src/__Libraries/StellaOps.Cryptography.Kms/`). REST endpoints documented in OpenAPI.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Requires new pages for overview, payloads, policies, workflows, key management, transparency, air gap, console UI, CLI, and updated security invariants, all with the imposed rule banner.
|
||||
|
||||
---
|
||||
|
||||
## 6) Implementation plan
|
||||
|
||||
Six phases: Foundations; Policies & UI; Scan & VEX support; Transparency & keys; Bulk & air gap; Performance & hardening.
|
||||
|
||||
---
|
||||
|
||||
## 7) Engineering tasks
|
||||
|
||||
Detailed tasks across envelope/crypto, payload schemas, APIs, Policy Studio integration, Console UI, CLI, transparency, bulk operations, observability, security, docs, and testing.
|
||||
|
||||
---
|
||||
|
||||
## 8) Feature changes required elsewhere
|
||||
|
||||
Policy Studio (VerificationPolicy), Export Center (attestation bundles), Authority & Tenancy, SBOM/Vulnerability explorers, Notifications, Observability. All must inherit the imposed rule and update docs accordingly.
|
||||
|
||||
---
|
||||
|
||||
## 9) Acceptance criteria
|
||||
|
||||
Signing and verification for all payloads, policy enforcement, Console views, bulk verification, export/import for air gap, observability coverage, AOC invariants respected, documented OpenAPI endpoints. Tests and performance targets met.
|
||||
|
||||
---
|
||||
|
||||
## 10) Risks & mitigations
|
||||
|
||||
Key compromise, parsing bugs, policy complexity, transparency outages addressed via rotation workflows, fuzz tests, curated starter policies, and fallback/mirroring strategies.
|
||||
|
||||
---
|
||||
|
||||
## 11) Philosophy
|
||||
|
||||
Evidence first, scoped identities, cheap verification, portable attestations, truth preservation.
|
||||
|
||||
---
|
||||
|
||||
## 12) Examples
|
||||
|
||||
Includes abbreviated SBOM attestation and verification report JSON samples illustrating required fields and outcomes.
|
||||
|
||||
---
|
||||
|
||||
## 13) Cross-epic documentation updates
|
||||
|
||||
Cross-link Attestor docs from Policy Studio, Export Center, Air-Gapped, Observability, Risk Profiles, SBOM Graph, and Vulnerability Explorer pages, maintaining the imposed rule banner.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
@@ -131,7 +131,7 @@ The script should emit a parity report that feeds into the Downloads workspace (
|
||||
|
||||
- [ ] Matrix reflects latest command availability (statuses accurate, task IDs linked).
|
||||
- [ ] Notes include owning backlog items for every 🟩 / 🟡 row.
|
||||
- [ ] CLI commands marked ✅ have corresponding entries in `/docs/cli/*.md` or module-specific docs.
|
||||
- [ ] CLI commands marked ✅ have corresponding entries in `/docs/modules/cli/guides/*.md` or module-specific docs.
|
||||
- [ ] CI parity workflow description kept in sync with CONSOLE-DOC-23-502 implementation.
|
||||
- [ ] Downloads workspace links to latest parity report.
|
||||
- [ ] Install / observability guides reference this matrix for pending CLI parity.
|
||||
|
||||
@@ -8,7 +8,7 @@ Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national
|
||||
|
||||
## CCCS (Canada)
|
||||
- JSON endpoint (`https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=<lang>&content_type=cccs_threat`) returns ~5 100 records per language; `page=<n>` still works for segmented pulls and the earliest `date_created` seen is 2018‑06‑08 (EN) / 2018‑06‑08 (FR). Use an explicit `User-Agent` to avoid 403 responses.
|
||||
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/ops/concelier-cccs-operations.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
|
||||
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/modules/concelier/operations/connectors/cccs.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
|
||||
|
||||
## CERT-Bund (Germany)
|
||||
- `https://wid.cert-bund.de/content/public/securityAdvisory/rss` responds 200 without cookies (≈250-item window, German taxonomy). Detail links load an Angular SPA that fetches JSON behind the bootstrap session.
|
||||
|
||||
@@ -1,228 +1,228 @@
|
||||
# Deploying the StellaOps Console
|
||||
|
||||
> **Audience:** Deployment Guild, Console Guild, operators rolling out the web console.
|
||||
> **Scope:** Helm and Docker Compose deployment steps, ingress/TLS configuration, required environment variables, health checks, offline/air-gap operation, and compliance checklist (Sprint 23).
|
||||
|
||||
The StellaOps Console ships as part of the `stellaops` stack Helm chart and Compose bundles maintained under `deploy/`. This guide describes the supported deployment paths, the configuration surface, and operational checks needed to run the console in connected or air-gapped environments.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Kubernetes cluster (v1.28+) with ingress controller (NGINX, Traefik, or equivalent) and Cert-Manager for automated TLS, or Docker host for Compose deployments.
|
||||
- Container registry access to `registry.stella-ops.org` (or mirrored registry) for all images listed in `deploy/releases/*.yaml`.
|
||||
- Authority service configured with console client (`aud=ui`, scopes `ui.read`, `ui.admin`).
|
||||
- DNS entry pointing to the console hostname (for example, `console.acme.internal`).
|
||||
- Cosign public key for manifest verification (`deploy/releases/manifest.json.sig`).
|
||||
- Optional: Offline Kit bundle for air-gapped sites (`stella-ops-offline-kit-<ver>.tar.gz`).
|
||||
|
||||
---
|
||||
|
||||
## 2. Helm deployment (recommended)
|
||||
|
||||
### 2.1 Install chart repository
|
||||
|
||||
```bash
|
||||
helm repo add stellaops https://downloads.stella-ops.org/helm
|
||||
helm repo update stellaops
|
||||
```
|
||||
|
||||
If operating offline, copy the chart archive from the Offline Kit (`deploy/helm/stellaops-<ver>.tgz`) and run:
|
||||
|
||||
```bash
|
||||
helm install stellaops ./stellaops-<ver>.tgz --namespace stellaops --create-namespace
|
||||
```
|
||||
|
||||
### 2.2 Base installation
|
||||
|
||||
```bash
|
||||
helm install stellaops stellaops/stellaops \
|
||||
--namespace stellaops \
|
||||
--create-namespace \
|
||||
--values deploy/helm/stellaops/values-prod.yaml
|
||||
```
|
||||
|
||||
The chart deploys Authority, Console web/API gateway, Scanner API, Scheduler, and supporting services. The console frontend pod is labelled `app=stellaops-web-ui`.
|
||||
|
||||
### 2.3 Helm values highlights
|
||||
|
||||
Key sections in `deploy/helm/stellaops/values-prod.yaml`:
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `console.ingress.host` | Hostname served by the console (`console.example.com`). |
|
||||
| `console.ingress.tls.secretName` | Kubernetes secret containing TLS certificate (generated by Cert-Manager or uploaded manually). |
|
||||
| `console.config.apiGateway.baseUrl` | Internal base URL the UI uses to reach the gateway (defaults to `https://stellaops-web`). |
|
||||
| `console.env.AUTHORITY_ISSUER` | Authority issuer URL (for example, `https://authority.example.com`). |
|
||||
| `console.env.AUTHORITY_CLIENT_ID` | Authority client ID for the console UI. |
|
||||
| `console.env.AUTHORITY_SCOPES` | Space-separated scopes required by UI (`ui.read ui.admin`). |
|
||||
| `console.resources` | CPU/memory requests and limits (default 250m CPU / 512Mi memory). |
|
||||
| `console.podAnnotations` | Optional annotations for service mesh or monitoring. |
|
||||
|
||||
Use `values-stage.yaml`, `values-dev.yaml`, or `values-airgap.yaml` as templates for other environments.
|
||||
|
||||
### 2.4 TLS and ingress
|
||||
|
||||
Example ingress override:
|
||||
|
||||
```yaml
|
||||
console:
|
||||
ingress:
|
||||
enabled: true
|
||||
className: nginx
|
||||
host: console.acme.internal
|
||||
tls:
|
||||
enabled: true
|
||||
secretName: console-tls
|
||||
```
|
||||
|
||||
Generate certificates using Cert-Manager or provide an existing secret. For air-gapped deployments, pre-create the secret with the mirrored CA chain.
|
||||
|
||||
### 2.5 Health checks
|
||||
|
||||
Console pods expose:
|
||||
|
||||
| Path | Purpose | Notes |
|
||||
|------|---------|-------|
|
||||
| `/health/live` | Liveness probe | Confirms process responsive. |
|
||||
| `/health/ready` | Readiness probe | Verifies configuration bootstrap and Authority reachability. |
|
||||
| `/metrics` | Prometheus metrics | Enabled when `console.metrics.enabled=true`. |
|
||||
|
||||
Helm chart sets default probes (`initialDelaySeconds: 10`, `periodSeconds: 15`). Adjust via `console.livenessProbe` and `console.readinessProbe`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Docker Compose deployment
|
||||
|
||||
Located in `deploy/compose/docker-compose.console.yaml`. Quick start:
|
||||
|
||||
```bash
|
||||
cd deploy/compose
|
||||
docker compose -f docker-compose.console.yaml --env-file console.env up -d
|
||||
```
|
||||
|
||||
`console.env` should define:
|
||||
|
||||
```
|
||||
CONSOLE_PUBLIC_BASE_URL=https://console.acme.internal
|
||||
AUTHORITY_ISSUER=https://authority.acme.internal
|
||||
AUTHORITY_CLIENT_ID=console-ui
|
||||
AUTHORITY_CLIENT_SECRET=<if using confidential client>
|
||||
AUTHORITY_SCOPES=ui.read ui.admin
|
||||
CONSOLE_GATEWAY_BASE_URL=https://api.acme.internal
|
||||
```
|
||||
|
||||
The compose bundle includes Traefik as reverse proxy with TLS termination. Update `traefik/dynamic/console.yml` for custom certificates or additional middlewares (CSP headers, rate limits).
|
||||
|
||||
---
|
||||
|
||||
## 4. Environment variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `CONSOLE_PUBLIC_BASE_URL` | External URL used for redirects, deep links, and telemetry. | None (required). |
|
||||
| `CONSOLE_GATEWAY_BASE_URL` | URL of the web gateway that proxies API calls (`/console/*`). | Chart service name. |
|
||||
| `AUTHORITY_ISSUER` | Authority issuer (`https://authority.example.com`). | None (required). |
|
||||
| `AUTHORITY_CLIENT_ID` | OIDC client configured in Authority. | None (required). |
|
||||
| `AUTHORITY_SCOPES` | Space-separated scopes assigned to the console client. | `ui.read ui.admin`. |
|
||||
| `AUTHORITY_DPOP_ENABLED` | Enables DPoP challenge/response (recommended true). | `true`. |
|
||||
| `CONSOLE_FEATURE_FLAGS` | Comma-separated feature flags (`runs`, `downloads.offline`, etc.). | `runs,downloads,policies`. |
|
||||
| `CONSOLE_LOG_LEVEL` | Minimum log level (`Information`, `Debug`, etc.). | `Information`. |
|
||||
| `CONSOLE_METRICS_ENABLED` | Expose `/metrics` endpoint. | `true`. |
|
||||
| `CONSOLE_SENTRY_DSN` | Optional error reporting DSN. | Blank. |
|
||||
|
||||
When running behind additional proxies, set `ASPNETCORE_FORWARDEDHEADERS_ENABLED=true` to honour `X-Forwarded-*` headers.
|
||||
|
||||
---
|
||||
|
||||
## 5. Security headers and CSP
|
||||
|
||||
The console serves a strict Content Security Policy (CSP) by default:
|
||||
|
||||
```
|
||||
default-src 'self';
|
||||
connect-src 'self' https://*.stella-ops.local;
|
||||
script-src 'self';
|
||||
style-src 'self' 'unsafe-inline';
|
||||
img-src 'self' data:;
|
||||
font-src 'self';
|
||||
frame-ancestors 'none';
|
||||
```
|
||||
|
||||
Adjust via `console.config.cspOverrides` if additional domains are required. For integrations embedding the console, update OIDC redirect URIs and Authority scopes accordingly.
|
||||
|
||||
TLS recommendations:
|
||||
|
||||
- Use TLS 1.2+ with modern cipher suite policy.
|
||||
- Enable HSTS (`Strict-Transport-Security: max-age=31536000; includeSubDomains`).
|
||||
- Provide custom trust bundles via `console.config.trustBundleSecret` when using private CAs.
|
||||
|
||||
---
|
||||
|
||||
## 6. Logging and metrics
|
||||
|
||||
- Structured logs emitted to stdout with correlation IDs. Configure log shipping via Fluent Bit or similar.
|
||||
- Metrics available at `/metrics` in Prometheus format. Key metrics include `ui_request_duration_seconds`, `ui_tenant_switch_total`, and `ui_download_manifest_refresh_seconds`.
|
||||
- Enable OpenTelemetry exporter by setting `OTEL_EXPORTER_OTLP_ENDPOINT` and associated headers in environment variables.
|
||||
|
||||
---
|
||||
|
||||
## 7. Offline and air-gap deployment
|
||||
|
||||
- Mirror container images using the Downloads workspace or Offline Kit manifest. Example:
|
||||
|
||||
```bash
|
||||
oras copy registry.stella-ops.org/stellaops/web-ui@sha256:<digest> \
|
||||
registry.airgap.local/stellaops/web-ui:2025.10.0
|
||||
```
|
||||
|
||||
- Import Offline Kit using `stella ouk import` before starting the console so manifest parity checks succeed.
|
||||
- Use `values-airgap.yaml` to disable external telemetry endpoints and configure internal certificate chains.
|
||||
- Run `helm upgrade --install` using the mirrored chart (`stellaops-<ver>.tgz`) and set `console.offlineMode=true` to surface offline banners.
|
||||
|
||||
---
|
||||
|
||||
## 8. Health checks and remediation
|
||||
|
||||
| Check | Command | Expected result |
|
||||
|-------|---------|-----------------|
|
||||
| Pod status | `kubectl get pods -n stellaops` | `Running` state with restarts = 0. |
|
||||
| Liveness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/live` | Returns `{"status":"Healthy"}`. |
|
||||
| Readiness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/ready` | Returns `{"status":"Ready"}`. |
|
||||
| Gateway reachability | `curl -I https://console.example.com/api/console/status` | `200 OK` with CSP headers. |
|
||||
| Static assets | `curl -I https://console.example.com/static/assets/app.js` | `200 OK` with long cache headers. |
|
||||
|
||||
Troubleshooting steps:
|
||||
|
||||
- **Authority unreachable:** readiness fails with `AUTHORITY_UNREACHABLE`. Check DNS, trust bundles, and Authority service health.
|
||||
- **Manifest mismatch:** console logs `DOWNLOAD_MANIFEST_SIGNATURE_INVALID`. Verify cosign key and re-sync manifest.
|
||||
- **Ingress 404:** ensure ingress controller routes host to `stellaops-web-ui` service; check TLS secret name.
|
||||
- **SSE blocked:** confirm proxy allows HTTP/1.1 and disables buffering on `/console/runs/*`.
|
||||
|
||||
---
|
||||
|
||||
## 9. References
|
||||
|
||||
- `deploy/helm/stellaops/values-*.yaml` - environment-specific overrides.
|
||||
- `deploy/compose/docker-compose.console.yaml` - Compose bundle.
|
||||
- `/docs/ui/downloads.md` - manifest and offline bundle guidance.
|
||||
- `/docs/security/console-security.md` - CSP and Authority scopes.
|
||||
- `/docs/24_OFFLINE_KIT.md` - Offline kit packaging and verification.
|
||||
- `/docs/ops/deployment-runbook.md` (pending) - wider platform deployment steps.
|
||||
|
||||
---
|
||||
|
||||
## 10. Compliance checklist
|
||||
|
||||
- [ ] Helm and Compose instructions verified against `deploy/` assets.
|
||||
- [ ] Ingress/TLS guidance aligns with Security Guild recommendations.
|
||||
- [ ] Environment variables documented with defaults and required values.
|
||||
- [ ] Health/liveness/readiness endpoints tested and listed.
|
||||
- [ ] Offline workflow (mirrors, manifest parity) captured.
|
||||
- [ ] Logging and metrics surface documented metrics.
|
||||
- [ ] CSP and security header defaults stated alongside override guidance.
|
||||
- [ ] Troubleshooting section linked to relevant runbooks.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 23).*
|
||||
# Deploying the StellaOps Console
|
||||
|
||||
> **Audience:** Deployment Guild, Console Guild, operators rolling out the web console.
|
||||
> **Scope:** Helm and Docker Compose deployment steps, ingress/TLS configuration, required environment variables, health checks, offline/air-gap operation, and compliance checklist (Sprint 23).
|
||||
|
||||
The StellaOps Console ships as part of the `stellaops` stack Helm chart and Compose bundles maintained under `deploy/`. This guide describes the supported deployment paths, the configuration surface, and operational checks needed to run the console in connected or air-gapped environments.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Kubernetes cluster (v1.28+) with ingress controller (NGINX, Traefik, or equivalent) and Cert-Manager for automated TLS, or Docker host for Compose deployments.
|
||||
- Container registry access to `registry.stella-ops.org` (or mirrored registry) for all images listed in `deploy/releases/*.yaml`.
|
||||
- Authority service configured with console client (`aud=ui`, scopes `ui.read`, `ui.admin`).
|
||||
- DNS entry pointing to the console hostname (for example, `console.acme.internal`).
|
||||
- Cosign public key for manifest verification (`deploy/releases/manifest.json.sig`).
|
||||
- Optional: Offline Kit bundle for air-gapped sites (`stella-ops-offline-kit-<ver>.tar.gz`).
|
||||
|
||||
---
|
||||
|
||||
## 2. Helm deployment (recommended)
|
||||
|
||||
### 2.1 Install chart repository
|
||||
|
||||
```bash
|
||||
helm repo add stellaops https://downloads.stella-ops.org/helm
|
||||
helm repo update stellaops
|
||||
```
|
||||
|
||||
If operating offline, copy the chart archive from the Offline Kit (`deploy/helm/stellaops-<ver>.tgz`) and run:
|
||||
|
||||
```bash
|
||||
helm install stellaops ./stellaops-<ver>.tgz --namespace stellaops --create-namespace
|
||||
```
|
||||
|
||||
### 2.2 Base installation
|
||||
|
||||
```bash
|
||||
helm install stellaops stellaops/stellaops \
|
||||
--namespace stellaops \
|
||||
--create-namespace \
|
||||
--values deploy/helm/stellaops/values-prod.yaml
|
||||
```
|
||||
|
||||
The chart deploys Authority, Console web/API gateway, Scanner API, Scheduler, and supporting services. The console frontend pod is labelled `app=stellaops-web-ui`.
|
||||
|
||||
### 2.3 Helm values highlights
|
||||
|
||||
Key sections in `deploy/helm/stellaops/values-prod.yaml`:
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `console.ingress.host` | Hostname served by the console (`console.example.com`). |
|
||||
| `console.ingress.tls.secretName` | Kubernetes secret containing TLS certificate (generated by Cert-Manager or uploaded manually). |
|
||||
| `console.config.apiGateway.baseUrl` | Internal base URL the UI uses to reach the gateway (defaults to `https://stellaops-web`). |
|
||||
| `console.env.AUTHORITY_ISSUER` | Authority issuer URL (for example, `https://authority.example.com`). |
|
||||
| `console.env.AUTHORITY_CLIENT_ID` | Authority client ID for the console UI. |
|
||||
| `console.env.AUTHORITY_SCOPES` | Space-separated scopes required by UI (`ui.read ui.admin`). |
|
||||
| `console.resources` | CPU/memory requests and limits (default 250m CPU / 512Mi memory). |
|
||||
| `console.podAnnotations` | Optional annotations for service mesh or monitoring. |
|
||||
|
||||
Use `values-stage.yaml`, `values-dev.yaml`, or `values-airgap.yaml` as templates for other environments.
|
||||
|
||||
### 2.4 TLS and ingress
|
||||
|
||||
Example ingress override:
|
||||
|
||||
```yaml
|
||||
console:
|
||||
ingress:
|
||||
enabled: true
|
||||
className: nginx
|
||||
host: console.acme.internal
|
||||
tls:
|
||||
enabled: true
|
||||
secretName: console-tls
|
||||
```
|
||||
|
||||
Generate certificates using Cert-Manager or provide an existing secret. For air-gapped deployments, pre-create the secret with the mirrored CA chain.
|
||||
|
||||
### 2.5 Health checks
|
||||
|
||||
Console pods expose:
|
||||
|
||||
| Path | Purpose | Notes |
|
||||
|------|---------|-------|
|
||||
| `/health/live` | Liveness probe | Confirms process responsive. |
|
||||
| `/health/ready` | Readiness probe | Verifies configuration bootstrap and Authority reachability. |
|
||||
| `/metrics` | Prometheus metrics | Enabled when `console.metrics.enabled=true`. |
|
||||
|
||||
Helm chart sets default probes (`initialDelaySeconds: 10`, `periodSeconds: 15`). Adjust via `console.livenessProbe` and `console.readinessProbe`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Docker Compose deployment
|
||||
|
||||
Located in `deploy/compose/docker-compose.console.yaml`. Quick start:
|
||||
|
||||
```bash
|
||||
cd deploy/compose
|
||||
docker compose -f docker-compose.console.yaml --env-file console.env up -d
|
||||
```
|
||||
|
||||
`console.env` should define:
|
||||
|
||||
```
|
||||
CONSOLE_PUBLIC_BASE_URL=https://console.acme.internal
|
||||
AUTHORITY_ISSUER=https://authority.acme.internal
|
||||
AUTHORITY_CLIENT_ID=console-ui
|
||||
AUTHORITY_CLIENT_SECRET=<if using confidential client>
|
||||
AUTHORITY_SCOPES=ui.read ui.admin
|
||||
CONSOLE_GATEWAY_BASE_URL=https://api.acme.internal
|
||||
```
|
||||
|
||||
The compose bundle includes Traefik as reverse proxy with TLS termination. Update `traefik/dynamic/console.yml` for custom certificates or additional middlewares (CSP headers, rate limits).
|
||||
|
||||
---
|
||||
|
||||
## 4. Environment variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `CONSOLE_PUBLIC_BASE_URL` | External URL used for redirects, deep links, and telemetry. | None (required). |
|
||||
| `CONSOLE_GATEWAY_BASE_URL` | URL of the web gateway that proxies API calls (`/console/*`). | Chart service name. |
|
||||
| `AUTHORITY_ISSUER` | Authority issuer (`https://authority.example.com`). | None (required). |
|
||||
| `AUTHORITY_CLIENT_ID` | OIDC client configured in Authority. | None (required). |
|
||||
| `AUTHORITY_SCOPES` | Space-separated scopes assigned to the console client. | `ui.read ui.admin`. |
|
||||
| `AUTHORITY_DPOP_ENABLED` | Enables DPoP challenge/response (recommended true). | `true`. |
|
||||
| `CONSOLE_FEATURE_FLAGS` | Comma-separated feature flags (`runs`, `downloads.offline`, etc.). | `runs,downloads,policies`. |
|
||||
| `CONSOLE_LOG_LEVEL` | Minimum log level (`Information`, `Debug`, etc.). | `Information`. |
|
||||
| `CONSOLE_METRICS_ENABLED` | Expose `/metrics` endpoint. | `true`. |
|
||||
| `CONSOLE_SENTRY_DSN` | Optional error reporting DSN. | Blank. |
|
||||
|
||||
When running behind additional proxies, set `ASPNETCORE_FORWARDEDHEADERS_ENABLED=true` to honour `X-Forwarded-*` headers.
|
||||
|
||||
---
|
||||
|
||||
## 5. Security headers and CSP
|
||||
|
||||
The console serves a strict Content Security Policy (CSP) by default:
|
||||
|
||||
```
|
||||
default-src 'self';
|
||||
connect-src 'self' https://*.stella-ops.local;
|
||||
script-src 'self';
|
||||
style-src 'self' 'unsafe-inline';
|
||||
img-src 'self' data:;
|
||||
font-src 'self';
|
||||
frame-ancestors 'none';
|
||||
```
|
||||
|
||||
Adjust via `console.config.cspOverrides` if additional domains are required. For integrations embedding the console, update OIDC redirect URIs and Authority scopes accordingly.
|
||||
|
||||
TLS recommendations:
|
||||
|
||||
- Use TLS 1.2+ with modern cipher suite policy.
|
||||
- Enable HSTS (`Strict-Transport-Security: max-age=31536000; includeSubDomains`).
|
||||
- Provide custom trust bundles via `console.config.trustBundleSecret` when using private CAs.
|
||||
|
||||
---
|
||||
|
||||
## 6. Logging and metrics
|
||||
|
||||
- Structured logs emitted to stdout with correlation IDs. Configure log shipping via Fluent Bit or similar.
|
||||
- Metrics available at `/metrics` in Prometheus format. Key metrics include `ui_request_duration_seconds`, `ui_tenant_switch_total`, and `ui_download_manifest_refresh_seconds`.
|
||||
- Enable OpenTelemetry exporter by setting `OTEL_EXPORTER_OTLP_ENDPOINT` and associated headers in environment variables.
|
||||
|
||||
---
|
||||
|
||||
## 7. Offline and air-gap deployment
|
||||
|
||||
- Mirror container images using the Downloads workspace or Offline Kit manifest. Example:
|
||||
|
||||
```bash
|
||||
oras copy registry.stella-ops.org/stellaops/web-ui@sha256:<digest> \
|
||||
registry.airgap.local/stellaops/web-ui:2025.10.0
|
||||
```
|
||||
|
||||
- Import Offline Kit using `stella ouk import` before starting the console so manifest parity checks succeed.
|
||||
- Use `values-airgap.yaml` to disable external telemetry endpoints and configure internal certificate chains.
|
||||
- Run `helm upgrade --install` using the mirrored chart (`stellaops-<ver>.tgz`) and set `console.offlineMode=true` to surface offline banners.
|
||||
|
||||
---
|
||||
|
||||
## 8. Health checks and remediation
|
||||
|
||||
| Check | Command | Expected result |
|
||||
|-------|---------|-----------------|
|
||||
| Pod status | `kubectl get pods -n stellaops` | `Running` state with restarts = 0. |
|
||||
| Liveness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/live` | Returns `{"status":"Healthy"}`. |
|
||||
| Readiness | `kubectl exec deploy/stellaops-web-ui -- curl -fsS http://localhost:8080/health/ready` | Returns `{"status":"Ready"}`. |
|
||||
| Gateway reachability | `curl -I https://console.example.com/api/console/status` | `200 OK` with CSP headers. |
|
||||
| Static assets | `curl -I https://console.example.com/static/assets/app.js` | `200 OK` with long cache headers. |
|
||||
|
||||
Troubleshooting steps:
|
||||
|
||||
- **Authority unreachable:** readiness fails with `AUTHORITY_UNREACHABLE`. Check DNS, trust bundles, and Authority service health.
|
||||
- **Manifest mismatch:** console logs `DOWNLOAD_MANIFEST_SIGNATURE_INVALID`. Verify cosign key and re-sync manifest.
|
||||
- **Ingress 404:** ensure ingress controller routes host to `stellaops-web-ui` service; check TLS secret name.
|
||||
- **SSE blocked:** confirm proxy allows HTTP/1.1 and disables buffering on `/console/runs/*`.
|
||||
|
||||
---
|
||||
|
||||
## 9. References
|
||||
|
||||
- `deploy/helm/stellaops/values-*.yaml` - environment-specific overrides.
|
||||
- `deploy/compose/docker-compose.console.yaml` - Compose bundle.
|
||||
- `/docs/ui/downloads.md` - manifest and offline bundle guidance.
|
||||
- `/docs/security/console-security.md` - CSP and Authority scopes.
|
||||
- `/docs/24_OFFLINE_KIT.md` - Offline kit packaging and verification.
|
||||
- `/docs/modules/devops/runbooks/deployment-runbook.md` (pending) - wider platform deployment steps.
|
||||
|
||||
---
|
||||
|
||||
## 10. Compliance checklist
|
||||
|
||||
- [ ] Helm and Compose instructions verified against `deploy/` assets.
|
||||
- [ ] Ingress/TLS guidance aligns with Security Guild recommendations.
|
||||
- [ ] Environment variables documented with defaults and required values.
|
||||
- [ ] Health/liveness/readiness endpoints tested and listed.
|
||||
- [ ] Offline workflow (mirrors, manifest parity) captured.
|
||||
- [ ] Logging and metrics surface documented metrics.
|
||||
- [ ] CSP and security header defaults stated alongside override guidance.
|
||||
- [ ] Troubleshooting section linked to relevant runbooks.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 23).*
|
||||
|
||||
@@ -1,160 +1,160 @@
|
||||
# Container Deployment Guide — AOC Update
|
||||
|
||||
> **Audience:** DevOps Guild, platform operators deploying StellaOps services.
|
||||
> **Scope:** Deployment configuration changes required by the Aggregation-Only Contract (AOC), including schema validators, guard environment flags, and verifier identities.
|
||||
|
||||
This guide supplements existing deployment manuals with AOC-specific configuration. It assumes familiarity with the base Compose/Helm manifests described in `ops/deployment/` and `docs/ARCHITECTURE_DEVOPS.md`.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Schema validator enablement
|
||||
|
||||
### 1.1 MongoDB validators
|
||||
|
||||
- Apply JSON schema validators to `advisory_raw` and `vex_raw` collections before enabling AOC guards.
|
||||
- Before enabling validators or the idempotency index, run the duplicate audit helper to confirm no conflicting raw advisories remain:
|
||||
```bash
|
||||
mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'
|
||||
```
|
||||
Resolve any reported rows prior to rollout.
|
||||
- Use the migration script provided in `ops/devops/scripts/apply-aoc-validators.js`:
|
||||
|
||||
```bash
|
||||
kubectl exec -n concelier deploy/concelier-mongo -- \
|
||||
mongo concelier ops/devops/scripts/apply-aoc-validators.js
|
||||
|
||||
kubectl exec -n excititor deploy/excititor-mongo -- \
|
||||
mongo excititor ops/devops/scripts/apply-aoc-validators.js
|
||||
```
|
||||
|
||||
- Validators enforce required fields (`tenant`, `source`, `upstream`, `linkset`) and reject forbidden keys at DB level.
|
||||
- Rollback plan: validators are applied with `validationLevel: moderate`—downgrade via the same script with `--remove` if required.
|
||||
|
||||
### 1.2 Migration order
|
||||
|
||||
1. Deploy validators in maintenance window.
|
||||
2. Roll out Concelier/Excititor images with guard middleware enabled (`AOC_GUARD_ENABLED=true`).
|
||||
3. Run smoke tests (`stella sources ingest --dry-run` fixtures) before resuming production ingestion.
|
||||
|
||||
### 1.3 Supersedes backfill verification
|
||||
|
||||
1. **Duplicate audit:** Confirm `mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'` reports no conflicts before restarting Concelier with the new migrations.
|
||||
2. **Post-migration check:** After the service restarts, validate that `db.advisory` is a view pointing to `advisory_backup_20251028`:
|
||||
```bash
|
||||
mongo concelier --quiet --eval 'db.getCollectionInfos({ name: "advisory" })[0]'
|
||||
```
|
||||
The `type` should be `"view"` and `options.viewOn` should equal `"advisory_backup_20251028"`.
|
||||
3. **Supersedes chain spot-check:** Inspect a sample set to ensure deterministic chaining:
|
||||
```bash
|
||||
mongo concelier --quiet --eval '
|
||||
db.advisory_raw.aggregate([
|
||||
{ $match: { "upstream.upstream_id": { $exists: true } } },
|
||||
{ $sort: { "tenant": 1, "source.vendor": 1, "upstream.upstream_id": 1, "upstream.retrieved_at": 1 } },
|
||||
{ $limit: 5 },
|
||||
{ $project: { _id: 1, supersedes: 1 } }
|
||||
]).forEach(printjson)'
|
||||
```
|
||||
Each revision should reference the previous `_id` (or `null` for the first revision). Record findings in the change ticket before proceeding to production.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Container environment flags
|
||||
|
||||
Add the following environment variables to Concelier/Excititor deployments:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `AOC_GUARD_ENABLED` | `true` | Enables `AOCWriteGuard` interception. Set `false` only for controlled rollback. |
|
||||
| `AOC_ALLOW_SUPERSEDES_RETROFIT` | `false` | Allows temporary supersedes backfill during migration. Remove after cutover. |
|
||||
| `AOC_METRICS_ENABLED` | `true` | Emits `ingestion_write_total`, `aoc_violation_total`, etc. |
|
||||
| `AOC_TENANT_HEADER` | `X-Stella-Tenant` | Header name expected from Gateway. |
|
||||
| `AOC_VERIFIER_USER` | `stella-aoc-verify` | Read-only service user used by UI/CLI verification. |
|
||||
|
||||
Compose snippet:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- AOC_GUARD_ENABLED=true
|
||||
- AOC_ALLOW_SUPERSEDES_RETROFIT=false
|
||||
- AOC_METRICS_ENABLED=true
|
||||
- AOC_TENANT_HEADER=X-Stella-Tenant
|
||||
- AOC_VERIFIER_USER=stella-aoc-verify
|
||||
```
|
||||
|
||||
Ensure `AOC_VERIFIER_USER` exists in Authority with `aoc:verify` scope and no write permissions.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Verifier identity
|
||||
|
||||
- Create a dedicated client (`stella-aoc-verify`) via Authority bootstrap:
|
||||
|
||||
```yaml
|
||||
clients:
|
||||
- clientId: stella-aoc-verify
|
||||
grantTypes: [client_credentials]
|
||||
scopes: [aoc:verify, advisory:read, vex:read]
|
||||
tenants: [default]
|
||||
```
|
||||
|
||||
- Store credentials in secret store (`Kubernetes Secret`, `Docker swarm secret`).
|
||||
- Bind credentials to `stella aoc verify` CI jobs and Console verification service.
|
||||
- Rotate quarterly; document in `ops/authority-key-rotation.md`.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Deployment steps
|
||||
|
||||
1. **Pre-checks:** Confirm database backups, alerting in maintenance mode, and staging environment validated.
|
||||
2. **Apply validators:** Run scripts per § 1.1.
|
||||
3. **Update manifests:** Inject environment variables (§ 2) and mount guard configuration configmaps.
|
||||
4. **Redeploy services:** Rolling restart Concelier/Excititor pods. Monitor `ingestion_write_total` for steady throughput.
|
||||
5. **Seed verifier:** Deploy read-only verifier user and store credentials.
|
||||
6. **Run verification:** Execute `stella aoc verify --since 24h` and ensure exit code `0`.
|
||||
7. **Update dashboards:** Point Grafana panels to new metrics (`aoc_violation_total`).
|
||||
8. **Record handoff:** Capture console screenshots and verification logs for release notes.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Offline Kit updates
|
||||
|
||||
- Ship validator scripts with Offline Kit (`offline-kit/scripts/apply-aoc-validators.js`).
|
||||
- Include pre-generated verification reports for air-gapped deployments.
|
||||
- Document offline CLI workflow in bundle README referencing `docs/cli/cli-reference.md`.
|
||||
- Ensure `stella-aoc-verify` credentials are scoped to offline tenant and rotated during bundle refresh.
|
||||
|
||||
---
|
||||
|
||||
## 6 · Rollback plan
|
||||
|
||||
1. Disable guard via `AOC_GUARD_ENABLED=false` on Concelier/Excititor and rollout.
|
||||
2. Remove validators with the migration script (`--remove`).
|
||||
3. Pause verification jobs to prevent noise.
|
||||
4. Investigate and remediate upstream issues before re-enabling guards.
|
||||
|
||||
---
|
||||
|
||||
## 7 · References
|
||||
|
||||
- [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md)
|
||||
- [Authority scopes & tenancy](../security/authority-scopes.md)
|
||||
- [Observability guide](../observability/observability.md)
|
||||
- [CLI AOC commands](../cli/cli-reference.md)
|
||||
- [Concelier architecture](../ARCHITECTURE_CONCELIER.md)
|
||||
- [Excititor architecture](../ARCHITECTURE_EXCITITOR.md)
|
||||
|
||||
---
|
||||
|
||||
## 8 · Compliance checklist
|
||||
|
||||
- [ ] Validators documented and scripts referenced for online/offline deployments.
|
||||
- [ ] Environment variables cover guard enablement, metrics, and tenant header.
|
||||
- [ ] Read-only verifier user installation steps included.
|
||||
- [ ] Offline kit instructions align with validator/verification workflow.
|
||||
- [ ] Rollback procedure captured.
|
||||
- [ ] Cross-links to AOC docs, Authority scopes, and observability guides present.
|
||||
- [ ] DevOps Guild sign-off tracked (owner: @devops-guild, due 2025-10-29).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-26 (Sprint 19).*
|
||||
# Container Deployment Guide — AOC Update
|
||||
|
||||
> **Audience:** DevOps Guild, platform operators deploying StellaOps services.
|
||||
> **Scope:** Deployment configuration changes required by the Aggregation-Only Contract (AOC), including schema validators, guard environment flags, and verifier identities.
|
||||
|
||||
This guide supplements existing deployment manuals with AOC-specific configuration. It assumes familiarity with the base Compose/Helm manifests described in `ops/deployment/` and `docs/modules/devops/architecture.md`.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Schema validator enablement
|
||||
|
||||
### 1.1 MongoDB validators
|
||||
|
||||
- Apply JSON schema validators to `advisory_raw` and `vex_raw` collections before enabling AOC guards.
|
||||
- Before enabling validators or the idempotency index, run the duplicate audit helper to confirm no conflicting raw advisories remain:
|
||||
```bash
|
||||
mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'
|
||||
```
|
||||
Resolve any reported rows prior to rollout.
|
||||
- Use the migration script provided in `ops/devops/scripts/apply-aoc-validators.js`:
|
||||
|
||||
```bash
|
||||
kubectl exec -n concelier deploy/concelier-mongo -- \
|
||||
mongo concelier ops/devops/scripts/apply-aoc-validators.js
|
||||
|
||||
kubectl exec -n excititor deploy/excititor-mongo -- \
|
||||
mongo excititor ops/devops/scripts/apply-aoc-validators.js
|
||||
```
|
||||
|
||||
- Validators enforce required fields (`tenant`, `source`, `upstream`, `linkset`) and reject forbidden keys at DB level.
|
||||
- Rollback plan: validators are applied with `validationLevel: moderate`—downgrade via the same script with `--remove` if required.
|
||||
|
||||
### 1.2 Migration order
|
||||
|
||||
1. Deploy validators in maintenance window.
|
||||
2. Roll out Concelier/Excititor images with guard middleware enabled (`AOC_GUARD_ENABLED=true`).
|
||||
3. Run smoke tests (`stella sources ingest --dry-run` fixtures) before resuming production ingestion.
|
||||
|
||||
### 1.3 Supersedes backfill verification
|
||||
|
||||
1. **Duplicate audit:** Confirm `mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'` reports no conflicts before restarting Concelier with the new migrations.
|
||||
2. **Post-migration check:** After the service restarts, validate that `db.advisory` is a view pointing to `advisory_backup_20251028`:
|
||||
```bash
|
||||
mongo concelier --quiet --eval 'db.getCollectionInfos({ name: "advisory" })[0]'
|
||||
```
|
||||
The `type` should be `"view"` and `options.viewOn` should equal `"advisory_backup_20251028"`.
|
||||
3. **Supersedes chain spot-check:** Inspect a sample set to ensure deterministic chaining:
|
||||
```bash
|
||||
mongo concelier --quiet --eval '
|
||||
db.advisory_raw.aggregate([
|
||||
{ $match: { "upstream.upstream_id": { $exists: true } } },
|
||||
{ $sort: { "tenant": 1, "source.vendor": 1, "upstream.upstream_id": 1, "upstream.retrieved_at": 1 } },
|
||||
{ $limit: 5 },
|
||||
{ $project: { _id: 1, supersedes: 1 } }
|
||||
]).forEach(printjson)'
|
||||
```
|
||||
Each revision should reference the previous `_id` (or `null` for the first revision). Record findings in the change ticket before proceeding to production.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Container environment flags
|
||||
|
||||
Add the following environment variables to Concelier/Excititor deployments:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `AOC_GUARD_ENABLED` | `true` | Enables `AOCWriteGuard` interception. Set `false` only for controlled rollback. |
|
||||
| `AOC_ALLOW_SUPERSEDES_RETROFIT` | `false` | Allows temporary supersedes backfill during migration. Remove after cutover. |
|
||||
| `AOC_METRICS_ENABLED` | `true` | Emits `ingestion_write_total`, `aoc_violation_total`, etc. |
|
||||
| `AOC_TENANT_HEADER` | `X-Stella-Tenant` | Header name expected from Gateway. |
|
||||
| `AOC_VERIFIER_USER` | `stella-aoc-verify` | Read-only service user used by UI/CLI verification. |
|
||||
|
||||
Compose snippet:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- AOC_GUARD_ENABLED=true
|
||||
- AOC_ALLOW_SUPERSEDES_RETROFIT=false
|
||||
- AOC_METRICS_ENABLED=true
|
||||
- AOC_TENANT_HEADER=X-Stella-Tenant
|
||||
- AOC_VERIFIER_USER=stella-aoc-verify
|
||||
```
|
||||
|
||||
Ensure `AOC_VERIFIER_USER` exists in Authority with `aoc:verify` scope and no write permissions.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Verifier identity
|
||||
|
||||
- Create a dedicated client (`stella-aoc-verify`) via Authority bootstrap:
|
||||
|
||||
```yaml
|
||||
clients:
|
||||
- clientId: stella-aoc-verify
|
||||
grantTypes: [client_credentials]
|
||||
scopes: [aoc:verify, advisory:read, vex:read]
|
||||
tenants: [default]
|
||||
```
|
||||
|
||||
- Store credentials in secret store (`Kubernetes Secret`, `Docker swarm secret`).
|
||||
- Bind credentials to `stella aoc verify` CI jobs and Console verification service.
|
||||
- Rotate quarterly; document in `ops/authority-key-rotation.md`.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Deployment steps
|
||||
|
||||
1. **Pre-checks:** Confirm database backups, alerting in maintenance mode, and staging environment validated.
|
||||
2. **Apply validators:** Run scripts per § 1.1.
|
||||
3. **Update manifests:** Inject environment variables (§ 2) and mount guard configuration configmaps.
|
||||
4. **Redeploy services:** Rolling restart Concelier/Excititor pods. Monitor `ingestion_write_total` for steady throughput.
|
||||
5. **Seed verifier:** Deploy read-only verifier user and store credentials.
|
||||
6. **Run verification:** Execute `stella aoc verify --since 24h` and ensure exit code `0`.
|
||||
7. **Update dashboards:** Point Grafana panels to new metrics (`aoc_violation_total`).
|
||||
8. **Record handoff:** Capture console screenshots and verification logs for release notes.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Offline Kit updates
|
||||
|
||||
- Ship validator scripts with Offline Kit (`offline-kit/scripts/apply-aoc-validators.js`).
|
||||
- Include pre-generated verification reports for air-gapped deployments.
|
||||
- Document offline CLI workflow in bundle README referencing `docs/modules/cli/guides/cli-reference.md`.
|
||||
- Ensure `stella-aoc-verify` credentials are scoped to offline tenant and rotated during bundle refresh.
|
||||
|
||||
---
|
||||
|
||||
## 6 · Rollback plan
|
||||
|
||||
1. Disable guard via `AOC_GUARD_ENABLED=false` on Concelier/Excititor and rollout.
|
||||
2. Remove validators with the migration script (`--remove`).
|
||||
3. Pause verification jobs to prevent noise.
|
||||
4. Investigate and remediate upstream issues before re-enabling guards.
|
||||
|
||||
---
|
||||
|
||||
## 7 · References
|
||||
|
||||
- [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md)
|
||||
- [Authority scopes & tenancy](../security/authority-scopes.md)
|
||||
- [Observability guide](../observability/observability.md)
|
||||
- [CLI AOC commands](../modules/cli/guides/cli-reference.md)
|
||||
- [Concelier architecture](../modules/concelier/architecture.md)
|
||||
- [Excititor architecture](../modules/excititor/architecture.md)
|
||||
|
||||
---
|
||||
|
||||
## 8 · Compliance checklist
|
||||
|
||||
- [ ] Validators documented and scripts referenced for online/offline deployments.
|
||||
- [ ] Environment variables cover guard enablement, metrics, and tenant header.
|
||||
- [ ] Read-only verifier user installation steps included.
|
||||
- [ ] Offline kit instructions align with validator/verification workflow.
|
||||
- [ ] Rollback procedure captured.
|
||||
- [ ] Cross-links to AOC docs, Authority scopes, and observability guides present.
|
||||
- [ ] DevOps Guild sign-off tracked (owner: @devops-guild, due 2025-10-29).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-26 (Sprint 19).*
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Audience:** teams implementing new Excititor provider plug‑ins (CSAF feeds,
|
||||
> OpenVEX attestations, etc.)
|
||||
> **Prerequisites:** read `docs/ARCHITECTURE_EXCITITOR.md` and the module
|
||||
> **Prerequisites:** read `docs/modules/excititor/architecture.md` and the module
|
||||
> `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`.
|
||||
|
||||
The Excititor connector SDK gives you:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Audience:** teams implementing new Vexer provider plug‑ins (CSAF feeds,
|
||||
> OpenVEX attestations, etc.)
|
||||
> **Prerequisites:** read `docs/ARCHITECTURE_VEXER.md` and the module
|
||||
> **Prerequisites:** read `docs/modules/vexer/architecture.md` and the module
|
||||
> `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`.
|
||||
|
||||
The Vexer connector SDK gives you:
|
||||
|
||||
@@ -116,7 +116,7 @@ Capability flags let the host reason about what your plug-in supports:
|
||||
- Support environment overrides using `STELLAOPS_AUTHORITY_PLUGINS__DESCRIPTORS__<NAME>__...`.
|
||||
- Never store raw secrets in git: allow operators to supply them via `.local.yaml`, environment variables, or injected secret files. Document which keys are mandatory.
|
||||
- Validate configuration as soon as the registrar runs; use explicit error messages to guide operators. The Standard plug-in now enforces complete bootstrap credentials (username + password) and positive lockout windows via `StandardPluginOptions.Validate`.
|
||||
- Cross-reference bootstrap workflows with `docs/ops/authority_bootstrap.md` (to be published alongside CORE6) so operators can reuse the same payload formats for manual provisioning.
|
||||
- Cross-reference bootstrap workflows with `docs/modules/authority/operations/bootstrap.md` (to be published alongside CORE6) so operators can reuse the same payload formats for manual provisioning.
|
||||
- `passwordHashing` inherits defaults from `authority.security.passwordHashing`. Override only when hardware constraints differ per plug-in:
|
||||
```yaml
|
||||
passwordHashing:
|
||||
|
||||
@@ -116,4 +116,4 @@ Add the workflow to your repository (or call it via `workflow_call`) and adjust
|
||||
|
||||
---
|
||||
|
||||
For deeper integration guidance (custom SBOM builders, exporting DSSE bundles), track ADRs in `docs/ARCHITECTURE_SCANNER.md` §7 and follow upcoming Attestor API releases.
|
||||
For deeper integration guidance (custom SBOM builders, exporting DSSE bundles), track ADRs in `docs/modules/scanner/architecture.md` §7 and follow upcoming Attestor API releases.
|
||||
|
||||
@@ -134,13 +134,13 @@
|
||||
2. Introduce configuration classes and bind from YAML/environment.
|
||||
3. Implement nonce store (Redis + in-memory), handler integration, and OpenIddict transaction plumbing.
|
||||
4. Stamp `cnf.jkt`, audit events, and metrics; update Mongo documents and migrations.
|
||||
5. Extend docs: `docs/ARCHITECTURE_AUTHORITY.md`, `docs/security/audit-events.md`, `docs/security/rate-limits.md`, CLI/UI references.
|
||||
5. Extend docs: `docs/modules/authority/architecture.md`, `docs/security/audit-events.md`, `docs/security/rate-limits.md`, CLI/UI references.
|
||||
|
||||
**mTLS work-stream**
|
||||
1. Extend client document/schema and provisioning stores with certificate bindings + sender constraint flag.
|
||||
2. Configure Kestrel/middleware for optional client certificates and validation service.
|
||||
3. Update token issuance/introspection to honour certificate bindings and emit `cnf.x5t#S256`.
|
||||
4. Add auditing/metrics and integration tests (happy path + failure).
|
||||
5. Refresh operator documentation (`docs/ops/authority-backup-restore.md`, `docs/ops/authority-monitoring.md`, sample `authority.yaml`) to cover certificate lifecycle.
|
||||
5. Refresh operator documentation (`docs/modules/authority/operations/backup-restore.md`, `docs/modules/authority/operations/monitoring.md`, sample `authority.yaml`) to cover certificate lifecycle.
|
||||
|
||||
Both streams should conclude with `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.sln` and documentation cross-links so dependent guilds can unblock UI/Signer work.
|
||||
|
||||
@@ -11,7 +11,7 @@ fixture sets, where they live, and how to regenerate them safely.
|
||||
- **Purpose:** Exercised by `OsvGhsaParityRegressionTests` to ensure OSV + GHSA outputs stay aligned on aliases,
|
||||
ranges, references, and credits.
|
||||
- **Regeneration:** Either run the test harness with online regeneration (`UPDATE_PARITY_FIXTURES=1 dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`)
|
||||
or execute the fixture updater (`dotnet run --project tools/FixtureUpdater/FixtureUpdater.csproj`). Both paths
|
||||
or execute the fixture updater (`dotnet run --project src/Tools/FixtureUpdater/FixtureUpdater.csproj`). Both paths
|
||||
normalise timestamps and canonical ordering.
|
||||
- **SemVer provenance:** The regenerated fixtures should show `normalizedVersions[].notes` in the
|
||||
`osv:{ecosystem}:{advisoryId}:{identifier}` shape emitted by `SemVerRangeRuleBuilder`. Confirm the
|
||||
@@ -22,7 +22,7 @@ fixture sets, where they live, and how to regenerate them safely.
|
||||
|
||||
- **Location:** `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/Fixtures/credit-parity.{ghsa,osv,nvd}.json`
|
||||
- **Purpose:** Exercised by `GhsaCreditParityRegressionTests` to guarantee GHSA/NVD/OSV acknowledgements remain in lockstep.
|
||||
- **Regeneration:** `dotnet run --project tools/FixtureUpdater/FixtureUpdater.csproj` rewrites all three canonical snapshots.
|
||||
- **Regeneration:** `dotnet run --project src/Tools/FixtureUpdater/FixtureUpdater.csproj` rewrites all three canonical snapshots.
|
||||
- **Verification:** `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`.
|
||||
|
||||
> Always commit fixture changes together with the code that motivated them and reference the regression test that guards the behaviour.
|
||||
|
||||
@@ -82,10 +82,10 @@ Coordinate with the Storage team when enabling these indexes so deployment windo
|
||||
|
||||
## 6. Dual-write rollout
|
||||
|
||||
Follow the operational checklist in `docs/ops/migrations/SEMVER_STYLE.md`. The summary:
|
||||
Follow the operational checklist in `docs/modules/devops/migrations/semver-style.md`. The summary:
|
||||
|
||||
1. **Dual write (now)** – emit both legacy `versionRanges` and the new `normalizedVersions`.
|
||||
2. **Backfill** – follow the storage migration in `docs/ops/migrations/SEMVER_STYLE.md` to rewrite historical advisories before switching consumers.
|
||||
2. **Backfill** – follow the storage migration in `docs/modules/devops/migrations/semver-style.md` to rewrite historical advisories before switching consumers.
|
||||
3. **Verify** – run the aggregation above (with `explain("executionStats")`) to ensure the new indexes are used.
|
||||
4. **Cutover** – after consumers switch to normalized rules, mark the old `rangeExpression` as deprecated.
|
||||
|
||||
|
||||
@@ -1,121 +1,121 @@
|
||||
# Scanner Orchestrator Events (ORCH-SVC-38-101)
|
||||
|
||||
Last updated: 2025-10-26
|
||||
|
||||
The Notifications Studio initiative (NOTIFY-SVC-38-001) and orchestrator backlog (ORCH-SVC-38-101) standardise how platform services emit lifecycle events. This document describes the Scanner WebService contract for the new **orchestrator envelopes** (`scanner.event.*`) and how they supersede the legacy Redis-backed `scanner.report.ready` / `scanner.scan.completed` events.
|
||||
|
||||
## 1. Envelope overview
|
||||
|
||||
Orchestrator events share a deterministic JSON envelope:
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `eventId` | `uuid` | Globally unique identifier generated per occurrence. |
|
||||
| `kind` | `string` | Event identifier; Scanner emits `scanner.event.report.ready` and `scanner.event.scan.completed`. |
|
||||
| `version` | `integer` | Schema version. Initial release uses `1`. |
|
||||
| `tenant` | `string` | Tenant that owns the scan/report. Mirrors Authority claims. |
|
||||
| `occurredAt` | `date-time` | UTC instant when the underlying state transition happened (e.g., report persisted). |
|
||||
| `recordedAt` | `date-time` | UTC instant when the event was durably written. Optional but recommended. |
|
||||
| `source` | `string` | Producer identifier (`scanner.webservice`). |
|
||||
| `idempotencyKey` | `string` | Deterministic key for duplicate suppression (see §4). |
|
||||
| `correlationId` | `string` | Maps back to the API request or scan identifier. |
|
||||
| `traceId` / `spanId` | `string` | W3C trace context propagated into downstream telemetry. |
|
||||
| `scope` | `object` | Describes the affected artefact. Requires `repo` and `digest`; optional `namespace`, `component`, `image`. |
|
||||
| `attributes` | `object` | Flat string map for frequently queried metadata (e.g., policy revision). |
|
||||
| `payload` | `object` | Event-specific body (see §2). |
|
||||
|
||||
Canonical schemas live under `docs/events/scanner.event.*@1.json`. Samples that round-trip through `NotifyCanonicalJsonSerializer` are stored in `docs/events/samples/`.
|
||||
|
||||
## 2. Event kinds and payloads
|
||||
|
||||
### 2.1 `scanner.event.report.ready`
|
||||
|
||||
Emitted once a signed report is persisted and attested. Payload highlights:
|
||||
|
||||
- `reportId` / `scanId` — identifiers for the persisted report and originating scan. Until Scan IDs are surfaced by the API, `scanId` mirrors `reportId` so downstream correlators can stabilise on a single key.
|
||||
- **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict` — pre-sorted for deterministic routing.
|
||||
- **Links:**
|
||||
- `ui` → `/ui/reports/{reportId}` on the current host.
|
||||
- `report` → `{apiBasePath}/{reportsSegment}/{reportId}` (defaults to `/api/v1/reports/{reportId}`).
|
||||
- `policy` → `{apiBasePath}/{policySegment}/revisions/{revisionId}` when a revision is present.
|
||||
- `attestation` → `/ui/attestations/{reportId}` when a DSSE envelope is included.
|
||||
- `imageDigest` — OCI image digest associated with the analysis.
|
||||
- `generatedAt` — report generation timestamp (ISO-8601 UTC).
|
||||
- `verdict` — `pass`, `warn`, or `fail` after policy evaluation.
|
||||
- `summary` — blocked/warned/ignored/quieted counters (all non-negative integers).
|
||||
- `delta` — newly critical/high counts and optional `kev` array.
|
||||
- `quietedFindingCount` — mirrors `summary.quieted`.
|
||||
- `policy` — revision metadata (`digest`, `revisionId`) surfaced for routing.
|
||||
- `links` — UI/report/policy URLs suitable for operators.
|
||||
- `dsse` — embedded DSSE envelope (payload, type, signature list).
|
||||
- `report` — canonical report document; identical to the DSSE payload.
|
||||
|
||||
Schema: `docs/events/scanner.event.report.ready@1.json`
|
||||
Sample: `docs/events/samples/scanner.event.report.ready@1.sample.json`
|
||||
|
||||
### 2.2 `scanner.event.scan.completed`
|
||||
|
||||
Emitted after scan execution finishes (success or policy failure). Payload highlights:
|
||||
|
||||
- `reportId` / `scanId` / `imageDigest` — identifiers mirroring the report-ready event. As with the report-ready payload, `scanId` currently mirrors `reportId` as a temporary shim.
|
||||
- **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict`.
|
||||
- **Links:** same as above (`ui`, `report`, `policy`) with `attestation` populated when DSSE metadata exists.
|
||||
- `verdict`, `summary`, `delta`, `policy` — same semantics as above.
|
||||
- `findings` — array of surfaced findings with `id`, `severity`, optional `cve`, `purl`, and `reachability`.
|
||||
- `links`, `dsse`, `report` — same structure as §2.1 (allows Notifier to reuse signatures).
|
||||
|
||||
Schema: `docs/events/scanner.event.scan.completed@1.json`
|
||||
Sample: `docs/events/samples/scanner.event.scan.completed@1.sample.json`
|
||||
|
||||
### 2.3 Relationship to legacy events
|
||||
|
||||
| Legacy Redis event | Replacement orchestrator event | Notes |
|
||||
|--------------------|-------------------------------|-------|
|
||||
| `scanner.report.ready` | `scanner.event.report.ready` | Adds versioning, idempotency, trace context. Payload is a superset of the legacy fields. |
|
||||
| `scanner.scan.completed` | `scanner.event.scan.completed` | Same data plus explicit scan identifiers and orchestrator metadata. |
|
||||
|
||||
Legacy schemas remain for backwards-compatibility during migration, but new integrations **must** target the orchestrator variants.
|
||||
|
||||
## 3. Deterministic serialization
|
||||
|
||||
- Producers must serialise events using `NotifyCanonicalJsonSerializer` to guarantee consistent key ordering and whitespace.
|
||||
- Timestamps (`occurredAt`, `recordedAt`, `payload.generatedAt`) use `DateTimeOffset.UtcDateTime.ToString("O")`.
|
||||
- Payload arrays (`delta.kev`, `findings`) should be pre-sorted (e.g., alphabetical CVE order) so hash-based consumers remain stable.
|
||||
- Optional fields are omitted rather than emitted as `null`.
|
||||
|
||||
## 4. Idempotency and correlation
|
||||
|
||||
Idempotency keys dedupe repeated publishes and align with the orchestrator’s outbox pattern:
|
||||
|
||||
| Event kind | Idempotency key template |
|
||||
|------------|-------------------------|
|
||||
| `scanner.event.report.ready` | `scanner.event.report.ready:<tenant>:<reportId>` |
|
||||
| `scanner.event.scan.completed` | `scanner.event.scan.completed:<tenant>:<scanId>` |
|
||||
|
||||
Keys are ASCII lowercase; components should be trimmed and validated before concatenation. Retries must reuse the same key.
|
||||
|
||||
`correlationId` should match the scan identifier that appears in REST responses (`scanId`). Re-using the same value across the pair of events allows Notifier and orchestrator analytics to stitch lifecycle data together.
|
||||
|
||||
## 5. Versioning and evolution
|
||||
|
||||
- Increment the `version` field and the `@<version>` suffix for **breaking** changes (field removals, type changes, semantic shifts).
|
||||
- Additive optional fields may remain within version 1; update the JSON schema and samples accordingly.
|
||||
- When introducing `@2`, keep the `@1` schema/docs in place until orchestrator subscribers confirm migration.
|
||||
|
||||
## 6. Consumer checklist
|
||||
|
||||
1. Validate incoming payloads against the schema for the targeted version.
|
||||
2. Use `idempotencyKey` for dedupe, not `eventId`.
|
||||
3. Map `traceId`/`spanId` into telemetry spans to preserve causality.
|
||||
4. Prefer `payload.report` → `policy.revisionId` when populating templates; the top-level `attributes` are convenience duplicates for quick routing.
|
||||
5. Reserve the legacy Redis events for transitional compatibility only; downstream systems should subscribe to the orchestrator bus exposed by ORCH-SVC-38-101.
|
||||
|
||||
## 7. Implementation status and next actions
|
||||
|
||||
- **Scanner WebService** — `SCANNER-EVENTS-16-301` (blocked) and `SCANNER-EVENTS-16-302` (doing) track the production of these envelopes. The remaining blocker is the .NET 10 preview OpenAPI/Auth dependency drift that currently breaks `dotnet test`. Once Gateway and Notifier owners land the replacement packages, rerun the full test suite and capture fresh fixtures under `docs/events/samples/`.
|
||||
- **Gateway/Notifier consumers** — subscribe to the orchestrator stream documented in ORCH-SVC-38-101. When the Scanner tasks unblock, regenerate notifier contract tests against the sample events included here.
|
||||
- **Docs cadence** — update this file and the matching JSON schemas whenever payload fields change. Use the rehearsal checklist in `docs/ops/launch-cutover.md` to confirm downstream validation before the production cutover. Record gaps or newly required fields in `docs/ops/launch-readiness.md` so they land in the launch checklist.
|
||||
|
||||
---
|
||||
|
||||
**Imposed rule reminder:** work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
# Scanner Orchestrator Events (ORCH-SVC-38-101)
|
||||
|
||||
Last updated: 2025-10-26
|
||||
|
||||
The Notifications Studio initiative (NOTIFY-SVC-38-001) and orchestrator backlog (ORCH-SVC-38-101) standardise how platform services emit lifecycle events. This document describes the Scanner WebService contract for the new **orchestrator envelopes** (`scanner.event.*`) and how they supersede the legacy Redis-backed `scanner.report.ready` / `scanner.scan.completed` events.
|
||||
|
||||
## 1. Envelope overview
|
||||
|
||||
Orchestrator events share a deterministic JSON envelope:
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `eventId` | `uuid` | Globally unique identifier generated per occurrence. |
|
||||
| `kind` | `string` | Event identifier; Scanner emits `scanner.event.report.ready` and `scanner.event.scan.completed`. |
|
||||
| `version` | `integer` | Schema version. Initial release uses `1`. |
|
||||
| `tenant` | `string` | Tenant that owns the scan/report. Mirrors Authority claims. |
|
||||
| `occurredAt` | `date-time` | UTC instant when the underlying state transition happened (e.g., report persisted). |
|
||||
| `recordedAt` | `date-time` | UTC instant when the event was durably written. Optional but recommended. |
|
||||
| `source` | `string` | Producer identifier (`scanner.webservice`). |
|
||||
| `idempotencyKey` | `string` | Deterministic key for duplicate suppression (see §4). |
|
||||
| `correlationId` | `string` | Maps back to the API request or scan identifier. |
|
||||
| `traceId` / `spanId` | `string` | W3C trace context propagated into downstream telemetry. |
|
||||
| `scope` | `object` | Describes the affected artefact. Requires `repo` and `digest`; optional `namespace`, `component`, `image`. |
|
||||
| `attributes` | `object` | Flat string map for frequently queried metadata (e.g., policy revision). |
|
||||
| `payload` | `object` | Event-specific body (see §2). |
|
||||
|
||||
Canonical schemas live under `docs/events/scanner.event.*@1.json`. Samples that round-trip through `NotifyCanonicalJsonSerializer` are stored in `docs/events/samples/`.
|
||||
|
||||
## 2. Event kinds and payloads
|
||||
|
||||
### 2.1 `scanner.event.report.ready`
|
||||
|
||||
Emitted once a signed report is persisted and attested. Payload highlights:
|
||||
|
||||
- `reportId` / `scanId` — identifiers for the persisted report and originating scan. Until Scan IDs are surfaced by the API, `scanId` mirrors `reportId` so downstream correlators can stabilise on a single key.
|
||||
- **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict` — pre-sorted for deterministic routing.
|
||||
- **Links:**
|
||||
- `ui` → `/ui/reports/{reportId}` on the current host.
|
||||
- `report` → `{apiBasePath}/{reportsSegment}/{reportId}` (defaults to `/api/v1/reports/{reportId}`).
|
||||
- `policy` → `{apiBasePath}/{policySegment}/revisions/{revisionId}` when a revision is present.
|
||||
- `attestation` → `/ui/attestations/{reportId}` when a DSSE envelope is included.
|
||||
- `imageDigest` — OCI image digest associated with the analysis.
|
||||
- `generatedAt` — report generation timestamp (ISO-8601 UTC).
|
||||
- `verdict` — `pass`, `warn`, or `fail` after policy evaluation.
|
||||
- `summary` — blocked/warned/ignored/quieted counters (all non-negative integers).
|
||||
- `delta` — newly critical/high counts and optional `kev` array.
|
||||
- `quietedFindingCount` — mirrors `summary.quieted`.
|
||||
- `policy` — revision metadata (`digest`, `revisionId`) surfaced for routing.
|
||||
- `links` — UI/report/policy URLs suitable for operators.
|
||||
- `dsse` — embedded DSSE envelope (payload, type, signature list).
|
||||
- `report` — canonical report document; identical to the DSSE payload.
|
||||
|
||||
Schema: `docs/events/scanner.event.report.ready@1.json`
|
||||
Sample: `docs/events/samples/scanner.event.report.ready@1.sample.json`
|
||||
|
||||
### 2.2 `scanner.event.scan.completed`
|
||||
|
||||
Emitted after scan execution finishes (success or policy failure). Payload highlights:
|
||||
|
||||
- `reportId` / `scanId` / `imageDigest` — identifiers mirroring the report-ready event. As with the report-ready payload, `scanId` currently mirrors `reportId` as a temporary shim.
|
||||
- **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict`.
|
||||
- **Links:** same as above (`ui`, `report`, `policy`) with `attestation` populated when DSSE metadata exists.
|
||||
- `verdict`, `summary`, `delta`, `policy` — same semantics as above.
|
||||
- `findings` — array of surfaced findings with `id`, `severity`, optional `cve`, `purl`, and `reachability`.
|
||||
- `links`, `dsse`, `report` — same structure as §2.1 (allows Notifier to reuse signatures).
|
||||
|
||||
Schema: `docs/events/scanner.event.scan.completed@1.json`
|
||||
Sample: `docs/events/samples/scanner.event.scan.completed@1.sample.json`
|
||||
|
||||
### 2.3 Relationship to legacy events
|
||||
|
||||
| Legacy Redis event | Replacement orchestrator event | Notes |
|
||||
|--------------------|-------------------------------|-------|
|
||||
| `scanner.report.ready` | `scanner.event.report.ready` | Adds versioning, idempotency, trace context. Payload is a superset of the legacy fields. |
|
||||
| `scanner.scan.completed` | `scanner.event.scan.completed` | Same data plus explicit scan identifiers and orchestrator metadata. |
|
||||
|
||||
Legacy schemas remain for backwards-compatibility during migration, but new integrations **must** target the orchestrator variants.
|
||||
|
||||
## 3. Deterministic serialization
|
||||
|
||||
- Producers must serialise events using `NotifyCanonicalJsonSerializer` to guarantee consistent key ordering and whitespace.
|
||||
- Timestamps (`occurredAt`, `recordedAt`, `payload.generatedAt`) use `DateTimeOffset.UtcDateTime.ToString("O")`.
|
||||
- Payload arrays (`delta.kev`, `findings`) should be pre-sorted (e.g., alphabetical CVE order) so hash-based consumers remain stable.
|
||||
- Optional fields are omitted rather than emitted as `null`.
|
||||
|
||||
## 4. Idempotency and correlation
|
||||
|
||||
Idempotency keys dedupe repeated publishes and align with the orchestrator’s outbox pattern:
|
||||
|
||||
| Event kind | Idempotency key template |
|
||||
|------------|-------------------------|
|
||||
| `scanner.event.report.ready` | `scanner.event.report.ready:<tenant>:<reportId>` |
|
||||
| `scanner.event.scan.completed` | `scanner.event.scan.completed:<tenant>:<scanId>` |
|
||||
|
||||
Keys are ASCII lowercase; components should be trimmed and validated before concatenation. Retries must reuse the same key.
|
||||
|
||||
`correlationId` should match the scan identifier that appears in REST responses (`scanId`). Re-using the same value across the pair of events allows Notifier and orchestrator analytics to stitch lifecycle data together.
|
||||
|
||||
## 5. Versioning and evolution
|
||||
|
||||
- Increment the `version` field and the `@<version>` suffix for **breaking** changes (field removals, type changes, semantic shifts).
|
||||
- Additive optional fields may remain within version 1; update the JSON schema and samples accordingly.
|
||||
- When introducing `@2`, keep the `@1` schema/docs in place until orchestrator subscribers confirm migration.
|
||||
|
||||
## 6. Consumer checklist
|
||||
|
||||
1. Validate incoming payloads against the schema for the targeted version.
|
||||
2. Use `idempotencyKey` for dedupe, not `eventId`.
|
||||
3. Map `traceId`/`spanId` into telemetry spans to preserve causality.
|
||||
4. Prefer `payload.report` → `policy.revisionId` when populating templates; the top-level `attributes` are convenience duplicates for quick routing.
|
||||
5. Reserve the legacy Redis events for transitional compatibility only; downstream systems should subscribe to the orchestrator bus exposed by ORCH-SVC-38-101.
|
||||
|
||||
## 7. Implementation status and next actions
|
||||
|
||||
- **Scanner WebService** — `SCANNER-EVENTS-16-301` (blocked) and `SCANNER-EVENTS-16-302` (doing) track the production of these envelopes. The remaining blocker is the .NET 10 preview OpenAPI/Auth dependency drift that currently breaks `dotnet test`. Once Gateway and Notifier owners land the replacement packages, rerun the full test suite and capture fresh fixtures under `docs/events/samples/`.
|
||||
- **Gateway/Notifier consumers** — subscribe to the orchestrator stream documented in ORCH-SVC-38-101. When the Scanner tasks unblock, regenerate notifier contract tests against the sample events included here.
|
||||
- **Docs cadence** — update this file and the matching JSON schemas whenever payload fields change. Use the rehearsal checklist in `docs/modules/devops/runbooks/launch-cutover.md` to confirm downstream validation before the production cutover. Record gaps or newly required fields in `docs/modules/devops/runbooks/launch-readiness.md` so they land in the launch checklist.
|
||||
|
||||
---
|
||||
|
||||
**Imposed rule reminder:** work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
@@ -1,112 +0,0 @@
|
||||
# 1) What is StellaOps?
|
||||
|
||||
**StellaOps** an open, sovereign, modular container-security toolkit built for high-speed, offline operation, released under AGPL-3.0-or-later.
|
||||
|
||||
It follows an SBOM-first model—analyzing each container layer or ingesting existing CycloneDX/SPDX SBOMs, then enriching them with vulnerability, licence, secret-leak, and misconfiguration data to produce cryptographically signed reports.
|
||||
|
||||
Vulnerability detection maps OS and language dependencies to sources such as NVD, GHSA, OSV, ENISA.
|
||||
Secrets sweep flags exposed credentials or keys in files or environment variables.
|
||||
Licence audit identifies potential conflicts, especially copyleft obligations.
|
||||
Misconfiguration checks detect unsafe Dockerfile patterns (root user, latest tags, permissive modes).
|
||||
Provenance features include in-toto/SLSA attestations signed with cosign for supply-chain trust.
|
||||
|
||||
| Guiding principle | What it means for Concelier |
|
||||
|-------------------|---------------------------|
|
||||
| **SBOM-first ingest** | Prefer signed SBOMs or reproducible layer diffs before falling back to raw scraping; connectors treat source docs as provenance, never as mutable truth. |
|
||||
| **Deterministic outputs** | Same inputs yield identical canonical advisories and exported JSON/Trivy DB artefacts; merge hashes and export manifests are reproducible across machines. |
|
||||
| **Restart-time plug-ins only** | Connector/exporter plug-ins load at service start, keeping runtime sandboxing simple and avoiding hot-patch attack surface. |
|
||||
| **Sovereign/offline-first** | No mandatory outbound calls beyond allow-listed advisories; Offline Kit bundles Mongo snapshots and exporter artefacts for air-gapped installs. |
|
||||
| **Operational transparency** | Every stage logs structured events (fetch, parse, merge, export) with correlation IDs so parallel agents can debug without shared state. |
|
||||
|
||||
Performance: warm scans < 5 s, cold scans < 30 s on a 4 vCPU runner.
|
||||
Deployment: entirely SaaS-free, suitable for air-gapped or on-prem use through its Offline Kit.
|
||||
Policy: anonymous users → 33 scans/day; verified → 333 /day; nearing 90 % quota triggers throttling but never full blocks.
|
||||
|
||||
More documention is available ./docs/*.md files. Read `docs/README.md` to gather information about the available documentation. You could inquiry specific documents as your work requires it
|
||||
|
||||
---
|
||||
|
||||
# 3) Practices
|
||||
|
||||
## 3.1) Naming
|
||||
All modules are .NET projects based on .NET 10 (preview). Exclussion is the UI. It is based on Angular
|
||||
All modules are contained by one or more projects. Each project goes in its dedicated folder. Each project starts with StellaOps.<ModuleName>. In case it is common for for all StellaOps modules it is library or plugin and it is named StellaOps.<LibraryOrPlugin>.
|
||||
|
||||
## 3.2) Key technologies & integrations
|
||||
|
||||
- **Runtime**: .NET 10 (`net10.0`) preview SDK; C# latest preview features.
|
||||
- **Data**: MongoDB (canonical store and job/export state).
|
||||
- **Observability**: structured logs, counters, and (optional) OpenTelemetry traces.
|
||||
- **Ops posture**: offline‑first, allowlist for remote hosts, strict schema validation, gated LLM fallback (only where explicitly configured).
|
||||
|
||||
# 4) Modules
|
||||
StellaOps is contained by different modules installable via docker containers
|
||||
- Concelier. Responsible for aggregation and delivery of vulnerability database
|
||||
- Cli. Command line tool to unlock full potential - request database operations, install scanner, request scan, configure backend
|
||||
- Backend. Configures and Manages scans
|
||||
- UI. UI to access the backend (and scanners)
|
||||
- Agent. Installable daemon that does the scanning
|
||||
- Zastava. Realtime monitor for allowed (verified) installations.
|
||||
|
||||
For more information of the architecture see `./docs/*ARCHITECTURE*.md` files.
|
||||
|
||||
---
|
||||
|
||||
### 4.1.4) Glossary (quick)
|
||||
|
||||
- **OVAL** — Vendor/distro security definition format; authoritative for OS packages.
|
||||
- **NEVRA / EVR** — RPM and Debian version semantics for OS packages.
|
||||
- **PURL / SemVer** — Coordinates and version semantics for OSS ecosystems.
|
||||
- **KEV** — Known Exploited Vulnerabilities (flag only).
|
||||
|
||||
---
|
||||
# 5) Your role as StellaOps contributor
|
||||
|
||||
You acting as information technology engineer that will take different type of roles in goal achieving StellaOps production implementation
|
||||
In order you to work - you have to be supplied with directory that contains `AGENTS.md`,`TASKS.md` files. There will you have more information about the role you have, the scope of your work and the tasks you will have.
|
||||
|
||||
Boundaries:
|
||||
- You operate only in the working directories I gave you, unless there is dependencies that makes you to work on dependency in shared directory. Then you ask for confirmation.
|
||||
|
||||
You main characteristics:
|
||||
- Keep endpoints small, deterministic, and cancellation-aware.
|
||||
- Improve logs/metrics as per tasks.
|
||||
- Update `TASKS.md` when moving tasks forward.
|
||||
- When you are done with all task you state explicitly you are done.
|
||||
- Impersonate the role described on working directory `AGENTS.md` you will read, if role is not available - take role of the CTO of the StellaOps in early stages.
|
||||
- You always strive for best practices
|
||||
- You always strive for re-usability
|
||||
- When in doubt of design decision - you ask then act
|
||||
- You are autonomus - meaning that you will work for long time alone and achieve maximum without stopping for stupid questions
|
||||
- You operate on the same directory where other agents will work. In case you need to work on directory that is dependency on provided `AGENTS.md`,`TASKS.md` files you have to ask for confirmation first.
|
||||
|
||||
## 5.1) Type of contributions
|
||||
|
||||
- **BE‑Base (Platform & Pipeline)**
|
||||
Owns DI, plugin host, job scheduler/coordinator, configuration binding, minimal API endpoints, and Mongo bootstrapping.
|
||||
- **BE‑Conn‑X (Connectors)**
|
||||
One agent per source family (NVD, Red Hat, Ubuntu, Debian, SUSE, GHSA, OSV, PSIRTs, CERTs, KEV, ICS). Implements fetch/parse/map with incremental watermarks.
|
||||
- **BE‑Merge (Canonical Merge & Dedupe)**
|
||||
Identity graph, precedence policies, canonical JSON serializer, and deterministic hashing (`merge_event`).
|
||||
- **BE‑Export (JSON & Trivy DB)**
|
||||
Deterministic export trees, Trivy DB packaging, optional ORAS push, and offline bundle.
|
||||
- **QA (Validation & Observability)**
|
||||
Schema tests, fixture goldens, determinism checks, metrics/logs/traces, e2e reproducibility runs.
|
||||
- **DevEx/Docs**
|
||||
Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews.
|
||||
|
||||
|
||||
## 5.2) Work rules (important)
|
||||
|
||||
- **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description.
|
||||
- **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting.
|
||||
- **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests.
|
||||
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED). Before starting of actual work - ensure you have set the task to DOING. When complete or stop update the status in corresponding TASKS.md and in ./SPRINTS.md file.
|
||||
- **Coordination**: In case task is discovered as blocked on other team or task, according TASKS.md files that dependency is on needs to be changed by adding new tasks describing the requirement. the current task must be updated as completed. In case task changes, scope or requirements or rules - other documentations needs be updated accordingly.
|
||||
- **Sprint synchronization**: When given task seek for relevant directory to work on from SPRINTS.md. Confirm its state on both SPRINTS.md and the relevant TASKS.md file. Always check the AGENTS.md in the relevant TASKS.md directory.
|
||||
- **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence.
|
||||
- **Test layout**: Use module-specific projects in `StellaOps.Concelier.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Concelier.Testing`.
|
||||
- **Execution autonomous**: In case you need to continue with more than one options just continue sequentially, unless the continue requires design decision.
|
||||
- **EPIC reference**: In case task (on relevant TASKS.md file) references in direct or indirect way an EPIC - then seek for epic document like ./EPIC_*.md. There will be more information how to implement the task.
|
||||
|
||||
---
|
||||
@@ -1,524 +0,0 @@
|
||||
Here’s the full write‑up you can drop into your repo as the canonical reference for Epic 1. It’s written in clean product‑doc style so it’s safe to check in as Markdown. No fluff, just everything you need to build, test, and police it.
|
||||
|
||||
---
|
||||
|
||||
# Epic 1: Aggregation‑Only Contract (AOC) Enforcement
|
||||
|
||||
> Short name: **AOC enforcement**
|
||||
> Services touched: **Conseiller (advisory ingestion), Excitator (VEX ingestion), Web API, Workers, Policy Engine, CLI, Console, Authority**
|
||||
> Data stores: **MongoDB primary, optional Redis/NATS for jobs**
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
**Aggregation‑Only Contract (AOC)** is the ingestion covenant for StellaOps. It defines a hard boundary between **collection** and **interpretation**:
|
||||
|
||||
* **Ingestion (Conseiller/Excitator)** only **collects** data and preserves it as immutable raw facts with provenance. It does not decide, merge, normalize, prioritize, or assign severity. It may compute **links** that help future joins (aliases, PURLs, CPEs), but never derived judgments.
|
||||
* **Policy evaluation** is the only place where merges, deduplication, consensus, severity computation, and status folding are allowed. It’s reproducible and traceable.
|
||||
|
||||
The AOC establishes:
|
||||
|
||||
* **Immutable raw stores**: `advisory_raw` and `vex_raw` documents with full provenance, signatures, checksums, and upstream identifiers.
|
||||
* **Linksets**: machine‑generated join hints (aliases, PURLs, CPEs, CVE/GHSA IDs) that never change the underlying source content.
|
||||
* **Invariants**: a strict set of “never do this in ingestion” rules enforced by schema validation, runtime guards, and CI checks.
|
||||
* **AOC Verifier**: a build‑time and runtime watchdog that blocks non‑compliant code and data writes.
|
||||
|
||||
This epic delivers: schemas, guards, error codes, APIs, tests, migration, docs, and ops dashboards to make AOC non‑negotiable across the platform.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
AOC makes results **auditable, deterministic, and organization‑specific**. Source vendors disagree; your policies decide. By removing hidden heuristics from ingestion, we avoid unexplainable risk changes, race conditions between collectors, and vendor bias. Policy‑time evaluation yields reproducible deltas with complete “why” traces.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (deep details)
|
||||
|
||||
### 3.1 Core invariants
|
||||
|
||||
The following must be true for every write to `advisory_raw` and `vex_raw` and for every ingestion pipeline:
|
||||
|
||||
1. **No severity in ingestion**
|
||||
|
||||
* Forbidden fields: `severity`, `cvss`, `cvss_vector`, `effective_status`, `effective_range`, `merged_from`, `consensus_provider`, `reachability`, `asset_criticality`, `risk_score`.
|
||||
2. **No merges or de‑dups in ingestion**
|
||||
|
||||
* No combining two upstream advisories into one. No picking a single truth when multiple VEX statements exist.
|
||||
3. **Provenance is mandatory**
|
||||
|
||||
* Every raw doc includes `provenance` and `signature/checksum`.
|
||||
4. **Idempotent upserts**
|
||||
|
||||
* Same upstream document (by `upstream_id` + `source` + `content_hash`) must not create duplicates.
|
||||
5. **Append‑only versioning**
|
||||
|
||||
* Revisions from the source create new immutable documents with `supersedes` pointers; no in‑place edits.
|
||||
6. **Linkset only**
|
||||
|
||||
* Ingestion can compute and store a `linkset` for join performance. Linkset does not alter or infer severity/status.
|
||||
7. **Policy‑time only for effective findings**
|
||||
|
||||
* Only the Policy Engine can write `effective_finding_*` materializations.
|
||||
8. **Schema safety**
|
||||
|
||||
* Strict JSON schema validation at DB level; unknown fields reject writes.
|
||||
9. **Clock discipline**
|
||||
|
||||
* Timestamps are UTC, monotonic within a batch; collectors record `fetched_at` and `received_at`.
|
||||
|
||||
### 3.2 Data model
|
||||
|
||||
#### 3.2.1 `advisory_raw` (Mongo collection)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "advisory_raw:osv:GHSA-xxxx-....:v3",
|
||||
"source": {
|
||||
"vendor": "OSV",
|
||||
"stream": "github",
|
||||
"api": "https://api.osv.dev/v1/.../GHSA-...",
|
||||
"collector_version": "conseiller/1.7.3"
|
||||
},
|
||||
"upstream": {
|
||||
"upstream_id": "GHSA-xxxx-....",
|
||||
"document_version": "2024-09-01T12:13:14Z",
|
||||
"fetched_at": "2025-01-02T03:04:05Z",
|
||||
"received_at": "2025-01-02T03:04:06Z",
|
||||
"content_hash": "sha256:...",
|
||||
"signature": {
|
||||
"present": true,
|
||||
"format": "dsse",
|
||||
"key_id": "rekor:.../key/abc",
|
||||
"sig": "base64..."
|
||||
}
|
||||
},
|
||||
"content": {
|
||||
"format": "OSV",
|
||||
"spec_version": "1.6",
|
||||
"raw": { /* full upstream JSON, unmodified */ }
|
||||
},
|
||||
"identifiers": {
|
||||
"cve": ["CVE-2023-1234"],
|
||||
"ghsa": ["GHSA-xxxx-...."],
|
||||
"aliases": ["CVE-2023-1234", "GHSA-xxxx-...."]
|
||||
},
|
||||
"linkset": {
|
||||
"purls": ["pkg:npm/lodash@4.17.21", "pkg:maven/..."],
|
||||
"cpes": ["cpe:2.3:a:..."],
|
||||
"references": [
|
||||
{"type":"advisory","url":"https://..."},
|
||||
{"type":"fix","url":"https://..."}
|
||||
],
|
||||
"reconciled_from": ["content.raw.affected.ranges", "content.raw.pkg"]
|
||||
},
|
||||
"supersedes": "advisory_raw:osv:GHSA-xxxx-....:v2",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
> Note: No `severity`, no `cvss`, no `effective_*`. If the upstream payload includes CVSS, it stays inside `content.raw` and is not promoted or normalized at ingestion.
|
||||
|
||||
#### 3.2.2 `vex_raw` (Mongo collection)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "vex_raw:vendorX:doc-123:v4",
|
||||
"source": {
|
||||
"vendor": "VendorX",
|
||||
"stream": "vex",
|
||||
"api": "https://.../vex/doc-123",
|
||||
"collector_version": "excitator/0.9.2"
|
||||
},
|
||||
"upstream": {
|
||||
"upstream_id": "doc-123",
|
||||
"document_version": "2025-01-15T08:09:10Z",
|
||||
"fetched_at": "2025-01-16T01:02:03Z",
|
||||
"received_at": "2025-01-16T01:02:03Z",
|
||||
"content_hash": "sha256:...",
|
||||
"signature": { "present": true, "format": "cms", "key_id": "kid:...", "sig": "..." }
|
||||
},
|
||||
"content": {
|
||||
"format": "CycloneDX-VEX", // or "CSAF-VEX"
|
||||
"spec_version": "1.5",
|
||||
"raw": { /* full upstream VEX */ }
|
||||
},
|
||||
"identifiers": {
|
||||
"statements": [
|
||||
{
|
||||
"advisory_ids": ["CVE-2023-1234","GHSA-..."],
|
||||
"component_purls": ["pkg:deb/openssl@1.1.1"],
|
||||
"status": "not_affected",
|
||||
"justification": "component_not_present"
|
||||
}
|
||||
]
|
||||
},
|
||||
"linkset": {
|
||||
"purls": ["pkg:deb/openssl@1.1.1"],
|
||||
"cves": ["CVE-2023-1234"],
|
||||
"ghsas": ["GHSA-..."]
|
||||
},
|
||||
"supersedes": "vex_raw:vendorX:doc-123:v3",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
> VEX statuses remain as raw facts. No cross‑provider consensus is computed here.
|
||||
|
||||
### 3.3 Database validation
|
||||
|
||||
* MongoDB JSON Schema validators on both collections:
|
||||
|
||||
* Reject forbidden fields at the top level.
|
||||
* Enforce presence of `source`, `upstream`, `content`, `linkset`, `tenant`.
|
||||
* Enforce string formats for timestamps and hashes.
|
||||
|
||||
### 3.4 Write paths
|
||||
|
||||
1. **Collector fetches upstream**
|
||||
|
||||
* Normalize transport (gzip/json), compute `content_hash`, verify signature if available.
|
||||
2. **Build raw doc**
|
||||
|
||||
* Populate `source`, `upstream`, `content.raw`, `identifiers`, `linkset`.
|
||||
3. **Idempotent upsert**
|
||||
|
||||
* Lookup by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. If exists, skip; if new content hash, insert new revision with `supersedes`.
|
||||
4. **AOC guard**
|
||||
|
||||
* Runtime interceptor inspects write payload; if any forbidden field detected, reject with `ERR_AOC_001`.
|
||||
5. **Metrics**
|
||||
|
||||
* Emit `ingestion_write_ok` or `ingestion_write_reject` with reason code.
|
||||
|
||||
### 3.5 Read paths (ingestion scope)
|
||||
|
||||
* Allow only listing, getting raw docs, and searching by linkset. No endpoints return “effective findings” from ingestion services.
|
||||
|
||||
### 3.6 Error codes
|
||||
|
||||
| Code | Meaning | HTTP |
|
||||
| ------------- | ------------------------------------------------------------ | ---- |
|
||||
| `ERR_AOC_001` | Forbidden field present (severity/consensus/normalized data) | 400 |
|
||||
| `ERR_AOC_002` | Merge attempt detected (multiple upstreams fused) | 400 |
|
||||
| `ERR_AOC_003` | Idempotency violation (duplicate without supersedes) | 409 |
|
||||
| `ERR_AOC_004` | Missing provenance fields | 422 |
|
||||
| `ERR_AOC_005` | Signature/checksum mismatch | 422 |
|
||||
| `ERR_AOC_006` | Attempt to write effective findings from ingestion context | 403 |
|
||||
| `ERR_AOC_007` | Unknown top‑level fields (schema violation) | 400 |
|
||||
|
||||
### 3.7 AOC Verifier
|
||||
|
||||
A build‑time and runtime safeguard:
|
||||
|
||||
* **Static checks (CI)**
|
||||
|
||||
* Block imports of `*.Policy*` or `*.Merge*` from ingestion modules.
|
||||
* AST lint rule: any write to `advisory_raw` or `vex_raw` setting a forbidden key fails the build.
|
||||
* **Runtime checks**
|
||||
|
||||
* Repository layer interceptor inspects documents before insert/update; rejects forbidden fields and multi‑source merges.
|
||||
* **Drift detection job**
|
||||
|
||||
* Nightly job scans newest N docs; if violation found, pages ops and blocks new pipeline runs.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.8 Indexing strategy
|
||||
|
||||
* `advisory_raw`:
|
||||
|
||||
* `{ "identifiers.cve": 1 }`, `{ "identifiers.ghsa": 1 }`, `{ "linkset.purls": 1 }`, `{ "source.vendor": 1, "upstream.upstream_id": 1, "upstream.content_hash": 1 }` (unique), `{ "tenant": 1 }`.
|
||||
* `vex_raw`:
|
||||
|
||||
* `{ "identifiers.statements.advisory_ids": 1 }`, `{ "linkset.purls": 1 }`, `{ "source.vendor": 1, "upstream.upstream_id": 1, "upstream.content_hash": 1 }` (unique), `{ "tenant": 1 }`.
|
||||
|
||||
### 3.9 Interaction with Policy Engine
|
||||
|
||||
* Policy Engine pulls raw docs by identifiers/linksets and computes:
|
||||
|
||||
* De‑dup/merge per policy
|
||||
* Consensus for VEX statements
|
||||
* Severity normalization and risk scoring
|
||||
* Writes **only** to `effective_finding_{policyId}` collections.
|
||||
|
||||
A dedicated write guard refuses `effective_finding_*` writes from any caller that isn’t the Policy Engine service identity.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.10 Migration plan
|
||||
|
||||
1. **Freeze ingestion writes** except raw pass‑through.
|
||||
2. **Backfill**: copy existing ingestion collections to `_backup_*`.
|
||||
3. **Strip forbidden fields** from raw copies, move them into a temporary `advisory_view_legacy` used only by Policy Engine for parity.
|
||||
4. **Enable DB schema validators**.
|
||||
5. **Run collectors** in dry‑run; ensure only allowed keys land.
|
||||
6. **Switch Policy Engine** to pull exclusively from `*_raw` and to compute everything else.
|
||||
7. **Delete legacy normalized fields** in ingestion codepaths.
|
||||
8. **Enable runtime guards** and CI lint.
|
||||
|
||||
### 3.11 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `aoc_violation_total{code=...}`, `ingestion_write_total{result=ok|reject}`, `ingestion_signature_verified_total{result=ok|fail}`, `ingestion_latency_seconds`, `advisory_revision_count`.
|
||||
* Tracing: span `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`.
|
||||
* Logs: include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, `correlation_id`.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.12 Security and tenancy
|
||||
|
||||
* Every raw doc carries a `tenant` field.
|
||||
* Authority enforces `advisory:ingest` and `vex:ingest` scopes for ingestion endpoints.
|
||||
* Cross‑tenant reads/writes are blocked by default.
|
||||
* Secrets never logged; signatures verified with pinned trust stores.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.13 CLI and Console behavior
|
||||
|
||||
* **CLI**
|
||||
|
||||
* `stella sources ingest --dry-run` prints would‑write payload and explicitly shows that no severity/status fields are present.
|
||||
* `stella aoc verify` scans last K documents and reports violations with exit codes.
|
||||
* **Console**
|
||||
|
||||
* Sources dashboard shows AOC pass/fail per job, most recent violation codes, and a drill‑down to the offending document.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 4) API surface (ingestion scope)
|
||||
|
||||
### 4.1 Conseiller (Advisories)
|
||||
|
||||
* `POST /ingest/advisory`
|
||||
|
||||
* Body: raw upstream advisory with metadata; server constructs document, not the client.
|
||||
* Rejections: `ERR_AOC_00x` per table above.
|
||||
* `GET /advisories/raw/{id}`
|
||||
* `GET /advisories/raw?cve=CVE-...&purl=pkg:...&tenant=...`
|
||||
* `GET /advisories/raw/{id}/provenance`
|
||||
* `POST /aoc/verify?since=ISO8601` returns summary stats and first N violations.
|
||||
|
||||
### 4.2 Excitator (VEX)
|
||||
|
||||
* `POST /ingest/vex`
|
||||
* `GET /vex/raw/{id}`
|
||||
* `GET /vex/raw?advisory_id=CVE-...&purl=pkg:...`
|
||||
* `POST /aoc/verify?since=ISO8601`
|
||||
|
||||
All endpoints require `tenant` scope and appropriate `:write` or `:read`.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 5) Example: end‑to‑end flow
|
||||
|
||||
1. Collector fetches `GHSA-1234` from OSV.
|
||||
2. Build `advisory_raw` with linkset PURLs.
|
||||
3. Insert; AOC guard approves.
|
||||
4. Policy Engine later evaluates SBOM `S-42` under `policy P-7`, reads raw advisory and any VEX raw docs, computes effective findings, and writes to `effective_finding_P-7`.
|
||||
5. CLI `stella aoc verify --since 24h` returns `0` violations.
|
||||
|
||||
---
|
||||
|
||||
## 6) Implementation tasks
|
||||
|
||||
Breakdown by component with exact work items. Each section ends with the imposed sentence you requested.
|
||||
|
||||
### 6.1 Conseiller (advisory ingestion, WS + Worker)
|
||||
|
||||
* [ ] Add Mongo JSON schema validation for `advisory_raw`.
|
||||
* [ ] Implement repository layer with **write interceptors** that reject forbidden fields.
|
||||
* [ ] Compute `linkset` from upstream using deterministic mappers.
|
||||
* [ ] Enforce idempotency by unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)`.
|
||||
* [ ] Remove any normalization pipelines; relocate to Policy Engine.
|
||||
* [ ] Add `POST /ingest/advisory` and `GET /advisories/raw*` endpoints with Authority scope checks.
|
||||
* [ ] Emit observability metrics and traces.
|
||||
* [ ] Unit tests: schema violations, idempotency, supersedes chain, forbidden fields.
|
||||
* [ ] Integration tests: large batch ingest, linkset correctness against golden fixtures.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.2 Excitator (VEX ingestion, WS + Worker)
|
||||
|
||||
* [ ] Add Mongo JSON schema validation for `vex_raw`.
|
||||
* [ ] Implement repository layer guard identical to Conseiller.
|
||||
* [ ] Deterministic `linkset` extraction for advisory IDs and PURLs.
|
||||
* [ ] Endpoints `POST /ingest/vex`, `GET /vex/raw*` with scopes.
|
||||
* [ ] Remove any consensus or folding logic; leave VEX statements as raw.
|
||||
* [ ] Tests as per Conseiller, with rich fixtures for CycloneDX‑VEX and CSAF.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.3 Web API shared library
|
||||
|
||||
* [ ] Define `AOCForbiddenKeys` and export for both services.
|
||||
* [ ] Provide `AOCWriteGuard` middleware and `AOCError` types.
|
||||
* [ ] Provide `ProvenanceBuilder` utility.
|
||||
* [ ] Provide `SignatureVerifier` and `Checksum` helpers.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.4 Policy Engine
|
||||
|
||||
* [ ] Block any import/use from ingestion modules by lint rule.
|
||||
* [ ] Add hard gate on `effective_finding_*` writes that verifies caller identity is Policy Engine.
|
||||
* [ ] Update readers to pull fields only from `content.raw`, `identifiers`, `linkset`, not any legacy normalized fields.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.5 Authority
|
||||
|
||||
* [ ] Introduce scopes: `advisory:ingest`, `advisory:read`, `vex:ingest`, `vex:read`, `aoc:verify`.
|
||||
* [ ] Add `tenant` claim propagation to ingestion services.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.6 CLI
|
||||
|
||||
* [ ] `stella sources ingest --dry-run` and `stella aoc verify` commands.
|
||||
* [ ] Exit codes mapping to `ERR_AOC_00x`.
|
||||
* [ ] JSON output schema including violation list.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.7 Console
|
||||
|
||||
* [ ] Sources dashboard tiles: last run, AOC violations, top error codes.
|
||||
* [ ] Drill‑down page rendering offending doc with highlight on forbidden keys.
|
||||
* [ ] “Verify last 24h” action calling the AOC Verifier endpoint.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.8 CI/CD
|
||||
|
||||
* [ ] AST linter to forbid writes of banned keys in ingestion modules.
|
||||
* [ ] Unit test coverage gates for AOC guard code.
|
||||
* [ ] Pipeline stage that runs `stella aoc verify` against seeded DB snapshots.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Documentation changes (create/update these files)
|
||||
|
||||
1. **`/docs/ingestion/aggregation-only-contract.md`**
|
||||
|
||||
* Add: philosophy, invariants, schemas for `advisory_raw`/`vex_raw`, error codes, linkset definition, examples, idempotency rules, supersedes, API references, migration steps, observability, security.
|
||||
2. **`/docs/architecture/overview.md`**
|
||||
|
||||
* Update system diagram to show AOC boundary and raw stores; add sequence diagram: fetch → guard → raw insert → policy evaluation.
|
||||
3. **`/docs/architecture/policy-engine.md`**
|
||||
|
||||
* Clarify ingestion boundary; list inputs consumed from raw; note that any severity/consensus is policy‑time only.
|
||||
4. **`/docs/ui/console.md`**
|
||||
|
||||
* Add Sources dashboard section: AOC tiles and violation drill‑down.
|
||||
5. **`/docs/cli/cli-reference.md`**
|
||||
|
||||
* Add `stella aoc verify` and `stella sources ingest --dry-run` usage and exit codes.
|
||||
6. **`/docs/observability/observability.md`**
|
||||
|
||||
* Document new metrics, traces, logs keys for AOC.
|
||||
7. **`/docs/security/authority-scopes.md`**
|
||||
|
||||
* Add new scopes and tenancy enforcement for ingestion endpoints.
|
||||
8. **`/docs/deploy/containers.md`**
|
||||
|
||||
* Note DB validators must be enabled; environment flags for AOC guards; read‑only user for verify endpoint.
|
||||
|
||||
Each file should include a “Compliance checklist” subsection for AOC.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 8) Acceptance criteria
|
||||
|
||||
* DB validators are active and reject writes with forbidden fields.
|
||||
* AOC runtime guards log and reject violations with correct error codes.
|
||||
* CI linter prevents shipping code that writes forbidden keys to raw stores.
|
||||
* Ingestion of known fixture sets results in zero normalized fields outside `content.raw`.
|
||||
* Policy Engine is the only writer of `effective_finding_*` materializations.
|
||||
* CLI `stella aoc verify` returns success on clean datasets and non‑zero on seeded violations.
|
||||
* Console shows AOC status and violation drill‑downs.
|
||||
|
||||
---
|
||||
|
||||
## 9) Risks and mitigations
|
||||
|
||||
* **Collector drift**: new upstream fields tempt developers to normalize.
|
||||
|
||||
* Mitigation: CI linter + guard + schema validators; require RFC to extend linkset.
|
||||
* **Performance impact**: extra validation on write.
|
||||
|
||||
* Mitigation: guard is O(number of keys) and schema check is bounded; indexes sized appropriately.
|
||||
* **Migration complexity**: moving legacy normalized fields out.
|
||||
|
||||
* Mitigation: temporary `advisory_view_legacy` for parity; stepwise cutover.
|
||||
* **Tenant leakage**: missing tenant on write.
|
||||
|
||||
* Mitigation: schema requires `tenant`; middleware injects and asserts.
|
||||
|
||||
---
|
||||
|
||||
## 10) Test plan
|
||||
|
||||
* **Unit tests**
|
||||
|
||||
* Guard rejects forbidden keys; idempotency; supersedes chain; provenance required.
|
||||
* Signature verification paths: good, bad, absent.
|
||||
* **Property tests**
|
||||
|
||||
* Randomized upstream docs never produce forbidden keys at top level.
|
||||
* **Integration tests**
|
||||
|
||||
* Batch ingest of 50k advisories: throughput, zero violations.
|
||||
* Mixed VEX sources with contradictory statements remain separate in raw.
|
||||
* **Contract tests**
|
||||
|
||||
* Policy Engine refuses to run without raw inputs; writes only to `effective_finding_*`.
|
||||
* **End‑to‑end**
|
||||
|
||||
* Seed SBOM + advisories + VEX; ensure findings are identical pre/post migration.
|
||||
|
||||
---
|
||||
|
||||
## 11) Developer checklists
|
||||
|
||||
**Definition of Ready**
|
||||
|
||||
* Upstream spec reference attached.
|
||||
* Linkset mappers defined.
|
||||
* Example fixtures added.
|
||||
|
||||
**Definition of Done**
|
||||
|
||||
* DB validators deployed and tested.
|
||||
* Runtime guards enabled.
|
||||
* CI linter merged and enforced.
|
||||
* Docs updated (files in section 7).
|
||||
* Metrics visible on dashboard.
|
||||
* CLI verify passes.
|
||||
|
||||
---
|
||||
|
||||
## 12) Glossary
|
||||
|
||||
* **Raw document**: exact upstream content plus provenance, with join hints.
|
||||
* **Linkset**: PURLs/CPEs/IDs extracted to accelerate joins later.
|
||||
* **Supersedes**: pointer from a newer raw doc to the previous revision of the same upstream doc.
|
||||
* **Policy‑time**: evaluation phase where merges, consensus, and severity are computed.
|
||||
* **AOC**: Aggregation‑Only Contract.
|
||||
|
||||
---
|
||||
|
||||
### Final imposed reminder
|
||||
|
||||
**Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,566 +0,0 @@
|
||||
Fine. Here’s your next brick of “maximum documentation.” Try not to drop it on your foot.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 10: Export Center (JSON, Trivy DB, Mirror bundles)
|
||||
|
||||
**Short name:** `Export Center`
|
||||
**Primary service:** `exporter`
|
||||
**Surfaces:** Console (Web UI), CLI, Web API
|
||||
**Touches:** Conseiller (Feedser), Excitator (Vexer), SBOM Service, Policy Engine, VEX Consensus Lens, Findings Ledger, Authority (authN/Z), Object Storage, Orchestrator, Signing/Attestation, Telemetry
|
||||
|
||||
**AOC ground rule:** Conseiller and Excitator aggregate but never merge. The Export Center serializes evidence and policy results; it does not rewrite or “improve” your data in-flight.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The Export Center is the unified system for packaging StellaOps data into portable, verifiable bundles:
|
||||
|
||||
* **Canonical JSON exports** for advisories, VEX, SBOMs, findings, and policy-evaluation snapshots.
|
||||
* **Trivy DB compatible bundles** so downstream scanners can use Stella’s curated vulnerability knowledge without custom integrations.
|
||||
* **Mirror bundles** for air‑gapped or disconnected environments containing raw evidence, normalized records, indexes, and policy snapshots, with provenance, signatures, and optional encryption.
|
||||
|
||||
It centralizes format adapters, compliance and provenance, scheduling, versioning, and distribution (download, OCI push, file share). Every export is reproducible by `run_id`, cryptographically signed, and audit‑traceable back to source artifacts.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (brief)
|
||||
|
||||
Teams need to move results into other scanners, CI systems, and isolated networks without babysitting ten different scripts. The Export Center gives a single, policy‑aware, verifiable exit point that doesn’t surprise compliance or set your ops team on fire.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Capabilities
|
||||
|
||||
* **Profiles**
|
||||
|
||||
* `json:raw` canonical JSON lines for advisories, VEX, SBOMs.
|
||||
* `json:policy` adds policy evaluation results (allow/deny/risk, rationales).
|
||||
* `trivy:db` Trivy DB‑compatible export (core vulnerability DB).
|
||||
* `trivy:java-db` optional Java ecosystem DB export if enabled.
|
||||
* `mirror:full` air‑gap bundle with raw + normalized + indexes + policy + VEX consensus + SBOMs.
|
||||
* `mirror:delta` incremental bundle based on a previous export manifest.
|
||||
|
||||
* **Scope selectors**
|
||||
|
||||
* By time window, product, ecosystem, package, image digest, repository, tenant, tag.
|
||||
* Include/exclude SBOMs, advisories, VEX, findings, policy snapshots.
|
||||
|
||||
* **Policy awareness**
|
||||
|
||||
* Optionally bake a **policy snapshot** into the bundle, including the policy version, inputs and decision traces.
|
||||
* Can export **raw** evidence only (AOC) or **raw + evaluated**.
|
||||
|
||||
* **Provenance & signing**
|
||||
|
||||
* Generate attestation metadata with source URIs, artifact hashes, schema versions, export profile and filters.
|
||||
* Sign manifests and bundle using configured KMS; support detached and in‑bundle signatures.
|
||||
|
||||
* **Distribution**
|
||||
|
||||
* Download via Console or API (streaming).
|
||||
* Push to OCI registry as an artifact/image with annotations.
|
||||
* Write to object storage prefix for batch pickup.
|
||||
|
||||
* **Scheduling & automation**
|
||||
|
||||
* One‑off, cron, and event‑triggered exports (e.g., after a “VEX consensus recompute” run).
|
||||
* Retention policies and automatic expiry for old bundles.
|
||||
|
||||
* **Observability**
|
||||
|
||||
* Export metrics, throughput, size, downstream pull counts (when pushed to registry with report backs).
|
||||
|
||||
### 3.2 Architecture
|
||||
|
||||
* **exporter (service)**
|
||||
|
||||
* Orchestrates export runs, gathers records from the ledger and indexes, calls format adapters, writes bundles, signs, and publishes distribution tasks.
|
||||
* Stateless workers pull “export.jobs” from the orchestrator, stream data, and write manifests into object storage.
|
||||
|
||||
* **format adapters**
|
||||
|
||||
* Pluggable adapters:
|
||||
|
||||
* `adapter-json`: canonicalized JSONL writers per record type.
|
||||
* `adapter-trivy`: translates Stella’s normalized advisory model into Trivy DB format (and Java DB if enabled).
|
||||
* `adapter-mirror`: constructs a portable filesystem/OCI layout with manifests, indexes, and data subtrees.
|
||||
|
||||
* **manifesting & provenance**
|
||||
|
||||
* `export.json` (export manifest): profile, filters, counts, schema versions, content checksums, start/finish times, inputs list (artifact ids + hashes).
|
||||
* `provenance.json`: full chain back to source runs and artifacts; linked signatures.
|
||||
|
||||
* **distribution engines**
|
||||
|
||||
* `dist-http` streaming for downloads.
|
||||
* `dist-oci` layer writer with descriptors and annotations.
|
||||
* `dist-objstore` for staging to buckets.
|
||||
|
||||
* **security**
|
||||
|
||||
* Tenant scoping, RBAC on export creation and retrieval.
|
||||
* Optional in‑bundle encryption (age/AES‑GCM) with key wrapping using KMS.
|
||||
|
||||
### 3.3 Data model (selected tables)
|
||||
|
||||
* `export_profiles`
|
||||
|
||||
* `id`, `name`, `kind` (`json|trivy|mirror`), `variant` (`raw|policy|db|java-db|full|delta`), `config_json`, `created_by`, `created_at`.
|
||||
|
||||
* `export_runs`
|
||||
|
||||
* `id`, `profile_id`, `trigger` (`manual|schedule|event|api`), `state`, `filters_json`, `policy_version`, `started_at`, `finished_at`, `artifact_uri`, `size_bytes`, `sig_uri`, `provenance_uri`, `tenant_id`, `error_class`, `error_message`.
|
||||
|
||||
* `export_inputs`
|
||||
|
||||
* Link table between `export_runs` and source artifacts. `export_run_id`, `artifact_id`, `hash`.
|
||||
|
||||
* `export_distributions`
|
||||
|
||||
* `export_run_id`, `type` (`download|oci|objstore`), `target`, `state`, `meta_json`, `created_at`, `updated_at`.
|
||||
|
||||
### 3.4 Canonical file layouts
|
||||
|
||||
**JSON profile output**
|
||||
Directory layout under export root:
|
||||
|
||||
```
|
||||
/export/
|
||||
export.json # export manifest
|
||||
provenance.json # provenance and source artifact chain
|
||||
signatures/
|
||||
export.sig # detached signature for export.json
|
||||
advisories/
|
||||
normalized.jsonl # normalized advisory records
|
||||
vex/
|
||||
normalized.jsonl # normalized VEX records
|
||||
sboms/
|
||||
<subject>/sbom.spdx.json # one per subject; SPDX JSON or CycloneDX JSON
|
||||
findings/
|
||||
policy_evaluated.jsonl # if profile=json:policy
|
||||
```
|
||||
|
||||
**Trivy DB profile output**
|
||||
Produced as a compressed artifact:
|
||||
|
||||
```
|
||||
/export/
|
||||
export.json
|
||||
provenance.json
|
||||
trivy/
|
||||
db.bundle # Trivy DB compatible archive
|
||||
java-db.bundle # optional Java DB bundle (if enabled)
|
||||
signatures/
|
||||
trivy-db.sig
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
* The adapter keeps an internal mapping of Stella normalized fields to Trivy’s expected fields and namespaces. The mapping is versioned to track upstream schema evolution.
|
||||
|
||||
**Mirror bundle (filesystem layout)**
|
||||
|
||||
```
|
||||
/mirror/
|
||||
manifest.yaml # high-level bundle manifest (profile, filters, counts)
|
||||
export.json # same as JSON profile
|
||||
provenance.json
|
||||
indexes/
|
||||
advisories.index.json # quick lookups (pkg -> advisory ids)
|
||||
vex.index.json
|
||||
sbom.index.json
|
||||
advisories/raw/...
|
||||
advisories/normalized/...
|
||||
vex/raw/...
|
||||
vex/normalized/...
|
||||
sboms/raw/...
|
||||
sboms/graph/...
|
||||
policy/
|
||||
snapshot.yaml # full policy set used for evaluation
|
||||
evaluations.jsonl # decision outputs if requested
|
||||
consensus/
|
||||
vex_consensus.jsonl
|
||||
signatures/
|
||||
manifest.sig
|
||||
export.sig
|
||||
README.md
|
||||
```
|
||||
|
||||
**Mirror bundle (OCI layout)**
|
||||
Following standard OCI image artifact layout with annotations (`org.opencontainers.artifact.description`, `com.stella.export.profile`, `com.stella.export.filters`), and manifest lists for large bundles.
|
||||
|
||||
### 3.5 Export workflow
|
||||
|
||||
1. **Plan**
|
||||
Exporter computes candidates based on filters. For `mirror:delta`, compares with previous manifest to compute changes.
|
||||
|
||||
2. **Stream & write**
|
||||
Records are streamed from the Findings Ledger and stores. Writers are forward‑only, emitting JSONL or adapter‑specific structures, chunked for memory safety.
|
||||
|
||||
3. **Sign & attest**
|
||||
Once all content hashes are stable, Export Center writes `export.json`, `provenance.json`, and signs using KMS. Optional encryption wraps data layers.
|
||||
|
||||
4. **Distribute**
|
||||
Depending on profile settings, it exposes a download URL, pushes an OCI artifact, or writes to object storage. Distribution metadata is recorded.
|
||||
|
||||
5. **Audit & retention**
|
||||
Run, manifest, and signatures are immutable. Retention policy prunes large data folders after N days with manifests retained longer.
|
||||
|
||||
### 3.6 APIs
|
||||
|
||||
```
|
||||
POST /export/profiles
|
||||
GET /export/profiles?kind=&variant=
|
||||
GET /export/profiles/{id}
|
||||
PATCH /export/profiles/{id}
|
||||
DELETE /export/profiles/{id}
|
||||
|
||||
POST /export/runs
|
||||
GET /export/runs?state=&profile_id=&from=&to=&tenant_id=
|
||||
GET /export/runs/{run_id}
|
||||
POST /export/runs/{run_id}/cancel
|
||||
|
||||
GET /export/runs/{run_id}/download # presigned URL or streaming
|
||||
POST /export/runs/{run_id}/distribute # { "type":"oci|objstore", "target":"..." }
|
||||
GET /export/runs/{run_id}/manifest # export.json
|
||||
GET /export/runs/{run_id}/provenance
|
||||
GET /export/runs/{run_id}/signatures
|
||||
|
||||
GET /export/metrics/overview
|
||||
WS /export/streams/updates
|
||||
```
|
||||
|
||||
**Request example (create run):**
|
||||
|
||||
```json
|
||||
{
|
||||
"profile_id": "prof_json_policy",
|
||||
"filters": {
|
||||
"time_from": "2025-01-01T00:00:00Z",
|
||||
"time_to": "2025-01-31T23:59:59Z",
|
||||
"ecosystems": ["pypi", "npm"],
|
||||
"include": ["advisories", "vex", "sboms", "findings"]
|
||||
},
|
||||
"distribution": { "type": "download" },
|
||||
"policy_version": "pol-v1.8.2"
|
||||
}
|
||||
```
|
||||
|
||||
### 3.7 CLI
|
||||
|
||||
```
|
||||
stella export profiles list --kind mirror
|
||||
stella export profiles create --file profile.yaml
|
||||
stella export run create --profile prof_json_policy --from 2025-01-01 --to 2025-01-31 --include advisories,vex,sboms --download
|
||||
stella export run status <run-id>
|
||||
stella export run cancel <run-id>
|
||||
stella export run get <run-id> --manifest
|
||||
stella export run download <run-id> --out export-jan.tar.zst
|
||||
stella export distribute <run-id> --oci ghcr.io/org/stella-export:jan2025
|
||||
stella export verify <bundle> --manifest export.json --sig signatures/export.sig
|
||||
```
|
||||
|
||||
Exit codes: `0` ok, `2` bad args, `4` not found, `5` denied, `6` integrity failed, `8` export error.
|
||||
|
||||
### 3.8 RBAC & security
|
||||
|
||||
* Roles:
|
||||
|
||||
* `Export.Viewer`: list runs, download completed bundles.
|
||||
* `Export.Operator`: create runs, cancel, schedule, distribute.
|
||||
* `Export.Admin`: manage profiles, set retention, manage signing keys.
|
||||
|
||||
* Tenancy:
|
||||
|
||||
* Every run and artifact scoped by tenant; cross‑tenant export is disallowed.
|
||||
|
||||
* Secrets:
|
||||
|
||||
* KMS references for signing and encryption; never store private keys.
|
||||
|
||||
* PII & redaction:
|
||||
|
||||
* Exporters must not include secrets or credentials. Redaction rules enforced at writer level with schema‑based allowlists.
|
||||
|
||||
### 3.9 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `export_bytes_total{profile,tenant}`
|
||||
* `export_records_total{type}`
|
||||
* `export_duration_ms{profile}`
|
||||
* `export_failures_total{error_class}`
|
||||
* `export_distributions_total{type}`
|
||||
* `export_verify_fail_total`
|
||||
|
||||
* Traces:
|
||||
|
||||
* Spans per export phase: plan, stream, write, sign, distribute; baggage includes `export_run_id`.
|
||||
|
||||
* Logs:
|
||||
|
||||
* Structured JSON with counts, sizes, hashes, and redaction hints.
|
||||
|
||||
### 3.10 Performance targets
|
||||
|
||||
* Stream throughput ≥ 25k records/sec per worker for JSONL writing with compression.
|
||||
* Trivy bundle generation for 1M advisories ≤ 8 minutes on a standard worker.
|
||||
* Mirror delta export for 5% change set ≤ 2 minutes.
|
||||
|
||||
### 3.11 Edge cases & behavior
|
||||
|
||||
* **Schema drift**: adapter refuses to emit unknown fields without explicit mapping; run fails with `error_class=schema_mismatch`.
|
||||
* **Oversized bundles**: automatic sharding by time or content type; mirror OCI uses multi‑manifest indices.
|
||||
* **Missing policy snapshot**: profile `json:policy` will auto‑pull latest version unless pinned; pinning is recommended for reproducibility.
|
||||
* **Duplicate evidence**: writers dedupe by artifact hash and advisory id; AOC forbids merging.
|
||||
* **Air‑gap encryption**: if `encrypt=true`, mirror bundles require recipient public key material; decryption tooling documented.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Modules
|
||||
|
||||
* **New service:** `src/ExportCenter/StellaOps.ExportCenter`
|
||||
|
||||
* `api/` REST + WS
|
||||
* `planner/` scope planning, delta computation, sampling
|
||||
* `adapters/`
|
||||
|
||||
* `json/` canonical writers
|
||||
* `trivy/` db builders and schema mapping
|
||||
* `mirror/` fs/OCI builders, sharding, delta logic
|
||||
* `signing/` KMS clients, attestations
|
||||
* `dist/` download streaming, OCI push, object storage writer
|
||||
* `state/` repositories, migrations
|
||||
* `metrics/`, `audit/`, `security/`
|
||||
|
||||
* **SDK/CLI**
|
||||
|
||||
* `src/Cli/StellaOps.Cli` subcommands with streaming download and verification.
|
||||
|
||||
* **Console**
|
||||
|
||||
* `console/apps/export-center/` pages:
|
||||
|
||||
* Overview, Profiles, Runs, Run Detail, Distributions, Settings.
|
||||
* Components: `ExportPlanPreview`, `ProfileEditor`, `RunDiff`, `VerifyPanel`.
|
||||
|
||||
* **Existing services updates**
|
||||
|
||||
* Findings Ledger: new paginated streaming endpoints for advisories/VEX/SBOM/findings by filters and snapshots.
|
||||
* Policy Engine: “policy snapshot” exportable endpoint.
|
||||
* VEX Lens: consensus snapshot endpoint.
|
||||
|
||||
### 4.2 Packaging & deployment
|
||||
|
||||
* Containers:
|
||||
|
||||
* `stella/exporter:<ver>`
|
||||
* `stella/exporter-worker:<ver>` (optional separated worker pool)
|
||||
* Helm:
|
||||
|
||||
* WS replicas, concurrent export limits, default compression (`zstd`), default retention, KMS settings, OCI creds secrets.
|
||||
* DB migrations:
|
||||
|
||||
* Create `export_*` tables with proper indices (tenant, time, state).
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
* Phase 1: JSON (raw, policy) and Mirror (full) as download only.
|
||||
* Phase 2: Trivy DB adapters, OCI distribution.
|
||||
* Phase 3: Mirror deltas, encryption, verification tooling, scheduling.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Create/update the following docs; each page must end with the imposed rule statement.
|
||||
|
||||
1. `/docs/export-center/overview.md`
|
||||
Purpose, profiles, supported targets, AOC alignment, security model.
|
||||
|
||||
2. `/docs/export-center/architecture.md`
|
||||
Service components, adapters, manifests, signing, distribution flows.
|
||||
|
||||
3. `/docs/export-center/profiles.md`
|
||||
Profile schemas, examples, versioning, compatibility notes.
|
||||
|
||||
4. `/docs/export-center/api.md`
|
||||
All endpoints with request/response examples and error codes.
|
||||
|
||||
5. `/docs/export-center/cli.md`
|
||||
Commands with examples, scripts for CI/CD, verification.
|
||||
|
||||
6. `/docs/export-center/mirror-bundles.md`
|
||||
Filesystem and OCI layouts, delta exports, encryption, air‑gap import guide.
|
||||
|
||||
7. `/docs/export-center/trivy-adapter.md`
|
||||
Field mapping, supported ecosystems, compatibility and test matrix.
|
||||
|
||||
8. `/docs/export-center/provenance-and-signing.md`
|
||||
Manifest format, attestation details, verification process.
|
||||
|
||||
9. `/docs/operations/export-runbook.md`
|
||||
Common failures, recovery, tuning, capacity planning.
|
||||
|
||||
10. `/docs/security/export-hardening.md`
|
||||
RBAC, tenant isolation, secret redaction, encryption keys.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend: exporter
|
||||
|
||||
* [ ] Migrations for `export_profiles`, `export_runs`, `export_inputs`, `export_distributions`.
|
||||
* [ ] Planner to resolve filters to iterators over advisory/VEX/SBOM/findings datasets with pagination.
|
||||
* [ ] JSON adapter: canonical JSONL writers with schema normalization and redaction enforcement.
|
||||
* [ ] Policy snapshot embedder: pull policy version and evaluation outputs when requested.
|
||||
* [ ] Trivy adapter: implement schema mapping, writer, integrity validation, and compatibility version flag.
|
||||
* [ ] Mirror adapter: filesystem and OCI writer, sharding, manifest creation, delta computation.
|
||||
* [ ] Signing/attestation using KMS; detached and embedded options.
|
||||
* [ ] Distribution engines: download streaming, OCI push, object storage staging.
|
||||
* [ ] API layer with async export run handling and WebSocket updates.
|
||||
* [ ] Rate limit and concurrency controls per tenant/profile.
|
||||
* [ ] Audit logging for all create/cancel/distribute/verify actions.
|
||||
|
||||
### Integrations
|
||||
|
||||
* [ ] Findings Ledger streaming APIs for each content type.
|
||||
* [ ] Policy Engine endpoint to return deterministic policy snapshot and decision set by run.
|
||||
* [ ] VEX Lens endpoint to expose consensus snapshot.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Profiles CRUD with validation and test preview.
|
||||
* [ ] Create Run wizard with live count estimates and storage footprint prediction.
|
||||
* [ ] Runs list + detail page with manifest, provenance, and quick verify.
|
||||
* [ ] Download and distribution actions with progress and logs.
|
||||
* [ ] Verification panel to check signatures and hashes client‑side.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella export` commands as defined; include `verify` that checks signatures and hashes.
|
||||
* [ ] Auto‑resume of interrupted downloads with range requests.
|
||||
* [ ] Friendly error messages for schema mismatch and verification failure.
|
||||
|
||||
### Observability
|
||||
|
||||
* [ ] Metrics and traces per §3.9; dashboards for throughput, durations, failures, sizes.
|
||||
* [ ] Alerts for export failure rate and verify failures.
|
||||
|
||||
### Security & RBAC
|
||||
|
||||
* [ ] Enforce tenant scoping at query level; fuzz tests for leakage.
|
||||
* [ ] Role matrix checks on each API; Console hides forbidden actions.
|
||||
* [ ] Encryption test vectors and key rotation procedure.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author all files in §5 with concrete examples and diagrams.
|
||||
* [ ] Cross‑link from Orchestrator, Policy Studio, VEX Lens, and SBOM docs to the Export Center pages.
|
||||
* [ ] Append imposed rule line at the end of each page.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Implementation notes per profile
|
||||
|
||||
### 7.1 JSON: raw
|
||||
|
||||
* **Content:** advisories.normalized, vex.normalized, sboms (SPDX/CycloneDX), optional findings.raw.
|
||||
* **Normalization:** enforce Stella field casing, timestamps in RFC3339, unicode NFC.
|
||||
* **Compression:** `.jsonl.zst` per file to allow split/merge.
|
||||
|
||||
### 7.2 JSON: policy
|
||||
|
||||
* **Adds:** `policy_snapshot` and `findings.policy_evaluated.jsonl` with decision, rule_id, rationale, inputs fingerprint.
|
||||
* **Determinism:** include `policy_version` and `inputs_hash`; replays should match exactly.
|
||||
|
||||
### 7.3 Trivy DB
|
||||
|
||||
* **Mapping:**
|
||||
|
||||
* Package name, ecosystem, version ranges, CVE/CWE/aliases, severity mapping, vendor statements, fixed versions.
|
||||
* Ensure namespace mapping avoids collisions (e.g., distro vs ecosystem).
|
||||
* **Compatibility:** version flag in manifest; adapter throws if upstream schema version not supported.
|
||||
* **Validation:** run post‑build sanity checks (counts, required indexes).
|
||||
|
||||
### 7.4 Mirror: full/delta
|
||||
|
||||
* **Full:** everything needed to spin up an isolated read‑only Stella mirror with local search.
|
||||
* **Delta:** compute changed/added/removed advisory ids, VEX statements, SBOM subjects; update indexes and manifest with `base_export_id`.
|
||||
* **Encryption:** if enabled, encrypt data subtrees; leave `manifest.yaml` unencrypted for discoverability unless `strict=true`.
|
||||
|
||||
---
|
||||
|
||||
## 8) Acceptance criteria
|
||||
|
||||
* Operators can create an export with filters, download it, verify signatures, and trace back to source artifacts via provenance.
|
||||
* Trivy adapter produces a bundle consumable by Trivy without custom flags (basic validation in CI).
|
||||
* Mirror bundle imports successfully in an air‑gapped “mirror‑reader” sample app and serves queries from indexes.
|
||||
* Policy‑aware exports include deterministic decisions matching the specified `policy_version`.
|
||||
* RBAC prevents a Viewer from creating or canceling exports; tenancy prevents cross‑tenant leakage.
|
||||
* Metrics and dashboards show per‑profile throughput and error classes; alerts trigger on failure spikes.
|
||||
* Export retries are idempotent and do not duplicate content; hashes stable across re‑runs with identical inputs.
|
||||
|
||||
---
|
||||
|
||||
## 9) Risks & mitigations
|
||||
|
||||
* **Upstream schema changes break Trivy export.**
|
||||
Mitigation: versioned adapter with compatibility gate; integration tests against known fixtures; fail early with clear remediation.
|
||||
|
||||
* **Bundle size explosion.**
|
||||
Mitigation: zstd compression, sharding, delta exports, content‑addressed storage reuse for mirror OCI.
|
||||
|
||||
* **Data leakage via exports.**
|
||||
Mitigation: strict allowlist schemas, redaction filters, RBAC, tenant scoping tests, encryption for mirror.
|
||||
|
||||
* **Non‑deterministic policy outputs.**
|
||||
Mitigation: pin policy version and inputs hash; snapshot embedded rules; deterministic evaluation mode only.
|
||||
|
||||
* **Slow downloads/timeouts.**
|
||||
Mitigation: streaming with range support, resumable downloads, CDN integration if needed.
|
||||
|
||||
---
|
||||
|
||||
## 10) Test plan
|
||||
|
||||
* **Unit**
|
||||
Schema normalization; Trivy mapping; mirror delta computation; manifest hashing; signing.
|
||||
|
||||
* **Integration**
|
||||
End‑to‑end export with each profile; verify bundles; replay determinism of policy exports; OCI push/pull.
|
||||
|
||||
* **Compatibility**
|
||||
Validate Trivy bundle against a matrix of versions; import mirror bundle into a reference reader and run queries.
|
||||
|
||||
* **Security**
|
||||
Tenant isolation fuzzing; redaction checks; encryption round‑trip; signature verification with rotated keys.
|
||||
|
||||
* **Performance**
|
||||
Large dataset generation; parallel writer stress; OCI multi‑manifest publishing; download resume under packet loss.
|
||||
|
||||
* **Chaos**
|
||||
Kill exporter mid‑write; ensure resume or clean failure without partial corrupt bundles.
|
||||
|
||||
---
|
||||
|
||||
## 11) Philosophy
|
||||
|
||||
* **Ports, not prisons.** Exports should free your data to move with integrity and context, not trap it in a proprietary maze.
|
||||
* **Reproducible or it didn’t happen.** Every bit derived from known inputs, signed, traceable.
|
||||
* **Air‑gap is a first‑class citizen.** Mirror bundles are not an afterthought; they’re how serious orgs actually run.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1 +0,0 @@
|
||||
See `docs/airgap/EPIC_16_AIRGAP_MODE.md` for the full Epic 16 specification.
|
||||
@@ -1 +0,0 @@
|
||||
See `docs/api/EPIC_17_SDKS_OPENAPI.md` for the complete Epic 17 specification.
|
||||
@@ -1 +0,0 @@
|
||||
See `docs/risk/EPIC_18_RISK_PROFILES.md` for the complete Epic 18 specification.
|
||||
@@ -1 +0,0 @@
|
||||
See `docs/attestor/EPIC_19_ATTESTOR_CONSOLE.md` for the complete Epic 19 specification.
|
||||
@@ -1,567 +0,0 @@
|
||||
Fine. Here’s the next epic, written so you can paste it straight into the repo without having to babysit me. Same structure as before, maximum detail, zero hand‑waving.
|
||||
|
||||
---
|
||||
|
||||
# Epic 2: Policy Engine & Policy Editor (VEX + Advisory Application Rules)
|
||||
|
||||
> Short name: **Policy Engine v2**
|
||||
> Services touched: **Policy Engine, Web API, Console (Policy Editor), CLI, Conseiller, Excitator, SBOM Service, Authority, Workers/Scheduler**
|
||||
> Data stores: **MongoDB (policies, runs, effective findings), optional Redis/NATS for jobs**
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
This epic delivers the **organization‑specific decision layer** for Stella. Ingestion is now AOC‑compliant (Epic 1). That means advisories and VEX arrive as immutable raw facts. This epic builds the place where those facts become **effective findings** under policies you control.
|
||||
|
||||
Core deliverables:
|
||||
|
||||
* **Policy Engine**: deterministic evaluator that applies rule sets to inputs:
|
||||
|
||||
* Inputs: `advisory_raw`, `vex_raw`, SBOMs, optional telemetry hooks (reachability stubs), org metadata.
|
||||
* Outputs: `effective_finding_{policyId}` materializations, with full explanation traces.
|
||||
* **Policy Editor (Console + CLI)**: versioned policy authoring, simulation, review/approval workflow, and change diffs.
|
||||
* **Rules DSL v1**: safe, declarative language for VEX application, advisory normalization, and risk scoring. No arbitrary code execution, no network calls.
|
||||
* **Run Orchestrator**: incremental re‑evaluation when new raw facts or SBOM changes arrive; efficient partial updates.
|
||||
|
||||
The philosophy is boring on purpose: policy is a **pure function of inputs**. Same inputs and same policy yield the same outputs, every time, on every machine. If you want drama, watch reality TV, not your risk pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* Vendors disagree, contexts differ, and your tolerance for risk is not universal.
|
||||
* VEX means nothing until you decide **how** to apply it to **your** assets.
|
||||
* Auditors care about the “why.” You’ll need consistent, replayable answers, with traces.
|
||||
* Security teams need **simulation** before rollouts, and **diffs** after.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (deep details)
|
||||
|
||||
### 3.1 Data model
|
||||
|
||||
#### 3.1.1 Policy documents (Mongo: `policies`)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "policy:P-7:v3",
|
||||
"policy_id": "P-7",
|
||||
"version": 3,
|
||||
"name": "Default Org Policy",
|
||||
"status": "approved", // draft | submitted | approved | archived
|
||||
"owned_by": "team:sec-plat",
|
||||
"valid_from": "2025-01-15T00:00:00Z",
|
||||
"valid_to": null,
|
||||
"dsl": {
|
||||
"syntax": "stella-dsl@1",
|
||||
"source": "rule-set text or compiled IR ref"
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Baseline scoring + VEX precedence",
|
||||
"tags": ["baseline","vex","cvss"]
|
||||
},
|
||||
"provenance": {
|
||||
"created_by": "user:ali",
|
||||
"created_at": "2025-01-15T08:00:00Z",
|
||||
"submitted_by": "user:kay",
|
||||
"approved_by": "user:root",
|
||||
"approval_at": "2025-01-16T10:00:00Z",
|
||||
"checksum": "sha256:..."
|
||||
},
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
* `status=approved` is required to run in production.
|
||||
* Version increments are append‑only. Old versions remain runnable for replay.
|
||||
|
||||
#### 3.1.2 Policy runs (Mongo: `policy_runs`)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "run:P-7:2025-02-20T12:34:56Z:abcd",
|
||||
"policy_id": "P-7",
|
||||
"policy_version": 3,
|
||||
"inputs": {
|
||||
"sbom_set": ["sbom:S-42"],
|
||||
"advisory_cursor": "2025-02-20T00:00:00Z",
|
||||
"vex_cursor": "2025-02-20T00:00:00Z"
|
||||
},
|
||||
"mode": "incremental", // full | incremental | simulate
|
||||
"stats": {
|
||||
"components": 1742,
|
||||
"advisories_considered": 9210,
|
||||
"vex_considered": 1187,
|
||||
"rules_fired": 68023,
|
||||
"findings_out": 4321
|
||||
},
|
||||
"trace": {
|
||||
"location": "blob://traces/run-.../index.json",
|
||||
"sampling": "smart-10pct"
|
||||
},
|
||||
"status": "succeeded", // queued | running | failed | succeeded | canceled
|
||||
"started_at": "2025-02-20T12:34:56Z",
|
||||
"finished_at": "2025-02-20T12:35:41Z",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
#### 3.1.3 Effective findings (Mongo: `effective_finding_P-7`)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "P-7:S-42:pkg:npm/lodash@4.17.21:CVE-2021-23337",
|
||||
"policy_id": "P-7",
|
||||
"policy_version": 3,
|
||||
"sbom_id": "S-42",
|
||||
"component_purl": "pkg:npm/lodash@4.17.21",
|
||||
"advisory_ids": ["CVE-2021-23337", "GHSA-..."],
|
||||
"status": "affected", // affected | not_affected | fixed | under_investigation | suppressed
|
||||
"severity": {
|
||||
"normalized": "High",
|
||||
"score": 7.5,
|
||||
"vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
|
||||
"rationale": "cvss_base(OSV) + vendor_weighting + env_modifiers"
|
||||
},
|
||||
"rationale": [
|
||||
{"rule":"vex.precedence","detail":"VendorX not_affected justified=component_not_present wins"},
|
||||
{"rule":"advisory.cvss.normalization","detail":"mapped GHSA severity to CVSS 3.1 = 7.5"}
|
||||
],
|
||||
"references": {
|
||||
"advisory_raw_ids": ["advisory_raw:osv:GHSA-...:v3"],
|
||||
"vex_raw_ids": ["vex_raw:VendorX:doc-123:v4"]
|
||||
},
|
||||
"run_id": "run:P-7:2025-02-20T12:34:56Z:abcd",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
Write protection: only the **Policy Engine** service identity may write any `effective_finding_*` collection.
|
||||
|
||||
---
|
||||
|
||||
### 3.2 Rules DSL v1 (stella‑dsl@1)
|
||||
|
||||
**Design goals**
|
||||
|
||||
* Declarative, composable, deterministic.
|
||||
* No loops, no network IO, no non‑deterministic time.
|
||||
* Policy authors see readable text; the engine compiles to a safe IR.
|
||||
|
||||
**Concepts**
|
||||
|
||||
* **WHEN** condition matches a tuple `(sbom_component, advisory, optional vex_statements)`
|
||||
* **THEN** actions set `status`, compute `severity`, attach `rationale`, or `suppress` with reason.
|
||||
* **Profiles** for severity and scoring; **Maps** for vendor weighting; **Guards** for VEX justification.
|
||||
|
||||
**Mini‑grammar (subset)**
|
||||
|
||||
```
|
||||
policy "Default Org Policy" syntax "stella-dsl@1" {
|
||||
|
||||
profile severity {
|
||||
map vendor_weight {
|
||||
source "GHSA" => +0.5
|
||||
source "OSV" => +0.0
|
||||
source "VendorX" => -0.2
|
||||
}
|
||||
env base_cvss {
|
||||
if env.runtime == "serverless" then -0.5
|
||||
if env.exposure == "internal-only" then -1.0
|
||||
}
|
||||
}
|
||||
|
||||
rule vex_precedence {
|
||||
when vex.any(status in ["not_affected","fixed"])
|
||||
and vex.justification in ["component_not_present","vulnerable_code_not_present"]
|
||||
then status := vex.status
|
||||
because "VEX strong justification prevails";
|
||||
}
|
||||
|
||||
rule advisory_to_cvss {
|
||||
when advisory.source in ["GHSA","OSV"]
|
||||
then severity := normalize_cvss(advisory)
|
||||
because "Map vendor severity or CVSS vector";
|
||||
}
|
||||
|
||||
rule reachability_soft_suppress {
|
||||
when severity.normalized <= "Medium"
|
||||
and telemetry.reachability == "none"
|
||||
then status := "suppressed"
|
||||
because "not reachable and low severity";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Built‑ins** (non‑exhaustive)
|
||||
|
||||
* `normalize_cvss(advisory)` maps GHSA/OSV/CSAF severity fields to CVSS v3.1 numbers when possible; otherwise vendor‑to‑numeric mapping table in policy.
|
||||
* `vex.any(...)` tests across matching VEX statements for the same `(component, advisory)`.
|
||||
* `telemetry.*` is an optional input namespace reserved for future reachability data; if absent, expressions evaluate to `unknown` (no effect).
|
||||
|
||||
**Determinism**
|
||||
|
||||
* Rules are evaluated in **stable order**: explicit `priority` attribute or lexical order.
|
||||
* **First‑match** semantics for conflicting status unless `combine` is used.
|
||||
* Severity computations are pure; numeric maps are part of policy document.
|
||||
|
||||
---
|
||||
|
||||
### 3.3 Evaluation model
|
||||
|
||||
1. **Selection**
|
||||
|
||||
* For each SBOM component PURL, find candidate advisories from `advisory_raw` via linkset PURLs or identifiers.
|
||||
* For each pair `(component, advisory)`, load all matching VEX facts from `vex_raw`.
|
||||
|
||||
2. **Context assembly**
|
||||
|
||||
* Build an evaluation context from:
|
||||
|
||||
* `sbom_component`: PURL, licenses, relationships.
|
||||
* `advisory`: source, identifiers, references, embedded vendor severity (kept in `content.raw`).
|
||||
* `vex`: list of statements with status and justification.
|
||||
* `env`: org‑specific env vars configured per policy run (e.g., exposure).
|
||||
* Optional `telemetry` if available.
|
||||
|
||||
3. **Rule execution**
|
||||
|
||||
* Compile DSL to IR once per policy version; cache.
|
||||
* Execute rules per tuple; record which rules fired and the order.
|
||||
* If no rule sets status, default is `affected`.
|
||||
* If no rule sets severity, default severity uses `normalize_cvss(advisory)` with vendor defaults.
|
||||
|
||||
4. **Materialization**
|
||||
|
||||
* Write to `effective_finding_{policyId}` with `rationale` chain and references to raw docs.
|
||||
* Emit per‑tuple trace events; sample and store full traces per run.
|
||||
|
||||
5. **Incremental updates**
|
||||
|
||||
* A watch job observes new `advisory_raw` and `vex_raw` inserts and SBOM deltas.
|
||||
* The orchestrator computes the affected tuples and re‑evaluates only those.
|
||||
|
||||
6. **Replay**
|
||||
|
||||
* Any `policy_run` is fully reproducible by `(policy_id, version, input set, cursors)`.
|
||||
|
||||
---
|
||||
|
||||
### 3.4 VEX application semantics
|
||||
|
||||
* **Precedence**: a `not_affected` with strong justification (`component_not_present`, `vulnerable_code_not_present`, `fix_not_required`) wins unless another rule explicitly overrides by environment context.
|
||||
* **Scoping**: VEX statements often specify product/component scope. Matching uses PURL equivalence and version ranges extracted during ingestion linkset generation.
|
||||
* **Conflicts**: If multiple VEX statements conflict, the default is **most‑specific scope wins** (component > product > vendor), then newest `document_version`. Policies can override with explicit rules.
|
||||
* **Explainability**: Every VEX‑driven decision records which statement IDs were considered and which one won.
|
||||
|
||||
---
|
||||
|
||||
### 3.5 Advisory normalization rules
|
||||
|
||||
* **Vendor severity mapping**: Map GHSA levels or CSAF product‑tree severities to CVSS‑like numeric bands via policy maps.
|
||||
* **CVSS vector use**: If a valid vector exists in `content.raw`, parse and compute; apply policy modifiers from `profile severity`.
|
||||
* **Temporal/environment modifiers**: Optional reductions for network exposure, isolation, or compensating controls, all encoded in policy.
|
||||
|
||||
---
|
||||
|
||||
### 3.6 Performance and scale
|
||||
|
||||
* Partition evaluation by SBOM ID and hash ranges of PURLs.
|
||||
* Pre‑index `advisory_raw.linkset.purls` and `vex_raw.linkset.purls` (already in Epic 1).
|
||||
* Use streaming iterators; avoid loading entire SBOM or advisory sets into memory.
|
||||
* Materialize only changed findings (diff‑aware writes).
|
||||
* Target: 100k components, 1M advisories considered, 5 minutes incremental SLA on commodity hardware.
|
||||
|
||||
---
|
||||
|
||||
### 3.7 Error codes
|
||||
|
||||
| Code | Meaning | HTTP |
|
||||
| ------------- | ----------------------------------------------------- | ---- |
|
||||
| `ERR_POL_001` | Policy syntax error | 400 |
|
||||
| `ERR_POL_002` | Policy not approved for run | 403 |
|
||||
| `ERR_POL_003` | Missing inputs (SBOM/advisory/vex fetch failed) | 424 |
|
||||
| `ERR_POL_004` | Determinism guard triggered (non‑pure function usage) | 500 |
|
||||
| `ERR_POL_005` | Write denied to effective findings (caller invalid) | 403 |
|
||||
| `ERR_POL_006` | Run canceled or timed out | 408 |
|
||||
|
||||
---
|
||||
|
||||
### 3.8 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `policy_compile_seconds`, `policy_run_seconds{mode=...}`, `rules_fired_total`, `findings_written_total`, `vex_overrides_total`, `simulate_diff_total{delta=up|down|unchanged}`.
|
||||
* Tracing:
|
||||
|
||||
* Spans: `policy.compile`, `policy.select`, `policy.eval`, `policy.materialize`.
|
||||
* Logs:
|
||||
|
||||
* Include `policy_id`, `version`, `run_id`, `sbom_id`, `component_purl`, `advisory_id`, `vex_count`, `rule_hits`.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
### 3.9 Security and tenancy
|
||||
|
||||
* Only users with `policy:write` can create/modify policies.
|
||||
* `policy:approve` is a separate privileged role.
|
||||
* Only Policy Engine service identity has `effective:write`.
|
||||
* Tenancy is explicit on all documents and queries.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 4) API surface
|
||||
|
||||
### 4.1 Policy CRUD and lifecycle
|
||||
|
||||
* `POST /policies` create draft
|
||||
* `GET /policies?status=...` list
|
||||
* `GET /policies/{policyId}/versions/{v}` fetch
|
||||
* `POST /policies/{policyId}/submit` move draft to submitted
|
||||
* `POST /policies/{policyId}/approve` approve version
|
||||
* `POST /policies/{policyId}/archive` archive version
|
||||
|
||||
### 4.2 Compilation and validation
|
||||
|
||||
* `POST /policies/{policyId}/versions/{v}/compile`
|
||||
|
||||
* Returns IR checksum, syntax diagnostics, rule stats.
|
||||
|
||||
### 4.3 Runs
|
||||
|
||||
* `POST /policies/{policyId}/runs` body: `{mode, sbom_set, advisory_cursor?, vex_cursor?, env?}`
|
||||
* `GET /policies/{policyId}/runs/{runId}` status + stats
|
||||
* `POST /policies/{policyId}/simulate` returns **diff** vs current approved version on a sample SBOM set.
|
||||
|
||||
### 4.4 Findings and explanations
|
||||
|
||||
* `GET /findings/{policyId}?sbom_id=S-42&status=affected&severity=High+Critical`
|
||||
* `GET /findings/{policyId}/{findingId}/explain` returns ordered rule hits and linked raw IDs.
|
||||
|
||||
All endpoints require tenant scoping and appropriate `policy:*` or `findings:*` roles.
|
||||
|
||||
---
|
||||
|
||||
## 5) Console (Policy Editor) and CLI behavior
|
||||
|
||||
**Console**
|
||||
|
||||
* Monaco‑style editor with DSL syntax highlighting, lint, quick docs.
|
||||
* Side‑by‑side **Simulation** panel: show count of affected findings before/after.
|
||||
* Approval workflow: submit, review comments, approve with rationale.
|
||||
* Diffs: show rule‑wise changes and estimated impact.
|
||||
* Read‑only run viewer: heatmap of rules fired, top suppressions, VEX wins.
|
||||
|
||||
**CLI**
|
||||
|
||||
* `stella policy new --name "Default Org Policy"`
|
||||
* `stella policy edit P-7` opens local editor -> `submit`
|
||||
* `stella policy approve P-7 --version 3`
|
||||
* `stella policy simulate P-7 --sbom S-42 --env exposure=internal-only`
|
||||
* `stella findings ls --policy P-7 --sbom S-42 --status affected`
|
||||
|
||||
Exit codes map to `ERR_POL_*`.
|
||||
|
||||
---
|
||||
|
||||
## 6) Implementation tasks
|
||||
|
||||
### 6.1 Policy Engine service
|
||||
|
||||
* [ ] Implement DSL parser and IR compiler (`stella-dsl@1`).
|
||||
* [ ] Build evaluator with stable ordering and first‑match semantics.
|
||||
* [ ] Implement selection joiners for SBOM↔advisory↔vex using linksets.
|
||||
* [ ] Materialization writer with upsert‑only semantics to `effective_finding_{policyId}`.
|
||||
* [ ] Determinism guard (ban wall‑clock, network, and RNG during eval).
|
||||
* [ ] Incremental orchestrator listening to advisory/vex/SBOM change streams.
|
||||
* [ ] Trace emitter with rule‑hit sampling.
|
||||
* [ ] Unit tests, property tests, golden fixtures; perf tests to target SLA.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.2 Web API
|
||||
|
||||
* [ ] Policy CRUD, compile, run, simulate, findings, explain endpoints.
|
||||
* [ ] Pagination, filters, and tenant enforcement on all list endpoints.
|
||||
* [ ] Error mapping to `ERR_POL_*`.
|
||||
* [ ] Rate limits on simulate endpoints.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.3 Console (Policy Editor)
|
||||
|
||||
* [ ] Editor with DSL syntax highlighting and inline diagnostics.
|
||||
* [ ] Simulation UI with pre/post counts and top deltas.
|
||||
* [ ] Approval workflow UI with audit trail.
|
||||
* [ ] Run viewer dashboards (rule heatmap, VEX wins, suppressions).
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.4 CLI
|
||||
|
||||
* [ ] New commands: `policy new|edit|submit|approve|simulate`, `findings ls|get`.
|
||||
* [ ] Json/YAML output formats for CI consumption.
|
||||
* [ ] Non‑zero exits on syntax errors or simulation failures; map to `ERR_POL_*`.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.5 Conseiller & Excitator integration
|
||||
|
||||
* [ ] Provide search endpoints optimized for policy selection (batch by PURLs and IDs).
|
||||
* [ ] Harden linkset extraction to maximize join recall.
|
||||
* [ ] Add cursors for incremental selection windows per run.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.6 SBOM Service
|
||||
|
||||
* [ ] Ensure fast PURL index and component metadata projection for policy queries.
|
||||
* [ ] Provide relationship graph API for future transitive logic.
|
||||
* [ ] Emit change events on SBOM updates.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.7 Authority
|
||||
|
||||
* [ ] Define scopes: `policy:write`, `policy:approve`, `policy:run`, `findings:read`, `effective:write`.
|
||||
* [ ] Issue service identity for Policy Engine with `effective:write` only.
|
||||
* [ ] Enforce tenant claims at gateway.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.8 CI/CD
|
||||
|
||||
* [ ] Lint policy DSL in PRs; block invalid syntax.
|
||||
* [ ] Run `simulate` against golden SBOMs to detect explosive deltas.
|
||||
* [ ] Determinism CI: two runs with identical seeds produce identical outputs.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Documentation changes (create/update these files)
|
||||
|
||||
1. **`/docs/policy/overview.md`**
|
||||
|
||||
* What the Policy Engine is, high‑level concepts, inputs, outputs, determinism.
|
||||
2. **`/docs/policy/dsl.md`**
|
||||
|
||||
* Full grammar, built‑ins, examples, best practices, anti‑patterns.
|
||||
3. **`/docs/policy/lifecycle.md`**
|
||||
|
||||
* Draft → submitted → approved → archived, roles, and audit trail.
|
||||
4. **`/docs/policy/runs.md`**
|
||||
|
||||
* Run modes, incremental mechanics, cursors, replay.
|
||||
5. **`/docs/api/policy.md`**
|
||||
|
||||
* Endpoints, request/response schemas, error codes.
|
||||
6. **`/docs/cli/policy.md`**
|
||||
|
||||
* Command usage, examples, exit codes, JSON output contracts.
|
||||
7. **`/docs/ui/policy-editor.md`**
|
||||
|
||||
* Screens, workflows, simulation, diffs, approvals.
|
||||
8. **`/docs/architecture/policy-engine.md`**
|
||||
|
||||
* Detailed sequence diagrams, selection/join strategy, materialization schema.
|
||||
9. **`/docs/observability/policy.md`**
|
||||
|
||||
* Metrics, tracing, logs, sample dashboards.
|
||||
10. **`/docs/security/policy-governance.md`**
|
||||
|
||||
* Scopes, approvals, tenancy, least privilege.
|
||||
11. **`/docs/examples/policies/`**
|
||||
|
||||
* `baseline.pol`, `serverless.pol`, `internal-only.pol`, each with commentary.
|
||||
12. **`/docs/faq/policy-faq.md`**
|
||||
|
||||
* Common pitfalls, VEX conflict handling, determinism gotchas.
|
||||
|
||||
Each file includes a **Compliance checklist** for authors and reviewers.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 8) Acceptance criteria
|
||||
|
||||
* Policies are versioned, approvable, and compilable; invalid DSL blocks merges.
|
||||
* Engine produces deterministic outputs with full rationale chains.
|
||||
* VEX precedence rules work per spec and are overridable by policy.
|
||||
* Simulation yields accurate pre/post deltas and diffs.
|
||||
* Only Policy Engine can write to `effective_finding_*`.
|
||||
* Incremental runs pick up new advisories/VEX/SBOM changes without full re‑runs.
|
||||
* Console and CLI cover authoring, simulation, approval, and retrieval.
|
||||
* Observability dashboards show rule hits, VEX wins, and run timings.
|
||||
|
||||
---
|
||||
|
||||
## 9) Risks and mitigations
|
||||
|
||||
* **Policy sprawl**: too many similar policies.
|
||||
|
||||
* Mitigation: templates, policy inheritance in v1.1, tagging, ownership metadata.
|
||||
* **Non‑determinism creep**: someone sneaks wall‑clock or network into evaluation.
|
||||
|
||||
* Mitigation: determinism guard, static analyzer, and CI replay check.
|
||||
* **Join miss‑rate**: weak linksets cause under‑matching.
|
||||
|
||||
* Mitigation: linkset strengthening in ingestion, PURL equivalence tables, monitoring for “zero‑hit” rates.
|
||||
* **Approval bottlenecks**: blocked rollouts.
|
||||
|
||||
* Mitigation: RBAC with delegated approvers and time‑boxed SLAs.
|
||||
|
||||
---
|
||||
|
||||
## 10) Test plan
|
||||
|
||||
* **Unit**: parser, compiler, evaluator; conflict resolution; precedence.
|
||||
* **Property**: random policies over synthetic inputs; ensure no panics and stable outputs.
|
||||
* **Golden**: fixed SBOM + curated advisories/VEX → expected findings; compare every run.
|
||||
* **Performance**: large SBOMs with heavy rule sets; assert run times and memory ceilings.
|
||||
* **Integration**: end‑to‑end simulate → approve → run → diff; verify write protections.
|
||||
* **Chaos**: inject malformed VEX, missing advisories; ensure graceful degradation and clear errors.
|
||||
|
||||
---
|
||||
|
||||
## 11) Developer checklists
|
||||
|
||||
**Definition of Ready**
|
||||
|
||||
* Policy grammar finalized; examples prepared.
|
||||
* Linkset join queries benchmarked.
|
||||
* Owner and approvers assigned.
|
||||
|
||||
**Definition of Done**
|
||||
|
||||
* All APIs live with RBAC.
|
||||
* CLI and Console features shipped.
|
||||
* Determinism and golden tests green.
|
||||
* Observability dashboards deployed.
|
||||
* Docs in section 7 merged.
|
||||
* Two real org policies migrated and in production.
|
||||
|
||||
---
|
||||
|
||||
## 12) Glossary
|
||||
|
||||
* **Policy**: versioned rule set controlling status and severity.
|
||||
* **DSL**: domain‑specific language used to express rules.
|
||||
* **Run**: a single evaluation execution with defined inputs and outputs.
|
||||
* **Simulation**: a run that doesn’t write findings; returns diffs.
|
||||
* **Materialization**: persisted effective findings for fast queries.
|
||||
* **Determinism**: same inputs + same policy = same outputs. Always.
|
||||
|
||||
---
|
||||
|
||||
### Final imposed reminder
|
||||
|
||||
**Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,409 +0,0 @@
|
||||
Here’s Epic 4 in the same paste‑into‑repo, implementation‑ready style as the prior epics. It’s exhaustive, formal, and slots directly into the existing AOC model, Policy Engine, and Console.
|
||||
|
||||
---
|
||||
|
||||
# Epic 4: Policy Studio (author, version, simulate)
|
||||
|
||||
> Short name: **Policy Studio**
|
||||
> Services touched: **Policy Engine**, **Policy Registry** (new), **Web API Gateway**, **Authority** (authN/Z), **Scheduler/Workers**, **SBOM Service**, **Conseiller (Feedser)**, **Excitator (Vexer)**, **Telemetry**
|
||||
> Surfaces: **Console (Web UI)** feature module, **CLI**, **CI hooks**
|
||||
> Deliverables: Authoring workspace, policy versioning, static checks, simulation at scale, reviews/approvals, signing/publishing, promotion
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
**Policy Studio** is the end‑to‑end system for creating, evolving, and safely rolling out the rules that turn AOC facts (SBOM, advisories, VEX) into **effective findings**. It provides:
|
||||
|
||||
* A **workspace** where authors write policies in the DSL (Epic 2), with linting, autocompletion, snippets, and templates.
|
||||
* A **Policy Registry** that stores immutable versions, compiled artifacts, metadata, provenance, and signatures.
|
||||
* **Simulation** at two levels: quick local samples and large batch simulations across real SBOM inventories with full deltas.
|
||||
* A **review/approval** workflow with comments, diffs, required approvers, and promotion to environments (dev/test/prod).
|
||||
* **Publishing** semantics: signed, immutable versions bound to tenants; rollback and deprecation.
|
||||
* Tight integration with **Explain** traces so any change can show exactly which rules fired and why outcomes shifted.
|
||||
|
||||
The Studio respects **AOC enforcement**: policies never edit or merge facts. They only interpret facts and produce determinations consistent with precedence rules defined in the DSL.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* Policy errors are expensive. Authors need safe sandboxes, deterministic builds, and evidence before rollout.
|
||||
* Auditors require immutability, provenance, and reproducibility from “source policy” to “effective finding.”
|
||||
* Teams want gradual rollout: simulate, canary, promote, observe, rollback.
|
||||
* Policy knowledge should be modular, reusable, and testable, not tribal.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Domain model
|
||||
|
||||
* **PolicyPackage**: `{name, tenant, description, owners[], tags[], created_at}`
|
||||
* **PolicyVersion** (immutable): `{package, semver, source_sha, compiled_sha, status: draft|review|approved|published|deprecated|archived, created_by, created_at, signatures[], changelog, metadata{}}`
|
||||
* **Workspace**: mutable working area for authors; holds unversioned edits until compiled.
|
||||
* **CompilationArtifact**: `{policy_version, compiler_version, diagnostics[], rule_index[], symbol_table}`
|
||||
* **SimulationSpec**: `{policy_version|workspace, sbom_selector, time_window?, environment?, sample_size?, severity_floor?, includes{advisories?, vex?}}`
|
||||
* **SimulationRun**: `{run_id, spec, started_at, finished_at, result{counts_before, counts_after, top_deltas[], by_rule_hit[], sample_explains[]}}`
|
||||
* **Review**: `{policy_version, required_approvers[], votes[], comments[], files_changed[], diffs[]}`
|
||||
* **Promotion**: `{policy_version, environment: dev|test|prod, promoted_by, promoted_at, rollout_strategy: All|Percent|TenantSubset}`
|
||||
* **Attestation**: OIDC‑backed signature metadata binding `source_sha` and `compiled_sha` to an actor and time.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.2 Authoring workflow
|
||||
|
||||
1. **Create** a workspace from a template (e.g., “Default Risk Model,” “License Tilted,” “Cloud‑Native SBOM”).
|
||||
2. **Edit** in the Studio: Monaco editor with DSL grammar, intelligent completion for predicates, policies, attributes.
|
||||
3. **Lint & compile** locally: semantic checks, forbidden rules detection, policy size limits, constant‑folding.
|
||||
4. **Unit tests**: run policy test cases on bundled fixtures and golden expectations.
|
||||
5. **Quick simulate** on selected SBOMs (10–50 items) to preview counts, examples, and rule heatmap.
|
||||
6. **Propose version**: bump semver, enter changelog; create a **PolicyVersion** in `review` with compiled artifacts.
|
||||
7. **Review & approval**: side‑by‑side diff, comments, required approvers enforced by RBAC.
|
||||
8. **Batch simulation**: run at scale across tenant inventory; produce deltas, sample explainer evidence.
|
||||
9. **Publish**: sign and move to `published`; optional **Promotion** to target environment(s).
|
||||
10. **Run** evaluation with the selected policy version; verify outcomes; optionally promote to default.
|
||||
11. **Rollback**: select an older version; promotion updates references without mutating older versions.
|
||||
|
||||
### 3.3 Editing experience (Console)
|
||||
|
||||
* **Three‑pane layout**: file tree, editor, diagnostics/simulation.
|
||||
* **Features**: autocomplete from symbol table, in‑editor docs on hover, go‑to definition, rule references, rename symbols across files, snippet library, policy templates.
|
||||
* **Validations**:
|
||||
|
||||
* AOC guardrails: no edit/merge actions on source facts, only interpretation.
|
||||
* Precedence correctness: if rules conflict, studio shows explicit order and effective winner.
|
||||
* Severity floor and normalization mapping validated against registry configuration.
|
||||
* **Diagnostics panel**: errors, warnings, performance hints (e.g., “predicate X loads N advisories per component; consider indexing”).
|
||||
* **Rule heatmap**: during simulation, bar chart of rule firings and the objects they impact.
|
||||
* **Explain sampler**: click any delta bucket to open a sampled finding with full trace.
|
||||
|
||||
### 3.4 Simulation
|
||||
|
||||
* **Quick Sim**: synchronous; runs in browser‑orchestrated job against API, constrained by `sample_size`.
|
||||
* **Batch Sim**: asynchronous run in workers:
|
||||
|
||||
* Input selection: all SBOMs, labels, artifact regex, last N ingests, or a curated set.
|
||||
* Outputs: counts by severity before/after, by status, top deltas by component and advisory, rule heatmap, top K affected artifacts.
|
||||
* Evidence: NDJSON of sampled findings with traces; CSV summary; signed result manifest.
|
||||
* Guardrails: cannot publish if batch sim drift > configurable threshold without an override justification.
|
||||
|
||||
### 3.5 Versioning & promotion
|
||||
|
||||
* Semver enforced: `major` implies compatibility break (e.g., precedence changes), `minor` adds rules, `patch` fixes.
|
||||
* **Immutable**: after `published`, the version cannot change; deprecate instead.
|
||||
* **Environment bindings**: dev/test/prod mapping per tenant; default policy per environment.
|
||||
* **Canary**: promote to a subset of tenants or artifacts; the Runs page displays A/B comparisons.
|
||||
|
||||
### 3.6 Review & approval
|
||||
|
||||
* Require N approvers by role; self‑approval optionally prohibited.
|
||||
* Line and file comments; overall decision with justification.
|
||||
* Review snapshot captures: diffs, diagnostics, simulation summary.
|
||||
* Webhooks to notify external systems of review events.
|
||||
|
||||
### 3.7 RBAC (Authority)
|
||||
|
||||
Roles per tenant:
|
||||
|
||||
* **Policy Author**: create/edit workspace, quick sim, propose versions.
|
||||
* **Policy Reviewer**: comment, request changes, approve/reject.
|
||||
* **Policy Approver**: final approve, publish.
|
||||
* **Policy Operator**: promote, rollback, schedule runs.
|
||||
* **Read‑only Auditor**: view everything, download evidence.
|
||||
|
||||
All actions server‑checked; UI only hides affordances.
|
||||
|
||||
### 3.8 CLI + CI integration
|
||||
|
||||
CLI verbs (examples):
|
||||
|
||||
```
|
||||
stella policy init --template default
|
||||
stella policy lint
|
||||
stella policy compile
|
||||
stella policy test --golden ./tests
|
||||
stella policy simulate --sboms label:prod --sample 1000
|
||||
stella policy version bump --level minor --changelog "Normalize GHSA CVSS"
|
||||
stella policy submit --reviewers alice@example.com,bob@example.com
|
||||
stella policy approve --version 1.3.0
|
||||
stella policy publish --version 1.3.0 --sign
|
||||
stella policy promote --version 1.3.0 --env test --percent 20
|
||||
stella policy rollback --env prod --to 1.2.1
|
||||
```
|
||||
|
||||
CI usage:
|
||||
|
||||
* Lint, compile, and run unit tests on PRs that modify `/policies/**`.
|
||||
* Optionally trigger **Batch Sim** against a staging inventory and post a Markdown report to the PR.
|
||||
* Block merge if diagnostics include errors or drift exceeds thresholds.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.9 APIs (representative)
|
||||
|
||||
* `POST /policies/workspaces` create from template
|
||||
* `PUT /policies/workspaces/{id}/files` edit source files
|
||||
* `POST /policies/workspaces/{id}/compile` get diagnostics + compiled artifact
|
||||
* `POST /policies/workspaces/{id}/simulate` quick sim
|
||||
* `POST /policies/versions` create version from workspace with semver + changelog
|
||||
* `GET /policies/versions/{id}` fetch version + diagnostics + sim summary
|
||||
* `POST /policies/versions/{id}/reviews` open review
|
||||
* `POST /policies/versions/{id}/approve` record approval
|
||||
* `POST /policies/versions/{id}/publish` sign + publish
|
||||
* `POST /policies/versions/{id}/promote` bind to env/canary
|
||||
* `POST /policies/versions/{id}/simulate-batch` start batch sim (async)
|
||||
* `GET /policies/simulations/{run_id}` get sim results and artifacts
|
||||
* `GET /policies/registry` list packages/versions, status and bindings
|
||||
|
||||
All calls require tenant scoping and RBAC.
|
||||
|
||||
### 3.10 Storage & data
|
||||
|
||||
* **Policy Registry DB** (MongoDB): packages, versions, workspaces, metadata.
|
||||
* **Object storage**: source bundles, compiled artifacts, simulation result bundles, evidence.
|
||||
* **Indexing**: compound indexes by `{tenant, package}`, `{tenant, status}`, `{tenant, environment}`.
|
||||
* **Retention**: configurable retention for workspaces and simulation artifacts; versions never deleted, only archived.
|
||||
|
||||
### 3.11 Evidence & provenance
|
||||
|
||||
* Every published version has:
|
||||
|
||||
* `source_sha` (content digest of the policy source bundle)
|
||||
* `compiled_sha` (digest of compiled artifact)
|
||||
* Attestation: signed envelope binding digests to an identity, time, and tenant.
|
||||
* Links to the exact compiler version, inputs, and environment.
|
||||
|
||||
### 3.12 Observability
|
||||
|
||||
* Metrics: compile time, diagnostics rate, simulation queue depth, delta magnitude distribution, approval latencies.
|
||||
* Logs: structured events for lifecycle transitions.
|
||||
* Traces: long simulations emit span per shard.
|
||||
|
||||
### 3.13 Performance & scale
|
||||
|
||||
* Compilation should complete under 3 seconds for typical policies; warn at 10s.
|
||||
* Batch sim uses workers with partitioning by SBOM id; results reduced by the API.
|
||||
* Memory guardrails on rule execution; deny policies that exceed configured complexity limits.
|
||||
|
||||
### 3.14 Security
|
||||
|
||||
* OIDC‑backed signing and attestation.
|
||||
* Policy sources are scanned on upload for secrets; blocked if found.
|
||||
* Strict CSP in Studio pages; tokens stored in memory, not localStorage.
|
||||
* Tenant isolation in buckets and DB collections.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **Policy Registry (new microservice)**
|
||||
|
||||
* REST API and background workers for batch simulation orchestration.
|
||||
* Stores workspaces, versions, metadata, bindings, reviews.
|
||||
* Generates signed attestations at publish time.
|
||||
* Coordinates with **Policy Engine** for compile/simulate invocations.
|
||||
|
||||
* **Policy Engine (existing)**
|
||||
|
||||
* Expose compile and simulate endpoints with deterministic outputs.
|
||||
* Provide rule coverage, symbol table, and explain traces for samples.
|
||||
|
||||
* **Web API Gateway**
|
||||
|
||||
* Routes requests; injects tenant context; enforces RBAC.
|
||||
|
||||
### 4.2 Console (Web UI) feature module
|
||||
|
||||
* `packages/features/policies` (shared with Epic 3):
|
||||
|
||||
* **Studio** routes: `/policies/studio`, `/policies/:id/versions/:v/edit`, `/simulate`, `/review`.
|
||||
* Monaco editor wrapper for DSL with hover docs, autocomplete.
|
||||
* Diff viewer, diagnostics, heatmap, explain sampler, review UI.
|
||||
|
||||
### 4.3 CLI
|
||||
|
||||
* New commands under `stella policy *`; typed client generated from OpenAPI.
|
||||
* Outputs machine‑readable JSON and pretty tables.
|
||||
|
||||
### 4.4 Workers
|
||||
|
||||
* **Simulation workers**: pull shards of SBOMs, run policy, emit partials, reduce into result bundle.
|
||||
* **Notification worker**: sends webhooks on review, approval, publish, promote.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. **`/docs/policy/studio-overview.md`**
|
||||
|
||||
* Concepts, roles, lifecycle, glossary.
|
||||
2. **`/docs/policy/authoring.md`**
|
||||
|
||||
* Workspace, templates, snippets, lint rules, best practices.
|
||||
3. **`/docs/policy/versioning-and-publishing.md`**
|
||||
|
||||
* Semver, immutability, deprecation, rollback, attestations.
|
||||
4. **`/docs/policy/simulation.md`**
|
||||
|
||||
* Quick vs batch sim, selection strategies, thresholds, evidence artifacts.
|
||||
5. **`/docs/policy/review-and-approval.md`**
|
||||
|
||||
* Required approvers, comments, webhooks, audit trail.
|
||||
6. **`/docs/policy/promotion.md`**
|
||||
|
||||
* Environments, canary, default policy binding, rollback.
|
||||
7. **`/docs/policy/cli.md`**
|
||||
|
||||
* Command reference with examples and JSON outputs.
|
||||
8. **`/docs/policy/api.md`**
|
||||
|
||||
* REST endpoints, request/response schemas, error codes.
|
||||
9. **`/docs/security/policy-attestations.md`**
|
||||
|
||||
* Signatures, digests, verifier steps.
|
||||
10. **`/docs/architecture/policy-registry.md`**
|
||||
|
||||
* Service design, schemas, queues, failure modes.
|
||||
11. **`/docs/observability/policy-telemetry.md`**
|
||||
|
||||
* Metrics, logs, tracing, dashboards.
|
||||
12. **`/docs/runbooks/policy-incident.md`**
|
||||
|
||||
* Rolling back a bad policy, freezing publishes, forensic steps.
|
||||
13. **`/docs/examples/policy-templates.md`**
|
||||
|
||||
* Ready‑made templates and snippet catalog.
|
||||
14. **`/docs/aoc/aoc-guardrails.md`**
|
||||
|
||||
* How Studio enforces AOC in authoring and review.
|
||||
|
||||
Each doc ends with a “Compliance checklist.”
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Tasks
|
||||
|
||||
### 6.1 Backend: Policy Registry
|
||||
|
||||
* [ ] Define OpenAPI spec for Registry (workspaces, versions, reviews, sim).
|
||||
* [ ] Implement workspace storage and file CRUD.
|
||||
* [ ] Integrate with Policy Engine compile endpoint; return diagnostics, symbol table.
|
||||
* [ ] Implement quick simulation with request limits.
|
||||
* [ ] Implement batch simulation orchestration: enqueue shards, collect results, reduce deltas, store artifacts.
|
||||
* [ ] Implement review model: comments, required approvers, decisions.
|
||||
* [ ] Implement publish: sign, persist attestation, set status=published.
|
||||
* [ ] Implement promotion bindings per tenant/environment; canary subsets.
|
||||
* [ ] RBAC checks for all endpoints.
|
||||
* [ ] Unit/integration tests; load tests for batch sim.
|
||||
|
||||
### 6.2 Policy Engine enhancements
|
||||
|
||||
* [ ] Return rule coverage and firing counts with compile/simulate.
|
||||
* [ ] Return symbol table and inline docs for editor autocomplete.
|
||||
* [ ] Expose deterministic Explain traces for sampled findings.
|
||||
* [ ] Enforce complexity/time limits and report breaches.
|
||||
|
||||
### 6.3 Console (Web UI)
|
||||
|
||||
* [ ] Build Studio editor wrapper with Monaco + DSL language server hooks.
|
||||
* [ ] Implement file tree, snippets, templates, hotkeys, search/replace.
|
||||
* [ ] Diagnostics panel with jump‑to‑line, quick fixes.
|
||||
* [ ] Simulation panel: quick sim UI, charts, heatmap, sample explains.
|
||||
* [ ] Review UI: diff, comments, approvals, status badges.
|
||||
* [ ] Publish & Promote flows with confirmation and post‑actions.
|
||||
* [ ] Batch sim results pages with export buttons.
|
||||
* [ ] Accessibility audits and keyboard‑only authoring flow.
|
||||
|
||||
### 6.4 CLI
|
||||
|
||||
* [ ] Implement commands listed in 3.8 with rich help and examples.
|
||||
* [ ] Add `--json` flag for machine consumption; emit stable schemas.
|
||||
* [ ] Exit codes aligned with CI usage (lint errors → non‑zero).
|
||||
|
||||
### 6.5 CI/CD & Security
|
||||
|
||||
* [ ] Add CI job that runs `stella policy lint/compile/test` on PRs.
|
||||
* [ ] Optional job that triggers batch sim against staging inventory; post summary to PR.
|
||||
* [ ] Policy source secret scanning; block on findings.
|
||||
* [ ] Signing keys configuration; verify pipeline for attestation on publish.
|
||||
|
||||
### 6.6 Docs
|
||||
|
||||
* [ ] Write all docs in section 5 with screenshots and CLI transcripts.
|
||||
* [ ] Add cookbook examples and templates in `/docs/examples/policy-templates.md`.
|
||||
* [ ] Wire contextual Help links from Studio to relevant docs.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Authors can create, edit, lint, compile policies with inline diagnostics and autocomplete.
|
||||
* Quick simulation produces counts, rule heatmap, and sample explains within UI.
|
||||
* Batch simulation scales across large SBOM sets, producing deltas and downloadable evidence.
|
||||
* Review requires configured approvers; comments and diffs are preserved.
|
||||
* Publish generates immutable, signed versions with attestations.
|
||||
* Promotion binds versions to environments and supports canary and rollback.
|
||||
* CLI supports full lifecycle and is usable in CI.
|
||||
* All actions are tenant‑scoped, RBAC‑enforced, and logged.
|
||||
* AOC guardrails prevent any mutation of raw facts.
|
||||
* Documentation shipped and linked contextually from the Studio.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks & mitigations
|
||||
|
||||
* **Policy complexity causes timeouts** → compile‑time complexity scoring, execution limits, early diagnostics.
|
||||
* **Simulation cost at scale** → sharding and streaming reducers; sampling; configurable quotas.
|
||||
* **RBAC misconfiguration** → server‑enforced checks, defense‑in‑depth tests, deny‑by‑default.
|
||||
* **Attestation key management** → OIDC‑backed signatures; auditable verifier tool; time‑boxed credentials.
|
||||
* **Editor usability** → language server with accurate completions; docs on hover; snippet library.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit**: compiler adapters, registry models, reviewers workflow, CLI options.
|
||||
* **Integration**: compile→simulate→publish→promote on seeded data.
|
||||
* **E2E**: Playwright flows for author→review→batch sim→publish→promote→rollback.
|
||||
* **Performance**: load test batch simulation with 100k components spread across SBOMs.
|
||||
* **Security**: RBAC matrix tests; secret scanning; signing and verification.
|
||||
* **Determinism**: same inputs produce identical `compiled_sha` and simulation summaries.
|
||||
|
||||
---
|
||||
|
||||
## 10) Feature flags
|
||||
|
||||
* `policy.studio` (enables editor and quick sim)
|
||||
* `policy.batch-sim`
|
||||
* `policy.canary-promotion`
|
||||
* `policy.signature-required` (enforce signing on publish)
|
||||
|
||||
Flags documented in `/docs/observability/policy-telemetry.md`.
|
||||
|
||||
---
|
||||
|
||||
## 11) Non‑goals (this epic)
|
||||
|
||||
* Building a general IDE for arbitrary languages; the editor is purpose‑built for the DSL.
|
||||
* Auto‑generated policies from AI without human approval.
|
||||
* Cross‑tenant policies; all policies are tenant‑scoped.
|
||||
|
||||
---
|
||||
|
||||
## 12) Philosophy
|
||||
|
||||
* **Safety first**: it’s cheaper to prevent a bad policy than to fix its fallout.
|
||||
* **Determinism**: same inputs, same outputs, verifiably.
|
||||
* **Immutability**: versions and evidence are forever; we deprecate, not mutate.
|
||||
* **Transparency**: every change is explainable with traces and proofs.
|
||||
* **Reusability**: templates, snippets, and tests turn policy from art into engineering.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,431 +0,0 @@
|
||||
Here’s Epic 5 in the same paste‑into‑repo, implementation‑ready format as the prior epics. It’s exhaustive, formal, and designed to slot into AOC, Policy Engine, Conseiller/Excitator, and the Console.
|
||||
|
||||
---
|
||||
|
||||
# Epic 5: SBOM Graph Explorer
|
||||
|
||||
> Short name: **Graph Explorer**
|
||||
> Services touched: **SBOM Service**, **Graph Indexer** (new), **Graph API** (new), **Policy Engine**, **Conseiller (Feedser)**, **Excitator (Vexer)**, **Web API Gateway**, **Authority** (authN/Z), **Workers/Scheduler**, **Telemetry**
|
||||
> Surfaces: **Console (Web UI)** graph module, **CLI**, **Exports**
|
||||
> Deliverables: Interactive graph UI with semantic zoom, saved queries, policy/VEX/advisory overlays, diff views, impact analysis, exports
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
**SBOM Graph Explorer** is the interactive, tenant‑scoped view of all supply‑chain relationships the platform knows about, rendered as a navigable graph. It connects:
|
||||
|
||||
* **Artifacts** (applications, images, libs), **Packages/Versions**, **Files/Paths**, **Licenses**, **Advisories** (from Conseiller), **VEX statements** (from Excitator), **Provenance** (builds, sources), and **Policies** (overlays of determinations)
|
||||
* **Edges** like `depends_on`, `contains`, `built_from`, `declared_in`, `affected_by`, `vex_exempts`, `governs_with`
|
||||
* **Time/version** dimension: multiple SBOM snapshots with diffs
|
||||
|
||||
It’s built for investigation and review: find where a vulnerable package enters; see which apps are impacted; understand why a finding exists; simulate a policy version and see the delta. The explorer observes **AOC enforcement**: it never mutates facts; it aggregates and visualizes them. Only the Policy Engine may classify, and classification is displayed as overlays.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* SBOMs are graphs. Tables flatten what matters and hide transitive risk.
|
||||
* Engineers, security, and auditors need impact answers quickly: “What pulls in `log4j:2.17` and where is it at runtime?”
|
||||
* Policy/VEX/advisory interactions are nuanced. A visual overlay makes precedence and outcomes obvious.
|
||||
* Review is collaborative; you need saved queries, deep links, exports, and consistent evidence.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Domain model
|
||||
|
||||
**Nodes** (typed, versioned, tenant‑scoped):
|
||||
|
||||
* `Artifact`: application, service, container image, library, module
|
||||
* `Package`: name + ecosystem (purl), `PackageVersion` with resolved version
|
||||
* `File`: path within artifact or image layer
|
||||
* `License`: SPDX id
|
||||
* `Advisory`: normalized advisory id (GHSA, CVE, vendor), source = Conseiller
|
||||
* `VEX`: statement with product context, status, justification, source = Excitator
|
||||
* `SBOM`: ingestion unit; includes metadata (tool, sha, build info)
|
||||
* `PolicyDetermination`: materialized view of Policy Engine results (read‑only overlay)
|
||||
* `Build`: provenance, commit, workflow run
|
||||
* `Source`: repo, tag, commit
|
||||
|
||||
**Edges** (directed):
|
||||
|
||||
* `declared_in` (PackageVersion → SBOM)
|
||||
* `contains` (Artifact → PackageVersion | File)
|
||||
* `depends_on` (PackageVersion → PackageVersion) with scope attr (prod|dev|test|optional)
|
||||
* `built_from` (Artifact → Build), `provenance_of` (Build → Source)
|
||||
* `affected_by` (PackageVersion → Advisory) with range semantics
|
||||
* `vex_exempts` (Advisory ↔ VEX) scoped by product/component
|
||||
* `licensed_under` (Artifact|PackageVersion → License)
|
||||
* `governs_with` (Artifact|PackageVersion → PolicyDetermination)
|
||||
* `derived_from` (SBOM → SBOM) for superseding snapshots
|
||||
|
||||
**Identity & versioning**
|
||||
|
||||
* Every node has a stable key: `{tenant}:{type}:{natural_id}` (e.g., purl for packages, digest for images).
|
||||
* SBOM snapshots are immutable; edges carry `valid_from`/`valid_to` for time travel and diffing.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.2 User capabilities (end‑to‑end)
|
||||
|
||||
* **Search & Navigate**: global search (purls, CVEs, repos, licenses), keyboard nav, breadcrumbs, semantic zoom.
|
||||
* **Lenses**: toggle views (Security, License, Provenance, Runtime vs Dev, Policy effect).
|
||||
* **Overlays**:
|
||||
|
||||
* **Advisory overlay**: show affected nodes/edges with source, severity, ranges.
|
||||
* **VEX overlay**: show suppressions/justifications; collapse exempted paths.
|
||||
* **Policy overlay**: choose a policy version; nodes/edges reflect determinations (severity, status) with explain sampling.
|
||||
* **Impact analysis**: pick a vulnerable node; highlight upstream/downstream dependents, scope filters, shortest/all paths with constraints.
|
||||
* **Diff view**: compare SBOM A vs B; show added/removed nodes/edges, changed versions, changed determinations.
|
||||
* **Saved queries**: visual builder + JSON query; shareable permalinks scoped by tenant and environment.
|
||||
* **Exports**: GraphML, CSV edge list, NDJSON of findings, PNG/SVG snapshot.
|
||||
* **Evidence details**: side panel with raw facts, advisory links, VEX statements, policy explain trace, provenance.
|
||||
* **Accessibility**: tab‑navigable, high‑contrast, screen‑reader labels for nodes and sidebars.
|
||||
|
||||
### 3.3 Query model
|
||||
|
||||
* **Visual builder** for common queries:
|
||||
|
||||
* “Show all paths from Artifact X to Advisory Y up to depth 6.”
|
||||
* “All runtime dependencies with license = GPL‑3.0.”
|
||||
* “All artifacts affected by GHSA‑… with no applicable VEX.”
|
||||
* “Which SBOMs introduced/removed `openssl` between build 120 and 130?”
|
||||
* **JSON query** (internal, POST body) with:
|
||||
|
||||
* `start`: list of node selectors (type + id or attributes)
|
||||
* `expand`: edge types and depth, direction, scope filters
|
||||
* `where`: predicates on node/edge attributes
|
||||
* `overlay`: policy version id, advisory sources, VEX filters
|
||||
* `limit`: nodes, edges, timebox, cost budget
|
||||
* **Cost control**: server estimates cost, denies or pages results; UI streams partial graph tiles.
|
||||
|
||||
### 3.4 UI architecture (Console)
|
||||
|
||||
* **Canvas**: WebGL renderer with level‑of‑detail, edge bundling, and label culling; deterministic layout when possible (seeded).
|
||||
* **Semantic zoom**:
|
||||
|
||||
* Far: clusters by artifact/repo/ecosystem, color by lens
|
||||
* Mid: package groups, advisory badges, license swatches
|
||||
* Near: concrete versions, direct edges, inline badges for policy determinations
|
||||
* **Panels**:
|
||||
|
||||
* Left: search, filters, lens selector, saved queries
|
||||
* Right: details, explain trace, evidence tabs (Advisory/VEX/Policy/Provenance)
|
||||
* Bottom: query expression, diagnostics, performance/stream status
|
||||
* **Diff mode**: split or overlay, color legend (add/remove/changed), filter by node type.
|
||||
* **Deep links**: URL encodes query + viewport; shareable respecting RBAC.
|
||||
* **Keyboard**: space drag, +/- zoom, F to focus, G to expand neighbors, P to show paths.
|
||||
|
||||
### 3.5 Back‑end architecture
|
||||
|
||||
**Graph Indexer (new)**
|
||||
|
||||
* Consumes SBOM ingests, Conseiller advisories, Excitator VEX statements, Policy Engine determinations (read‑only).
|
||||
* Projects facts into a **property graph** persisted in:
|
||||
|
||||
* Primary: document store + adjacency sets (e.g., Mongo collections + compressed adjacency lists)
|
||||
* Optional driver for graph DB backends if needed (pluggable)
|
||||
* Maintains materialized aggregates: degree, critical paths cache, affected artifact counts, license distribution.
|
||||
* Emits **graph snapshots** per SBOM with lineage to original ingestion.
|
||||
|
||||
**Graph API (new)**
|
||||
|
||||
* Endpoints for search, neighbor expansion, path queries, diffs, overlays, exports.
|
||||
* Streaming responses for large graphs (chunked NDJSON tiles).
|
||||
* Cost accounting + quotas per tenant.
|
||||
|
||||
**Workers**
|
||||
|
||||
* **Centrality & clustering** precompute on idle: betweenness approximations, connected components, Louvain clusters.
|
||||
* **Diff compute** on new SBOM ingestion pairs (previous vs current).
|
||||
* **Overlay materialization** cache for popular policy versions.
|
||||
|
||||
**Policy Engine integration**
|
||||
|
||||
* Graph API requests can specify a policy version.
|
||||
* For sampled nodes, the API fetches explain traces; for counts, uses precomputed overlay materializations where available.
|
||||
|
||||
**AOC enforcement**
|
||||
|
||||
* Graph Indexer never merges or edits advisories/VEX; it links them and exposes overlays that the Policy Engine evaluates.
|
||||
* Conseiller and Excitator remain authoritative sources; severities come from Policy‑governed normalization.
|
||||
|
||||
### 3.6 APIs (representative)
|
||||
|
||||
* `GET /graph/search?q=...&type=package|artifact|advisory|license`
|
||||
* `POST /graph/query` ⇒ stream tiles `{nodes[], edges[], stats, cursor}`
|
||||
* `POST /graph/paths` body: `{from, to, depth<=6, constraints{scope, runtime_only}}`
|
||||
* `POST /graph/diff` body: `{sbom_a, sbom_b, filters}`
|
||||
* `GET /graph/snapshots/{sbom_id}` ⇒ graph metadata, counts, top advisories
|
||||
* `POST /graph/export` body: `{format: graphml|csv|ndjson|png|svg, query|snapshot}`
|
||||
* `GET /graph/saved` / `POST /graph/saved` save and list tenant queries
|
||||
* `GET /graph/overlays/policy/{version_id}` ⇒ summary stats for caching
|
||||
|
||||
All endpoints tenant‑scoped, RBAC‑checked. Timeouts and pagination by server. Errors return structured diagnostics.
|
||||
|
||||
### 3.7 CLI
|
||||
|
||||
```
|
||||
stella sbom graph search "purl:pkg:maven/org.apache.logging.log4j/log4j-core"
|
||||
stella sbom graph query --file ./query.json --export graphml > graph.graphml
|
||||
stella sbom graph impacted --advisory GHSA-xxxx --runtime-only --limit 100
|
||||
stella sbom graph paths --from artifact:service-a --to advisory:GHSA-xxxx --depth 5 --policy 1.3.0
|
||||
stella sbom graph diff --sbom-a 2025-03-15T10:00Z --sbom-b 2025-03-22T10:00Z --export csv > diff.csv
|
||||
stella sbom graph save --name "openssl-runtime" --file ./query.json
|
||||
```
|
||||
|
||||
Exit codes: 0 ok, 2 query validation error, 3 over‑budget, 4 not found, 5 RBAC denied.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.8 Performance & scale
|
||||
|
||||
* **Progressive loading**: server pages tiles by BFS frontier; client renders incrementally.
|
||||
* **Viewport culling**: only visible nodes/edges in canvas; offscreen demoted to aggregates.
|
||||
* **Level‑of‑detail**: simplified glyphs and collapsed clusters at distance.
|
||||
* **Query budgets**: per‑tenant rate + node/edge caps; interactive paths limited to depth ≤ 6.
|
||||
* **Caching**: hot queries memoized per tenant + overlay version; diffs precomputed for consecutive SBOMs.
|
||||
|
||||
### 3.9 Security
|
||||
|
||||
* Multi‑tenant isolation at storage and API layers.
|
||||
* RBAC roles:
|
||||
|
||||
* **Viewer**: browse graphs, saved queries
|
||||
* **Investigator**: run queries, export data
|
||||
* **Operator**: configure budgets, purge caches
|
||||
* **Auditor**: download evidence bundles
|
||||
* Input validation for query JSON; deny disallowed edge traversals; strict CSP in web app.
|
||||
|
||||
### 3.10 Observability
|
||||
|
||||
* Metrics: tile latency, nodes/edges per tile, cache hit rate, query denials, memory pressure.
|
||||
* Logs: structured, include query hash, cost, truncation flags.
|
||||
* Traces: server spans per stage (parse, plan, fetch, overlay, stream).
|
||||
|
||||
### 3.11 Accessibility & UX guarantees
|
||||
|
||||
* Keyboard complete, ARIA roles for graph and panels, high‑contrast theme.
|
||||
* Deterministic layout on reload for shareable investigations.
|
||||
|
||||
### 3.12 Data retention
|
||||
|
||||
* Graph nodes derived from SBOMs share retention with SBOM artifacts; overlays are ephemeral caches.
|
||||
* Saved queries retained until deleted; references to missing objects show warnings.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **Graph Indexer (new microservice)**
|
||||
|
||||
* Subscribes to SBOM ingest events, Conseiller advisory updates, Excitator VEX updates, Policy overlay materializations.
|
||||
* Builds adjacency lists and node documents; computes aggregates and clusters.
|
||||
|
||||
* **Graph API (new microservice)**
|
||||
|
||||
* Validates and executes queries; streams tiles; composes overlays; serves diffs and exports.
|
||||
* Integrates with Policy Engine for explain sample retrieval.
|
||||
|
||||
* **SBOM Service (existing)**
|
||||
|
||||
* Emits ingestion events with SBOM ids and lineage; exposes SBOM metadata to Graph API.
|
||||
|
||||
* **Web API Gateway**
|
||||
|
||||
* Routes `/graph/*`, injects tenant context, enforces RBAC.
|
||||
|
||||
### 4.2 Console (Web UI) feature module
|
||||
|
||||
* `packages/features/graph-explorer`
|
||||
|
||||
* Canvas renderer (WebGL), panels, query builder, diff mode, overlays, exports.
|
||||
* Deep‑link router and viewport state serializer.
|
||||
|
||||
### 4.3 Workers
|
||||
|
||||
* Centrality/clustering worker, diff worker, overlay materialization worker.
|
||||
* Schedules on low‑traffic windows; backpressure aware.
|
||||
|
||||
### 4.4 Data model (storage)
|
||||
|
||||
* Collections:
|
||||
|
||||
* `graph_nodes`: `{_id, tenant, type, natural_id, attrs, degree, created_at, updated_at}`
|
||||
* `graph_edges`: `{_id, tenant, from_id, to_id, type, attrs, valid_from, valid_to}`
|
||||
* `graph_snapshots`: per‑SBOM node/edge references
|
||||
* `graph_saved_queries`: `{_id, tenant, name, query_json, created_by}`
|
||||
* `graph_overlays_cache`: keyed by `{tenant, policy_version, hash(query)}`
|
||||
* Indexes: compound on `{tenant, type, natural_id}`, `{tenant, from_id}`, `{tenant, to_id}`, time bounds.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. **`/docs/sbom/graph-explorer-overview.md`**
|
||||
|
||||
* Concepts, node/edge taxonomy, lenses, overlays, roles, limitations.
|
||||
2. **`/docs/sbom/graph-using-the-console.md`**
|
||||
|
||||
* Walkthroughs: search, navigate, impact, diff, export; screenshots and keyboard cheatsheet.
|
||||
3. **`/docs/sbom/graph-query-language.md`**
|
||||
|
||||
* JSON schema, examples, constraints, cost/budget rules.
|
||||
4. **`/docs/sbom/graph-api.md`**
|
||||
|
||||
* REST endpoints, request/response examples, streaming and pagination.
|
||||
5. **`/docs/sbom/graph-cli.md`**
|
||||
|
||||
* CLI command reference and example pipelines.
|
||||
6. **`/docs/policy/graph-overlays.md`**
|
||||
|
||||
* How policy versions render in Graph; explain sampling; AOC guardrails.
|
||||
7. **`/docs/vex/graph-integration.md`**
|
||||
|
||||
* How VEX suppressions appear and how to validate product scoping.
|
||||
8. **`/docs/advisories/graph-integration.md`**
|
||||
|
||||
* Advisory linkage and severity normalization by policy.
|
||||
9. **`/docs/architecture/graph-services.md`**
|
||||
|
||||
* Graph Indexer, Graph API, storage choices, failure modes.
|
||||
10. **`/docs/observability/graph-telemetry.md`**
|
||||
|
||||
* Metrics, logs, tracing, dashboards.
|
||||
11. **`/docs/runbooks/graph-incidents.md`**
|
||||
|
||||
* Handling runaway queries, cache poisoning, degraded render.
|
||||
12. **`/docs/security/graph-rbac.md`**
|
||||
|
||||
* Permissions matrix, multi‑tenant boundaries.
|
||||
|
||||
Every doc should end with a “Compliance checklist.”
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Tasks
|
||||
|
||||
### 6.1 Backend: Graph Indexer
|
||||
|
||||
* [ ] Define node/edge schemas and attribute dictionaries for each type.
|
||||
* [ ] Implement event consumers for SBOM ingests, Conseiller updates, Excitator updates.
|
||||
* [ ] Build ingestion pipeline that populates nodes/edges and maintains `valid_from/valid_to`.
|
||||
* [ ] Implement aggregate counters and degree metrics.
|
||||
* [ ] Implement clustering job and persist cluster ids per node.
|
||||
* [ ] Implement snapshot materialization per SBOM and lineage tracking.
|
||||
* [ ] Unit tests for each node/edge builder; property‑based tests for identity stability.
|
||||
|
||||
### 6.2 Backend: Graph API
|
||||
|
||||
* [ ] Implement `/graph/search` with prefix and exact match across node types.
|
||||
* [ ] Implement `/graph/query` with validation, planning, cost estimation, and streaming tile results.
|
||||
* [ ] Implement `/graph/paths` with constraints and depth limits; shortest path heuristic.
|
||||
* [ ] Implement `/graph/diff` computing adds/removes/changed versions; stream results.
|
||||
* [ ] Implement overlays: advisory join, VEX join, policy materialization and explain sampling.
|
||||
* [ ] Implement exports: GraphML, CSV edge list, NDJSON findings, PNG/SVG snapshots.
|
||||
* [ ] RBAC middleware integration; multi‑tenant scoping.
|
||||
* [ ] Load tests with synthetic large SBOMs; define default budgets.
|
||||
|
||||
### 6.3 Policy Engine integration
|
||||
|
||||
* [ ] Add endpoint to fetch explain traces for specific node ids in batch.
|
||||
* [ ] Add materialization export that Graph API can cache per policy version.
|
||||
|
||||
### 6.4 Console (Web UI)
|
||||
|
||||
* [ ] Create `graph-explorer` module with routes `/graph`, `/graph/diff`, `/graph/q/:id`.
|
||||
* [ ] Implement WebGL canvas with LOD, culling, edge bundling, deterministic layout seed.
|
||||
* [ ] Build search, filter, lens, and overlay toolbars.
|
||||
* [ ] Side panels: details, evidence tabs, explain viewer.
|
||||
* [ ] Diff mode: split/overlay toggles and color legend.
|
||||
* [ ] Saved queries: create, update, run; deep links.
|
||||
* [ ] Export UI: formats, server round‑trip, progress indicators.
|
||||
* [ ] a11y audit and keyboard‑only flow.
|
||||
|
||||
### 6.5 CLI
|
||||
|
||||
* [ ] Implement `stella sbom graph *` subcommands with JSON IO and piping support.
|
||||
* [ ] Document examples and stable output schemas for CI consumption.
|
||||
|
||||
### 6.6 Observability & Ops
|
||||
|
||||
* [ ] Dashboards for tile latency, query denials, cache hit rate, memory.
|
||||
* [ ] Alerting on query error spikes, OOM risk, cache churn.
|
||||
* [ ] Runbooks in `/docs/runbooks/graph-incidents.md`.
|
||||
|
||||
### 6.7 Docs
|
||||
|
||||
* [ ] Author all docs in section 5, link from Console contextual help.
|
||||
* [ ] Add end‑to‑end tutorial: “Investigate GHSA‑XXXX across prod artifacts.”
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Console renders large SBOM graphs with semantic zoom, overlays, and responsive interactions.
|
||||
* Users can run impact and path queries with bounded depth and get results within budget.
|
||||
* VEX suppressions and advisory severities appear correctly and are consistent with policy.
|
||||
* Diff view clearly shows added/removed/changed nodes/edges between two SBOMs.
|
||||
* Saved queries and deep links reproduce the same view deterministically (given same data).
|
||||
* Exports produce valid GraphML/CSV/NDJSON and image snapshots.
|
||||
* CLI supports search, query, paths, impacted, diff, and export with stable schemas.
|
||||
* AOC guardrails: explorer never mutates facts; overlays reflect Policy Engine decisions.
|
||||
* RBAC enforced; all actions logged and observable.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks & mitigations
|
||||
|
||||
* **Graph explosion on large monorepos** → tiling, clustering, budgets, and strict depth limits.
|
||||
* **Inconsistent identities across tools** → canonicalize purls/digests; property‑based tests for identity stability.
|
||||
* **Policy overlay latency** → precompute materializations for hot policy versions; sample explains only on focus.
|
||||
* **User confusion** → strong lens defaults, deterministic layouts, legends, in‑context help.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit**: node/edge builders, identity normalization, cost estimator.
|
||||
* **Integration**: ingest SBOM + advisories + VEX, verify overlays and counts.
|
||||
* **E2E**: Playwright flows for search→impact→diff→export; deep link determinism.
|
||||
* **Performance**: simulate 500k nodes/2M edges; measure tile latency and memory.
|
||||
* **Security**: RBAC matrix; tenant isolation tests; query validation fuzzing.
|
||||
* **Determinism**: snapshot round‑trip: same query and seed produce identical layout and stats.
|
||||
|
||||
---
|
||||
|
||||
## 10) Feature flags
|
||||
|
||||
* `graph.explorer` (UI feature module)
|
||||
* `graph.paths` (advanced path queries)
|
||||
* `graph.diff` (SBOM diff mode)
|
||||
* `graph.overlays.policy` (policy overlay + explain sampling)
|
||||
* `graph.export` (exports enabled)
|
||||
|
||||
Documented in `/docs/observability/graph-telemetry.md`.
|
||||
|
||||
---
|
||||
|
||||
## 11) Non‑goals (this epic)
|
||||
|
||||
* Real‑time process/runtime call graphs.
|
||||
* Full substitution for text reports; Explorer complements Reports.
|
||||
* Cross‑tenant graphs; all queries are tenant‑scoped.
|
||||
|
||||
---
|
||||
|
||||
## 12) Philosophy
|
||||
|
||||
* **See the system**: security and license risk are structural. If you cannot see structure, you will miss risk.
|
||||
* **Evidence over assertion**: every colored node corresponds to raw facts and explainable determinations.
|
||||
* **Bounded interactivity**: fast, partial answers beat slow “complete” ones.
|
||||
* **Immutability**: graphs mirror SBOM snapshots and are never rewritten; we add context, not edits.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,650 +0,0 @@
|
||||
Below is the expanded, “maximum documentation” package for Epic 6. It is paste‑ready for your repo and deliberately formal so engineering, docs, and audit folks can work from the same source of truth.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 6: Vulnerability Explorer (policy‑aware)
|
||||
|
||||
**Short name:** `Vuln Explorer`
|
||||
**Services touched:** Conseiller (Feedser), Excitator (Vexer), SBOM Service, Policy Engine, Findings Ledger (new), Web API Gateway, Authority (authN/Z), Workers/Scheduler, Telemetry/Analytics, Console (Web UI), CLI
|
||||
**AOC ground rule:** Conseiller/Excitator aggregate but never merge or rewrite source documents. The Explorer only renders **effective** results as decided by Policy Engine and records human workflow as immutable ledger events.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The Vulnerability Explorer is the API and UI for operational triage, investigation, and reporting of vulnerabilities across all artifacts tracked by StellaOps. It correlates SBOM inventory with advisory and VEX evidence, then displays the **effective** status and obligations per the currently selected **Policy version**. It never edits source evidence. It provides:
|
||||
|
||||
* Policy‑aware lists, pivots, and detail views
|
||||
* Triage workflow with immutable audit ledger
|
||||
* Risk acceptance with expiry and evidence
|
||||
* SLA tracking by severity and business tier
|
||||
* Simulation against other policy versions
|
||||
* Exports and cryptographically signed “evidence bundles”
|
||||
|
||||
**Identity principle:** one **Finding** per tuple `(artifact_id, purl, version, advisory_key)`, with links to every contributing AdvisoryEvidence and VexEvidence.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (concise)
|
||||
|
||||
* Prioritization must reflect **policy and VEX**, not raw feeds.
|
||||
* Audit requires complete, reproducible lineage: what changed, why, and who decided.
|
||||
* Operators need consistent APIs, CLI, and a UI that explain determinations, not just list CVEs.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Domain model and contracts
|
||||
|
||||
**Evidence (immutable)**
|
||||
|
||||
* `AdvisoryEvidence` (from Conseiller)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "adv_evd:...uuid...",
|
||||
"tenant": "acme",
|
||||
"source": "ghsa|nvd|vendor|ossindex|... ",
|
||||
"source_id": "GHSA-xxxx",
|
||||
"schema": "GHSA|CVE|CSAF-ADV",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"affected": [{"ecosystem":"npm","purl":"pkg:npm/lodash","ranges":[{"type":"semver","events":[{"introduced":"0.0.0"},{"fixed":"4.17.21"}]}]}],
|
||||
"cvss": {"version":"3.1","baseScore":7.5,"vectorString":"AV:N/AC:L/..."},
|
||||
"severity": "HIGH",
|
||||
"urls": ["https://..."],
|
||||
"published": "2024-06-10T12:00:00Z",
|
||||
"withdrawn": null,
|
||||
"ingested_at": "2024-06-11T08:43:21Z"
|
||||
}
|
||||
```
|
||||
* `VexEvidence` (from Excitator)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "vex_evd:...uuid...",
|
||||
"tenant": "acme",
|
||||
"source": "vendor-csaf",
|
||||
"schema": "CSAF-VEX",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"product_scope": [{"purl":"pkg:npm/lodash@4.17.20"}],
|
||||
"status": "not_affected|affected|fixed|under_investigation",
|
||||
"justification": "component_not_present|vulnerable_code_not_in_execute_path|... ",
|
||||
"impact_statement": "Not reachable in Acme Payment Service.",
|
||||
"timestamp": "2024-06-12T10:00:00Z",
|
||||
"ingested_at": "2024-06-12T10:10:02Z"
|
||||
}
|
||||
```
|
||||
* `InventoryEvidence` (from SBOM Service)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "inv_evd:...uuid...",
|
||||
"tenant": "acme",
|
||||
"artifact_id": "svc:payments@1.14.0",
|
||||
"sbom_id": "sbom:sha256:...",
|
||||
"purl": "pkg:npm/lodash@4.17.20",
|
||||
"scope": "runtime|dev|test",
|
||||
"runtime_flag": true,
|
||||
"paths": [["root", "pkg:npm/a", "pkg:npm/lodash"]],
|
||||
"discovered_at": "2024-06-12T11:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**PolicyDetermination** (read‑only from Policy Engine)
|
||||
|
||||
```json
|
||||
{
|
||||
"key": {
|
||||
"artifact_id": "svc:payments@1.14.0",
|
||||
"purl": "pkg:npm/lodash",
|
||||
"version": "4.17.20",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"policy_version": "1.3.0"
|
||||
},
|
||||
"applicable": true,
|
||||
"effective_severity": "HIGH",
|
||||
"exploitability": "ACTIVE|LIKELY|UNKNOWN|UNLIKELY",
|
||||
"signals": {"epss": 0.86, "kev": true, "maturity": "weaponized"},
|
||||
"suppression_state": "none|policy|vex",
|
||||
"obligations": [{"type":"fix_by","due":"2024-07-15"},{"type":"document_risk"}],
|
||||
"sla": {"due":"2024-07-15","tier":"gold"},
|
||||
"rationale": [
|
||||
{"rule":"sev.base=nvd","detail":"NVD base:7.5"},
|
||||
{"rule":"exploit.upsell.kev","detail":"KEV flag → HIGH"},
|
||||
{"rule":"env.weight.prod","detail":"Env=prod no downgrade"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Finding identity**
|
||||
|
||||
```
|
||||
finding_id = hash(tenant, artifact_id, purl, version, advisory_key)
|
||||
```
|
||||
|
||||
**Ledger event** (append‑only, tamper‑evident)
|
||||
|
||||
```json
|
||||
{
|
||||
"event_id": "led:...uuid...",
|
||||
"finding_id": "f:...hash...",
|
||||
"tenant": "acme",
|
||||
"type": "assign|comment|attach|change_state|accept_risk|set_target_fix|verify_fix|reopen",
|
||||
"payload": {"to":"team:platform","reason":"oncall triage"},
|
||||
"actor": {"user_id":"u:42","display":"Dana S."},
|
||||
"ts": "2024-06-12T14:01:02Z",
|
||||
"prev_event_hash": "sha256:...",
|
||||
"event_hash": "sha256:sha256(canonical_json(event) + prev_event_hash)"
|
||||
}
|
||||
```
|
||||
|
||||
**Projection** (materialized current state for fast lists)
|
||||
|
||||
```json
|
||||
{
|
||||
"finding_id":"f:...hash...",
|
||||
"tenant":"acme",
|
||||
"artifact_id":"svc:payments@1.14.0",
|
||||
"purl":"pkg:npm/lodash",
|
||||
"version":"4.17.20",
|
||||
"advisory_key":"CVE-2024-12345",
|
||||
"effective_severity":"HIGH",
|
||||
"exploitability":"ACTIVE",
|
||||
"suppression_state":"none",
|
||||
"status":"UNDER_INVESTIGATION|REMEDIATING|ACCEPTED|RESOLVED|NEW|SUPPRESSED",
|
||||
"sla_due":"2024-07-15",
|
||||
"owner":"team:platform",
|
||||
"kev":true,
|
||||
"epss":0.86,
|
||||
"new_since":"2024-06-12",
|
||||
"has_fix":true,
|
||||
"envs":["prod"],
|
||||
"runtime_flag":true,
|
||||
"updated_at":"2024-06-12T14:01:02Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Advisory key normalization**
|
||||
|
||||
* Input identifiers: `CVE-*`, `GHSA-*`, vendor IDs.
|
||||
* Preference order: CVE, then GHSA, else vendor id prefixed with namespace.
|
||||
* Canonicalization: uppercase, trim, map withdrawn to same key but mark `withdrawn=true` in evidence.
|
||||
* Conseiller must publish `links: [all source ids]` for provenance.
|
||||
|
||||
### 3.2 Resolver algorithm (candidate findings)
|
||||
|
||||
**Goal:** produce tuples `(artifact_id, purl, version, advisory_key)` where inventory intersects affected ranges and policy deems the path relevant.
|
||||
|
||||
**Pseudocode**
|
||||
|
||||
```
|
||||
for each artifact sbom S:
|
||||
inv = inventory(S)
|
||||
for each advisory evidence A:
|
||||
for each affected package spec in A.affected:
|
||||
for each inv_item in inv where inv_item.purl package == spec.purl package:
|
||||
if version_in_ranges(inv_item.version, spec.ranges, ecosystem):
|
||||
if policy.path_scope_allows(inv_item.scope, inv_item.runtime_flag, inv_item.paths):
|
||||
yield candidate (artifact_id, inv_item.purl, inv_item.version, A.advisory_key)
|
||||
```
|
||||
|
||||
**Version semantics per ecosystem**
|
||||
|
||||
* npm: semver, pre-release excluded unless explicitly in range.
|
||||
* Maven: Maven version rules, handle `-SNAPSHOT`, use maven‑resolver semantics.
|
||||
* PyPI: PEP 440 versioning.
|
||||
* Go: semver with `+incompatible` handling.
|
||||
* OS packages: RPM/DEB epoch:version‑release ordering.
|
||||
|
||||
**Edge cases**
|
||||
|
||||
* Multiple paths: store shortest path and count.
|
||||
* Dev/test scope: policy may exclude or downgrade.
|
||||
* Withdrawn advisories: keep as evidence; Policy can set severity to `NONE`.
|
||||
|
||||
### 3.3 VEX precedence and scoping
|
||||
|
||||
* If any matching **VEX** says `not_affected` scoped to the artifact product/component per CSAF product tree, set `suppression_state="vex"` and `applicable=false`.
|
||||
* If VEX says `fixed` and inventory version is >= fixed version, mark as **Resolved (verified)** after SBOM recrawl confirms.
|
||||
* If VEX `under_investigation`, no suppression; may add a policy grace period obligation.
|
||||
|
||||
### 3.4 Policy evaluation
|
||||
|
||||
* Inputs: candidate tuple + context (artifact env, business tier, ownership, signals, fix availability).
|
||||
* Determinations: applicability, effective severity, exploitability, obligations, SLA, suppression.
|
||||
* Suppression by policy examples: test‑scope only; path through optional deps; package vendored but not linked at runtime.
|
||||
* Simulation: identical input, alternate `policy_version`; returns determinations without side effects.
|
||||
|
||||
### 3.5 API surface (authoritative)
|
||||
|
||||
**List**
|
||||
|
||||
```
|
||||
GET /vuln/findings?policy=1.3.0&sev=high,critical&group_by=artifact&exploit=kev&env=prod&page=1&page_size=100
|
||||
```
|
||||
|
||||
Response
|
||||
|
||||
```json
|
||||
{
|
||||
"page": 1,
|
||||
"page_size": 100,
|
||||
"total": 740,
|
||||
"group_by": "artifact",
|
||||
"results": [
|
||||
{"group":"svc:payments","counts":{"CRITICAL":3,"HIGH":12,"MEDIUM":8},"sla_breaches":2},
|
||||
...
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Query (complex filters)**
|
||||
|
||||
```
|
||||
POST /vuln/findings/query
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"policy": "1.3.0",
|
||||
"filter": {
|
||||
"severity": [">=MEDIUM"],
|
||||
"exploit": ["kev", "epss>=0.8"],
|
||||
"artifact": ["svc:payments", "svc:checkouts"],
|
||||
"status": ["NEW","UNDER_INVESTIGATION"],
|
||||
"env": ["prod"]
|
||||
},
|
||||
"sort": [{"field":"effective_severity","dir":"desc"},{"field":"epss","dir":"desc"}],
|
||||
"page": {"number":1,"size":200}
|
||||
}
|
||||
```
|
||||
|
||||
**Detail**
|
||||
|
||||
```
|
||||
GET /vuln/findings/{finding_id}?policy=1.3.0
|
||||
```
|
||||
|
||||
Returns projection, evidence links, policy rationale, paths, history summary.
|
||||
|
||||
**Workflow**
|
||||
|
||||
```
|
||||
POST /vuln/findings/{id}/assign { "to": "team:platform" }
|
||||
POST /vuln/findings/{id}/comment { "text": "triage notes..." }
|
||||
POST /vuln/findings/{id}/accept-risk { "until":"2025-06-30","reason":"vendor patch pending","evidence":["url|upload_id"] }
|
||||
POST /vuln/findings/{id}/verify-fix { "sbom_id": "sbom:sha256:..." }
|
||||
POST /vuln/findings/{id}/target-fix { "version": "4.17.21" }
|
||||
```
|
||||
|
||||
**Simulation**
|
||||
|
||||
```
|
||||
POST /vuln/simulate
|
||||
{
|
||||
"policy_from": "1.3.0",
|
||||
"policy_to": "1.4.0",
|
||||
"query": { "severity":[">=MEDIUM"], "env":["prod"] }
|
||||
}
|
||||
```
|
||||
|
||||
Response includes per‑finding delta `{before, after, diff}`.
|
||||
|
||||
**Export**
|
||||
|
||||
```
|
||||
POST /vuln/export { "format":"ndjson","scope":{"query":{...}} }
|
||||
```
|
||||
|
||||
Returns a signed bundle (see §3.10).
|
||||
|
||||
**Errors**
|
||||
|
||||
* `400` validation, `403` RBAC, `404` not found, `409` state conflict (idempotency), `429` rate limited, `5xx` server.
|
||||
|
||||
### 3.6 Console (Web UI)
|
||||
|
||||
**Routes**
|
||||
|
||||
* `/vuln` list with saved views
|
||||
* `/vuln/:id` detail drawer state
|
||||
* `/vuln/simulate/:policyVersion` diff mode
|
||||
|
||||
**State shape (client)**
|
||||
|
||||
```ts
|
||||
interface VulnListState {
|
||||
policyVersion: string;
|
||||
filters: {...};
|
||||
sort: [...];
|
||||
columns: string[];
|
||||
viewId?: string;
|
||||
page: {number: number; size: number};
|
||||
}
|
||||
```
|
||||
|
||||
**UX**
|
||||
|
||||
* Virtualized grid with server paging; column chooser; density toggle.
|
||||
* Quick filters: severity, exploit signals, status, env, owner, fix availability.
|
||||
* Detail tabs: Summary, Evidence (raw docs with provenance), Policy (rationale chain), Paths (deep link to Graph Explorer), Fixes, History.
|
||||
* Simulation bar shows delta chips: `+21 HIGH`, `-9 Suppressed by VEX` etc.
|
||||
* Evidence bundle dialog previews scope and size.
|
||||
* a11y: ARIA roles on grid, keyboard shortcuts: `A` assign, `C` comment, `R` accept risk, `V` verify fix.
|
||||
|
||||
### 3.7 CLI
|
||||
|
||||
Commands
|
||||
|
||||
```
|
||||
stella vuln list --policy 1.3.0 --sev high,critical --group-by artifact --env prod --json
|
||||
stella vuln show --id <finding-id> --policy 1.3.0
|
||||
stella vuln simulate --from 1.3.0 --to 1.4.0 --sev '>=medium' --delta --json
|
||||
stella vuln assign --filter 'advisory:CVE-2024-12345 artifact:payments' --to team:platform
|
||||
stella vuln accept-risk --id <finding-id> --until 2025-06-30 --reason "vendor patch pending" --evidence url:https://ticket/123
|
||||
stella vuln verify-fix --id <finding-id> --sbom <sbom-id>
|
||||
```
|
||||
|
||||
Return codes: `0 ok`, `2 invalid args`, `3 budget exceeded`, `4 not found`, `5 denied`.
|
||||
|
||||
### 3.8 Storage schema (illustrative)
|
||||
|
||||
**Tables**
|
||||
|
||||
```sql
|
||||
-- Evidence
|
||||
CREATE TABLE evidence_advisory (...);
|
||||
CREATE INDEX ea_tenant_key ON evidence_advisory(tenant, advisory_key);
|
||||
CREATE TABLE evidence_vex (...);
|
||||
CREATE INDEX ev_tenant_key ON evidence_vex(tenant, advisory_key);
|
||||
CREATE TABLE evidence_inventory (...);
|
||||
CREATE INDEX ei_artifact_purl ON evidence_inventory(tenant, artifact_id, purl);
|
||||
|
||||
-- Ledger
|
||||
CREATE TABLE findings_ledger_events (
|
||||
event_id uuid PRIMARY KEY,
|
||||
finding_id bytea NOT NULL,
|
||||
tenant text NOT NULL,
|
||||
type text NOT NULL,
|
||||
payload jsonb NOT NULL,
|
||||
actor jsonb NOT NULL,
|
||||
ts timestamptz NOT NULL,
|
||||
prev_event_hash bytea,
|
||||
event_hash bytea NOT NULL
|
||||
);
|
||||
CREATE INDEX fle_find_ts ON findings_ledger_events(tenant, finding_id, ts);
|
||||
|
||||
-- Projection
|
||||
CREATE TABLE findings_projection (
|
||||
finding_id bytea PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
artifact_id text NOT NULL,
|
||||
purl text NOT NULL,
|
||||
version text NOT NULL,
|
||||
advisory_key text NOT NULL,
|
||||
policy_version text NOT NULL,
|
||||
effective_severity text NOT NULL,
|
||||
exploitability text,
|
||||
suppression_state text,
|
||||
status text NOT NULL,
|
||||
sla_due date,
|
||||
owner text,
|
||||
kev boolean,
|
||||
epss double precision,
|
||||
envs text[],
|
||||
runtime_flag boolean,
|
||||
updated_at timestamptz NOT NULL
|
||||
);
|
||||
CREATE INDEX fp_query ON findings_projection(tenant, policy_version, effective_severity, status);
|
||||
```
|
||||
|
||||
**Tamper‑evidence**
|
||||
|
||||
* Ledger events use chained SHA‑256 hashes over canonical JSON + previous hash.
|
||||
* Daily Merkle root of all event hashes is anchored to the audit store (and optionally external timestamping service).
|
||||
|
||||
### 3.9 Performance and scaling
|
||||
|
||||
* P95 list endpoint under 600 ms for 100‑row pages at 5M findings/tenant.
|
||||
* Projections denormalize heavy joins; background projector uses idempotent jobs keyed by `(tenant,finding_id,policy_version)`.
|
||||
* Rate limits per tenant and per API key; backpressure on export jobs; exponential retry for projector.
|
||||
|
||||
### 3.10 Evidence bundle format
|
||||
|
||||
* **Container:** ZIP with `manifest.json`, `findings.ndjson`, `advisory_evidence.ndjson`, `vex_evidence.ndjson`, `inventory_evidence.ndjson`, `policy_version.json`, `ledger_events.ndjson`, `CHECKSUMS`.
|
||||
* **Signing:** Detached signature `bundle.sig` using tenant’s org key (Ed25519).
|
||||
* **Manifest**
|
||||
|
||||
```json
|
||||
{"generated_at":"2024-06-12T15:00:00Z","tenant":"acme","policy_version":"1.3.0","scope":{"query":{...}},"counts":{"findings":421}}
|
||||
```
|
||||
|
||||
### 3.11 Observability
|
||||
|
||||
* Metrics (OpenTelemetry):
|
||||
|
||||
* `vuln_findings_list_latency_ms` (histogram)
|
||||
* `vuln_projection_lag_seconds` (gauge)
|
||||
* `vuln_new_findings_total` (counter)
|
||||
* `vuln_sla_breaches_total` (counter by sev, owner)
|
||||
* `vuln_simulation_latency_ms` (histogram)
|
||||
* Logs: structured JSON with `tenant`, `policy_version`, `query_hash`, `result_count`.
|
||||
* Traces: spans for resolver, policy calls, projection builds, export assembly.
|
||||
* PII: redact comments in logs; store attachments encrypted at rest (KMS).
|
||||
|
||||
### 3.12 Security & RBAC
|
||||
|
||||
**Roles**
|
||||
|
||||
* Viewer: GET list/detail/export read scope.
|
||||
* Investigator: Viewer + workflow actions except risk acceptance.
|
||||
* Operator: Investigator + risk acceptance, verify fix, bulk actions.
|
||||
* Auditor: Viewer + evidence bundles and ledger integrity checks.
|
||||
|
||||
**ABAC**
|
||||
|
||||
* Attribute constraints: by `artifact.owner`, `env`, and `business_tier`.
|
||||
* CSRF protection for Console; all POST require anti‑forgery tokens.
|
||||
* Attachments stored with envelope encryption; signed URLs for limited time access.
|
||||
|
||||
### 3.13 Rollout and migrations
|
||||
|
||||
* Feature flags: `vuln.explorer.ui`, `vuln.explorer.simulation`, `vuln.explorer.bulk_actions`, `vuln.explorer.evidence_bundle`.
|
||||
* Phase 1: dark launch API and projections.
|
||||
* Phase 2: UI read‑only list and detail.
|
||||
* Phase 3: workflow actions and exports.
|
||||
* Data backfill: replay advisory/VEX/SBOM events to seed projections.
|
||||
* Compatibility: maintain projection v1 schema for two releases; migration scripts in `/migrations/vuln/`.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **Findings Ledger (new)**
|
||||
|
||||
* Append‑only event store with projector to `findings_projection`.
|
||||
* Event validation and canonicalization; hashing and Merkle root anchoring.
|
||||
|
||||
* **Vuln Explorer API (new)**
|
||||
|
||||
* Query/filter engine with policy parameterization and grouping.
|
||||
* Simulation endpoint.
|
||||
* Export job orchestrator.
|
||||
|
||||
* **Conseiller / Excitator (updates)**
|
||||
|
||||
* Guarantee canonical `advisory_key` and publish `links[]`.
|
||||
* No merges; maintain raw payload snapshots.
|
||||
|
||||
* **Policy Engine (updates)**
|
||||
|
||||
* Batch evaluation endpoint `POST /policy/eval/batch` with `simulate` support.
|
||||
* Return rationale chain with rule IDs.
|
||||
|
||||
* **SBOM Service (updates)**
|
||||
|
||||
* Publish inventory deltas; include `scope`, `runtime_flag`, `paths`.
|
||||
* Nearest safe version hints.
|
||||
|
||||
* **Workers/Scheduler**
|
||||
|
||||
* Resolver job keyed by `(tenant, artifact_id, sbom_id)`; emits candidate tuples.
|
||||
* Recompute on policy activation and evidence changes.
|
||||
|
||||
### 4.2 Code structure
|
||||
|
||||
```
|
||||
/src/Findings/StellaOps.Findings.Ledger
|
||||
/api
|
||||
/projector
|
||||
/storage
|
||||
/src/VulnExplorer/StellaOps.VulnExplorer.Api
|
||||
/routes
|
||||
/query
|
||||
/simulation
|
||||
/export
|
||||
/packages/console/features/vuln-explorer
|
||||
/components
|
||||
/pages
|
||||
/state
|
||||
/src/Cli/StellaOps.Cli
|
||||
```
|
||||
|
||||
### 4.3 Performance tasks
|
||||
|
||||
* Projection indexes and covering queries; explain plans in `/docs/vuln/perf-notes.md`.
|
||||
* Cache hot groupings per tenant with TTL and invalidation on ledger projector tick.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. `/docs/vuln/explorer-overview.md`
|
||||
Conceptual model, identities, evidence vs determinations, AOC guarantees.
|
||||
2. `/docs/vuln/explorer-using-console.md`
|
||||
Workflows with screenshots, keyboard shortcuts, saved views, deep links.
|
||||
3. `/docs/vuln/explorer-api.md`
|
||||
Endpoint specs, query language, grouping, pagination, errors, rate limits.
|
||||
4. `/docs/vuln/explorer-cli.md`
|
||||
Commands, flags, examples, exit codes.
|
||||
5. `/docs/vuln/findings-ledger.md`
|
||||
Event schema, state machine, hashing, Merkle roots, integrity checks.
|
||||
6. `/docs/policy/vuln-determinations.md`
|
||||
Inputs, outputs, precedence rules, simulation semantics.
|
||||
7. `/docs/vex/explorer-integration.md`
|
||||
CSAF mapping, scoping to product tree, precedence.
|
||||
8. `/docs/advisories/explorer-integration.md`
|
||||
Advisory key normalization, provenance, withdrawn handling.
|
||||
9. `/docs/sbom/vuln-resolution.md`
|
||||
Ecosystem version semantics, path sensitivity, scope rules.
|
||||
10. `/docs/observability/vuln-telemetry.md`
|
||||
Metrics, logs, traces, dashboards, SLOs.
|
||||
11. `/docs/security/vuln-rbac.md`
|
||||
Role mapping, ABAC, attachment encryption, CSRF.
|
||||
12. `/docs/runbooks/vuln-ops.md`
|
||||
Recompute storms, projector lag, policy activation drains, export failures.
|
||||
13. `/docs/install/containers.md`
|
||||
Add `findings-ledger`, `vuln-explorer-api` images, compose/k8s manifests, resource sizing, health checks.
|
||||
|
||||
> Each doc ends with: **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend: Findings & API
|
||||
|
||||
* [ ] Define evidence and ledger schemas; migrations.
|
||||
* [ ] Implement resolver for npm, Maven, PyPI, Go, OS packages with property‑based tests for version comparisons.
|
||||
* [ ] Implement ledger API and projector with idempotency and hashing.
|
||||
* [ ] Implement list/detail/grouping endpoints with server‑side paging.
|
||||
* [ ] Implement simulation and export (bundle assembly, signing).
|
||||
* [ ] Integrate Policy Engine batch eval with rationale traces.
|
||||
* [ ] RBAC via Authority with ABAC filters.
|
||||
* [ ] Load tests at 5M findings/tenant; tune indexes.
|
||||
|
||||
### Conseiller/Excitator
|
||||
|
||||
* [ ] Normalize `advisory_key` and persist `links[]`.
|
||||
* [ ] Ensure raw payload snapshots are retrievable by Explorer for Evidence tab.
|
||||
|
||||
### SBOM Service
|
||||
|
||||
* [ ] Emit `scope`, `runtime_flag`, `paths`; safe version hints.
|
||||
* [ ] Inventory delta events to trigger resolver.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Build grid with virtualization, saved views, deep link serializer.
|
||||
* [ ] Implement detail tabs and path deep‑links to Graph Explorer.
|
||||
* [ ] Add simulation bar and delta chips.
|
||||
* [ ] Evidence bundle dialog.
|
||||
* [ ] a11y keyboard flow and ARIA labeling; unit and E2E tests.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella vuln list|show|simulate|assign|accept-risk|verify-fix` with `--json` and CSV export.
|
||||
* [ ] Stable output schemas; pipe‑friendly defaults.
|
||||
|
||||
### Observability/Ops
|
||||
|
||||
* [ ] Dashboards for list latency, projection lag, new/reopened, SLA breaches.
|
||||
* [ ] Alerts on projector backlog, API 5xx spikes, export failures.
|
||||
* [ ] Runbooks in `/docs/runbooks/vuln-ops.md`.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author files listed in §5 with cross‑links to Policy Studio and SBOM Graph Explorer.
|
||||
* [ ] Update `/docs/install/containers.md` with new images and compose/k8s snippets.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* List and detail views reflect **effective** policy outcomes and update instantly when switching policy versions.
|
||||
* Evidence tab shows all raw advisory/VEX documents with provenance; no source merging.
|
||||
* Resolver respects ecosystem semantics, scope, and paths; path tab round‑trips to Graph Explorer.
|
||||
* Ledger events are immutable and reconstruct historical list states accurately.
|
||||
* Simulation returns diffs without side effects and matches Policy Engine outputs.
|
||||
* CLI/API support paging, grouping, export, simulation; contracts stable and documented.
|
||||
* RBAC and tenant isolation validated by tests; attachments encrypted.
|
||||
* P95 performance budgets met; dashboards green for SLOs.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks and mitigations
|
||||
|
||||
* **Advisory identity collisions** → strict canonicalization; preserve `links[]`; never merge raw docs.
|
||||
* **Projection lag** → backpressure, worker autoscaling, health checks; alerting on lag.
|
||||
* **Resolver false positives** → path evidence required; dev/test scope rules explicit; ecosystem‑specific tests.
|
||||
* **User confusion over suppression** → explicit badges; Policy tab with rationale and “why changed.”
|
||||
* **Export size** → NDJSON streaming, size estimator in UI, scope previews.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit:** version comparators, resolver per ecosystem, policy mapping, ledger state machine.
|
||||
* **Integration:** SBOM + advisories + VEX ingestion, candidate generation, policy application, suppression precedence.
|
||||
* **E2E Console:** triage, bulk assign, simulation, evidence bundle download; keyboard‑only flow.
|
||||
* **Performance:** list/grouping at target scale; projector rebuild; export assembly.
|
||||
* **Security:** RBAC matrix, ABAC filters, CSRF, signed URL lifetimes, tamper‑evidence verification.
|
||||
* **Determinism:** time‑travel snapshots reproduce prior states byte‑for‑byte.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Facts first, decisions second.** Evidence is immutable; decisions and workflow sit on top in a ledger.
|
||||
* **Policy is the lens.** The same facts can imply different obligations; the system must make that explicit and reproducible.
|
||||
* **Audit > convenience.** Every state change is justified, signed, and verifiable.
|
||||
* **No hidden magic.** If anything is suppressed, the UI shows the rule or VEX that did it, with documents attached.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,545 +0,0 @@
|
||||
Below is the expanded, “maximum documentation” package for Epic 7. It is paste‑ready for your repo and deliberately formal so engineering, docs, governance, and audit can work from the same source of truth.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 7: VEX Consensus Lens
|
||||
|
||||
**Short name:** `VEX Lens`
|
||||
**Services touched:** Excitator (Vexer), Conseiller (Feedser), SBOM Service, Policy Engine, Findings Ledger, Web API Gateway, Authority (authN/Z), Console (Web UI), CLI, Telemetry/Analytics
|
||||
**AOC ground rule:** Excitator and Conseiller aggregate but never merge or rewrite source documents. The Lens only computes a **derived, reproducible** consensus view while preserving all raw evidence and provenance.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The VEX Consensus Lens is a deterministic computation and presentation layer that ingests all available VEX statements about a given advisory key (e.g., CVE) and product/context, then produces an **explicit consensus state** for each `(artifact_id, purl, version, advisory_key)` tuple. It answers: “Given multiple, possibly conflicting VEX statements from different issuers, what is the most trustworthy, policy‑consistent interpretation for this artifact and version?”
|
||||
|
||||
It never edits or merges VEX documents. It normalizes them, aligns their **product scope** to SBOM inventory, scores issuers via a tenant‑configurable **trust model**, applies time and version scoping, and outputs:
|
||||
|
||||
* `consensus_state`: `NOT_AFFECTED | AFFECTED | FIXED | UNDER_INVESTIGATION | INCONCLUSIVE | DISPUTED`
|
||||
* `confidence`: 0.0–1.0 numeric score
|
||||
* `rationale`: structured explanation of evidence, weights, and rules applied
|
||||
* `quorum`: weighted vote breakdown by issuer, timestamp, justification, and version applicability
|
||||
|
||||
The Lens plugs into the Policy Engine. Policy remains the source of truth for applicability, severity, and obligations; the Lens supplies structured signals and provenance that policy can use to suppress findings, downgrade risk, or set SLAs.
|
||||
|
||||
Key properties:
|
||||
|
||||
* **Reproducible:** Same inputs and policy yield same outputs.
|
||||
* **Explainable:** Every output carries a machine‑readable rationale chain.
|
||||
* **Scoped:** Product trees and version ranges are resolved against actual SBOM inventory.
|
||||
* **Immutable evidence:** All raw VEX docs remain intact, retrievable, and linkable.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (concise)
|
||||
|
||||
Real ecosystems have overlapping VEX statements from upstream maintainers, vendors, distros, and third parties. Operators need a single, defensible view without losing provenance. The Lens reduces noise, flags conflicts, and turns fragmented VEX evidence into auditable signals that policy and humans can act on.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Input model
|
||||
|
||||
**From Excitator (Vexer):**
|
||||
|
||||
* Raw `VexEvidence` documents (CSAF VEX, OpenVEX, CycloneDX VEX) with:
|
||||
|
||||
* `advisory_key` (canonicalized)
|
||||
* product tree or component coordinates (purl, CPE, vendor IDs)
|
||||
* status (`not_affected`, `affected`, `fixed`, `under_investigation`)
|
||||
* justifications (CSAF, OpenVEX enumerations)
|
||||
* version ranges or fixed versions if available
|
||||
* timestamps (issued, last_updated), source URLs
|
||||
* cryptographic metadata (signature, issuer, certificate chain) if present
|
||||
|
||||
**From SBOM Service:**
|
||||
|
||||
* Inventory for each artifact: purls, versions, dependency paths, scopes, env flags.
|
||||
|
||||
**From Issuer Directory (new)**
|
||||
|
||||
* Directory of known VEX issuers:
|
||||
|
||||
* identity, organization, domain, CSAF publisher metadata
|
||||
* public keys and trust anchors (Ed25519/X.509/PKIX/PKCS7/DSSE)
|
||||
* default trust weight
|
||||
* tenancy‑specific overrides
|
||||
|
||||
**From Policy Engine:**
|
||||
|
||||
* Policy version and options relevant to VEX evaluation:
|
||||
|
||||
* trust model parameters
|
||||
* confidence thresholds per environment/tier
|
||||
* justification whitelist/blacklist
|
||||
* recency requirements and expiry windows
|
||||
* precedence rules for status conflicts
|
||||
|
||||
### 3.2 Normalization
|
||||
|
||||
1. **Canonical advisory key:** CVE preferred; GHSA mapped; vendor IDs namespaced.
|
||||
2. **Status mapping:** Normalize all encodings to `NOT_AFFECTED | AFFECTED | FIXED | UNDER_INVESTIGATION`.
|
||||
3. **Product scope alignment:** Convert CSAF product tree/CPE to purl variants using deterministic mappings; store mapping evidence.
|
||||
4. **Version scoping:** For each evidence, compute `applies_to(version)` using ecosystem comparators (npm semver, Maven, PEP 440, Go semver, RPM/DEB EVR).
|
||||
5. **Signature verification:** If signed, verify against Issuer Directory trust anchors; attach `sig_verified=true/false` and chain metadata.
|
||||
6. **Temporal fields:** Compute `effective_at` (issued), `observed_at` (ingested), and staleness.
|
||||
|
||||
### 3.3 Trust model
|
||||
|
||||
Each `VexEvidence` gets a base weight `w_base` from issuer type and verification:
|
||||
|
||||
* Maintainer/Upstream signed: 0.9
|
||||
* Maintainer/Upstream unsigned: 0.7
|
||||
* Distro/SIG signed: 0.8
|
||||
* Vendor of downstream product signed: 0.8
|
||||
* Third‑party scanner VEX: 0.4
|
||||
* Unknown/unsigned with weak provenance: 0.2
|
||||
|
||||
Weights are then adjusted:
|
||||
|
||||
```
|
||||
w = w_base
|
||||
* f_signature(sig_verified) // e.g., 1.1 if verified, 0.8 if unverifiable
|
||||
* f_recency(age_days) // decay after T days (policy)
|
||||
* f_justification(type) // e.g., "component_not_present" lower if SBOM shows present
|
||||
* f_scope_match(score) // quality of product match: exact purl > family > CPE wildcard
|
||||
* f_env(app_env) // optional env-specific multipliers
|
||||
```
|
||||
|
||||
All `f_*` are bounded to [0.5, 1.2] by default to prevent runaway effects. Tenants can override per Policy Studio.
|
||||
|
||||
### 3.4 Consensus algorithm
|
||||
|
||||
For a tuple `(artifact_id, purl, version, advisory_key)`:
|
||||
|
||||
1. **Collect** all normalized `VexEvidence` whose product scope maps to `purl` and `applies_to(version)==true`.
|
||||
|
||||
2. **Bucket** by normalized status; compute `W(status) = Σ w(evidence)` per status.
|
||||
|
||||
3. **Apply precedence** rules from policy:
|
||||
|
||||
* If `W(NOT_AFFECTED)` exceeds threshold `T_na` and there is no `FIXED` evidence contradictory for the same version, propose `NOT_AFFECTED`.
|
||||
* If any `FIXED` applies and inventory version ≥ fixed version, propose `FIXED`.
|
||||
* If `W(AFFECTED)` ≥ `T_aff` and no dominating `NOT_AFFECTED`, propose `AFFECTED`.
|
||||
* If `W(UNDER_INVESTIGATION)` dominates and others below thresholds, propose `UNDER_INVESTIGATION`.
|
||||
* If both `AFFECTED` and `NOT_AFFECTED` exceed thresholds within a small margin `δ`, mark `DISPUTED`.
|
||||
* Otherwise `INCONCLUSIVE`.
|
||||
|
||||
4. **Confidence score:**
|
||||
|
||||
```
|
||||
confidence = W(winning_status) / (W(AFFECTED)+W(NOT_AFFECTED)+W(FIXED)+W(UNDER_INVESTIGATION) + ε)
|
||||
```
|
||||
|
||||
Clip to [0.0, 1.0].
|
||||
|
||||
5. **Rationale chain:**
|
||||
|
||||
* Winning status, thresholds used, top 5 contributing evidences with weights and reasons, product mapping quality, version scoping evidence, and policy knobs that influenced the result.
|
||||
|
||||
6. **Quorum summary:**
|
||||
|
||||
* List issuers and their votes, signature state, timestamps, and justifications.
|
||||
|
||||
### 3.5 Policy interaction
|
||||
|
||||
* The Lens returns `consensus_state`, `confidence`, and structured `signals`.
|
||||
* Policy Engine consumes these and may:
|
||||
|
||||
* Suppress findings automatically on `NOT_AFFECTED` confidence ≥ `P_na`.
|
||||
* Downgrade severity or extend SLA when `UNDER_INVESTIGATION` from high‑trust issuers.
|
||||
* Require human approval when `DISPUTED`.
|
||||
* Treat `FIXED` as resolved only when SBOM crawl verifies the fixed version or a `verify_fix` ledger event exists.
|
||||
|
||||
**Simulation:** Policy Studio can simulate different trust weights and thresholds and see consensus deltas without side effects.
|
||||
|
||||
### 3.6 Data contracts
|
||||
|
||||
**ConsensusRecord (materialized)**
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cons:sha256(tenant|artifact|purl|version|advisory|policy)",
|
||||
"tenant": "acme",
|
||||
"artifact_id": "svc:payments@1.14.0",
|
||||
"purl": "pkg:npm/lodash@4.17.20",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"policy_version": "1.3.0",
|
||||
"consensus_state": "NOT_AFFECTED",
|
||||
"confidence": 0.87,
|
||||
"weights": {"AFFECTED":0.31,"NOT_AFFECTED":1.25,"FIXED":0.00,"UNDER_INVESTIGATION":0.12},
|
||||
"top_evidence": ["vex_evd:...","vex_evd:..."],
|
||||
"quorum": [
|
||||
{"issuer":"lodash-maintainers","status":"NOT_AFFECTED","w":0.9,"sig_verified":true,"just":"vulnerable_code_not_present","issued":"2024-06-08"},
|
||||
{"issuer":"vendorX-distro","status":"AFFECTED","w":0.25,"sig_verified":false,"just":"generic_advisory","issued":"2024-06-07"}
|
||||
],
|
||||
"rationale": [
|
||||
{"rule":"trust.weight.issuer","detail":"maintainer signed evidence 0.9"},
|
||||
{"rule":"scope.match.exact","detail":"exact purl match"},
|
||||
{"rule":"justification.vcnp","detail":"supported by SBOM callgraph hint"}
|
||||
],
|
||||
"updated_at": "2024-06-12T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Issuer Directory entry**
|
||||
|
||||
```json
|
||||
{
|
||||
"issuer_id": "iss:lodash",
|
||||
"org": "Lodash Maintainers",
|
||||
"domains": ["lodash.com"],
|
||||
"keys": [{"type":"ed25519","pub":"...","expires":"2026-12-31"}],
|
||||
"default_weight": 0.9,
|
||||
"metadata": {"csaf_publisher": true}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.7 APIs
|
||||
|
||||
**Compute/Query**
|
||||
|
||||
```
|
||||
GET /vex/consensus?artifact=svc:payments@1.14.0&purl=pkg:npm/lodash@4.17.20&advisory=CVE-2024-12345&policy=1.3.0
|
||||
POST /vex/consensus/query { "policy":"1.3.0", "filter": { "state":["DISPUTED","INCONCLUSIVE"], "confidence":"<0.6", "env":["prod"] }, "page":{...} }
|
||||
GET /vex/consensus/{id} // full record
|
||||
POST /vex/consensus/simulate // override trust knobs and thresholds for what-if
|
||||
```
|
||||
|
||||
**Issuer Directory**
|
||||
|
||||
```
|
||||
GET /vex/issuers
|
||||
POST /vex/issuers (admin)
|
||||
POST /vex/issuers/{id}/keys (admin)
|
||||
```
|
||||
|
||||
**Exports**
|
||||
|
||||
```
|
||||
POST /vex/consensus/export { "format":"ndjson","scope":{"filter":{...}} }
|
||||
```
|
||||
|
||||
**Errors**
|
||||
|
||||
* `400` invalid mapping, `403` RBAC, `404` not found, `409` conflict, `429` rate limit.
|
||||
|
||||
### 3.8 Console (Web UI)
|
||||
|
||||
Routes:
|
||||
|
||||
* `/vex/consensus` overview with filters: state, confidence, issuer, advisory, artifact, env.
|
||||
* `/vex/consensus/:id` detail: Evidence pane, Quorum graph, Policy impact, SBOM path links.
|
||||
|
||||
UX elements:
|
||||
|
||||
* **Quorum bar:** stacked bar showing weights per status; hover reveals issuer contributions.
|
||||
* **Confidence chip:** numeric and qualitative band (Low/Med/High).
|
||||
* **Evidence table:** paged list of VEX docs with signature icon, scope match quality tag, justification tag, issued/updated timestamps.
|
||||
* **Conflict view:** for `DISPUTED`, show side-by-side issuer rationales and suggested next steps.
|
||||
* **Deep link** into Vulnerability Explorer detail, preselecting the Policy version used for the consensus.
|
||||
|
||||
A11y:
|
||||
|
||||
* ARIA roles on grid and bars; keyboard shortcuts `S` switch policy, `T` trust presets, `E` export.
|
||||
|
||||
### 3.9 CLI
|
||||
|
||||
Commands:
|
||||
|
||||
```
|
||||
stella vex consensus list --policy 1.3.0 --state disputed,inconclusive --confidence '<0.6' --artifact payments --json
|
||||
stella vex consensus show --id <cons-id> --policy 1.3.0
|
||||
stella vex simulate --policy 1.3.0 --trust 'issuer:lodash=1.0,issuer:vendorX=0.5' --thresholds 'na=1.0,aff=0.6' --json
|
||||
stella vex issuers list
|
||||
stella vex export --filter 'artifact:payments advisory:CVE-2024-12345' --out vex-consensus.ndjson
|
||||
```
|
||||
|
||||
Exit codes: `0` ok, `2` invalid args, `4` not found, `5` denied.
|
||||
|
||||
### 3.10 Storage schema (illustrative)
|
||||
|
||||
```sql
|
||||
-- Normalized VEX (reference Excitator id; do not alter raw)
|
||||
CREATE TABLE vex_normalized (
|
||||
id uuid PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
evidence_id text NOT NULL, -- link to Excitator
|
||||
advisory_key text NOT NULL,
|
||||
issuer_id text,
|
||||
status text NOT NULL, -- NOT_AFFECTED|AFFECTED|FIXED|UNDER_INVESTIGATION
|
||||
justification text,
|
||||
purl text, -- normalized mapping target
|
||||
version_range jsonb, -- ecosystem-specific encoding
|
||||
sig_verified boolean,
|
||||
scope_score real, -- 0..1 quality of mapping
|
||||
issued timestamptz,
|
||||
updated timestamptz,
|
||||
w_base real,
|
||||
UNIQUE (tenant, evidence_id)
|
||||
);
|
||||
|
||||
-- Issuer Directory
|
||||
CREATE TABLE vex_issuers (
|
||||
issuer_id text PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
org text NOT NULL,
|
||||
default_weight real NOT NULL,
|
||||
metadata jsonb,
|
||||
UNIQUE(tenant, org)
|
||||
);
|
||||
|
||||
CREATE TABLE vex_issuer_keys (
|
||||
id uuid PRIMARY KEY,
|
||||
issuer_id text NOT NULL REFERENCES vex_issuers(issuer_id),
|
||||
key_type text NOT NULL,
|
||||
pubkey text NOT NULL,
|
||||
expires timestamptz
|
||||
);
|
||||
|
||||
-- Consensus projection
|
||||
CREATE TABLE vex_consensus (
|
||||
id bytea PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
artifact_id text NOT NULL,
|
||||
purl text NOT NULL,
|
||||
version text NOT NULL,
|
||||
advisory_key text NOT NULL,
|
||||
policy_version text NOT NULL,
|
||||
consensus_state text NOT NULL,
|
||||
confidence real NOT NULL,
|
||||
weights jsonb NOT NULL,
|
||||
top_evidence text[] NOT NULL,
|
||||
updated_at timestamptz NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX vc_query ON vex_consensus(tenant, policy_version, consensus_state, confidence);
|
||||
```
|
||||
|
||||
### 3.11 Integration with Findings Ledger and Vuln Explorer
|
||||
|
||||
* The Vuln Explorer reads `vex_consensus` for each finding and renders:
|
||||
|
||||
* Consensus chip, confidence, and a link to full quorum.
|
||||
* For `NOT_AFFECTED` with confidence ≥ policy threshold, show “Suppressed by VEX (Consensus)” badge.
|
||||
* For `DISPUTED`, open a triage banner prompting manual review and optional ledger comment/assignment.
|
||||
|
||||
* Ledger receives no new event type from lens computation itself. Human actions triggered by lens views produce standard events (`comment`, `assign`, `change_state`).
|
||||
|
||||
### 3.12 Security & RBAC
|
||||
|
||||
Roles:
|
||||
|
||||
* Viewer: query consensus read‑only.
|
||||
* Investigator: Viewer + export.
|
||||
* Operator: Investigator + trust simulation.
|
||||
* Admin: manage Issuer Directory entries and keys.
|
||||
|
||||
CSRF for Console; ABAC scoping by artifact ownership and environment.
|
||||
|
||||
### 3.13 Observability
|
||||
|
||||
Metrics:
|
||||
|
||||
* `vex_consensus_compute_latency_ms` (histogram)
|
||||
* `vex_consensus_records_total` (counter)
|
||||
* `vex_consensus_disputed_total` (counter by issuer combinations)
|
||||
* `vex_consensus_staleness_seconds` (gauge)
|
||||
* `vex_signature_verification_rate` (gauge)
|
||||
|
||||
Logs: structured events with `tenant`, `policy_version`, `advisory_key`, `quorum_summary`.
|
||||
Traces: spans for normalization, mapping, trust weighting, consensus decision, DB writes.
|
||||
|
||||
### 3.14 Performance & scaling
|
||||
|
||||
Targets:
|
||||
|
||||
* P95 query under 500 ms for 100‑row pages at 10M consensus records/tenant.
|
||||
* Projection jobs are idempotent and keyed by `(tenant, artifact, purl, version, advisory, policy)`; backpressure with work queues.
|
||||
* Cache popular queries with tenant‑scoped TTL; invalidate on Excitator or policy changes.
|
||||
|
||||
### 3.15 Edge cases
|
||||
|
||||
* **Ambiguous product mapping:** mark low `scope_score`, cap weight, surface warning in UI.
|
||||
* **VEX “not present” vs SBOM shows present:** down‑weight with `f_justification`, require manual check.
|
||||
* **Withdrawn or superseded VEX:** decay to near zero; keep provenance.
|
||||
* **Partial fixes:** if fixed version applies to subset of platforms, map to env or arch dimension when available.
|
||||
* **Time travel:** consensus recalculated as of a timestamp using only evidence ≤ `as_of` and the corresponding policy version.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **VEX Lens Service (new)**
|
||||
|
||||
* Normalization pipeline, trust weighting, consensus computation, and projections.
|
||||
* Batch recompute on policy activation and Excitator deltas.
|
||||
|
||||
* **Excitator (updates)**
|
||||
|
||||
* Ensure all VEX evidence carries issuer hints and raw signature blobs when present.
|
||||
* Publish product trees and original coordinates intact.
|
||||
|
||||
* **Policy Engine (updates)**
|
||||
|
||||
* Add VEX trust knobs, thresholds, recency decay, and status precedence.
|
||||
* Batch eval endpoint accepts `consensus inputs` where needed.
|
||||
|
||||
* **Issuer Directory (new)**
|
||||
|
||||
* Manage issuer metadata and keys; tenant overrides; audit logs.
|
||||
|
||||
### 4.2 Code structure
|
||||
|
||||
```
|
||||
/src/VexLens/StellaOps.VexLens
|
||||
/normalizer
|
||||
/mapping # CPE/purl translators
|
||||
/trust # weighting functions
|
||||
/consensus # algorithm and projections
|
||||
/api
|
||||
/src/Excititor # updates
|
||||
/src/Policy/__Libraries/StellaOps.Policy # updates
|
||||
/src/IssuerDirectory/StellaOps.IssuerDirectory
|
||||
/packages/console/features/vex-consensus
|
||||
/src/Cli/StellaOps.Cli
|
||||
```
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
* Phase 1: API read‑only with basic trust model, Console list/detail, no simulation.
|
||||
* Phase 2: Policy Studio integrations and simulation.
|
||||
* Phase 3: Issuer Directory admin flows, exports, and advanced mapping diagnostics.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. `/docs/vex/consensus-overview.md`
|
||||
Purpose, scope, terminology, evidence vs derived view, AOC guarantees.
|
||||
|
||||
2. `/docs/vex/consensus-algorithm.md`
|
||||
Normalization, mapping, weighting, thresholds, precedence, formulas, examples.
|
||||
|
||||
3. `/docs/vex/issuer-directory.md`
|
||||
Managing issuers, keys, trust overrides, security model.
|
||||
|
||||
4. `/docs/vex/consensus-api.md`
|
||||
Endpoints, request/response schemas, errors, pagination, rate limits.
|
||||
|
||||
5. `/docs/vex/consensus-console.md`
|
||||
Screens, filters, conflict workflows, a11y, deep links.
|
||||
|
||||
6. `/docs/policy/vex-trust-model.md`
|
||||
Policy knobs, thresholds, decay, simulation.
|
||||
|
||||
7. `/docs/sbom/vex-mapping.md`
|
||||
Product tree mapping to purl/version, ecosystem comparators, edge cases.
|
||||
|
||||
8. `/docs/security/vex-signatures.md`
|
||||
Signature verification flows, key management, auditing.
|
||||
|
||||
9. `/docs/runbooks/vex-ops.md`
|
||||
Recompute storms, mapping failures, signature errors, lag, quotas.
|
||||
|
||||
All docs end with the imposed rule statement.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend core
|
||||
|
||||
* [ ] Implement normalization for CSAF VEX, OpenVEX, CycloneDX VEX.
|
||||
* [ ] Build product mapping library (CPE→purl, vendor tokens→purl families).
|
||||
* [ ] Implement signature verification (Ed25519/PKIX/DSSE) using Issuer Directory keys.
|
||||
* [ ] Implement trust weighting functions and configurable parameters.
|
||||
* [ ] Implement consensus algorithm with unit tests and property tests.
|
||||
* [ ] Materialize `vex_consensus` projection with indexes and idempotent workers.
|
||||
* [ ] Batch recompute on policy activation and Excitator deltas.
|
||||
|
||||
### APIs & Integrations
|
||||
|
||||
* [ ] `/vex/consensus` query, detail, simulate, export.
|
||||
* [ ] Policy Engine: consume consensus signals; add thresholds and precedence.
|
||||
* [ ] Vuln Explorer: show consensus chip and triage banners; deep link to Lens.
|
||||
|
||||
### Issuer Directory
|
||||
|
||||
* [ ] CRUD for issuers and keys, audit logs, RBAC.
|
||||
* [ ] Import common CSAF publishers; seed with sane defaults.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Build list grid with filters and saved views.
|
||||
* [ ] Quorum bar and Evidence table with signature icons and scope quality tags.
|
||||
* [ ] Conflict view for `DISPUTED`.
|
||||
* [ ] Simulation drawer integrated with Policy Studio.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella vex consensus list|show|simulate|export` with JSON/CSV.
|
||||
* [ ] Stable schemas; tests for piping and scripting.
|
||||
|
||||
### Observability/Perf
|
||||
|
||||
* [ ] Metrics, logs, traces as specified; dashboards.
|
||||
* [ ] Load tests at 10M consensus records/tenant; optimize indexes and caches.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author and cross‑link all docs listed in §5.
|
||||
* [ ] Add examples and screenshots to Console doc.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Normalization supports CSAF VEX, OpenVEX, CycloneDX VEX with product tree mapping to purls across npm, Maven, PyPI, Go, RPM/DEB.
|
||||
* Signature verification works and affects weights; unverifiable signatures do not crash flows.
|
||||
* Consensus outputs are reproducible, explainable, and queryable at scale.
|
||||
* Vuln Explorer displays consensus state and affects policy outcomes per thresholds.
|
||||
* Simulation reflects policy trust changes without side effects and returns rationale deltas.
|
||||
* CLI/API feature parity; evidence and quorum are exportable.
|
||||
* P95 performance budgets met; dashboards reflect health.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks and mitigations
|
||||
|
||||
* **Mapping errors (CPE→purl):** use conservative scope scores, cap weights, surface warnings, manual override hooks in Policy Studio.
|
||||
* **Malicious or mistaken issuer:** signature verification plus trust weighting and tenancy overrides.
|
||||
* **Evidence storms:** debounce recompute; batch and shard workers; backpressure and prioritization.
|
||||
* **User confusion with conflicting VEX:** clear conflict UI, rationale chains, suggested actions, and policy banners.
|
||||
* **Stale statements:** recency decay and expiry windows in policy.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit:** status mapping, comparators per ecosystem, trust weighting, threshold math.
|
||||
* **Property tests:** invariants such as monotonicity with added supporting evidence and idempotent recompute.
|
||||
* **Integration:** Excitator→Lens→Policy→Vuln Explorer pipeline with signed/unsigned, conflicting, and stale evidence.
|
||||
* **E2E Console:** list filters, detail with quorum, conflict workflows, export, simulation.
|
||||
* **Security:** RBAC on Issuer Directory, CSRF for Console, signature verification path traversal guards.
|
||||
* **Performance:** cold/hot query latencies, recompute throughput, cache hit ratios.
|
||||
* **Determinism:** time‑travel snapshots reproduce prior consensus states.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Consensus, not replacement.** The Lens summarizes without erasing dissent.
|
||||
* **Trust is contextual.** Tenants must tune weights and thresholds to their environments.
|
||||
* **Proof over prose.** Every decision comes with math, provenance, and a rationale chain.
|
||||
* **Safety by design.** When in doubt, surface conflicts instead of silently suppressing.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,440 +0,0 @@
|
||||
Below is the “maximum documentation” bundle for Epic 8. It’s engineered to be pasted into your repo without turning into yet another unread wiki tomb. Slight sarcasm included to keep blood flowing.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 8: Advisory AI Assistant (summaries, conflict explain, remediation hints)
|
||||
|
||||
**Short name:** `Advisory AI`
|
||||
**Services touched:** Conseiller (Feedser), Excitator (Vexer), VEX Lens, SBOM Service, Policy Engine, Findings Ledger, Web API Gateway, Authority (authN/Z), Console (Web UI), CLI, Telemetry/Analytics
|
||||
**AOC ground rule:** Conseiller and Excitator aggregate but never merge or mutate source docs. Advisory AI produces derived summaries and plans with strict provenance and citations. No silent rewriting of evidence. Ever.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
Advisory AI is a tenant‑scoped, retrieval‑augmented assistant that turns noisy security advisories and VEX statements into three consumable artifacts:
|
||||
|
||||
1. **Advisory Summary**
|
||||
Condenses one or more advisories (CSAF, OSV, GHSA, vendor PDFs, distro notices) into a concise brief with key facts: affected ranges, exploit status, impact, known workarounds, fixed versions, and links. Always cites the exact sources and sections used.
|
||||
|
||||
2. **Conflict Explain**
|
||||
Explains why VEX statements or advisories disagree for a specific artifact and version. Uses the VEX Consensus Lens outputs and issuer trust model to produce a human‑readable, step‑by‑step explanation: who said what, where the product scoping diverges, and what policy thresholds caused the final state.
|
||||
|
||||
3. **Remediation Hints**
|
||||
Suggests practical next steps: upgrade paths compatible with your dependency graph, backports, config toggles, temporary policy suppressions, or compensating controls. Every hint is grounded in SBOM, environment, and policy. It ships as structured JSON plus a human summary, ready to paste into a ticket.
|
||||
|
||||
It lives in the Console as a side panel, in the CLI for batch runs, and via APIs for automation. It does not change scanner results or consensus on its own. Humans remain in charge. The machine does the skimming and the math so humans can keep the judgment and the coffee.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (brief)
|
||||
|
||||
Advisories are long, inconsistent, and sometimes contradictory. Teams waste cycles reconciling PDFs with package manifests. The assistant eliminates that sludge: fast summaries, explicit conflict explanations, and remediation hints that are actually applicable to your software, not to an imaginary ideal project from 2013.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Capabilities
|
||||
|
||||
* **Summaries**
|
||||
|
||||
* Input: one advisory or a bundle linked by the same advisory key (CVE, GHSA, vendor‑ID), product scope, and environment.
|
||||
* Output:
|
||||
|
||||
* 150 to 300 words summary
|
||||
* `AdvisorySummary JSON` (schema below)
|
||||
* Citations with paragraph anchors
|
||||
* Confidence label and coverage score (how much of the advisory set is represented)
|
||||
* **Conflict Explain**
|
||||
|
||||
* Input: `(artifact_id, purl, version, advisory_key)` tuple.
|
||||
* Output: narrative plus a structured breakdown of consensus math, issuer votes, product mapping mismatches, and the exact policy knobs that tipped the result.
|
||||
* **Remediation Hints**
|
||||
|
||||
* Input: same tuple plus SBOM context and environment.
|
||||
* Output: ranked list of remediation options with feasibility score, blast radius estimate (derived from dependency paths), effort class, and links to fixed versions. Includes “do nothing” when the VEX consensus is not affected.
|
||||
|
||||
### 3.2 System design
|
||||
|
||||
**Architecture diagram in words (because ASCII art is a crime):**
|
||||
|
||||
1. **Retrievers**
|
||||
|
||||
* Structured retriever over Conseiller’s normalized advisory fields.
|
||||
* Vector retriever over advisory text chunks with paragraph anchors.
|
||||
* VEX retriever over Excitator evidence and VEX Lens consensus.
|
||||
* SBOM retriever for purl, version, dependency paths, env flags.
|
||||
2. **Deterministic resolvers**
|
||||
|
||||
* Version comparators per ecosystem.
|
||||
* Range satisfaction checks.
|
||||
* Dependency path scorers and blast radius estimator.
|
||||
3. **Orchestrator**
|
||||
|
||||
* Task‑specific prompt templates for Summary, Conflict, Remediation.
|
||||
* Tool calls to deterministics (version check, graph crawl) with results injected into the prompt as structured context.
|
||||
* Strict token budgets and truncation rules to avoid model babble.
|
||||
4. **Models**
|
||||
|
||||
* Default: on‑prem inference container with mid‑sized model.
|
||||
* Optional: tenant‑enabled remote inference. Disabled by default.
|
||||
* Temperature locked low for summary and conflict. Slightly higher for remediation narrative phrasing. No creativity in facts.
|
||||
5. **Guardrails**
|
||||
|
||||
* Prompt injection defense by stripping or quarantining advisory text that tries to instruct the model.
|
||||
* Fact boundary tagger. The assistant must only state facts that appear in structured inputs or cited chunks.
|
||||
* Redaction of secrets before prompts.
|
||||
* Output validator checks: required JSON fields, numeric ranges, valid version strings.
|
||||
|
||||
### 3.3 Data contracts
|
||||
|
||||
**AdvisorySummary JSON**
|
||||
|
||||
```json
|
||||
{
|
||||
"advisory_key": "CVE-2025-12345",
|
||||
"sources": [
|
||||
{"id":"csaf:vendorA:2025-001","uri":"...","sections":["2.1","3.4"]},
|
||||
{"id":"osv:pkg:npm/lodash","uri":"...","sections":["affected","references"]}
|
||||
],
|
||||
"affected_ranges": [
|
||||
{"ecosystem":"npm","purl_family":"pkg:npm/lodash","introduced":"<4.17.15","fixed": "4.17.21"}
|
||||
],
|
||||
"exploit_status": "no_known_exploit | poc_public | exploited_in_the_wild | n/a",
|
||||
"impact": {"cvss":[{"vector":"CVSS:3.1/AV:N/...","score":7.5}], "cwes":["CWE-79"]},
|
||||
"workarounds": ["Disable feature X", "Set flag Y=false"],
|
||||
"fixed_versions": ["4.17.21"],
|
||||
"notes": "Vendor states not affected on platform Z due to build option W.",
|
||||
"coverage_score": 0.86,
|
||||
"generated_at": "2025-10-25T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**ConflictExplanation JSON**
|
||||
|
||||
```json
|
||||
{
|
||||
"tuple": {"artifact_id":"svc:checkout@1.9.0","purl":"pkg:npm/lodash@4.17.20","advisory_key":"CVE-2025-12345"},
|
||||
"consensus": {"state":"NOT_AFFECTED","confidence":0.82},
|
||||
"quorum": [
|
||||
{"issuer":"lodash-maintainers","status":"NOT_AFFECTED","weight":0.9,"sig":true,"justification":"component_not_present"},
|
||||
{"issuer":"vendorX-distro","status":"AFFECTED","weight":0.25,"sig":false,"justification":"generic"}
|
||||
],
|
||||
"policy_factors": {"na_threshold":1.0,"aff_threshold":0.6,"recency_decay_days":90},
|
||||
"mapping_issues": [{"kind":"cpe_to_purl","score":0.6,"detail":"CPE wildcard matched multiple purls"}],
|
||||
"explanation_steps": [
|
||||
"Exact purl match found for maintainer VEX; weight 0.9",
|
||||
"Distro advisory generic; scope score 0.5; effective weight 0.25",
|
||||
"NA threshold met. Result set to NOT_AFFECTED"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**RemediationPlan JSON**
|
||||
|
||||
```json
|
||||
{
|
||||
"tuple": {"artifact_id":"svc:checkout@1.9.0","purl":"pkg:npm/lodash@4.17.20","advisory_key":"CVE-2025-12345"},
|
||||
"options": [
|
||||
{
|
||||
"kind": "upgrade",
|
||||
"target_version": "4.17.21",
|
||||
"feasibility": 0.92,
|
||||
"blast_radius": {"direct_callers":3,"transitive_depth":2},
|
||||
"effort": "low | medium | high",
|
||||
"rationale": "Semver patch, no breaking APIs in release notes",
|
||||
"links": ["release_notes_uri"]
|
||||
},
|
||||
{
|
||||
"kind": "workaround",
|
||||
"action": "Set SAFE_MODE=true",
|
||||
"feasibility": 0.6,
|
||||
"blast_radius": {"feature_flags":["SAFE_MODE"]},
|
||||
"effort": "low",
|
||||
"rationale": "Vendor states mitigation reduces attack surface on feature X"
|
||||
}
|
||||
],
|
||||
"preferred": 0,
|
||||
"policy_effects": {"sla_days": 7, "severity_override": "medium_if_not_fixed"},
|
||||
"generated_at": "2025-10-25T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### 3.4 APIs
|
||||
|
||||
```
|
||||
POST /advisory/ai/summary
|
||||
{
|
||||
"advisory_key":"CVE-2025-12345",
|
||||
"artifact_id":"svc:checkout@1.9.0",
|
||||
"purl":"pkg:npm/lodash@4.17.20",
|
||||
"sources":["csaf:*","osv:*"], // optional filters
|
||||
"policy_version":"1.3.0",
|
||||
"lang":"en"
|
||||
}
|
||||
-> 200 { "summary_text":"...", "summary": {AdvisorySummary}, "citations":[...] }
|
||||
|
||||
POST /advisory/ai/conflict
|
||||
{
|
||||
"artifact_id":"svc:checkout@1.9.0",
|
||||
"purl":"pkg:npm/lodash@4.17.20",
|
||||
"advisory_key":"CVE-2025-12345",
|
||||
"policy_version":"1.3.0"
|
||||
}
|
||||
-> 200 { "explanation_text":"...", "explanation": {ConflictExplanation} }
|
||||
|
||||
POST /advisory/ai/remediation
|
||||
{
|
||||
"artifact_id":"svc:checkout@1.9.0",
|
||||
"purl":"pkg:npm/lodash@4.17.20",
|
||||
"advisory_key":"CVE-2025-12345",
|
||||
"policy_version":"1.3.0",
|
||||
"max_options":5,
|
||||
"strategy_preference":["upgrade","backport","workaround"]
|
||||
}
|
||||
-> 200 { "plan_text":"...", "plan": {RemediationPlan} }
|
||||
|
||||
POST /advisory/ai/batch
|
||||
{
|
||||
"items":[ {tuple}, {tuple}, ... ],
|
||||
"task":"summary | conflict | remediation",
|
||||
"policy_version":"1.3.0"
|
||||
}
|
||||
-> 207 multi-status
|
||||
```
|
||||
|
||||
Status codes: `400` invalid, `403` RBAC, `404` missing evidence, `409` conflict lock, `422` output validation failed, `429` rate limit.
|
||||
|
||||
### 3.5 Console (Web UI)
|
||||
|
||||
* Surfaces:
|
||||
|
||||
* Vuln Explorer detail: “Advisory AI” side panel with 3 tabs: Summary, Conflict, Remediation.
|
||||
* Consensus Lens detail: prominent “Explain conflict” button.
|
||||
* Policy Studio sim: “Show effect on assistant output” preview.
|
||||
|
||||
* UX details:
|
||||
|
||||
* Citations are footnotes with hover to show source paragraph.
|
||||
* “Copy as ticket” produces Markdown and JSON.
|
||||
* Plan options show feasibility bar, blast radius chips, and required approvals per policy.
|
||||
* Injection warnings appear if advisory text included unsafe instructions.
|
||||
|
||||
* A11y: ARIA tags for tabs, keyboard shortcuts `G` to generate, `R` to refresh, `C` to copy JSON.
|
||||
|
||||
### 3.6 CLI
|
||||
|
||||
```
|
||||
stella advise summarize --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0 --json
|
||||
stella advise explain --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0
|
||||
stella advise remediate --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0 --strategy upgrade,workaround --out plan.json
|
||||
stella advise batch --file tuples.json --task remediation --policy 1.3.0
|
||||
```
|
||||
|
||||
Exit codes: `0` ok, `2` invalid args, `4` not found, `5` denied, `7` validation fail.
|
||||
|
||||
### 3.7 RBAC and security
|
||||
|
||||
* Roles:
|
||||
|
||||
* Viewer can run summaries and read explanations.
|
||||
* Operator can run remediation and export plans.
|
||||
* Admin can toggle model endpoints and guardrail settings.
|
||||
|
||||
* Defaults:
|
||||
|
||||
* Remote model calls disabled.
|
||||
* Redaction on.
|
||||
* Prompt logging anonymized.
|
||||
* Outputs stored as derived artifacts with TTL (default 30 days) unless pinned to a ticket.
|
||||
|
||||
### 3.8 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `advisory_ai_latency_ms` by task type.
|
||||
* `advisory_ai_guardrail_blocks_total`.
|
||||
* `advisory_ai_output_validation_fail_total`.
|
||||
* `advisory_ai_citation_coverage` gauge.
|
||||
* Traces: retriever spans, tool calls, model inference, validator.
|
||||
* Logs: include tuple key, token usage, truncation events, and guardrail outcomes.
|
||||
|
||||
### 3.9 Performance
|
||||
|
||||
* Targets:
|
||||
|
||||
* P95 under 1.5 s for Summary and Conflict with warm caches.
|
||||
* P95 under 2.5 s for Remediation on medium SBOMs (1000 packages).
|
||||
* Batch throughput 10 tuples per second per worker.
|
||||
|
||||
### 3.10 Edge cases
|
||||
|
||||
* Advisory missing fixed versions: produce workaround‑only plan and mark feasibility low.
|
||||
* Conflicts with near‑tie weights: declare “DISPUTED” and require human approval, no auto plan preferred.
|
||||
* Exotic version schemes: fallback to string compare with warning and feasibility cap.
|
||||
* Private packages: no public release notes. Prefer internal changelog links if attached to artifact metadata.
|
||||
* Multi‑env differences: render per‑env deltas when policy knobs differ (dev vs prod).
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services and components
|
||||
|
||||
* **New:** `src/AdvisoryAI/StellaOps.AdvisoryAI`
|
||||
|
||||
* `retriever/` wrappers for Conseiller, Excitator, VEX Lens, SBOM.
|
||||
* `deterministic/` version and path analyzers.
|
||||
* `orchestrator/` task routers and prompt builders.
|
||||
* `guardrails/` injection, redaction, output validator.
|
||||
* `api/` REST endpoints and schema enforcement.
|
||||
|
||||
* **Updates:**
|
||||
|
||||
* Conseiller: expose paragraph‑level anchors for advisories.
|
||||
* Excitator: expose justifications and product trees in normalized form.
|
||||
* VEX Lens: stable API for quorum and rationale.
|
||||
* SBOM Service: efficient path queries and versions timeline per purl.
|
||||
|
||||
### 4.2 Packaging
|
||||
|
||||
* Container images:
|
||||
|
||||
* `stella/advisory-ai:<<version>>`
|
||||
* `stella/inference:<<version>>` (if using on‑prem model)
|
||||
* Helm values to toggle remote inference and GPU.
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
* Phase 1: Summary and Conflict read‑only.
|
||||
* Phase 2: Remediation with “Copy as ticket”.
|
||||
* Phase 3: Batch APIs, CLI, and Policy Studio simulation hooks.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Create or update the following files. Each doc ends with the imposed rule statement.
|
||||
|
||||
1. `/docs/advisory-ai/overview.md`
|
||||
What it is, capabilities, guardrails, AOC alignment, RBAC.
|
||||
|
||||
2. `/docs/advisory-ai/architecture.md`
|
||||
RAG design, retrievers, orchestrator, deterministics, models, caching.
|
||||
|
||||
3. `/docs/advisory-ai/api.md`
|
||||
Endpoint specs, payload schemas, error codes, examples.
|
||||
|
||||
4. `/docs/advisory-ai/console.md`
|
||||
Screens, actions, a11y, how citations work, copy‑as‑ticket.
|
||||
|
||||
5. `/docs/advisory-ai/cli.md`
|
||||
Command usage, exit codes, piping examples.
|
||||
|
||||
6. `/docs/policy/assistant-parameters.md`
|
||||
Temperature, max tokens, plan ranking weights, TTLs.
|
||||
|
||||
7. `/docs/security/assistant-guardrails.md`
|
||||
Redaction rules, injection defense, output validation, logging.
|
||||
|
||||
8. `/docs/sbom/remediation-heuristics.md`
|
||||
Feasibility scoring, blast radius, effort classes.
|
||||
|
||||
9. `/docs/runbooks/assistant-ops.md`
|
||||
Warmup, cache priming, model outages, scaling, on‑call steps.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend core
|
||||
|
||||
* [ ] Implement structured and vector retrievers with paragraph anchors from Conseiller.
|
||||
* [ ] Implement VEX retriever using Lens APIs with caching.
|
||||
* [ ] Build deterministics: ecosystem comparators, range checks, dependency path scorer.
|
||||
* [ ] Implement orchestrator with task‑specific templates and tool call pipeline.
|
||||
* [ ] Implement guardrails and validators with hard failure on invalid JSON.
|
||||
* [ ] Add RBAC to endpoints and anonymized prompt logging.
|
||||
* [ ] Add caching layer with tuple‑keyed entries and policy version scoping.
|
||||
|
||||
### Integrations
|
||||
|
||||
* [ ] Conseiller: expose advisory chunk API and metadata needed for citations.
|
||||
* [ ] Excitator: ensure justifications and product trees are queryable.
|
||||
* [ ] VEX Lens: add “policy factors” endpoint for explanation rendering.
|
||||
* [ ] SBOM Service: implement `GET /sbom/paths?purl=...` and version timeline.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Build Advisory AI panel with 3 tabs and citation tooltips.
|
||||
* [ ] Implement “Copy as ticket” (Markdown + JSON) and download.
|
||||
* [ ] Add injection warning banner when triggered.
|
||||
* [ ] Respect a11y requirements and shortcuts.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella advise summarize|explain|remediate|batch` with JSON output.
|
||||
* [ ] Add `--out` option to save plans and summaries.
|
||||
* [ ] Tests for piping and jq workflows.
|
||||
|
||||
### Observability
|
||||
|
||||
* [ ] Emit metrics and traces listed in §3.8.
|
||||
* [ ] Dashboards: latency, guardrail blocks, validation fails, coverage.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Write all files in §5 with examples and screenshots.
|
||||
* [ ] Cross‑link to VEX Lens and Vulnerability Explorer docs.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Summaries cite specific source sections and reflect affected ranges and fixed versions correctly for at least 95% of a validation set.
|
||||
* Conflict explanations enumerate issuers, weights, justifications, mapping issues, and policy thresholds that caused the consensus state.
|
||||
* Remediation plans output at least one feasible option when a fixed version exists and correctly flag “no public fix” cases.
|
||||
* JSON schemas validate for all outputs.
|
||||
* Console shows the panel with citations, copy‑as‑ticket, and a11y passes.
|
||||
* CLI produces identical JSON to API responses.
|
||||
* Guardrails block injection attempts and redact secrets in prompts.
|
||||
* P95 latency targets are met with warm caches.
|
||||
* No mutation of raw advisory or VEX evidence occurs anywhere in the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks and mitigations
|
||||
|
||||
* **Prompt injection in advisory text.** Strip instructions, sandbox chunks, and highlight to user when removed.
|
||||
* **Hallucinated facts.** Hard validation requires facts to appear in structured inputs or cited text. Fail closed if not provable.
|
||||
* **Mapping errors produce bad hints.** Depend on SBOM Graph and VEX Lens scope scores; cap feasibility when scope is weak.
|
||||
* **Model outage.** Degrade to deterministic summaries (shorter, but accurate).
|
||||
* **Privacy concerns.** Default on‑prem inference, remote endpoints opt‑in with clear flags and audit logs.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit:** version comparators, range checks, feasibility scoring, output validators.
|
||||
* **Golden files:** advisory sets mapped to expected summaries and plans; diff on each build.
|
||||
* **Injection tests:** adversarial advisories with “ignore prior instructions” payloads must be neutralized.
|
||||
* **Integration:** Conseiller→Advisory AI→Console loop, with VEX Lens conflicts and SBOM graph lookups.
|
||||
* **E2E:** generate summary, explanation, and plan for representative ecosystems (npm, Maven, PyPI, Go, RPM/DEB).
|
||||
* **Perf:** soak tests with 5k tuples batch; observe cache hit ratios and P95.
|
||||
* **A11y:** keyboard navigation and screen reader labels.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Facts first.** If it is not in structured inputs or citations, it does not exist.
|
||||
* **Explain everything.** Humans should see exactly why the tool said what it said.
|
||||
* **Helpful by default.** Plans must consider the real dependency graph and environment, not fantasy.
|
||||
* **No silent merges.** Evidence is sacred. Summaries and plans are separate, auditable derivatives.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1,523 +0,0 @@
|
||||
Below is the “maximum documentation” bundle for Epic 9. Paste it into your repo and pretend the ingestion chaos was always under control.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 9: Source & Job Orchestrator Dashboard
|
||||
|
||||
**Short name:** `Orchestrator Dashboard`
|
||||
**Primary service:** `orchestrator` (scheduler, queues, rate‑limits, job state)
|
||||
**Surfaces:** Console (Web UI), CLI, Web API
|
||||
**Touches:** Conseiller (Feedser), Excitator (Vexer), VEX Consensus Lens, SBOM Service, Policy Engine, Findings Ledger, Authority (authN/Z), Telemetry/Analytics, Object Storage, Relational DB, Message Bus
|
||||
|
||||
**AOC ground rule:** Conseiller and Excitator aggregate but never merge. The orchestrator schedules, tracks and recovers jobs; it does not transform evidence beyond transport and storage. No “smart” merging in flight.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The Source & Job Orchestrator Dashboard is the control surface for every data source and pipeline run across StellaOps. It gives operators:
|
||||
|
||||
* Live health of all advisory/VEX/SBOM sources and derived jobs.
|
||||
* End‑to‑end pipeline visibility as DAGs and timelines.
|
||||
* Controls for pausing, backfilling, replaying, throttling and retrying.
|
||||
* Error pattern analysis, rate‑limit observability and backpressure insights.
|
||||
* Provenance and audit trails from initial fetch through parse, normalize, index and policy evaluation.
|
||||
|
||||
The dashboard sits over the `orchestrator` service, which maintains job state, schedules runs, enforces quotas and rate limits, and collects metrics from worker pools embedded in Conseiller, Excitator, SBOM and related services.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (brief)
|
||||
|
||||
Ingestion breaks quietly and then loudly. Without a unified control plane, you learn about it from angry users or empty indexes. This dashboard shortens incident MTTR, enables safe backfills, and makes compliance reviewers stop sending emails with twelve attachments and one emoji.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Capabilities
|
||||
|
||||
* **Source registry**
|
||||
|
||||
* Register, tag and version connectors (OSV, GHSA, CSAF endpoints, vendor PDF scrapers, distro feeds, RSS, S3 drops, internal registries).
|
||||
* Store connection details, secrets (via KMS), rate‑limit policy, schedules, and ownership metadata.
|
||||
* Validate and “test connection” safely.
|
||||
|
||||
* **Job orchestration**
|
||||
|
||||
* Create DAGs composed of job types: `fetch`, `parse`, `normalize`, `dedupe`, `index`, `consensus_compute`, `policy_eval`, `crosslink`, `sbom_ingest`, `sbom_index`.
|
||||
* Priorities, queues, concurrency caps, exponential backoff, circuit breakers.
|
||||
* Idempotency keys and output artifact hashing to avoid duplicate work.
|
||||
* Event‑time watermarks for backfills without double counting.
|
||||
|
||||
* **Observability & control**
|
||||
|
||||
* Gantt timeline and real‑time DAG view with critical path highlighting.
|
||||
* Backpressure and queue depth heatmaps.
|
||||
* Error clustering by class (HTTP 429, TLS, schema mismatch, parse failure, upstream 5xx).
|
||||
* Per‑source SLOs and SLA budgets with burn‑rate alerts.
|
||||
* One‑click actions: retry, replay range, pause/resume, throttle/unthrottle, reroute to canary workers.
|
||||
|
||||
* **Provenance & audit**
|
||||
|
||||
* Immutable run ledger linking input artifact → every job → output artifact.
|
||||
* Schema version tracking and drift detection.
|
||||
* Operator actions recorded with reason and ticket reference.
|
||||
|
||||
* **Safety**
|
||||
|
||||
* Secret redaction everywhere.
|
||||
* Tenant isolation at API, queue and storage layers.
|
||||
* AOC: no in‑flight merges of advisory or VEX content.
|
||||
|
||||
### 3.2 Core architecture
|
||||
|
||||
* **orchestrator (service)**
|
||||
|
||||
* Maintains job state in Postgres (`sources`, `runs`, `jobs`, `artifacts`, `dag_edges`, `quotas`, `schedules`).
|
||||
* Publishes work to a message bus (e.g., `topic.jobs.ready.<queue>`).
|
||||
* Distributed token‑bucket rate limiter per source/tenant/host.
|
||||
* Watchdog for stuck jobs and circuit breakers for flapping sources.
|
||||
* Watermark manager for backfills (event‑time windows).
|
||||
|
||||
* **worker SDK**
|
||||
|
||||
* Lightweight library embedded in Conseiller/Excitator/SBOM workers to:
|
||||
|
||||
* Claim work, heartbeat, update progress, report metrics.
|
||||
* Emit artifact metadata and checksums.
|
||||
* Enforce idempotency via orchestrator‑supplied key.
|
||||
|
||||
* **object store**
|
||||
|
||||
* Raw payloads and intermediate artifacts organized by schema and hash:
|
||||
|
||||
* `advisory/raw/<source_id>/<event_time>/<sha256>.json|pdf`
|
||||
* `advisory/normalized/<schema_ver>/<hash>.json`
|
||||
* `vex/raw|normalized/...`
|
||||
* `sbom/raw|graph/...`
|
||||
|
||||
* **web API**
|
||||
|
||||
* CRUD for sources, runs, jobs, schedules, quotas.
|
||||
* Control actions (retry, cancel, pause, backfill).
|
||||
* Streaming updates via WebSocket/SSE for the Console.
|
||||
|
||||
* **console**
|
||||
|
||||
* React app consuming Orchestrator APIs, rendering DAGs, timelines, health charts and action panels with RBAC.
|
||||
|
||||
### 3.3 Data model (selected tables)
|
||||
|
||||
* `sources`
|
||||
|
||||
* `id`, `kind` (`advisory|vex|sbom|internal`), `subtype` (e.g., `osv`, `ghsa`, `csaf`, `vendor_pdf`), `display_name`, `owner_team`, `schedule_cron`, `rate_policy`, `enabled`, `secrets_ref`, `tags`, `schema_hint`, `created_at`, `updated_at`.
|
||||
|
||||
* `runs`
|
||||
|
||||
* `id`, `source_id`, `trigger` (`schedule|manual|event|backfill`), `window_start`, `window_end`, `state`, `started_at`, `finished_at`, `stats_json`.
|
||||
|
||||
* `jobs`
|
||||
|
||||
* `id`, `run_id`, `type`, `queue`, `priority`, `state` (`pending|running|succeeded|failed|canceled|deadletter`), `attempt`, `max_attempt`, `idempotency_key`, `input_artifact_id`, `output_artifact_id`, `worker_id`, `created_at`, `started_at`, `finished_at`, `error_class`, `error_message`, `metrics_json`.
|
||||
|
||||
* `dag_edges`
|
||||
|
||||
* `from_job_id`, `to_job_id`, `edge_kind` (`success_only|always`).
|
||||
|
||||
* `artifacts`
|
||||
|
||||
* `id`, `kind` (`raw|normalized|index|consensus`), `schema_ver`, `hash`, `uri`, `bytes`, `meta_json`, `created_at`.
|
||||
|
||||
* `quotas`
|
||||
|
||||
* `tenant_id`, `resource` (`requests_per_min`, `concurrent_jobs`), `limit`, `window_sec`.
|
||||
|
||||
* `schedules`
|
||||
|
||||
* Per‑source cron plus jitter, timezone, blackout windows.
|
||||
|
||||
### 3.4 Job lifecycle
|
||||
|
||||
1. **Plan**
|
||||
Scheduler creates a `run` for a source and plans a DAG: e.g., `fetch → parse → normalize → dedupe → index → policy_eval` (advisory) or `fetch → parse → normalize → consensus_compute` (VEX).
|
||||
|
||||
2. **Enqueue**
|
||||
Ready nodes become `jobs` with queue, priority, idempotency key and optional rate‑limit tokens reserved.
|
||||
|
||||
3. **Execute**
|
||||
Worker claims job, heartbeats every N seconds. Output artifacts are stored and linked. Failures are classified and retried with exponential backoff and jitter, up to `max_attempt`.
|
||||
|
||||
4. **Complete**
|
||||
Downstream nodes unblock. On run completion, orchestrator computes SLO deltas and emits run summary.
|
||||
|
||||
5. **Dead‑letter**
|
||||
Jobs exceeding attempts move to a DLQ with structured context and suggested remediation.
|
||||
|
||||
### 3.5 Scheduling, backpressure, rate‑limits
|
||||
|
||||
* **Token bucket** per `{tenant, source.host}` with adaptive refill if upstream 429/503 seen.
|
||||
* **Concurrency caps** per source and per job type to avoid thundering herd.
|
||||
* **Backpressure signals** from queue depth, worker CPU, and upstream error rates; scheduler reduces inflight issuance accordingly.
|
||||
* **Backfills** use event‑time windows with immutable watermarks to avoid re‑processing.
|
||||
* **Blackout windows** for vendor maintenance periods.
|
||||
|
||||
### 3.6 APIs
|
||||
|
||||
```
|
||||
POST /orchestrator/sources
|
||||
GET /orchestrator/sources?kind=&tag=&q=
|
||||
GET /orchestrator/sources/{id}
|
||||
PATCH /orchestrator/sources/{id}
|
||||
POST /orchestrator/sources/{id}/actions:test|pause|resume|sync-now
|
||||
POST /orchestrator/sources/{id}/backfill { "from":"2024-01-01", "to":"2024-03-01" }
|
||||
|
||||
GET /orchestrator/runs?source_id=&state=&from=&to=
|
||||
GET /orchestrator/runs/{run_id}
|
||||
GET /orchestrator/runs/{run_id}/dag
|
||||
POST /orchestrator/runs/{run_id}/cancel
|
||||
|
||||
GET /orchestrator/jobs?state=&type=&queue=&source_id=
|
||||
GET /orchestrator/jobs/{job_id}
|
||||
POST /orchestrator/jobs/{job_id}/actions:retry|cancel|prioritize
|
||||
|
||||
GET /orchestrator/metrics/overview
|
||||
GET /orchestrator/errors/top?window=1h
|
||||
GET /orchestrator/quotas
|
||||
PATCH /orchestrator/quotas/{tenant_id}
|
||||
WS /orchestrator/streams/updates
|
||||
```
|
||||
|
||||
### 3.7 Console (Web UI)
|
||||
|
||||
* **Overview**
|
||||
|
||||
* KPI tiles: sources healthy, runs in progress, queue depth, error rate, burn‑rate to SLO.
|
||||
* Heatmap of source health by last 24h success ratio.
|
||||
|
||||
* **Sources**
|
||||
|
||||
* Grid with filters, inline status (active, paused, throttled), next run eta, last error class.
|
||||
* Detail panel: config, secrets status (redacted), schedule, rate limits, ownership, run history, action buttons.
|
||||
|
||||
* **Runs**
|
||||
|
||||
* Timeline (Gantt) with critical path, duration distribution, and per‑stage breakdown.
|
||||
* Run detail: DAG view with node metrics, artifacts, logs, action menu (cancel).
|
||||
|
||||
* **Jobs**
|
||||
|
||||
* Live table with state filters and “tail” view.
|
||||
* Job detail: payload preview (redacted), worker, attempts, stack traces, linked artifacts.
|
||||
|
||||
* **Errors**
|
||||
|
||||
* Clusters by class and signature, suggested remediations (pause source, lower concurrency, patch parser).
|
||||
|
||||
* **Queues & Backpressure**
|
||||
|
||||
* Per‑queue depth, service rate, inflight, age percentiles.
|
||||
* Rate‑limit tokens graphs per source host.
|
||||
|
||||
* **Controls**
|
||||
|
||||
* Backfill wizard with event‑time preview and safety checks.
|
||||
* Canary routing: route 5% of next 100 runs to a new worker pool.
|
||||
|
||||
* **A11y**
|
||||
|
||||
* Keyboard nav, ARIA roles for DAG nodes, live regions for updates, color‑blind friendly graphs.
|
||||
|
||||
### 3.8 CLI
|
||||
|
||||
```
|
||||
stella orch sources list --kind advisory --tag prod
|
||||
stella orch sources add --file source.yaml
|
||||
stella orch sources test <source-id>
|
||||
stella orch sources pause <source-id> # or resume
|
||||
stella orch sources sync-now <source-id>
|
||||
stella orch sources backfill <source-id> --from 2024-01-01 --to 2024-03-01
|
||||
|
||||
stella orch runs list --source <id> --state running
|
||||
stella orch runs show <run-id> --dag
|
||||
stella orch runs cancel <run-id>
|
||||
|
||||
stella orch jobs list --state failed --type parse --limit 100
|
||||
stella orch jobs retry <job-id>
|
||||
stella orch jobs cancel <job-id>
|
||||
stella orch jobs tail --queue normalize --follow
|
||||
|
||||
stella orch quotas get --tenant default
|
||||
stella orch quotas set --tenant default --concurrent-jobs 50 --rpm 1200
|
||||
```
|
||||
|
||||
Exit codes: `0` success, `2` invalid args, `4` not found, `5` denied, `7` precondition failed, `8` rate‑limited.
|
||||
|
||||
### 3.9 RBAC & security
|
||||
|
||||
* **Roles**
|
||||
|
||||
* `Orch.Viewer`: read‑only sources/runs/jobs/metrics.
|
||||
* `Orch.Operator`: perform actions on sources and jobs, launch backfills.
|
||||
* `Orch.Admin`: manage quotas, schedules, connector versions, and delete sources.
|
||||
|
||||
* **Secrets**
|
||||
|
||||
* Stored only as references to your KMS; never persisted in cleartext.
|
||||
* Console shows redact badges and last rotated timestamp.
|
||||
|
||||
* **Tenancy**
|
||||
|
||||
* Source, run, job rows scoped by tenant id.
|
||||
* Queue names and token buckets namespaced per tenant.
|
||||
|
||||
* **Compliance**
|
||||
|
||||
* Full audit log for every operator action with “reason” and optional ticket link.
|
||||
* Exportable run ledger for audits.
|
||||
|
||||
### 3.10 Observability
|
||||
|
||||
* **Metrics (examples)**
|
||||
|
||||
* `orch_jobs_inflight{type,queue}`
|
||||
* `orch_jobs_latency_ms{type,percentile}`
|
||||
* `orch_rate_tokens_available{source}`
|
||||
* `orch_error_rate{source,error_class}`
|
||||
* `orch_slo_burn_rate{source,slo}`
|
||||
* `orch_deadletter_total{source,type}`
|
||||
|
||||
* **Traces**
|
||||
|
||||
* Span per job with baggage: `run_id`, `source_id`, `artifact_id`.
|
||||
* Links across services to Conseiller/Excitator/SBOM workers.
|
||||
|
||||
* **Logs**
|
||||
|
||||
* Structured JSON with correlation ids, attempt numbers and redacted payload previews.
|
||||
|
||||
### 3.11 Performance targets
|
||||
|
||||
* Job dispatch P95 < 150 ms after dependency satisfied.
|
||||
* Scheduler loop P95 < 500 ms for 10k pending jobs.
|
||||
* Console live updates sub‑second at 1k events/sec per tenant.
|
||||
* Backfill throughput ≥ 200 jobs/sec per worker pool with zero dupes.
|
||||
|
||||
### 3.12 Edge cases & behaviors
|
||||
|
||||
* **Upstream 429 storms:** auto‑throttle, pause optional, recommend extended jitter.
|
||||
* **Schema drift:** parser moves job to DLQ with `error_class=schema_mismatch` and opens a change ticket via webhook.
|
||||
* **Flapping source:** circuit breaker opens after N consecutive failures; requires human “resume”.
|
||||
* **Clock skew:** watermark logic uses upstream event time; large skews flagged.
|
||||
* **Idempotency collisions:** new attempt yields no‑op if artifact hash already exists.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Modules (new and updated)
|
||||
|
||||
* New service: `src/Orchestrator/StellaOps.Orchestrator`
|
||||
|
||||
* `api/` REST + WS handlers
|
||||
* `scheduler/` run planner, DAG builder, watermark/backfill logic
|
||||
* `queues/` publisher and consumer abstractions
|
||||
* `ratelimit/` token bucket and adaptive controller
|
||||
* `state/` Postgres repositories and migrations
|
||||
* `audit/` action logging and export
|
||||
* `metrics/` Prometheus exporters
|
||||
* `security/` tenant scoping, KMS client, secret refs
|
||||
|
||||
* Worker SDKs:
|
||||
|
||||
* `src/Orchestrator/StellaOps.Orchestrator.WorkerSdk.Go` and `src/Orchestrator/StellaOps.Orchestrator.WorkerSdk.Python` with job claim, heartbeat, progress, artifact publish, and structured error reporting.
|
||||
|
||||
* Console:
|
||||
|
||||
* `console/apps/orch/` pages: Overview, Sources, Runs, Jobs, Errors, Queues.
|
||||
* `components/dag-view/`, `components/gantt/`, `components/health-heatmap/`.
|
||||
|
||||
* Updates to existing services:
|
||||
|
||||
* Conseiller/Excitator/SBOM workers adopt SDK and emit artifacts with schema/version/fingerprint.
|
||||
* VEX Lens exposes `consensus_compute` as a jobable operation.
|
||||
* Policy Engine exposes `policy_eval` as a job type for scheduled recalcs.
|
||||
|
||||
### 4.2 Packaging & deployment
|
||||
|
||||
* Containers:
|
||||
|
||||
* `stella/orchestrator:<ver>`
|
||||
* `stella/worker-sdk-examples:<ver>` for canary pools
|
||||
|
||||
* Helm values:
|
||||
|
||||
* Queues/topics, per‑tenant concurrency, rate‑limit defaults, WS replica count.
|
||||
* KMS integration secrets.
|
||||
|
||||
* Migrations:
|
||||
|
||||
* Flyway/Goose migrations for new tables and indexes.
|
||||
|
||||
### 4.3 Rollout strategy
|
||||
|
||||
* Phase 1: Read‑only dashboard fed by existing job tables; no controls.
|
||||
* Phase 2: Control actions enabled for non‑prod tenants.
|
||||
* Phase 3: Backfills and quota management, then GA.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Create/update the following, each ending with the imposed rule statement.
|
||||
|
||||
1. `/docs/orchestrator/overview.md`
|
||||
Concepts, roles, responsibilities, AOC alignment.
|
||||
|
||||
2. `/docs/orchestrator/architecture.md`
|
||||
Scheduler, DAGs, watermarks, queues, rate‑limits, data model.
|
||||
|
||||
3. `/docs/orchestrator/api.md`
|
||||
Endpoints, WebSocket events, error codes, examples.
|
||||
|
||||
4. `/docs/orchestrator/console.md`
|
||||
Screens, actions, a11y, live updates.
|
||||
|
||||
5. `/docs/orchestrator/cli.md`
|
||||
Commands, examples, exit codes, scripting patterns.
|
||||
|
||||
6. `/docs/orchestrator/run‑ledger.md`
|
||||
Provenance and audit export format.
|
||||
|
||||
7. `/docs/security/secrets‑handling.md`
|
||||
KMS references, redaction rules, operator hygiene.
|
||||
|
||||
8. `/docs/operations/orchestrator‑runbook.md`
|
||||
Common failures, backfill guide, circuit breakers, tuning.
|
||||
|
||||
9. `/docs/schemas/artifacts.md`
|
||||
Artifact kinds, schema versions, hashing, storage layout.
|
||||
|
||||
10. `/docs/slo/orchestrator‑slo.md`
|
||||
SLO definitions, measurement, alerting.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend (orchestrator)
|
||||
|
||||
* [ ] Stand up Postgres schemas and indices for sources, runs, jobs, dag_edges, artifacts, quotas, schedules.
|
||||
* [ ] Implement scheduler: DAG planner, dependency resolver, critical path computation.
|
||||
* [ ] Implement rate limiter with adaptive behavior on 429/503 and per‑tenant tokens.
|
||||
* [ ] Implement watermark/backfill manager with event‑time windows and idempotency keys.
|
||||
* [ ] Implement API endpoints + OpenAPI spec + request validation.
|
||||
* [ ] Implement WebSocket/SSE event stream for live updates.
|
||||
* [ ] Implement audit logging and export.
|
||||
* [ ] Implement dead‑letter store and replay.
|
||||
|
||||
### Worker SDKs and integrations
|
||||
|
||||
* [ ] Build Go/Python SDKs with claim/heartbeat/progress API.
|
||||
* [ ] Integrate SDK into Conseiller, Excitator, SBOM workers; ensure artifact emission with schema ver.
|
||||
* [ ] Add `consensus_compute` and `policy_eval` as job types with deterministic inputs/outputs.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Overview tiles and health heatmap.
|
||||
* [ ] Source list/detail with actions and config view.
|
||||
* [ ] Runs timeline (Gantt) and DAG visualization with node inspector.
|
||||
* [ ] Jobs “tail” with live updates and filters.
|
||||
* [ ] Errors clustering and suggested remediations.
|
||||
* [ ] Queues/backpressure dashboard.
|
||||
* [ ] Backfill wizard with safety checks.
|
||||
|
||||
### Observability
|
||||
|
||||
* [ ] Emit metrics listed in §3.10 and wire traces across services.
|
||||
* [ ] Dashboards: health, queue depth, error classes, burn‑rate, dispatch latency.
|
||||
* [ ] Alerts for SLO burn and circuit breaker opens.
|
||||
|
||||
### Security & RBAC
|
||||
|
||||
* [ ] Enforce tenant scoping on all endpoints; test leakage.
|
||||
* [ ] Wire KMS for secret refs and redact everywhere.
|
||||
* [ ] Implement `Orch.Viewer|Operator|Admin` roles and check in Console and API.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author all files in §5 with examples and screenshots.
|
||||
* [ ] Cross‑link from Conseiller/Excitator/SBOM pages to the dashboard docs.
|
||||
* [ ] Append imposed rule to each page.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Operators can: pause/resume a source, run “sync‑now,” initiate a backfill for a date range, and retry/cancel individual jobs from Console and CLI.
|
||||
* DAG and timeline reflect reality within 1 second of job state changes at P95.
|
||||
* Backfills do not create duplicate artifacts; idempotency proven by hash equality.
|
||||
* Rate limiter reduces 429s by ≥80% under simulated throttle tests.
|
||||
* Audit log includes who/when/why for every operator action.
|
||||
* Provenance ledger exports a complete chain for any artifact.
|
||||
* RBAC prevents non‑admins from quota changes; tenancy isolation proven via automated tests.
|
||||
* SLO dashboard shows burn‑rate and triggers alerts under injected failure.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks & mitigations
|
||||
|
||||
* **Orchestrator becomes a single bottleneck.**
|
||||
Horizontal scale stateless workers; DB indexes tuned; job state updates batched; cache hot paths.
|
||||
|
||||
* **Secret spillage.**
|
||||
Only KMS references stored; aggressive redaction; log scrubbing in SDK.
|
||||
|
||||
* **Over‑eager backfills overwhelm upstream.**
|
||||
Enforce per‑source quotas and sandbox previews; dry‑run backfills first.
|
||||
|
||||
* **Schema drift silently corrupts normalization.**
|
||||
Hard‑fail on mismatch; DLQ with clear signatures; schema registry gating.
|
||||
|
||||
* **Flapping sources cause alert fatigue.**
|
||||
Circuit breaker with cool‑down and deduped alerts; error budget policy.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit**
|
||||
Scheduler DAG building, topological sort, backoff math, token bucket, watermark math.
|
||||
|
||||
* **Integration**
|
||||
Orchestrator ↔ worker SDK, artifact store wiring, DLQ replay, audit pipeline.
|
||||
|
||||
* **Chaos**
|
||||
Inject 429 storms, packet loss, worker crashes; verify throttling and recovery.
|
||||
|
||||
* **Backfill**
|
||||
Simulate overlapping windows and verify idempotency and watermark correctness.
|
||||
|
||||
* **Perf**
|
||||
10k concurrent jobs: dispatch latency, DB contention, WebSocket fan‑out.
|
||||
|
||||
* **Security**
|
||||
Multi‑tenant isolation tests; KMS mock tests for secret access; RBAC matrix.
|
||||
|
||||
* **UX/A11y**
|
||||
Screen reader labels on DAG, keyboard navigation, live region updates.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Make the invisible visible.** Pipelines should be legible at a glance.
|
||||
* **Prefer reproducibility to heroics.** Idempotency and provenance over “we think it ran.”
|
||||
* **Safeguards before speed.** Throttle first, retry thoughtfully, never melt upstreams.
|
||||
* **No silent merges.** Evidence remains immutable; transformations are explicit, logged and reversible.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
@@ -1244,7 +1244,7 @@ Generated from SPRINTS.md and module TASKS.md files on 2025-10-19. Waves cluster
|
||||
5. [TODO] DOCS-POLICY-20-005 — `/docs/api/policy.md` endpoints + schemas.
|
||||
• Prereqs: WEB-POLICY-20-001 (Wave 7)
|
||||
• Current: TODO
|
||||
6. [TODO] DOCS-POLICY-20-006 — `/docs/cli/policy.md` with command usage.
|
||||
6. [TODO] DOCS-POLICY-20-006 — `/docs/modules/cli/guides/policy.md` with command usage.
|
||||
• Prereqs: CLI-POLICY-20-002 (Wave 7)
|
||||
• Current: TODO
|
||||
7. [TODO] DOCS-POLICY-20-007 — `/docs/ui/policy-editor.md` flows + screenshots.
|
||||
|
||||
@@ -535,9 +535,9 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 35 | EPDR Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md | TODO | Scanner EPDR Guild | SCANNER-ANALYZERS-LANG-11-001 | Build entrypoint resolver (identity + environment profiles) and emit normalized entrypoint records. |
|
||||
| Sprint 35 | EPDR Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md | TODO | Scanner EPDR Guild | SCANNER-ANALYZERS-LANG-11-002 | Static IL/reflection/ALC heuristics producing dependency edges with reason codes and confidence. |
|
||||
| Sprint 35 | EPDR Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md | TODO | Scanner EPDR Guild, Signals Guild | SCANNER-ANALYZERS-LANG-11-003 | Runtime loader/PInvoke signal ingestion merged with static/declared edges (confidence & explain). |
|
||||
| Sprint 35 | Export Center Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-35-001 | Author `/docs/export-center/overview.md` with purpose, profiles, security, and imposed rule reminder. |
|
||||
| Sprint 35 | Export Center Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-35-002 | Author `/docs/export-center/architecture.md` detailing service components, adapters, manifests, signing, and distribution. |
|
||||
| Sprint 35 | Export Center Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-35-003 | Publish `/docs/export-center/profiles.md` covering schemas, examples, and compatibility. |
|
||||
| Sprint 35 | Export Center Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-35-001 | Author `/docs/modules/export-center/overview.md` with purpose, profiles, security, and imposed rule reminder. |
|
||||
| Sprint 35 | Export Center Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-35-002 | Author `/docs/modules/export-center/architecture.md` detailing service components, adapters, manifests, signing, and distribution. |
|
||||
| Sprint 35 | Export Center Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-35-003 | Publish `/docs/modules/export-center/profiles.md` covering schemas, examples, and compatibility. |
|
||||
| Sprint 35 | Export Center Phase 1 | ops/deployment/TASKS.md | TODO | Deployment Guild | DEPLOY-EXPORT-35-001 | Package exporter service/worker containers, Helm overlays (download-only), and rollout guide. |
|
||||
| Sprint 35 | Export Center Phase 1 | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-EXPORT-35-001 | Create exporter CI pipeline (lint/test/perf smoke), object storage fixtures, and initial Grafana dashboards. |
|
||||
| Sprint 35 | Export Center Phase 1 | src/ExportCenter/StellaOps.ExportCenter/TASKS.md | TODO | Exporter Service Guild | EXPORT-SVC-35-001 | Bootstrap exporter service, configuration, and migrations for export profiles/runs/inputs/distributions with tenant scopes. |
|
||||
@@ -553,9 +553,9 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 35 | Export Center Phase 1 | src/Web/StellaOps.Web/TASKS.md | TODO | BE-Base Platform Guild | WEB-EXPORT-35-001 | Route Export Center APIs through gateway with tenant scoping, viewer/operator scopes, and streaming downloads. |
|
||||
| Sprint 36 | EPDR Observations | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md | TODO | Scanner EPDR Guild, SBOM Service Guild | SCANNER-ANALYZERS-LANG-11-004 | Normalize EPDR output to Scanner observation writer (entrypoints + edges + env profiles). |
|
||||
| Sprint 36 | EPDR Observations | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md | TODO | Scanner EPDR Guild, QA Guild | SCANNER-ANALYZERS-LANG-11-005 | End-to-end fixtures/benchmarks covering publish modes, RIDs, trimming, NativeAOT with explain traces. |
|
||||
| Sprint 36 | Export Center Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-36-004 | Author `/docs/export-center/api.md` with endpoint examples and imposed rule note. |
|
||||
| Sprint 36 | Export Center Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-36-005 | Publish `/docs/export-center/cli.md` covering commands, scripts, verification, and imposed rule reminder. |
|
||||
| Sprint 36 | Export Center Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-36-006 | Write `/docs/export-center/trivy-adapter.md` detailing mappings, compatibility, and test matrix. |
|
||||
| Sprint 36 | Export Center Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-36-004 | Author `/docs/modules/export-center/api.md` with endpoint examples and imposed rule note. |
|
||||
| Sprint 36 | Export Center Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-36-005 | Publish `/docs/modules/export-center/cli.md` covering commands, scripts, verification, and imposed rule reminder. |
|
||||
| Sprint 36 | Export Center Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-36-006 | Write `/docs/modules/export-center/trivy-adapter.md` detailing mappings, compatibility, and test matrix. |
|
||||
| Sprint 36 | Export Center Phase 2 | ops/deployment/TASKS.md | TODO | Deployment Guild | DEPLOY-EXPORT-36-001 | Document registry credentials, OCI push workflows, and automation for export distributions. |
|
||||
| Sprint 36 | Export Center Phase 2 | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-EXPORT-36-001 | Integrate Trivy compatibility validation, OCI push smoke tests, and metrics dashboards for export throughput. |
|
||||
| Sprint 36 | Export Center Phase 2 | src/Cli/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI Guild | CLI-EXPORT-36-001 | Add `stella export distribute` (OCI/objstore), `run download --resume`, and status polling enhancements. |
|
||||
@@ -565,8 +565,8 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 36 | Export Center Phase 2 | src/ExportCenter/StellaOps.ExportCenter/TASKS.md | TODO | Exporter Service Guild | EXPORT-SVC-36-004 | Extend planner/run lifecycle for OCI/object storage distributions with retry + idempotency. |
|
||||
| Sprint 36 | Export Center Phase 2 | src/Orchestrator/StellaOps.Orchestrator/TASKS.md | TODO | Orchestrator Service Guild | ORCH-SVC-36-101 | Add distribution job follow-ups, retention metadata, and metrics for export runs. |
|
||||
| Sprint 36 | Export Center Phase 2 | src/Web/StellaOps.Web/TASKS.md | TODO | BE-Base Platform Guild | WEB-EXPORT-36-001 | Expose distribution endpoints (OCI/object storage) and manifest/provenance download proxies with RBAC. |
|
||||
| Sprint 37 | Export Center Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-37-001 | Publish `/docs/export-center/mirror-bundles.md` detailing layouts, deltas, encryption, imposed rule reminder. |
|
||||
| Sprint 37 | Export Center Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-37-002 | Publish `/docs/export-center/provenance-and-signing.md` covering manifests, attestation, verification. |
|
||||
| Sprint 37 | Export Center Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-37-001 | Publish `/docs/modules/export-center/mirror-bundles.md` detailing layouts, deltas, encryption, imposed rule reminder. |
|
||||
| Sprint 37 | Export Center Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-37-002 | Publish `/docs/modules/export-center/provenance-and-signing.md` covering manifests, attestation, verification. |
|
||||
| Sprint 37 | Export Center Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-37-003 | Publish `/docs/operations/export-runbook.md` for failures, tuning, capacity, with imposed rule note. |
|
||||
| Sprint 37 | Export Center Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-EXPORT-37-004 | Publish `/docs/security/export-hardening.md` covering RBAC, isolation, encryption, and imposed rule. |
|
||||
| Sprint 37 | Export Center Phase 3 | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-EXPORT-37-001 | Finalize dashboards/alerts for exports (failure, verify), retention jobs, and chaos testing harness. |
|
||||
@@ -631,7 +631,7 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 40 | Notifications Studio Phase 3 | src/Notifier/StellaOps.Notifier/TASKS.md | TODO | Notifications Service Guild | NOTIFY-SVC-40-003 | Harden security: signed ack links, webhook HMAC/IP allowlists, tenant isolation fuzzing, localization fallback. |
|
||||
| Sprint 40 | Notifications Studio Phase 3 | src/Notifier/StellaOps.Notifier/TASKS.md | TODO | Notifications Service Guild | NOTIFY-SVC-40-004 | Finalize observability (incident metrics, escalation latency) and chaos tests for channel outages. |
|
||||
| Sprint 40 | Notifications Studio Phase 3 | src/Web/StellaOps.Web/TASKS.md | TODO | BE-Base Platform Guild | WEB-NOTIFY-40-001 | Expose escalation, localization, channel health endpoints and verification of signed links. |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-CLI-41-001 | Publish `/docs/cli/overview.md`, `/cli/configuration.md`, `/cli/output-and-exit-codes.md` (with imposed rule). |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | docs/TASKS.md | TODO | Docs Guild | DOCS-CLI-41-001 | Publish `/docs/modules/cli/guides/overview.md`, `/cli/configuration.md`, `/cli/output-and-exit-codes.md` (with imposed rule). |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | ops/deployment/TASKS.md | TODO | Deployment Guild | DEPLOY-CLI-41-001 | Package CLI release artifacts (tarballs, completions, container image) with distribution docs. |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-CLI-41-001 | Establish CLI build pipeline (multi-platform binaries, SBOM, checksums) and parity matrix CI enforcement. |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | src/Authority/StellaOps.Authority/TASKS.md | TODO | Authority Core & Security Guild | AUTH-PACKS-41-001 | Define CLI SSO scopes and Packs (`Packs.Read/Write/Run/Approve`) roles; update discovery/offline defaults. |
|
||||
@@ -641,7 +641,7 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | src/Orchestrator/StellaOps.Orchestrator/TASKS.md | TODO | Orchestrator Service Guild | ORCH-SVC-41-101 | Register `pack-run` job type, integrate logs/artifacts, expose pack run metadata. |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | src/PacksRegistry/StellaOps.PacksRegistry/TASKS.md | TODO | Packs Registry Guild | PACKS-REG-41-001 | Implement packs index API, signature verification, provenance storage, and RBAC. |
|
||||
| Sprint 41 | CLI Parity & Task Packs Phase 1 | src/TaskRunner/StellaOps.TaskRunner/TASKS.md | TODO | Task Runner Guild | TASKRUN-41-001 | Bootstrap Task Runner service, migrations, run API, local executor, approvals pause, artifact capture. |
|
||||
| Sprint 42 | CLI Parity & Task Packs Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-CLI-42-001 | Publish `/docs/cli/parity-matrix.md`, `/cli/commands/*.md`, `/docs/task-packs/spec.md` (imposed rule). |
|
||||
| Sprint 42 | CLI Parity & Task Packs Phase 2 | docs/TASKS.md | TODO | Docs Guild | DOCS-CLI-42-001 | Publish `/docs/modules/cli/guides/parity-matrix.md`, `/cli/commands/*.md`, `/docs/task-packs/spec.md` (imposed rule). |
|
||||
| Sprint 42 | CLI Parity & Task Packs Phase 2 | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-CLI-42-001 | Add CLI golden output tests, parity diff automation, and pack run CI harness. |
|
||||
| Sprint 42 | CLI Parity & Task Packs Phase 2 | src/Cli/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI Guild | CLI-PACKS-42-001 | Implement Task Pack CLI commands (`pack plan/run/push/pull/verify`) with plan/simulate engine and expression sandbox. |
|
||||
| Sprint 42 | CLI Parity & Task Packs Phase 2 | src/Cli/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI Guild | CLI-PARITY-41-001..002 | Close parity gaps for Notifications, Policy Studio advanced features, SBOM graph, Vuln Explorer; parity matrix green. |
|
||||
@@ -695,7 +695,7 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 48 | Authority-Backed Scopes & Tenancy Phase 2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | TODO | Policy Guild | POLICY-TEN-48-001 | Add `tenant_id`/`project_id` to policy data, enable Postgres RLS, and expose rationale IDs with tenant context. |
|
||||
| Sprint 48 | Authority-Backed Scopes & Tenancy Phase 2 | src/TaskRunner/StellaOps.TaskRunner/TASKS.md | TODO | Task Runner Guild | TASKRUN-TEN-48-001 | Propagate tenant/project to all steps, enforce object store prefix, and validate before execution. |
|
||||
| Sprint 48 | Authority-Backed Scopes & Tenancy Phase 2 | src/Web/StellaOps.Web/TASKS.md | TODO | BE-Base Platform Guild | WEB-TEN-48-001 | Enforce tenant context through persistence (DB GUC, object store prefix), add request annotations, and emit audit events. |
|
||||
| Sprint 49 | Authority-Backed Scopes & Tenancy Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-TEN-49-001 | Publish `/docs/cli/authentication.md`, `/docs/api/authentication.md`, `/docs/policy/examples/abac-overlays.md`, `/docs/install/configuration-reference.md` updates (imposed rule). |
|
||||
| Sprint 49 | Authority-Backed Scopes & Tenancy Phase 3 | docs/TASKS.md | TODO | Docs Guild | DOCS-TEN-49-001 | Publish `/docs/modules/cli/guides/authentication.md`, `/docs/api/authentication.md`, `/docs/policy/examples/abac-overlays.md`, `/docs/install/configuration-reference.md` updates (imposed rule). |
|
||||
| Sprint 49 | Authority-Backed Scopes & Tenancy Phase 3 | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-TEN-49-001 | Implement audit log pipeline, monitor scope usage, chaos tests for JWKS outage, and tenant load/perf tests. |
|
||||
| Sprint 49 | Authority-Backed Scopes & Tenancy Phase 3 | src/Authority/StellaOps.Authority/TASKS.md | TODO | Authority Core & Security Guild | AUTH-TEN-49-001 | Implement service accounts, delegation tokens (`act` chain), per-tenant quotas, and audit log streaming. |
|
||||
| Sprint 49 | Authority-Backed Scopes & Tenancy Phase 3 | src/Cli/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI Guild | CLI-TEN-49-001 | Add service account token minting, delegation, and `--impersonate` banner/controls. |
|
||||
@@ -707,7 +707,7 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | docs/TASKS.md | TODO | Docs Guild | DOCS-OBS-50-004 | Publish tracing guide `/docs/observability/tracing.md` covering context propagation and sampling. |
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | docs/TASKS.md | TODO | Docs Guild | DOCS-SEC-OBS-50-001 | Update `/docs/security/redaction-and-privacy.md` for telemetry privacy controls. |
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DOING (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-002 | Stand up multi-tenant metrics/logs/traces backends with retention and isolation. |
|
||||
> Staging rollout plan recorded in `docs/ops/telemetry-storage.md`; waiting on Authority-issued tokens and namespace bootstrap.
|
||||
> Staging rollout plan recorded in `docs/modules/telemetry/operations/storage.md`; waiting on Authority-issued tokens and namespace bootstrap.
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | src/Authority/StellaOps.Authority/TASKS.md | TODO | Authority Core & Security Guild | AUTH-OBS-50-001 | Introduce observability/timeline/evidence/attestation scopes and update discovery metadata. |
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | src/Cli/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI Guild | CLI-OBS-50-001 | Propagate trace headers from CLI commands and print correlation IDs. |
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | TODO | Concelier Core Guild | CONCELIER-OBS-50-001 | Replace ad-hoc logging with telemetry core across advisory ingestion/linking. |
|
||||
|
||||
@@ -36,7 +36,7 @@ Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-003 | Telemetry & documentation<br>Apple meter metrics wired into Concelier WebService OpenTelemetry configuration; README and fixtures document normalizedVersions coverage. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-004 | Live HTML regression sweep<br>Sanitised HT125326/HT125328/HT106355/HT214108/HT215500 fixtures recorded and regression tests green on 2025-10-12. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-005 | Fixture regeneration tooling<br>`UPDATE_APPLE_FIXTURES=1` flow fetches & rewrites fixtures; README documents usage.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/AGENTS.md. Resume stalled tasks, ensuring normalizedVersions output and fixtures align with ./src/FASTER_MODELING_AND_NORMALIZATION.md before handing data to the conflict sprint. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `tools/FixtureUpdater` updates across connectors. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `src/Tools/FixtureUpdater` updates across connectors. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-CVE-02-003 | CVE normalized versions uplift |
|
||||
@@ -72,14 +72,14 @@ Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-004 | OSV references & credits alignment |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-005 | Fixture updater workflow<br>Resolved 2025-10-12: OSV mapper now derives canonical PURLs for Go + scoped npm packages when raw payloads omit `purl`; conflict fixtures unchanged for invalid npm names. Verified via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests`, and backbone normalization/storage suites. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Acsc/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ACSC-02-001 … 02-008 | Fetch→parse→map pipeline, fixtures, diagnostics, and README finished 2025-10-12; downstream export parity captured via FEEDEXPORT-JSON-04-001 / FEEDEXPORT-TRIVY-04-001 (completed). |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cccs/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CCCS-02-001 … 02-008 | Observability meter, historical harvest plan, and DOM sanitizer refinements wrapped; ops notes live under `docs/ops/concelier-cccs-operations.md` with fixtures validating EN/FR list handling. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertBund/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CERTBUND-02-001 … 02-008 | Telemetry/docs (02-006) and history/locale sweep (02-007) completed alongside pipeline; runbook `docs/ops/concelier-certbund-operations.md` captures locale guidance and offline packaging. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cccs/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CCCS-02-001 … 02-008 | Observability meter, historical harvest plan, and DOM sanitizer refinements wrapped; ops notes live under `docs/modules/concelier/operations/connectors/cccs.md` with fixtures validating EN/FR list handling. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertBund/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CERTBUND-02-001 … 02-008 | Telemetry/docs (02-006) and history/locale sweep (02-007) completed alongside pipeline; runbook `docs/modules/concelier/operations/connectors/certbund.md` captures locale guidance and offline packaging. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kisa/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-KISA-02-001 … 02-007 | Connector, tests, and telemetry/docs (02-006) finalized; localisation notes in `docs/dev/kisa_connector_notes.md` complete rollout. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-RUBDU-02-001 … 02-008 | Fetch/parser/mapper refinements, regression fixtures, telemetry/docs, access options, and trusted root packaging all landed; README documents offline access strategy. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md | DONE (2025-10-13) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-NKCKI-02-001 … 02-008 | Listing fetch, parser, mapper, fixtures, telemetry/docs, and archive plan finished; Mongo2Go/libcrypto dependency resolved via bundled OpenSSL noted in ops guide. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ICSCISA-02-001 … 02-011 | Feed parser attachment fixes, SemVer exact values, regression suites, telemetry/docs updates, and handover complete; ops runbook now details attachment verification + proxy usage. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CISCO-02-001 … 02-007 | OAuth fetch pipeline, DTO/mapping, tests, and telemetry/docs shipped; monitoring/export integration follow-ups recorded in Ops docs and exporter backlog (completed). |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-MSRC-02-001 … 02-008 | Azure AD onboarding (02-008) unblocked fetch/parse/map pipeline; fixtures, telemetry/docs, and Offline Kit guidance published in `docs/ops/concelier-msrc-operations.md`. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-MSRC-02-001 … 02-008 | Azure AD onboarding (02-008) unblocked fetch/parse/map pipeline; fixtures, telemetry/docs, and Offline Kit guidance published in `docs/modules/concelier/operations/connectors/msrc.md`. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-15) | Team Connector Support & Monitoring | FEEDCONN-CVE-02-001 … 02-002 | CVE data-source selection, fetch pipeline, and docs landed 2025-10-10. 2025-10-15: smoke verified using the seeded mirror fallback; connector now logs a warning and pulls from `seed-data/cve/` until live CVE Services credentials arrive. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Support & Monitoring | FEEDCONN-KEV-02-001 … 02-002 | KEV catalog ingestion, fixtures, telemetry, and schema validation completed 2025-10-12; ops dashboard published. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-01-001 | Canonical schema docs refresh<br>Updated canonical schema + provenance guides with SemVer style, normalized version rules, decision reason change log, and migration notes. |
|
||||
@@ -97,8 +97,8 @@ Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-002 | GHSA conflict regression fixtures |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-NVD-04-002 | NVD conflict regression fixtures |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-OSV-04-002 | OSV conflict regression fixtures<br>Instructions to work:<br>Read ./AGENTS.md and module AGENTS. Produce fixture triples supporting the precedence/tie-breaker paths defined in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md and hand them to Merge QA. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-11) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-001 | Concelier Conflict Rules<br>Runbook published at `docs/ops/concelier-conflict-resolution.md`; metrics/log guidance aligned with Sprint 3 merge counters. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-16) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-002 | Conflict runbook ops rollout<br>Ops review completed, alert thresholds applied, and change log appended in `docs/ops/concelier-conflict-resolution.md`; task closed after connector signals verified. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-11) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-001 | Concelier Conflict Rules<br>Runbook published at `docs/modules/concelier/operations/conflict-resolution.md`; metrics/log guidance aligned with Sprint 3 merge counters. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-16) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-002 | Conflict runbook ops rollout<br>Ops review completed, alert thresholds applied, and change log appended in `docs/modules/concelier/operations/conflict-resolution.md`; task closed after connector signals verified. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-15) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-04-001 | Advisory schema parity (description/CWE/canonical metric)<br>Extend `Advisory` and related records with description text, CWE collection, and canonical metric pointer; refresh validation + serializer determinism tests. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-003 | Canonical merger parity for new fields<br>Teach `CanonicalMerger` to populate description, CWEResults, and canonical metric pointer with provenance + regression coverage. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-004 | Reference normalization & freshness instrumentation cleanup<br>Implement URL normalization for reference dedupe, align freshness-sensitive instrumentation, and add analytics tests. |
|
||||
@@ -146,7 +146,7 @@ Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-19) | Team Excititor Core & Policy | EXCITITOR-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-19) | Team Excititor Policy | EXCITITOR-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-501 | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `ARCHITECTURE_SCANNER.md` §3–§4. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-501 | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `modules/scanner/architecture.md` §3–§4. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-502 | Observability helpers (correlation IDs, logging scopes, metric namespacing, deterministic hashes) consumed by WebService/Worker. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-503 | Security utilities: Authority client factory, OpTok caching, DPoP verifier, restart-time plug-in guardrails for scanner components. |
|
||||
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-001 | Buildx driver scaffold + handshake with Scanner.Emit (local CAS). |
|
||||
@@ -174,7 +174,7 @@ Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-401 | Queue abstraction + Redis Streams adapter with ack/claim APIs and idempotency tokens. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-402 | Pluggable backend support (Redis, NATS) with configuration binding, health probes, failover docs. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-403 | Retry + dead-letter strategy with structured logs/metrics for offline deployments. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `tools/FixtureUpdater` updates across connectors.<br>Progress 2025-10-20: Coordination matrix + rollout dashboard refreshed; upcoming deadlines tracked (Cccs/Cisco 2025-10-21, CertBund 2025-10-22, ICS-CISA 2025-10-23, KISA 2025-10-24) with escalation path documented in FEEDMERGE-COORD-02-900.|
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `src/Tools/FixtureUpdater` updates across connectors.<br>Progress 2025-10-20: Coordination matrix + rollout dashboard refreshed; upcoming deadlines tracked (Cccs/Cisco 2025-10-21, CertBund 2025-10-22, ICS-CISA 2025-10-23, KISA 2025-10-24) with escalation path documented in FEEDMERGE-COORD-02-900.|
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs now point at `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`; defaults/docs/tests updated to reflect the new layout. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-02-001 | Statement events & scoring signals – immutable VEX statements store, consensus signal fields, and migration `20251019-consensus-signals-statements` with tests (`dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`, `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`). |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-19) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. |
|
||||
|
||||
@@ -40,9 +40,9 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-POLICY-20-003 | Write `/docs/policy/lifecycle.md` covering workflow + roles. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Scheduler Guild | DOCS-POLICY-20-004 | Document policy run modes + cursors in `/docs/policy/runs.md`. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Platform Guild | DOCS-POLICY-20-005 | Produce `/docs/api/policy.md` with endpoint schemas + errors. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-POLICY-20-006 | Author `/docs/cli/policy.md` with commands, exit codes, JSON output. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-POLICY-20-006 | Author `/docs/modules/cli/guides/policy.md` with commands, exit codes, JSON output. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-POLICY-20-007 | Create `/docs/ui/policy-editor.md` covering editor, simulation, approvals. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-POLICY-20-008 | Publish `/docs/architecture/policy-engine.md` with sequence diagrams. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-POLICY-20-008 | Publish `/docs/modules/policy/architecture.md` with sequence diagrams. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-POLICY-20-009 | Document metrics/traces/logs in `/docs/observability/policy.md`. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Security Guild | DOCS-POLICY-20-010 | Publish `/docs/security/policy-governance.md` for scopes + approvals. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-POLICY-20-011 | Add example policies under `/docs/examples/policies/` with commentary. |
|
||||
|
||||
@@ -1,180 +1,180 @@
|
||||
# Aggregation-Only Contract Reference
|
||||
|
||||
> The Aggregation-Only Contract (AOC) is the governing rule set that keeps StellaOps ingestion services deterministic, policy-neutral, and auditable. It applies to Concelier, Excititor, and any future collectors that write raw advisory or VEX documents.
|
||||
|
||||
## 1. Purpose and Scope
|
||||
|
||||
- Defines the canonical behaviour for `advisory_raw` and `vex_raw` collections and the linkset hints they may emit.
|
||||
- Applies to every ingestion runtime (`StellaOps.Concelier.*`, `StellaOps.Excititor.*`), the Authority scopes that guard them, and the DevOps/QA surfaces that verify compliance.
|
||||
- Complements the high-level architecture in [Concelier](../ARCHITECTURE_CONCELIER.md) and Authority enforcement documented in [Authority Architecture](../ARCHITECTURE_AUTHORITY.md).
|
||||
- Paired guidance: see the guard-rail checkpoints in [AOC Guardrails](../aoc/aoc-guardrails.md) and CLI usage that will land in `/docs/cli/` as part of Sprint 19 follow-up.
|
||||
|
||||
## 2. Philosophy and Goals
|
||||
|
||||
- Preserve upstream truth: ingestion only captures immutable raw facts plus provenance, never derived severity or policy decisions.
|
||||
- Defer interpretation: Policy Engine and downstream overlays remain the sole writers of materialised findings, severity, consensus, or risk scores.
|
||||
- Make every write explainable: provenance, signatures, and content hashes are required so operators can prove where each fact originated.
|
||||
- Keep outputs reproducible: identical inputs must yield identical documents, hashes, and linksets across replays and air-gapped installs.
|
||||
|
||||
## 3. Contract Invariants
|
||||
|
||||
| # | Invariant | What it forbids or requires | Enforcement surfaces |
|
||||
|---|-----------|-----------------------------|----------------------|
|
||||
| 1 | No derived severity at ingest | Reject top-level keys such as `severity`, `cvss`, `effective_status`, `consensus_provider`, `risk_score`. Raw upstream CVSS remains inside `content.raw`. | Mongo schema validator, `AOCWriteGuard`, Roslyn analyzer, `stella aoc verify`. |
|
||||
| 2 | No merges or opinionated dedupe | Each upstream document persists on its own; ingestion never collapses multiple vendors into one document. | Repository interceptors, unit/fixture suites. |
|
||||
| 3 | Provenance is mandatory | `source.*`, `upstream.*`, and `signature` metadata must be present; missing provenance triggers `ERR_AOC_004`. | Schema validator, guard, CLI verifier. |
|
||||
| 4 | Idempotent upserts | Writes keyed by `(vendor, upstream_id, content_hash)` either no-op or insert a new revision with `supersedes`. Duplicate hashes map to the same document. | Repository guard, storage unique index, CI smoke tests. |
|
||||
| 5 | Append-only revisions | Updates create a new document with `supersedes` pointer; no in-place mutation of content. | Mongo schema (`supersedes` format), guard, data migration scripts. |
|
||||
| 6 | Linkset only | Ingestion may compute link hints (`purls`, `cpes`, IDs) to accelerate joins, but must not transform or infer severity or policy. | Linkset builders reviewed via fixtures and analyzers. |
|
||||
| 7 | Policy-only effective findings | Only Policy Engine identities can write `effective_finding_*`; ingestion callers receive `ERR_AOC_006` if they attempt it. | Authority scopes, Policy Engine guard. |
|
||||
| 8 | Schema safety | Unknown top-level keys reject with `ERR_AOC_007`; timestamps use ISO 8601 UTC strings; tenant is required. | Mongo validator, JSON schema tests. |
|
||||
| 9 | Clock discipline | Collectors stamp `fetched_at` and `received_at` monotonically per batch to support reproducibility windows. | Collector contracts, QA fixtures. |
|
||||
|
||||
## 4. Raw Schemas
|
||||
|
||||
### 4.1 `advisory_raw`
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `_id` | string | `advisory_raw:{source}:{upstream_id}:{revision}`; deterministic and tenant-scoped. |
|
||||
| `tenant` | string | Required; injected by Authority middleware and asserted by schema validator. |
|
||||
| `source.vendor` | string | Provider identifier (e.g., `redhat`, `osv`, `ghsa`). |
|
||||
| `source.stream` | string | Connector stream name (`csaf`, `osv`, etc.). |
|
||||
| `source.api` | string | Absolute URI of upstream document; stored for traceability. |
|
||||
| `source.collector_version` | string | Semantic version of the collector. |
|
||||
| `upstream.upstream_id` | string | Vendor- or ecosystem-provided identifier (CVE, GHSA, vendor ID). |
|
||||
| `upstream.document_version` | string | Upstream issued timestamp or revision string. |
|
||||
| `upstream.fetched_at` / `received_at` | string | ISO 8601 UTC timestamps recorded by the collector. |
|
||||
| `upstream.content_hash` | string | `sha256:` digest of the raw payload used for idempotency. |
|
||||
| `upstream.signature` | object | Required structure storing `present`, `format`, `key_id`, `sig`; even unsigned payloads set `present: false`. |
|
||||
| `content.format` | string | Source format (`CSAF`, `OSV`, etc.). |
|
||||
| `content.spec_version` | string | Upstream spec version when known. |
|
||||
| `content.raw` | object | Full upstream payload, untouched except for transport normalisation. |
|
||||
| `identifiers` | object | Normalised identifiers (`cve`, `ghsa`, `aliases`, etc.) derived losslessly from raw content. |
|
||||
| `linkset` | object | Join hints (see section 4.3). |
|
||||
| `supersedes` | string or null | Points to previous revision of same upstream doc when content hash changes. |
|
||||
|
||||
### 4.2 `vex_raw`
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `_id` | string | `vex_raw:{source}:{upstream_id}:{revision}`. |
|
||||
| `tenant` | string | Required; matches advisory collection requirements. |
|
||||
| `source.*` | object | Same shape and requirements as `advisory_raw`. |
|
||||
| `upstream.*` | object | Includes `document_version`, timestamps, `content_hash`, and `signature`. |
|
||||
| `content.format` | string | Typically `CycloneDX-VEX` or `CSAF-VEX`. |
|
||||
| `content.raw` | object | Entire upstream VEX payload. |
|
||||
| `identifiers.statements` | array | Normalised statement summaries (IDs, PURLs, status, justification) to accelerate policy joins. |
|
||||
| `linkset` | object | CVEs, GHSA IDs, and PURLs referenced in the document. |
|
||||
| `supersedes` | string or null | Same convention as advisory documents. |
|
||||
|
||||
### 4.3 Linkset Fields
|
||||
|
||||
- `purls`: fully qualified Package URLs extracted from raw ranges or product nodes.
|
||||
- `cpes`: Common Platform Enumerations when upstream docs provide them.
|
||||
- `aliases`: Any alternate advisory identifiers present in the payload.
|
||||
- `references`: Array of `{ type, url }` pairs pointing back to vendor advisories, patches, or exploits.
|
||||
- `reconciled_from`: Provenance of linkset entries (JSON Pointer or field origin) to make automated checks auditable.
|
||||
|
||||
Canonicalisation rules:
|
||||
- Package URLs are rendered in canonical form without qualifiers/subpaths (`pkg:type/namespace/name@version`).
|
||||
- CPE values are normalised to the 2.3 binding (`cpe:2.3:part:vendor:product:version:*:*:*:*:*:*:*`).
|
||||
|
||||
### 4.4 `advisory_observations`
|
||||
|
||||
`advisory_observations` is an immutable projection of the validated raw document used by Link‑Not‑Merge overlays. Fields mirror the JSON contract surfaced by `StellaOps.Concelier.Models.Observations.AdvisoryObservation`.
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `_id` | string | Deterministic observation id — `{tenant}:{source.vendor}:{upstreamId}:{revision}`. |
|
||||
| `tenant` | string | Lower-case tenant identifier. |
|
||||
| `source.vendor` / `source.stream` | string | Connector identity (e.g., `vendor/redhat`, `ecosystem/osv`). |
|
||||
| `source.api` | string | Absolute URI the connector fetched from. |
|
||||
| `source.collectorVersion` | string | Optional semantic version of the connector build. |
|
||||
| `upstream.upstream_id` | string | Advisory identifier as issued by the provider (CVE, vendor ID, etc.). |
|
||||
| `upstream.document_version` | string | Upstream revision/version string. |
|
||||
| `upstream.fetchedAt` / `upstream.receivedAt` | datetime | UTC timestamps recorded by the connector. |
|
||||
| `upstream.contentHash` | string | `sha256:` digest used for idempotency. |
|
||||
| `upstream.signature` | object | `{present, format?, keyId?, signature?}` describing upstream signature material. |
|
||||
| `content.format` / `content.specVersion` | string | Raw payload format metadata (CSAF, OSV, JSON, etc.). |
|
||||
| `content.raw` | object | Full upstream document stored losslessly (Relaxed Extended JSON). |
|
||||
| `content.metadata` | object | Optional connector-specific metadata (batch ids, hints). |
|
||||
| `linkset.aliases` | array | Normalized aliases (lower-case, sorted). |
|
||||
| `linkset.purls` | array | Normalized PURLs extracted from the document. |
|
||||
| `linkset.cpes` | array | Normalized CPE URIs. |
|
||||
| `linkset.references` | array | `{ type, url }` pairs (type lower-case). |
|
||||
| `createdAt` | datetime | Timestamp when Concelier persisted the observation. |
|
||||
| `attributes` | object | Optional provenance attributes keyed by connector. |
|
||||
|
||||
## 5. Error Model
|
||||
|
||||
| Code | Description | HTTP status | Surfaces |
|
||||
|------|-------------|-------------|----------|
|
||||
| `ERR_AOC_001` | Forbidden field detected (severity, cvss, effective data). | 400 | Ingestion APIs, CLI verifier, CI guard. |
|
||||
| `ERR_AOC_002` | Merge attempt detected (multiple upstream sources fused into one document). | 400 | Ingestion APIs, CLI verifier. |
|
||||
| `ERR_AOC_003` | Idempotency violation (duplicate without supersedes pointer). | 409 | Repository guard, Mongo unique index, CLI verifier. |
|
||||
| `ERR_AOC_004` | Missing provenance metadata (`source`, `upstream`, `signature`). | 422 | Schema validator, ingestion endpoints. |
|
||||
| `ERR_AOC_005` | Signature or checksum mismatch. | 422 | Collector validation, CLI verifier. |
|
||||
| `ERR_AOC_006` | Attempt to persist derived findings from ingestion context. | 403 | Policy engine guard, Authority scopes. |
|
||||
| `ERR_AOC_007` | Unknown top-level fields (schema violation). | 400 | Mongo validator, CLI verifier. |
|
||||
|
||||
Consumers should map these codes to CLI exit codes and structured log events so automation can fail fast and produce actionable guidance.
|
||||
|
||||
## 6. API and Tooling Interfaces
|
||||
|
||||
- **Concelier ingestion** (`StellaOps.Concelier.WebService`)
|
||||
- `POST /ingest/advisory`: accepts upstream payload metadata; server-side guard constructs and persists raw document.
|
||||
- `GET /advisories/raw/{id}` and filterable list endpoints expose raw documents for debugging and offline analysis.
|
||||
- `POST /aoc/verify`: runs guard checks over recent documents and returns summary totals plus first violations.
|
||||
- **Excititor ingestion** (`StellaOps.Excititor.WebService`) mirrors the same surface for VEX documents.
|
||||
- **CLI workflows** (`stella aoc verify`, `stella sources ingest --dry-run`) surface pre-flight verification; documentation will live in `/docs/cli/` alongside Sprint 19 CLI updates.
|
||||
- **Authority scopes**: new `advisory:ingest`, `advisory:read`, `vex:ingest`, and `vex:read` scopes enforce least privilege; see [Authority Architecture](../ARCHITECTURE_AUTHORITY.md) for scope grammar.
|
||||
|
||||
## 7. Idempotency and Supersedes Rules
|
||||
|
||||
1. Compute `content_hash` before any transformation; use it with `(source.vendor, upstream.upstream_id)` to detect duplicates.
|
||||
2. If a document with the same hash already exists, skip the write and log a no-op.
|
||||
3. When a new hash arrives for an existing upstream document, insert a new record and set `supersedes` to the previous `_id`.
|
||||
4. Keep supersedes chains acyclic; collectors must resolve conflicts by rewinding before they insert.
|
||||
5. Expose idempotency counters via metrics (`ingestion_write_total{result=ok|noop}`) to catch regressions early.
|
||||
|
||||
## 8. Migration Playbook
|
||||
|
||||
1. Freeze ingestion writes except for raw pass-through paths while deploying schema validators.
|
||||
2. Snapshot existing collections to `_backup_*` for rollback safety.
|
||||
3. Strip forbidden fields from historical documents into a temporary `advisory_view_legacy` used only during transition.
|
||||
4. Enable Mongo JSON schema validators for `advisory_raw` and `vex_raw`.
|
||||
5. Run collectors in `--dry-run` to confirm only allowed keys appear; fix violations before lifting the freeze.
|
||||
6. Point Policy Engine to consume exclusively from raw collections and compute derived outputs downstream.
|
||||
7. Delete legacy normalisation paths from ingestion code and enable runtime guards plus CI linting.
|
||||
8. Roll forward CLI, Console, and dashboards so operators can monitor AOC status end-to-end.
|
||||
|
||||
## 9. Observability and Diagnostics
|
||||
|
||||
- **Metrics**: `ingestion_write_total{result=ok|reject}`, `aoc_violation_total{code}`, `ingestion_signature_verified_total{result}`, `ingestion_latency_seconds`, `advisory_revision_count`.
|
||||
- **Traces**: spans `ingest.fetch`, `ingest.transform`, `ingest.write`, and `aoc.guard` with correlation IDs shared across workers.
|
||||
- **Logs**: structured entries must include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, and `violation_code` when applicable.
|
||||
- **Dashboards**: DevOps should add panels for violation counts, signature failures, supersedes growth, and CLI verifier outcomes for each tenant.
|
||||
|
||||
## 10. Security and Tenancy Checklist
|
||||
|
||||
- Enforce Authority scopes (`advisory:ingest`, `vex:ingest`, `advisory:read`, `vex:read`) and require tenant claims on every request.
|
||||
- Maintain pinned trust stores for signature verification; capture verification result in metrics and logs.
|
||||
- Ensure collectors never log secrets or raw authentication headers; redact tokens before persistence.
|
||||
- Validate that Policy Engine remains the only identity with permission to write `effective_finding_*` documents.
|
||||
- Verify offline bundles include the raw collections, guard configuration, and verifier binaries so air-gapped installs can audit parity.
|
||||
- Document operator steps for recovering from violations, including rollback to superseded revisions and re-running policy evaluation.
|
||||
|
||||
## 11. Compliance Checklist
|
||||
|
||||
- [ ] Deterministic guard enabled in Concelier and Excititor repositories.
|
||||
- [ ] Mongo validators deployed for `advisory_raw` and `vex_raw`.
|
||||
- [ ] Authority scopes and tenant enforcement verified via integration tests.
|
||||
- [ ] CLI and CI pipelines run `stella aoc verify` against seeded snapshots.
|
||||
- [ ] Observability feeds (metrics, logs, traces) wired into dashboards with alerts.
|
||||
- [ ] Offline kit instructions updated to bundle validators and verifier tooling.
|
||||
- [ ] Security review recorded covering ingestion, tenancy, and rollback procedures.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 19).*
|
||||
# Aggregation-Only Contract Reference
|
||||
|
||||
> The Aggregation-Only Contract (AOC) is the governing rule set that keeps StellaOps ingestion services deterministic, policy-neutral, and auditable. It applies to Concelier, Excititor, and any future collectors that write raw advisory or VEX documents.
|
||||
|
||||
## 1. Purpose and Scope
|
||||
|
||||
- Defines the canonical behaviour for `advisory_raw` and `vex_raw` collections and the linkset hints they may emit.
|
||||
- Applies to every ingestion runtime (`StellaOps.Concelier.*`, `StellaOps.Excititor.*`), the Authority scopes that guard them, and the DevOps/QA surfaces that verify compliance.
|
||||
- Complements the high-level architecture in [Concelier](../modules/concelier/architecture.md) and Authority enforcement documented in [Authority Architecture](../modules/authority/architecture.md).
|
||||
- Paired guidance: see the guard-rail checkpoints in [AOC Guardrails](../aoc/aoc-guardrails.md) and CLI usage that will land in `/docs/modules/cli/guides/` as part of Sprint 19 follow-up.
|
||||
|
||||
## 2. Philosophy and Goals
|
||||
|
||||
- Preserve upstream truth: ingestion only captures immutable raw facts plus provenance, never derived severity or policy decisions.
|
||||
- Defer interpretation: Policy Engine and downstream overlays remain the sole writers of materialised findings, severity, consensus, or risk scores.
|
||||
- Make every write explainable: provenance, signatures, and content hashes are required so operators can prove where each fact originated.
|
||||
- Keep outputs reproducible: identical inputs must yield identical documents, hashes, and linksets across replays and air-gapped installs.
|
||||
|
||||
## 3. Contract Invariants
|
||||
|
||||
| # | Invariant | What it forbids or requires | Enforcement surfaces |
|
||||
|---|-----------|-----------------------------|----------------------|
|
||||
| 1 | No derived severity at ingest | Reject top-level keys such as `severity`, `cvss`, `effective_status`, `consensus_provider`, `risk_score`. Raw upstream CVSS remains inside `content.raw`. | Mongo schema validator, `AOCWriteGuard`, Roslyn analyzer, `stella aoc verify`. |
|
||||
| 2 | No merges or opinionated dedupe | Each upstream document persists on its own; ingestion never collapses multiple vendors into one document. | Repository interceptors, unit/fixture suites. |
|
||||
| 3 | Provenance is mandatory | `source.*`, `upstream.*`, and `signature` metadata must be present; missing provenance triggers `ERR_AOC_004`. | Schema validator, guard, CLI verifier. |
|
||||
| 4 | Idempotent upserts | Writes keyed by `(vendor, upstream_id, content_hash)` either no-op or insert a new revision with `supersedes`. Duplicate hashes map to the same document. | Repository guard, storage unique index, CI smoke tests. |
|
||||
| 5 | Append-only revisions | Updates create a new document with `supersedes` pointer; no in-place mutation of content. | Mongo schema (`supersedes` format), guard, data migration scripts. |
|
||||
| 6 | Linkset only | Ingestion may compute link hints (`purls`, `cpes`, IDs) to accelerate joins, but must not transform or infer severity or policy. | Linkset builders reviewed via fixtures and analyzers. |
|
||||
| 7 | Policy-only effective findings | Only Policy Engine identities can write `effective_finding_*`; ingestion callers receive `ERR_AOC_006` if they attempt it. | Authority scopes, Policy Engine guard. |
|
||||
| 8 | Schema safety | Unknown top-level keys reject with `ERR_AOC_007`; timestamps use ISO 8601 UTC strings; tenant is required. | Mongo validator, JSON schema tests. |
|
||||
| 9 | Clock discipline | Collectors stamp `fetched_at` and `received_at` monotonically per batch to support reproducibility windows. | Collector contracts, QA fixtures. |
|
||||
|
||||
## 4. Raw Schemas
|
||||
|
||||
### 4.1 `advisory_raw`
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `_id` | string | `advisory_raw:{source}:{upstream_id}:{revision}`; deterministic and tenant-scoped. |
|
||||
| `tenant` | string | Required; injected by Authority middleware and asserted by schema validator. |
|
||||
| `source.vendor` | string | Provider identifier (e.g., `redhat`, `osv`, `ghsa`). |
|
||||
| `source.stream` | string | Connector stream name (`csaf`, `osv`, etc.). |
|
||||
| `source.api` | string | Absolute URI of upstream document; stored for traceability. |
|
||||
| `source.collector_version` | string | Semantic version of the collector. |
|
||||
| `upstream.upstream_id` | string | Vendor- or ecosystem-provided identifier (CVE, GHSA, vendor ID). |
|
||||
| `upstream.document_version` | string | Upstream issued timestamp or revision string. |
|
||||
| `upstream.fetched_at` / `received_at` | string | ISO 8601 UTC timestamps recorded by the collector. |
|
||||
| `upstream.content_hash` | string | `sha256:` digest of the raw payload used for idempotency. |
|
||||
| `upstream.signature` | object | Required structure storing `present`, `format`, `key_id`, `sig`; even unsigned payloads set `present: false`. |
|
||||
| `content.format` | string | Source format (`CSAF`, `OSV`, etc.). |
|
||||
| `content.spec_version` | string | Upstream spec version when known. |
|
||||
| `content.raw` | object | Full upstream payload, untouched except for transport normalisation. |
|
||||
| `identifiers` | object | Normalised identifiers (`cve`, `ghsa`, `aliases`, etc.) derived losslessly from raw content. |
|
||||
| `linkset` | object | Join hints (see section 4.3). |
|
||||
| `supersedes` | string or null | Points to previous revision of same upstream doc when content hash changes. |
|
||||
|
||||
### 4.2 `vex_raw`
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `_id` | string | `vex_raw:{source}:{upstream_id}:{revision}`. |
|
||||
| `tenant` | string | Required; matches advisory collection requirements. |
|
||||
| `source.*` | object | Same shape and requirements as `advisory_raw`. |
|
||||
| `upstream.*` | object | Includes `document_version`, timestamps, `content_hash`, and `signature`. |
|
||||
| `content.format` | string | Typically `CycloneDX-VEX` or `CSAF-VEX`. |
|
||||
| `content.raw` | object | Entire upstream VEX payload. |
|
||||
| `identifiers.statements` | array | Normalised statement summaries (IDs, PURLs, status, justification) to accelerate policy joins. |
|
||||
| `linkset` | object | CVEs, GHSA IDs, and PURLs referenced in the document. |
|
||||
| `supersedes` | string or null | Same convention as advisory documents. |
|
||||
|
||||
### 4.3 Linkset Fields
|
||||
|
||||
- `purls`: fully qualified Package URLs extracted from raw ranges or product nodes.
|
||||
- `cpes`: Common Platform Enumerations when upstream docs provide them.
|
||||
- `aliases`: Any alternate advisory identifiers present in the payload.
|
||||
- `references`: Array of `{ type, url }` pairs pointing back to vendor advisories, patches, or exploits.
|
||||
- `reconciled_from`: Provenance of linkset entries (JSON Pointer or field origin) to make automated checks auditable.
|
||||
|
||||
Canonicalisation rules:
|
||||
- Package URLs are rendered in canonical form without qualifiers/subpaths (`pkg:type/namespace/name@version`).
|
||||
- CPE values are normalised to the 2.3 binding (`cpe:2.3:part:vendor:product:version:*:*:*:*:*:*:*`).
|
||||
|
||||
### 4.4 `advisory_observations`
|
||||
|
||||
`advisory_observations` is an immutable projection of the validated raw document used by Link‑Not‑Merge overlays. Fields mirror the JSON contract surfaced by `StellaOps.Concelier.Models.Observations.AdvisoryObservation`.
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| `_id` | string | Deterministic observation id — `{tenant}:{source.vendor}:{upstreamId}:{revision}`. |
|
||||
| `tenant` | string | Lower-case tenant identifier. |
|
||||
| `source.vendor` / `source.stream` | string | Connector identity (e.g., `vendor/redhat`, `ecosystem/osv`). |
|
||||
| `source.api` | string | Absolute URI the connector fetched from. |
|
||||
| `source.collectorVersion` | string | Optional semantic version of the connector build. |
|
||||
| `upstream.upstream_id` | string | Advisory identifier as issued by the provider (CVE, vendor ID, etc.). |
|
||||
| `upstream.document_version` | string | Upstream revision/version string. |
|
||||
| `upstream.fetchedAt` / `upstream.receivedAt` | datetime | UTC timestamps recorded by the connector. |
|
||||
| `upstream.contentHash` | string | `sha256:` digest used for idempotency. |
|
||||
| `upstream.signature` | object | `{present, format?, keyId?, signature?}` describing upstream signature material. |
|
||||
| `content.format` / `content.specVersion` | string | Raw payload format metadata (CSAF, OSV, JSON, etc.). |
|
||||
| `content.raw` | object | Full upstream document stored losslessly (Relaxed Extended JSON). |
|
||||
| `content.metadata` | object | Optional connector-specific metadata (batch ids, hints). |
|
||||
| `linkset.aliases` | array | Normalized aliases (lower-case, sorted). |
|
||||
| `linkset.purls` | array | Normalized PURLs extracted from the document. |
|
||||
| `linkset.cpes` | array | Normalized CPE URIs. |
|
||||
| `linkset.references` | array | `{ type, url }` pairs (type lower-case). |
|
||||
| `createdAt` | datetime | Timestamp when Concelier persisted the observation. |
|
||||
| `attributes` | object | Optional provenance attributes keyed by connector. |
|
||||
|
||||
## 5. Error Model
|
||||
|
||||
| Code | Description | HTTP status | Surfaces |
|
||||
|------|-------------|-------------|----------|
|
||||
| `ERR_AOC_001` | Forbidden field detected (severity, cvss, effective data). | 400 | Ingestion APIs, CLI verifier, CI guard. |
|
||||
| `ERR_AOC_002` | Merge attempt detected (multiple upstream sources fused into one document). | 400 | Ingestion APIs, CLI verifier. |
|
||||
| `ERR_AOC_003` | Idempotency violation (duplicate without supersedes pointer). | 409 | Repository guard, Mongo unique index, CLI verifier. |
|
||||
| `ERR_AOC_004` | Missing provenance metadata (`source`, `upstream`, `signature`). | 422 | Schema validator, ingestion endpoints. |
|
||||
| `ERR_AOC_005` | Signature or checksum mismatch. | 422 | Collector validation, CLI verifier. |
|
||||
| `ERR_AOC_006` | Attempt to persist derived findings from ingestion context. | 403 | Policy engine guard, Authority scopes. |
|
||||
| `ERR_AOC_007` | Unknown top-level fields (schema violation). | 400 | Mongo validator, CLI verifier. |
|
||||
|
||||
Consumers should map these codes to CLI exit codes and structured log events so automation can fail fast and produce actionable guidance.
|
||||
|
||||
## 6. API and Tooling Interfaces
|
||||
|
||||
- **Concelier ingestion** (`StellaOps.Concelier.WebService`)
|
||||
- `POST /ingest/advisory`: accepts upstream payload metadata; server-side guard constructs and persists raw document.
|
||||
- `GET /advisories/raw/{id}` and filterable list endpoints expose raw documents for debugging and offline analysis.
|
||||
- `POST /aoc/verify`: runs guard checks over recent documents and returns summary totals plus first violations.
|
||||
- **Excititor ingestion** (`StellaOps.Excititor.WebService`) mirrors the same surface for VEX documents.
|
||||
- **CLI workflows** (`stella aoc verify`, `stella sources ingest --dry-run`) surface pre-flight verification; documentation will live in `/docs/modules/cli/guides/` alongside Sprint 19 CLI updates.
|
||||
- **Authority scopes**: new `advisory:ingest`, `advisory:read`, `vex:ingest`, and `vex:read` scopes enforce least privilege; see [Authority Architecture](../modules/authority/architecture.md) for scope grammar.
|
||||
|
||||
## 7. Idempotency and Supersedes Rules
|
||||
|
||||
1. Compute `content_hash` before any transformation; use it with `(source.vendor, upstream.upstream_id)` to detect duplicates.
|
||||
2. If a document with the same hash already exists, skip the write and log a no-op.
|
||||
3. When a new hash arrives for an existing upstream document, insert a new record and set `supersedes` to the previous `_id`.
|
||||
4. Keep supersedes chains acyclic; collectors must resolve conflicts by rewinding before they insert.
|
||||
5. Expose idempotency counters via metrics (`ingestion_write_total{result=ok|noop}`) to catch regressions early.
|
||||
|
||||
## 8. Migration Playbook
|
||||
|
||||
1. Freeze ingestion writes except for raw pass-through paths while deploying schema validators.
|
||||
2. Snapshot existing collections to `_backup_*` for rollback safety.
|
||||
3. Strip forbidden fields from historical documents into a temporary `advisory_view_legacy` used only during transition.
|
||||
4. Enable Mongo JSON schema validators for `advisory_raw` and `vex_raw`.
|
||||
5. Run collectors in `--dry-run` to confirm only allowed keys appear; fix violations before lifting the freeze.
|
||||
6. Point Policy Engine to consume exclusively from raw collections and compute derived outputs downstream.
|
||||
7. Delete legacy normalisation paths from ingestion code and enable runtime guards plus CI linting.
|
||||
8. Roll forward CLI, Console, and dashboards so operators can monitor AOC status end-to-end.
|
||||
|
||||
## 9. Observability and Diagnostics
|
||||
|
||||
- **Metrics**: `ingestion_write_total{result=ok|reject}`, `aoc_violation_total{code}`, `ingestion_signature_verified_total{result}`, `ingestion_latency_seconds`, `advisory_revision_count`.
|
||||
- **Traces**: spans `ingest.fetch`, `ingest.transform`, `ingest.write`, and `aoc.guard` with correlation IDs shared across workers.
|
||||
- **Logs**: structured entries must include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, and `violation_code` when applicable.
|
||||
- **Dashboards**: DevOps should add panels for violation counts, signature failures, supersedes growth, and CLI verifier outcomes for each tenant.
|
||||
|
||||
## 10. Security and Tenancy Checklist
|
||||
|
||||
- Enforce Authority scopes (`advisory:ingest`, `vex:ingest`, `advisory:read`, `vex:read`) and require tenant claims on every request.
|
||||
- Maintain pinned trust stores for signature verification; capture verification result in metrics and logs.
|
||||
- Ensure collectors never log secrets or raw authentication headers; redact tokens before persistence.
|
||||
- Validate that Policy Engine remains the only identity with permission to write `effective_finding_*` documents.
|
||||
- Verify offline bundles include the raw collections, guard configuration, and verifier binaries so air-gapped installs can audit parity.
|
||||
- Document operator steps for recovering from violations, including rollback to superseded revisions and re-running policy evaluation.
|
||||
|
||||
## 11. Compliance Checklist
|
||||
|
||||
- [ ] Deterministic guard enabled in Concelier and Excititor repositories.
|
||||
- [ ] Mongo validators deployed for `advisory_raw` and `vex_raw`.
|
||||
- [ ] Authority scopes and tenant enforcement verified via integration tests.
|
||||
- [ ] CLI and CI pipelines run `stella aoc verify` against seeded snapshots.
|
||||
- [ ] Observability feeds (metrics, logs, traces) wired into dashboards with alerts.
|
||||
- [ ] Offline kit instructions updated to bundle validators and verifier tooling.
|
||||
- [ ] Security review recorded covering ingestion, tenancy, and rollback procedures.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 19).*
|
||||
|
||||
430
docs/moat.md
Normal file
430
docs/moat.md
Normal file
@@ -0,0 +1,430 @@
|
||||
# StellaOps Moat Track — Spec Outline v0.3
|
||||
|
||||
**Scope of this doc:**
|
||||
(1) Deterministic Replayable Scans (SRM), (2) Policy Engine & Lattice UI, (3) Sovereign Readiness (CryptoProfile + RootPack), (4) Attestation Observability Graph (AOG), (5) Procurement‑Grade Trust Statement, (6) Third‑Party Proof Channel, (7) Zastava differential SBOM + AI scheduler.
|
||||
|
||||
Cross‑cutting principles: offline‑first, cryptographic determinism, evidence‑bound decisions, regional crypto compliance, minimal operational friction.
|
||||
|
||||
---
|
||||
|
||||
## 0) Shared Concepts (applies to all 7)
|
||||
|
||||
* **Artifact identity:** digest-first (OCI image digest, file sha256).
|
||||
* **Canonicalization:** all structured payloads (SBOM, SRM, Trust Statement JSON, VEX) are normalized via Canonical JSON (RFC‑8785‑like) prior to hashing/signing.
|
||||
* **Signatures:** DSSE envelopes; **dual‑signing** supported (e.g., FIPS ECDSA + GOST R 34.10; or ECDSA + SM2).
|
||||
* **Attestation chain:** each decision (scan, VEX merge, policy evaluation) yields a signed, replayable record.
|
||||
* **Profiles & Packs:** **CryptoProfile** (algorithm + root policy) and **RootPack** (trust anchors + OCSP/CRL/TSA mirrors) are versioned and importable.
|
||||
* **Policy Unit Tests:** any policy/lattice bundle ships with fixtures expected to pass during CI.
|
||||
|
||||
---
|
||||
|
||||
## 1) Deterministic Replayable Scans — SRM
|
||||
|
||||
### Objective
|
||||
|
||||
Make every scan a **provable, re‑executable fact**. Auditors can replay; results are bit‑for‑bit reproducible.
|
||||
|
||||
### Deliverables
|
||||
|
||||
* **SRM v0.1** schema (YAML/JSON)
|
||||
* Deterministic executor + `stella replay`
|
||||
* Replay diffing and result hashing
|
||||
|
||||
### SRM (Stella Replay Manifest) — schema (abridged)
|
||||
|
||||
```yaml
|
||||
apiVersion: srm.stellaops.dev/v0.1
|
||||
scan:
|
||||
id: uuid
|
||||
timestamp: ISO8601
|
||||
engine: { name: "stella-scan", version: "1.7.3", build_sha: "<sha>" }
|
||||
environment:
|
||||
os_image: <digest>
|
||||
kernel: { uname: "...", cgroups: "v2" }
|
||||
cpu_features: [avx2, sse4.2]
|
||||
inputs:
|
||||
image: { name: "reg/app:1.9.2", digest: "<sha>", layers: ["<sha>", ...] }
|
||||
sbom: { type: "cyclonedx@1.5", digest: "<sha>" }
|
||||
vex_set: [{ type: "openvex", digest: "<sha>" }]
|
||||
lattice_policy: { id: "corp-policy@2025-08-15", digest: "<sha>" }
|
||||
rules_and_feeds:
|
||||
rulesets: [{ name: "vuln-core", version: "2025.08.30", digest: "<sha>" }]
|
||||
feeds:
|
||||
- { name: "nvd", snapshot_date: "2025-08-30", archive_digest: "<sha>" }
|
||||
execution:
|
||||
mode: deterministic
|
||||
random_seed: 314159
|
||||
ordering: lexical
|
||||
heuristics: { binary_scan: true, secrets: false }
|
||||
evidence:
|
||||
files_hashed: 12873
|
||||
samples: [{ path: "/usr/lib/libssl.so.3", sha256: "<sha>" }]
|
||||
outputs:
|
||||
report_digest: "<sha>" # canonical JSON root hash
|
||||
artifacts:
|
||||
- { name: "findings.json", sha256: "<sha>" }
|
||||
signatures:
|
||||
- { scheme: "DSSE", CryptoProfile: "FIPS-140-3", signer: "build-ca@corp" }
|
||||
- { scheme: "DSSE", CryptoProfile: "GOST-2012", signer: "ru-ca@corp" }
|
||||
rekor: { entries: ["<uuid>", ...] } # optional (offline allowed)
|
||||
```
|
||||
|
||||
### CLI & API
|
||||
|
||||
* `stella scan --image reg/app@sha256:... --srm-out srm.yaml --findings findings.json`
|
||||
* `stella replay srm.yaml --out replay.json --assert-digest <sha>`
|
||||
* `POST /v1/srm/replay` → returns `ok`, `replay_report_digest`, `diff` (if any).
|
||||
|
||||
### Determinism Rules
|
||||
|
||||
* Single thread or ordered parallel with stable scheduling; sorted inputs; fixed random seed; pinned rules/feeds/policies from SRM; identical canonicalization routines.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* Replaying SRM on a different host returns identical `report_digest`.
|
||||
* If any feed archive differs by 1 bit, replay fails with a precise diff.
|
||||
* SRM size ≤ 25 MB for a typical microservice image (excludes large feed archives, which may be referenced by digest and bundled as a side‑car tar).
|
||||
|
||||
---
|
||||
|
||||
## 2) Policy Engine & Lattice UI
|
||||
|
||||
### Objective
|
||||
|
||||
Turn VEX merging and severity logic into **programmable, testable algebra** with explainability.
|
||||
|
||||
### Model
|
||||
|
||||
* **Domain:** partial order over vulnerability states:
|
||||
`unknown < under_investigation < affected || not_affected < fixed`.
|
||||
Cross‑product with *scope*: `{runtime_path, build_path, optional_path}` and *confidence*: `{low, med, high}`.
|
||||
* **Merge semantics:** monotonic lattice joins; conflict resolution rules prioritized by signed source trust and policy precedence.
|
||||
|
||||
### DSL (sketch)
|
||||
|
||||
```hocon
|
||||
policy "corp-runtime" version "2025.08.15" {
|
||||
sources {
|
||||
trust_order = ["vendor:redhat", "internal:appsec", "public:nvd"]
|
||||
require_signatures = true
|
||||
}
|
||||
|
||||
rules {
|
||||
when vex.statement == "not_affected"
|
||||
and evidence.entrypoint_exposes == false
|
||||
then state := not_affected with confidence := high;
|
||||
|
||||
when package.is_dev_dependency == true
|
||||
then scope := optional_path;
|
||||
|
||||
when cvss >= 9.0 and reachable == true
|
||||
then priority := "block";
|
||||
}
|
||||
|
||||
guards {
|
||||
forbid unsigned_sources;
|
||||
forbid downgrade_of_state_below previous_state;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### UI (“Trust Algebra Studio”)
|
||||
|
||||
* Drag‑and‑drop rule blocks, precedence editor, **simulation mode** on sample SBOM/VEX; **policy unit tests**.
|
||||
* Export **signed** `.lattice.json`; importable into CI.
|
||||
|
||||
### CLI & API
|
||||
|
||||
* `stella policy lint corp-runtime.lattice.json`
|
||||
* `stella policy test --fixtures fixtures/`
|
||||
* `POST /v1/policy/evaluate` → normalized decision + proof trail.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* Given same inputs, policy evaluation yields identical decision + proof trail hash.
|
||||
* UI can round‑trip DSL ⇄ JSON with no semantic drift.
|
||||
* Policy pack signature required to run in “enforced” mode.
|
||||
|
||||
---
|
||||
|
||||
## 3) Sovereign Readiness — CryptoProfile + RootPack
|
||||
|
||||
### Objective
|
||||
|
||||
**Drop‑in regional cryptography** (Russia/China/EU/US) with offline operation.
|
||||
|
||||
### CryptoProfile (attached to every signature/attestation)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "GOST-2012@v1",
|
||||
"algorithms": {"sign":"GOST R 34.10-2012","hash":"GOST R 34.11-2012","cipher":"GOST 34.12-2015"},
|
||||
"key_policy": {"curve":"id-tc26-gost-3410-2012-256","hsm_required": true},
|
||||
"time_stamping": {"tsa": "rootpack://ru/tsa1"},
|
||||
"roots": ["rootpack://ru/trustanchors/gost-ca1"]
|
||||
}
|
||||
```
|
||||
|
||||
### RootPack
|
||||
|
||||
* Tarball containing: trust anchors, intermediate CAs, OCSP/CRL snapshots, TSA profiles, policy constraints (e.g., no cross‑sign with foreign roots), and region metadata.
|
||||
* Installed via: `stella rootpack import rootpack_ru_v1.tar.gz`.
|
||||
|
||||
### Dual‑Signing & Guardrails
|
||||
|
||||
* Policy flags like `allow_dual_signing FIPS+GOST`, `forbid_sm2_for_us_exports` (example only).
|
||||
* Enforcement happens at signing time and verification time.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* Offline verification using RootPack succeeds; online OCSP disabled per policy.
|
||||
* Mis‑profiled signatures are rejected with explicit reason codes.
|
||||
* Dual‑signed DSSE verifies under both profiles when allowed.
|
||||
|
||||
---
|
||||
|
||||
## 4) Attestation Observability Graph (AOG)
|
||||
|
||||
### Objective
|
||||
|
||||
Make trust **observable**. Expose SLIs/SLOs for cryptographic posture & policy compliance.
|
||||
|
||||
### Data Model
|
||||
|
||||
* Nodes: `{artifact, sbom, policy, vex, srm, signature, rootpack, runtime_instance}`
|
||||
* Edges: `derived_from`, `signed_by`, `evaluated_by`, `replayed_by`, `runs_as`
|
||||
|
||||
### Metrics (OpenTelemetry/Prometheus)
|
||||
|
||||
* `stella_trust_sli{service,env}` = fraction of running pods whose image has a valid SRM‑backed attestation chain under the active policy.
|
||||
* `stella_attestation_latency_seconds` (P50/P95) from build to verified‑ready.
|
||||
* `stella_policy_drift_events_total` (increment when running policy != signed policy).
|
||||
* `stella_exception_without_proof_total` (must be 0).
|
||||
* `stella_replay_success_ratio` (per week).
|
||||
|
||||
**SLO Example**
|
||||
|
||||
* **Trust SLO ≥ 99.9%** measured hourly; error budget resets monthly.
|
||||
|
||||
### Interfaces
|
||||
|
||||
* `stella aog export --format otlp`
|
||||
* `GET /v1/aog/graph?artifactDigest=...` → subgraph JSON (with signed edge proofs).
|
||||
* Grafana dashboards (packaged).
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* AOG can reconstruct the full trust lineage for any running pod in ≤ 2s (p95) on a 1k‑service cluster.
|
||||
* Metrics cardinality bounded (service/env/policy only).
|
||||
|
||||
---
|
||||
|
||||
## 5) Procurement‑Grade “Trust Statement”
|
||||
|
||||
### Objective
|
||||
|
||||
One **board‑ready** artifact that unifies security posture across vendors; machine‑readable twin for ERP/GRC.
|
||||
|
||||
### Outputs
|
||||
|
||||
* **PDF** (human): signed, watermark, controlled fields for vendor name/version/date, summary graphs, SLOs, exceptions with PCE (proof‑carrying exceptions).
|
||||
* **JSON** (machine): normalized schema below; DSSE‑signed; includes SRM and policy references.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "trust-statement.stellaops.dev/v1",
|
||||
"vendor": {"name": "Acme","product":"Payments API","version":"1.9.2"},
|
||||
"build": {"image_digest":"sha256:...","srm_digest":"sha256:..."},
|
||||
"policy": {"id":"corp-runtime@2025-08-15","digest":"sha256:..."},
|
||||
"summary": {
|
||||
"trust_sli": 0.9992,
|
||||
"exceptions": 1,
|
||||
"open_findings": {"critical":0,"high":2,"medium":4,"low":12}
|
||||
},
|
||||
"exceptions": [{
|
||||
"id":"EXC-2025-0912",
|
||||
"reason":"not_affected-via-vex",
|
||||
"proof_digest":"sha256:...",
|
||||
"expiry":"2026-01-15"
|
||||
}],
|
||||
"signatures": [{ "CryptoProfile":"FIPS-140-3" }]
|
||||
}
|
||||
```
|
||||
|
||||
### Integrations
|
||||
|
||||
* Push connectors: **SAP Ariba, ServiceNow, Archer, Jira** (webhooks or SFTP in offline flows).
|
||||
* CLI: `stella trust-statement generate --srm srm.yaml --policy corp-runtime.lattice.json --out acme-1.9.2.trust.json --pdf`.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* JSON validates against schema; PDF and JSON hashes match the DSSE statement.
|
||||
* ERP ingest POC: Ariba/ServiceNow field mapping validated.
|
||||
|
||||
---
|
||||
|
||||
## 6) Third‑Party Proof Channel
|
||||
|
||||
### Objective
|
||||
|
||||
Create a **publisher ecosystem** for upstream proofs: SBOM, VEX, and **VDR (Vulnerability Derivation Reason)**.
|
||||
|
||||
### Publisher Model
|
||||
|
||||
* **Identity:** publishers obtain a **Publisher Certificate** (could be verified via RootPack‑anchored CA or cross‑signed).
|
||||
* **Submission:** `stella ledger publish --type {sbom|vex|vdr} --artifact <digest> --file proof.json --sign`.
|
||||
* **Moderation & Revocation:** CRL‑like **Proof Revocation List (PRL)** with signed reasons.
|
||||
|
||||
### VDR (schema sketch)
|
||||
|
||||
```json
|
||||
{
|
||||
"schema":"vdr.stellaops.dev/v1",
|
||||
"artifact":"sha256:...",
|
||||
"cve":"CVE-2025-12345",
|
||||
"claim":"not_affected",
|
||||
"method":"entrypoint_unreachable|abi_mismatch|dead_code",
|
||||
"evidence_refs":[{"type":"symbol_map","digest":"sha256:..."}],
|
||||
"publisher":"redhat://rhel",
|
||||
"signatures":[...]
|
||||
}
|
||||
```
|
||||
|
||||
### Consumption
|
||||
|
||||
* Policies can **prioritize** publisher channels by trust level.
|
||||
* AOG shows which proofs originated from which publishers.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* At least one upstream (e.g., base image vendor) can publish and your policy can consume & rank it.
|
||||
* PRL revokes a proof and AOG reflects the change within 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
## 7) Zastava — differential SBOM + AI enrichment scheduler
|
||||
|
||||
### Objective
|
||||
|
||||
Produce **entrypoint‑aware differential SBOMs** and continually **re‑enrich** new/old SBOMs with AI context and exposure‑aware prioritization.
|
||||
|
||||
### Concepts
|
||||
|
||||
* **dSBOM:** SBOM that reflects effective dependency set for a specific `ENTRYPOINT/CMD` and runtime flags (e.g., `--server.urls`, `DOTNET_...`).
|
||||
* **Scheduler:** rescans **old SBOMs** when: (a) new CVE feeds arrive, (b) new VEX/VDR appear, (c) policy changes, or (d) AI models learn new exploitability signals.
|
||||
|
||||
### Pipeline
|
||||
|
||||
1. **Static slice:** infer reachable packages from entrypoint (e.g., `.NET Kestrel` vs CLI tool).
|
||||
2. **Runtime slice (optional):** collect process tree, open sockets, and imported modules at startup in a **shadow run** or mirrored traffic.
|
||||
3. **Diff:** `dSBOM = SBOM ∩ (static_reachable ∪ runtime_observed)`.
|
||||
4. **AI Enrichment:** Zastava annotates each finding with *context weights* (exposed/not exposed, network scope, RBAC, secrets proximity).
|
||||
5. **Plan:** produce PRs (Dockerfile base bump, package pin, k8s Service change).
|
||||
6. **Scheduler:**
|
||||
|
||||
* `stella zastava schedule --query 'service=payments AND env=prod' --interval 6h`
|
||||
* Triggers re‑evaluation and emits updated SRM + Trust deltas.
|
||||
|
||||
### dSBOM format (addon)
|
||||
|
||||
```json
|
||||
{
|
||||
"schema":"cyclonedx+stella-diff@1.0",
|
||||
"base_sbom":"sha256:...",
|
||||
"entrypoint": ["/app/bin/Release/net8.0/app.dll"],
|
||||
"cmd": ["--urls","http://127.0.0.1:8080"],
|
||||
"static_reachable": ["pkg:nuget/Kestrel@*", "pkg:nuget/System.Data@*"],
|
||||
"runtime_observed": ["pkg:rpm/openssl@3.0.9"],
|
||||
"excluded_paths": ["/usr/share/docs/**"],
|
||||
"digest":"sha256:..."
|
||||
}
|
||||
```
|
||||
|
||||
### Kestrel example (priority logic)
|
||||
|
||||
* If Kestrel present but Service is `ClusterIP` and no Ingress, **deprioritize**; if `LoadBalancer`/`NodePort` with 0‑RTT TLS disabled, **prioritize**.
|
||||
* Zastava produces an **explainable card**: “Priority lowered due to non‑exposed runtime path; evidence: `kubectl get svc`, `netstat`, policy rule #42.”
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
* Changing `ENTRYPOINT` produces a different, signed dSBOM and updated priorities.
|
||||
* Scheduler updates stale SBOMs and issues signed deltas without network access (when RootPacks + feed mirrors are available).
|
||||
|
||||
---
|
||||
|
||||
## Cross‑Cutting Security & Compliance
|
||||
|
||||
* **Sanctions & Guardrails:** per‑profile constraints (e.g., forbid dual‑signing across certain jurisdictions). Policy‑enforced at sign/verify time.
|
||||
* **HSM Integrations:** PKCS#11 providers for each profile (FIPS, GOST, SM2).
|
||||
* **Data handling:** SRMs and Trust Statements may contain paths and hashes; optional redaction profiles for vendor sharing.
|
||||
|
||||
---
|
||||
|
||||
## Interfaces Summary (CLI)
|
||||
|
||||
```bash
|
||||
# Scans / Replay
|
||||
stella scan --image <digest> --srm-out srm.yaml
|
||||
stella replay srm.yaml --assert-digest <sha>
|
||||
|
||||
# Policy & Lattice
|
||||
stella policy lint corp.lattice.json
|
||||
stella policy test --fixtures fixtures/
|
||||
stella policy sign --profile FIPS-140-3
|
||||
|
||||
# Crypto Sovereign
|
||||
stella rootpack import rootpack_ru_v1.tar.gz
|
||||
stella attest sign --profile GOST-2012 --in srm.yaml
|
||||
stella attest verify --profiles GOST-2012,FIPS-140-3
|
||||
|
||||
# AOG
|
||||
stella aog export --format otlp
|
||||
stella aog graph --artifact <digest>
|
||||
|
||||
# Trust Statement
|
||||
stella trust-statement generate --srm srm.yaml --policy corp.lattice.json --pdf out.pdf --json out.json
|
||||
|
||||
# Third-Party Proof Channel
|
||||
stella ledger publish --type vdr --artifact <digest> --file vdr.json --sign
|
||||
stella ledger revoke --id <proof-id> --reason "superseded"
|
||||
|
||||
# Zastava
|
||||
stella zastava diff-sbom --image <digest> --entrypoint "<cmd>" --out dsbom.json
|
||||
stella zastava enrich --sbom dsbom.json --findings findings.json
|
||||
stella zastava schedule --query 'env=prod' --interval 6h
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics (per pillar)
|
||||
|
||||
* **SRM:** ≥ 99% of production images have SRM attached; **Replay Success Ratio** ≥ 0.99 weekly.
|
||||
* **Policy/Lattice:** 100% of exceptions carry proof; policy test coverage ≥ 90% on top CVE classes.
|
||||
* **Sovereign:** RootPacks for RU/CN/EU/US verified; dual‑signing working in CI; offline verification median < 200 ms.
|
||||
* **AOG:** Trust SLO ≥ 99.9%; lineage lookup p95 ≤ 2s.
|
||||
* **Trust Statement:** Accepted by at least one ERP (pilot); generation time ≤ 15s.
|
||||
* **Third‑Party Channel:** ≥ 3 upstream publishers integrated; PRL revocations flow to AOG within 5 minutes.
|
||||
* **Zastava:** dSBOM reduces non‑reachable high/critical findings by ≥ 35% without raising exposure incidents.
|
||||
|
||||
---
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
* **Crypto complexity:** profile misuse → strict guardrails + default safe profiles, strong linting.
|
||||
* **Cardinality/telemetry blow‑ups:** AOG label hygiene + sampling.
|
||||
* **Vendor adoption (Proof Channel):** seed with your own base images and internal frameworks; provide SDKs and reference publishers.
|
||||
* **Determinism regressions:** CI runs replay tests on golden SRMs; any drift fails the build.
|
||||
|
||||
---
|
||||
|
||||
## 90‑Day Moat‑First Milestones
|
||||
|
||||
1. **SRM v0.1**: schema, deterministic executor, CLI replay, golden tests.
|
||||
2. **Policy Engine MVP**: DSL + evaluator + UI simulation; policy unit tests; signed policy packs.
|
||||
3. **CryptoProfile/RootPack MVP**: FIPS + GOST working; dual‑signing; offline verify.
|
||||
4. **AOG MVP**: lineage service + OTLP exporter + Grafana pack; Trust SLI.
|
||||
5. **Trust Statement MVP**: JSON + PDF; ServiceNow ingest POC.
|
||||
6. **Proof Channel alpha**: publisher identity + VDR schema + local ledger; PRL.
|
||||
7. **Zastava α**: `diff-sbom` + exposure heuristics for `.NET Kestrel`; scheduler with offline mirrors.
|
||||
|
||||
---
|
||||
22
docs/modules/advisory-ai/AGENTS.md
Normal file
22
docs/modules/advisory-ai/AGENTS.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Advisory AI agent guide
|
||||
|
||||
## Mission
|
||||
Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance.
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
|
||||
## How to get started
|
||||
1. Review ./architecture.md for retrieval pipeline, guardrails, and profile support.
|
||||
2. Open ../../implplan/SPRINTS.md and locate stories for this component.
|
||||
3. Check ./TASKS.md and update status before/after work.
|
||||
4. Read README/architecture for design context and update as the implementation evolves.
|
||||
|
||||
## Guardrails
|
||||
- Uphold Aggregation-Only Contract boundaries when consuming ingestion data.
|
||||
- Preserve determinism and provenance in all derived outputs.
|
||||
- Document offline/air-gap pathways for any new feature.
|
||||
- Update telemetry/observability assets alongside feature work.
|
||||
29
docs/modules/advisory-ai/README.md
Normal file
29
docs/modules/advisory-ai/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# StellaOps Advisory AI
|
||||
|
||||
Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance.
|
||||
|
||||
## Responsibilities
|
||||
- Generate policy-aware advisory summaries with citations back to Conseiller and Excititor evidence.
|
||||
- Explain conflicting advisories/VEX statements using weights from VEX Lens and Policy Engine.
|
||||
- Propose remediation hints aligned with Offline Kit staging and export bundles.
|
||||
- Expose API/UI surfaces with guardrails on model prompts, outputs, and retention.
|
||||
|
||||
## Key components
|
||||
- RAG pipeline drawing from Conseiller, Excititor, VEX Lens, Policy Engine, and SBOM Service data.
|
||||
- Prompt templates and guard models enforcing provenance and redaction policies.
|
||||
- Vercel/offline inference workers with deterministic caching of generated artefacts.
|
||||
|
||||
## Integrations & dependencies
|
||||
- Authority for tenant-aware access control.
|
||||
- Policy Engine for context-specific decisions and explain traces.
|
||||
- Console/CLI for interaction surfaces.
|
||||
- Export Center/Vuln Explorer for embedding generated briefs.
|
||||
|
||||
## Operational notes
|
||||
- Model cache management and offline bundle packaging per Epic 8 requirements.
|
||||
- Usage/latency dashboards for prompt/response monitoring.
|
||||
- Redaction policies validated against security/LLM guardrail tests.
|
||||
|
||||
## Epic alignment
|
||||
- Epic 8: Advisory AI Assistant.
|
||||
- DOCS-AI stories to be tracked in ../../TASKS.md.
|
||||
9
docs/modules/advisory-ai/TASKS.md
Normal file
9
docs/modules/advisory-ai/TASKS.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Task board — Advisory AI
|
||||
|
||||
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| ADVISORY-AI-DOCS-0001 | TODO | Docs Guild | Ensure ./README.md reflects the latest epic deliverables. | Align with ./AGENTS.md |
|
||||
| ADVISORY-AI-ENG-0001 | TODO | Module Team | Break down epic milestones into actionable stories. | Sync into ../../TASKS.md |
|
||||
| ADVISORY-AI-OPS-0001 | TODO | Ops Guild | Prepare runbooks/observability assets once MVP lands. | Document outputs in ./README.md |
|
||||
100
docs/modules/advisory-ai/architecture.md
Normal file
100
docs/modules/advisory-ai/architecture.md
Normal file
@@ -0,0 +1,100 @@
|
||||
# Advisory AI architecture
|
||||
|
||||
> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
|
||||
|
||||
## 1) Goals
|
||||
|
||||
- Summarise advisories/VEX evidence into operator-ready briefs with citations.
|
||||
- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
|
||||
- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
|
||||
- Operate deterministically where possible; cache generated artefacts with digests for audit.
|
||||
|
||||
## 2) Pipeline overview
|
||||
|
||||
```
|
||||
+---------------------+
|
||||
Concelier/VEX Lens | Evidence Retriever |
|
||||
Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON)
|
||||
Zastava runtime +---------------------+
|
||||
|
|
||||
v
|
||||
+-------------+
|
||||
| Prompt |
|
||||
| Assembler |
|
||||
+-------------+
|
||||
|
|
||||
v
|
||||
+-------------+
|
||||
| Guarded LLM |
|
||||
| (local/host)|
|
||||
+-------------+
|
||||
|
|
||||
v
|
||||
+-----------------+
|
||||
| Citation & |
|
||||
| Validation |
|
||||
+-----------------+
|
||||
|
|
||||
v
|
||||
+----------------+
|
||||
| Output cache |
|
||||
| (hash, bundle) |
|
||||
+----------------+
|
||||
```
|
||||
|
||||
## 3) Retrieval & context
|
||||
|
||||
- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
|
||||
- Context packs include:
|
||||
- Advisory raw excerpts with highlighted sections and source URLs.
|
||||
- VEX statements (normalized tuples + trust metadata).
|
||||
- Policy explain traces for the affected finding.
|
||||
- Runtime/impact hints from Zastava (exposure, entrypoints).
|
||||
- Export-ready remediation data (fixed versions, patches).
|
||||
|
||||
All context references include `content_hash` and `source_id` enabling verifiable citations.
|
||||
|
||||
## 4) Guardrails
|
||||
|
||||
- Prompt templates enforce structure: summary, conflicts, remediation, references.
|
||||
- Response validator ensures:
|
||||
- No hallucinated advisories (every fact must map to input context).
|
||||
- Citations follow `[n]` indexing referencing actual sources.
|
||||
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
|
||||
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
|
||||
|
||||
## 5) Output persistence
|
||||
|
||||
- Cached artefacts stored in `advisory_ai_outputs` with fields:
|
||||
- `output_hash` (sha256 of JSON response).
|
||||
- `input_digest` (hash of context pack).
|
||||
- `summary`, `conflicts`, `remediation`, `citations`.
|
||||
- `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
|
||||
- `signatures` (optional DSSE if run in deterministic mode).
|
||||
- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
|
||||
|
||||
## 6) Profiles & sovereignty
|
||||
|
||||
- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
|
||||
- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
|
||||
|
||||
## 7) APIs
|
||||
|
||||
- `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`.
|
||||
- `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking.
|
||||
- `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps.
|
||||
- `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center).
|
||||
|
||||
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
|
||||
|
||||
## 8) Observability
|
||||
|
||||
- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
|
||||
- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
|
||||
- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
|
||||
|
||||
## 9) Operational controls
|
||||
|
||||
- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
|
||||
- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
|
||||
- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.
|
||||
19
docs/modules/advisory-ai/implementation_plan.md
Normal file
19
docs/modules/advisory-ai/implementation_plan.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Implementation plan — Advisory AI
|
||||
|
||||
## Current objectives
|
||||
- Deliver Epic milestones summarised below while maintaining determinism and offline parity.
|
||||
- Keep documentation, telemetry, and runbooks aligned with sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Roadmap: reconcile open stories in ../../TASKS.md with module backlog.
|
||||
- Delivery: ship features outlined in the epic while preserving AOC guardrails.
|
||||
- Validation: extend tests/fixtures to guarantee reproducibility and provenance.
|
||||
|
||||
## Epic milestones
|
||||
- Epic 8: Advisory AI Assistant.
|
||||
- DOCS-AI stories to be tracked in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up work.
|
||||
- Sync with owners listed in docs/implplan/SPRINTS.md.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
22
docs/modules/attestor/AGENTS.md
Normal file
22
docs/modules/attestor/AGENTS.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Attestor agent guide
|
||||
|
||||
## Mission
|
||||
Attestor moves signed evidence through the trust chain by accepting DSSE bundles from Signer, registering them with Rekor v2, and serving deterministic verification payloads to other services.
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
|
||||
## How to get started
|
||||
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
|
||||
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
|
||||
3. Read the architecture and README for domain context before editing code or docs.
|
||||
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
|
||||
|
||||
## Guardrails
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
|
||||
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
|
||||
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
|
||||
- Update runbooks/observability assets when operational characteristics change.
|
||||
54
docs/modules/attestor/README.md
Normal file
54
docs/modules/attestor/README.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# StellaOps Attestor
|
||||
|
||||
Attestor converts signed DSSE evidence from the Signer into transparency-log proofs and verifiable reports for every downstream surface (Policy Engine, Export Center, CLI, Console, Scheduler). It is the trust backbone that proves SBOM, scan, VEX, and policy artefacts were signed, witnessed, and preserved without tampering.
|
||||
|
||||
## Why it exists
|
||||
- **Evidence first:** organisations need portable, verifiable attestations that prove build provenance, SBOM availability, policy verdicts, and VEX statements.
|
||||
- **Policy enforcement:** verification policies ensure only approved issuers, key types, witnesses, and freshness windows are accepted.
|
||||
- **Sovereign/offline-ready:** Attestor archives envelopes, signatures, and proofs so air-gapped deployments can replay verification without contacting external services.
|
||||
|
||||
## Roles & surfaces
|
||||
- **Subjects:** immutable digests for container images, SBOMs, reports, and policy bundles.
|
||||
- **Issuers:** builders, scanners, policy engines, or operators signing DSSE envelopes using keyless (Fulcio), KMS/HSM, or FIDO2 keys.
|
||||
- **Consumers:** CLI/SDK, Console, Export Center, Scanner, Policy Engine, and Notify retrieving verification bundles or triggering policy checks.
|
||||
- **Scopes:** Authority issues `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for issuer/key management; every call is bound with mTLS + DPoP.
|
||||
|
||||
## Supported payloads
|
||||
- `StellaOps.BuildProvenance@1`, `StellaOps.SBOMAttestation@1`
|
||||
- `StellaOps.ScanResults@1`, `StellaOps.VEXAttestation@1`
|
||||
- `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1`
|
||||
All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`.
|
||||
|
||||
## Trust & envelope model
|
||||
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
|
||||
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
|
||||
- Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts.
|
||||
- CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors.
|
||||
|
||||
## UI, CLI, and SDK workflows
|
||||
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
|
||||
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
|
||||
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
|
||||
|
||||
## Storage, offline & air-gap posture
|
||||
- MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
|
||||
- Export Center packages attestation bundles (`stella export attestation-bundle`) for Offline Kit delivery.
|
||||
- Transparency logs can be mirrored; offline mode records gaps and provides compensating controls.
|
||||
|
||||
## Observability & performance
|
||||
- Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_cache_hit_ratio`, `attestor_rekor_latency_seconds`.
|
||||
- Logs capture tenant, issuer, subject digests, Rekor UUID, proof status, and policy verdict.
|
||||
- Performance target: ≥1 000 envelopes/minute per worker with cached verification, batched operations, and concurrency controls.
|
||||
|
||||
## Key integrations
|
||||
- Signer (DSSE source), Authority (scopes & tenancy), Export Center (attestation bundles), Policy Engine (verification policies), Scanner/Excititor (subject evidence), Notify (key rotation & verification alerts), Observability stack (dashboards/alerts).
|
||||
|
||||
## Backlog references
|
||||
- DOCS-ATTEST-73-001 … DOCS-ATTEST-75-002 (Attestor console, key management, air-gap bundles) in ../../TASKS.md.
|
||||
- EXPORT-ATTEST-75-002 (Export Center attestation packaging) in ../export-center/TASKS.md.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 19 – Attestor Console:** console experience, verification APIs, issuer/key governance, transparency integration, and offline bundles.
|
||||
- **Epic 10 – Export Center:** provenance alignment so exports carry signed manifests and attestation bundles.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
9
docs/modules/attestor/TASKS.md
Normal file
9
docs/modules/attestor/TASKS.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Task board — Attestor
|
||||
|
||||
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| ATTESTOR-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
|
||||
| ATTESTOR-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
|
||||
| ATTESTOR-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
|
||||
@@ -1,5 +1,7 @@
|
||||
# component_architecture_attestor.md — **Stella Ops Attestor** (2025Q4)
|
||||
|
||||
> Derived from Epic 19 – Attestor Console with provenance hooks aligned to the Export Center bundle workflows scoped in Epic 10.
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for the **Attestor**: the service that **submits** DSSE envelopes to **Rekor v2**, retrieves/validates inclusion proofs, caches results, and exposes verification APIs. It accepts DSSE **only** from the **Signer** over mTLS, enforces chain‑of‑trust to Stella Ops roots, and returns `{uuid, index, proof, logURL}` to calling services (Scanner.WebService for SBOMs; backend for final reports; Excititor exports when configured).
|
||||
|
||||
---
|
||||
@@ -33,6 +35,47 @@ Trust boundary: **Only the Signer** is allowed to call submission endpoints; enf
|
||||
|
||||
---
|
||||
|
||||
### Roles, identities & scopes
|
||||
- **Subjects** — immutable digests for artifacts (container images, SBOMs, reports) referenced in DSSE envelopes.
|
||||
- **Issuers** — authenticated builders/scanners/policy engines signing evidence; tracked with mode (`keyless`, `kms`, `hsm`, `fido2`) and tenant scope.
|
||||
- **Consumers** — Scanner, Export Center, CLI, Console, Policy Engine that verify proofs using Attestor APIs.
|
||||
- **Authority scopes** — `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for key management; all calls mTLS/DPoP-bound.
|
||||
|
||||
### Supported predicate types
|
||||
- `StellaOps.BuildProvenance@1`
|
||||
- `StellaOps.SBOMAttestation@1`
|
||||
- `StellaOps.ScanResults@1`
|
||||
- `StellaOps.PolicyEvaluation@1`
|
||||
- `StellaOps.VEXAttestation@1`
|
||||
- `StellaOps.RiskProfileEvidence@1`
|
||||
|
||||
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
|
||||
|
||||
### Envelope & signature model
|
||||
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
|
||||
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
|
||||
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
|
||||
- Archive CAS retains original envelope plus metadata for offline verification.
|
||||
|
||||
### Verification pipeline overview
|
||||
1. Fetch envelope (from request, cache, or storage) and validate DSSE structure.
|
||||
2. Verify signature(s) against configured trust roots; evaluate issuer policy.
|
||||
3. Retrieve or acquire inclusion proof from Rekor (primary + optional mirror).
|
||||
4. Validate Merkle proof against checkpoint; optionally verify witness endorsement.
|
||||
5. Return cached verification bundle including policy verdict and timestamps.
|
||||
|
||||
### UI & CLI touchpoints
|
||||
- Console: Evidence browser, verification report, chain-of-custody graph, issuer/key management, attestation workbench, bulk verification views.
|
||||
- CLI: `stella attest sign|verify|list|fetch|key` with offline verification and export bundle support.
|
||||
- SDKs expose sign/verify primitives for build pipelines.
|
||||
|
||||
### Performance & observability targets
|
||||
- Throughput goal: ≥1 000 envelopes/minute per worker with cached verification.
|
||||
- Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_rekor_latency_seconds`, `attestor_cache_hit_ratio`.
|
||||
- Logs include `tenant`, `issuer`, `subjectDigest`, `rekorUuid`, `proofStatus`; traces cover submission → Rekor → cache → response path.
|
||||
|
||||
---
|
||||
|
||||
## 2) Data model (Mongo)
|
||||
|
||||
Database: `attestor`
|
||||
74
docs/modules/attestor/implementation_plan.md
Normal file
74
docs/modules/attestor/implementation_plan.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Implementation plan — Attestor
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Foundations**
|
||||
Build the Attestor service skeleton, DSSE bundle ingestion, mTLS/OpTok enforcement, Rekor v2 client, and cache the `{uuid,index,proof}` tuple. Publish base API (`POST /rekor/entries`, `GET /entries/{uuid}`) and Mongo schemas.
|
||||
- **Phase 2 – Policies & UI**
|
||||
Deliver verification policy authoring (Policy Studio integration), console views (evidence browser, verification reports, issuer management), and CLI verbs (`stella attest sign|verify|list|fetch`).
|
||||
- **Phase 3 – Scan & VEX support**
|
||||
Accept SBOM, ScanResults, VEX, and PolicyEvaluation predicates; integrate with Scanner, Export Center, Excititor, and Policy Engine pipelines. Ensure AOC invariants on ingestion.
|
||||
- **Phase 4 – Transparency & keys**
|
||||
Add multi-log submission (primary + mirror), witness endorsements, KMS/HSM/FIDO2 drivers, key rotation/revocation workflows, and audit trails.
|
||||
- **Phase 5 – Bulk & air gap**
|
||||
Implement batch submission/verification, DSSE archival to CAS/object storage, export/import bundles for Offline Kit, and mirror transparency log snapshots.
|
||||
- **Phase 6 – Performance & hardening**
|
||||
Optimise cache usage, parallel verification (target ≥1 k envelopes/minute per worker), extend observability (metrics/logs/traces), fuzz parsers, and finalise incident playbooks.
|
||||
|
||||
## Work breakdown
|
||||
- **Attestor service & libraries**
|
||||
- DSSE validation pipeline (payload whitelist, signature verification, trust roots).
|
||||
- Rekor client with inclusion-proof acquisition, retry/backoff, mirroring controls.
|
||||
- Mongo repositories for entries, dedupe, audit; CAS storage for DSSE envelopes.
|
||||
- Batch submission/verification APIs, verification cache, deterministic serialization.
|
||||
- Observability hooks: metrics (`attestor_submission_total`, `attestor_verify_seconds`), structured logs, OpenTelemetry traces.
|
||||
- **Signer & Authority integration**
|
||||
- Enforce mTLS peer validation, Authority scope mapping (`attestor.write`, `attestor.verify`), and DPoP binding.
|
||||
- Provide signer identity attestation metadata consumed by Attestor.
|
||||
- **Policy & Console**
|
||||
- Extend Policy Studio with `VerificationPolicy` authoring, approvals, and simulated results.
|
||||
- Console workflows: Evidence browser, verification reports, chain-of-custody graph, key management UI, bulk verification screens.
|
||||
- **CLI & SDK**
|
||||
- `stella attest` command group (sign/verify/list/fetch/key management) with DSSE canonicalisation and cosign interoperability.
|
||||
- SDK helpers for DSSE envelope creation, verification, and proof inspection.
|
||||
- **Export Center & Offline Kit**
|
||||
- Export Center adapters for attestation bundles; CLI/Console flows to export & import evidence in air-gapped environments.
|
||||
- Offline Kit scripts for replaying verification, mirroring transparency logs, and reporting gaps.
|
||||
- **Security & key management**
|
||||
- KMS/HSM/FIDO2 driver abstraction, key rotation and revocation runbooks, witness endorsements, and revocation telemetry.
|
||||
- **Docs & training**
|
||||
- Update module dossier (overview, architecture, implementation plan), key management guides, transparency reference, CLI/Console documentation, and air-gap runbooks.
|
||||
|
||||
## Cross-module dependencies
|
||||
- **Policy Studio / Policy Engine:** verification policy artefacts, explain integration, remediation hints.
|
||||
- **Export Center:** attestation bundle export/import, provenance linking.
|
||||
- **Authority & Tenancy:** scopes, identity attestations, tenant-aware issuer catalogues.
|
||||
- **Notifications:** attestation success/failure events, key rotation alerts.
|
||||
- **Observability:** dashboards and alerting for signing/verification pipelines.
|
||||
|
||||
## Acceptance criteria
|
||||
- Service ingests DSSE envelopes for all supported predicate types, logs them to configured transparency logs, and returns proofs with deterministic hashes.
|
||||
- Verification APIs/CLI/UI validate signatures, inclusion proofs, and policy compliance; cached verification accelerates repeated checks.
|
||||
- Verification policies gate attestation usage, enforcing issuer, freshness, signature count, and witness requirements.
|
||||
- Export Center and Offline Kit workflows bundle attestations and replay verification offline.
|
||||
- Observability coverage includes metrics, traces, logs, audit events, and alert triggers for key compromise, log outages, and verification failure spikes.
|
||||
- Performance target met (≥1 k envelopes/minute per worker) with horizontal scaling.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Key compromise or leakage:** enforce hardware-backed keys, rotation procedures, revocation checks, and incident runbooks.
|
||||
- **Parser bugs / malformed DSSE:** fuzz DSSE and predicate schemas, strict schema validation, fail closed.
|
||||
- **Transparency outage:** mirror logs, support witness endorsements, queue submissions for retry with exponential backoff.
|
||||
- **Policy complexity:** ship curated starter policies, provide simulation tooling, and document common scenarios.
|
||||
- **Offline gaps:** archive bundles and proof material, surface gaps to operators, and document compensating controls.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** DSSE validation, Rekor client, dedupe logic, key drivers, policy enforcement.
|
||||
- **Integration:** submit/verify flows across predicate types, multi-log publishing, batch operations, CLI/UI end-to-end exercises.
|
||||
- **Security:** tenant isolation, scope enforcement, key rotation regression, tamper detection.
|
||||
- **Performance:** throughput benchmarks, cache hit-rate monitoring, large batch verification.
|
||||
- **Chaos:** inject Rekor outages, network failures, corrupt bundles; ensure graceful degradation and auditable alerts.
|
||||
|
||||
## Definition of done
|
||||
- Phased milestones delivered with telemetry, documentation, and runbooks in place.
|
||||
- CLI/Console parity verified; Offline Kit procedures validated in sealed environment.
|
||||
- Cross-module dependencies acknowledged in ./TASKS.md and ../../TASKS.md.
|
||||
- Documentation set refreshed (overview, architecture, key management, transparency, CLI/UI) with imposed rule statement.
|
||||
22
docs/modules/authority/AGENTS.md
Normal file
22
docs/modules/authority/AGENTS.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Authority agent guide
|
||||
|
||||
## Mission
|
||||
Authority is the platform OIDC/OAuth2 control plane that mints short-lived, sender-constrained operational tokens (OpToks) for every StellaOps service and tool.
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
|
||||
## How to get started
|
||||
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
|
||||
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
|
||||
3. Read the architecture and README for domain context before editing code or docs.
|
||||
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
|
||||
|
||||
## Guardrails
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
|
||||
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
|
||||
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
|
||||
- Update runbooks/observability assets when operational characteristics change.
|
||||
40
docs/modules/authority/README.md
Normal file
40
docs/modules/authority/README.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# StellaOps Authority
|
||||
|
||||
Authority is the platform OIDC/OAuth2 control plane that mints short-lived, sender-constrained operational tokens (OpToks) for every StellaOps service and tool.
|
||||
|
||||
## Responsibilities
|
||||
- Expose device-code, auth-code, and client-credential flows with DPoP or mTLS binding.
|
||||
- Manage signing keys, JWKS rotation, and PoE integration for plan enforcement.
|
||||
- Emit structured audit events and enforce tenant-aware scope policies.
|
||||
- Provide plugin surface for custom identity providers and credential validators.
|
||||
|
||||
## Key components
|
||||
- `StellaOps.Authority` web host.
|
||||
- `StellaOps.Authority.Plugin.*` extensions for secret stores, identity bridges, and OpTok validation.
|
||||
- Telemetry and audit pipeline feeding Security/Observability stacks.
|
||||
|
||||
## Integrations & dependencies
|
||||
- Signer/Attestor for PoE and OpTok introspection.
|
||||
- CLI/UI for login flows and token management.
|
||||
- Scheduler/Scanner for machine-to-machine scope enforcement.
|
||||
|
||||
## Operational notes
|
||||
- MongoDB for tenant, client, and token state.
|
||||
- Key material in KMS/HSM with rotation runbooks (see ./operations/key-rotation.md).
|
||||
- Grafana/Prometheus dashboards for auth latency/issuance.
|
||||
|
||||
## Related resources
|
||||
- ./operations/backup-restore.md
|
||||
- ./operations/key-rotation.md
|
||||
- ./operations/monitoring.md
|
||||
- ./operations/grafana-dashboard.json
|
||||
|
||||
## Backlog references
|
||||
- DOCS-SEC-62-001 (scope hardening doc) in ../../TASKS.md.
|
||||
- AUTH-POLICY-20-001/002 follow-ups in src/Authority/StellaOps.Authority/TASKS.md.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 1 – AOC enforcement:** enforce OpTok scopes and guardrails supporting raw ingestion boundaries.
|
||||
- **Epic 2 – Policy Engine & Editor:** supply policy evaluation/principal scopes and short-lived tokens for evaluator workflows.
|
||||
- **Epic 4 – Policy Studio:** integrate approval/promotion signatures and policy registry access controls.
|
||||
- **Epic 14 – Identity & Tenancy:** deliver tenant isolation, RBAC hierarchies, and governance tooling for authentication.
|
||||
9
docs/modules/authority/TASKS.md
Normal file
9
docs/modules/authority/TASKS.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Task board — Authority
|
||||
|
||||
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| AUTHORITY-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
|
||||
| AUTHORITY-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
|
||||
| AUTHORITY-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
|
||||
@@ -1,4 +1,6 @@
|
||||
# component_architecture_authority.md — **Stella Ops Authority** (2025Q4)
|
||||
# component_architecture_authority.md — **Stella Ops Authority** (2025Q4)
|
||||
|
||||
> Consolidates identity and tenancy requirements documented across the AOC, Policy, and Platform guides, along with the dedicated Authority implementation plan.
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops Authority**: the on‑prem **OIDC/OAuth2** service that issues **short‑lived, sender‑constrained operational tokens (OpToks)** to first‑party services and tools. Covers protocols (DPoP & mTLS binding), token shapes, endpoints, storage, rotation, HA, RBAC, audit, and testing. This component is the trust anchor for *who* is calling inside a Stella Ops installation. (Entitlement is proven separately by **PoE** from the cloud Licensing Service; Authority does not issue PoE.)
|
||||
|
||||
22
docs/modules/authority/implementation_plan.md
Normal file
22
docs/modules/authority/implementation_plan.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Implementation plan — Authority
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** deliver OpTok scopes, guardrails, and AOC verifier hooks for ingestion services.
|
||||
- **Epic 2 – Policy Engine & Editor:** support policy evaluator flows (device-code, client credentials, scope sandboxing).
|
||||
- **Epic 4 – Policy Studio:** provide registry/promotion signing, approvals, and fresh-auth prompts.
|
||||
- **Epic 14 – Identity & Tenancy:** implement tenant isolation, RBAC hierarchies, audit trails, and PoE integration.
|
||||
- Track additional work (DOCS-SEC-62-001, AUTH-POLICY-20-001/002) in ../../TASKS.md and src/Authority/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -1,94 +1,94 @@
|
||||
# Authority Signing Key Rotation Playbook
|
||||
|
||||
> **Status:** Authored 2025-10-12 as part of OPS3.KEY-ROTATION rollout.
|
||||
> Use together with `docs/11_AUTHORITY.md` (Authority service guide) and the automation shipped under `ops/authority/`.
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Authority publishes JWKS and revocation bundles signed with ES256 keys. To rotate those keys without downtime we now provide:
|
||||
|
||||
- **Automation script:** `ops/authority/key-rotation.sh`
|
||||
Shell helper that POSTS to `/internal/signing/rotate`, supports metadata, dry-run, and confirms JWKS afterwards.
|
||||
- **CI workflow:** `.gitea/workflows/authority-key-rotation.yml`
|
||||
Manual dispatch workflow that pulls environment-specific secrets, runs the script, and records the result. Works across staging/production by passing the `environment` input.
|
||||
|
||||
This playbook documents the repeatable sequence for all environments.
|
||||
|
||||
## 2. Pre-requisites
|
||||
|
||||
1. **Generate a new PEM key (per environment)**
|
||||
```bash
|
||||
openssl ecparam -name prime256v1 -genkey -noout \
|
||||
-out certificates/authority-signing-<env>-<year>.pem
|
||||
chmod 600 certificates/authority-signing-<env>-<year>.pem
|
||||
```
|
||||
2. **Stash the previous key** under the same volume so it can be referenced in `signing.additionalKeys` after rotation.
|
||||
3. **Ensure secrets/vars exist in Gitea**
|
||||
- `<ENV>_AUTHORITY_BOOTSTRAP_KEY`
|
||||
- `<ENV>_AUTHORITY_URL`
|
||||
- Optional shared defaults `AUTHORITY_BOOTSTRAP_KEY`, `AUTHORITY_URL`.
|
||||
|
||||
## 3. Executing the rotation
|
||||
|
||||
### Option A – via CI workflow (recommended)
|
||||
|
||||
1. Navigate to **Actions → Authority Key Rotation**.
|
||||
2. Provide inputs:
|
||||
- `environment`: `staging`, `production`, etc.
|
||||
- `key_id`: new `kid` (e.g. `authority-signing-2025-dev`).
|
||||
- `key_path`: path as seen by the Authority service (e.g. `../certificates/authority-signing-2025-dev.pem`).
|
||||
- Optional `metadata`: comma-separated `key=value` pairs (for audit trails).
|
||||
3. Trigger. The workflow:
|
||||
- Reads the bootstrap key/URL from secrets.
|
||||
- Runs `ops/authority/key-rotation.sh`.
|
||||
- Prints the JWKS response for verification.
|
||||
|
||||
### Option B – manual shell invocation
|
||||
|
||||
```bash
|
||||
AUTHORITY_BOOTSTRAP_KEY=$(cat /secure/authority-bootstrap.key) \
|
||||
./ops/authority/key-rotation.sh \
|
||||
--authority-url https://authority.example.com \
|
||||
--key-id authority-signing-2025-dev \
|
||||
--key-path ../certificates/authority-signing-2025-dev.pem \
|
||||
--meta rotatedBy=ops --meta changeTicket=OPS-1234
|
||||
```
|
||||
|
||||
Use `--dry-run` to inspect the payload before execution.
|
||||
|
||||
## 4. Post-rotation checklist
|
||||
|
||||
1. Update `authority.yaml` (or environment-specific overrides):
|
||||
- Set `signing.activeKeyId` to the new key.
|
||||
- Set `signing.keyPath` to the new PEM.
|
||||
- Append the previous key into `signing.additionalKeys`.
|
||||
- Ensure `keySource`/`provider` match the values passed to the script.
|
||||
2. Run `stellaops-cli auth revoke export` so revocation bundles are re-signed with the new key.
|
||||
3. Confirm `/jwks` lists the new `kid` with `status: "active"` and the previous one as `retired`.
|
||||
4. Archive the old key securely; keep it available until all tokens/bundles signed with it have expired.
|
||||
|
||||
## 5. Development key state
|
||||
|
||||
For the sample configuration (`etc/authority.yaml.sample`) we minted a placeholder dev key:
|
||||
|
||||
- Active: `authority-signing-2025-dev` (`certificates/authority-signing-2025-dev.pem`)
|
||||
- Retired: `authority-signing-dev`
|
||||
|
||||
Treat these as examples; real environments must maintain their own PEM material.
|
||||
|
||||
## 6. References
|
||||
|
||||
- `docs/11_AUTHORITY.md` – Architecture and rotation SOP (Section 5).
|
||||
- `docs/ops/authority-backup-restore.md` – Recovery flow referencing this playbook.
|
||||
- `ops/authority/README.md` – CLI usage and examples.
|
||||
- `scripts/rotate-policy-cli-secret.sh` – Helper to mint new `policy-cli` shared secrets when policy scope bundles change.
|
||||
|
||||
## 7. Appendix — Policy CLI secret rotation
|
||||
|
||||
Scope migrations such as AUTH-POLICY-23-004 require issuing fresh credentials for the `policy-cli` client. Use the helper script committed with the repo to keep secrets deterministic across environments.
|
||||
|
||||
```bash
|
||||
./scripts/rotate-policy-cli-secret.sh --output etc/secrets/policy-cli.secret
|
||||
```
|
||||
|
||||
The script writes a timestamped header and a random secret into the target file. Use `--dry-run` when generating material for external secret stores. After updating secrets in staging/production, recycle the Authority pods and confirm the new client credentials work before the next release freeze.
|
||||
# Authority Signing Key Rotation Playbook
|
||||
|
||||
> **Status:** Authored 2025-10-12 as part of OPS3.KEY-ROTATION rollout.
|
||||
> Use together with `docs/11_AUTHORITY.md` (Authority service guide) and the automation shipped under `ops/authority/`.
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Authority publishes JWKS and revocation bundles signed with ES256 keys. To rotate those keys without downtime we now provide:
|
||||
|
||||
- **Automation script:** `ops/authority/key-rotation.sh`
|
||||
Shell helper that POSTS to `/internal/signing/rotate`, supports metadata, dry-run, and confirms JWKS afterwards.
|
||||
- **CI workflow:** `.gitea/workflows/authority-key-rotation.yml`
|
||||
Manual dispatch workflow that pulls environment-specific secrets, runs the script, and records the result. Works across staging/production by passing the `environment` input.
|
||||
|
||||
This playbook documents the repeatable sequence for all environments.
|
||||
|
||||
## 2. Pre-requisites
|
||||
|
||||
1. **Generate a new PEM key (per environment)**
|
||||
```bash
|
||||
openssl ecparam -name prime256v1 -genkey -noout \
|
||||
-out certificates/authority-signing-<env>-<year>.pem
|
||||
chmod 600 certificates/authority-signing-<env>-<year>.pem
|
||||
```
|
||||
2. **Stash the previous key** under the same volume so it can be referenced in `signing.additionalKeys` after rotation.
|
||||
3. **Ensure secrets/vars exist in Gitea**
|
||||
- `<ENV>_AUTHORITY_BOOTSTRAP_KEY`
|
||||
- `<ENV>_AUTHORITY_URL`
|
||||
- Optional shared defaults `AUTHORITY_BOOTSTRAP_KEY`, `AUTHORITY_URL`.
|
||||
|
||||
## 3. Executing the rotation
|
||||
|
||||
### Option A – via CI workflow (recommended)
|
||||
|
||||
1. Navigate to **Actions → Authority Key Rotation**.
|
||||
2. Provide inputs:
|
||||
- `environment`: `staging`, `production`, etc.
|
||||
- `key_id`: new `kid` (e.g. `authority-signing-2025-dev`).
|
||||
- `key_path`: path as seen by the Authority service (e.g. `../certificates/authority-signing-2025-dev.pem`).
|
||||
- Optional `metadata`: comma-separated `key=value` pairs (for audit trails).
|
||||
3. Trigger. The workflow:
|
||||
- Reads the bootstrap key/URL from secrets.
|
||||
- Runs `ops/authority/key-rotation.sh`.
|
||||
- Prints the JWKS response for verification.
|
||||
|
||||
### Option B – manual shell invocation
|
||||
|
||||
```bash
|
||||
AUTHORITY_BOOTSTRAP_KEY=$(cat /secure/authority-bootstrap.key) \
|
||||
./ops/authority/key-rotation.sh \
|
||||
--authority-url https://authority.example.com \
|
||||
--key-id authority-signing-2025-dev \
|
||||
--key-path ../certificates/authority-signing-2025-dev.pem \
|
||||
--meta rotatedBy=ops --meta changeTicket=OPS-1234
|
||||
```
|
||||
|
||||
Use `--dry-run` to inspect the payload before execution.
|
||||
|
||||
## 4. Post-rotation checklist
|
||||
|
||||
1. Update `authority.yaml` (or environment-specific overrides):
|
||||
- Set `signing.activeKeyId` to the new key.
|
||||
- Set `signing.keyPath` to the new PEM.
|
||||
- Append the previous key into `signing.additionalKeys`.
|
||||
- Ensure `keySource`/`provider` match the values passed to the script.
|
||||
2. Run `stellaops-cli auth revoke export` so revocation bundles are re-signed with the new key.
|
||||
3. Confirm `/jwks` lists the new `kid` with `status: "active"` and the previous one as `retired`.
|
||||
4. Archive the old key securely; keep it available until all tokens/bundles signed with it have expired.
|
||||
|
||||
## 5. Development key state
|
||||
|
||||
For the sample configuration (`etc/authority.yaml.sample`) we minted a placeholder dev key:
|
||||
|
||||
- Active: `authority-signing-2025-dev` (`certificates/authority-signing-2025-dev.pem`)
|
||||
- Retired: `authority-signing-dev`
|
||||
|
||||
Treat these as examples; real environments must maintain their own PEM material.
|
||||
|
||||
## 6. References
|
||||
|
||||
- `docs/11_AUTHORITY.md` – Architecture and rotation SOP (Section 5).
|
||||
- `docs/modules/authority/operations/backup-restore.md` – Recovery flow referencing this playbook.
|
||||
- `ops/authority/README.md` – CLI usage and examples.
|
||||
- `scripts/rotate-policy-cli-secret.sh` – Helper to mint new `policy-cli` shared secrets when policy scope bundles change.
|
||||
|
||||
## 7. Appendix — Policy CLI secret rotation
|
||||
|
||||
Scope migrations such as AUTH-POLICY-23-004 require issuing fresh credentials for the `policy-cli` client. Use the helper script committed with the repo to keep secrets deterministic across environments.
|
||||
|
||||
```bash
|
||||
./scripts/rotate-policy-cli-secret.sh --output etc/secrets/policy-cli.secret
|
||||
```
|
||||
|
||||
The script writes a timestamped header and a random secret into the target file. Use `--dry-run` when generating material for external secret stores. After updating secrets in staging/production, recycle the Authority pods and confirm the new client credentials work before the next release freeze.
|
||||
@@ -1,83 +1,83 @@
|
||||
# Authority Monitoring & Alerting Playbook
|
||||
|
||||
## Telemetry Sources
|
||||
- **Traces:** Activity source `StellaOps.Authority` emits spans for every token flow (`authority.token.validate_*`, `authority.token.handle_*`, `authority.token.validate_access`). Key tags include `authority.endpoint`, `authority.grant_type`, `authority.username`, `authority.client_id`, and `authority.identity_provider`.
|
||||
- **Metrics:** OpenTelemetry instrumentation (`AddAspNetCoreInstrumentation`, `AddHttpClientInstrumentation`, custom meter `StellaOps.Authority`) exports:
|
||||
- `http.server.request.duration` histogram (`http_route`, `http_status_code`, `authority.endpoint` tag via `aspnetcore` enrichment).
|
||||
- `process.runtime.gc.*`, `process.runtime.dotnet.*` (from `AddRuntimeInstrumentation`).
|
||||
- **Logs:** Serilog writes structured events to stdout. Notable templates:
|
||||
- `"Password grant verification failed ..."` and `"Plugin {PluginName} denied access ... due to lockout"` (lockout spike detector).
|
||||
- `"Password grant validation failed for {Username}: provider '{Provider}' does not support MFA required for exception approvals."` (identifies users attempting `exceptions:approve` without MFA support; tie to fresh-auth errors).
|
||||
- `"Client credentials validation failed for {ClientId}: exception scopes require tenant assignment."` (signals misconfigured exception service identities).
|
||||
- `"Granting StellaOps bypass for remote {RemoteIp}"` (bypass usage).
|
||||
- `"Rate limit exceeded for path {Path} from {RemoteIp}"` (limiter alerts).
|
||||
|
||||
## Prometheus Metrics to Collect
|
||||
| Metric | Query | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `token_requests_total` | `sum by (grant_type, status) (rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Token issuance volume per grant type (`grant_type` comes via `authority.grant_type` span attribute → Exemplars in Grafana). |
|
||||
| `token_failure_ratio` | `sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token", http_status_code=~"4..|5.."}[5m])) / sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Alert when > 5 % for 10 min. |
|
||||
| `authorize_rate_limit_hits` | `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority", limiter="authority-token"}[5m]))` | Detect rate limiting saturations (requires OTEL ASP.NET rate limiter exporter). |
|
||||
| `lockout_events` | `sum by (plugin) (rate(log_messages_total{app="stellaops-authority", level="Warning", message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[5m]))` | Derived from Loki/Promtail log counter. |
|
||||
| `bypass_usage_total` | `sum(rate(log_messages_total{app="stellaops-authority", level="Information", message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m]))` | Track trusted bypass invocations. |
|
||||
|
||||
> **Exporter note:** Enable `aspnetcore` meters (`dotnet-counters` name `Microsoft.AspNetCore.Hosting`), or configure the OpenTelemetry Collector `metrics` pipeline with `metric_statements` to remap histogram counts into the shown series.
|
||||
|
||||
## Alert Rules
|
||||
1. **Token Failure Surge**
|
||||
- _Expression_: `token_failure_ratio > 0.05`
|
||||
- _For_: `10m`
|
||||
- _Labels_: `severity="critical"`
|
||||
- _Annotations_: Include `topk(5, sum by (authority_identity_provider) (increase(authority_token_rejections_total[10m])))` as diagnostic hint (requires span → metric transformation).
|
||||
2. **Lockout Spike**
|
||||
- _Expression_: `sum(rate(log_messages_total{message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[15m])) > 10`
|
||||
- _For_: `15m`
|
||||
- Investigate credential stuffing; consider temporarily tightening `RateLimiting.Token`.
|
||||
3. **Bypass Threshold**
|
||||
- _Expression_: `sum(rate(log_messages_total{message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m])) > 1`
|
||||
- _For_: `5m`
|
||||
- Alert severity `warning` — verify the calling host list.
|
||||
4. **Rate Limiter Saturation**
|
||||
- _Expression_: `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority"}[5m])) > 0`
|
||||
- Escalate if sustained for 5 min; confirm trusted clients aren’t misconfigured.
|
||||
|
||||
## Grafana Dashboard
|
||||
- Import `docs/ops/authority-grafana-dashboard.json` to provision baseline panels:
|
||||
- **Token Success vs Failure** – stacked rate visualization split by grant type.
|
||||
- **Rate Limiter Hits** – bar chart showing `authority-token` and `authority-authorize`.
|
||||
- **Bypass & Lockout Events** – dual-stat panel using Loki-derived counters.
|
||||
- **Trace Explorer Link** – panel links to `StellaOps.Authority` span search pre-filtered by `authority.grant_type`.
|
||||
|
||||
## Collector Configuration Snippets
|
||||
```yaml
|
||||
receivers:
|
||||
otlp:
|
||||
protocols:
|
||||
http:
|
||||
exporters:
|
||||
prometheus:
|
||||
endpoint: "0.0.0.0:9464"
|
||||
processors:
|
||||
batch:
|
||||
attributes/token_grant:
|
||||
actions:
|
||||
- key: grant_type
|
||||
action: upsert
|
||||
from_attribute: authority.grant_type
|
||||
service:
|
||||
pipelines:
|
||||
metrics:
|
||||
receivers: [otlp]
|
||||
processors: [attributes/token_grant, batch]
|
||||
exporters: [prometheus]
|
||||
logs:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [loki]
|
||||
```
|
||||
|
||||
## Operational Checklist
|
||||
- [ ] Confirm `STELLAOPS_AUTHORITY__OBSERVABILITY__EXPORTERS` enables OTLP in production builds.
|
||||
- [ ] Ensure Promtail captures container stdout with Serilog structured formatting.
|
||||
- [ ] Periodically validate alert noise by running load tests that trigger the rate limiter.
|
||||
- [ ] Include dashboard JSON in Offline Kit for air-gapped clusters; update version header when metrics change.
|
||||
# Authority Monitoring & Alerting Playbook
|
||||
|
||||
## Telemetry Sources
|
||||
- **Traces:** Activity source `StellaOps.Authority` emits spans for every token flow (`authority.token.validate_*`, `authority.token.handle_*`, `authority.token.validate_access`). Key tags include `authority.endpoint`, `authority.grant_type`, `authority.username`, `authority.client_id`, and `authority.identity_provider`.
|
||||
- **Metrics:** OpenTelemetry instrumentation (`AddAspNetCoreInstrumentation`, `AddHttpClientInstrumentation`, custom meter `StellaOps.Authority`) exports:
|
||||
- `http.server.request.duration` histogram (`http_route`, `http_status_code`, `authority.endpoint` tag via `aspnetcore` enrichment).
|
||||
- `process.runtime.gc.*`, `process.runtime.dotnet.*` (from `AddRuntimeInstrumentation`).
|
||||
- **Logs:** Serilog writes structured events to stdout. Notable templates:
|
||||
- `"Password grant verification failed ..."` and `"Plugin {PluginName} denied access ... due to lockout"` (lockout spike detector).
|
||||
- `"Password grant validation failed for {Username}: provider '{Provider}' does not support MFA required for exception approvals."` (identifies users attempting `exceptions:approve` without MFA support; tie to fresh-auth errors).
|
||||
- `"Client credentials validation failed for {ClientId}: exception scopes require tenant assignment."` (signals misconfigured exception service identities).
|
||||
- `"Granting StellaOps bypass for remote {RemoteIp}"` (bypass usage).
|
||||
- `"Rate limit exceeded for path {Path} from {RemoteIp}"` (limiter alerts).
|
||||
|
||||
## Prometheus Metrics to Collect
|
||||
| Metric | Query | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `token_requests_total` | `sum by (grant_type, status) (rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Token issuance volume per grant type (`grant_type` comes via `authority.grant_type` span attribute → Exemplars in Grafana). |
|
||||
| `token_failure_ratio` | `sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token", http_status_code=~"4..|5.."}[5m])) / sum(rate(http_server_duration_seconds_count{service_name="stellaops-authority", http_route="/token"}[5m]))` | Alert when > 5 % for 10 min. |
|
||||
| `authorize_rate_limit_hits` | `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority", limiter="authority-token"}[5m]))` | Detect rate limiting saturations (requires OTEL ASP.NET rate limiter exporter). |
|
||||
| `lockout_events` | `sum by (plugin) (rate(log_messages_total{app="stellaops-authority", level="Warning", message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[5m]))` | Derived from Loki/Promtail log counter. |
|
||||
| `bypass_usage_total` | `sum(rate(log_messages_total{app="stellaops-authority", level="Information", message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m]))` | Track trusted bypass invocations. |
|
||||
|
||||
> **Exporter note:** Enable `aspnetcore` meters (`dotnet-counters` name `Microsoft.AspNetCore.Hosting`), or configure the OpenTelemetry Collector `metrics` pipeline with `metric_statements` to remap histogram counts into the shown series.
|
||||
|
||||
## Alert Rules
|
||||
1. **Token Failure Surge**
|
||||
- _Expression_: `token_failure_ratio > 0.05`
|
||||
- _For_: `10m`
|
||||
- _Labels_: `severity="critical"`
|
||||
- _Annotations_: Include `topk(5, sum by (authority_identity_provider) (increase(authority_token_rejections_total[10m])))` as diagnostic hint (requires span → metric transformation).
|
||||
2. **Lockout Spike**
|
||||
- _Expression_: `sum(rate(log_messages_total{message_template="Plugin {PluginName} denied access for {Username} due to lockout (retry after {RetryAfter})."}[15m])) > 10`
|
||||
- _For_: `15m`
|
||||
- Investigate credential stuffing; consider temporarily tightening `RateLimiting.Token`.
|
||||
3. **Bypass Threshold**
|
||||
- _Expression_: `sum(rate(log_messages_total{message_template="Granting StellaOps bypass for remote {RemoteIp}; required scopes {RequiredScopes}."}[5m])) > 1`
|
||||
- _For_: `5m`
|
||||
- Alert severity `warning` — verify the calling host list.
|
||||
4. **Rate Limiter Saturation**
|
||||
- _Expression_: `sum(rate(aspnetcore_rate_limiting_rejections_total{service_name="stellaops-authority"}[5m])) > 0`
|
||||
- Escalate if sustained for 5 min; confirm trusted clients aren’t misconfigured.
|
||||
|
||||
## Grafana Dashboard
|
||||
- Import `docs/modules/authority/operations/grafana-dashboard.json` to provision baseline panels:
|
||||
- **Token Success vs Failure** – stacked rate visualization split by grant type.
|
||||
- **Rate Limiter Hits** – bar chart showing `authority-token` and `authority-authorize`.
|
||||
- **Bypass & Lockout Events** – dual-stat panel using Loki-derived counters.
|
||||
- **Trace Explorer Link** – panel links to `StellaOps.Authority` span search pre-filtered by `authority.grant_type`.
|
||||
|
||||
## Collector Configuration Snippets
|
||||
```yaml
|
||||
receivers:
|
||||
otlp:
|
||||
protocols:
|
||||
http:
|
||||
exporters:
|
||||
prometheus:
|
||||
endpoint: "0.0.0.0:9464"
|
||||
processors:
|
||||
batch:
|
||||
attributes/token_grant:
|
||||
actions:
|
||||
- key: grant_type
|
||||
action: upsert
|
||||
from_attribute: authority.grant_type
|
||||
service:
|
||||
pipelines:
|
||||
metrics:
|
||||
receivers: [otlp]
|
||||
processors: [attributes/token_grant, batch]
|
||||
exporters: [prometheus]
|
||||
logs:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [loki]
|
||||
```
|
||||
|
||||
## Operational Checklist
|
||||
- [ ] Confirm `STELLAOPS_AUTHORITY__OBSERVABILITY__EXPORTERS` enables OTLP in production builds.
|
||||
- [ ] Ensure Promtail captures container stdout with Serilog structured formatting.
|
||||
- [ ] Periodically validate alert noise by running load tests that trigger the rate limiter.
|
||||
- [ ] Include dashboard JSON in Offline Kit for air-gapped clusters; update version header when metrics change.
|
||||
22
docs/modules/ci/AGENTS.md
Normal file
22
docs/modules/ci/AGENTS.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# CI Recipes agent guide
|
||||
|
||||
## Mission
|
||||
CI module collects reproducible pipeline recipes for builds, tests, and release promotion across supported platforms.
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
|
||||
## How to get started
|
||||
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
|
||||
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
|
||||
3. Read the architecture and README for domain context before editing code or docs.
|
||||
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
|
||||
|
||||
## Guardrails
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
|
||||
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
|
||||
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
|
||||
- Update runbooks/observability assets when operational characteristics change.
|
||||
29
docs/modules/ci/README.md
Normal file
29
docs/modules/ci/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# StellaOps CI Recipes
|
||||
|
||||
CI module collects reproducible pipeline recipes for builds, tests, and release promotion across supported platforms.
|
||||
|
||||
## Responsibilities
|
||||
- Provide ready-to-use pipeline snippets for ingestion, scanning, policy evaluation, and exports.
|
||||
- Document required secrets/scopes and deterministic build knobs.
|
||||
- Highlight offline-compatible workflows and cache strategies.
|
||||
|
||||
## Key components
|
||||
- Recipe catalogue in ./recipes.md.
|
||||
|
||||
## Integrations & dependencies
|
||||
- DevOps release workflows.
|
||||
- Module-specific test suites referenced in recipes.
|
||||
|
||||
## Operational notes
|
||||
- Encourage reuse through templated YAML/JSON fragments.
|
||||
|
||||
## Related resources
|
||||
- ./recipes.md
|
||||
|
||||
## Backlog references
|
||||
- CI recipes refresh tracked in ../../TASKS.md under DOCS-CI stories.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 1 – AOC enforcement:** bake ingestion/verifier guardrails into CI recipes.
|
||||
- **Epic 10 – Export Center:** provide pipeline snippets for export packaging, signing, and Offline Kit publication.
|
||||
- **Epic 11 – Notifications Studio:** offer CI hooks for notification previews/tests where relevant.
|
||||
9
docs/modules/ci/TASKS.md
Normal file
9
docs/modules/ci/TASKS.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Task board — CI Recipes
|
||||
|
||||
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| CI RECIPES-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
|
||||
| CI RECIPES-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
|
||||
| CI RECIPES-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
|
||||
7
docs/modules/ci/architecture.md
Normal file
7
docs/modules/ci/architecture.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# CI Recipes architecture
|
||||
|
||||
> Reference the AOC guardrails, export workflows, and notification patterns documented in the Authority, Export Center, and Notify module guides when designing CI templates.
|
||||
|
||||
This placeholder summarises the planned architecture for CI Recipes. Consolidate design details from implementation plans and upcoming epics before coding.
|
||||
|
||||
Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised.
|
||||
21
docs/modules/ci/implementation_plan.md
Normal file
21
docs/modules/ci/implementation_plan.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Implementation plan — CI Recipes
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** ensure pipelines enforce schemas, provenance, and verifier jobs.
|
||||
- **Epic 10 – Export Center:** add export/signing/Offline Kit automation templates.
|
||||
- **Epic 11 – Notifications Studio:** document CI hooks for notification previews/tests.
|
||||
- Track DOCS-CI stories in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -1,248 +1,248 @@
|
||||
# Stella Ops CI Recipes — (2025‑08‑04)
|
||||
|
||||
## 0 · Key variables (export these once)
|
||||
|
||||
| Variable | Meaning | Typical value |
|
||||
| ------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
|
||||
| `STELLA_URL` | Host that: ① stores the **CLI** & **SBOM‑builder** images under `/registry` **and** ② receives API calls at `https://$STELLA_URL` | `stella-ops.ci.acme.example` |
|
||||
| `DOCKER_HOST` | How containers reach your Docker daemon (because we no longer mount `/var/run/docker.sock`) | `tcp://docker:2375` |
|
||||
| `WORKSPACE` | Directory where the pipeline stores artefacts (SBOM file) | `$(pwd)` |
|
||||
| `IMAGE` | The image you are building & scanning | `acme/backend:sha-${COMMIT_SHA}` |
|
||||
# Stella Ops CI Recipes — (2025‑08‑04)
|
||||
|
||||
## 0 · Key variables (export these once)
|
||||
|
||||
| Variable | Meaning | Typical value |
|
||||
| ------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
|
||||
| `STELLA_URL` | Host that: ① stores the **CLI** & **SBOM‑builder** images under `/registry` **and** ② receives API calls at `https://$STELLA_URL` | `stella-ops.ci.acme.example` |
|
||||
| `DOCKER_HOST` | How containers reach your Docker daemon (because we no longer mount `/var/run/docker.sock`) | `tcp://docker:2375` |
|
||||
| `WORKSPACE` | Directory where the pipeline stores artefacts (SBOM file) | `$(pwd)` |
|
||||
| `IMAGE` | The image you are building & scanning | `acme/backend:sha-${COMMIT_SHA}` |
|
||||
| `SBOM_FILE` | Immutable SBOM name – `<image-ref>‑YYYYMMDDThhmmssZ.sbom.json` | `acme_backend_sha‑abc123‑20250804T153050Z.sbom.json` |
|
||||
|
||||
> **Authority graph scopes note (2025-10-27):** CI stages that spin up the Authority compose profile now rely on the checked-in `etc/authority.yaml`. Before running integration smoke jobs, inject real secrets for every `etc/secrets/*.secret` file (Cartographer, Graph API, Policy Engine, Concelier, Excititor). The repository defaults contain `*-change-me` placeholders and Authority will reject tokens if those secrets are not overridden. Reissue CI tokens that previously used `policy:write`/`policy:submit`/`policy:edit` scopes—new bundles must request `policy:read`, `policy:author`, `policy:review`, `policy:simulate`, and (`policy:approve`/`policy:operate`/`policy:activate` when pipelines promote policies).
|
||||
|
||||
```bash
|
||||
export STELLA_URL="stella-ops.ci.acme.example"
|
||||
export DOCKER_HOST="tcp://docker:2375" # Jenkins/Circle often expose it like this
|
||||
export WORKSPACE="$(pwd)"
|
||||
export IMAGE="acme/backend:sha-${COMMIT_SHA}"
|
||||
export SBOM_FILE="$(echo "${IMAGE}" | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 1 · SBOM creation strategies
|
||||
|
||||
### Option A – **Buildx attested SBOM** (preferred if you can use BuildKit)
|
||||
|
||||
You pass **two build args** so the Dockerfile can run the builder and copy the result out of the build context.
|
||||
|
||||
```bash
|
||||
docker buildx build \
|
||||
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
--provenance=true --sbom=true \
|
||||
--build-arg SBOM_FILE="$SBOM_FILE" \
|
||||
-t "$IMAGE" .
|
||||
```
|
||||
|
||||
**If you **cannot** use Buildx, use Option B below.** The older “run a builder stage inside the Dockerfile” pattern is unreliable for producing an SBOM of the final image.
|
||||
|
||||
```Dockerfile
|
||||
|
||||
ARG STELLA_SBOM_BUILDER
|
||||
ARG SBOM_FILE
|
||||
|
||||
FROM $STELLA_SBOM_BUILDER as sbom
|
||||
ARG IMAGE
|
||||
ARG SBOM_FILE
|
||||
RUN $STELLA_SBOM_BUILDER build --image $IMAGE --output /out/$SBOM_FILE
|
||||
|
||||
# ---- actual build stages … ----
|
||||
FROM alpine:3.20
|
||||
COPY --from=sbom /out/$SBOM_FILE / # (optional) keep or discard
|
||||
|
||||
# (rest of your Dockerfile)
|
||||
```
|
||||
|
||||
### Option B – **External builder step** (works everywhere; recommended baseline if Buildx isn’t available)
|
||||
|
||||
*(keep this block if your pipeline already has an image‑build step that you can’t modify)*
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-e DOCKER_HOST="$DOCKER_HOST" \ # let builder reach the daemon remotely
|
||||
-v "$WORKSPACE:/workspace" \ # place SBOM beside the source code
|
||||
"$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2 · Scan the image & upload results
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-e DOCKER_HOST="$DOCKER_HOST" \ # remote‑daemon pointer
|
||||
-v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \ # mount SBOM under same name at container root
|
||||
-e STELLA_OPS_URL="https://${STELLA_URL}" \ # where the CLI posts findings
|
||||
"$STELLA_URL/registry/stella-cli:latest" \
|
||||
scan --sbom "/${SBOM_FILE}" "$IMAGE"
|
||||
```
|
||||
|
||||
The CLI returns **exit 0** if policies pass, **>0** if blocked — perfect for failing the job.
|
||||
|
||||
---
|
||||
|
||||
## 3 · CI templates
|
||||
|
||||
Below are minimal, cut‑and‑paste snippets.
|
||||
**Feel free to delete Option B** if you adopt Option A.
|
||||
|
||||
### 3.1 Jenkins (Declarative Pipeline)
|
||||
|
||||
```groovy
|
||||
pipeline {
|
||||
agent { docker { image 'docker:25' args '--privileged' } } // gives us /usr/bin/docker
|
||||
environment {
|
||||
STELLA_URL = 'stella-ops.ci.acme.example'
|
||||
DOCKER_HOST = 'tcp://docker:2375'
|
||||
IMAGE = "acme/backend:${env.BUILD_NUMBER}"
|
||||
SBOM_FILE = "acme_backend_${env.BUILD_NUMBER}-${new Date().format('yyyyMMdd\'T\'HHmmss\'Z\'', TimeZone.getTimeZone('UTC'))}.sbom.json"
|
||||
}
|
||||
stages {
|
||||
stage('Build image + SBOM (Option A)') {
|
||||
steps {
|
||||
sh '''
|
||||
docker build \
|
||||
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
--build-arg SBOM_FILE="$SBOM_FILE" \
|
||||
-t "$IMAGE" .
|
||||
'''
|
||||
}
|
||||
}
|
||||
/* ---------- Option B fallback (when you must keep the existing build step as‑is) ----------
|
||||
stage('SBOM builder (Option B)') {
|
||||
steps {
|
||||
sh '''
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$WORKSPACE:/workspace" \
|
||||
"$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
|
||||
'''
|
||||
}
|
||||
}
|
||||
------------------------------------------------------------------------------------------ */
|
||||
stage('Scan & upload') {
|
||||
steps {
|
||||
sh '''
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \
|
||||
-e STELLA_OPS_URL="https://$STELLA_URL" \
|
||||
"$STELLA_URL/registry/stella-cli:latest" \
|
||||
scan --sbom "/${SBOM_FILE}" "$IMAGE"
|
||||
'''
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.2 CircleCI `.circleci/config.yml`
|
||||
|
||||
```yaml
|
||||
version: 2.1
|
||||
jobs:
|
||||
stella_scan:
|
||||
docker:
|
||||
- image: cimg/base:stable # baremetal image with Docker CLI
|
||||
environment:
|
||||
STELLA_URL: stella-ops.ci.acme.example
|
||||
DOCKER_HOST: tcp://docker:2375 # Circle’s “remote Docker” socket
|
||||
steps:
|
||||
- checkout
|
||||
|
||||
- run:
|
||||
name: Compute vars
|
||||
command: |
|
||||
echo 'export IMAGE="acme/backend:${CIRCLE_SHA1}"' >> $BASH_ENV
|
||||
echo 'export SBOM_FILE="$(echo acme/backend:${CIRCLE_SHA1} | tr "/:+" "__")-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"' >> $BASH_ENV
|
||||
- run:
|
||||
name: Build image + SBOM (Option A)
|
||||
command: |
|
||||
docker build \
|
||||
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
--build-arg SBOM_FILE="$SBOM_FILE" \
|
||||
-t "$IMAGE" .
|
||||
# --- Option B fallback (when you must keep the existing build step as‑is) ---
|
||||
#- run:
|
||||
# name: SBOM builder (Option B)
|
||||
# command: |
|
||||
# docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
# -v "$PWD:/workspace" \
|
||||
# "$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
# build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
|
||||
- run:
|
||||
name: Scan
|
||||
command: |
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$PWD/${SBOM_FILE}:/${SBOM_FILE}:ro" \
|
||||
-e STELLA_OPS_URL="https://$STELLA_URL" \
|
||||
"$STELLA_URL/registry/stella-cli:latest" \
|
||||
scan --sbom "/${SBOM_FILE}" "$IMAGE"
|
||||
workflows:
|
||||
stella:
|
||||
jobs: [stella_scan]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.3 Gitea Actions `.gitea/workflows/stella.yml`
|
||||
|
||||
*(Gitea 1.22+ ships native Actions compatible with GitHub syntax)*
|
||||
|
||||
```yaml
|
||||
name: Stella Scan
|
||||
on: [push]
|
||||
|
||||
jobs:
|
||||
stella:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
STELLA_URL: ${{ secrets.STELLA_URL }}
|
||||
DOCKER_HOST: tcp://docker:2375 # provided by the docker:dind service
|
||||
services:
|
||||
docker:
|
||||
image: docker:dind
|
||||
options: >-
|
||||
--privileged
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Compute vars
|
||||
id: vars
|
||||
run: |
|
||||
echo "IMAGE=ghcr.io/${{ gitea.repository }}:${{ gitea.sha }}" >> $GITEA_OUTPUT
|
||||
echo "SBOM_FILE=$(echo ghcr.io/${{ gitea.repository }}:${{ gitea.sha }} | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json" >> $GITEA_OUTPUT
|
||||
|
||||
- name: Build image + SBOM (Option A)
|
||||
run: |
|
||||
docker build \
|
||||
--build-arg STELLA_SBOM_BUILDER="${STELLA_URL}/registry/stella-sbom-builder:latest" \
|
||||
--build-arg SBOM_FILE="${{ steps.vars.outputs.SBOM_FILE }}" \
|
||||
-t "${{ steps.vars.outputs.IMAGE }}" .
|
||||
|
||||
# --- Option B fallback (when you must keep the existing build step as‑is) ---
|
||||
#- name: SBOM builder (Option B)
|
||||
# run: |
|
||||
# docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
# -v "$(pwd):/workspace" \
|
||||
# "${STELLA_URL}/registry/stella-sbom-builder:latest" \
|
||||
# build --image "${{ steps.vars.outputs.IMAGE }}" --output "/workspace/${{ steps.vars.outputs.SBOM_FILE }}"
|
||||
|
||||
- name: Scan
|
||||
run: |
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$(pwd)/${{ steps.vars.outputs.SBOM_FILE }}:/${{ steps.vars.outputs.SBOM_FILE }}:ro" \
|
||||
-e STELLA_OPS_URL="https://${STELLA_URL}" \
|
||||
"${STELLA_URL}/registry/stella-cli:latest" \
|
||||
scan --sbom "/${{ steps.vars.outputs.SBOM_FILE }}" "${{ steps.vars.outputs.IMAGE }}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
||||
```bash
|
||||
export STELLA_URL="stella-ops.ci.acme.example"
|
||||
export DOCKER_HOST="tcp://docker:2375" # Jenkins/Circle often expose it like this
|
||||
export WORKSPACE="$(pwd)"
|
||||
export IMAGE="acme/backend:sha-${COMMIT_SHA}"
|
||||
export SBOM_FILE="$(echo "${IMAGE}" | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 1 · SBOM creation strategies
|
||||
|
||||
### Option A – **Buildx attested SBOM** (preferred if you can use BuildKit)
|
||||
|
||||
You pass **two build args** so the Dockerfile can run the builder and copy the result out of the build context.
|
||||
|
||||
```bash
|
||||
docker buildx build \
|
||||
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
--provenance=true --sbom=true \
|
||||
--build-arg SBOM_FILE="$SBOM_FILE" \
|
||||
-t "$IMAGE" .
|
||||
```
|
||||
|
||||
**If you **cannot** use Buildx, use Option B below.** The older “run a builder stage inside the Dockerfile” pattern is unreliable for producing an SBOM of the final image.
|
||||
|
||||
```Dockerfile
|
||||
|
||||
ARG STELLA_SBOM_BUILDER
|
||||
ARG SBOM_FILE
|
||||
|
||||
FROM $STELLA_SBOM_BUILDER as sbom
|
||||
ARG IMAGE
|
||||
ARG SBOM_FILE
|
||||
RUN $STELLA_SBOM_BUILDER build --image $IMAGE --output /out/$SBOM_FILE
|
||||
|
||||
# ---- actual build stages … ----
|
||||
FROM alpine:3.20
|
||||
COPY --from=sbom /out/$SBOM_FILE / # (optional) keep or discard
|
||||
|
||||
# (rest of your Dockerfile)
|
||||
```
|
||||
|
||||
### Option B – **External builder step** (works everywhere; recommended baseline if Buildx isn’t available)
|
||||
|
||||
*(keep this block if your pipeline already has an image‑build step that you can’t modify)*
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-e DOCKER_HOST="$DOCKER_HOST" \ # let builder reach the daemon remotely
|
||||
-v "$WORKSPACE:/workspace" \ # place SBOM beside the source code
|
||||
"$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2 · Scan the image & upload results
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-e DOCKER_HOST="$DOCKER_HOST" \ # remote‑daemon pointer
|
||||
-v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \ # mount SBOM under same name at container root
|
||||
-e STELLA_OPS_URL="https://${STELLA_URL}" \ # where the CLI posts findings
|
||||
"$STELLA_URL/registry/stella-cli:latest" \
|
||||
scan --sbom "/${SBOM_FILE}" "$IMAGE"
|
||||
```
|
||||
|
||||
The CLI returns **exit 0** if policies pass, **>0** if blocked — perfect for failing the job.
|
||||
|
||||
---
|
||||
|
||||
## 3 · CI templates
|
||||
|
||||
Below are minimal, cut‑and‑paste snippets.
|
||||
**Feel free to delete Option B** if you adopt Option A.
|
||||
|
||||
### 3.1 Jenkins (Declarative Pipeline)
|
||||
|
||||
```groovy
|
||||
pipeline {
|
||||
agent { docker { image 'docker:25' args '--privileged' } } // gives us /usr/bin/docker
|
||||
environment {
|
||||
STELLA_URL = 'stella-ops.ci.acme.example'
|
||||
DOCKER_HOST = 'tcp://docker:2375'
|
||||
IMAGE = "acme/backend:${env.BUILD_NUMBER}"
|
||||
SBOM_FILE = "acme_backend_${env.BUILD_NUMBER}-${new Date().format('yyyyMMdd\'T\'HHmmss\'Z\'', TimeZone.getTimeZone('UTC'))}.sbom.json"
|
||||
}
|
||||
stages {
|
||||
stage('Build image + SBOM (Option A)') {
|
||||
steps {
|
||||
sh '''
|
||||
docker build \
|
||||
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
--build-arg SBOM_FILE="$SBOM_FILE" \
|
||||
-t "$IMAGE" .
|
||||
'''
|
||||
}
|
||||
}
|
||||
/* ---------- Option B fallback (when you must keep the existing build step as‑is) ----------
|
||||
stage('SBOM builder (Option B)') {
|
||||
steps {
|
||||
sh '''
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$WORKSPACE:/workspace" \
|
||||
"$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
|
||||
'''
|
||||
}
|
||||
}
|
||||
------------------------------------------------------------------------------------------ */
|
||||
stage('Scan & upload') {
|
||||
steps {
|
||||
sh '''
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$WORKSPACE/${SBOM_FILE}:/${SBOM_FILE}:ro" \
|
||||
-e STELLA_OPS_URL="https://$STELLA_URL" \
|
||||
"$STELLA_URL/registry/stella-cli:latest" \
|
||||
scan --sbom "/${SBOM_FILE}" "$IMAGE"
|
||||
'''
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.2 CircleCI `.circleci/config.yml`
|
||||
|
||||
```yaml
|
||||
version: 2.1
|
||||
jobs:
|
||||
stella_scan:
|
||||
docker:
|
||||
- image: cimg/base:stable # baremetal image with Docker CLI
|
||||
environment:
|
||||
STELLA_URL: stella-ops.ci.acme.example
|
||||
DOCKER_HOST: tcp://docker:2375 # Circle’s “remote Docker” socket
|
||||
steps:
|
||||
- checkout
|
||||
|
||||
- run:
|
||||
name: Compute vars
|
||||
command: |
|
||||
echo 'export IMAGE="acme/backend:${CIRCLE_SHA1}"' >> $BASH_ENV
|
||||
echo 'export SBOM_FILE="$(echo acme/backend:${CIRCLE_SHA1} | tr "/:+" "__")-$(date -u +%Y%m%dT%H%M%SZ).sbom.json"' >> $BASH_ENV
|
||||
- run:
|
||||
name: Build image + SBOM (Option A)
|
||||
command: |
|
||||
docker build \
|
||||
--build-arg STELLA_SBOM_BUILDER="$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
--build-arg SBOM_FILE="$SBOM_FILE" \
|
||||
-t "$IMAGE" .
|
||||
# --- Option B fallback (when you must keep the existing build step as‑is) ---
|
||||
#- run:
|
||||
# name: SBOM builder (Option B)
|
||||
# command: |
|
||||
# docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
# -v "$PWD:/workspace" \
|
||||
# "$STELLA_URL/registry/stella-sbom-builder:latest" \
|
||||
# build --image "$IMAGE" --output "/workspace/${SBOM_FILE}"
|
||||
- run:
|
||||
name: Scan
|
||||
command: |
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$PWD/${SBOM_FILE}:/${SBOM_FILE}:ro" \
|
||||
-e STELLA_OPS_URL="https://$STELLA_URL" \
|
||||
"$STELLA_URL/registry/stella-cli:latest" \
|
||||
scan --sbom "/${SBOM_FILE}" "$IMAGE"
|
||||
workflows:
|
||||
stella:
|
||||
jobs: [stella_scan]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.3 Gitea Actions `.gitea/workflows/stella.yml`
|
||||
|
||||
*(Gitea 1.22+ ships native Actions compatible with GitHub syntax)*
|
||||
|
||||
```yaml
|
||||
name: Stella Scan
|
||||
on: [push]
|
||||
|
||||
jobs:
|
||||
stella:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
STELLA_URL: ${{ secrets.STELLA_URL }}
|
||||
DOCKER_HOST: tcp://docker:2375 # provided by the docker:dind service
|
||||
services:
|
||||
docker:
|
||||
image: docker:dind
|
||||
options: >-
|
||||
--privileged
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Compute vars
|
||||
id: vars
|
||||
run: |
|
||||
echo "IMAGE=ghcr.io/${{ gitea.repository }}:${{ gitea.sha }}" >> $GITEA_OUTPUT
|
||||
echo "SBOM_FILE=$(echo ghcr.io/${{ gitea.repository }}:${{ gitea.sha }} | tr '/:+' '__')-$(date -u +%Y%m%dT%H%M%SZ).sbom.json" >> $GITEA_OUTPUT
|
||||
|
||||
- name: Build image + SBOM (Option A)
|
||||
run: |
|
||||
docker build \
|
||||
--build-arg STELLA_SBOM_BUILDER="${STELLA_URL}/registry/stella-sbom-builder:latest" \
|
||||
--build-arg SBOM_FILE="${{ steps.vars.outputs.SBOM_FILE }}" \
|
||||
-t "${{ steps.vars.outputs.IMAGE }}" .
|
||||
|
||||
# --- Option B fallback (when you must keep the existing build step as‑is) ---
|
||||
#- name: SBOM builder (Option B)
|
||||
# run: |
|
||||
# docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
# -v "$(pwd):/workspace" \
|
||||
# "${STELLA_URL}/registry/stella-sbom-builder:latest" \
|
||||
# build --image "${{ steps.vars.outputs.IMAGE }}" --output "/workspace/${{ steps.vars.outputs.SBOM_FILE }}"
|
||||
|
||||
- name: Scan
|
||||
run: |
|
||||
docker run --rm -e DOCKER_HOST="$DOCKER_HOST" \
|
||||
-v "$(pwd)/${{ steps.vars.outputs.SBOM_FILE }}:/${{ steps.vars.outputs.SBOM_FILE }}:ro" \
|
||||
-e STELLA_OPS_URL="https://${STELLA_URL}" \
|
||||
"${STELLA_URL}/registry/stella-cli:latest" \
|
||||
scan --sbom "/${{ steps.vars.outputs.SBOM_FILE }}" "${{ steps.vars.outputs.IMAGE }}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4 · Docs CI (Gitea Actions & Offline Mirror)
|
||||
|
||||
StellaOps ships a dedicated Docs workflow at `.gitea/workflows/docs.yml`. When mirroring the pipeline offline or running it locally, install the same toolchain so markdown linting, schema validation, and HTML preview stay deterministic.
|
||||
@@ -305,7 +305,7 @@ Policy Engine v2 pipelines now fail fast if policy documents are malformed. Afte
|
||||
|
||||
```bash
|
||||
dotnet run \
|
||||
--project tools/PolicyDslValidator/PolicyDslValidator.csproj \
|
||||
--project src/Tools/PolicyDslValidator/PolicyDslValidator.csproj \
|
||||
-- \
|
||||
--strict docs/examples/policies/*.yaml
|
||||
```
|
||||
@@ -322,7 +322,7 @@ Catch unexpected policy regressions by exercising a small set of golden SBOM fin
|
||||
|
||||
```bash
|
||||
dotnet run \
|
||||
--project tools/PolicySimulationSmoke/PolicySimulationSmoke.csproj \
|
||||
--project src/Tools/PolicySimulationSmoke/PolicySimulationSmoke.csproj \
|
||||
-- \
|
||||
--scenario-root samples/policy/simulations \
|
||||
--output artifacts/policy-simulations
|
||||
@@ -336,17 +336,17 @@ dotnet run \
|
||||
---
|
||||
|
||||
## 5 · Troubleshooting cheat‑sheet
|
||||
|
||||
| Symptom | Root cause | First things to try |
|
||||
| ------------------------------------- | --------------------------- | --------------------------------------------------------------- |
|
||||
| `no such host $STELLA_URL` | DNS typo or VPN outage | `ping $STELLA_URL` from runner |
|
||||
| `connection refused` when CLI uploads | Port 443 blocked | open firewall / check ingress |
|
||||
| `failed to stat /<sbom>.json` | SBOM wasn’t produced | Did Option A actually run builder? If not, enable Option B |
|
||||
| `registry unauthorized` | Runner lacks registry creds | `docker login $STELLA_URL/registry` (store creds in CI secrets) |
|
||||
| Non‑zero scan exit | Blocking vuln/licence | Open project in Ops UI → triage or waive |
|
||||
|
||||
---
|
||||
|
||||
|
||||
| Symptom | Root cause | First things to try |
|
||||
| ------------------------------------- | --------------------------- | --------------------------------------------------------------- |
|
||||
| `no such host $STELLA_URL` | DNS typo or VPN outage | `ping $STELLA_URL` from runner |
|
||||
| `connection refused` when CLI uploads | Port 443 blocked | open firewall / check ingress |
|
||||
| `failed to stat /<sbom>.json` | SBOM wasn’t produced | Did Option A actually run builder? If not, enable Option B |
|
||||
| `registry unauthorized` | Runner lacks registry creds | `docker login $STELLA_URL/registry` (store creds in CI secrets) |
|
||||
| Non‑zero scan exit | Blocking vuln/licence | Open project in Ops UI → triage or waive |
|
||||
|
||||
---
|
||||
|
||||
### Change log
|
||||
|
||||
* **2025‑10‑18** – Documented Docs CI toolchain (Ajv validation, static preview) and offline checklist.
|
||||
22
docs/modules/cli/AGENTS.md
Normal file
22
docs/modules/cli/AGENTS.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# CLI agent guide
|
||||
|
||||
## Mission
|
||||
The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting.
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
|
||||
## How to get started
|
||||
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
|
||||
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
|
||||
3. Read the architecture and README for domain context before editing code or docs.
|
||||
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
|
||||
|
||||
## Guardrails
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
|
||||
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
|
||||
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
|
||||
- Update runbooks/observability assets when operational characteristics change.
|
||||
40
docs/modules/cli/README.md
Normal file
40
docs/modules/cli/README.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# StellaOps CLI
|
||||
|
||||
The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting.
|
||||
|
||||
## Responsibilities
|
||||
- Deliver deterministic verbs for scan, diff, export, policy, and observability operations.
|
||||
- Handle interactive and non-interactive authentication via Authority (device code, client credentials).
|
||||
- Support offline kit workflows including bundle verification and seed installation.
|
||||
- Expose JSON outputs suitable for CI parity and golden tests.
|
||||
|
||||
## Key components
|
||||
- `StellaOps.Cli` native AOT host.
|
||||
- Shared helpers in `StellaOps.Cli.Core`.
|
||||
- Restart-time plug-ins under `StellaOps.Cli.Plugins.*`.
|
||||
|
||||
## Integrations & dependencies
|
||||
- Authority for token exchange.
|
||||
- Backend APIs (Scanner, Policy, Export Center, Notify).
|
||||
- Offline kit bundles and local keychain/DPoP storage.
|
||||
|
||||
## Operational notes
|
||||
- Deterministic output fixtures under `src/Cli/StellaOps.Cli.Tests`.
|
||||
- Versioned command docs in `docs/modules/cli/guides`.
|
||||
- Plugin catalogue in `plugins/cli/**` (restart-only).
|
||||
|
||||
## Related resources
|
||||
- ./guides/20_REFERENCE.md
|
||||
- ./guides/cli-reference.md
|
||||
- ./guides/policy.md
|
||||
|
||||
## Backlog references
|
||||
- DOCS-CLI-OBS-52-001 / DOCS-CLI-FORENSICS-53-001 in ../../TASKS.md.
|
||||
- CLI-CORE-41-001 epic in `src/Cli/StellaOps.Cli/TASKS.md`.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy authoring, simulation, and explain verbs.
|
||||
- **Epic 4 – Policy Studio:** integrate registry/promotion workflows, approvals, and lint tooling.
|
||||
- **Epic 6 – Vulnerability Explorer:** surface triage and ledger operations.
|
||||
- **Epic 10 – Export Center:** orchestrate export requests, verification, and Offline Kit automation.
|
||||
- **Epic 11 – Notifications Studio:** manage notification authoring/previews from the command line.
|
||||
9
docs/modules/cli/TASKS.md
Normal file
9
docs/modules/cli/TASKS.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Task board — CLI
|
||||
|
||||
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| CLI-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
|
||||
| CLI-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
|
||||
| CLI-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
|
||||
@@ -1,4 +1,6 @@
|
||||
# component_architecture_cli.md — **Stella Ops CLI** (2025Q4)
|
||||
# component_architecture_cli.md — **Stella Ops CLI** (2025Q4)
|
||||
|
||||
> Consolidates requirements captured in the Policy Engine, Policy Studio, Vulnerability Explorer, Export Center, and Notifications implementation plans and module guides.
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops CLI**: command surface, process model, auth (Authority/DPoP), integration with Scanner/Excititor/Concelier/Signer/Attestor, Buildx plug‑in management, offline kit behavior, packaging, observability, security posture, and CI ergonomics.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# CLI Reference (`stella --help`)
|
||||
|
||||
> **Auto‑generated file — do not edit manually.**
|
||||
> On every tagged release the CI pipeline runs
|
||||
> `stella --help --markdown > docs/cli/20_REFERENCE.md`
|
||||
> ensuring this document always matches the shipped binary.
|
||||
|
||||
*(The reference will appear after the first public α release.)*
|
||||
# CLI Reference (`stella --help`)
|
||||
|
||||
> **Auto‑generated file — do not edit manually.**
|
||||
> On every tagged release the CI pipeline runs
|
||||
> `stella --help --markdown > docs/modules/cli/guides/20_REFERENCE.md`
|
||||
> ensuring this document always matches the shipped binary.
|
||||
|
||||
*(The reference will appear after the first public α release.)*
|
||||
@@ -1,316 +1,316 @@
|
||||
# CLI AOC Commands Reference
|
||||
|
||||
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
|
||||
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19.
|
||||
|
||||
Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../ingestion/aggregation-only-contract.md) and the [architecture overview](../architecture/overview.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Prerequisites
|
||||
|
||||
- CLI version: `stella` ≥ 0.19.0 (AOC feature gate enabled).
|
||||
- Required scopes (DPoP-bound):
|
||||
- `advisory:read` for Concelier sources.
|
||||
- `vex:read` for Excititor sources (optional but required for VEX checks).
|
||||
- `aoc:verify` to invoke guard verification endpoints.
|
||||
- `tenant:select` if your deployment uses tenant switching.
|
||||
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see § 4).
|
||||
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
|
||||
|
||||
---
|
||||
|
||||
## 2 · `stella sources ingest --dry-run`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source <source-key> \
|
||||
--input <path-or-uri> \
|
||||
[--tenant <tenant-id>] \
|
||||
[--format json|table] \
|
||||
[--no-color] \
|
||||
[--output <file>]
|
||||
```
|
||||
|
||||
### 2.2 Description
|
||||
|
||||
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
|
||||
|
||||
### 2.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
|
||||
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
|
||||
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
|
||||
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
|
||||
| `--no-color` | Disables ANSI colour output for CI logs. |
|
||||
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
|
||||
|
||||
### 2.4 Output schema (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "redhat",
|
||||
"tenant": "default",
|
||||
"guardVersion": "1.0.0",
|
||||
"status": "ok",
|
||||
"document": {
|
||||
"contentHash": "sha256:…",
|
||||
"supersedes": null,
|
||||
"provenance": {
|
||||
"signature": { "format": "pgp", "present": true }
|
||||
}
|
||||
},
|
||||
"violations": []
|
||||
}
|
||||
```
|
||||
|
||||
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
|
||||
|
||||
### 2.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Guard passed; would-write payload is AOC compliant. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field (`severity`, `cvss`, etc.) detected. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt (multiple upstream sources fused). |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation (duplicate without supersedes). |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance fields. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings present (Policy-only data). |
|
||||
| `17` | `ERR_AOC_007` – Unknown top-level fields / schema violation. |
|
||||
| `70` | Transport error (network, auth, malformed input). |
|
||||
|
||||
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
|
||||
|
||||
### 2.6 Examples
|
||||
|
||||
Dry-run a local CSAF file:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source redhat \
|
||||
--input ./fixtures/redhat/RHSA-2025-1234.json
|
||||
```
|
||||
|
||||
Stream from HTTPS and emit JSON for CI:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source osv \
|
||||
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
|
||||
--format json \
|
||||
--output artifacts/osv-dry-run.json
|
||||
|
||||
cat artifacts/osv-dry-run.json | jq '.violations'
|
||||
```
|
||||
|
||||
### 2.7 Offline notes
|
||||
|
||||
When operating in sealed/offline mode:
|
||||
|
||||
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
|
||||
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
|
||||
- The command does not attempt network access when given a file path.
|
||||
- Store reports with `--output` to include in transfer packages for policy review.
|
||||
|
||||
---
|
||||
|
||||
## 3 · `stella aoc verify`
|
||||
|
||||
### 3.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
[--since <iso8601|duration>] \
|
||||
[--limit <count>] \
|
||||
[--sources <list>] \
|
||||
[--codes <ERR_AOC_00x,...>] \
|
||||
[--format table|json] \
|
||||
[--export <file>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--no-color]
|
||||
```
|
||||
|
||||
### 3.2 Description
|
||||
|
||||
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24 hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
|
||||
|
||||
### 3.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--since <value>` | Verification window. Accepts ISO 8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
|
||||
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
|
||||
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
|
||||
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
|
||||
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
|
||||
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
|
||||
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
|
||||
| `--no-color` | Disables ANSI colours. |
|
||||
|
||||
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
|
||||
|
||||
### 3.4 Report structure (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"tenant": "default",
|
||||
"window": {
|
||||
"from": "2025-10-25T12:00:00Z",
|
||||
"to": "2025-10-26T12:00:00Z"
|
||||
},
|
||||
"checked": {
|
||||
"advisories": 482,
|
||||
"vex": 75
|
||||
},
|
||||
"violations": [
|
||||
{
|
||||
"code": "ERR_AOC_001",
|
||||
"count": 2,
|
||||
"examples": [
|
||||
{
|
||||
"source": "redhat",
|
||||
"documentId": "advisory_raw:redhat:RHSA-2025:1",
|
||||
"contentHash": "sha256:…",
|
||||
"path": "/content/raw/cvss"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metrics": {
|
||||
"ingestion_write_total": 557,
|
||||
"aoc_violation_total": 2
|
||||
},
|
||||
"truncated": false
|
||||
}
|
||||
```
|
||||
|
||||
### 3.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Verification succeeded with zero violations. |
|
||||
| `11…17` | Same mapping as § 2.5 when violations are detected. Highest-priority code returned. |
|
||||
| `18` | Verification ran but results truncated (limit reached) – treat as warning; rerun with higher `--limit`. |
|
||||
| `70` | Transport/authentication error. |
|
||||
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
|
||||
|
||||
### 3.6 Examples
|
||||
|
||||
Daily verification across all sources:
|
||||
|
||||
```bash
|
||||
stella aoc verify --since 24h --format table
|
||||
```
|
||||
|
||||
CI pipeline focusing on errant sources and exporting evidence:
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--sources redhat,ubuntu \
|
||||
--codes ERR_AOC_001,ERR_AOC_004 \
|
||||
--format json \
|
||||
--limit 100 \
|
||||
--export artifacts/aoc-verify.json
|
||||
|
||||
jq '.violations[] | {code, count}' artifacts/aoc-verify.json
|
||||
```
|
||||
|
||||
Air-gapped verification using Offline Kit snapshot (example script):
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--since 7d \
|
||||
--format json \
|
||||
--export /mnt/offline/aoc-verify-$(date +%F).json
|
||||
|
||||
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
|
||||
```
|
||||
|
||||
### 3.7 Automation tips
|
||||
|
||||
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥ 11.
|
||||
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
|
||||
- Push JSON exports to observability pipelines for historical tracking of violation counts.
|
||||
|
||||
### 3.8 Offline notes
|
||||
|
||||
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
|
||||
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
|
||||
- Include verification output in compliance packages alongside Offline Kit manifests.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Global exit-code reference
|
||||
|
||||
| Code | Summary |
|
||||
|------|---------|
|
||||
| `0` | Success / no violations. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field present. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt detected. |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation. |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance/signature metadata. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings in ingestion payload. |
|
||||
| `17` | `ERR_AOC_007` – Schema violation / unknown fields. |
|
||||
| `18` | Partial verification (limit reached). |
|
||||
| `70` | Transport or HTTP failure. |
|
||||
| `71` | CLI usage error (invalid arguments, missing tenant). |
|
||||
|
||||
Use these codes in CI to map outcomes to build statuses or alert severities.
|
||||
|
||||
---
|
||||
|
||||
## 4 · `stella vuln observations` (Overlay paging)
|
||||
|
||||
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
|
||||
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
|
||||
|
||||
Additional notes:
|
||||
|
||||
- Table mode prints a hint when `hasMore` is `true`:
|
||||
`[yellow]More observations available. Continue with --cursor <token>[/]`.
|
||||
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
|
||||
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Related references
|
||||
|
||||
- [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md)
|
||||
- [Architecture overview](../architecture/overview.md)
|
||||
- [Console AOC dashboard](../ui/console.md)
|
||||
- [Authority scopes](../ARCHITECTURE_AUTHORITY.md)
|
||||
|
||||
---
|
||||
|
||||
## 6 · Compliance checklist
|
||||
|
||||
- [ ] Usage documented for both table and JSON formats.
|
||||
- [ ] Exit-code mapping matches `ERR_AOC_00x` definitions and automation guidance.
|
||||
- [ ] Offline/air-gap workflow captured for both commands.
|
||||
- [ ] References to AOC architecture and console docs included.
|
||||
- [ ] Examples validated against current CLI syntax (update post-implementation).
|
||||
- [ ] Docs guild screenshot/narrative placeholder logged for release notes (pending CLI team capture).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-29 (Sprint 24).*
|
||||
|
||||
## 13. Authority configuration quick reference
|
||||
|
||||
| Setting | Purpose | How to set |
|
||||
|---------|---------|------------|
|
||||
| `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. |
|
||||
| `StellaOps:Authority:OperatorTicket` | Change/incident ticket reference paired with orchestrator control actions. | CLI flag `--Authority:OperatorTicket=...` or env `STELLAOPS_ORCH_TICKET`. |
|
||||
|
||||
> Tokens requesting `orch:operate` will fail with `invalid_request` unless both values are present. Choose concise strings (≤256 chars for reason, ≤128 chars for ticket) and avoid sensitive data.
|
||||
|
||||
# CLI AOC Commands Reference
|
||||
|
||||
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
|
||||
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19.
|
||||
|
||||
Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../ingestion/aggregation-only-contract.md) and the [architecture overview](../../platform/architecture-overview.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Prerequisites
|
||||
|
||||
- CLI version: `stella` ≥ 0.19.0 (AOC feature gate enabled).
|
||||
- Required scopes (DPoP-bound):
|
||||
- `advisory:read` for Concelier sources.
|
||||
- `vex:read` for Excititor sources (optional but required for VEX checks).
|
||||
- `aoc:verify` to invoke guard verification endpoints.
|
||||
- `tenant:select` if your deployment uses tenant switching.
|
||||
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see § 4).
|
||||
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
|
||||
|
||||
---
|
||||
|
||||
## 2 · `stella sources ingest --dry-run`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source <source-key> \
|
||||
--input <path-or-uri> \
|
||||
[--tenant <tenant-id>] \
|
||||
[--format json|table] \
|
||||
[--no-color] \
|
||||
[--output <file>]
|
||||
```
|
||||
|
||||
### 2.2 Description
|
||||
|
||||
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
|
||||
|
||||
### 2.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
|
||||
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
|
||||
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
|
||||
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
|
||||
| `--no-color` | Disables ANSI colour output for CI logs. |
|
||||
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
|
||||
|
||||
### 2.4 Output schema (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "redhat",
|
||||
"tenant": "default",
|
||||
"guardVersion": "1.0.0",
|
||||
"status": "ok",
|
||||
"document": {
|
||||
"contentHash": "sha256:…",
|
||||
"supersedes": null,
|
||||
"provenance": {
|
||||
"signature": { "format": "pgp", "present": true }
|
||||
}
|
||||
},
|
||||
"violations": []
|
||||
}
|
||||
```
|
||||
|
||||
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
|
||||
|
||||
### 2.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Guard passed; would-write payload is AOC compliant. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field (`severity`, `cvss`, etc.) detected. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt (multiple upstream sources fused). |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation (duplicate without supersedes). |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance fields. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings present (Policy-only data). |
|
||||
| `17` | `ERR_AOC_007` – Unknown top-level fields / schema violation. |
|
||||
| `70` | Transport error (network, auth, malformed input). |
|
||||
|
||||
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
|
||||
|
||||
### 2.6 Examples
|
||||
|
||||
Dry-run a local CSAF file:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source redhat \
|
||||
--input ./fixtures/redhat/RHSA-2025-1234.json
|
||||
```
|
||||
|
||||
Stream from HTTPS and emit JSON for CI:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source osv \
|
||||
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
|
||||
--format json \
|
||||
--output artifacts/osv-dry-run.json
|
||||
|
||||
cat artifacts/osv-dry-run.json | jq '.violations'
|
||||
```
|
||||
|
||||
### 2.7 Offline notes
|
||||
|
||||
When operating in sealed/offline mode:
|
||||
|
||||
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
|
||||
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
|
||||
- The command does not attempt network access when given a file path.
|
||||
- Store reports with `--output` to include in transfer packages for policy review.
|
||||
|
||||
---
|
||||
|
||||
## 3 · `stella aoc verify`
|
||||
|
||||
### 3.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
[--since <iso8601|duration>] \
|
||||
[--limit <count>] \
|
||||
[--sources <list>] \
|
||||
[--codes <ERR_AOC_00x,...>] \
|
||||
[--format table|json] \
|
||||
[--export <file>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--no-color]
|
||||
```
|
||||
|
||||
### 3.2 Description
|
||||
|
||||
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24 hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
|
||||
|
||||
### 3.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--since <value>` | Verification window. Accepts ISO 8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
|
||||
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
|
||||
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
|
||||
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
|
||||
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
|
||||
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
|
||||
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
|
||||
| `--no-color` | Disables ANSI colours. |
|
||||
|
||||
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
|
||||
|
||||
### 3.4 Report structure (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"tenant": "default",
|
||||
"window": {
|
||||
"from": "2025-10-25T12:00:00Z",
|
||||
"to": "2025-10-26T12:00:00Z"
|
||||
},
|
||||
"checked": {
|
||||
"advisories": 482,
|
||||
"vex": 75
|
||||
},
|
||||
"violations": [
|
||||
{
|
||||
"code": "ERR_AOC_001",
|
||||
"count": 2,
|
||||
"examples": [
|
||||
{
|
||||
"source": "redhat",
|
||||
"documentId": "advisory_raw:redhat:RHSA-2025:1",
|
||||
"contentHash": "sha256:…",
|
||||
"path": "/content/raw/cvss"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metrics": {
|
||||
"ingestion_write_total": 557,
|
||||
"aoc_violation_total": 2
|
||||
},
|
||||
"truncated": false
|
||||
}
|
||||
```
|
||||
|
||||
### 3.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Verification succeeded with zero violations. |
|
||||
| `11…17` | Same mapping as § 2.5 when violations are detected. Highest-priority code returned. |
|
||||
| `18` | Verification ran but results truncated (limit reached) – treat as warning; rerun with higher `--limit`. |
|
||||
| `70` | Transport/authentication error. |
|
||||
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
|
||||
|
||||
### 3.6 Examples
|
||||
|
||||
Daily verification across all sources:
|
||||
|
||||
```bash
|
||||
stella aoc verify --since 24h --format table
|
||||
```
|
||||
|
||||
CI pipeline focusing on errant sources and exporting evidence:
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--sources redhat,ubuntu \
|
||||
--codes ERR_AOC_001,ERR_AOC_004 \
|
||||
--format json \
|
||||
--limit 100 \
|
||||
--export artifacts/aoc-verify.json
|
||||
|
||||
jq '.violations[] | {code, count}' artifacts/aoc-verify.json
|
||||
```
|
||||
|
||||
Air-gapped verification using Offline Kit snapshot (example script):
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--since 7d \
|
||||
--format json \
|
||||
--export /mnt/offline/aoc-verify-$(date +%F).json
|
||||
|
||||
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
|
||||
```
|
||||
|
||||
### 3.7 Automation tips
|
||||
|
||||
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥ 11.
|
||||
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
|
||||
- Push JSON exports to observability pipelines for historical tracking of violation counts.
|
||||
|
||||
### 3.8 Offline notes
|
||||
|
||||
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
|
||||
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
|
||||
- Include verification output in compliance packages alongside Offline Kit manifests.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Global exit-code reference
|
||||
|
||||
| Code | Summary |
|
||||
|------|---------|
|
||||
| `0` | Success / no violations. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field present. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt detected. |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation. |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance/signature metadata. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings in ingestion payload. |
|
||||
| `17` | `ERR_AOC_007` – Schema violation / unknown fields. |
|
||||
| `18` | Partial verification (limit reached). |
|
||||
| `70` | Transport or HTTP failure. |
|
||||
| `71` | CLI usage error (invalid arguments, missing tenant). |
|
||||
|
||||
Use these codes in CI to map outcomes to build statuses or alert severities.
|
||||
|
||||
---
|
||||
|
||||
## 4 · `stella vuln observations` (Overlay paging)
|
||||
|
||||
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
|
||||
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
|
||||
|
||||
Additional notes:
|
||||
|
||||
- Table mode prints a hint when `hasMore` is `true`:
|
||||
`[yellow]More observations available. Continue with --cursor <token>[/]`.
|
||||
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
|
||||
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Related references
|
||||
|
||||
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
|
||||
- [Architecture overview](../../platform/architecture-overview.md)
|
||||
- [Console AOC dashboard](../../ui/console.md)
|
||||
- [Authority scopes](../../authority/architecture.md)
|
||||
|
||||
---
|
||||
|
||||
## 6 · Compliance checklist
|
||||
|
||||
- [ ] Usage documented for both table and JSON formats.
|
||||
- [ ] Exit-code mapping matches `ERR_AOC_00x` definitions and automation guidance.
|
||||
- [ ] Offline/air-gap workflow captured for both commands.
|
||||
- [ ] References to AOC architecture and console docs included.
|
||||
- [ ] Examples validated against current CLI syntax (update post-implementation).
|
||||
- [ ] Docs guild screenshot/narrative placeholder logged for release notes (pending CLI team capture).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-29 (Sprint 24).*
|
||||
|
||||
## 13. Authority configuration quick reference
|
||||
|
||||
| Setting | Purpose | How to set |
|
||||
|---------|---------|------------|
|
||||
| `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. |
|
||||
| `StellaOps:Authority:OperatorTicket` | Change/incident ticket reference paired with orchestrator control actions. | CLI flag `--Authority:OperatorTicket=...` or env `STELLAOPS_ORCH_TICKET`. |
|
||||
|
||||
> Tokens requesting `orch:operate` will fail with `invalid_request` unless both values are present. Choose concise strings (≤256 chars for reason, ≤128 chars for ticket) and avoid sensitive data.
|
||||
|
||||
23
docs/modules/cli/implementation_plan.md
Normal file
23
docs/modules/cli/implementation_plan.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Implementation plan — CLI
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy verbs, simulation, and explain outputs.
|
||||
- **Epic 4 – Policy Studio:** add registry/promotion workflows, lint tooling, and approvals UX.
|
||||
- **Epic 6 – Vulnerability Explorer:** integrate ledger/triage operations.
|
||||
- **Epic 10 – Export Center:** automate export verification and Offline Kit flows.
|
||||
- **Epic 11 – Notifications Studio:** manage rule/channel authoring and previews via CLI.
|
||||
- Track CLI-specific work (e.g., CLI-CORE-41-001, DOCS-CLI-OBS-52-001) in ../../TASKS.md and src/Cli/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in ../../implplan/SPRINTS.md.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -1,134 +1,134 @@
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
# CLI Release & Packaging Runbook
|
||||
|
||||
This runbook describes how to build, sign, package, and distribute the StellaOps CLI with Task Pack support. It covers connected and air-gapped workflows, SBOM generation, parity gating, and distribution artifacts required by Sprint 43 (`DEVOPS-CLI-43-001`, `DEPLOY-PACKS-43-001`).
|
||||
|
||||
---
|
||||
|
||||
## 1 · Release Artifacts
|
||||
|
||||
| Artifact | Description | Notes |
|
||||
|----------|-------------|-------|
|
||||
| `stella-<version>-linux-x64.tar.gz` | Linux binary + completions | Includes man pages, localization files. |
|
||||
| `stella-<version>-macos-universal.tar.gz` | macOS universal binary | Signed/notarized where applicable. |
|
||||
| `stella-<version>-windows-x64.zip` | Windows binary + PowerShell modules | Code-signed. |
|
||||
| `stella-cli-container:<version>` | OCI image with CLI + pack runtime | Deterministic rootfs (scratch/distroless). |
|
||||
| SBOM (`.cdx.json`) | CycloneDX SBOM per artifact | Generated via `stella sbom generate` or `syft`. |
|
||||
| Checksums (`SHA256SUMS`) | Aggregated digest list | Signed with cosign. |
|
||||
| Provenance (`.intoto.jsonl`) | DSSE attestation (SLSA L2) | Contains build metadata. |
|
||||
| Release notes | Markdown summary | Links to task packs docs, parity matrix. |
|
||||
|
||||
---
|
||||
|
||||
## 2 · Build Pipeline
|
||||
|
||||
1. **Source checkout** – pinned commit, reproducible environment (Docker).
|
||||
2. **Dependency lock** – `dotnet restore`, `npm ci` (for CLI frontends), ensure deterministic build flags.
|
||||
3. **Build binaries** – cross-platform targets with reproducible timestamps.
|
||||
4. **Run tests** – unit + integration; include `stella pack` commands (plan/run/verify) in CI.
|
||||
5. **Generate SBOM** – `syft packages dist/stella-linux-x64 --output cyclonedx-json`.
|
||||
6. **Bundle** – compress artifacts, include completions (`bash`, `zsh`, `fish`, PowerShell).
|
||||
7. **Sign** – cosign signatures for binaries, checksums, container image.
|
||||
8. **Publish** – upload to `downloads.stella-ops.org`, container registry, Packs Registry (for CLI container).
|
||||
9. **Parity gating** – run CLI parity matrix tests vs Console features (automation in `DEVOPS-CLI-43-001`).
|
||||
|
||||
CI must run in isolated environment (no network beyond allowlist). Cache dependencies for offline bundling.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Versioning & Channels
|
||||
|
||||
- Semantic versioning (`YYYY.MM.patch`), e.g., `2025.10.0`.
|
||||
- Channels:
|
||||
- `edge` – nightly builds, limited support.
|
||||
- `beta` – pre-release candidates.
|
||||
- `stable` – production-ready, after parity gating.
|
||||
- Release promotions mirror Task Pack channels; update downloads manifest (`deploy/downloads/manifest.json`).
|
||||
|
||||
---
|
||||
|
||||
## 4 · Signing & Verification
|
||||
|
||||
- Binaries signed with cosign (`cosign sign-blob`).
|
||||
- Container image signed (`cosign sign stella-cli-container:<version>`).
|
||||
- DSSE provenance includes:
|
||||
- Build pipeline ID.
|
||||
- Source commit and repo.
|
||||
- Dependencies SBOM digest.
|
||||
- Test results summary.
|
||||
- Verification command for operators:
|
||||
|
||||
```bash
|
||||
cosign verify-blob \
|
||||
--certificate-identity https://ci.stella-ops.org \
|
||||
--certificate-oidc-issuer https://fulcio.sigstore.dev \
|
||||
--signature stella-2025.10.0-linux-x64.sig \
|
||||
stella-2025.10.0-linux-x64.tar.gz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5 · Distribution
|
||||
|
||||
### 5.1 Online
|
||||
|
||||
- Publish artifacts to Downloads service; update manifest with digests, SBOM URLs, attestations.
|
||||
- Update CLI parity docs (`docs/cli-vs-ui-parity.md`) and release notes.
|
||||
- Push container image to registry with SBOM + attestations referenced as OCI referrers.
|
||||
- Notify stakeholders via `#release-cli` channel and release mailing list.
|
||||
|
||||
### 5.2 Offline / Air-Gap
|
||||
|
||||
- Bundle CLI artifacts, Task Pack samples, and registry mirror:
|
||||
|
||||
```bash
|
||||
stella pack bundle export \
|
||||
--packs "sbom-remediation:1.3.0" \
|
||||
--output offline/packs-bundle-2025.10.0.tgz
|
||||
|
||||
stella cli bundle export \
|
||||
--output offline/cli-2025.10.0.tgz \
|
||||
--include-container \
|
||||
--include-sbom
|
||||
```
|
||||
|
||||
- Update Offline Kit manifest with new CLI version and pack bundle entries.
|
||||
- Provide import scripts (`ouk import`) for sealed sites.
|
||||
|
||||
---
|
||||
|
||||
## 6 · Parity Gating
|
||||
|
||||
- `stella cli parity check` compares CLI commands vs parity matrix.
|
||||
- CI fails release if any required command flagged `🟥` or `🟡` with severity > threshold.
|
||||
- Parity report uploaded to Downloads workspace and linked in docs.
|
||||
- Manual review required for new commands (ensure `man` pages and help text localized).
|
||||
|
||||
---
|
||||
|
||||
## 7 · Localization & Documentation
|
||||
|
||||
- CLI includes localization bundles; ensure `i18n.txz` packaged.
|
||||
- Update man pages (`man/stella-pack.1`) and HTML docs.
|
||||
- Sync docs: `docs/cli/overview.md`, pack authoring guide, release notes.
|
||||
- Document new flags/commands in `docs/cli/commands/pack.md` (tracked in Sprint 42 tasks).
|
||||
|
||||
---
|
||||
|
||||
## 8 · Release Checklist
|
||||
|
||||
- [ ] All binaries built reproducibly (CI logs archived).
|
||||
- [ ] Tests + parity matrix passing.
|
||||
- [ ] SBOM + provenance generated and published.
|
||||
- [ ] Cosign signatures created and verified.
|
||||
- [ ] Downloads manifest updated (edge/beta/stable).
|
||||
- [ ] Offline bundle exported and validated.
|
||||
- [ ] Release notes + documentation updates merged.
|
||||
- [ ] Notifications sent (chat/email).
|
||||
- [ ] Imposed rule reminder present at top of document.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 43).*
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
# CLI Release & Packaging Runbook
|
||||
|
||||
This runbook describes how to build, sign, package, and distribute the StellaOps CLI with Task Pack support. It covers connected and air-gapped workflows, SBOM generation, parity gating, and distribution artifacts required by Sprint 43 (`DEVOPS-CLI-43-001`, `DEPLOY-PACKS-43-001`).
|
||||
|
||||
---
|
||||
|
||||
## 1 · Release Artifacts
|
||||
|
||||
| Artifact | Description | Notes |
|
||||
|----------|-------------|-------|
|
||||
| `stella-<version>-linux-x64.tar.gz` | Linux binary + completions | Includes man pages, localization files. |
|
||||
| `stella-<version>-macos-universal.tar.gz` | macOS universal binary | Signed/notarized where applicable. |
|
||||
| `stella-<version>-windows-x64.zip` | Windows binary + PowerShell modules | Code-signed. |
|
||||
| `stella-cli-container:<version>` | OCI image with CLI + pack runtime | Deterministic rootfs (scratch/distroless). |
|
||||
| SBOM (`.cdx.json`) | CycloneDX SBOM per artifact | Generated via `stella sbom generate` or `syft`. |
|
||||
| Checksums (`SHA256SUMS`) | Aggregated digest list | Signed with cosign. |
|
||||
| Provenance (`.intoto.jsonl`) | DSSE attestation (SLSA L2) | Contains build metadata. |
|
||||
| Release notes | Markdown summary | Links to task packs docs, parity matrix. |
|
||||
|
||||
---
|
||||
|
||||
## 2 · Build Pipeline
|
||||
|
||||
1. **Source checkout** – pinned commit, reproducible environment (Docker).
|
||||
2. **Dependency lock** – `dotnet restore`, `npm ci` (for CLI frontends), ensure deterministic build flags.
|
||||
3. **Build binaries** – cross-platform targets with reproducible timestamps.
|
||||
4. **Run tests** – unit + integration; include `stella pack` commands (plan/run/verify) in CI.
|
||||
5. **Generate SBOM** – `syft packages dist/stella-linux-x64 --output cyclonedx-json`.
|
||||
6. **Bundle** – compress artifacts, include completions (`bash`, `zsh`, `fish`, PowerShell).
|
||||
7. **Sign** – cosign signatures for binaries, checksums, container image.
|
||||
8. **Publish** – upload to `downloads.stella-ops.org`, container registry, Packs Registry (for CLI container).
|
||||
9. **Parity gating** – run CLI parity matrix tests vs Console features (automation in `DEVOPS-CLI-43-001`).
|
||||
|
||||
CI must run in isolated environment (no network beyond allowlist). Cache dependencies for offline bundling.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Versioning & Channels
|
||||
|
||||
- Semantic versioning (`YYYY.MM.patch`), e.g., `2025.10.0`.
|
||||
- Channels:
|
||||
- `edge` – nightly builds, limited support.
|
||||
- `beta` – pre-release candidates.
|
||||
- `stable` – production-ready, after parity gating.
|
||||
- Release promotions mirror Task Pack channels; update downloads manifest (`deploy/downloads/manifest.json`).
|
||||
|
||||
---
|
||||
|
||||
## 4 · Signing & Verification
|
||||
|
||||
- Binaries signed with cosign (`cosign sign-blob`).
|
||||
- Container image signed (`cosign sign stella-cli-container:<version>`).
|
||||
- DSSE provenance includes:
|
||||
- Build pipeline ID.
|
||||
- Source commit and repo.
|
||||
- Dependencies SBOM digest.
|
||||
- Test results summary.
|
||||
- Verification command for operators:
|
||||
|
||||
```bash
|
||||
cosign verify-blob \
|
||||
--certificate-identity https://ci.stella-ops.org \
|
||||
--certificate-oidc-issuer https://fulcio.sigstore.dev \
|
||||
--signature stella-2025.10.0-linux-x64.sig \
|
||||
stella-2025.10.0-linux-x64.tar.gz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5 · Distribution
|
||||
|
||||
### 5.1 Online
|
||||
|
||||
- Publish artifacts to Downloads service; update manifest with digests, SBOM URLs, attestations.
|
||||
- Update CLI parity docs (`docs/cli-vs-ui-parity.md`) and release notes.
|
||||
- Push container image to registry with SBOM + attestations referenced as OCI referrers.
|
||||
- Notify stakeholders via `#release-cli` channel and release mailing list.
|
||||
|
||||
### 5.2 Offline / Air-Gap
|
||||
|
||||
- Bundle CLI artifacts, Task Pack samples, and registry mirror:
|
||||
|
||||
```bash
|
||||
stella pack bundle export \
|
||||
--packs "sbom-remediation:1.3.0" \
|
||||
--output offline/packs-bundle-2025.10.0.tgz
|
||||
|
||||
stella cli bundle export \
|
||||
--output offline/cli-2025.10.0.tgz \
|
||||
--include-container \
|
||||
--include-sbom
|
||||
```
|
||||
|
||||
- Update Offline Kit manifest with new CLI version and pack bundle entries.
|
||||
- Provide import scripts (`ouk import`) for sealed sites.
|
||||
|
||||
---
|
||||
|
||||
## 6 · Parity Gating
|
||||
|
||||
- `stella cli parity check` compares CLI commands vs parity matrix.
|
||||
- CI fails release if any required command flagged `🟥` or `🟡` with severity > threshold.
|
||||
- Parity report uploaded to Downloads workspace and linked in docs.
|
||||
- Manual review required for new commands (ensure `man` pages and help text localized).
|
||||
|
||||
---
|
||||
|
||||
## 7 · Localization & Documentation
|
||||
|
||||
- CLI includes localization bundles; ensure `i18n.txz` packaged.
|
||||
- Update man pages (`man/stella-pack.1`) and HTML docs.
|
||||
- Sync docs: `docs/modules/cli/guides/overview.md`, pack authoring guide, release notes.
|
||||
- Document new flags/commands in `docs/modules/cli/guides/commands/pack.md` (tracked in Sprint 42 tasks).
|
||||
|
||||
---
|
||||
|
||||
## 8 · Release Checklist
|
||||
|
||||
- [ ] All binaries built reproducibly (CI logs archived).
|
||||
- [ ] Tests + parity matrix passing.
|
||||
- [ ] SBOM + provenance generated and published.
|
||||
- [ ] Cosign signatures created and verified.
|
||||
- [ ] Downloads manifest updated (edge/beta/stable).
|
||||
- [ ] Offline bundle exported and validated.
|
||||
- [ ] Release notes + documentation updates merged.
|
||||
- [ ] Notifications sent (chat/email).
|
||||
- [ ] Imposed rule reminder present at top of document.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 43).*
|
||||
|
||||
22
docs/modules/concelier/AGENTS.md
Normal file
22
docs/modules/concelier/AGENTS.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Concelier agent guide
|
||||
|
||||
## Mission
|
||||
Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC).
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
|
||||
## How to get started
|
||||
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
|
||||
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
|
||||
3. Read the architecture and README for domain context before editing code or docs.
|
||||
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
|
||||
|
||||
## Guardrails
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
|
||||
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
|
||||
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
|
||||
- Update runbooks/observability assets when operational characteristics change.
|
||||
36
docs/modules/concelier/README.md
Normal file
36
docs/modules/concelier/README.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# StellaOps Concelier
|
||||
|
||||
Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC).
|
||||
|
||||
## Responsibilities
|
||||
- Fetch and normalise vulnerability advisories via restart-time connectors.
|
||||
- Persist observations and correlation linksets without precedence decisions.
|
||||
- Emit deterministic exports (JSON, Trivy DB) for downstream policy evaluation.
|
||||
- Coordinate offline/air-gap updates via Offline Kit bundles.
|
||||
|
||||
## Key components
|
||||
- `StellaOps.Concelier.WebService` orchestration host.
|
||||
- Connector libraries under `StellaOps.Concelier.Connector.*`.
|
||||
- Exporter packages (`StellaOps.Concelier.Exporter.*`).
|
||||
|
||||
## Integrations & dependencies
|
||||
- MongoDB for canonical observations and schedules.
|
||||
- Policy Engine / Export Center / CLI for evidence consumption.
|
||||
- Notify and UI for advisory deltas.
|
||||
|
||||
## Operational notes
|
||||
- Connector runbooks in ./operations/connectors/.
|
||||
- Mirror operations for Offline Kit parity.
|
||||
- Grafana dashboards for connector health.
|
||||
|
||||
## Related resources
|
||||
- ./operations/conflict-resolution.md
|
||||
- ./operations/mirror.md
|
||||
|
||||
## Backlog references
|
||||
- DOCS-LNM-22-001, DOCS-LNM-22-007 in ../../TASKS.md.
|
||||
- Connector-specific TODOs in `src/Concelier/**/TASKS.md`.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 1 – AOC enforcement:** uphold raw observation invariants, provenance requirements, linkset-only enrichment, and AOC verifier guardrails across every connector.
|
||||
- **Epic 10 – Export Center:** expose deterministic advisory exports and metadata required by JSON/Trivy/mirror bundles.
|
||||
9
docs/modules/concelier/TASKS.md
Normal file
9
docs/modules/concelier/TASKS.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Task board — Concelier
|
||||
|
||||
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| CONCELIER-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
|
||||
| CONCELIER-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
|
||||
| CONCELIER-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
|
||||
File diff suppressed because it is too large
Load Diff
67
docs/modules/concelier/implementation_plan.md
Normal file
67
docs/modules/concelier/implementation_plan.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Implementation plan — Concelier
|
||||
|
||||
## Delivery timeline
|
||||
- **Phase 1 — Guardrails & schema**
|
||||
Stand up Mongo JSON validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
|
||||
- **Phase 2 — API & observability**
|
||||
Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps.
|
||||
- **Phase 3 — Experience polish**
|
||||
Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures).
|
||||
|
||||
## Work breakdown by component
|
||||
- **Concelier WebService & worker**
|
||||
- Add Mongo validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
|
||||
- Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts.
|
||||
- Deterministically compute linksets and persist canonical JSON payloads.
|
||||
- Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes.
|
||||
- Emit guard metrics/traces and surface supersedes/violation audit logs.
|
||||
- **Excititor (shared ingestion contract)**
|
||||
- Mirror Concelier guard and schema changes for `vex_raw`.
|
||||
- Maintain restart-time plug-in determinism and linkset extraction parity.
|
||||
- **Shared libraries**
|
||||
- Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification).
|
||||
- Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities.
|
||||
- **Policy Engine integration**
|
||||
- Enforce `effective_finding_*` write exclusivity.
|
||||
- Consume only raw documents + linksets, removing any implicit normalisation.
|
||||
- **Authority scopes**
|
||||
- Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services.
|
||||
- **CLI & Console**
|
||||
- Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`).
|
||||
- Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console.
|
||||
- **CI/CD**
|
||||
- Add Roslyn analyzer / AST lint to block forbidden writes.
|
||||
- Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating.
|
||||
|
||||
## Documentation deliverables
|
||||
- Update `docs/ingestion/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
|
||||
- Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards.
|
||||
- Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes.
|
||||
- Ensure Offline Kit documentation captures validator bootstrap and verify workflows.
|
||||
|
||||
## Acceptance criteria
|
||||
- Mongo validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
|
||||
- Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents.
|
||||
- CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status.
|
||||
- Export Center consumes advisory datasets without relying on legacy normalised fields.
|
||||
- CI fails if lint rules detect forbidden writes or if seeded guard tests regress.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes.
|
||||
- **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity.
|
||||
- **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets.
|
||||
- **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers.
|
||||
|
||||
## Test strategy
|
||||
- **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism.
|
||||
- **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge.
|
||||
- **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes.
|
||||
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
|
||||
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.
|
||||
|
||||
## Definition of done
|
||||
- Validators deployed and verified in staging/offline environments.
|
||||
- Runtime guards, CLI/Console workflows, and CI linting all active.
|
||||
- Observability dashboards and runbooks updated; metrics visible.
|
||||
- Documentation updates merged; Offline Kit instructions published.
|
||||
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.
|
||||
@@ -1,159 +1,159 @@
|
||||
# Concelier Authority Audit Runbook
|
||||
|
||||
_Last updated: 2025-10-22_
|
||||
|
||||
This runbook helps operators verify and monitor the StellaOps Concelier ⇆ Authority integration. It focuses on the `/jobs*` surface, which now requires StellaOps Authority tokens, and the corresponding audit/metric signals that expose authentication and bypass activity.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Authority integration is enabled in `concelier.yaml` (or via `CONCELIER_AUTHORITY__*` environment variables) with a valid `clientId`, secret, audience, and required scopes.
|
||||
- OTLP metrics/log exporters are configured (`concelier.telemetry.*`) or container stdout is shipped to your SIEM.
|
||||
- Operators have access to the Concelier job trigger endpoints via CLI or REST for smoke tests.
|
||||
- The rollout table in `docs/10_CONCELIER_CLI_QUICKSTART.md` has been reviewed so stakeholders align on the staged → enforced toggle timeline.
|
||||
|
||||
### Configuration snippet
|
||||
|
||||
```yaml
|
||||
concelier:
|
||||
authority:
|
||||
enabled: true
|
||||
allowAnonymousFallback: false # keep true only during initial rollout
|
||||
issuer: "https://authority.internal"
|
||||
audiences:
|
||||
- "api://concelier"
|
||||
requiredScopes:
|
||||
- "concelier.jobs.trigger"
|
||||
- "advisory:read"
|
||||
- "advisory:ingest"
|
||||
requiredTenants:
|
||||
- "tenant-default"
|
||||
bypassNetworks:
|
||||
- "127.0.0.1/32"
|
||||
- "::1/128"
|
||||
clientId: "concelier-jobs"
|
||||
clientSecretFile: "/run/secrets/concelier_authority_client"
|
||||
tokenClockSkewSeconds: 60
|
||||
resilience:
|
||||
enableRetries: true
|
||||
retryDelays:
|
||||
- "00:00:01"
|
||||
- "00:00:02"
|
||||
- "00:00:05"
|
||||
allowOfflineCacheFallback: true
|
||||
offlineCacheTolerance: "00:10:00"
|
||||
```
|
||||
|
||||
> Store secrets outside source control. Concelier reads `clientSecretFile` on startup; rotate by updating the mounted file and restarting the service.
|
||||
|
||||
### Resilience tuning
|
||||
|
||||
- **Connected sites:** keep the default 1 s / 2 s / 5 s retry ladder so Concelier retries transient Authority hiccups but still surfaces outages quickly. Leave `allowOfflineCacheFallback=true` so cached discovery/JWKS data can bridge short Pathfinder restarts.
|
||||
- **Air-gapped/Offline Kit installs:** extend `offlineCacheTolerance` (15–30 minutes) to keep the cached metadata valid between manual synchronisations. You can also disable retries (`enableRetries=false`) if infrastructure teams prefer to handle exponential backoff at the network layer; Concelier will fail fast but keep deterministic logs.
|
||||
- Concelier resolves these knobs through `IOptionsMonitor<StellaOpsAuthClientOptions>`. Edits to `concelier.yaml` are applied on configuration reload; restart the container if you change environment variables or do not have file-watch reloads enabled.
|
||||
|
||||
## 2. Key Signals
|
||||
|
||||
### 2.1 Audit log channel
|
||||
|
||||
Concelier emits structured audit entries via the `Concelier.Authorization.Audit` logger for every `/jobs*` request once Authority enforcement is active.
|
||||
|
||||
```
|
||||
Concelier authorization audit route=/jobs/definitions status=200 subject=ops@example.com clientId=concelier-cli scopes=concelier.jobs.trigger advisory:ingest bypass=False remote=10.1.4.7
|
||||
```
|
||||
|
||||
| Field | Sample value | Meaning |
|
||||
|--------------|-------------------------|------------------------------------------------------------------------------------------|
|
||||
| `route` | `/jobs/definitions` | Endpoint that processed the request. |
|
||||
| `status` | `200` / `401` / `409` | Final HTTP status code returned to the caller. |
|
||||
| `subject` | `ops@example.com` | User or service principal subject (falls back to `(anonymous)` when unauthenticated). |
|
||||
| `clientId` | `concelier-cli` | OAuth client ID provided by Authority (`(none)` if the token lacked the claim). |
|
||||
| `scopes` | `concelier.jobs.trigger advisory:ingest advisory:read` | Normalised scope list extracted from token claims; `(none)` if the token carried none. |
|
||||
| `tenant` | `tenant-default` | Tenant claim extracted from the Authority token (`(none)` when the token lacked it). |
|
||||
| `bypass` | `True` / `False` | Indicates whether the request succeeded because its source IP matched a bypass CIDR. |
|
||||
| `remote` | `10.1.4.7` | Remote IP recorded from the connection / forwarded header test hooks. |
|
||||
|
||||
Use your logging backend (e.g., Loki) to index the logger name and filter for suspicious combinations:
|
||||
|
||||
- `status=401 AND bypass=True` – bypass network accepted an unauthenticated call (should be temporary during rollout).
|
||||
- `status=202 AND scopes="(none)"` – a token without scopes triggered a job; tighten client configuration.
|
||||
- `status=202 AND NOT contains(scopes,"advisory:ingest")` – ingestion attempted without the new AOC scopes; confirm the Authority client registration matches the sample above.
|
||||
- `tenant!=(tenant-default)` – indicates a cross-tenant token was accepted. Ensure Concelier `requiredTenants` is aligned with Authority client registration.
|
||||
- Spike in `clientId="(none)"` – indicates upstream Authority is not issuing `client_id` claims or the CLI is outdated.
|
||||
|
||||
### 2.2 Metrics
|
||||
|
||||
Concelier publishes counters under the OTEL meter `StellaOps.Concelier.WebService.Jobs`. Tags: `job.kind`, `job.trigger`, `job.outcome`.
|
||||
|
||||
| Metric name | Description | PromQL example |
|
||||
|-------------------------------|----------------------------------------------------|----------------|
|
||||
| `web.jobs.triggered` | Accepted job trigger requests. | `sum by (job_kind) (rate(web_jobs_triggered_total[5m]))` |
|
||||
| `web.jobs.trigger.conflict` | Rejected triggers (already running, disabled…). | `sum(rate(web_jobs_trigger_conflict_total[5m]))` |
|
||||
| `web.jobs.trigger.failed` | Server-side job failures. | `sum(rate(web_jobs_trigger_failed_total[5m]))` |
|
||||
|
||||
> Prometheus/OTEL collectors typically surface counters with `_total` suffix. Adjust queries to match your pipeline’s generated metric names.
|
||||
|
||||
Correlate audit logs with the following global meter exported via `Concelier.SourceDiagnostics`:
|
||||
|
||||
- `concelier.source.http.requests_total{concelier_source="jobs-run"}` – ensures REST/manual triggers route through Authority.
|
||||
- If Grafana dashboards are deployed, extend the “Concelier Jobs” board with the above counters plus a table of recent audit log entries.
|
||||
|
||||
## 3. Alerting Guidance
|
||||
|
||||
1. **Unauthorized bypass attempt**
|
||||
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", status="401", bypass="True"}[5m])) > 0`
|
||||
- Action: verify `bypassNetworks` list; confirm expected maintenance windows; rotate credentials if suspicious.
|
||||
|
||||
2. **Missing scopes**
|
||||
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", scopes="(none)", status="200"}[5m])) > 0`
|
||||
- Action: audit Authority client registration; ensure `requiredScopes` includes `concelier.jobs.trigger`, `advisory:ingest`, and `advisory:read`.
|
||||
|
||||
3. **Trigger failure surge**
|
||||
- Query: `sum(rate(web_jobs_trigger_failed_total[10m])) > 0` with severity `warning` if sustained for 10 minutes.
|
||||
- Action: inspect correlated audit entries and `Concelier.Telemetry` traces for job execution errors.
|
||||
|
||||
4. **Conflict spike**
|
||||
- Query: `sum(rate(web_jobs_trigger_conflict_total[10m])) > 5` (tune threshold).
|
||||
- Action: downstream scheduling may be firing repetitive triggers; ensure precedence is configured properly.
|
||||
|
||||
5. **Authority offline**
|
||||
- Watch `Concelier.Authorization.Audit` logs for `status=503` or `status=500` along with `clientId="(none)"`. Investigate Authority availability before re-enabling anonymous fallback.
|
||||
|
||||
## 4. Rollout & Verification Procedure
|
||||
|
||||
1. **Pre-checks**
|
||||
- Align with the rollout phases documented in `docs/10_CONCELIER_CLI_QUICKSTART.md` (validation → rehearsal → enforced) and record the target dates in your change request.
|
||||
- Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation.
|
||||
- Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host).
|
||||
|
||||
2. **Smoke test with valid token**
|
||||
- Obtain a token via CLI: `stella auth login --scope "concelier.jobs.trigger advisory:ingest" --scope advisory:read`.
|
||||
- Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://concelier.internal/jobs/definitions`.
|
||||
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger advisory:ingest advisory:read`, and `tenant=tenant-default`.
|
||||
|
||||
3. **Negative test without token**
|
||||
- Call the same endpoint without a token. Expect HTTP 401, `bypass=False`.
|
||||
- If the request succeeds, double-check `bypassNetworks` and ensure fallback is disabled.
|
||||
|
||||
4. **Bypass check (if applicable)**
|
||||
- From an allowed maintenance IP, call `/jobs/definitions` without a token. Confirm the audit log shows `bypass=True`. Review business justification and expiry date for such entries.
|
||||
|
||||
5. **Metrics validation**
|
||||
- Ensure `web.jobs.triggered` counter increments during accepted runs.
|
||||
- Exporters should show corresponding spans (`concelier.job.trigger`) if tracing is enabled.
|
||||
|
||||
## 5. Troubleshooting
|
||||
|
||||
| Symptom | Probable cause | Remediation |
|
||||
|---------|----------------|-------------|
|
||||
| Audit log shows `clientId=(none)` for all requests | Authority not issuing `client_id` claim or CLI outdated | Update StellaOps Authority configuration (`StellaOpsAuthorityOptions.Token.Claims.ClientId`), or upgrade the CLI token acquisition flow. |
|
||||
| Requests succeed with `bypass=True` unexpectedly | Local network added to `bypassNetworks` or fallback still enabled | Remove/adjust the CIDR list, disable anonymous fallback, restart Concelier. |
|
||||
| HTTP 401 with valid token | `requiredScopes` missing from client registration or token audience mismatch | Verify Authority client scopes (`concelier.jobs.trigger`) and ensure the token audience matches `audiences` config. |
|
||||
| Metrics missing from Prometheus | Telemetry exporters disabled or filter missing OTEL meter | Set `concelier.telemetry.enableMetrics=true`, ensure collector includes `StellaOps.Concelier.WebService.Jobs` meter. |
|
||||
| Sudden spike in `web.jobs.trigger.failed` | Downstream job failure or Authority timeout mid-request | Inspect Concelier job logs, re-run with tracing enabled, validate Authority latency. |
|
||||
|
||||
## 6. References
|
||||
|
||||
- `docs/21_INSTALL_GUIDE.md` – Authority configuration quick start.
|
||||
- `docs/17_SECURITY_HARDENING_GUIDE.md` – Security guardrails and enforcement deadlines.
|
||||
- `docs/ops/authority-monitoring.md` – Authority-side monitoring and alerting playbook.
|
||||
- `StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields.
|
||||
# Concelier Authority Audit Runbook
|
||||
|
||||
_Last updated: 2025-10-22_
|
||||
|
||||
This runbook helps operators verify and monitor the StellaOps Concelier ⇆ Authority integration. It focuses on the `/jobs*` surface, which now requires StellaOps Authority tokens, and the corresponding audit/metric signals that expose authentication and bypass activity.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Authority integration is enabled in `concelier.yaml` (or via `CONCELIER_AUTHORITY__*` environment variables) with a valid `clientId`, secret, audience, and required scopes.
|
||||
- OTLP metrics/log exporters are configured (`concelier.telemetry.*`) or container stdout is shipped to your SIEM.
|
||||
- Operators have access to the Concelier job trigger endpoints via CLI or REST for smoke tests.
|
||||
- The rollout table in `docs/10_CONCELIER_CLI_QUICKSTART.md` has been reviewed so stakeholders align on the staged → enforced toggle timeline.
|
||||
|
||||
### Configuration snippet
|
||||
|
||||
```yaml
|
||||
concelier:
|
||||
authority:
|
||||
enabled: true
|
||||
allowAnonymousFallback: false # keep true only during initial rollout
|
||||
issuer: "https://authority.internal"
|
||||
audiences:
|
||||
- "api://concelier"
|
||||
requiredScopes:
|
||||
- "concelier.jobs.trigger"
|
||||
- "advisory:read"
|
||||
- "advisory:ingest"
|
||||
requiredTenants:
|
||||
- "tenant-default"
|
||||
bypassNetworks:
|
||||
- "127.0.0.1/32"
|
||||
- "::1/128"
|
||||
clientId: "concelier-jobs"
|
||||
clientSecretFile: "/run/secrets/concelier_authority_client"
|
||||
tokenClockSkewSeconds: 60
|
||||
resilience:
|
||||
enableRetries: true
|
||||
retryDelays:
|
||||
- "00:00:01"
|
||||
- "00:00:02"
|
||||
- "00:00:05"
|
||||
allowOfflineCacheFallback: true
|
||||
offlineCacheTolerance: "00:10:00"
|
||||
```
|
||||
|
||||
> Store secrets outside source control. Concelier reads `clientSecretFile` on startup; rotate by updating the mounted file and restarting the service.
|
||||
|
||||
### Resilience tuning
|
||||
|
||||
- **Connected sites:** keep the default 1 s / 2 s / 5 s retry ladder so Concelier retries transient Authority hiccups but still surfaces outages quickly. Leave `allowOfflineCacheFallback=true` so cached discovery/JWKS data can bridge short Pathfinder restarts.
|
||||
- **Air-gapped/Offline Kit installs:** extend `offlineCacheTolerance` (15–30 minutes) to keep the cached metadata valid between manual synchronisations. You can also disable retries (`enableRetries=false`) if infrastructure teams prefer to handle exponential backoff at the network layer; Concelier will fail fast but keep deterministic logs.
|
||||
- Concelier resolves these knobs through `IOptionsMonitor<StellaOpsAuthClientOptions>`. Edits to `concelier.yaml` are applied on configuration reload; restart the container if you change environment variables or do not have file-watch reloads enabled.
|
||||
|
||||
## 2. Key Signals
|
||||
|
||||
### 2.1 Audit log channel
|
||||
|
||||
Concelier emits structured audit entries via the `Concelier.Authorization.Audit` logger for every `/jobs*` request once Authority enforcement is active.
|
||||
|
||||
```
|
||||
Concelier authorization audit route=/jobs/definitions status=200 subject=ops@example.com clientId=concelier-cli scopes=concelier.jobs.trigger advisory:ingest bypass=False remote=10.1.4.7
|
||||
```
|
||||
|
||||
| Field | Sample value | Meaning |
|
||||
|--------------|-------------------------|------------------------------------------------------------------------------------------|
|
||||
| `route` | `/jobs/definitions` | Endpoint that processed the request. |
|
||||
| `status` | `200` / `401` / `409` | Final HTTP status code returned to the caller. |
|
||||
| `subject` | `ops@example.com` | User or service principal subject (falls back to `(anonymous)` when unauthenticated). |
|
||||
| `clientId` | `concelier-cli` | OAuth client ID provided by Authority (`(none)` if the token lacked the claim). |
|
||||
| `scopes` | `concelier.jobs.trigger advisory:ingest advisory:read` | Normalised scope list extracted from token claims; `(none)` if the token carried none. |
|
||||
| `tenant` | `tenant-default` | Tenant claim extracted from the Authority token (`(none)` when the token lacked it). |
|
||||
| `bypass` | `True` / `False` | Indicates whether the request succeeded because its source IP matched a bypass CIDR. |
|
||||
| `remote` | `10.1.4.7` | Remote IP recorded from the connection / forwarded header test hooks. |
|
||||
|
||||
Use your logging backend (e.g., Loki) to index the logger name and filter for suspicious combinations:
|
||||
|
||||
- `status=401 AND bypass=True` – bypass network accepted an unauthenticated call (should be temporary during rollout).
|
||||
- `status=202 AND scopes="(none)"` – a token without scopes triggered a job; tighten client configuration.
|
||||
- `status=202 AND NOT contains(scopes,"advisory:ingest")` – ingestion attempted without the new AOC scopes; confirm the Authority client registration matches the sample above.
|
||||
- `tenant!=(tenant-default)` – indicates a cross-tenant token was accepted. Ensure Concelier `requiredTenants` is aligned with Authority client registration.
|
||||
- Spike in `clientId="(none)"` – indicates upstream Authority is not issuing `client_id` claims or the CLI is outdated.
|
||||
|
||||
### 2.2 Metrics
|
||||
|
||||
Concelier publishes counters under the OTEL meter `StellaOps.Concelier.WebService.Jobs`. Tags: `job.kind`, `job.trigger`, `job.outcome`.
|
||||
|
||||
| Metric name | Description | PromQL example |
|
||||
|-------------------------------|----------------------------------------------------|----------------|
|
||||
| `web.jobs.triggered` | Accepted job trigger requests. | `sum by (job_kind) (rate(web_jobs_triggered_total[5m]))` |
|
||||
| `web.jobs.trigger.conflict` | Rejected triggers (already running, disabled…). | `sum(rate(web_jobs_trigger_conflict_total[5m]))` |
|
||||
| `web.jobs.trigger.failed` | Server-side job failures. | `sum(rate(web_jobs_trigger_failed_total[5m]))` |
|
||||
|
||||
> Prometheus/OTEL collectors typically surface counters with `_total` suffix. Adjust queries to match your pipeline’s generated metric names.
|
||||
|
||||
Correlate audit logs with the following global meter exported via `Concelier.SourceDiagnostics`:
|
||||
|
||||
- `concelier.source.http.requests_total{concelier_source="jobs-run"}` – ensures REST/manual triggers route through Authority.
|
||||
- If Grafana dashboards are deployed, extend the “Concelier Jobs” board with the above counters plus a table of recent audit log entries.
|
||||
|
||||
## 3. Alerting Guidance
|
||||
|
||||
1. **Unauthorized bypass attempt**
|
||||
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", status="401", bypass="True"}[5m])) > 0`
|
||||
- Action: verify `bypassNetworks` list; confirm expected maintenance windows; rotate credentials if suspicious.
|
||||
|
||||
2. **Missing scopes**
|
||||
- Query: `sum(rate(log_messages_total{logger="Concelier.Authorization.Audit", scopes="(none)", status="200"}[5m])) > 0`
|
||||
- Action: audit Authority client registration; ensure `requiredScopes` includes `concelier.jobs.trigger`, `advisory:ingest`, and `advisory:read`.
|
||||
|
||||
3. **Trigger failure surge**
|
||||
- Query: `sum(rate(web_jobs_trigger_failed_total[10m])) > 0` with severity `warning` if sustained for 10 minutes.
|
||||
- Action: inspect correlated audit entries and `Concelier.Telemetry` traces for job execution errors.
|
||||
|
||||
4. **Conflict spike**
|
||||
- Query: `sum(rate(web_jobs_trigger_conflict_total[10m])) > 5` (tune threshold).
|
||||
- Action: downstream scheduling may be firing repetitive triggers; ensure precedence is configured properly.
|
||||
|
||||
5. **Authority offline**
|
||||
- Watch `Concelier.Authorization.Audit` logs for `status=503` or `status=500` along with `clientId="(none)"`. Investigate Authority availability before re-enabling anonymous fallback.
|
||||
|
||||
## 4. Rollout & Verification Procedure
|
||||
|
||||
1. **Pre-checks**
|
||||
- Align with the rollout phases documented in `docs/10_CONCELIER_CLI_QUICKSTART.md` (validation → rehearsal → enforced) and record the target dates in your change request.
|
||||
- Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation.
|
||||
- Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host).
|
||||
|
||||
2. **Smoke test with valid token**
|
||||
- Obtain a token via CLI: `stella auth login --scope "concelier.jobs.trigger advisory:ingest" --scope advisory:read`.
|
||||
- Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://concelier.internal/jobs/definitions`.
|
||||
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger advisory:ingest advisory:read`, and `tenant=tenant-default`.
|
||||
|
||||
3. **Negative test without token**
|
||||
- Call the same endpoint without a token. Expect HTTP 401, `bypass=False`.
|
||||
- If the request succeeds, double-check `bypassNetworks` and ensure fallback is disabled.
|
||||
|
||||
4. **Bypass check (if applicable)**
|
||||
- From an allowed maintenance IP, call `/jobs/definitions` without a token. Confirm the audit log shows `bypass=True`. Review business justification and expiry date for such entries.
|
||||
|
||||
5. **Metrics validation**
|
||||
- Ensure `web.jobs.triggered` counter increments during accepted runs.
|
||||
- Exporters should show corresponding spans (`concelier.job.trigger`) if tracing is enabled.
|
||||
|
||||
## 5. Troubleshooting
|
||||
|
||||
| Symptom | Probable cause | Remediation |
|
||||
|---------|----------------|-------------|
|
||||
| Audit log shows `clientId=(none)` for all requests | Authority not issuing `client_id` claim or CLI outdated | Update StellaOps Authority configuration (`StellaOpsAuthorityOptions.Token.Claims.ClientId`), or upgrade the CLI token acquisition flow. |
|
||||
| Requests succeed with `bypass=True` unexpectedly | Local network added to `bypassNetworks` or fallback still enabled | Remove/adjust the CIDR list, disable anonymous fallback, restart Concelier. |
|
||||
| HTTP 401 with valid token | `requiredScopes` missing from client registration or token audience mismatch | Verify Authority client scopes (`concelier.jobs.trigger`) and ensure the token audience matches `audiences` config. |
|
||||
| Metrics missing from Prometheus | Telemetry exporters disabled or filter missing OTEL meter | Set `concelier.telemetry.enableMetrics=true`, ensure collector includes `StellaOps.Concelier.WebService.Jobs` meter. |
|
||||
| Sudden spike in `web.jobs.trigger.failed` | Downstream job failure or Authority timeout mid-request | Inspect Concelier job logs, re-run with tracing enabled, validate Authority latency. |
|
||||
|
||||
## 6. References
|
||||
|
||||
- `docs/21_INSTALL_GUIDE.md` – Authority configuration quick start.
|
||||
- `docs/17_SECURITY_HARDENING_GUIDE.md` – Security guardrails and enforcement deadlines.
|
||||
- `docs/modules/authority/operations/monitoring.md` – Authority-side monitoring and alerting playbook.
|
||||
- `StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields.
|
||||
@@ -96,7 +96,7 @@ curl -s -b cookies.txt \
|
||||
|
||||
Iterate `page` until the response `content` array is empty. Pages 0–9 currently cover 2014→present. Persist JSON responses (plus SHA256) for Offline Kit parity.
|
||||
|
||||
> **Shortcut** – run `python tools/certbund_offline_snapshot.py --output seed-data/cert-bund`
|
||||
> **Shortcut** – run `python src/Tools/certbund_offline_snapshot.py --output seed-data/cert-bund`
|
||||
> to bootstrap the session, capture the paginated search responses, and regenerate
|
||||
> the manifest/checksum files automatically. Supply `--cookie-file` and `--xsrf-token`
|
||||
> if the portal requires a browser-derived session (see options via `--help`).
|
||||
@@ -104,7 +104,7 @@ Iterate `page` until the response `content` array is empty. Pages 0–9 currentl
|
||||
### 3.3 Export bundles
|
||||
|
||||
```bash
|
||||
python tools/certbund_offline_snapshot.py \
|
||||
python src/Tools/certbund_offline_snapshot.py \
|
||||
--output seed-data/cert-bund \
|
||||
--start-year 2014 \
|
||||
--end-year "$(date -u +%Y)"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user