Restructure solution layout by module
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
This commit is contained in:
root
112
docs/implplan/AGENTS.md
Normal file
112
docs/implplan/AGENTS.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# 1) What is StellaOps?
|
||||
|
||||
**StellaOps** an open, sovereign, modular container-security toolkit built for high-speed, offline operation, released under AGPL-3.0-or-later.
|
||||
|
||||
It follows an SBOM-first model—analyzing each container layer or ingesting existing CycloneDX/SPDX SBOMs, then enriching them with vulnerability, licence, secret-leak, and misconfiguration data to produce cryptographically signed reports.
|
||||
|
||||
Vulnerability detection maps OS and language dependencies to sources such as NVD, GHSA, OSV, ENISA.
|
||||
Secrets sweep flags exposed credentials or keys in files or environment variables.
|
||||
Licence audit identifies potential conflicts, especially copyleft obligations.
|
||||
Misconfiguration checks detect unsafe Dockerfile patterns (root user, latest tags, permissive modes).
|
||||
Provenance features include in-toto/SLSA attestations signed with cosign for supply-chain trust.
|
||||
|
||||
| Guiding principle | What it means for Concelier |
|
||||
|-------------------|---------------------------|
|
||||
| **SBOM-first ingest** | Prefer signed SBOMs or reproducible layer diffs before falling back to raw scraping; connectors treat source docs as provenance, never as mutable truth. |
|
||||
| **Deterministic outputs** | Same inputs yield identical canonical advisories and exported JSON/Trivy DB artefacts; merge hashes and export manifests are reproducible across machines. |
|
||||
| **Restart-time plug-ins only** | Connector/exporter plug-ins load at service start, keeping runtime sandboxing simple and avoiding hot-patch attack surface. |
|
||||
| **Sovereign/offline-first** | No mandatory outbound calls beyond allow-listed advisories; Offline Kit bundles Mongo snapshots and exporter artefacts for air-gapped installs. |
|
||||
| **Operational transparency** | Every stage logs structured events (fetch, parse, merge, export) with correlation IDs so parallel agents can debug without shared state. |
|
||||
|
||||
Performance: warm scans < 5 s, cold scans < 30 s on a 4 vCPU runner.
|
||||
Deployment: entirely SaaS-free, suitable for air-gapped or on-prem use through its Offline Kit.
|
||||
Policy: anonymous users → 33 scans/day; verified → 333 /day; nearing 90 % quota triggers throttling but never full blocks.
|
||||
|
||||
More documention is available ./docs/*.md files. Read `docs/README.md` to gather information about the available documentation. You could inquiry specific documents as your work requires it
|
||||
|
||||
---
|
||||
|
||||
# 3) Practices
|
||||
|
||||
## 3.1) Naming
|
||||
All modules are .NET projects based on .NET 10 (preview). Exclussion is the UI. It is based on Angular
|
||||
All modules are contained by one or more projects. Each project goes in its dedicated folder. Each project starts with StellaOps.<ModuleName>. In case it is common for for all StellaOps modules it is library or plugin and it is named StellaOps.<LibraryOrPlugin>.
|
||||
|
||||
## 3.2) Key technologies & integrations
|
||||
|
||||
- **Runtime**: .NET 10 (`net10.0`) preview SDK; C# latest preview features.
|
||||
- **Data**: MongoDB (canonical store and job/export state).
|
||||
- **Observability**: structured logs, counters, and (optional) OpenTelemetry traces.
|
||||
- **Ops posture**: offline‑first, allowlist for remote hosts, strict schema validation, gated LLM fallback (only where explicitly configured).
|
||||
|
||||
# 4) Modules
|
||||
StellaOps is contained by different modules installable via docker containers
|
||||
- Concelier. Responsible for aggregation and delivery of vulnerability database
|
||||
- Cli. Command line tool to unlock full potential - request database operations, install scanner, request scan, configure backend
|
||||
- Backend. Configures and Manages scans
|
||||
- UI. UI to access the backend (and scanners)
|
||||
- Agent. Installable daemon that does the scanning
|
||||
- Zastava. Realtime monitor for allowed (verified) installations.
|
||||
|
||||
For more information of the architecture see `./docs/*ARCHITECTURE*.md` files.
|
||||
|
||||
---
|
||||
|
||||
### 4.1.4) Glossary (quick)
|
||||
|
||||
- **OVAL** — Vendor/distro security definition format; authoritative for OS packages.
|
||||
- **NEVRA / EVR** — RPM and Debian version semantics for OS packages.
|
||||
- **PURL / SemVer** — Coordinates and version semantics for OSS ecosystems.
|
||||
- **KEV** — Known Exploited Vulnerabilities (flag only).
|
||||
|
||||
---
|
||||
# 5) Your role as StellaOps contributor
|
||||
|
||||
You acting as information technology engineer that will take different type of roles in goal achieving StellaOps production implementation
|
||||
In order you to work - you have to be supplied with directory that contains `AGENTS.md`,`TASKS.md` files. There will you have more information about the role you have, the scope of your work and the tasks you will have.
|
||||
|
||||
Boundaries:
|
||||
- You operate only in the working directories I gave you, unless there is dependencies that makes you to work on dependency in shared directory. Then you ask for confirmation.
|
||||
|
||||
You main characteristics:
|
||||
- Keep endpoints small, deterministic, and cancellation-aware.
|
||||
- Improve logs/metrics as per tasks.
|
||||
- Update `TASKS.md` when moving tasks forward.
|
||||
- When you are done with all task you state explicitly you are done.
|
||||
- Impersonate the role described on working directory `AGENTS.md` you will read, if role is not available - take role of the CTO of the StellaOps in early stages.
|
||||
- You always strive for best practices
|
||||
- You always strive for re-usability
|
||||
- When in doubt of design decision - you ask then act
|
||||
- You are autonomus - meaning that you will work for long time alone and achieve maximum without stopping for stupid questions
|
||||
- You operate on the same directory where other agents will work. In case you need to work on directory that is dependency on provided `AGENTS.md`,`TASKS.md` files you have to ask for confirmation first.
|
||||
|
||||
## 5.1) Type of contributions
|
||||
|
||||
- **BE‑Base (Platform & Pipeline)**
|
||||
Owns DI, plugin host, job scheduler/coordinator, configuration binding, minimal API endpoints, and Mongo bootstrapping.
|
||||
- **BE‑Conn‑X (Connectors)**
|
||||
One agent per source family (NVD, Red Hat, Ubuntu, Debian, SUSE, GHSA, OSV, PSIRTs, CERTs, KEV, ICS). Implements fetch/parse/map with incremental watermarks.
|
||||
- **BE‑Merge (Canonical Merge & Dedupe)**
|
||||
Identity graph, precedence policies, canonical JSON serializer, and deterministic hashing (`merge_event`).
|
||||
- **BE‑Export (JSON & Trivy DB)**
|
||||
Deterministic export trees, Trivy DB packaging, optional ORAS push, and offline bundle.
|
||||
- **QA (Validation & Observability)**
|
||||
Schema tests, fixture goldens, determinism checks, metrics/logs/traces, e2e reproducibility runs.
|
||||
- **DevEx/Docs**
|
||||
Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews.
|
||||
|
||||
|
||||
## 5.2) Work rules (important)
|
||||
|
||||
- **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description.
|
||||
- **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting.
|
||||
- **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests.
|
||||
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED). Before starting of actual work - ensure you have set the task to DOING. When complete or stop update the status in corresponding TASKS.md and in ./SPRINTS.md file.
|
||||
- **Coordination**: In case task is discovered as blocked on other team or task, according TASKS.md files that dependency is on needs to be changed by adding new tasks describing the requirement. the current task must be updated as completed. In case task changes, scope or requirements or rules - other documentations needs be updated accordingly.
|
||||
- **Sprint synchronization**: When given task seek for relevant directory to work on from SPRINTS.md. Confirm its state on both SPRINTS.md and the relevant TASKS.md file. Always check the AGENTS.md in the relevant TASKS.md directory.
|
||||
- **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence.
|
||||
- **Test layout**: Use module-specific projects in `StellaOps.Concelier.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Concelier.Testing`.
|
||||
- **Execution autonomous**: In case you need to continue with more than one options just continue sequentially, unless the continue requires design decision.
|
||||
- **EPIC reference**: In case task (on relevant TASKS.md file) references in direct or indirect way an EPIC - then seek for epic document like ./EPIC_*.md. There will be more information how to implement the task.
|
||||
|
||||
---
|
||||
524
docs/implplan/EPIC_1.md
Normal file
524
docs/implplan/EPIC_1.md
Normal file
@@ -0,0 +1,524 @@
|
||||
Here’s the full write‑up you can drop into your repo as the canonical reference for Epic 1. It’s written in clean product‑doc style so it’s safe to check in as Markdown. No fluff, just everything you need to build, test, and police it.
|
||||
|
||||
---
|
||||
|
||||
# Epic 1: Aggregation‑Only Contract (AOC) Enforcement
|
||||
|
||||
> Short name: **AOC enforcement**
|
||||
> Services touched: **Conseiller (advisory ingestion), Excitator (VEX ingestion), Web API, Workers, Policy Engine, CLI, Console, Authority**
|
||||
> Data stores: **MongoDB primary, optional Redis/NATS for jobs**
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
**Aggregation‑Only Contract (AOC)** is the ingestion covenant for StellaOps. It defines a hard boundary between **collection** and **interpretation**:
|
||||
|
||||
* **Ingestion (Conseiller/Excitator)** only **collects** data and preserves it as immutable raw facts with provenance. It does not decide, merge, normalize, prioritize, or assign severity. It may compute **links** that help future joins (aliases, PURLs, CPEs), but never derived judgments.
|
||||
* **Policy evaluation** is the only place where merges, deduplication, consensus, severity computation, and status folding are allowed. It’s reproducible and traceable.
|
||||
|
||||
The AOC establishes:
|
||||
|
||||
* **Immutable raw stores**: `advisory_raw` and `vex_raw` documents with full provenance, signatures, checksums, and upstream identifiers.
|
||||
* **Linksets**: machine‑generated join hints (aliases, PURLs, CPEs, CVE/GHSA IDs) that never change the underlying source content.
|
||||
* **Invariants**: a strict set of “never do this in ingestion” rules enforced by schema validation, runtime guards, and CI checks.
|
||||
* **AOC Verifier**: a build‑time and runtime watchdog that blocks non‑compliant code and data writes.
|
||||
|
||||
This epic delivers: schemas, guards, error codes, APIs, tests, migration, docs, and ops dashboards to make AOC non‑negotiable across the platform.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
AOC makes results **auditable, deterministic, and organization‑specific**. Source vendors disagree; your policies decide. By removing hidden heuristics from ingestion, we avoid unexplainable risk changes, race conditions between collectors, and vendor bias. Policy‑time evaluation yields reproducible deltas with complete “why” traces.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (deep details)
|
||||
|
||||
### 3.1 Core invariants
|
||||
|
||||
The following must be true for every write to `advisory_raw` and `vex_raw` and for every ingestion pipeline:
|
||||
|
||||
1. **No severity in ingestion**
|
||||
|
||||
* Forbidden fields: `severity`, `cvss`, `cvss_vector`, `effective_status`, `effective_range`, `merged_from`, `consensus_provider`, `reachability`, `asset_criticality`, `risk_score`.
|
||||
2. **No merges or de‑dups in ingestion**
|
||||
|
||||
* No combining two upstream advisories into one. No picking a single truth when multiple VEX statements exist.
|
||||
3. **Provenance is mandatory**
|
||||
|
||||
* Every raw doc includes `provenance` and `signature/checksum`.
|
||||
4. **Idempotent upserts**
|
||||
|
||||
* Same upstream document (by `upstream_id` + `source` + `content_hash`) must not create duplicates.
|
||||
5. **Append‑only versioning**
|
||||
|
||||
* Revisions from the source create new immutable documents with `supersedes` pointers; no in‑place edits.
|
||||
6. **Linkset only**
|
||||
|
||||
* Ingestion can compute and store a `linkset` for join performance. Linkset does not alter or infer severity/status.
|
||||
7. **Policy‑time only for effective findings**
|
||||
|
||||
* Only the Policy Engine can write `effective_finding_*` materializations.
|
||||
8. **Schema safety**
|
||||
|
||||
* Strict JSON schema validation at DB level; unknown fields reject writes.
|
||||
9. **Clock discipline**
|
||||
|
||||
* Timestamps are UTC, monotonic within a batch; collectors record `fetched_at` and `received_at`.
|
||||
|
||||
### 3.2 Data model
|
||||
|
||||
#### 3.2.1 `advisory_raw` (Mongo collection)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "advisory_raw:osv:GHSA-xxxx-....:v3",
|
||||
"source": {
|
||||
"vendor": "OSV",
|
||||
"stream": "github",
|
||||
"api": "https://api.osv.dev/v1/.../GHSA-...",
|
||||
"collector_version": "conseiller/1.7.3"
|
||||
},
|
||||
"upstream": {
|
||||
"upstream_id": "GHSA-xxxx-....",
|
||||
"document_version": "2024-09-01T12:13:14Z",
|
||||
"fetched_at": "2025-01-02T03:04:05Z",
|
||||
"received_at": "2025-01-02T03:04:06Z",
|
||||
"content_hash": "sha256:...",
|
||||
"signature": {
|
||||
"present": true,
|
||||
"format": "dsse",
|
||||
"key_id": "rekor:.../key/abc",
|
||||
"sig": "base64..."
|
||||
}
|
||||
},
|
||||
"content": {
|
||||
"format": "OSV",
|
||||
"spec_version": "1.6",
|
||||
"raw": { /* full upstream JSON, unmodified */ }
|
||||
},
|
||||
"identifiers": {
|
||||
"cve": ["CVE-2023-1234"],
|
||||
"ghsa": ["GHSA-xxxx-...."],
|
||||
"aliases": ["CVE-2023-1234", "GHSA-xxxx-...."]
|
||||
},
|
||||
"linkset": {
|
||||
"purls": ["pkg:npm/lodash@4.17.21", "pkg:maven/..."],
|
||||
"cpes": ["cpe:2.3:a:..."],
|
||||
"references": [
|
||||
{"type":"advisory","url":"https://..."},
|
||||
{"type":"fix","url":"https://..."}
|
||||
],
|
||||
"reconciled_from": ["content.raw.affected.ranges", "content.raw.pkg"]
|
||||
},
|
||||
"supersedes": "advisory_raw:osv:GHSA-xxxx-....:v2",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
> Note: No `severity`, no `cvss`, no `effective_*`. If the upstream payload includes CVSS, it stays inside `content.raw` and is not promoted or normalized at ingestion.
|
||||
|
||||
#### 3.2.2 `vex_raw` (Mongo collection)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "vex_raw:vendorX:doc-123:v4",
|
||||
"source": {
|
||||
"vendor": "VendorX",
|
||||
"stream": "vex",
|
||||
"api": "https://.../vex/doc-123",
|
||||
"collector_version": "excitator/0.9.2"
|
||||
},
|
||||
"upstream": {
|
||||
"upstream_id": "doc-123",
|
||||
"document_version": "2025-01-15T08:09:10Z",
|
||||
"fetched_at": "2025-01-16T01:02:03Z",
|
||||
"received_at": "2025-01-16T01:02:03Z",
|
||||
"content_hash": "sha256:...",
|
||||
"signature": { "present": true, "format": "cms", "key_id": "kid:...", "sig": "..." }
|
||||
},
|
||||
"content": {
|
||||
"format": "CycloneDX-VEX", // or "CSAF-VEX"
|
||||
"spec_version": "1.5",
|
||||
"raw": { /* full upstream VEX */ }
|
||||
},
|
||||
"identifiers": {
|
||||
"statements": [
|
||||
{
|
||||
"advisory_ids": ["CVE-2023-1234","GHSA-..."],
|
||||
"component_purls": ["pkg:deb/openssl@1.1.1"],
|
||||
"status": "not_affected",
|
||||
"justification": "component_not_present"
|
||||
}
|
||||
]
|
||||
},
|
||||
"linkset": {
|
||||
"purls": ["pkg:deb/openssl@1.1.1"],
|
||||
"cves": ["CVE-2023-1234"],
|
||||
"ghsas": ["GHSA-..."]
|
||||
},
|
||||
"supersedes": "vex_raw:vendorX:doc-123:v3",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
> VEX statuses remain as raw facts. No cross‑provider consensus is computed here.
|
||||
|
||||
### 3.3 Database validation
|
||||
|
||||
* MongoDB JSON Schema validators on both collections:
|
||||
|
||||
* Reject forbidden fields at the top level.
|
||||
* Enforce presence of `source`, `upstream`, `content`, `linkset`, `tenant`.
|
||||
* Enforce string formats for timestamps and hashes.
|
||||
|
||||
### 3.4 Write paths
|
||||
|
||||
1. **Collector fetches upstream**
|
||||
|
||||
* Normalize transport (gzip/json), compute `content_hash`, verify signature if available.
|
||||
2. **Build raw doc**
|
||||
|
||||
* Populate `source`, `upstream`, `content.raw`, `identifiers`, `linkset`.
|
||||
3. **Idempotent upsert**
|
||||
|
||||
* Lookup by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. If exists, skip; if new content hash, insert new revision with `supersedes`.
|
||||
4. **AOC guard**
|
||||
|
||||
* Runtime interceptor inspects write payload; if any forbidden field detected, reject with `ERR_AOC_001`.
|
||||
5. **Metrics**
|
||||
|
||||
* Emit `ingestion_write_ok` or `ingestion_write_reject` with reason code.
|
||||
|
||||
### 3.5 Read paths (ingestion scope)
|
||||
|
||||
* Allow only listing, getting raw docs, and searching by linkset. No endpoints return “effective findings” from ingestion services.
|
||||
|
||||
### 3.6 Error codes
|
||||
|
||||
| Code | Meaning | HTTP |
|
||||
| ------------- | ------------------------------------------------------------ | ---- |
|
||||
| `ERR_AOC_001` | Forbidden field present (severity/consensus/normalized data) | 400 |
|
||||
| `ERR_AOC_002` | Merge attempt detected (multiple upstreams fused) | 400 |
|
||||
| `ERR_AOC_003` | Idempotency violation (duplicate without supersedes) | 409 |
|
||||
| `ERR_AOC_004` | Missing provenance fields | 422 |
|
||||
| `ERR_AOC_005` | Signature/checksum mismatch | 422 |
|
||||
| `ERR_AOC_006` | Attempt to write effective findings from ingestion context | 403 |
|
||||
| `ERR_AOC_007` | Unknown top‑level fields (schema violation) | 400 |
|
||||
|
||||
### 3.7 AOC Verifier
|
||||
|
||||
A build‑time and runtime safeguard:
|
||||
|
||||
* **Static checks (CI)**
|
||||
|
||||
* Block imports of `*.Policy*` or `*.Merge*` from ingestion modules.
|
||||
* AST lint rule: any write to `advisory_raw` or `vex_raw` setting a forbidden key fails the build.
|
||||
* **Runtime checks**
|
||||
|
||||
* Repository layer interceptor inspects documents before insert/update; rejects forbidden fields and multi‑source merges.
|
||||
* **Drift detection job**
|
||||
|
||||
* Nightly job scans newest N docs; if violation found, pages ops and blocks new pipeline runs.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.8 Indexing strategy
|
||||
|
||||
* `advisory_raw`:
|
||||
|
||||
* `{ "identifiers.cve": 1 }`, `{ "identifiers.ghsa": 1 }`, `{ "linkset.purls": 1 }`, `{ "source.vendor": 1, "upstream.upstream_id": 1, "upstream.content_hash": 1 }` (unique), `{ "tenant": 1 }`.
|
||||
* `vex_raw`:
|
||||
|
||||
* `{ "identifiers.statements.advisory_ids": 1 }`, `{ "linkset.purls": 1 }`, `{ "source.vendor": 1, "upstream.upstream_id": 1, "upstream.content_hash": 1 }` (unique), `{ "tenant": 1 }`.
|
||||
|
||||
### 3.9 Interaction with Policy Engine
|
||||
|
||||
* Policy Engine pulls raw docs by identifiers/linksets and computes:
|
||||
|
||||
* De‑dup/merge per policy
|
||||
* Consensus for VEX statements
|
||||
* Severity normalization and risk scoring
|
||||
* Writes **only** to `effective_finding_{policyId}` collections.
|
||||
|
||||
A dedicated write guard refuses `effective_finding_*` writes from any caller that isn’t the Policy Engine service identity.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.10 Migration plan
|
||||
|
||||
1. **Freeze ingestion writes** except raw pass‑through.
|
||||
2. **Backfill**: copy existing ingestion collections to `_backup_*`.
|
||||
3. **Strip forbidden fields** from raw copies, move them into a temporary `advisory_view_legacy` used only by Policy Engine for parity.
|
||||
4. **Enable DB schema validators**.
|
||||
5. **Run collectors** in dry‑run; ensure only allowed keys land.
|
||||
6. **Switch Policy Engine** to pull exclusively from `*_raw` and to compute everything else.
|
||||
7. **Delete legacy normalized fields** in ingestion codepaths.
|
||||
8. **Enable runtime guards** and CI lint.
|
||||
|
||||
### 3.11 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `aoc_violation_total{code=...}`, `ingestion_write_total{result=ok|reject}`, `ingestion_signature_verified_total{result=ok|fail}`, `ingestion_latency_seconds`, `advisory_revision_count`.
|
||||
* Tracing: span `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`.
|
||||
* Logs: include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, `correlation_id`.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.12 Security and tenancy
|
||||
|
||||
* Every raw doc carries a `tenant` field.
|
||||
* Authority enforces `advisory:ingest` and `vex:ingest` scopes for ingestion endpoints.
|
||||
* Cross‑tenant reads/writes are blocked by default.
|
||||
* Secrets never logged; signatures verified with pinned trust stores.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.13 CLI and Console behavior
|
||||
|
||||
* **CLI**
|
||||
|
||||
* `stella sources ingest --dry-run` prints would‑write payload and explicitly shows that no severity/status fields are present.
|
||||
* `stella aoc verify` scans last K documents and reports violations with exit codes.
|
||||
* **Console**
|
||||
|
||||
* Sources dashboard shows AOC pass/fail per job, most recent violation codes, and a drill‑down to the offending document.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 4) API surface (ingestion scope)
|
||||
|
||||
### 4.1 Conseiller (Advisories)
|
||||
|
||||
* `POST /ingest/advisory`
|
||||
|
||||
* Body: raw upstream advisory with metadata; server constructs document, not the client.
|
||||
* Rejections: `ERR_AOC_00x` per table above.
|
||||
* `GET /advisories/raw/{id}`
|
||||
* `GET /advisories/raw?cve=CVE-...&purl=pkg:...&tenant=...`
|
||||
* `GET /advisories/raw/{id}/provenance`
|
||||
* `POST /aoc/verify?since=ISO8601` returns summary stats and first N violations.
|
||||
|
||||
### 4.2 Excitator (VEX)
|
||||
|
||||
* `POST /ingest/vex`
|
||||
* `GET /vex/raw/{id}`
|
||||
* `GET /vex/raw?advisory_id=CVE-...&purl=pkg:...`
|
||||
* `POST /aoc/verify?since=ISO8601`
|
||||
|
||||
All endpoints require `tenant` scope and appropriate `:write` or `:read`.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 5) Example: end‑to‑end flow
|
||||
|
||||
1. Collector fetches `GHSA-1234` from OSV.
|
||||
2. Build `advisory_raw` with linkset PURLs.
|
||||
3. Insert; AOC guard approves.
|
||||
4. Policy Engine later evaluates SBOM `S-42` under `policy P-7`, reads raw advisory and any VEX raw docs, computes effective findings, and writes to `effective_finding_P-7`.
|
||||
5. CLI `stella aoc verify --since 24h` returns `0` violations.
|
||||
|
||||
---
|
||||
|
||||
## 6) Implementation tasks
|
||||
|
||||
Breakdown by component with exact work items. Each section ends with the imposed sentence you requested.
|
||||
|
||||
### 6.1 Conseiller (advisory ingestion, WS + Worker)
|
||||
|
||||
* [ ] Add Mongo JSON schema validation for `advisory_raw`.
|
||||
* [ ] Implement repository layer with **write interceptors** that reject forbidden fields.
|
||||
* [ ] Compute `linkset` from upstream using deterministic mappers.
|
||||
* [ ] Enforce idempotency by unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)`.
|
||||
* [ ] Remove any normalization pipelines; relocate to Policy Engine.
|
||||
* [ ] Add `POST /ingest/advisory` and `GET /advisories/raw*` endpoints with Authority scope checks.
|
||||
* [ ] Emit observability metrics and traces.
|
||||
* [ ] Unit tests: schema violations, idempotency, supersedes chain, forbidden fields.
|
||||
* [ ] Integration tests: large batch ingest, linkset correctness against golden fixtures.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.2 Excitator (VEX ingestion, WS + Worker)
|
||||
|
||||
* [ ] Add Mongo JSON schema validation for `vex_raw`.
|
||||
* [ ] Implement repository layer guard identical to Conseiller.
|
||||
* [ ] Deterministic `linkset` extraction for advisory IDs and PURLs.
|
||||
* [ ] Endpoints `POST /ingest/vex`, `GET /vex/raw*` with scopes.
|
||||
* [ ] Remove any consensus or folding logic; leave VEX statements as raw.
|
||||
* [ ] Tests as per Conseiller, with rich fixtures for CycloneDX‑VEX and CSAF.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.3 Web API shared library
|
||||
|
||||
* [ ] Define `AOCForbiddenKeys` and export for both services.
|
||||
* [ ] Provide `AOCWriteGuard` middleware and `AOCError` types.
|
||||
* [ ] Provide `ProvenanceBuilder` utility.
|
||||
* [ ] Provide `SignatureVerifier` and `Checksum` helpers.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.4 Policy Engine
|
||||
|
||||
* [ ] Block any import/use from ingestion modules by lint rule.
|
||||
* [ ] Add hard gate on `effective_finding_*` writes that verifies caller identity is Policy Engine.
|
||||
* [ ] Update readers to pull fields only from `content.raw`, `identifiers`, `linkset`, not any legacy normalized fields.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.5 Authority
|
||||
|
||||
* [ ] Introduce scopes: `advisory:ingest`, `advisory:read`, `vex:ingest`, `vex:read`, `aoc:verify`.
|
||||
* [ ] Add `tenant` claim propagation to ingestion services.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.6 CLI
|
||||
|
||||
* [ ] `stella sources ingest --dry-run` and `stella aoc verify` commands.
|
||||
* [ ] Exit codes mapping to `ERR_AOC_00x`.
|
||||
* [ ] JSON output schema including violation list.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.7 Console
|
||||
|
||||
* [ ] Sources dashboard tiles: last run, AOC violations, top error codes.
|
||||
* [ ] Drill‑down page rendering offending doc with highlight on forbidden keys.
|
||||
* [ ] “Verify last 24h” action calling the AOC Verifier endpoint.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.8 CI/CD
|
||||
|
||||
* [ ] AST linter to forbid writes of banned keys in ingestion modules.
|
||||
* [ ] Unit test coverage gates for AOC guard code.
|
||||
* [ ] Pipeline stage that runs `stella aoc verify` against seeded DB snapshots.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Documentation changes (create/update these files)
|
||||
|
||||
1. **`/docs/ingestion/aggregation-only-contract.md`**
|
||||
|
||||
* Add: philosophy, invariants, schemas for `advisory_raw`/`vex_raw`, error codes, linkset definition, examples, idempotency rules, supersedes, API references, migration steps, observability, security.
|
||||
2. **`/docs/architecture/overview.md`**
|
||||
|
||||
* Update system diagram to show AOC boundary and raw stores; add sequence diagram: fetch → guard → raw insert → policy evaluation.
|
||||
3. **`/docs/architecture/policy-engine.md`**
|
||||
|
||||
* Clarify ingestion boundary; list inputs consumed from raw; note that any severity/consensus is policy‑time only.
|
||||
4. **`/docs/ui/console.md`**
|
||||
|
||||
* Add Sources dashboard section: AOC tiles and violation drill‑down.
|
||||
5. **`/docs/cli/cli-reference.md`**
|
||||
|
||||
* Add `stella aoc verify` and `stella sources ingest --dry-run` usage and exit codes.
|
||||
6. **`/docs/observability/observability.md`**
|
||||
|
||||
* Document new metrics, traces, logs keys for AOC.
|
||||
7. **`/docs/security/authority-scopes.md`**
|
||||
|
||||
* Add new scopes and tenancy enforcement for ingestion endpoints.
|
||||
8. **`/docs/deploy/containers.md`**
|
||||
|
||||
* Note DB validators must be enabled; environment flags for AOC guards; read‑only user for verify endpoint.
|
||||
|
||||
Each file should include a “Compliance checklist” subsection for AOC.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 8) Acceptance criteria
|
||||
|
||||
* DB validators are active and reject writes with forbidden fields.
|
||||
* AOC runtime guards log and reject violations with correct error codes.
|
||||
* CI linter prevents shipping code that writes forbidden keys to raw stores.
|
||||
* Ingestion of known fixture sets results in zero normalized fields outside `content.raw`.
|
||||
* Policy Engine is the only writer of `effective_finding_*` materializations.
|
||||
* CLI `stella aoc verify` returns success on clean datasets and non‑zero on seeded violations.
|
||||
* Console shows AOC status and violation drill‑downs.
|
||||
|
||||
---
|
||||
|
||||
## 9) Risks and mitigations
|
||||
|
||||
* **Collector drift**: new upstream fields tempt developers to normalize.
|
||||
|
||||
* Mitigation: CI linter + guard + schema validators; require RFC to extend linkset.
|
||||
* **Performance impact**: extra validation on write.
|
||||
|
||||
* Mitigation: guard is O(number of keys) and schema check is bounded; indexes sized appropriately.
|
||||
* **Migration complexity**: moving legacy normalized fields out.
|
||||
|
||||
* Mitigation: temporary `advisory_view_legacy` for parity; stepwise cutover.
|
||||
* **Tenant leakage**: missing tenant on write.
|
||||
|
||||
* Mitigation: schema requires `tenant`; middleware injects and asserts.
|
||||
|
||||
---
|
||||
|
||||
## 10) Test plan
|
||||
|
||||
* **Unit tests**
|
||||
|
||||
* Guard rejects forbidden keys; idempotency; supersedes chain; provenance required.
|
||||
* Signature verification paths: good, bad, absent.
|
||||
* **Property tests**
|
||||
|
||||
* Randomized upstream docs never produce forbidden keys at top level.
|
||||
* **Integration tests**
|
||||
|
||||
* Batch ingest of 50k advisories: throughput, zero violations.
|
||||
* Mixed VEX sources with contradictory statements remain separate in raw.
|
||||
* **Contract tests**
|
||||
|
||||
* Policy Engine refuses to run without raw inputs; writes only to `effective_finding_*`.
|
||||
* **End‑to‑end**
|
||||
|
||||
* Seed SBOM + advisories + VEX; ensure findings are identical pre/post migration.
|
||||
|
||||
---
|
||||
|
||||
## 11) Developer checklists
|
||||
|
||||
**Definition of Ready**
|
||||
|
||||
* Upstream spec reference attached.
|
||||
* Linkset mappers defined.
|
||||
* Example fixtures added.
|
||||
|
||||
**Definition of Done**
|
||||
|
||||
* DB validators deployed and tested.
|
||||
* Runtime guards enabled.
|
||||
* CI linter merged and enforced.
|
||||
* Docs updated (files in section 7).
|
||||
* Metrics visible on dashboard.
|
||||
* CLI verify passes.
|
||||
|
||||
---
|
||||
|
||||
## 12) Glossary
|
||||
|
||||
* **Raw document**: exact upstream content plus provenance, with join hints.
|
||||
* **Linkset**: PURLs/CPEs/IDs extracted to accelerate joins later.
|
||||
* **Supersedes**: pointer from a newer raw doc to the previous revision of the same upstream doc.
|
||||
* **Policy‑time**: evaluation phase where merges, consensus, and severity are computed.
|
||||
* **AOC**: Aggregation‑Only Contract.
|
||||
|
||||
---
|
||||
|
||||
### Final imposed reminder
|
||||
|
||||
**Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
566
docs/implplan/EPIC_10.md
Normal file
566
docs/implplan/EPIC_10.md
Normal file
@@ -0,0 +1,566 @@
|
||||
Fine. Here’s your next brick of “maximum documentation.” Try not to drop it on your foot.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 10: Export Center (JSON, Trivy DB, Mirror bundles)
|
||||
|
||||
**Short name:** `Export Center`
|
||||
**Primary service:** `exporter`
|
||||
**Surfaces:** Console (Web UI), CLI, Web API
|
||||
**Touches:** Conseiller (Feedser), Excitator (Vexer), SBOM Service, Policy Engine, VEX Consensus Lens, Findings Ledger, Authority (authN/Z), Object Storage, Orchestrator, Signing/Attestation, Telemetry
|
||||
|
||||
**AOC ground rule:** Conseiller and Excitator aggregate but never merge. The Export Center serializes evidence and policy results; it does not rewrite or “improve” your data in-flight.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The Export Center is the unified system for packaging StellaOps data into portable, verifiable bundles:
|
||||
|
||||
* **Canonical JSON exports** for advisories, VEX, SBOMs, findings, and policy-evaluation snapshots.
|
||||
* **Trivy DB compatible bundles** so downstream scanners can use Stella’s curated vulnerability knowledge without custom integrations.
|
||||
* **Mirror bundles** for air‑gapped or disconnected environments containing raw evidence, normalized records, indexes, and policy snapshots, with provenance, signatures, and optional encryption.
|
||||
|
||||
It centralizes format adapters, compliance and provenance, scheduling, versioning, and distribution (download, OCI push, file share). Every export is reproducible by `run_id`, cryptographically signed, and audit‑traceable back to source artifacts.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (brief)
|
||||
|
||||
Teams need to move results into other scanners, CI systems, and isolated networks without babysitting ten different scripts. The Export Center gives a single, policy‑aware, verifiable exit point that doesn’t surprise compliance or set your ops team on fire.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Capabilities
|
||||
|
||||
* **Profiles**
|
||||
|
||||
* `json:raw` canonical JSON lines for advisories, VEX, SBOMs.
|
||||
* `json:policy` adds policy evaluation results (allow/deny/risk, rationales).
|
||||
* `trivy:db` Trivy DB‑compatible export (core vulnerability DB).
|
||||
* `trivy:java-db` optional Java ecosystem DB export if enabled.
|
||||
* `mirror:full` air‑gap bundle with raw + normalized + indexes + policy + VEX consensus + SBOMs.
|
||||
* `mirror:delta` incremental bundle based on a previous export manifest.
|
||||
|
||||
* **Scope selectors**
|
||||
|
||||
* By time window, product, ecosystem, package, image digest, repository, tenant, tag.
|
||||
* Include/exclude SBOMs, advisories, VEX, findings, policy snapshots.
|
||||
|
||||
* **Policy awareness**
|
||||
|
||||
* Optionally bake a **policy snapshot** into the bundle, including the policy version, inputs and decision traces.
|
||||
* Can export **raw** evidence only (AOC) or **raw + evaluated**.
|
||||
|
||||
* **Provenance & signing**
|
||||
|
||||
* Generate attestation metadata with source URIs, artifact hashes, schema versions, export profile and filters.
|
||||
* Sign manifests and bundle using configured KMS; support detached and in‑bundle signatures.
|
||||
|
||||
* **Distribution**
|
||||
|
||||
* Download via Console or API (streaming).
|
||||
* Push to OCI registry as an artifact/image with annotations.
|
||||
* Write to object storage prefix for batch pickup.
|
||||
|
||||
* **Scheduling & automation**
|
||||
|
||||
* One‑off, cron, and event‑triggered exports (e.g., after a “VEX consensus recompute” run).
|
||||
* Retention policies and automatic expiry for old bundles.
|
||||
|
||||
* **Observability**
|
||||
|
||||
* Export metrics, throughput, size, downstream pull counts (when pushed to registry with report backs).
|
||||
|
||||
### 3.2 Architecture
|
||||
|
||||
* **exporter (service)**
|
||||
|
||||
* Orchestrates export runs, gathers records from the ledger and indexes, calls format adapters, writes bundles, signs, and publishes distribution tasks.
|
||||
* Stateless workers pull “export.jobs” from the orchestrator, stream data, and write manifests into object storage.
|
||||
|
||||
* **format adapters**
|
||||
|
||||
* Pluggable adapters:
|
||||
|
||||
* `adapter-json`: canonicalized JSONL writers per record type.
|
||||
* `adapter-trivy`: translates Stella’s normalized advisory model into Trivy DB format (and Java DB if enabled).
|
||||
* `adapter-mirror`: constructs a portable filesystem/OCI layout with manifests, indexes, and data subtrees.
|
||||
|
||||
* **manifesting & provenance**
|
||||
|
||||
* `export.json` (export manifest): profile, filters, counts, schema versions, content checksums, start/finish times, inputs list (artifact ids + hashes).
|
||||
* `provenance.json`: full chain back to source runs and artifacts; linked signatures.
|
||||
|
||||
* **distribution engines**
|
||||
|
||||
* `dist-http` streaming for downloads.
|
||||
* `dist-oci` layer writer with descriptors and annotations.
|
||||
* `dist-objstore` for staging to buckets.
|
||||
|
||||
* **security**
|
||||
|
||||
* Tenant scoping, RBAC on export creation and retrieval.
|
||||
* Optional in‑bundle encryption (age/AES‑GCM) with key wrapping using KMS.
|
||||
|
||||
### 3.3 Data model (selected tables)
|
||||
|
||||
* `export_profiles`
|
||||
|
||||
* `id`, `name`, `kind` (`json|trivy|mirror`), `variant` (`raw|policy|db|java-db|full|delta`), `config_json`, `created_by`, `created_at`.
|
||||
|
||||
* `export_runs`
|
||||
|
||||
* `id`, `profile_id`, `trigger` (`manual|schedule|event|api`), `state`, `filters_json`, `policy_version`, `started_at`, `finished_at`, `artifact_uri`, `size_bytes`, `sig_uri`, `provenance_uri`, `tenant_id`, `error_class`, `error_message`.
|
||||
|
||||
* `export_inputs`
|
||||
|
||||
* Link table between `export_runs` and source artifacts. `export_run_id`, `artifact_id`, `hash`.
|
||||
|
||||
* `export_distributions`
|
||||
|
||||
* `export_run_id`, `type` (`download|oci|objstore`), `target`, `state`, `meta_json`, `created_at`, `updated_at`.
|
||||
|
||||
### 3.4 Canonical file layouts
|
||||
|
||||
**JSON profile output**
|
||||
Directory layout under export root:
|
||||
|
||||
```
|
||||
/export/
|
||||
export.json # export manifest
|
||||
provenance.json # provenance and source artifact chain
|
||||
signatures/
|
||||
export.sig # detached signature for export.json
|
||||
advisories/
|
||||
normalized.jsonl # normalized advisory records
|
||||
vex/
|
||||
normalized.jsonl # normalized VEX records
|
||||
sboms/
|
||||
<subject>/sbom.spdx.json # one per subject; SPDX JSON or CycloneDX JSON
|
||||
findings/
|
||||
policy_evaluated.jsonl # if profile=json:policy
|
||||
```
|
||||
|
||||
**Trivy DB profile output**
|
||||
Produced as a compressed artifact:
|
||||
|
||||
```
|
||||
/export/
|
||||
export.json
|
||||
provenance.json
|
||||
trivy/
|
||||
db.bundle # Trivy DB compatible archive
|
||||
java-db.bundle # optional Java DB bundle (if enabled)
|
||||
signatures/
|
||||
trivy-db.sig
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
* The adapter keeps an internal mapping of Stella normalized fields to Trivy’s expected fields and namespaces. The mapping is versioned to track upstream schema evolution.
|
||||
|
||||
**Mirror bundle (filesystem layout)**
|
||||
|
||||
```
|
||||
/mirror/
|
||||
manifest.yaml # high-level bundle manifest (profile, filters, counts)
|
||||
export.json # same as JSON profile
|
||||
provenance.json
|
||||
indexes/
|
||||
advisories.index.json # quick lookups (pkg -> advisory ids)
|
||||
vex.index.json
|
||||
sbom.index.json
|
||||
advisories/raw/...
|
||||
advisories/normalized/...
|
||||
vex/raw/...
|
||||
vex/normalized/...
|
||||
sboms/raw/...
|
||||
sboms/graph/...
|
||||
policy/
|
||||
snapshot.yaml # full policy set used for evaluation
|
||||
evaluations.jsonl # decision outputs if requested
|
||||
consensus/
|
||||
vex_consensus.jsonl
|
||||
signatures/
|
||||
manifest.sig
|
||||
export.sig
|
||||
README.md
|
||||
```
|
||||
|
||||
**Mirror bundle (OCI layout)**
|
||||
Following standard OCI image artifact layout with annotations (`org.opencontainers.artifact.description`, `com.stella.export.profile`, `com.stella.export.filters`), and manifest lists for large bundles.
|
||||
|
||||
### 3.5 Export workflow
|
||||
|
||||
1. **Plan**
|
||||
Exporter computes candidates based on filters. For `mirror:delta`, compares with previous manifest to compute changes.
|
||||
|
||||
2. **Stream & write**
|
||||
Records are streamed from the Findings Ledger and stores. Writers are forward‑only, emitting JSONL or adapter‑specific structures, chunked for memory safety.
|
||||
|
||||
3. **Sign & attest**
|
||||
Once all content hashes are stable, Export Center writes `export.json`, `provenance.json`, and signs using KMS. Optional encryption wraps data layers.
|
||||
|
||||
4. **Distribute**
|
||||
Depending on profile settings, it exposes a download URL, pushes an OCI artifact, or writes to object storage. Distribution metadata is recorded.
|
||||
|
||||
5. **Audit & retention**
|
||||
Run, manifest, and signatures are immutable. Retention policy prunes large data folders after N days with manifests retained longer.
|
||||
|
||||
### 3.6 APIs
|
||||
|
||||
```
|
||||
POST /export/profiles
|
||||
GET /export/profiles?kind=&variant=
|
||||
GET /export/profiles/{id}
|
||||
PATCH /export/profiles/{id}
|
||||
DELETE /export/profiles/{id}
|
||||
|
||||
POST /export/runs
|
||||
GET /export/runs?state=&profile_id=&from=&to=&tenant_id=
|
||||
GET /export/runs/{run_id}
|
||||
POST /export/runs/{run_id}/cancel
|
||||
|
||||
GET /export/runs/{run_id}/download # presigned URL or streaming
|
||||
POST /export/runs/{run_id}/distribute # { "type":"oci|objstore", "target":"..." }
|
||||
GET /export/runs/{run_id}/manifest # export.json
|
||||
GET /export/runs/{run_id}/provenance
|
||||
GET /export/runs/{run_id}/signatures
|
||||
|
||||
GET /export/metrics/overview
|
||||
WS /export/streams/updates
|
||||
```
|
||||
|
||||
**Request example (create run):**
|
||||
|
||||
```json
|
||||
{
|
||||
"profile_id": "prof_json_policy",
|
||||
"filters": {
|
||||
"time_from": "2025-01-01T00:00:00Z",
|
||||
"time_to": "2025-01-31T23:59:59Z",
|
||||
"ecosystems": ["pypi", "npm"],
|
||||
"include": ["advisories", "vex", "sboms", "findings"]
|
||||
},
|
||||
"distribution": { "type": "download" },
|
||||
"policy_version": "pol-v1.8.2"
|
||||
}
|
||||
```
|
||||
|
||||
### 3.7 CLI
|
||||
|
||||
```
|
||||
stella export profiles list --kind mirror
|
||||
stella export profiles create --file profile.yaml
|
||||
stella export run create --profile prof_json_policy --from 2025-01-01 --to 2025-01-31 --include advisories,vex,sboms --download
|
||||
stella export run status <run-id>
|
||||
stella export run cancel <run-id>
|
||||
stella export run get <run-id> --manifest
|
||||
stella export run download <run-id> --out export-jan.tar.zst
|
||||
stella export distribute <run-id> --oci ghcr.io/org/stella-export:jan2025
|
||||
stella export verify <bundle> --manifest export.json --sig signatures/export.sig
|
||||
```
|
||||
|
||||
Exit codes: `0` ok, `2` bad args, `4` not found, `5` denied, `6` integrity failed, `8` export error.
|
||||
|
||||
### 3.8 RBAC & security
|
||||
|
||||
* Roles:
|
||||
|
||||
* `Export.Viewer`: list runs, download completed bundles.
|
||||
* `Export.Operator`: create runs, cancel, schedule, distribute.
|
||||
* `Export.Admin`: manage profiles, set retention, manage signing keys.
|
||||
|
||||
* Tenancy:
|
||||
|
||||
* Every run and artifact scoped by tenant; cross‑tenant export is disallowed.
|
||||
|
||||
* Secrets:
|
||||
|
||||
* KMS references for signing and encryption; never store private keys.
|
||||
|
||||
* PII & redaction:
|
||||
|
||||
* Exporters must not include secrets or credentials. Redaction rules enforced at writer level with schema‑based allowlists.
|
||||
|
||||
### 3.9 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `export_bytes_total{profile,tenant}`
|
||||
* `export_records_total{type}`
|
||||
* `export_duration_ms{profile}`
|
||||
* `export_failures_total{error_class}`
|
||||
* `export_distributions_total{type}`
|
||||
* `export_verify_fail_total`
|
||||
|
||||
* Traces:
|
||||
|
||||
* Spans per export phase: plan, stream, write, sign, distribute; baggage includes `export_run_id`.
|
||||
|
||||
* Logs:
|
||||
|
||||
* Structured JSON with counts, sizes, hashes, and redaction hints.
|
||||
|
||||
### 3.10 Performance targets
|
||||
|
||||
* Stream throughput ≥ 25k records/sec per worker for JSONL writing with compression.
|
||||
* Trivy bundle generation for 1M advisories ≤ 8 minutes on a standard worker.
|
||||
* Mirror delta export for 5% change set ≤ 2 minutes.
|
||||
|
||||
### 3.11 Edge cases & behavior
|
||||
|
||||
* **Schema drift**: adapter refuses to emit unknown fields without explicit mapping; run fails with `error_class=schema_mismatch`.
|
||||
* **Oversized bundles**: automatic sharding by time or content type; mirror OCI uses multi‑manifest indices.
|
||||
* **Missing policy snapshot**: profile `json:policy` will auto‑pull latest version unless pinned; pinning is recommended for reproducibility.
|
||||
* **Duplicate evidence**: writers dedupe by artifact hash and advisory id; AOC forbids merging.
|
||||
* **Air‑gap encryption**: if `encrypt=true`, mirror bundles require recipient public key material; decryption tooling documented.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Modules
|
||||
|
||||
* **New service:** `src/ExportCenter/StellaOps.ExportCenter`
|
||||
|
||||
* `api/` REST + WS
|
||||
* `planner/` scope planning, delta computation, sampling
|
||||
* `adapters/`
|
||||
|
||||
* `json/` canonical writers
|
||||
* `trivy/` db builders and schema mapping
|
||||
* `mirror/` fs/OCI builders, sharding, delta logic
|
||||
* `signing/` KMS clients, attestations
|
||||
* `dist/` download streaming, OCI push, object storage writer
|
||||
* `state/` repositories, migrations
|
||||
* `metrics/`, `audit/`, `security/`
|
||||
|
||||
* **SDK/CLI**
|
||||
|
||||
* `src/Cli/StellaOps.Cli` subcommands with streaming download and verification.
|
||||
|
||||
* **Console**
|
||||
|
||||
* `console/apps/export-center/` pages:
|
||||
|
||||
* Overview, Profiles, Runs, Run Detail, Distributions, Settings.
|
||||
* Components: `ExportPlanPreview`, `ProfileEditor`, `RunDiff`, `VerifyPanel`.
|
||||
|
||||
* **Existing services updates**
|
||||
|
||||
* Findings Ledger: new paginated streaming endpoints for advisories/VEX/SBOM/findings by filters and snapshots.
|
||||
* Policy Engine: “policy snapshot” exportable endpoint.
|
||||
* VEX Lens: consensus snapshot endpoint.
|
||||
|
||||
### 4.2 Packaging & deployment
|
||||
|
||||
* Containers:
|
||||
|
||||
* `stella/exporter:<ver>`
|
||||
* `stella/exporter-worker:<ver>` (optional separated worker pool)
|
||||
* Helm:
|
||||
|
||||
* WS replicas, concurrent export limits, default compression (`zstd`), default retention, KMS settings, OCI creds secrets.
|
||||
* DB migrations:
|
||||
|
||||
* Create `export_*` tables with proper indices (tenant, time, state).
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
* Phase 1: JSON (raw, policy) and Mirror (full) as download only.
|
||||
* Phase 2: Trivy DB adapters, OCI distribution.
|
||||
* Phase 3: Mirror deltas, encryption, verification tooling, scheduling.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Create/update the following docs; each page must end with the imposed rule statement.
|
||||
|
||||
1. `/docs/export-center/overview.md`
|
||||
Purpose, profiles, supported targets, AOC alignment, security model.
|
||||
|
||||
2. `/docs/export-center/architecture.md`
|
||||
Service components, adapters, manifests, signing, distribution flows.
|
||||
|
||||
3. `/docs/export-center/profiles.md`
|
||||
Profile schemas, examples, versioning, compatibility notes.
|
||||
|
||||
4. `/docs/export-center/api.md`
|
||||
All endpoints with request/response examples and error codes.
|
||||
|
||||
5. `/docs/export-center/cli.md`
|
||||
Commands with examples, scripts for CI/CD, verification.
|
||||
|
||||
6. `/docs/export-center/mirror-bundles.md`
|
||||
Filesystem and OCI layouts, delta exports, encryption, air‑gap import guide.
|
||||
|
||||
7. `/docs/export-center/trivy-adapter.md`
|
||||
Field mapping, supported ecosystems, compatibility and test matrix.
|
||||
|
||||
8. `/docs/export-center/provenance-and-signing.md`
|
||||
Manifest format, attestation details, verification process.
|
||||
|
||||
9. `/docs/operations/export-runbook.md`
|
||||
Common failures, recovery, tuning, capacity planning.
|
||||
|
||||
10. `/docs/security/export-hardening.md`
|
||||
RBAC, tenant isolation, secret redaction, encryption keys.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend: exporter
|
||||
|
||||
* [ ] Migrations for `export_profiles`, `export_runs`, `export_inputs`, `export_distributions`.
|
||||
* [ ] Planner to resolve filters to iterators over advisory/VEX/SBOM/findings datasets with pagination.
|
||||
* [ ] JSON adapter: canonical JSONL writers with schema normalization and redaction enforcement.
|
||||
* [ ] Policy snapshot embedder: pull policy version and evaluation outputs when requested.
|
||||
* [ ] Trivy adapter: implement schema mapping, writer, integrity validation, and compatibility version flag.
|
||||
* [ ] Mirror adapter: filesystem and OCI writer, sharding, manifest creation, delta computation.
|
||||
* [ ] Signing/attestation using KMS; detached and embedded options.
|
||||
* [ ] Distribution engines: download streaming, OCI push, object storage staging.
|
||||
* [ ] API layer with async export run handling and WebSocket updates.
|
||||
* [ ] Rate limit and concurrency controls per tenant/profile.
|
||||
* [ ] Audit logging for all create/cancel/distribute/verify actions.
|
||||
|
||||
### Integrations
|
||||
|
||||
* [ ] Findings Ledger streaming APIs for each content type.
|
||||
* [ ] Policy Engine endpoint to return deterministic policy snapshot and decision set by run.
|
||||
* [ ] VEX Lens endpoint to expose consensus snapshot.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Profiles CRUD with validation and test preview.
|
||||
* [ ] Create Run wizard with live count estimates and storage footprint prediction.
|
||||
* [ ] Runs list + detail page with manifest, provenance, and quick verify.
|
||||
* [ ] Download and distribution actions with progress and logs.
|
||||
* [ ] Verification panel to check signatures and hashes client‑side.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella export` commands as defined; include `verify` that checks signatures and hashes.
|
||||
* [ ] Auto‑resume of interrupted downloads with range requests.
|
||||
* [ ] Friendly error messages for schema mismatch and verification failure.
|
||||
|
||||
### Observability
|
||||
|
||||
* [ ] Metrics and traces per §3.9; dashboards for throughput, durations, failures, sizes.
|
||||
* [ ] Alerts for export failure rate and verify failures.
|
||||
|
||||
### Security & RBAC
|
||||
|
||||
* [ ] Enforce tenant scoping at query level; fuzz tests for leakage.
|
||||
* [ ] Role matrix checks on each API; Console hides forbidden actions.
|
||||
* [ ] Encryption test vectors and key rotation procedure.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author all files in §5 with concrete examples and diagrams.
|
||||
* [ ] Cross‑link from Orchestrator, Policy Studio, VEX Lens, and SBOM docs to the Export Center pages.
|
||||
* [ ] Append imposed rule line at the end of each page.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Implementation notes per profile
|
||||
|
||||
### 7.1 JSON: raw
|
||||
|
||||
* **Content:** advisories.normalized, vex.normalized, sboms (SPDX/CycloneDX), optional findings.raw.
|
||||
* **Normalization:** enforce Stella field casing, timestamps in RFC3339, unicode NFC.
|
||||
* **Compression:** `.jsonl.zst` per file to allow split/merge.
|
||||
|
||||
### 7.2 JSON: policy
|
||||
|
||||
* **Adds:** `policy_snapshot` and `findings.policy_evaluated.jsonl` with decision, rule_id, rationale, inputs fingerprint.
|
||||
* **Determinism:** include `policy_version` and `inputs_hash`; replays should match exactly.
|
||||
|
||||
### 7.3 Trivy DB
|
||||
|
||||
* **Mapping:**
|
||||
|
||||
* Package name, ecosystem, version ranges, CVE/CWE/aliases, severity mapping, vendor statements, fixed versions.
|
||||
* Ensure namespace mapping avoids collisions (e.g., distro vs ecosystem).
|
||||
* **Compatibility:** version flag in manifest; adapter throws if upstream schema version not supported.
|
||||
* **Validation:** run post‑build sanity checks (counts, required indexes).
|
||||
|
||||
### 7.4 Mirror: full/delta
|
||||
|
||||
* **Full:** everything needed to spin up an isolated read‑only Stella mirror with local search.
|
||||
* **Delta:** compute changed/added/removed advisory ids, VEX statements, SBOM subjects; update indexes and manifest with `base_export_id`.
|
||||
* **Encryption:** if enabled, encrypt data subtrees; leave `manifest.yaml` unencrypted for discoverability unless `strict=true`.
|
||||
|
||||
---
|
||||
|
||||
## 8) Acceptance criteria
|
||||
|
||||
* Operators can create an export with filters, download it, verify signatures, and trace back to source artifacts via provenance.
|
||||
* Trivy adapter produces a bundle consumable by Trivy without custom flags (basic validation in CI).
|
||||
* Mirror bundle imports successfully in an air‑gapped “mirror‑reader” sample app and serves queries from indexes.
|
||||
* Policy‑aware exports include deterministic decisions matching the specified `policy_version`.
|
||||
* RBAC prevents a Viewer from creating or canceling exports; tenancy prevents cross‑tenant leakage.
|
||||
* Metrics and dashboards show per‑profile throughput and error classes; alerts trigger on failure spikes.
|
||||
* Export retries are idempotent and do not duplicate content; hashes stable across re‑runs with identical inputs.
|
||||
|
||||
---
|
||||
|
||||
## 9) Risks & mitigations
|
||||
|
||||
* **Upstream schema changes break Trivy export.**
|
||||
Mitigation: versioned adapter with compatibility gate; integration tests against known fixtures; fail early with clear remediation.
|
||||
|
||||
* **Bundle size explosion.**
|
||||
Mitigation: zstd compression, sharding, delta exports, content‑addressed storage reuse for mirror OCI.
|
||||
|
||||
* **Data leakage via exports.**
|
||||
Mitigation: strict allowlist schemas, redaction filters, RBAC, tenant scoping tests, encryption for mirror.
|
||||
|
||||
* **Non‑deterministic policy outputs.**
|
||||
Mitigation: pin policy version and inputs hash; snapshot embedded rules; deterministic evaluation mode only.
|
||||
|
||||
* **Slow downloads/timeouts.**
|
||||
Mitigation: streaming with range support, resumable downloads, CDN integration if needed.
|
||||
|
||||
---
|
||||
|
||||
## 10) Test plan
|
||||
|
||||
* **Unit**
|
||||
Schema normalization; Trivy mapping; mirror delta computation; manifest hashing; signing.
|
||||
|
||||
* **Integration**
|
||||
End‑to‑end export with each profile; verify bundles; replay determinism of policy exports; OCI push/pull.
|
||||
|
||||
* **Compatibility**
|
||||
Validate Trivy bundle against a matrix of versions; import mirror bundle into a reference reader and run queries.
|
||||
|
||||
* **Security**
|
||||
Tenant isolation fuzzing; redaction checks; encryption round‑trip; signature verification with rotated keys.
|
||||
|
||||
* **Performance**
|
||||
Large dataset generation; parallel writer stress; OCI multi‑manifest publishing; download resume under packet loss.
|
||||
|
||||
* **Chaos**
|
||||
Kill exporter mid‑write; ensure resume or clean failure without partial corrupt bundles.
|
||||
|
||||
---
|
||||
|
||||
## 11) Philosophy
|
||||
|
||||
* **Ports, not prisons.** Exports should free your data to move with integrity and context, not trap it in a proprietary maze.
|
||||
* **Reproducible or it didn’t happen.** Every bit derived from known inputs, signed, traceable.
|
||||
* **Air‑gap is a first‑class citizen.** Mirror bundles are not an afterthought; they’re how serious orgs actually run.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
BIN
docs/implplan/EPIC_11.md
Normal file
BIN
docs/implplan/EPIC_11.md
Normal file
Binary file not shown.
BIN
docs/implplan/EPIC_12.md
Normal file
BIN
docs/implplan/EPIC_12.md
Normal file
Binary file not shown.
BIN
docs/implplan/EPIC_13.md
Normal file
BIN
docs/implplan/EPIC_13.md
Normal file
Binary file not shown.
BIN
docs/implplan/EPIC_14.md
Normal file
BIN
docs/implplan/EPIC_14.md
Normal file
Binary file not shown.
BIN
docs/implplan/EPIC_15.md
Normal file
BIN
docs/implplan/EPIC_15.md
Normal file
Binary file not shown.
1
docs/implplan/EPIC_16.md
Normal file
1
docs/implplan/EPIC_16.md
Normal file
@@ -0,0 +1 @@
|
||||
See `docs/airgap/EPIC_16_AIRGAP_MODE.md` for the full Epic 16 specification.
|
||||
1
docs/implplan/EPIC_17.md
Normal file
1
docs/implplan/EPIC_17.md
Normal file
@@ -0,0 +1 @@
|
||||
See `docs/api/EPIC_17_SDKS_OPENAPI.md` for the complete Epic 17 specification.
|
||||
1
docs/implplan/EPIC_18.md
Normal file
1
docs/implplan/EPIC_18.md
Normal file
@@ -0,0 +1 @@
|
||||
See `docs/risk/EPIC_18_RISK_PROFILES.md` for the complete Epic 18 specification.
|
||||
1
docs/implplan/EPIC_19.md
Normal file
1
docs/implplan/EPIC_19.md
Normal file
@@ -0,0 +1 @@
|
||||
See `docs/attestor/EPIC_19_ATTESTOR_CONSOLE.md` for the complete Epic 19 specification.
|
||||
567
docs/implplan/EPIC_2.md
Normal file
567
docs/implplan/EPIC_2.md
Normal file
@@ -0,0 +1,567 @@
|
||||
Fine. Here’s the next epic, written so you can paste it straight into the repo without having to babysit me. Same structure as before, maximum detail, zero hand‑waving.
|
||||
|
||||
---
|
||||
|
||||
# Epic 2: Policy Engine & Policy Editor (VEX + Advisory Application Rules)
|
||||
|
||||
> Short name: **Policy Engine v2**
|
||||
> Services touched: **Policy Engine, Web API, Console (Policy Editor), CLI, Conseiller, Excitator, SBOM Service, Authority, Workers/Scheduler**
|
||||
> Data stores: **MongoDB (policies, runs, effective findings), optional Redis/NATS for jobs**
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
This epic delivers the **organization‑specific decision layer** for Stella. Ingestion is now AOC‑compliant (Epic 1). That means advisories and VEX arrive as immutable raw facts. This epic builds the place where those facts become **effective findings** under policies you control.
|
||||
|
||||
Core deliverables:
|
||||
|
||||
* **Policy Engine**: deterministic evaluator that applies rule sets to inputs:
|
||||
|
||||
* Inputs: `advisory_raw`, `vex_raw`, SBOMs, optional telemetry hooks (reachability stubs), org metadata.
|
||||
* Outputs: `effective_finding_{policyId}` materializations, with full explanation traces.
|
||||
* **Policy Editor (Console + CLI)**: versioned policy authoring, simulation, review/approval workflow, and change diffs.
|
||||
* **Rules DSL v1**: safe, declarative language for VEX application, advisory normalization, and risk scoring. No arbitrary code execution, no network calls.
|
||||
* **Run Orchestrator**: incremental re‑evaluation when new raw facts or SBOM changes arrive; efficient partial updates.
|
||||
|
||||
The philosophy is boring on purpose: policy is a **pure function of inputs**. Same inputs and same policy yield the same outputs, every time, on every machine. If you want drama, watch reality TV, not your risk pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* Vendors disagree, contexts differ, and your tolerance for risk is not universal.
|
||||
* VEX means nothing until you decide **how** to apply it to **your** assets.
|
||||
* Auditors care about the “why.” You’ll need consistent, replayable answers, with traces.
|
||||
* Security teams need **simulation** before rollouts, and **diffs** after.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (deep details)
|
||||
|
||||
### 3.1 Data model
|
||||
|
||||
#### 3.1.1 Policy documents (Mongo: `policies`)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "policy:P-7:v3",
|
||||
"policy_id": "P-7",
|
||||
"version": 3,
|
||||
"name": "Default Org Policy",
|
||||
"status": "approved", // draft | submitted | approved | archived
|
||||
"owned_by": "team:sec-plat",
|
||||
"valid_from": "2025-01-15T00:00:00Z",
|
||||
"valid_to": null,
|
||||
"dsl": {
|
||||
"syntax": "stella-dsl@1",
|
||||
"source": "rule-set text or compiled IR ref"
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Baseline scoring + VEX precedence",
|
||||
"tags": ["baseline","vex","cvss"]
|
||||
},
|
||||
"provenance": {
|
||||
"created_by": "user:ali",
|
||||
"created_at": "2025-01-15T08:00:00Z",
|
||||
"submitted_by": "user:kay",
|
||||
"approved_by": "user:root",
|
||||
"approval_at": "2025-01-16T10:00:00Z",
|
||||
"checksum": "sha256:..."
|
||||
},
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
* `status=approved` is required to run in production.
|
||||
* Version increments are append‑only. Old versions remain runnable for replay.
|
||||
|
||||
#### 3.1.2 Policy runs (Mongo: `policy_runs`)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "run:P-7:2025-02-20T12:34:56Z:abcd",
|
||||
"policy_id": "P-7",
|
||||
"policy_version": 3,
|
||||
"inputs": {
|
||||
"sbom_set": ["sbom:S-42"],
|
||||
"advisory_cursor": "2025-02-20T00:00:00Z",
|
||||
"vex_cursor": "2025-02-20T00:00:00Z"
|
||||
},
|
||||
"mode": "incremental", // full | incremental | simulate
|
||||
"stats": {
|
||||
"components": 1742,
|
||||
"advisories_considered": 9210,
|
||||
"vex_considered": 1187,
|
||||
"rules_fired": 68023,
|
||||
"findings_out": 4321
|
||||
},
|
||||
"trace": {
|
||||
"location": "blob://traces/run-.../index.json",
|
||||
"sampling": "smart-10pct"
|
||||
},
|
||||
"status": "succeeded", // queued | running | failed | succeeded | canceled
|
||||
"started_at": "2025-02-20T12:34:56Z",
|
||||
"finished_at": "2025-02-20T12:35:41Z",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
#### 3.1.3 Effective findings (Mongo: `effective_finding_P-7`)
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "P-7:S-42:pkg:npm/lodash@4.17.21:CVE-2021-23337",
|
||||
"policy_id": "P-7",
|
||||
"policy_version": 3,
|
||||
"sbom_id": "S-42",
|
||||
"component_purl": "pkg:npm/lodash@4.17.21",
|
||||
"advisory_ids": ["CVE-2021-23337", "GHSA-..."],
|
||||
"status": "affected", // affected | not_affected | fixed | under_investigation | suppressed
|
||||
"severity": {
|
||||
"normalized": "High",
|
||||
"score": 7.5,
|
||||
"vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
|
||||
"rationale": "cvss_base(OSV) + vendor_weighting + env_modifiers"
|
||||
},
|
||||
"rationale": [
|
||||
{"rule":"vex.precedence","detail":"VendorX not_affected justified=component_not_present wins"},
|
||||
{"rule":"advisory.cvss.normalization","detail":"mapped GHSA severity to CVSS 3.1 = 7.5"}
|
||||
],
|
||||
"references": {
|
||||
"advisory_raw_ids": ["advisory_raw:osv:GHSA-...:v3"],
|
||||
"vex_raw_ids": ["vex_raw:VendorX:doc-123:v4"]
|
||||
},
|
||||
"run_id": "run:P-7:2025-02-20T12:34:56Z:abcd",
|
||||
"tenant": "default"
|
||||
}
|
||||
```
|
||||
|
||||
Write protection: only the **Policy Engine** service identity may write any `effective_finding_*` collection.
|
||||
|
||||
---
|
||||
|
||||
### 3.2 Rules DSL v1 (stella‑dsl@1)
|
||||
|
||||
**Design goals**
|
||||
|
||||
* Declarative, composable, deterministic.
|
||||
* No loops, no network IO, no non‑deterministic time.
|
||||
* Policy authors see readable text; the engine compiles to a safe IR.
|
||||
|
||||
**Concepts**
|
||||
|
||||
* **WHEN** condition matches a tuple `(sbom_component, advisory, optional vex_statements)`
|
||||
* **THEN** actions set `status`, compute `severity`, attach `rationale`, or `suppress` with reason.
|
||||
* **Profiles** for severity and scoring; **Maps** for vendor weighting; **Guards** for VEX justification.
|
||||
|
||||
**Mini‑grammar (subset)**
|
||||
|
||||
```
|
||||
policy "Default Org Policy" syntax "stella-dsl@1" {
|
||||
|
||||
profile severity {
|
||||
map vendor_weight {
|
||||
source "GHSA" => +0.5
|
||||
source "OSV" => +0.0
|
||||
source "VendorX" => -0.2
|
||||
}
|
||||
env base_cvss {
|
||||
if env.runtime == "serverless" then -0.5
|
||||
if env.exposure == "internal-only" then -1.0
|
||||
}
|
||||
}
|
||||
|
||||
rule vex_precedence {
|
||||
when vex.any(status in ["not_affected","fixed"])
|
||||
and vex.justification in ["component_not_present","vulnerable_code_not_present"]
|
||||
then status := vex.status
|
||||
because "VEX strong justification prevails";
|
||||
}
|
||||
|
||||
rule advisory_to_cvss {
|
||||
when advisory.source in ["GHSA","OSV"]
|
||||
then severity := normalize_cvss(advisory)
|
||||
because "Map vendor severity or CVSS vector";
|
||||
}
|
||||
|
||||
rule reachability_soft_suppress {
|
||||
when severity.normalized <= "Medium"
|
||||
and telemetry.reachability == "none"
|
||||
then status := "suppressed"
|
||||
because "not reachable and low severity";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Built‑ins** (non‑exhaustive)
|
||||
|
||||
* `normalize_cvss(advisory)` maps GHSA/OSV/CSAF severity fields to CVSS v3.1 numbers when possible; otherwise vendor‑to‑numeric mapping table in policy.
|
||||
* `vex.any(...)` tests across matching VEX statements for the same `(component, advisory)`.
|
||||
* `telemetry.*` is an optional input namespace reserved for future reachability data; if absent, expressions evaluate to `unknown` (no effect).
|
||||
|
||||
**Determinism**
|
||||
|
||||
* Rules are evaluated in **stable order**: explicit `priority` attribute or lexical order.
|
||||
* **First‑match** semantics for conflicting status unless `combine` is used.
|
||||
* Severity computations are pure; numeric maps are part of policy document.
|
||||
|
||||
---
|
||||
|
||||
### 3.3 Evaluation model
|
||||
|
||||
1. **Selection**
|
||||
|
||||
* For each SBOM component PURL, find candidate advisories from `advisory_raw` via linkset PURLs or identifiers.
|
||||
* For each pair `(component, advisory)`, load all matching VEX facts from `vex_raw`.
|
||||
|
||||
2. **Context assembly**
|
||||
|
||||
* Build an evaluation context from:
|
||||
|
||||
* `sbom_component`: PURL, licenses, relationships.
|
||||
* `advisory`: source, identifiers, references, embedded vendor severity (kept in `content.raw`).
|
||||
* `vex`: list of statements with status and justification.
|
||||
* `env`: org‑specific env vars configured per policy run (e.g., exposure).
|
||||
* Optional `telemetry` if available.
|
||||
|
||||
3. **Rule execution**
|
||||
|
||||
* Compile DSL to IR once per policy version; cache.
|
||||
* Execute rules per tuple; record which rules fired and the order.
|
||||
* If no rule sets status, default is `affected`.
|
||||
* If no rule sets severity, default severity uses `normalize_cvss(advisory)` with vendor defaults.
|
||||
|
||||
4. **Materialization**
|
||||
|
||||
* Write to `effective_finding_{policyId}` with `rationale` chain and references to raw docs.
|
||||
* Emit per‑tuple trace events; sample and store full traces per run.
|
||||
|
||||
5. **Incremental updates**
|
||||
|
||||
* A watch job observes new `advisory_raw` and `vex_raw` inserts and SBOM deltas.
|
||||
* The orchestrator computes the affected tuples and re‑evaluates only those.
|
||||
|
||||
6. **Replay**
|
||||
|
||||
* Any `policy_run` is fully reproducible by `(policy_id, version, input set, cursors)`.
|
||||
|
||||
---
|
||||
|
||||
### 3.4 VEX application semantics
|
||||
|
||||
* **Precedence**: a `not_affected` with strong justification (`component_not_present`, `vulnerable_code_not_present`, `fix_not_required`) wins unless another rule explicitly overrides by environment context.
|
||||
* **Scoping**: VEX statements often specify product/component scope. Matching uses PURL equivalence and version ranges extracted during ingestion linkset generation.
|
||||
* **Conflicts**: If multiple VEX statements conflict, the default is **most‑specific scope wins** (component > product > vendor), then newest `document_version`. Policies can override with explicit rules.
|
||||
* **Explainability**: Every VEX‑driven decision records which statement IDs were considered and which one won.
|
||||
|
||||
---
|
||||
|
||||
### 3.5 Advisory normalization rules
|
||||
|
||||
* **Vendor severity mapping**: Map GHSA levels or CSAF product‑tree severities to CVSS‑like numeric bands via policy maps.
|
||||
* **CVSS vector use**: If a valid vector exists in `content.raw`, parse and compute; apply policy modifiers from `profile severity`.
|
||||
* **Temporal/environment modifiers**: Optional reductions for network exposure, isolation, or compensating controls, all encoded in policy.
|
||||
|
||||
---
|
||||
|
||||
### 3.6 Performance and scale
|
||||
|
||||
* Partition evaluation by SBOM ID and hash ranges of PURLs.
|
||||
* Pre‑index `advisory_raw.linkset.purls` and `vex_raw.linkset.purls` (already in Epic 1).
|
||||
* Use streaming iterators; avoid loading entire SBOM or advisory sets into memory.
|
||||
* Materialize only changed findings (diff‑aware writes).
|
||||
* Target: 100k components, 1M advisories considered, 5 minutes incremental SLA on commodity hardware.
|
||||
|
||||
---
|
||||
|
||||
### 3.7 Error codes
|
||||
|
||||
| Code | Meaning | HTTP |
|
||||
| ------------- | ----------------------------------------------------- | ---- |
|
||||
| `ERR_POL_001` | Policy syntax error | 400 |
|
||||
| `ERR_POL_002` | Policy not approved for run | 403 |
|
||||
| `ERR_POL_003` | Missing inputs (SBOM/advisory/vex fetch failed) | 424 |
|
||||
| `ERR_POL_004` | Determinism guard triggered (non‑pure function usage) | 500 |
|
||||
| `ERR_POL_005` | Write denied to effective findings (caller invalid) | 403 |
|
||||
| `ERR_POL_006` | Run canceled or timed out | 408 |
|
||||
|
||||
---
|
||||
|
||||
### 3.8 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `policy_compile_seconds`, `policy_run_seconds{mode=...}`, `rules_fired_total`, `findings_written_total`, `vex_overrides_total`, `simulate_diff_total{delta=up|down|unchanged}`.
|
||||
* Tracing:
|
||||
|
||||
* Spans: `policy.compile`, `policy.select`, `policy.eval`, `policy.materialize`.
|
||||
* Logs:
|
||||
|
||||
* Include `policy_id`, `version`, `run_id`, `sbom_id`, `component_purl`, `advisory_id`, `vex_count`, `rule_hits`.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
### 3.9 Security and tenancy
|
||||
|
||||
* Only users with `policy:write` can create/modify policies.
|
||||
* `policy:approve` is a separate privileged role.
|
||||
* Only Policy Engine service identity has `effective:write`.
|
||||
* Tenancy is explicit on all documents and queries.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 4) API surface
|
||||
|
||||
### 4.1 Policy CRUD and lifecycle
|
||||
|
||||
* `POST /policies` create draft
|
||||
* `GET /policies?status=...` list
|
||||
* `GET /policies/{policyId}/versions/{v}` fetch
|
||||
* `POST /policies/{policyId}/submit` move draft to submitted
|
||||
* `POST /policies/{policyId}/approve` approve version
|
||||
* `POST /policies/{policyId}/archive` archive version
|
||||
|
||||
### 4.2 Compilation and validation
|
||||
|
||||
* `POST /policies/{policyId}/versions/{v}/compile`
|
||||
|
||||
* Returns IR checksum, syntax diagnostics, rule stats.
|
||||
|
||||
### 4.3 Runs
|
||||
|
||||
* `POST /policies/{policyId}/runs` body: `{mode, sbom_set, advisory_cursor?, vex_cursor?, env?}`
|
||||
* `GET /policies/{policyId}/runs/{runId}` status + stats
|
||||
* `POST /policies/{policyId}/simulate` returns **diff** vs current approved version on a sample SBOM set.
|
||||
|
||||
### 4.4 Findings and explanations
|
||||
|
||||
* `GET /findings/{policyId}?sbom_id=S-42&status=affected&severity=High+Critical`
|
||||
* `GET /findings/{policyId}/{findingId}/explain` returns ordered rule hits and linked raw IDs.
|
||||
|
||||
All endpoints require tenant scoping and appropriate `policy:*` or `findings:*` roles.
|
||||
|
||||
---
|
||||
|
||||
## 5) Console (Policy Editor) and CLI behavior
|
||||
|
||||
**Console**
|
||||
|
||||
* Monaco‑style editor with DSL syntax highlighting, lint, quick docs.
|
||||
* Side‑by‑side **Simulation** panel: show count of affected findings before/after.
|
||||
* Approval workflow: submit, review comments, approve with rationale.
|
||||
* Diffs: show rule‑wise changes and estimated impact.
|
||||
* Read‑only run viewer: heatmap of rules fired, top suppressions, VEX wins.
|
||||
|
||||
**CLI**
|
||||
|
||||
* `stella policy new --name "Default Org Policy"`
|
||||
* `stella policy edit P-7` opens local editor -> `submit`
|
||||
* `stella policy approve P-7 --version 3`
|
||||
* `stella policy simulate P-7 --sbom S-42 --env exposure=internal-only`
|
||||
* `stella findings ls --policy P-7 --sbom S-42 --status affected`
|
||||
|
||||
Exit codes map to `ERR_POL_*`.
|
||||
|
||||
---
|
||||
|
||||
## 6) Implementation tasks
|
||||
|
||||
### 6.1 Policy Engine service
|
||||
|
||||
* [ ] Implement DSL parser and IR compiler (`stella-dsl@1`).
|
||||
* [ ] Build evaluator with stable ordering and first‑match semantics.
|
||||
* [ ] Implement selection joiners for SBOM↔advisory↔vex using linksets.
|
||||
* [ ] Materialization writer with upsert‑only semantics to `effective_finding_{policyId}`.
|
||||
* [ ] Determinism guard (ban wall‑clock, network, and RNG during eval).
|
||||
* [ ] Incremental orchestrator listening to advisory/vex/SBOM change streams.
|
||||
* [ ] Trace emitter with rule‑hit sampling.
|
||||
* [ ] Unit tests, property tests, golden fixtures; perf tests to target SLA.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.2 Web API
|
||||
|
||||
* [ ] Policy CRUD, compile, run, simulate, findings, explain endpoints.
|
||||
* [ ] Pagination, filters, and tenant enforcement on all list endpoints.
|
||||
* [ ] Error mapping to `ERR_POL_*`.
|
||||
* [ ] Rate limits on simulate endpoints.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.3 Console (Policy Editor)
|
||||
|
||||
* [ ] Editor with DSL syntax highlighting and inline diagnostics.
|
||||
* [ ] Simulation UI with pre/post counts and top deltas.
|
||||
* [ ] Approval workflow UI with audit trail.
|
||||
* [ ] Run viewer dashboards (rule heatmap, VEX wins, suppressions).
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.4 CLI
|
||||
|
||||
* [ ] New commands: `policy new|edit|submit|approve|simulate`, `findings ls|get`.
|
||||
* [ ] Json/YAML output formats for CI consumption.
|
||||
* [ ] Non‑zero exits on syntax errors or simulation failures; map to `ERR_POL_*`.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.5 Conseiller & Excitator integration
|
||||
|
||||
* [ ] Provide search endpoints optimized for policy selection (batch by PURLs and IDs).
|
||||
* [ ] Harden linkset extraction to maximize join recall.
|
||||
* [ ] Add cursors for incremental selection windows per run.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.6 SBOM Service
|
||||
|
||||
* [ ] Ensure fast PURL index and component metadata projection for policy queries.
|
||||
* [ ] Provide relationship graph API for future transitive logic.
|
||||
* [ ] Emit change events on SBOM updates.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.7 Authority
|
||||
|
||||
* [ ] Define scopes: `policy:write`, `policy:approve`, `policy:run`, `findings:read`, `effective:write`.
|
||||
* [ ] Issue service identity for Policy Engine with `effective:write` only.
|
||||
* [ ] Enforce tenant claims at gateway.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 6.8 CI/CD
|
||||
|
||||
* [ ] Lint policy DSL in PRs; block invalid syntax.
|
||||
* [ ] Run `simulate` against golden SBOMs to detect explosive deltas.
|
||||
* [ ] Determinism CI: two runs with identical seeds produce identical outputs.
|
||||
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Documentation changes (create/update these files)
|
||||
|
||||
1. **`/docs/policy/overview.md`**
|
||||
|
||||
* What the Policy Engine is, high‑level concepts, inputs, outputs, determinism.
|
||||
2. **`/docs/policy/dsl.md`**
|
||||
|
||||
* Full grammar, built‑ins, examples, best practices, anti‑patterns.
|
||||
3. **`/docs/policy/lifecycle.md`**
|
||||
|
||||
* Draft → submitted → approved → archived, roles, and audit trail.
|
||||
4. **`/docs/policy/runs.md`**
|
||||
|
||||
* Run modes, incremental mechanics, cursors, replay.
|
||||
5. **`/docs/api/policy.md`**
|
||||
|
||||
* Endpoints, request/response schemas, error codes.
|
||||
6. **`/docs/cli/policy.md`**
|
||||
|
||||
* Command usage, examples, exit codes, JSON output contracts.
|
||||
7. **`/docs/ui/policy-editor.md`**
|
||||
|
||||
* Screens, workflows, simulation, diffs, approvals.
|
||||
8. **`/docs/architecture/policy-engine.md`**
|
||||
|
||||
* Detailed sequence diagrams, selection/join strategy, materialization schema.
|
||||
9. **`/docs/observability/policy.md`**
|
||||
|
||||
* Metrics, tracing, logs, sample dashboards.
|
||||
10. **`/docs/security/policy-governance.md`**
|
||||
|
||||
* Scopes, approvals, tenancy, least privilege.
|
||||
11. **`/docs/examples/policies/`**
|
||||
|
||||
* `baseline.pol`, `serverless.pol`, `internal-only.pol`, each with commentary.
|
||||
12. **`/docs/faq/policy-faq.md`**
|
||||
|
||||
* Common pitfalls, VEX conflict handling, determinism gotchas.
|
||||
|
||||
Each file includes a **Compliance checklist** for authors and reviewers.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 8) Acceptance criteria
|
||||
|
||||
* Policies are versioned, approvable, and compilable; invalid DSL blocks merges.
|
||||
* Engine produces deterministic outputs with full rationale chains.
|
||||
* VEX precedence rules work per spec and are overridable by policy.
|
||||
* Simulation yields accurate pre/post deltas and diffs.
|
||||
* Only Policy Engine can write to `effective_finding_*`.
|
||||
* Incremental runs pick up new advisories/VEX/SBOM changes without full re‑runs.
|
||||
* Console and CLI cover authoring, simulation, approval, and retrieval.
|
||||
* Observability dashboards show rule hits, VEX wins, and run timings.
|
||||
|
||||
---
|
||||
|
||||
## 9) Risks and mitigations
|
||||
|
||||
* **Policy sprawl**: too many similar policies.
|
||||
|
||||
* Mitigation: templates, policy inheritance in v1.1, tagging, ownership metadata.
|
||||
* **Non‑determinism creep**: someone sneaks wall‑clock or network into evaluation.
|
||||
|
||||
* Mitigation: determinism guard, static analyzer, and CI replay check.
|
||||
* **Join miss‑rate**: weak linksets cause under‑matching.
|
||||
|
||||
* Mitigation: linkset strengthening in ingestion, PURL equivalence tables, monitoring for “zero‑hit” rates.
|
||||
* **Approval bottlenecks**: blocked rollouts.
|
||||
|
||||
* Mitigation: RBAC with delegated approvers and time‑boxed SLAs.
|
||||
|
||||
---
|
||||
|
||||
## 10) Test plan
|
||||
|
||||
* **Unit**: parser, compiler, evaluator; conflict resolution; precedence.
|
||||
* **Property**: random policies over synthetic inputs; ensure no panics and stable outputs.
|
||||
* **Golden**: fixed SBOM + curated advisories/VEX → expected findings; compare every run.
|
||||
* **Performance**: large SBOMs with heavy rule sets; assert run times and memory ceilings.
|
||||
* **Integration**: end‑to‑end simulate → approve → run → diff; verify write protections.
|
||||
* **Chaos**: inject malformed VEX, missing advisories; ensure graceful degradation and clear errors.
|
||||
|
||||
---
|
||||
|
||||
## 11) Developer checklists
|
||||
|
||||
**Definition of Ready**
|
||||
|
||||
* Policy grammar finalized; examples prepared.
|
||||
* Linkset join queries benchmarked.
|
||||
* Owner and approvers assigned.
|
||||
|
||||
**Definition of Done**
|
||||
|
||||
* All APIs live with RBAC.
|
||||
* CLI and Console features shipped.
|
||||
* Determinism and golden tests green.
|
||||
* Observability dashboards deployed.
|
||||
* Docs in section 7 merged.
|
||||
* Two real org policies migrated and in production.
|
||||
|
||||
---
|
||||
|
||||
## 12) Glossary
|
||||
|
||||
* **Policy**: versioned rule set controlling status and severity.
|
||||
* **DSL**: domain‑specific language used to express rules.
|
||||
* **Run**: a single evaluation execution with defined inputs and outputs.
|
||||
* **Simulation**: a run that doesn’t write findings; returns diffs.
|
||||
* **Materialization**: persisted effective findings for fast queries.
|
||||
* **Determinism**: same inputs + same policy = same outputs. Always.
|
||||
|
||||
---
|
||||
|
||||
### Final imposed reminder
|
||||
|
||||
**Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
409
docs/implplan/EPIC_4.md
Normal file
409
docs/implplan/EPIC_4.md
Normal file
@@ -0,0 +1,409 @@
|
||||
Here’s Epic 4 in the same paste‑into‑repo, implementation‑ready style as the prior epics. It’s exhaustive, formal, and slots directly into the existing AOC model, Policy Engine, and Console.
|
||||
|
||||
---
|
||||
|
||||
# Epic 4: Policy Studio (author, version, simulate)
|
||||
|
||||
> Short name: **Policy Studio**
|
||||
> Services touched: **Policy Engine**, **Policy Registry** (new), **Web API Gateway**, **Authority** (authN/Z), **Scheduler/Workers**, **SBOM Service**, **Conseiller (Feedser)**, **Excitator (Vexer)**, **Telemetry**
|
||||
> Surfaces: **Console (Web UI)** feature module, **CLI**, **CI hooks**
|
||||
> Deliverables: Authoring workspace, policy versioning, static checks, simulation at scale, reviews/approvals, signing/publishing, promotion
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
**Policy Studio** is the end‑to‑end system for creating, evolving, and safely rolling out the rules that turn AOC facts (SBOM, advisories, VEX) into **effective findings**. It provides:
|
||||
|
||||
* A **workspace** where authors write policies in the DSL (Epic 2), with linting, autocompletion, snippets, and templates.
|
||||
* A **Policy Registry** that stores immutable versions, compiled artifacts, metadata, provenance, and signatures.
|
||||
* **Simulation** at two levels: quick local samples and large batch simulations across real SBOM inventories with full deltas.
|
||||
* A **review/approval** workflow with comments, diffs, required approvers, and promotion to environments (dev/test/prod).
|
||||
* **Publishing** semantics: signed, immutable versions bound to tenants; rollback and deprecation.
|
||||
* Tight integration with **Explain** traces so any change can show exactly which rules fired and why outcomes shifted.
|
||||
|
||||
The Studio respects **AOC enforcement**: policies never edit or merge facts. They only interpret facts and produce determinations consistent with precedence rules defined in the DSL.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* Policy errors are expensive. Authors need safe sandboxes, deterministic builds, and evidence before rollout.
|
||||
* Auditors require immutability, provenance, and reproducibility from “source policy” to “effective finding.”
|
||||
* Teams want gradual rollout: simulate, canary, promote, observe, rollback.
|
||||
* Policy knowledge should be modular, reusable, and testable, not tribal.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Domain model
|
||||
|
||||
* **PolicyPackage**: `{name, tenant, description, owners[], tags[], created_at}`
|
||||
* **PolicyVersion** (immutable): `{package, semver, source_sha, compiled_sha, status: draft|review|approved|published|deprecated|archived, created_by, created_at, signatures[], changelog, metadata{}}`
|
||||
* **Workspace**: mutable working area for authors; holds unversioned edits until compiled.
|
||||
* **CompilationArtifact**: `{policy_version, compiler_version, diagnostics[], rule_index[], symbol_table}`
|
||||
* **SimulationSpec**: `{policy_version|workspace, sbom_selector, time_window?, environment?, sample_size?, severity_floor?, includes{advisories?, vex?}}`
|
||||
* **SimulationRun**: `{run_id, spec, started_at, finished_at, result{counts_before, counts_after, top_deltas[], by_rule_hit[], sample_explains[]}}`
|
||||
* **Review**: `{policy_version, required_approvers[], votes[], comments[], files_changed[], diffs[]}`
|
||||
* **Promotion**: `{policy_version, environment: dev|test|prod, promoted_by, promoted_at, rollout_strategy: All|Percent|TenantSubset}`
|
||||
* **Attestation**: OIDC‑backed signature metadata binding `source_sha` and `compiled_sha` to an actor and time.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.2 Authoring workflow
|
||||
|
||||
1. **Create** a workspace from a template (e.g., “Default Risk Model,” “License Tilted,” “Cloud‑Native SBOM”).
|
||||
2. **Edit** in the Studio: Monaco editor with DSL grammar, intelligent completion for predicates, policies, attributes.
|
||||
3. **Lint & compile** locally: semantic checks, forbidden rules detection, policy size limits, constant‑folding.
|
||||
4. **Unit tests**: run policy test cases on bundled fixtures and golden expectations.
|
||||
5. **Quick simulate** on selected SBOMs (10–50 items) to preview counts, examples, and rule heatmap.
|
||||
6. **Propose version**: bump semver, enter changelog; create a **PolicyVersion** in `review` with compiled artifacts.
|
||||
7. **Review & approval**: side‑by‑side diff, comments, required approvers enforced by RBAC.
|
||||
8. **Batch simulation**: run at scale across tenant inventory; produce deltas, sample explainer evidence.
|
||||
9. **Publish**: sign and move to `published`; optional **Promotion** to target environment(s).
|
||||
10. **Run** evaluation with the selected policy version; verify outcomes; optionally promote to default.
|
||||
11. **Rollback**: select an older version; promotion updates references without mutating older versions.
|
||||
|
||||
### 3.3 Editing experience (Console)
|
||||
|
||||
* **Three‑pane layout**: file tree, editor, diagnostics/simulation.
|
||||
* **Features**: autocomplete from symbol table, in‑editor docs on hover, go‑to definition, rule references, rename symbols across files, snippet library, policy templates.
|
||||
* **Validations**:
|
||||
|
||||
* AOC guardrails: no edit/merge actions on source facts, only interpretation.
|
||||
* Precedence correctness: if rules conflict, studio shows explicit order and effective winner.
|
||||
* Severity floor and normalization mapping validated against registry configuration.
|
||||
* **Diagnostics panel**: errors, warnings, performance hints (e.g., “predicate X loads N advisories per component; consider indexing”).
|
||||
* **Rule heatmap**: during simulation, bar chart of rule firings and the objects they impact.
|
||||
* **Explain sampler**: click any delta bucket to open a sampled finding with full trace.
|
||||
|
||||
### 3.4 Simulation
|
||||
|
||||
* **Quick Sim**: synchronous; runs in browser‑orchestrated job against API, constrained by `sample_size`.
|
||||
* **Batch Sim**: asynchronous run in workers:
|
||||
|
||||
* Input selection: all SBOMs, labels, artifact regex, last N ingests, or a curated set.
|
||||
* Outputs: counts by severity before/after, by status, top deltas by component and advisory, rule heatmap, top K affected artifacts.
|
||||
* Evidence: NDJSON of sampled findings with traces; CSV summary; signed result manifest.
|
||||
* Guardrails: cannot publish if batch sim drift > configurable threshold without an override justification.
|
||||
|
||||
### 3.5 Versioning & promotion
|
||||
|
||||
* Semver enforced: `major` implies compatibility break (e.g., precedence changes), `minor` adds rules, `patch` fixes.
|
||||
* **Immutable**: after `published`, the version cannot change; deprecate instead.
|
||||
* **Environment bindings**: dev/test/prod mapping per tenant; default policy per environment.
|
||||
* **Canary**: promote to a subset of tenants or artifacts; the Runs page displays A/B comparisons.
|
||||
|
||||
### 3.6 Review & approval
|
||||
|
||||
* Require N approvers by role; self‑approval optionally prohibited.
|
||||
* Line and file comments; overall decision with justification.
|
||||
* Review snapshot captures: diffs, diagnostics, simulation summary.
|
||||
* Webhooks to notify external systems of review events.
|
||||
|
||||
### 3.7 RBAC (Authority)
|
||||
|
||||
Roles per tenant:
|
||||
|
||||
* **Policy Author**: create/edit workspace, quick sim, propose versions.
|
||||
* **Policy Reviewer**: comment, request changes, approve/reject.
|
||||
* **Policy Approver**: final approve, publish.
|
||||
* **Policy Operator**: promote, rollback, schedule runs.
|
||||
* **Read‑only Auditor**: view everything, download evidence.
|
||||
|
||||
All actions server‑checked; UI only hides affordances.
|
||||
|
||||
### 3.8 CLI + CI integration
|
||||
|
||||
CLI verbs (examples):
|
||||
|
||||
```
|
||||
stella policy init --template default
|
||||
stella policy lint
|
||||
stella policy compile
|
||||
stella policy test --golden ./tests
|
||||
stella policy simulate --sboms label:prod --sample 1000
|
||||
stella policy version bump --level minor --changelog "Normalize GHSA CVSS"
|
||||
stella policy submit --reviewers alice@example.com,bob@example.com
|
||||
stella policy approve --version 1.3.0
|
||||
stella policy publish --version 1.3.0 --sign
|
||||
stella policy promote --version 1.3.0 --env test --percent 20
|
||||
stella policy rollback --env prod --to 1.2.1
|
||||
```
|
||||
|
||||
CI usage:
|
||||
|
||||
* Lint, compile, and run unit tests on PRs that modify `/policies/**`.
|
||||
* Optionally trigger **Batch Sim** against a staging inventory and post a Markdown report to the PR.
|
||||
* Block merge if diagnostics include errors or drift exceeds thresholds.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.9 APIs (representative)
|
||||
|
||||
* `POST /policies/workspaces` create from template
|
||||
* `PUT /policies/workspaces/{id}/files` edit source files
|
||||
* `POST /policies/workspaces/{id}/compile` get diagnostics + compiled artifact
|
||||
* `POST /policies/workspaces/{id}/simulate` quick sim
|
||||
* `POST /policies/versions` create version from workspace with semver + changelog
|
||||
* `GET /policies/versions/{id}` fetch version + diagnostics + sim summary
|
||||
* `POST /policies/versions/{id}/reviews` open review
|
||||
* `POST /policies/versions/{id}/approve` record approval
|
||||
* `POST /policies/versions/{id}/publish` sign + publish
|
||||
* `POST /policies/versions/{id}/promote` bind to env/canary
|
||||
* `POST /policies/versions/{id}/simulate-batch` start batch sim (async)
|
||||
* `GET /policies/simulations/{run_id}` get sim results and artifacts
|
||||
* `GET /policies/registry` list packages/versions, status and bindings
|
||||
|
||||
All calls require tenant scoping and RBAC.
|
||||
|
||||
### 3.10 Storage & data
|
||||
|
||||
* **Policy Registry DB** (MongoDB): packages, versions, workspaces, metadata.
|
||||
* **Object storage**: source bundles, compiled artifacts, simulation result bundles, evidence.
|
||||
* **Indexing**: compound indexes by `{tenant, package}`, `{tenant, status}`, `{tenant, environment}`.
|
||||
* **Retention**: configurable retention for workspaces and simulation artifacts; versions never deleted, only archived.
|
||||
|
||||
### 3.11 Evidence & provenance
|
||||
|
||||
* Every published version has:
|
||||
|
||||
* `source_sha` (content digest of the policy source bundle)
|
||||
* `compiled_sha` (digest of compiled artifact)
|
||||
* Attestation: signed envelope binding digests to an identity, time, and tenant.
|
||||
* Links to the exact compiler version, inputs, and environment.
|
||||
|
||||
### 3.12 Observability
|
||||
|
||||
* Metrics: compile time, diagnostics rate, simulation queue depth, delta magnitude distribution, approval latencies.
|
||||
* Logs: structured events for lifecycle transitions.
|
||||
* Traces: long simulations emit span per shard.
|
||||
|
||||
### 3.13 Performance & scale
|
||||
|
||||
* Compilation should complete under 3 seconds for typical policies; warn at 10s.
|
||||
* Batch sim uses workers with partitioning by SBOM id; results reduced by the API.
|
||||
* Memory guardrails on rule execution; deny policies that exceed configured complexity limits.
|
||||
|
||||
### 3.14 Security
|
||||
|
||||
* OIDC‑backed signing and attestation.
|
||||
* Policy sources are scanned on upload for secrets; blocked if found.
|
||||
* Strict CSP in Studio pages; tokens stored in memory, not localStorage.
|
||||
* Tenant isolation in buckets and DB collections.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **Policy Registry (new microservice)**
|
||||
|
||||
* REST API and background workers for batch simulation orchestration.
|
||||
* Stores workspaces, versions, metadata, bindings, reviews.
|
||||
* Generates signed attestations at publish time.
|
||||
* Coordinates with **Policy Engine** for compile/simulate invocations.
|
||||
|
||||
* **Policy Engine (existing)**
|
||||
|
||||
* Expose compile and simulate endpoints with deterministic outputs.
|
||||
* Provide rule coverage, symbol table, and explain traces for samples.
|
||||
|
||||
* **Web API Gateway**
|
||||
|
||||
* Routes requests; injects tenant context; enforces RBAC.
|
||||
|
||||
### 4.2 Console (Web UI) feature module
|
||||
|
||||
* `packages/features/policies` (shared with Epic 3):
|
||||
|
||||
* **Studio** routes: `/policies/studio`, `/policies/:id/versions/:v/edit`, `/simulate`, `/review`.
|
||||
* Monaco editor wrapper for DSL with hover docs, autocomplete.
|
||||
* Diff viewer, diagnostics, heatmap, explain sampler, review UI.
|
||||
|
||||
### 4.3 CLI
|
||||
|
||||
* New commands under `stella policy *`; typed client generated from OpenAPI.
|
||||
* Outputs machine‑readable JSON and pretty tables.
|
||||
|
||||
### 4.4 Workers
|
||||
|
||||
* **Simulation workers**: pull shards of SBOMs, run policy, emit partials, reduce into result bundle.
|
||||
* **Notification worker**: sends webhooks on review, approval, publish, promote.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. **`/docs/policy/studio-overview.md`**
|
||||
|
||||
* Concepts, roles, lifecycle, glossary.
|
||||
2. **`/docs/policy/authoring.md`**
|
||||
|
||||
* Workspace, templates, snippets, lint rules, best practices.
|
||||
3. **`/docs/policy/versioning-and-publishing.md`**
|
||||
|
||||
* Semver, immutability, deprecation, rollback, attestations.
|
||||
4. **`/docs/policy/simulation.md`**
|
||||
|
||||
* Quick vs batch sim, selection strategies, thresholds, evidence artifacts.
|
||||
5. **`/docs/policy/review-and-approval.md`**
|
||||
|
||||
* Required approvers, comments, webhooks, audit trail.
|
||||
6. **`/docs/policy/promotion.md`**
|
||||
|
||||
* Environments, canary, default policy binding, rollback.
|
||||
7. **`/docs/policy/cli.md`**
|
||||
|
||||
* Command reference with examples and JSON outputs.
|
||||
8. **`/docs/policy/api.md`**
|
||||
|
||||
* REST endpoints, request/response schemas, error codes.
|
||||
9. **`/docs/security/policy-attestations.md`**
|
||||
|
||||
* Signatures, digests, verifier steps.
|
||||
10. **`/docs/architecture/policy-registry.md`**
|
||||
|
||||
* Service design, schemas, queues, failure modes.
|
||||
11. **`/docs/observability/policy-telemetry.md`**
|
||||
|
||||
* Metrics, logs, tracing, dashboards.
|
||||
12. **`/docs/runbooks/policy-incident.md`**
|
||||
|
||||
* Rolling back a bad policy, freezing publishes, forensic steps.
|
||||
13. **`/docs/examples/policy-templates.md`**
|
||||
|
||||
* Ready‑made templates and snippet catalog.
|
||||
14. **`/docs/aoc/aoc-guardrails.md`**
|
||||
|
||||
* How Studio enforces AOC in authoring and review.
|
||||
|
||||
Each doc ends with a “Compliance checklist.”
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Tasks
|
||||
|
||||
### 6.1 Backend: Policy Registry
|
||||
|
||||
* [ ] Define OpenAPI spec for Registry (workspaces, versions, reviews, sim).
|
||||
* [ ] Implement workspace storage and file CRUD.
|
||||
* [ ] Integrate with Policy Engine compile endpoint; return diagnostics, symbol table.
|
||||
* [ ] Implement quick simulation with request limits.
|
||||
* [ ] Implement batch simulation orchestration: enqueue shards, collect results, reduce deltas, store artifacts.
|
||||
* [ ] Implement review model: comments, required approvers, decisions.
|
||||
* [ ] Implement publish: sign, persist attestation, set status=published.
|
||||
* [ ] Implement promotion bindings per tenant/environment; canary subsets.
|
||||
* [ ] RBAC checks for all endpoints.
|
||||
* [ ] Unit/integration tests; load tests for batch sim.
|
||||
|
||||
### 6.2 Policy Engine enhancements
|
||||
|
||||
* [ ] Return rule coverage and firing counts with compile/simulate.
|
||||
* [ ] Return symbol table and inline docs for editor autocomplete.
|
||||
* [ ] Expose deterministic Explain traces for sampled findings.
|
||||
* [ ] Enforce complexity/time limits and report breaches.
|
||||
|
||||
### 6.3 Console (Web UI)
|
||||
|
||||
* [ ] Build Studio editor wrapper with Monaco + DSL language server hooks.
|
||||
* [ ] Implement file tree, snippets, templates, hotkeys, search/replace.
|
||||
* [ ] Diagnostics panel with jump‑to‑line, quick fixes.
|
||||
* [ ] Simulation panel: quick sim UI, charts, heatmap, sample explains.
|
||||
* [ ] Review UI: diff, comments, approvals, status badges.
|
||||
* [ ] Publish & Promote flows with confirmation and post‑actions.
|
||||
* [ ] Batch sim results pages with export buttons.
|
||||
* [ ] Accessibility audits and keyboard‑only authoring flow.
|
||||
|
||||
### 6.4 CLI
|
||||
|
||||
* [ ] Implement commands listed in 3.8 with rich help and examples.
|
||||
* [ ] Add `--json` flag for machine consumption; emit stable schemas.
|
||||
* [ ] Exit codes aligned with CI usage (lint errors → non‑zero).
|
||||
|
||||
### 6.5 CI/CD & Security
|
||||
|
||||
* [ ] Add CI job that runs `stella policy lint/compile/test` on PRs.
|
||||
* [ ] Optional job that triggers batch sim against staging inventory; post summary to PR.
|
||||
* [ ] Policy source secret scanning; block on findings.
|
||||
* [ ] Signing keys configuration; verify pipeline for attestation on publish.
|
||||
|
||||
### 6.6 Docs
|
||||
|
||||
* [ ] Write all docs in section 5 with screenshots and CLI transcripts.
|
||||
* [ ] Add cookbook examples and templates in `/docs/examples/policy-templates.md`.
|
||||
* [ ] Wire contextual Help links from Studio to relevant docs.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Authors can create, edit, lint, compile policies with inline diagnostics and autocomplete.
|
||||
* Quick simulation produces counts, rule heatmap, and sample explains within UI.
|
||||
* Batch simulation scales across large SBOM sets, producing deltas and downloadable evidence.
|
||||
* Review requires configured approvers; comments and diffs are preserved.
|
||||
* Publish generates immutable, signed versions with attestations.
|
||||
* Promotion binds versions to environments and supports canary and rollback.
|
||||
* CLI supports full lifecycle and is usable in CI.
|
||||
* All actions are tenant‑scoped, RBAC‑enforced, and logged.
|
||||
* AOC guardrails prevent any mutation of raw facts.
|
||||
* Documentation shipped and linked contextually from the Studio.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks & mitigations
|
||||
|
||||
* **Policy complexity causes timeouts** → compile‑time complexity scoring, execution limits, early diagnostics.
|
||||
* **Simulation cost at scale** → sharding and streaming reducers; sampling; configurable quotas.
|
||||
* **RBAC misconfiguration** → server‑enforced checks, defense‑in‑depth tests, deny‑by‑default.
|
||||
* **Attestation key management** → OIDC‑backed signatures; auditable verifier tool; time‑boxed credentials.
|
||||
* **Editor usability** → language server with accurate completions; docs on hover; snippet library.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit**: compiler adapters, registry models, reviewers workflow, CLI options.
|
||||
* **Integration**: compile→simulate→publish→promote on seeded data.
|
||||
* **E2E**: Playwright flows for author→review→batch sim→publish→promote→rollback.
|
||||
* **Performance**: load test batch simulation with 100k components spread across SBOMs.
|
||||
* **Security**: RBAC matrix tests; secret scanning; signing and verification.
|
||||
* **Determinism**: same inputs produce identical `compiled_sha` and simulation summaries.
|
||||
|
||||
---
|
||||
|
||||
## 10) Feature flags
|
||||
|
||||
* `policy.studio` (enables editor and quick sim)
|
||||
* `policy.batch-sim`
|
||||
* `policy.canary-promotion`
|
||||
* `policy.signature-required` (enforce signing on publish)
|
||||
|
||||
Flags documented in `/docs/observability/policy-telemetry.md`.
|
||||
|
||||
---
|
||||
|
||||
## 11) Non‑goals (this epic)
|
||||
|
||||
* Building a general IDE for arbitrary languages; the editor is purpose‑built for the DSL.
|
||||
* Auto‑generated policies from AI without human approval.
|
||||
* Cross‑tenant policies; all policies are tenant‑scoped.
|
||||
|
||||
---
|
||||
|
||||
## 12) Philosophy
|
||||
|
||||
* **Safety first**: it’s cheaper to prevent a bad policy than to fix its fallout.
|
||||
* **Determinism**: same inputs, same outputs, verifiably.
|
||||
* **Immutability**: versions and evidence are forever; we deprecate, not mutate.
|
||||
* **Transparency**: every change is explainable with traces and proofs.
|
||||
* **Reusability**: templates, snippets, and tests turn policy from art into engineering.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
431
docs/implplan/EPIC_5.md
Normal file
431
docs/implplan/EPIC_5.md
Normal file
@@ -0,0 +1,431 @@
|
||||
Here’s Epic 5 in the same paste‑into‑repo, implementation‑ready format as the prior epics. It’s exhaustive, formal, and designed to slot into AOC, Policy Engine, Conseiller/Excitator, and the Console.
|
||||
|
||||
---
|
||||
|
||||
# Epic 5: SBOM Graph Explorer
|
||||
|
||||
> Short name: **Graph Explorer**
|
||||
> Services touched: **SBOM Service**, **Graph Indexer** (new), **Graph API** (new), **Policy Engine**, **Conseiller (Feedser)**, **Excitator (Vexer)**, **Web API Gateway**, **Authority** (authN/Z), **Workers/Scheduler**, **Telemetry**
|
||||
> Surfaces: **Console (Web UI)** graph module, **CLI**, **Exports**
|
||||
> Deliverables: Interactive graph UI with semantic zoom, saved queries, policy/VEX/advisory overlays, diff views, impact analysis, exports
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
**SBOM Graph Explorer** is the interactive, tenant‑scoped view of all supply‑chain relationships the platform knows about, rendered as a navigable graph. It connects:
|
||||
|
||||
* **Artifacts** (applications, images, libs), **Packages/Versions**, **Files/Paths**, **Licenses**, **Advisories** (from Conseiller), **VEX statements** (from Excitator), **Provenance** (builds, sources), and **Policies** (overlays of determinations)
|
||||
* **Edges** like `depends_on`, `contains`, `built_from`, `declared_in`, `affected_by`, `vex_exempts`, `governs_with`
|
||||
* **Time/version** dimension: multiple SBOM snapshots with diffs
|
||||
|
||||
It’s built for investigation and review: find where a vulnerable package enters; see which apps are impacted; understand why a finding exists; simulate a policy version and see the delta. The explorer observes **AOC enforcement**: it never mutates facts; it aggregates and visualizes them. Only the Policy Engine may classify, and classification is displayed as overlays.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why
|
||||
|
||||
* SBOMs are graphs. Tables flatten what matters and hide transitive risk.
|
||||
* Engineers, security, and auditors need impact answers quickly: “What pulls in `log4j:2.17` and where is it at runtime?”
|
||||
* Policy/VEX/advisory interactions are nuanced. A visual overlay makes precedence and outcomes obvious.
|
||||
* Review is collaborative; you need saved queries, deep links, exports, and consistent evidence.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Domain model
|
||||
|
||||
**Nodes** (typed, versioned, tenant‑scoped):
|
||||
|
||||
* `Artifact`: application, service, container image, library, module
|
||||
* `Package`: name + ecosystem (purl), `PackageVersion` with resolved version
|
||||
* `File`: path within artifact or image layer
|
||||
* `License`: SPDX id
|
||||
* `Advisory`: normalized advisory id (GHSA, CVE, vendor), source = Conseiller
|
||||
* `VEX`: statement with product context, status, justification, source = Excitator
|
||||
* `SBOM`: ingestion unit; includes metadata (tool, sha, build info)
|
||||
* `PolicyDetermination`: materialized view of Policy Engine results (read‑only overlay)
|
||||
* `Build`: provenance, commit, workflow run
|
||||
* `Source`: repo, tag, commit
|
||||
|
||||
**Edges** (directed):
|
||||
|
||||
* `declared_in` (PackageVersion → SBOM)
|
||||
* `contains` (Artifact → PackageVersion | File)
|
||||
* `depends_on` (PackageVersion → PackageVersion) with scope attr (prod|dev|test|optional)
|
||||
* `built_from` (Artifact → Build), `provenance_of` (Build → Source)
|
||||
* `affected_by` (PackageVersion → Advisory) with range semantics
|
||||
* `vex_exempts` (Advisory ↔ VEX) scoped by product/component
|
||||
* `licensed_under` (Artifact|PackageVersion → License)
|
||||
* `governs_with` (Artifact|PackageVersion → PolicyDetermination)
|
||||
* `derived_from` (SBOM → SBOM) for superseding snapshots
|
||||
|
||||
**Identity & versioning**
|
||||
|
||||
* Every node has a stable key: `{tenant}:{type}:{natural_id}` (e.g., purl for packages, digest for images).
|
||||
* SBOM snapshots are immutable; edges carry `valid_from`/`valid_to` for time travel and diffing.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.2 User capabilities (end‑to‑end)
|
||||
|
||||
* **Search & Navigate**: global search (purls, CVEs, repos, licenses), keyboard nav, breadcrumbs, semantic zoom.
|
||||
* **Lenses**: toggle views (Security, License, Provenance, Runtime vs Dev, Policy effect).
|
||||
* **Overlays**:
|
||||
|
||||
* **Advisory overlay**: show affected nodes/edges with source, severity, ranges.
|
||||
* **VEX overlay**: show suppressions/justifications; collapse exempted paths.
|
||||
* **Policy overlay**: choose a policy version; nodes/edges reflect determinations (severity, status) with explain sampling.
|
||||
* **Impact analysis**: pick a vulnerable node; highlight upstream/downstream dependents, scope filters, shortest/all paths with constraints.
|
||||
* **Diff view**: compare SBOM A vs B; show added/removed nodes/edges, changed versions, changed determinations.
|
||||
* **Saved queries**: visual builder + JSON query; shareable permalinks scoped by tenant and environment.
|
||||
* **Exports**: GraphML, CSV edge list, NDJSON of findings, PNG/SVG snapshot.
|
||||
* **Evidence details**: side panel with raw facts, advisory links, VEX statements, policy explain trace, provenance.
|
||||
* **Accessibility**: tab‑navigable, high‑contrast, screen‑reader labels for nodes and sidebars.
|
||||
|
||||
### 3.3 Query model
|
||||
|
||||
* **Visual builder** for common queries:
|
||||
|
||||
* “Show all paths from Artifact X to Advisory Y up to depth 6.”
|
||||
* “All runtime dependencies with license = GPL‑3.0.”
|
||||
* “All artifacts affected by GHSA‑… with no applicable VEX.”
|
||||
* “Which SBOMs introduced/removed `openssl` between build 120 and 130?”
|
||||
* **JSON query** (internal, POST body) with:
|
||||
|
||||
* `start`: list of node selectors (type + id or attributes)
|
||||
* `expand`: edge types and depth, direction, scope filters
|
||||
* `where`: predicates on node/edge attributes
|
||||
* `overlay`: policy version id, advisory sources, VEX filters
|
||||
* `limit`: nodes, edges, timebox, cost budget
|
||||
* **Cost control**: server estimates cost, denies or pages results; UI streams partial graph tiles.
|
||||
|
||||
### 3.4 UI architecture (Console)
|
||||
|
||||
* **Canvas**: WebGL renderer with level‑of‑detail, edge bundling, and label culling; deterministic layout when possible (seeded).
|
||||
* **Semantic zoom**:
|
||||
|
||||
* Far: clusters by artifact/repo/ecosystem, color by lens
|
||||
* Mid: package groups, advisory badges, license swatches
|
||||
* Near: concrete versions, direct edges, inline badges for policy determinations
|
||||
* **Panels**:
|
||||
|
||||
* Left: search, filters, lens selector, saved queries
|
||||
* Right: details, explain trace, evidence tabs (Advisory/VEX/Policy/Provenance)
|
||||
* Bottom: query expression, diagnostics, performance/stream status
|
||||
* **Diff mode**: split or overlay, color legend (add/remove/changed), filter by node type.
|
||||
* **Deep links**: URL encodes query + viewport; shareable respecting RBAC.
|
||||
* **Keyboard**: space drag, +/- zoom, F to focus, G to expand neighbors, P to show paths.
|
||||
|
||||
### 3.5 Back‑end architecture
|
||||
|
||||
**Graph Indexer (new)**
|
||||
|
||||
* Consumes SBOM ingests, Conseiller advisories, Excitator VEX statements, Policy Engine determinations (read‑only).
|
||||
* Projects facts into a **property graph** persisted in:
|
||||
|
||||
* Primary: document store + adjacency sets (e.g., Mongo collections + compressed adjacency lists)
|
||||
* Optional driver for graph DB backends if needed (pluggable)
|
||||
* Maintains materialized aggregates: degree, critical paths cache, affected artifact counts, license distribution.
|
||||
* Emits **graph snapshots** per SBOM with lineage to original ingestion.
|
||||
|
||||
**Graph API (new)**
|
||||
|
||||
* Endpoints for search, neighbor expansion, path queries, diffs, overlays, exports.
|
||||
* Streaming responses for large graphs (chunked NDJSON tiles).
|
||||
* Cost accounting + quotas per tenant.
|
||||
|
||||
**Workers**
|
||||
|
||||
* **Centrality & clustering** precompute on idle: betweenness approximations, connected components, Louvain clusters.
|
||||
* **Diff compute** on new SBOM ingestion pairs (previous vs current).
|
||||
* **Overlay materialization** cache for popular policy versions.
|
||||
|
||||
**Policy Engine integration**
|
||||
|
||||
* Graph API requests can specify a policy version.
|
||||
* For sampled nodes, the API fetches explain traces; for counts, uses precomputed overlay materializations where available.
|
||||
|
||||
**AOC enforcement**
|
||||
|
||||
* Graph Indexer never merges or edits advisories/VEX; it links them and exposes overlays that the Policy Engine evaluates.
|
||||
* Conseiller and Excitator remain authoritative sources; severities come from Policy‑governed normalization.
|
||||
|
||||
### 3.6 APIs (representative)
|
||||
|
||||
* `GET /graph/search?q=...&type=package|artifact|advisory|license`
|
||||
* `POST /graph/query` ⇒ stream tiles `{nodes[], edges[], stats, cursor}`
|
||||
* `POST /graph/paths` body: `{from, to, depth<=6, constraints{scope, runtime_only}}`
|
||||
* `POST /graph/diff` body: `{sbom_a, sbom_b, filters}`
|
||||
* `GET /graph/snapshots/{sbom_id}` ⇒ graph metadata, counts, top advisories
|
||||
* `POST /graph/export` body: `{format: graphml|csv|ndjson|png|svg, query|snapshot}`
|
||||
* `GET /graph/saved` / `POST /graph/saved` save and list tenant queries
|
||||
* `GET /graph/overlays/policy/{version_id}` ⇒ summary stats for caching
|
||||
|
||||
All endpoints tenant‑scoped, RBAC‑checked. Timeouts and pagination by server. Errors return structured diagnostics.
|
||||
|
||||
### 3.7 CLI
|
||||
|
||||
```
|
||||
stella sbom graph search "purl:pkg:maven/org.apache.logging.log4j/log4j-core"
|
||||
stella sbom graph query --file ./query.json --export graphml > graph.graphml
|
||||
stella sbom graph impacted --advisory GHSA-xxxx --runtime-only --limit 100
|
||||
stella sbom graph paths --from artifact:service-a --to advisory:GHSA-xxxx --depth 5 --policy 1.3.0
|
||||
stella sbom graph diff --sbom-a 2025-03-15T10:00Z --sbom-b 2025-03-22T10:00Z --export csv > diff.csv
|
||||
stella sbom graph save --name "openssl-runtime" --file ./query.json
|
||||
```
|
||||
|
||||
Exit codes: 0 ok, 2 query validation error, 3 over‑budget, 4 not found, 5 RBAC denied.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
### 3.8 Performance & scale
|
||||
|
||||
* **Progressive loading**: server pages tiles by BFS frontier; client renders incrementally.
|
||||
* **Viewport culling**: only visible nodes/edges in canvas; offscreen demoted to aggregates.
|
||||
* **Level‑of‑detail**: simplified glyphs and collapsed clusters at distance.
|
||||
* **Query budgets**: per‑tenant rate + node/edge caps; interactive paths limited to depth ≤ 6.
|
||||
* **Caching**: hot queries memoized per tenant + overlay version; diffs precomputed for consecutive SBOMs.
|
||||
|
||||
### 3.9 Security
|
||||
|
||||
* Multi‑tenant isolation at storage and API layers.
|
||||
* RBAC roles:
|
||||
|
||||
* **Viewer**: browse graphs, saved queries
|
||||
* **Investigator**: run queries, export data
|
||||
* **Operator**: configure budgets, purge caches
|
||||
* **Auditor**: download evidence bundles
|
||||
* Input validation for query JSON; deny disallowed edge traversals; strict CSP in web app.
|
||||
|
||||
### 3.10 Observability
|
||||
|
||||
* Metrics: tile latency, nodes/edges per tile, cache hit rate, query denials, memory pressure.
|
||||
* Logs: structured, include query hash, cost, truncation flags.
|
||||
* Traces: server spans per stage (parse, plan, fetch, overlay, stream).
|
||||
|
||||
### 3.11 Accessibility & UX guarantees
|
||||
|
||||
* Keyboard complete, ARIA roles for graph and panels, high‑contrast theme.
|
||||
* Deterministic layout on reload for shareable investigations.
|
||||
|
||||
### 3.12 Data retention
|
||||
|
||||
* Graph nodes derived from SBOMs share retention with SBOM artifacts; overlays are ephemeral caches.
|
||||
* Saved queries retained until deleted; references to missing objects show warnings.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **Graph Indexer (new microservice)**
|
||||
|
||||
* Subscribes to SBOM ingest events, Conseiller advisory updates, Excitator VEX updates, Policy overlay materializations.
|
||||
* Builds adjacency lists and node documents; computes aggregates and clusters.
|
||||
|
||||
* **Graph API (new microservice)**
|
||||
|
||||
* Validates and executes queries; streams tiles; composes overlays; serves diffs and exports.
|
||||
* Integrates with Policy Engine for explain sample retrieval.
|
||||
|
||||
* **SBOM Service (existing)**
|
||||
|
||||
* Emits ingestion events with SBOM ids and lineage; exposes SBOM metadata to Graph API.
|
||||
|
||||
* **Web API Gateway**
|
||||
|
||||
* Routes `/graph/*`, injects tenant context, enforces RBAC.
|
||||
|
||||
### 4.2 Console (Web UI) feature module
|
||||
|
||||
* `packages/features/graph-explorer`
|
||||
|
||||
* Canvas renderer (WebGL), panels, query builder, diff mode, overlays, exports.
|
||||
* Deep‑link router and viewport state serializer.
|
||||
|
||||
### 4.3 Workers
|
||||
|
||||
* Centrality/clustering worker, diff worker, overlay materialization worker.
|
||||
* Schedules on low‑traffic windows; backpressure aware.
|
||||
|
||||
### 4.4 Data model (storage)
|
||||
|
||||
* Collections:
|
||||
|
||||
* `graph_nodes`: `{_id, tenant, type, natural_id, attrs, degree, created_at, updated_at}`
|
||||
* `graph_edges`: `{_id, tenant, from_id, to_id, type, attrs, valid_from, valid_to}`
|
||||
* `graph_snapshots`: per‑SBOM node/edge references
|
||||
* `graph_saved_queries`: `{_id, tenant, name, query_json, created_by}`
|
||||
* `graph_overlays_cache`: keyed by `{tenant, policy_version, hash(query)}`
|
||||
* Indexes: compound on `{tenant, type, natural_id}`, `{tenant, from_id}`, `{tenant, to_id}`, time bounds.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. **`/docs/sbom/graph-explorer-overview.md`**
|
||||
|
||||
* Concepts, node/edge taxonomy, lenses, overlays, roles, limitations.
|
||||
2. **`/docs/sbom/graph-using-the-console.md`**
|
||||
|
||||
* Walkthroughs: search, navigate, impact, diff, export; screenshots and keyboard cheatsheet.
|
||||
3. **`/docs/sbom/graph-query-language.md`**
|
||||
|
||||
* JSON schema, examples, constraints, cost/budget rules.
|
||||
4. **`/docs/sbom/graph-api.md`**
|
||||
|
||||
* REST endpoints, request/response examples, streaming and pagination.
|
||||
5. **`/docs/sbom/graph-cli.md`**
|
||||
|
||||
* CLI command reference and example pipelines.
|
||||
6. **`/docs/policy/graph-overlays.md`**
|
||||
|
||||
* How policy versions render in Graph; explain sampling; AOC guardrails.
|
||||
7. **`/docs/vex/graph-integration.md`**
|
||||
|
||||
* How VEX suppressions appear and how to validate product scoping.
|
||||
8. **`/docs/advisories/graph-integration.md`**
|
||||
|
||||
* Advisory linkage and severity normalization by policy.
|
||||
9. **`/docs/architecture/graph-services.md`**
|
||||
|
||||
* Graph Indexer, Graph API, storage choices, failure modes.
|
||||
10. **`/docs/observability/graph-telemetry.md`**
|
||||
|
||||
* Metrics, logs, tracing, dashboards.
|
||||
11. **`/docs/runbooks/graph-incidents.md`**
|
||||
|
||||
* Handling runaway queries, cache poisoning, degraded render.
|
||||
12. **`/docs/security/graph-rbac.md`**
|
||||
|
||||
* Permissions matrix, multi‑tenant boundaries.
|
||||
|
||||
Every doc should end with a “Compliance checklist.”
|
||||
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Tasks
|
||||
|
||||
### 6.1 Backend: Graph Indexer
|
||||
|
||||
* [ ] Define node/edge schemas and attribute dictionaries for each type.
|
||||
* [ ] Implement event consumers for SBOM ingests, Conseiller updates, Excitator updates.
|
||||
* [ ] Build ingestion pipeline that populates nodes/edges and maintains `valid_from/valid_to`.
|
||||
* [ ] Implement aggregate counters and degree metrics.
|
||||
* [ ] Implement clustering job and persist cluster ids per node.
|
||||
* [ ] Implement snapshot materialization per SBOM and lineage tracking.
|
||||
* [ ] Unit tests for each node/edge builder; property‑based tests for identity stability.
|
||||
|
||||
### 6.2 Backend: Graph API
|
||||
|
||||
* [ ] Implement `/graph/search` with prefix and exact match across node types.
|
||||
* [ ] Implement `/graph/query` with validation, planning, cost estimation, and streaming tile results.
|
||||
* [ ] Implement `/graph/paths` with constraints and depth limits; shortest path heuristic.
|
||||
* [ ] Implement `/graph/diff` computing adds/removes/changed versions; stream results.
|
||||
* [ ] Implement overlays: advisory join, VEX join, policy materialization and explain sampling.
|
||||
* [ ] Implement exports: GraphML, CSV edge list, NDJSON findings, PNG/SVG snapshots.
|
||||
* [ ] RBAC middleware integration; multi‑tenant scoping.
|
||||
* [ ] Load tests with synthetic large SBOMs; define default budgets.
|
||||
|
||||
### 6.3 Policy Engine integration
|
||||
|
||||
* [ ] Add endpoint to fetch explain traces for specific node ids in batch.
|
||||
* [ ] Add materialization export that Graph API can cache per policy version.
|
||||
|
||||
### 6.4 Console (Web UI)
|
||||
|
||||
* [ ] Create `graph-explorer` module with routes `/graph`, `/graph/diff`, `/graph/q/:id`.
|
||||
* [ ] Implement WebGL canvas with LOD, culling, edge bundling, deterministic layout seed.
|
||||
* [ ] Build search, filter, lens, and overlay toolbars.
|
||||
* [ ] Side panels: details, evidence tabs, explain viewer.
|
||||
* [ ] Diff mode: split/overlay toggles and color legend.
|
||||
* [ ] Saved queries: create, update, run; deep links.
|
||||
* [ ] Export UI: formats, server round‑trip, progress indicators.
|
||||
* [ ] a11y audit and keyboard‑only flow.
|
||||
|
||||
### 6.5 CLI
|
||||
|
||||
* [ ] Implement `stella sbom graph *` subcommands with JSON IO and piping support.
|
||||
* [ ] Document examples and stable output schemas for CI consumption.
|
||||
|
||||
### 6.6 Observability & Ops
|
||||
|
||||
* [ ] Dashboards for tile latency, query denials, cache hit rate, memory.
|
||||
* [ ] Alerting on query error spikes, OOM risk, cache churn.
|
||||
* [ ] Runbooks in `/docs/runbooks/graph-incidents.md`.
|
||||
|
||||
### 6.7 Docs
|
||||
|
||||
* [ ] Author all docs in section 5, link from Console contextual help.
|
||||
* [ ] Add end‑to‑end tutorial: “Investigate GHSA‑XXXX across prod artifacts.”
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Console renders large SBOM graphs with semantic zoom, overlays, and responsive interactions.
|
||||
* Users can run impact and path queries with bounded depth and get results within budget.
|
||||
* VEX suppressions and advisory severities appear correctly and are consistent with policy.
|
||||
* Diff view clearly shows added/removed/changed nodes/edges between two SBOMs.
|
||||
* Saved queries and deep links reproduce the same view deterministically (given same data).
|
||||
* Exports produce valid GraphML/CSV/NDJSON and image snapshots.
|
||||
* CLI supports search, query, paths, impacted, diff, and export with stable schemas.
|
||||
* AOC guardrails: explorer never mutates facts; overlays reflect Policy Engine decisions.
|
||||
* RBAC enforced; all actions logged and observable.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks & mitigations
|
||||
|
||||
* **Graph explosion on large monorepos** → tiling, clustering, budgets, and strict depth limits.
|
||||
* **Inconsistent identities across tools** → canonicalize purls/digests; property‑based tests for identity stability.
|
||||
* **Policy overlay latency** → precompute materializations for hot policy versions; sample explains only on focus.
|
||||
* **User confusion** → strong lens defaults, deterministic layouts, legends, in‑context help.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit**: node/edge builders, identity normalization, cost estimator.
|
||||
* **Integration**: ingest SBOM + advisories + VEX, verify overlays and counts.
|
||||
* **E2E**: Playwright flows for search→impact→diff→export; deep link determinism.
|
||||
* **Performance**: simulate 500k nodes/2M edges; measure tile latency and memory.
|
||||
* **Security**: RBAC matrix; tenant isolation tests; query validation fuzzing.
|
||||
* **Determinism**: snapshot round‑trip: same query and seed produce identical layout and stats.
|
||||
|
||||
---
|
||||
|
||||
## 10) Feature flags
|
||||
|
||||
* `graph.explorer` (UI feature module)
|
||||
* `graph.paths` (advanced path queries)
|
||||
* `graph.diff` (SBOM diff mode)
|
||||
* `graph.overlays.policy` (policy overlay + explain sampling)
|
||||
* `graph.export` (exports enabled)
|
||||
|
||||
Documented in `/docs/observability/graph-telemetry.md`.
|
||||
|
||||
---
|
||||
|
||||
## 11) Non‑goals (this epic)
|
||||
|
||||
* Real‑time process/runtime call graphs.
|
||||
* Full substitution for text reports; Explorer complements Reports.
|
||||
* Cross‑tenant graphs; all queries are tenant‑scoped.
|
||||
|
||||
---
|
||||
|
||||
## 12) Philosophy
|
||||
|
||||
* **See the system**: security and license risk are structural. If you cannot see structure, you will miss risk.
|
||||
* **Evidence over assertion**: every colored node corresponds to raw facts and explainable determinations.
|
||||
* **Bounded interactivity**: fast, partial answers beat slow “complete” ones.
|
||||
* **Immutability**: graphs mirror SBOM snapshots and are never rewritten; we add context, not edits.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
650
docs/implplan/EPIC_6.md
Normal file
650
docs/implplan/EPIC_6.md
Normal file
@@ -0,0 +1,650 @@
|
||||
Below is the expanded, “maximum documentation” package for Epic 6. It is paste‑ready for your repo and deliberately formal so engineering, docs, and audit folks can work from the same source of truth.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 6: Vulnerability Explorer (policy‑aware)
|
||||
|
||||
**Short name:** `Vuln Explorer`
|
||||
**Services touched:** Conseiller (Feedser), Excitator (Vexer), SBOM Service, Policy Engine, Findings Ledger (new), Web API Gateway, Authority (authN/Z), Workers/Scheduler, Telemetry/Analytics, Console (Web UI), CLI
|
||||
**AOC ground rule:** Conseiller/Excitator aggregate but never merge or rewrite source documents. The Explorer only renders **effective** results as decided by Policy Engine and records human workflow as immutable ledger events.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The Vulnerability Explorer is the API and UI for operational triage, investigation, and reporting of vulnerabilities across all artifacts tracked by StellaOps. It correlates SBOM inventory with advisory and VEX evidence, then displays the **effective** status and obligations per the currently selected **Policy version**. It never edits source evidence. It provides:
|
||||
|
||||
* Policy‑aware lists, pivots, and detail views
|
||||
* Triage workflow with immutable audit ledger
|
||||
* Risk acceptance with expiry and evidence
|
||||
* SLA tracking by severity and business tier
|
||||
* Simulation against other policy versions
|
||||
* Exports and cryptographically signed “evidence bundles”
|
||||
|
||||
**Identity principle:** one **Finding** per tuple `(artifact_id, purl, version, advisory_key)`, with links to every contributing AdvisoryEvidence and VexEvidence.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (concise)
|
||||
|
||||
* Prioritization must reflect **policy and VEX**, not raw feeds.
|
||||
* Audit requires complete, reproducible lineage: what changed, why, and who decided.
|
||||
* Operators need consistent APIs, CLI, and a UI that explain determinations, not just list CVEs.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Domain model and contracts
|
||||
|
||||
**Evidence (immutable)**
|
||||
|
||||
* `AdvisoryEvidence` (from Conseiller)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "adv_evd:...uuid...",
|
||||
"tenant": "acme",
|
||||
"source": "ghsa|nvd|vendor|ossindex|... ",
|
||||
"source_id": "GHSA-xxxx",
|
||||
"schema": "GHSA|CVE|CSAF-ADV",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"affected": [{"ecosystem":"npm","purl":"pkg:npm/lodash","ranges":[{"type":"semver","events":[{"introduced":"0.0.0"},{"fixed":"4.17.21"}]}]}],
|
||||
"cvss": {"version":"3.1","baseScore":7.5,"vectorString":"AV:N/AC:L/..."},
|
||||
"severity": "HIGH",
|
||||
"urls": ["https://..."],
|
||||
"published": "2024-06-10T12:00:00Z",
|
||||
"withdrawn": null,
|
||||
"ingested_at": "2024-06-11T08:43:21Z"
|
||||
}
|
||||
```
|
||||
* `VexEvidence` (from Excitator)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "vex_evd:...uuid...",
|
||||
"tenant": "acme",
|
||||
"source": "vendor-csaf",
|
||||
"schema": "CSAF-VEX",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"product_scope": [{"purl":"pkg:npm/lodash@4.17.20"}],
|
||||
"status": "not_affected|affected|fixed|under_investigation",
|
||||
"justification": "component_not_present|vulnerable_code_not_in_execute_path|... ",
|
||||
"impact_statement": "Not reachable in Acme Payment Service.",
|
||||
"timestamp": "2024-06-12T10:00:00Z",
|
||||
"ingested_at": "2024-06-12T10:10:02Z"
|
||||
}
|
||||
```
|
||||
* `InventoryEvidence` (from SBOM Service)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "inv_evd:...uuid...",
|
||||
"tenant": "acme",
|
||||
"artifact_id": "svc:payments@1.14.0",
|
||||
"sbom_id": "sbom:sha256:...",
|
||||
"purl": "pkg:npm/lodash@4.17.20",
|
||||
"scope": "runtime|dev|test",
|
||||
"runtime_flag": true,
|
||||
"paths": [["root", "pkg:npm/a", "pkg:npm/lodash"]],
|
||||
"discovered_at": "2024-06-12T11:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**PolicyDetermination** (read‑only from Policy Engine)
|
||||
|
||||
```json
|
||||
{
|
||||
"key": {
|
||||
"artifact_id": "svc:payments@1.14.0",
|
||||
"purl": "pkg:npm/lodash",
|
||||
"version": "4.17.20",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"policy_version": "1.3.0"
|
||||
},
|
||||
"applicable": true,
|
||||
"effective_severity": "HIGH",
|
||||
"exploitability": "ACTIVE|LIKELY|UNKNOWN|UNLIKELY",
|
||||
"signals": {"epss": 0.86, "kev": true, "maturity": "weaponized"},
|
||||
"suppression_state": "none|policy|vex",
|
||||
"obligations": [{"type":"fix_by","due":"2024-07-15"},{"type":"document_risk"}],
|
||||
"sla": {"due":"2024-07-15","tier":"gold"},
|
||||
"rationale": [
|
||||
{"rule":"sev.base=nvd","detail":"NVD base:7.5"},
|
||||
{"rule":"exploit.upsell.kev","detail":"KEV flag → HIGH"},
|
||||
{"rule":"env.weight.prod","detail":"Env=prod no downgrade"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Finding identity**
|
||||
|
||||
```
|
||||
finding_id = hash(tenant, artifact_id, purl, version, advisory_key)
|
||||
```
|
||||
|
||||
**Ledger event** (append‑only, tamper‑evident)
|
||||
|
||||
```json
|
||||
{
|
||||
"event_id": "led:...uuid...",
|
||||
"finding_id": "f:...hash...",
|
||||
"tenant": "acme",
|
||||
"type": "assign|comment|attach|change_state|accept_risk|set_target_fix|verify_fix|reopen",
|
||||
"payload": {"to":"team:platform","reason":"oncall triage"},
|
||||
"actor": {"user_id":"u:42","display":"Dana S."},
|
||||
"ts": "2024-06-12T14:01:02Z",
|
||||
"prev_event_hash": "sha256:...",
|
||||
"event_hash": "sha256:sha256(canonical_json(event) + prev_event_hash)"
|
||||
}
|
||||
```
|
||||
|
||||
**Projection** (materialized current state for fast lists)
|
||||
|
||||
```json
|
||||
{
|
||||
"finding_id":"f:...hash...",
|
||||
"tenant":"acme",
|
||||
"artifact_id":"svc:payments@1.14.0",
|
||||
"purl":"pkg:npm/lodash",
|
||||
"version":"4.17.20",
|
||||
"advisory_key":"CVE-2024-12345",
|
||||
"effective_severity":"HIGH",
|
||||
"exploitability":"ACTIVE",
|
||||
"suppression_state":"none",
|
||||
"status":"UNDER_INVESTIGATION|REMEDIATING|ACCEPTED|RESOLVED|NEW|SUPPRESSED",
|
||||
"sla_due":"2024-07-15",
|
||||
"owner":"team:platform",
|
||||
"kev":true,
|
||||
"epss":0.86,
|
||||
"new_since":"2024-06-12",
|
||||
"has_fix":true,
|
||||
"envs":["prod"],
|
||||
"runtime_flag":true,
|
||||
"updated_at":"2024-06-12T14:01:02Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Advisory key normalization**
|
||||
|
||||
* Input identifiers: `CVE-*`, `GHSA-*`, vendor IDs.
|
||||
* Preference order: CVE, then GHSA, else vendor id prefixed with namespace.
|
||||
* Canonicalization: uppercase, trim, map withdrawn to same key but mark `withdrawn=true` in evidence.
|
||||
* Conseiller must publish `links: [all source ids]` for provenance.
|
||||
|
||||
### 3.2 Resolver algorithm (candidate findings)
|
||||
|
||||
**Goal:** produce tuples `(artifact_id, purl, version, advisory_key)` where inventory intersects affected ranges and policy deems the path relevant.
|
||||
|
||||
**Pseudocode**
|
||||
|
||||
```
|
||||
for each artifact sbom S:
|
||||
inv = inventory(S)
|
||||
for each advisory evidence A:
|
||||
for each affected package spec in A.affected:
|
||||
for each inv_item in inv where inv_item.purl package == spec.purl package:
|
||||
if version_in_ranges(inv_item.version, spec.ranges, ecosystem):
|
||||
if policy.path_scope_allows(inv_item.scope, inv_item.runtime_flag, inv_item.paths):
|
||||
yield candidate (artifact_id, inv_item.purl, inv_item.version, A.advisory_key)
|
||||
```
|
||||
|
||||
**Version semantics per ecosystem**
|
||||
|
||||
* npm: semver, pre-release excluded unless explicitly in range.
|
||||
* Maven: Maven version rules, handle `-SNAPSHOT`, use maven‑resolver semantics.
|
||||
* PyPI: PEP 440 versioning.
|
||||
* Go: semver with `+incompatible` handling.
|
||||
* OS packages: RPM/DEB epoch:version‑release ordering.
|
||||
|
||||
**Edge cases**
|
||||
|
||||
* Multiple paths: store shortest path and count.
|
||||
* Dev/test scope: policy may exclude or downgrade.
|
||||
* Withdrawn advisories: keep as evidence; Policy can set severity to `NONE`.
|
||||
|
||||
### 3.3 VEX precedence and scoping
|
||||
|
||||
* If any matching **VEX** says `not_affected` scoped to the artifact product/component per CSAF product tree, set `suppression_state="vex"` and `applicable=false`.
|
||||
* If VEX says `fixed` and inventory version is >= fixed version, mark as **Resolved (verified)** after SBOM recrawl confirms.
|
||||
* If VEX `under_investigation`, no suppression; may add a policy grace period obligation.
|
||||
|
||||
### 3.4 Policy evaluation
|
||||
|
||||
* Inputs: candidate tuple + context (artifact env, business tier, ownership, signals, fix availability).
|
||||
* Determinations: applicability, effective severity, exploitability, obligations, SLA, suppression.
|
||||
* Suppression by policy examples: test‑scope only; path through optional deps; package vendored but not linked at runtime.
|
||||
* Simulation: identical input, alternate `policy_version`; returns determinations without side effects.
|
||||
|
||||
### 3.5 API surface (authoritative)
|
||||
|
||||
**List**
|
||||
|
||||
```
|
||||
GET /vuln/findings?policy=1.3.0&sev=high,critical&group_by=artifact&exploit=kev&env=prod&page=1&page_size=100
|
||||
```
|
||||
|
||||
Response
|
||||
|
||||
```json
|
||||
{
|
||||
"page": 1,
|
||||
"page_size": 100,
|
||||
"total": 740,
|
||||
"group_by": "artifact",
|
||||
"results": [
|
||||
{"group":"svc:payments","counts":{"CRITICAL":3,"HIGH":12,"MEDIUM":8},"sla_breaches":2},
|
||||
...
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Query (complex filters)**
|
||||
|
||||
```
|
||||
POST /vuln/findings/query
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"policy": "1.3.0",
|
||||
"filter": {
|
||||
"severity": [">=MEDIUM"],
|
||||
"exploit": ["kev", "epss>=0.8"],
|
||||
"artifact": ["svc:payments", "svc:checkouts"],
|
||||
"status": ["NEW","UNDER_INVESTIGATION"],
|
||||
"env": ["prod"]
|
||||
},
|
||||
"sort": [{"field":"effective_severity","dir":"desc"},{"field":"epss","dir":"desc"}],
|
||||
"page": {"number":1,"size":200}
|
||||
}
|
||||
```
|
||||
|
||||
**Detail**
|
||||
|
||||
```
|
||||
GET /vuln/findings/{finding_id}?policy=1.3.0
|
||||
```
|
||||
|
||||
Returns projection, evidence links, policy rationale, paths, history summary.
|
||||
|
||||
**Workflow**
|
||||
|
||||
```
|
||||
POST /vuln/findings/{id}/assign { "to": "team:platform" }
|
||||
POST /vuln/findings/{id}/comment { "text": "triage notes..." }
|
||||
POST /vuln/findings/{id}/accept-risk { "until":"2025-06-30","reason":"vendor patch pending","evidence":["url|upload_id"] }
|
||||
POST /vuln/findings/{id}/verify-fix { "sbom_id": "sbom:sha256:..." }
|
||||
POST /vuln/findings/{id}/target-fix { "version": "4.17.21" }
|
||||
```
|
||||
|
||||
**Simulation**
|
||||
|
||||
```
|
||||
POST /vuln/simulate
|
||||
{
|
||||
"policy_from": "1.3.0",
|
||||
"policy_to": "1.4.0",
|
||||
"query": { "severity":[">=MEDIUM"], "env":["prod"] }
|
||||
}
|
||||
```
|
||||
|
||||
Response includes per‑finding delta `{before, after, diff}`.
|
||||
|
||||
**Export**
|
||||
|
||||
```
|
||||
POST /vuln/export { "format":"ndjson","scope":{"query":{...}} }
|
||||
```
|
||||
|
||||
Returns a signed bundle (see §3.10).
|
||||
|
||||
**Errors**
|
||||
|
||||
* `400` validation, `403` RBAC, `404` not found, `409` state conflict (idempotency), `429` rate limited, `5xx` server.
|
||||
|
||||
### 3.6 Console (Web UI)
|
||||
|
||||
**Routes**
|
||||
|
||||
* `/vuln` list with saved views
|
||||
* `/vuln/:id` detail drawer state
|
||||
* `/vuln/simulate/:policyVersion` diff mode
|
||||
|
||||
**State shape (client)**
|
||||
|
||||
```ts
|
||||
interface VulnListState {
|
||||
policyVersion: string;
|
||||
filters: {...};
|
||||
sort: [...];
|
||||
columns: string[];
|
||||
viewId?: string;
|
||||
page: {number: number; size: number};
|
||||
}
|
||||
```
|
||||
|
||||
**UX**
|
||||
|
||||
* Virtualized grid with server paging; column chooser; density toggle.
|
||||
* Quick filters: severity, exploit signals, status, env, owner, fix availability.
|
||||
* Detail tabs: Summary, Evidence (raw docs with provenance), Policy (rationale chain), Paths (deep link to Graph Explorer), Fixes, History.
|
||||
* Simulation bar shows delta chips: `+21 HIGH`, `-9 Suppressed by VEX` etc.
|
||||
* Evidence bundle dialog previews scope and size.
|
||||
* a11y: ARIA roles on grid, keyboard shortcuts: `A` assign, `C` comment, `R` accept risk, `V` verify fix.
|
||||
|
||||
### 3.7 CLI
|
||||
|
||||
Commands
|
||||
|
||||
```
|
||||
stella vuln list --policy 1.3.0 --sev high,critical --group-by artifact --env prod --json
|
||||
stella vuln show --id <finding-id> --policy 1.3.0
|
||||
stella vuln simulate --from 1.3.0 --to 1.4.0 --sev '>=medium' --delta --json
|
||||
stella vuln assign --filter 'advisory:CVE-2024-12345 artifact:payments' --to team:platform
|
||||
stella vuln accept-risk --id <finding-id> --until 2025-06-30 --reason "vendor patch pending" --evidence url:https://ticket/123
|
||||
stella vuln verify-fix --id <finding-id> --sbom <sbom-id>
|
||||
```
|
||||
|
||||
Return codes: `0 ok`, `2 invalid args`, `3 budget exceeded`, `4 not found`, `5 denied`.
|
||||
|
||||
### 3.8 Storage schema (illustrative)
|
||||
|
||||
**Tables**
|
||||
|
||||
```sql
|
||||
-- Evidence
|
||||
CREATE TABLE evidence_advisory (...);
|
||||
CREATE INDEX ea_tenant_key ON evidence_advisory(tenant, advisory_key);
|
||||
CREATE TABLE evidence_vex (...);
|
||||
CREATE INDEX ev_tenant_key ON evidence_vex(tenant, advisory_key);
|
||||
CREATE TABLE evidence_inventory (...);
|
||||
CREATE INDEX ei_artifact_purl ON evidence_inventory(tenant, artifact_id, purl);
|
||||
|
||||
-- Ledger
|
||||
CREATE TABLE findings_ledger_events (
|
||||
event_id uuid PRIMARY KEY,
|
||||
finding_id bytea NOT NULL,
|
||||
tenant text NOT NULL,
|
||||
type text NOT NULL,
|
||||
payload jsonb NOT NULL,
|
||||
actor jsonb NOT NULL,
|
||||
ts timestamptz NOT NULL,
|
||||
prev_event_hash bytea,
|
||||
event_hash bytea NOT NULL
|
||||
);
|
||||
CREATE INDEX fle_find_ts ON findings_ledger_events(tenant, finding_id, ts);
|
||||
|
||||
-- Projection
|
||||
CREATE TABLE findings_projection (
|
||||
finding_id bytea PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
artifact_id text NOT NULL,
|
||||
purl text NOT NULL,
|
||||
version text NOT NULL,
|
||||
advisory_key text NOT NULL,
|
||||
policy_version text NOT NULL,
|
||||
effective_severity text NOT NULL,
|
||||
exploitability text,
|
||||
suppression_state text,
|
||||
status text NOT NULL,
|
||||
sla_due date,
|
||||
owner text,
|
||||
kev boolean,
|
||||
epss double precision,
|
||||
envs text[],
|
||||
runtime_flag boolean,
|
||||
updated_at timestamptz NOT NULL
|
||||
);
|
||||
CREATE INDEX fp_query ON findings_projection(tenant, policy_version, effective_severity, status);
|
||||
```
|
||||
|
||||
**Tamper‑evidence**
|
||||
|
||||
* Ledger events use chained SHA‑256 hashes over canonical JSON + previous hash.
|
||||
* Daily Merkle root of all event hashes is anchored to the audit store (and optionally external timestamping service).
|
||||
|
||||
### 3.9 Performance and scaling
|
||||
|
||||
* P95 list endpoint under 600 ms for 100‑row pages at 5M findings/tenant.
|
||||
* Projections denormalize heavy joins; background projector uses idempotent jobs keyed by `(tenant,finding_id,policy_version)`.
|
||||
* Rate limits per tenant and per API key; backpressure on export jobs; exponential retry for projector.
|
||||
|
||||
### 3.10 Evidence bundle format
|
||||
|
||||
* **Container:** ZIP with `manifest.json`, `findings.ndjson`, `advisory_evidence.ndjson`, `vex_evidence.ndjson`, `inventory_evidence.ndjson`, `policy_version.json`, `ledger_events.ndjson`, `CHECKSUMS`.
|
||||
* **Signing:** Detached signature `bundle.sig` using tenant’s org key (Ed25519).
|
||||
* **Manifest**
|
||||
|
||||
```json
|
||||
{"generated_at":"2024-06-12T15:00:00Z","tenant":"acme","policy_version":"1.3.0","scope":{"query":{...}},"counts":{"findings":421}}
|
||||
```
|
||||
|
||||
### 3.11 Observability
|
||||
|
||||
* Metrics (OpenTelemetry):
|
||||
|
||||
* `vuln_findings_list_latency_ms` (histogram)
|
||||
* `vuln_projection_lag_seconds` (gauge)
|
||||
* `vuln_new_findings_total` (counter)
|
||||
* `vuln_sla_breaches_total` (counter by sev, owner)
|
||||
* `vuln_simulation_latency_ms` (histogram)
|
||||
* Logs: structured JSON with `tenant`, `policy_version`, `query_hash`, `result_count`.
|
||||
* Traces: spans for resolver, policy calls, projection builds, export assembly.
|
||||
* PII: redact comments in logs; store attachments encrypted at rest (KMS).
|
||||
|
||||
### 3.12 Security & RBAC
|
||||
|
||||
**Roles**
|
||||
|
||||
* Viewer: GET list/detail/export read scope.
|
||||
* Investigator: Viewer + workflow actions except risk acceptance.
|
||||
* Operator: Investigator + risk acceptance, verify fix, bulk actions.
|
||||
* Auditor: Viewer + evidence bundles and ledger integrity checks.
|
||||
|
||||
**ABAC**
|
||||
|
||||
* Attribute constraints: by `artifact.owner`, `env`, and `business_tier`.
|
||||
* CSRF protection for Console; all POST require anti‑forgery tokens.
|
||||
* Attachments stored with envelope encryption; signed URLs for limited time access.
|
||||
|
||||
### 3.13 Rollout and migrations
|
||||
|
||||
* Feature flags: `vuln.explorer.ui`, `vuln.explorer.simulation`, `vuln.explorer.bulk_actions`, `vuln.explorer.evidence_bundle`.
|
||||
* Phase 1: dark launch API and projections.
|
||||
* Phase 2: UI read‑only list and detail.
|
||||
* Phase 3: workflow actions and exports.
|
||||
* Data backfill: replay advisory/VEX/SBOM events to seed projections.
|
||||
* Compatibility: maintain projection v1 schema for two releases; migration scripts in `/migrations/vuln/`.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **Findings Ledger (new)**
|
||||
|
||||
* Append‑only event store with projector to `findings_projection`.
|
||||
* Event validation and canonicalization; hashing and Merkle root anchoring.
|
||||
|
||||
* **Vuln Explorer API (new)**
|
||||
|
||||
* Query/filter engine with policy parameterization and grouping.
|
||||
* Simulation endpoint.
|
||||
* Export job orchestrator.
|
||||
|
||||
* **Conseiller / Excitator (updates)**
|
||||
|
||||
* Guarantee canonical `advisory_key` and publish `links[]`.
|
||||
* No merges; maintain raw payload snapshots.
|
||||
|
||||
* **Policy Engine (updates)**
|
||||
|
||||
* Batch evaluation endpoint `POST /policy/eval/batch` with `simulate` support.
|
||||
* Return rationale chain with rule IDs.
|
||||
|
||||
* **SBOM Service (updates)**
|
||||
|
||||
* Publish inventory deltas; include `scope`, `runtime_flag`, `paths`.
|
||||
* Nearest safe version hints.
|
||||
|
||||
* **Workers/Scheduler**
|
||||
|
||||
* Resolver job keyed by `(tenant, artifact_id, sbom_id)`; emits candidate tuples.
|
||||
* Recompute on policy activation and evidence changes.
|
||||
|
||||
### 4.2 Code structure
|
||||
|
||||
```
|
||||
/src/Findings/StellaOps.Findings.Ledger
|
||||
/api
|
||||
/projector
|
||||
/storage
|
||||
/src/VulnExplorer/StellaOps.VulnExplorer.Api
|
||||
/routes
|
||||
/query
|
||||
/simulation
|
||||
/export
|
||||
/packages/console/features/vuln-explorer
|
||||
/components
|
||||
/pages
|
||||
/state
|
||||
/src/Cli/StellaOps.Cli
|
||||
```
|
||||
|
||||
### 4.3 Performance tasks
|
||||
|
||||
* Projection indexes and covering queries; explain plans in `/docs/vuln/perf-notes.md`.
|
||||
* Cache hot groupings per tenant with TTL and invalidation on ledger projector tick.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. `/docs/vuln/explorer-overview.md`
|
||||
Conceptual model, identities, evidence vs determinations, AOC guarantees.
|
||||
2. `/docs/vuln/explorer-using-console.md`
|
||||
Workflows with screenshots, keyboard shortcuts, saved views, deep links.
|
||||
3. `/docs/vuln/explorer-api.md`
|
||||
Endpoint specs, query language, grouping, pagination, errors, rate limits.
|
||||
4. `/docs/vuln/explorer-cli.md`
|
||||
Commands, flags, examples, exit codes.
|
||||
5. `/docs/vuln/findings-ledger.md`
|
||||
Event schema, state machine, hashing, Merkle roots, integrity checks.
|
||||
6. `/docs/policy/vuln-determinations.md`
|
||||
Inputs, outputs, precedence rules, simulation semantics.
|
||||
7. `/docs/vex/explorer-integration.md`
|
||||
CSAF mapping, scoping to product tree, precedence.
|
||||
8. `/docs/advisories/explorer-integration.md`
|
||||
Advisory key normalization, provenance, withdrawn handling.
|
||||
9. `/docs/sbom/vuln-resolution.md`
|
||||
Ecosystem version semantics, path sensitivity, scope rules.
|
||||
10. `/docs/observability/vuln-telemetry.md`
|
||||
Metrics, logs, traces, dashboards, SLOs.
|
||||
11. `/docs/security/vuln-rbac.md`
|
||||
Role mapping, ABAC, attachment encryption, CSRF.
|
||||
12. `/docs/runbooks/vuln-ops.md`
|
||||
Recompute storms, projector lag, policy activation drains, export failures.
|
||||
13. `/docs/install/containers.md`
|
||||
Add `findings-ledger`, `vuln-explorer-api` images, compose/k8s manifests, resource sizing, health checks.
|
||||
|
||||
> Each doc ends with: **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend: Findings & API
|
||||
|
||||
* [ ] Define evidence and ledger schemas; migrations.
|
||||
* [ ] Implement resolver for npm, Maven, PyPI, Go, OS packages with property‑based tests for version comparisons.
|
||||
* [ ] Implement ledger API and projector with idempotency and hashing.
|
||||
* [ ] Implement list/detail/grouping endpoints with server‑side paging.
|
||||
* [ ] Implement simulation and export (bundle assembly, signing).
|
||||
* [ ] Integrate Policy Engine batch eval with rationale traces.
|
||||
* [ ] RBAC via Authority with ABAC filters.
|
||||
* [ ] Load tests at 5M findings/tenant; tune indexes.
|
||||
|
||||
### Conseiller/Excitator
|
||||
|
||||
* [ ] Normalize `advisory_key` and persist `links[]`.
|
||||
* [ ] Ensure raw payload snapshots are retrievable by Explorer for Evidence tab.
|
||||
|
||||
### SBOM Service
|
||||
|
||||
* [ ] Emit `scope`, `runtime_flag`, `paths`; safe version hints.
|
||||
* [ ] Inventory delta events to trigger resolver.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Build grid with virtualization, saved views, deep link serializer.
|
||||
* [ ] Implement detail tabs and path deep‑links to Graph Explorer.
|
||||
* [ ] Add simulation bar and delta chips.
|
||||
* [ ] Evidence bundle dialog.
|
||||
* [ ] a11y keyboard flow and ARIA labeling; unit and E2E tests.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella vuln list|show|simulate|assign|accept-risk|verify-fix` with `--json` and CSV export.
|
||||
* [ ] Stable output schemas; pipe‑friendly defaults.
|
||||
|
||||
### Observability/Ops
|
||||
|
||||
* [ ] Dashboards for list latency, projection lag, new/reopened, SLA breaches.
|
||||
* [ ] Alerts on projector backlog, API 5xx spikes, export failures.
|
||||
* [ ] Runbooks in `/docs/runbooks/vuln-ops.md`.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author files listed in §5 with cross‑links to Policy Studio and SBOM Graph Explorer.
|
||||
* [ ] Update `/docs/install/containers.md` with new images and compose/k8s snippets.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* List and detail views reflect **effective** policy outcomes and update instantly when switching policy versions.
|
||||
* Evidence tab shows all raw advisory/VEX documents with provenance; no source merging.
|
||||
* Resolver respects ecosystem semantics, scope, and paths; path tab round‑trips to Graph Explorer.
|
||||
* Ledger events are immutable and reconstruct historical list states accurately.
|
||||
* Simulation returns diffs without side effects and matches Policy Engine outputs.
|
||||
* CLI/API support paging, grouping, export, simulation; contracts stable and documented.
|
||||
* RBAC and tenant isolation validated by tests; attachments encrypted.
|
||||
* P95 performance budgets met; dashboards green for SLOs.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks and mitigations
|
||||
|
||||
* **Advisory identity collisions** → strict canonicalization; preserve `links[]`; never merge raw docs.
|
||||
* **Projection lag** → backpressure, worker autoscaling, health checks; alerting on lag.
|
||||
* **Resolver false positives** → path evidence required; dev/test scope rules explicit; ecosystem‑specific tests.
|
||||
* **User confusion over suppression** → explicit badges; Policy tab with rationale and “why changed.”
|
||||
* **Export size** → NDJSON streaming, size estimator in UI, scope previews.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit:** version comparators, resolver per ecosystem, policy mapping, ledger state machine.
|
||||
* **Integration:** SBOM + advisories + VEX ingestion, candidate generation, policy application, suppression precedence.
|
||||
* **E2E Console:** triage, bulk assign, simulation, evidence bundle download; keyboard‑only flow.
|
||||
* **Performance:** list/grouping at target scale; projector rebuild; export assembly.
|
||||
* **Security:** RBAC matrix, ABAC filters, CSRF, signed URL lifetimes, tamper‑evidence verification.
|
||||
* **Determinism:** time‑travel snapshots reproduce prior states byte‑for‑byte.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Facts first, decisions second.** Evidence is immutable; decisions and workflow sit on top in a ledger.
|
||||
* **Policy is the lens.** The same facts can imply different obligations; the system must make that explicit and reproducible.
|
||||
* **Audit > convenience.** Every state change is justified, signed, and verifiable.
|
||||
* **No hidden magic.** If anything is suppressed, the UI shows the rule or VEX that did it, with documents attached.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
545
docs/implplan/EPIC_7.md
Normal file
545
docs/implplan/EPIC_7.md
Normal file
@@ -0,0 +1,545 @@
|
||||
Below is the expanded, “maximum documentation” package for Epic 7. It is paste‑ready for your repo and deliberately formal so engineering, docs, governance, and audit can work from the same source of truth.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 7: VEX Consensus Lens
|
||||
|
||||
**Short name:** `VEX Lens`
|
||||
**Services touched:** Excitator (Vexer), Conseiller (Feedser), SBOM Service, Policy Engine, Findings Ledger, Web API Gateway, Authority (authN/Z), Console (Web UI), CLI, Telemetry/Analytics
|
||||
**AOC ground rule:** Excitator and Conseiller aggregate but never merge or rewrite source documents. The Lens only computes a **derived, reproducible** consensus view while preserving all raw evidence and provenance.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The VEX Consensus Lens is a deterministic computation and presentation layer that ingests all available VEX statements about a given advisory key (e.g., CVE) and product/context, then produces an **explicit consensus state** for each `(artifact_id, purl, version, advisory_key)` tuple. It answers: “Given multiple, possibly conflicting VEX statements from different issuers, what is the most trustworthy, policy‑consistent interpretation for this artifact and version?”
|
||||
|
||||
It never edits or merges VEX documents. It normalizes them, aligns their **product scope** to SBOM inventory, scores issuers via a tenant‑configurable **trust model**, applies time and version scoping, and outputs:
|
||||
|
||||
* `consensus_state`: `NOT_AFFECTED | AFFECTED | FIXED | UNDER_INVESTIGATION | INCONCLUSIVE | DISPUTED`
|
||||
* `confidence`: 0.0–1.0 numeric score
|
||||
* `rationale`: structured explanation of evidence, weights, and rules applied
|
||||
* `quorum`: weighted vote breakdown by issuer, timestamp, justification, and version applicability
|
||||
|
||||
The Lens plugs into the Policy Engine. Policy remains the source of truth for applicability, severity, and obligations; the Lens supplies structured signals and provenance that policy can use to suppress findings, downgrade risk, or set SLAs.
|
||||
|
||||
Key properties:
|
||||
|
||||
* **Reproducible:** Same inputs and policy yield same outputs.
|
||||
* **Explainable:** Every output carries a machine‑readable rationale chain.
|
||||
* **Scoped:** Product trees and version ranges are resolved against actual SBOM inventory.
|
||||
* **Immutable evidence:** All raw VEX docs remain intact, retrievable, and linkable.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (concise)
|
||||
|
||||
Real ecosystems have overlapping VEX statements from upstream maintainers, vendors, distros, and third parties. Operators need a single, defensible view without losing provenance. The Lens reduces noise, flags conflicts, and turns fragmented VEX evidence into auditable signals that policy and humans can act on.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Input model
|
||||
|
||||
**From Excitator (Vexer):**
|
||||
|
||||
* Raw `VexEvidence` documents (CSAF VEX, OpenVEX, CycloneDX VEX) with:
|
||||
|
||||
* `advisory_key` (canonicalized)
|
||||
* product tree or component coordinates (purl, CPE, vendor IDs)
|
||||
* status (`not_affected`, `affected`, `fixed`, `under_investigation`)
|
||||
* justifications (CSAF, OpenVEX enumerations)
|
||||
* version ranges or fixed versions if available
|
||||
* timestamps (issued, last_updated), source URLs
|
||||
* cryptographic metadata (signature, issuer, certificate chain) if present
|
||||
|
||||
**From SBOM Service:**
|
||||
|
||||
* Inventory for each artifact: purls, versions, dependency paths, scopes, env flags.
|
||||
|
||||
**From Issuer Directory (new)**
|
||||
|
||||
* Directory of known VEX issuers:
|
||||
|
||||
* identity, organization, domain, CSAF publisher metadata
|
||||
* public keys and trust anchors (Ed25519/X.509/PKIX/PKCS7/DSSE)
|
||||
* default trust weight
|
||||
* tenancy‑specific overrides
|
||||
|
||||
**From Policy Engine:**
|
||||
|
||||
* Policy version and options relevant to VEX evaluation:
|
||||
|
||||
* trust model parameters
|
||||
* confidence thresholds per environment/tier
|
||||
* justification whitelist/blacklist
|
||||
* recency requirements and expiry windows
|
||||
* precedence rules for status conflicts
|
||||
|
||||
### 3.2 Normalization
|
||||
|
||||
1. **Canonical advisory key:** CVE preferred; GHSA mapped; vendor IDs namespaced.
|
||||
2. **Status mapping:** Normalize all encodings to `NOT_AFFECTED | AFFECTED | FIXED | UNDER_INVESTIGATION`.
|
||||
3. **Product scope alignment:** Convert CSAF product tree/CPE to purl variants using deterministic mappings; store mapping evidence.
|
||||
4. **Version scoping:** For each evidence, compute `applies_to(version)` using ecosystem comparators (npm semver, Maven, PEP 440, Go semver, RPM/DEB EVR).
|
||||
5. **Signature verification:** If signed, verify against Issuer Directory trust anchors; attach `sig_verified=true/false` and chain metadata.
|
||||
6. **Temporal fields:** Compute `effective_at` (issued), `observed_at` (ingested), and staleness.
|
||||
|
||||
### 3.3 Trust model
|
||||
|
||||
Each `VexEvidence` gets a base weight `w_base` from issuer type and verification:
|
||||
|
||||
* Maintainer/Upstream signed: 0.9
|
||||
* Maintainer/Upstream unsigned: 0.7
|
||||
* Distro/SIG signed: 0.8
|
||||
* Vendor of downstream product signed: 0.8
|
||||
* Third‑party scanner VEX: 0.4
|
||||
* Unknown/unsigned with weak provenance: 0.2
|
||||
|
||||
Weights are then adjusted:
|
||||
|
||||
```
|
||||
w = w_base
|
||||
* f_signature(sig_verified) // e.g., 1.1 if verified, 0.8 if unverifiable
|
||||
* f_recency(age_days) // decay after T days (policy)
|
||||
* f_justification(type) // e.g., "component_not_present" lower if SBOM shows present
|
||||
* f_scope_match(score) // quality of product match: exact purl > family > CPE wildcard
|
||||
* f_env(app_env) // optional env-specific multipliers
|
||||
```
|
||||
|
||||
All `f_*` are bounded to [0.5, 1.2] by default to prevent runaway effects. Tenants can override per Policy Studio.
|
||||
|
||||
### 3.4 Consensus algorithm
|
||||
|
||||
For a tuple `(artifact_id, purl, version, advisory_key)`:
|
||||
|
||||
1. **Collect** all normalized `VexEvidence` whose product scope maps to `purl` and `applies_to(version)==true`.
|
||||
|
||||
2. **Bucket** by normalized status; compute `W(status) = Σ w(evidence)` per status.
|
||||
|
||||
3. **Apply precedence** rules from policy:
|
||||
|
||||
* If `W(NOT_AFFECTED)` exceeds threshold `T_na` and there is no `FIXED` evidence contradictory for the same version, propose `NOT_AFFECTED`.
|
||||
* If any `FIXED` applies and inventory version ≥ fixed version, propose `FIXED`.
|
||||
* If `W(AFFECTED)` ≥ `T_aff` and no dominating `NOT_AFFECTED`, propose `AFFECTED`.
|
||||
* If `W(UNDER_INVESTIGATION)` dominates and others below thresholds, propose `UNDER_INVESTIGATION`.
|
||||
* If both `AFFECTED` and `NOT_AFFECTED` exceed thresholds within a small margin `δ`, mark `DISPUTED`.
|
||||
* Otherwise `INCONCLUSIVE`.
|
||||
|
||||
4. **Confidence score:**
|
||||
|
||||
```
|
||||
confidence = W(winning_status) / (W(AFFECTED)+W(NOT_AFFECTED)+W(FIXED)+W(UNDER_INVESTIGATION) + ε)
|
||||
```
|
||||
|
||||
Clip to [0.0, 1.0].
|
||||
|
||||
5. **Rationale chain:**
|
||||
|
||||
* Winning status, thresholds used, top 5 contributing evidences with weights and reasons, product mapping quality, version scoping evidence, and policy knobs that influenced the result.
|
||||
|
||||
6. **Quorum summary:**
|
||||
|
||||
* List issuers and their votes, signature state, timestamps, and justifications.
|
||||
|
||||
### 3.5 Policy interaction
|
||||
|
||||
* The Lens returns `consensus_state`, `confidence`, and structured `signals`.
|
||||
* Policy Engine consumes these and may:
|
||||
|
||||
* Suppress findings automatically on `NOT_AFFECTED` confidence ≥ `P_na`.
|
||||
* Downgrade severity or extend SLA when `UNDER_INVESTIGATION` from high‑trust issuers.
|
||||
* Require human approval when `DISPUTED`.
|
||||
* Treat `FIXED` as resolved only when SBOM crawl verifies the fixed version or a `verify_fix` ledger event exists.
|
||||
|
||||
**Simulation:** Policy Studio can simulate different trust weights and thresholds and see consensus deltas without side effects.
|
||||
|
||||
### 3.6 Data contracts
|
||||
|
||||
**ConsensusRecord (materialized)**
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cons:sha256(tenant|artifact|purl|version|advisory|policy)",
|
||||
"tenant": "acme",
|
||||
"artifact_id": "svc:payments@1.14.0",
|
||||
"purl": "pkg:npm/lodash@4.17.20",
|
||||
"advisory_key": "CVE-2024-12345",
|
||||
"policy_version": "1.3.0",
|
||||
"consensus_state": "NOT_AFFECTED",
|
||||
"confidence": 0.87,
|
||||
"weights": {"AFFECTED":0.31,"NOT_AFFECTED":1.25,"FIXED":0.00,"UNDER_INVESTIGATION":0.12},
|
||||
"top_evidence": ["vex_evd:...","vex_evd:..."],
|
||||
"quorum": [
|
||||
{"issuer":"lodash-maintainers","status":"NOT_AFFECTED","w":0.9,"sig_verified":true,"just":"vulnerable_code_not_present","issued":"2024-06-08"},
|
||||
{"issuer":"vendorX-distro","status":"AFFECTED","w":0.25,"sig_verified":false,"just":"generic_advisory","issued":"2024-06-07"}
|
||||
],
|
||||
"rationale": [
|
||||
{"rule":"trust.weight.issuer","detail":"maintainer signed evidence 0.9"},
|
||||
{"rule":"scope.match.exact","detail":"exact purl match"},
|
||||
{"rule":"justification.vcnp","detail":"supported by SBOM callgraph hint"}
|
||||
],
|
||||
"updated_at": "2024-06-12T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Issuer Directory entry**
|
||||
|
||||
```json
|
||||
{
|
||||
"issuer_id": "iss:lodash",
|
||||
"org": "Lodash Maintainers",
|
||||
"domains": ["lodash.com"],
|
||||
"keys": [{"type":"ed25519","pub":"...","expires":"2026-12-31"}],
|
||||
"default_weight": 0.9,
|
||||
"metadata": {"csaf_publisher": true}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.7 APIs
|
||||
|
||||
**Compute/Query**
|
||||
|
||||
```
|
||||
GET /vex/consensus?artifact=svc:payments@1.14.0&purl=pkg:npm/lodash@4.17.20&advisory=CVE-2024-12345&policy=1.3.0
|
||||
POST /vex/consensus/query { "policy":"1.3.0", "filter": { "state":["DISPUTED","INCONCLUSIVE"], "confidence":"<0.6", "env":["prod"] }, "page":{...} }
|
||||
GET /vex/consensus/{id} // full record
|
||||
POST /vex/consensus/simulate // override trust knobs and thresholds for what-if
|
||||
```
|
||||
|
||||
**Issuer Directory**
|
||||
|
||||
```
|
||||
GET /vex/issuers
|
||||
POST /vex/issuers (admin)
|
||||
POST /vex/issuers/{id}/keys (admin)
|
||||
```
|
||||
|
||||
**Exports**
|
||||
|
||||
```
|
||||
POST /vex/consensus/export { "format":"ndjson","scope":{"filter":{...}} }
|
||||
```
|
||||
|
||||
**Errors**
|
||||
|
||||
* `400` invalid mapping, `403` RBAC, `404` not found, `409` conflict, `429` rate limit.
|
||||
|
||||
### 3.8 Console (Web UI)
|
||||
|
||||
Routes:
|
||||
|
||||
* `/vex/consensus` overview with filters: state, confidence, issuer, advisory, artifact, env.
|
||||
* `/vex/consensus/:id` detail: Evidence pane, Quorum graph, Policy impact, SBOM path links.
|
||||
|
||||
UX elements:
|
||||
|
||||
* **Quorum bar:** stacked bar showing weights per status; hover reveals issuer contributions.
|
||||
* **Confidence chip:** numeric and qualitative band (Low/Med/High).
|
||||
* **Evidence table:** paged list of VEX docs with signature icon, scope match quality tag, justification tag, issued/updated timestamps.
|
||||
* **Conflict view:** for `DISPUTED`, show side-by-side issuer rationales and suggested next steps.
|
||||
* **Deep link** into Vulnerability Explorer detail, preselecting the Policy version used for the consensus.
|
||||
|
||||
A11y:
|
||||
|
||||
* ARIA roles on grid and bars; keyboard shortcuts `S` switch policy, `T` trust presets, `E` export.
|
||||
|
||||
### 3.9 CLI
|
||||
|
||||
Commands:
|
||||
|
||||
```
|
||||
stella vex consensus list --policy 1.3.0 --state disputed,inconclusive --confidence '<0.6' --artifact payments --json
|
||||
stella vex consensus show --id <cons-id> --policy 1.3.0
|
||||
stella vex simulate --policy 1.3.0 --trust 'issuer:lodash=1.0,issuer:vendorX=0.5' --thresholds 'na=1.0,aff=0.6' --json
|
||||
stella vex issuers list
|
||||
stella vex export --filter 'artifact:payments advisory:CVE-2024-12345' --out vex-consensus.ndjson
|
||||
```
|
||||
|
||||
Exit codes: `0` ok, `2` invalid args, `4` not found, `5` denied.
|
||||
|
||||
### 3.10 Storage schema (illustrative)
|
||||
|
||||
```sql
|
||||
-- Normalized VEX (reference Excitator id; do not alter raw)
|
||||
CREATE TABLE vex_normalized (
|
||||
id uuid PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
evidence_id text NOT NULL, -- link to Excitator
|
||||
advisory_key text NOT NULL,
|
||||
issuer_id text,
|
||||
status text NOT NULL, -- NOT_AFFECTED|AFFECTED|FIXED|UNDER_INVESTIGATION
|
||||
justification text,
|
||||
purl text, -- normalized mapping target
|
||||
version_range jsonb, -- ecosystem-specific encoding
|
||||
sig_verified boolean,
|
||||
scope_score real, -- 0..1 quality of mapping
|
||||
issued timestamptz,
|
||||
updated timestamptz,
|
||||
w_base real,
|
||||
UNIQUE (tenant, evidence_id)
|
||||
);
|
||||
|
||||
-- Issuer Directory
|
||||
CREATE TABLE vex_issuers (
|
||||
issuer_id text PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
org text NOT NULL,
|
||||
default_weight real NOT NULL,
|
||||
metadata jsonb,
|
||||
UNIQUE(tenant, org)
|
||||
);
|
||||
|
||||
CREATE TABLE vex_issuer_keys (
|
||||
id uuid PRIMARY KEY,
|
||||
issuer_id text NOT NULL REFERENCES vex_issuers(issuer_id),
|
||||
key_type text NOT NULL,
|
||||
pubkey text NOT NULL,
|
||||
expires timestamptz
|
||||
);
|
||||
|
||||
-- Consensus projection
|
||||
CREATE TABLE vex_consensus (
|
||||
id bytea PRIMARY KEY,
|
||||
tenant text NOT NULL,
|
||||
artifact_id text NOT NULL,
|
||||
purl text NOT NULL,
|
||||
version text NOT NULL,
|
||||
advisory_key text NOT NULL,
|
||||
policy_version text NOT NULL,
|
||||
consensus_state text NOT NULL,
|
||||
confidence real NOT NULL,
|
||||
weights jsonb NOT NULL,
|
||||
top_evidence text[] NOT NULL,
|
||||
updated_at timestamptz NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX vc_query ON vex_consensus(tenant, policy_version, consensus_state, confidence);
|
||||
```
|
||||
|
||||
### 3.11 Integration with Findings Ledger and Vuln Explorer
|
||||
|
||||
* The Vuln Explorer reads `vex_consensus` for each finding and renders:
|
||||
|
||||
* Consensus chip, confidence, and a link to full quorum.
|
||||
* For `NOT_AFFECTED` with confidence ≥ policy threshold, show “Suppressed by VEX (Consensus)” badge.
|
||||
* For `DISPUTED`, open a triage banner prompting manual review and optional ledger comment/assignment.
|
||||
|
||||
* Ledger receives no new event type from lens computation itself. Human actions triggered by lens views produce standard events (`comment`, `assign`, `change_state`).
|
||||
|
||||
### 3.12 Security & RBAC
|
||||
|
||||
Roles:
|
||||
|
||||
* Viewer: query consensus read‑only.
|
||||
* Investigator: Viewer + export.
|
||||
* Operator: Investigator + trust simulation.
|
||||
* Admin: manage Issuer Directory entries and keys.
|
||||
|
||||
CSRF for Console; ABAC scoping by artifact ownership and environment.
|
||||
|
||||
### 3.13 Observability
|
||||
|
||||
Metrics:
|
||||
|
||||
* `vex_consensus_compute_latency_ms` (histogram)
|
||||
* `vex_consensus_records_total` (counter)
|
||||
* `vex_consensus_disputed_total` (counter by issuer combinations)
|
||||
* `vex_consensus_staleness_seconds` (gauge)
|
||||
* `vex_signature_verification_rate` (gauge)
|
||||
|
||||
Logs: structured events with `tenant`, `policy_version`, `advisory_key`, `quorum_summary`.
|
||||
Traces: spans for normalization, mapping, trust weighting, consensus decision, DB writes.
|
||||
|
||||
### 3.14 Performance & scaling
|
||||
|
||||
Targets:
|
||||
|
||||
* P95 query under 500 ms for 100‑row pages at 10M consensus records/tenant.
|
||||
* Projection jobs are idempotent and keyed by `(tenant, artifact, purl, version, advisory, policy)`; backpressure with work queues.
|
||||
* Cache popular queries with tenant‑scoped TTL; invalidate on Excitator or policy changes.
|
||||
|
||||
### 3.15 Edge cases
|
||||
|
||||
* **Ambiguous product mapping:** mark low `scope_score`, cap weight, surface warning in UI.
|
||||
* **VEX “not present” vs SBOM shows present:** down‑weight with `f_justification`, require manual check.
|
||||
* **Withdrawn or superseded VEX:** decay to near zero; keep provenance.
|
||||
* **Partial fixes:** if fixed version applies to subset of platforms, map to env or arch dimension when available.
|
||||
* **Time travel:** consensus recalculated as of a timestamp using only evidence ≤ `as_of` and the corresponding policy version.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services
|
||||
|
||||
* **VEX Lens Service (new)**
|
||||
|
||||
* Normalization pipeline, trust weighting, consensus computation, and projections.
|
||||
* Batch recompute on policy activation and Excitator deltas.
|
||||
|
||||
* **Excitator (updates)**
|
||||
|
||||
* Ensure all VEX evidence carries issuer hints and raw signature blobs when present.
|
||||
* Publish product trees and original coordinates intact.
|
||||
|
||||
* **Policy Engine (updates)**
|
||||
|
||||
* Add VEX trust knobs, thresholds, recency decay, and status precedence.
|
||||
* Batch eval endpoint accepts `consensus inputs` where needed.
|
||||
|
||||
* **Issuer Directory (new)**
|
||||
|
||||
* Manage issuer metadata and keys; tenant overrides; audit logs.
|
||||
|
||||
### 4.2 Code structure
|
||||
|
||||
```
|
||||
/src/VexLens/StellaOps.VexLens
|
||||
/normalizer
|
||||
/mapping # CPE/purl translators
|
||||
/trust # weighting functions
|
||||
/consensus # algorithm and projections
|
||||
/api
|
||||
/src/Excititor # updates
|
||||
/src/Policy/__Libraries/StellaOps.Policy # updates
|
||||
/src/IssuerDirectory/StellaOps.IssuerDirectory
|
||||
/packages/console/features/vex-consensus
|
||||
/src/Cli/StellaOps.Cli
|
||||
```
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
* Phase 1: API read‑only with basic trust model, Console list/detail, no simulation.
|
||||
* Phase 2: Policy Studio integrations and simulation.
|
||||
* Phase 3: Issuer Directory admin flows, exports, and advanced mapping diagnostics.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes (create/update)
|
||||
|
||||
1. `/docs/vex/consensus-overview.md`
|
||||
Purpose, scope, terminology, evidence vs derived view, AOC guarantees.
|
||||
|
||||
2. `/docs/vex/consensus-algorithm.md`
|
||||
Normalization, mapping, weighting, thresholds, precedence, formulas, examples.
|
||||
|
||||
3. `/docs/vex/issuer-directory.md`
|
||||
Managing issuers, keys, trust overrides, security model.
|
||||
|
||||
4. `/docs/vex/consensus-api.md`
|
||||
Endpoints, request/response schemas, errors, pagination, rate limits.
|
||||
|
||||
5. `/docs/vex/consensus-console.md`
|
||||
Screens, filters, conflict workflows, a11y, deep links.
|
||||
|
||||
6. `/docs/policy/vex-trust-model.md`
|
||||
Policy knobs, thresholds, decay, simulation.
|
||||
|
||||
7. `/docs/sbom/vex-mapping.md`
|
||||
Product tree mapping to purl/version, ecosystem comparators, edge cases.
|
||||
|
||||
8. `/docs/security/vex-signatures.md`
|
||||
Signature verification flows, key management, auditing.
|
||||
|
||||
9. `/docs/runbooks/vex-ops.md`
|
||||
Recompute storms, mapping failures, signature errors, lag, quotas.
|
||||
|
||||
All docs end with the imposed rule statement.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend core
|
||||
|
||||
* [ ] Implement normalization for CSAF VEX, OpenVEX, CycloneDX VEX.
|
||||
* [ ] Build product mapping library (CPE→purl, vendor tokens→purl families).
|
||||
* [ ] Implement signature verification (Ed25519/PKIX/DSSE) using Issuer Directory keys.
|
||||
* [ ] Implement trust weighting functions and configurable parameters.
|
||||
* [ ] Implement consensus algorithm with unit tests and property tests.
|
||||
* [ ] Materialize `vex_consensus` projection with indexes and idempotent workers.
|
||||
* [ ] Batch recompute on policy activation and Excitator deltas.
|
||||
|
||||
### APIs & Integrations
|
||||
|
||||
* [ ] `/vex/consensus` query, detail, simulate, export.
|
||||
* [ ] Policy Engine: consume consensus signals; add thresholds and precedence.
|
||||
* [ ] Vuln Explorer: show consensus chip and triage banners; deep link to Lens.
|
||||
|
||||
### Issuer Directory
|
||||
|
||||
* [ ] CRUD for issuers and keys, audit logs, RBAC.
|
||||
* [ ] Import common CSAF publishers; seed with sane defaults.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Build list grid with filters and saved views.
|
||||
* [ ] Quorum bar and Evidence table with signature icons and scope quality tags.
|
||||
* [ ] Conflict view for `DISPUTED`.
|
||||
* [ ] Simulation drawer integrated with Policy Studio.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella vex consensus list|show|simulate|export` with JSON/CSV.
|
||||
* [ ] Stable schemas; tests for piping and scripting.
|
||||
|
||||
### Observability/Perf
|
||||
|
||||
* [ ] Metrics, logs, traces as specified; dashboards.
|
||||
* [ ] Load tests at 10M consensus records/tenant; optimize indexes and caches.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author and cross‑link all docs listed in §5.
|
||||
* [ ] Add examples and screenshots to Console doc.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Normalization supports CSAF VEX, OpenVEX, CycloneDX VEX with product tree mapping to purls across npm, Maven, PyPI, Go, RPM/DEB.
|
||||
* Signature verification works and affects weights; unverifiable signatures do not crash flows.
|
||||
* Consensus outputs are reproducible, explainable, and queryable at scale.
|
||||
* Vuln Explorer displays consensus state and affects policy outcomes per thresholds.
|
||||
* Simulation reflects policy trust changes without side effects and returns rationale deltas.
|
||||
* CLI/API feature parity; evidence and quorum are exportable.
|
||||
* P95 performance budgets met; dashboards reflect health.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks and mitigations
|
||||
|
||||
* **Mapping errors (CPE→purl):** use conservative scope scores, cap weights, surface warnings, manual override hooks in Policy Studio.
|
||||
* **Malicious or mistaken issuer:** signature verification plus trust weighting and tenancy overrides.
|
||||
* **Evidence storms:** debounce recompute; batch and shard workers; backpressure and prioritization.
|
||||
* **User confusion with conflicting VEX:** clear conflict UI, rationale chains, suggested actions, and policy banners.
|
||||
* **Stale statements:** recency decay and expiry windows in policy.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit:** status mapping, comparators per ecosystem, trust weighting, threshold math.
|
||||
* **Property tests:** invariants such as monotonicity with added supporting evidence and idempotent recompute.
|
||||
* **Integration:** Excitator→Lens→Policy→Vuln Explorer pipeline with signed/unsigned, conflicting, and stale evidence.
|
||||
* **E2E Console:** list filters, detail with quorum, conflict workflows, export, simulation.
|
||||
* **Security:** RBAC on Issuer Directory, CSRF for Console, signature verification path traversal guards.
|
||||
* **Performance:** cold/hot query latencies, recompute throughput, cache hit ratios.
|
||||
* **Determinism:** time‑travel snapshots reproduce prior consensus states.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Consensus, not replacement.** The Lens summarizes without erasing dissent.
|
||||
* **Trust is contextual.** Tenants must tune weights and thresholds to their environments.
|
||||
* **Proof over prose.** Every decision comes with math, provenance, and a rationale chain.
|
||||
* **Safety by design.** When in doubt, surface conflicts instead of silently suppressing.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
440
docs/implplan/EPIC_8.md
Normal file
440
docs/implplan/EPIC_8.md
Normal file
@@ -0,0 +1,440 @@
|
||||
Below is the “maximum documentation” bundle for Epic 8. It’s engineered to be pasted into your repo without turning into yet another unread wiki tomb. Slight sarcasm included to keep blood flowing.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 8: Advisory AI Assistant (summaries, conflict explain, remediation hints)
|
||||
|
||||
**Short name:** `Advisory AI`
|
||||
**Services touched:** Conseiller (Feedser), Excitator (Vexer), VEX Lens, SBOM Service, Policy Engine, Findings Ledger, Web API Gateway, Authority (authN/Z), Console (Web UI), CLI, Telemetry/Analytics
|
||||
**AOC ground rule:** Conseiller and Excitator aggregate but never merge or mutate source docs. Advisory AI produces derived summaries and plans with strict provenance and citations. No silent rewriting of evidence. Ever.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
Advisory AI is a tenant‑scoped, retrieval‑augmented assistant that turns noisy security advisories and VEX statements into three consumable artifacts:
|
||||
|
||||
1. **Advisory Summary**
|
||||
Condenses one or more advisories (CSAF, OSV, GHSA, vendor PDFs, distro notices) into a concise brief with key facts: affected ranges, exploit status, impact, known workarounds, fixed versions, and links. Always cites the exact sources and sections used.
|
||||
|
||||
2. **Conflict Explain**
|
||||
Explains why VEX statements or advisories disagree for a specific artifact and version. Uses the VEX Consensus Lens outputs and issuer trust model to produce a human‑readable, step‑by‑step explanation: who said what, where the product scoping diverges, and what policy thresholds caused the final state.
|
||||
|
||||
3. **Remediation Hints**
|
||||
Suggests practical next steps: upgrade paths compatible with your dependency graph, backports, config toggles, temporary policy suppressions, or compensating controls. Every hint is grounded in SBOM, environment, and policy. It ships as structured JSON plus a human summary, ready to paste into a ticket.
|
||||
|
||||
It lives in the Console as a side panel, in the CLI for batch runs, and via APIs for automation. It does not change scanner results or consensus on its own. Humans remain in charge. The machine does the skimming and the math so humans can keep the judgment and the coffee.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (brief)
|
||||
|
||||
Advisories are long, inconsistent, and sometimes contradictory. Teams waste cycles reconciling PDFs with package manifests. The assistant eliminates that sludge: fast summaries, explicit conflict explanations, and remediation hints that are actually applicable to your software, not to an imaginary ideal project from 2013.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Capabilities
|
||||
|
||||
* **Summaries**
|
||||
|
||||
* Input: one advisory or a bundle linked by the same advisory key (CVE, GHSA, vendor‑ID), product scope, and environment.
|
||||
* Output:
|
||||
|
||||
* 150 to 300 words summary
|
||||
* `AdvisorySummary JSON` (schema below)
|
||||
* Citations with paragraph anchors
|
||||
* Confidence label and coverage score (how much of the advisory set is represented)
|
||||
* **Conflict Explain**
|
||||
|
||||
* Input: `(artifact_id, purl, version, advisory_key)` tuple.
|
||||
* Output: narrative plus a structured breakdown of consensus math, issuer votes, product mapping mismatches, and the exact policy knobs that tipped the result.
|
||||
* **Remediation Hints**
|
||||
|
||||
* Input: same tuple plus SBOM context and environment.
|
||||
* Output: ranked list of remediation options with feasibility score, blast radius estimate (derived from dependency paths), effort class, and links to fixed versions. Includes “do nothing” when the VEX consensus is not affected.
|
||||
|
||||
### 3.2 System design
|
||||
|
||||
**Architecture diagram in words (because ASCII art is a crime):**
|
||||
|
||||
1. **Retrievers**
|
||||
|
||||
* Structured retriever over Conseiller’s normalized advisory fields.
|
||||
* Vector retriever over advisory text chunks with paragraph anchors.
|
||||
* VEX retriever over Excitator evidence and VEX Lens consensus.
|
||||
* SBOM retriever for purl, version, dependency paths, env flags.
|
||||
2. **Deterministic resolvers**
|
||||
|
||||
* Version comparators per ecosystem.
|
||||
* Range satisfaction checks.
|
||||
* Dependency path scorers and blast radius estimator.
|
||||
3. **Orchestrator**
|
||||
|
||||
* Task‑specific prompt templates for Summary, Conflict, Remediation.
|
||||
* Tool calls to deterministics (version check, graph crawl) with results injected into the prompt as structured context.
|
||||
* Strict token budgets and truncation rules to avoid model babble.
|
||||
4. **Models**
|
||||
|
||||
* Default: on‑prem inference container with mid‑sized model.
|
||||
* Optional: tenant‑enabled remote inference. Disabled by default.
|
||||
* Temperature locked low for summary and conflict. Slightly higher for remediation narrative phrasing. No creativity in facts.
|
||||
5. **Guardrails**
|
||||
|
||||
* Prompt injection defense by stripping or quarantining advisory text that tries to instruct the model.
|
||||
* Fact boundary tagger. The assistant must only state facts that appear in structured inputs or cited chunks.
|
||||
* Redaction of secrets before prompts.
|
||||
* Output validator checks: required JSON fields, numeric ranges, valid version strings.
|
||||
|
||||
### 3.3 Data contracts
|
||||
|
||||
**AdvisorySummary JSON**
|
||||
|
||||
```json
|
||||
{
|
||||
"advisory_key": "CVE-2025-12345",
|
||||
"sources": [
|
||||
{"id":"csaf:vendorA:2025-001","uri":"...","sections":["2.1","3.4"]},
|
||||
{"id":"osv:pkg:npm/lodash","uri":"...","sections":["affected","references"]}
|
||||
],
|
||||
"affected_ranges": [
|
||||
{"ecosystem":"npm","purl_family":"pkg:npm/lodash","introduced":"<4.17.15","fixed": "4.17.21"}
|
||||
],
|
||||
"exploit_status": "no_known_exploit | poc_public | exploited_in_the_wild | n/a",
|
||||
"impact": {"cvss":[{"vector":"CVSS:3.1/AV:N/...","score":7.5}], "cwes":["CWE-79"]},
|
||||
"workarounds": ["Disable feature X", "Set flag Y=false"],
|
||||
"fixed_versions": ["4.17.21"],
|
||||
"notes": "Vendor states not affected on platform Z due to build option W.",
|
||||
"coverage_score": 0.86,
|
||||
"generated_at": "2025-10-25T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**ConflictExplanation JSON**
|
||||
|
||||
```json
|
||||
{
|
||||
"tuple": {"artifact_id":"svc:checkout@1.9.0","purl":"pkg:npm/lodash@4.17.20","advisory_key":"CVE-2025-12345"},
|
||||
"consensus": {"state":"NOT_AFFECTED","confidence":0.82},
|
||||
"quorum": [
|
||||
{"issuer":"lodash-maintainers","status":"NOT_AFFECTED","weight":0.9,"sig":true,"justification":"component_not_present"},
|
||||
{"issuer":"vendorX-distro","status":"AFFECTED","weight":0.25,"sig":false,"justification":"generic"}
|
||||
],
|
||||
"policy_factors": {"na_threshold":1.0,"aff_threshold":0.6,"recency_decay_days":90},
|
||||
"mapping_issues": [{"kind":"cpe_to_purl","score":0.6,"detail":"CPE wildcard matched multiple purls"}],
|
||||
"explanation_steps": [
|
||||
"Exact purl match found for maintainer VEX; weight 0.9",
|
||||
"Distro advisory generic; scope score 0.5; effective weight 0.25",
|
||||
"NA threshold met. Result set to NOT_AFFECTED"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**RemediationPlan JSON**
|
||||
|
||||
```json
|
||||
{
|
||||
"tuple": {"artifact_id":"svc:checkout@1.9.0","purl":"pkg:npm/lodash@4.17.20","advisory_key":"CVE-2025-12345"},
|
||||
"options": [
|
||||
{
|
||||
"kind": "upgrade",
|
||||
"target_version": "4.17.21",
|
||||
"feasibility": 0.92,
|
||||
"blast_radius": {"direct_callers":3,"transitive_depth":2},
|
||||
"effort": "low | medium | high",
|
||||
"rationale": "Semver patch, no breaking APIs in release notes",
|
||||
"links": ["release_notes_uri"]
|
||||
},
|
||||
{
|
||||
"kind": "workaround",
|
||||
"action": "Set SAFE_MODE=true",
|
||||
"feasibility": 0.6,
|
||||
"blast_radius": {"feature_flags":["SAFE_MODE"]},
|
||||
"effort": "low",
|
||||
"rationale": "Vendor states mitigation reduces attack surface on feature X"
|
||||
}
|
||||
],
|
||||
"preferred": 0,
|
||||
"policy_effects": {"sla_days": 7, "severity_override": "medium_if_not_fixed"},
|
||||
"generated_at": "2025-10-25T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### 3.4 APIs
|
||||
|
||||
```
|
||||
POST /advisory/ai/summary
|
||||
{
|
||||
"advisory_key":"CVE-2025-12345",
|
||||
"artifact_id":"svc:checkout@1.9.0",
|
||||
"purl":"pkg:npm/lodash@4.17.20",
|
||||
"sources":["csaf:*","osv:*"], // optional filters
|
||||
"policy_version":"1.3.0",
|
||||
"lang":"en"
|
||||
}
|
||||
-> 200 { "summary_text":"...", "summary": {AdvisorySummary}, "citations":[...] }
|
||||
|
||||
POST /advisory/ai/conflict
|
||||
{
|
||||
"artifact_id":"svc:checkout@1.9.0",
|
||||
"purl":"pkg:npm/lodash@4.17.20",
|
||||
"advisory_key":"CVE-2025-12345",
|
||||
"policy_version":"1.3.0"
|
||||
}
|
||||
-> 200 { "explanation_text":"...", "explanation": {ConflictExplanation} }
|
||||
|
||||
POST /advisory/ai/remediation
|
||||
{
|
||||
"artifact_id":"svc:checkout@1.9.0",
|
||||
"purl":"pkg:npm/lodash@4.17.20",
|
||||
"advisory_key":"CVE-2025-12345",
|
||||
"policy_version":"1.3.0",
|
||||
"max_options":5,
|
||||
"strategy_preference":["upgrade","backport","workaround"]
|
||||
}
|
||||
-> 200 { "plan_text":"...", "plan": {RemediationPlan} }
|
||||
|
||||
POST /advisory/ai/batch
|
||||
{
|
||||
"items":[ {tuple}, {tuple}, ... ],
|
||||
"task":"summary | conflict | remediation",
|
||||
"policy_version":"1.3.0"
|
||||
}
|
||||
-> 207 multi-status
|
||||
```
|
||||
|
||||
Status codes: `400` invalid, `403` RBAC, `404` missing evidence, `409` conflict lock, `422` output validation failed, `429` rate limit.
|
||||
|
||||
### 3.5 Console (Web UI)
|
||||
|
||||
* Surfaces:
|
||||
|
||||
* Vuln Explorer detail: “Advisory AI” side panel with 3 tabs: Summary, Conflict, Remediation.
|
||||
* Consensus Lens detail: prominent “Explain conflict” button.
|
||||
* Policy Studio sim: “Show effect on assistant output” preview.
|
||||
|
||||
* UX details:
|
||||
|
||||
* Citations are footnotes with hover to show source paragraph.
|
||||
* “Copy as ticket” produces Markdown and JSON.
|
||||
* Plan options show feasibility bar, blast radius chips, and required approvals per policy.
|
||||
* Injection warnings appear if advisory text included unsafe instructions.
|
||||
|
||||
* A11y: ARIA tags for tabs, keyboard shortcuts `G` to generate, `R` to refresh, `C` to copy JSON.
|
||||
|
||||
### 3.6 CLI
|
||||
|
||||
```
|
||||
stella advise summarize --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0 --json
|
||||
stella advise explain --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0
|
||||
stella advise remediate --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0 --strategy upgrade,workaround --out plan.json
|
||||
stella advise batch --file tuples.json --task remediation --policy 1.3.0
|
||||
```
|
||||
|
||||
Exit codes: `0` ok, `2` invalid args, `4` not found, `5` denied, `7` validation fail.
|
||||
|
||||
### 3.7 RBAC and security
|
||||
|
||||
* Roles:
|
||||
|
||||
* Viewer can run summaries and read explanations.
|
||||
* Operator can run remediation and export plans.
|
||||
* Admin can toggle model endpoints and guardrail settings.
|
||||
|
||||
* Defaults:
|
||||
|
||||
* Remote model calls disabled.
|
||||
* Redaction on.
|
||||
* Prompt logging anonymized.
|
||||
* Outputs stored as derived artifacts with TTL (default 30 days) unless pinned to a ticket.
|
||||
|
||||
### 3.8 Observability
|
||||
|
||||
* Metrics:
|
||||
|
||||
* `advisory_ai_latency_ms` by task type.
|
||||
* `advisory_ai_guardrail_blocks_total`.
|
||||
* `advisory_ai_output_validation_fail_total`.
|
||||
* `advisory_ai_citation_coverage` gauge.
|
||||
* Traces: retriever spans, tool calls, model inference, validator.
|
||||
* Logs: include tuple key, token usage, truncation events, and guardrail outcomes.
|
||||
|
||||
### 3.9 Performance
|
||||
|
||||
* Targets:
|
||||
|
||||
* P95 under 1.5 s for Summary and Conflict with warm caches.
|
||||
* P95 under 2.5 s for Remediation on medium SBOMs (1000 packages).
|
||||
* Batch throughput 10 tuples per second per worker.
|
||||
|
||||
### 3.10 Edge cases
|
||||
|
||||
* Advisory missing fixed versions: produce workaround‑only plan and mark feasibility low.
|
||||
* Conflicts with near‑tie weights: declare “DISPUTED” and require human approval, no auto plan preferred.
|
||||
* Exotic version schemes: fallback to string compare with warning and feasibility cap.
|
||||
* Private packages: no public release notes. Prefer internal changelog links if attached to artifact metadata.
|
||||
* Multi‑env differences: render per‑env deltas when policy knobs differ (dev vs prod).
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Services and components
|
||||
|
||||
* **New:** `src/AdvisoryAI/StellaOps.AdvisoryAI`
|
||||
|
||||
* `retriever/` wrappers for Conseiller, Excitator, VEX Lens, SBOM.
|
||||
* `deterministic/` version and path analyzers.
|
||||
* `orchestrator/` task routers and prompt builders.
|
||||
* `guardrails/` injection, redaction, output validator.
|
||||
* `api/` REST endpoints and schema enforcement.
|
||||
|
||||
* **Updates:**
|
||||
|
||||
* Conseiller: expose paragraph‑level anchors for advisories.
|
||||
* Excitator: expose justifications and product trees in normalized form.
|
||||
* VEX Lens: stable API for quorum and rationale.
|
||||
* SBOM Service: efficient path queries and versions timeline per purl.
|
||||
|
||||
### 4.2 Packaging
|
||||
|
||||
* Container images:
|
||||
|
||||
* `stella/advisory-ai:<<version>>`
|
||||
* `stella/inference:<<version>>` (if using on‑prem model)
|
||||
* Helm values to toggle remote inference and GPU.
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
* Phase 1: Summary and Conflict read‑only.
|
||||
* Phase 2: Remediation with “Copy as ticket”.
|
||||
* Phase 3: Batch APIs, CLI, and Policy Studio simulation hooks.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Create or update the following files. Each doc ends with the imposed rule statement.
|
||||
|
||||
1. `/docs/advisory-ai/overview.md`
|
||||
What it is, capabilities, guardrails, AOC alignment, RBAC.
|
||||
|
||||
2. `/docs/advisory-ai/architecture.md`
|
||||
RAG design, retrievers, orchestrator, deterministics, models, caching.
|
||||
|
||||
3. `/docs/advisory-ai/api.md`
|
||||
Endpoint specs, payload schemas, error codes, examples.
|
||||
|
||||
4. `/docs/advisory-ai/console.md`
|
||||
Screens, actions, a11y, how citations work, copy‑as‑ticket.
|
||||
|
||||
5. `/docs/advisory-ai/cli.md`
|
||||
Command usage, exit codes, piping examples.
|
||||
|
||||
6. `/docs/policy/assistant-parameters.md`
|
||||
Temperature, max tokens, plan ranking weights, TTLs.
|
||||
|
||||
7. `/docs/security/assistant-guardrails.md`
|
||||
Redaction rules, injection defense, output validation, logging.
|
||||
|
||||
8. `/docs/sbom/remediation-heuristics.md`
|
||||
Feasibility scoring, blast radius, effort classes.
|
||||
|
||||
9. `/docs/runbooks/assistant-ops.md`
|
||||
Warmup, cache priming, model outages, scaling, on‑call steps.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend core
|
||||
|
||||
* [ ] Implement structured and vector retrievers with paragraph anchors from Conseiller.
|
||||
* [ ] Implement VEX retriever using Lens APIs with caching.
|
||||
* [ ] Build deterministics: ecosystem comparators, range checks, dependency path scorer.
|
||||
* [ ] Implement orchestrator with task‑specific templates and tool call pipeline.
|
||||
* [ ] Implement guardrails and validators with hard failure on invalid JSON.
|
||||
* [ ] Add RBAC to endpoints and anonymized prompt logging.
|
||||
* [ ] Add caching layer with tuple‑keyed entries and policy version scoping.
|
||||
|
||||
### Integrations
|
||||
|
||||
* [ ] Conseiller: expose advisory chunk API and metadata needed for citations.
|
||||
* [ ] Excitator: ensure justifications and product trees are queryable.
|
||||
* [ ] VEX Lens: add “policy factors” endpoint for explanation rendering.
|
||||
* [ ] SBOM Service: implement `GET /sbom/paths?purl=...` and version timeline.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Build Advisory AI panel with 3 tabs and citation tooltips.
|
||||
* [ ] Implement “Copy as ticket” (Markdown + JSON) and download.
|
||||
* [ ] Add injection warning banner when triggered.
|
||||
* [ ] Respect a11y requirements and shortcuts.
|
||||
|
||||
### CLI
|
||||
|
||||
* [ ] `stella advise summarize|explain|remediate|batch` with JSON output.
|
||||
* [ ] Add `--out` option to save plans and summaries.
|
||||
* [ ] Tests for piping and jq workflows.
|
||||
|
||||
### Observability
|
||||
|
||||
* [ ] Emit metrics and traces listed in §3.8.
|
||||
* [ ] Dashboards: latency, guardrail blocks, validation fails, coverage.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Write all files in §5 with examples and screenshots.
|
||||
* [ ] Cross‑link to VEX Lens and Vulnerability Explorer docs.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Summaries cite specific source sections and reflect affected ranges and fixed versions correctly for at least 95% of a validation set.
|
||||
* Conflict explanations enumerate issuers, weights, justifications, mapping issues, and policy thresholds that caused the consensus state.
|
||||
* Remediation plans output at least one feasible option when a fixed version exists and correctly flag “no public fix” cases.
|
||||
* JSON schemas validate for all outputs.
|
||||
* Console shows the panel with citations, copy‑as‑ticket, and a11y passes.
|
||||
* CLI produces identical JSON to API responses.
|
||||
* Guardrails block injection attempts and redact secrets in prompts.
|
||||
* P95 latency targets are met with warm caches.
|
||||
* No mutation of raw advisory or VEX evidence occurs anywhere in the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks and mitigations
|
||||
|
||||
* **Prompt injection in advisory text.** Strip instructions, sandbox chunks, and highlight to user when removed.
|
||||
* **Hallucinated facts.** Hard validation requires facts to appear in structured inputs or cited text. Fail closed if not provable.
|
||||
* **Mapping errors produce bad hints.** Depend on SBOM Graph and VEX Lens scope scores; cap feasibility when scope is weak.
|
||||
* **Model outage.** Degrade to deterministic summaries (shorter, but accurate).
|
||||
* **Privacy concerns.** Default on‑prem inference, remote endpoints opt‑in with clear flags and audit logs.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit:** version comparators, range checks, feasibility scoring, output validators.
|
||||
* **Golden files:** advisory sets mapped to expected summaries and plans; diff on each build.
|
||||
* **Injection tests:** adversarial advisories with “ignore prior instructions” payloads must be neutralized.
|
||||
* **Integration:** Conseiller→Advisory AI→Console loop, with VEX Lens conflicts and SBOM graph lookups.
|
||||
* **E2E:** generate summary, explanation, and plan for representative ecosystems (npm, Maven, PyPI, Go, RPM/DEB).
|
||||
* **Perf:** soak tests with 5k tuples batch; observe cache hit ratios and P95.
|
||||
* **A11y:** keyboard navigation and screen reader labels.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Facts first.** If it is not in structured inputs or citations, it does not exist.
|
||||
* **Explain everything.** Humans should see exactly why the tool said what it said.
|
||||
* **Helpful by default.** Plans must consider the real dependency graph and environment, not fantasy.
|
||||
* **No silent merges.** Evidence is sacred. Summaries and plans are separate, auditable derivatives.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
523
docs/implplan/EPIC_9.md
Normal file
523
docs/implplan/EPIC_9.md
Normal file
@@ -0,0 +1,523 @@
|
||||
Below is the “maximum documentation” bundle for Epic 9. Paste it into your repo and pretend the ingestion chaos was always under control.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
# Epic 9: Source & Job Orchestrator Dashboard
|
||||
|
||||
**Short name:** `Orchestrator Dashboard`
|
||||
**Primary service:** `orchestrator` (scheduler, queues, rate‑limits, job state)
|
||||
**Surfaces:** Console (Web UI), CLI, Web API
|
||||
**Touches:** Conseiller (Feedser), Excitator (Vexer), VEX Consensus Lens, SBOM Service, Policy Engine, Findings Ledger, Authority (authN/Z), Telemetry/Analytics, Object Storage, Relational DB, Message Bus
|
||||
|
||||
**AOC ground rule:** Conseiller and Excitator aggregate but never merge. The orchestrator schedules, tracks and recovers jobs; it does not transform evidence beyond transport and storage. No “smart” merging in flight.
|
||||
|
||||
---
|
||||
|
||||
## 1) What it is
|
||||
|
||||
The Source & Job Orchestrator Dashboard is the control surface for every data source and pipeline run across StellaOps. It gives operators:
|
||||
|
||||
* Live health of all advisory/VEX/SBOM sources and derived jobs.
|
||||
* End‑to‑end pipeline visibility as DAGs and timelines.
|
||||
* Controls for pausing, backfilling, replaying, throttling and retrying.
|
||||
* Error pattern analysis, rate‑limit observability and backpressure insights.
|
||||
* Provenance and audit trails from initial fetch through parse, normalize, index and policy evaluation.
|
||||
|
||||
The dashboard sits over the `orchestrator` service, which maintains job state, schedules runs, enforces quotas and rate limits, and collects metrics from worker pools embedded in Conseiller, Excitator, SBOM and related services.
|
||||
|
||||
---
|
||||
|
||||
## 2) Why (brief)
|
||||
|
||||
Ingestion breaks quietly and then loudly. Without a unified control plane, you learn about it from angry users or empty indexes. This dashboard shortens incident MTTR, enables safe backfills, and makes compliance reviewers stop sending emails with twelve attachments and one emoji.
|
||||
|
||||
---
|
||||
|
||||
## 3) How it should work (maximum detail)
|
||||
|
||||
### 3.1 Capabilities
|
||||
|
||||
* **Source registry**
|
||||
|
||||
* Register, tag and version connectors (OSV, GHSA, CSAF endpoints, vendor PDF scrapers, distro feeds, RSS, S3 drops, internal registries).
|
||||
* Store connection details, secrets (via KMS), rate‑limit policy, schedules, and ownership metadata.
|
||||
* Validate and “test connection” safely.
|
||||
|
||||
* **Job orchestration**
|
||||
|
||||
* Create DAGs composed of job types: `fetch`, `parse`, `normalize`, `dedupe`, `index`, `consensus_compute`, `policy_eval`, `crosslink`, `sbom_ingest`, `sbom_index`.
|
||||
* Priorities, queues, concurrency caps, exponential backoff, circuit breakers.
|
||||
* Idempotency keys and output artifact hashing to avoid duplicate work.
|
||||
* Event‑time watermarks for backfills without double counting.
|
||||
|
||||
* **Observability & control**
|
||||
|
||||
* Gantt timeline and real‑time DAG view with critical path highlighting.
|
||||
* Backpressure and queue depth heatmaps.
|
||||
* Error clustering by class (HTTP 429, TLS, schema mismatch, parse failure, upstream 5xx).
|
||||
* Per‑source SLOs and SLA budgets with burn‑rate alerts.
|
||||
* One‑click actions: retry, replay range, pause/resume, throttle/unthrottle, reroute to canary workers.
|
||||
|
||||
* **Provenance & audit**
|
||||
|
||||
* Immutable run ledger linking input artifact → every job → output artifact.
|
||||
* Schema version tracking and drift detection.
|
||||
* Operator actions recorded with reason and ticket reference.
|
||||
|
||||
* **Safety**
|
||||
|
||||
* Secret redaction everywhere.
|
||||
* Tenant isolation at API, queue and storage layers.
|
||||
* AOC: no in‑flight merges of advisory or VEX content.
|
||||
|
||||
### 3.2 Core architecture
|
||||
|
||||
* **orchestrator (service)**
|
||||
|
||||
* Maintains job state in Postgres (`sources`, `runs`, `jobs`, `artifacts`, `dag_edges`, `quotas`, `schedules`).
|
||||
* Publishes work to a message bus (e.g., `topic.jobs.ready.<queue>`).
|
||||
* Distributed token‑bucket rate limiter per source/tenant/host.
|
||||
* Watchdog for stuck jobs and circuit breakers for flapping sources.
|
||||
* Watermark manager for backfills (event‑time windows).
|
||||
|
||||
* **worker SDK**
|
||||
|
||||
* Lightweight library embedded in Conseiller/Excitator/SBOM workers to:
|
||||
|
||||
* Claim work, heartbeat, update progress, report metrics.
|
||||
* Emit artifact metadata and checksums.
|
||||
* Enforce idempotency via orchestrator‑supplied key.
|
||||
|
||||
* **object store**
|
||||
|
||||
* Raw payloads and intermediate artifacts organized by schema and hash:
|
||||
|
||||
* `advisory/raw/<source_id>/<event_time>/<sha256>.json|pdf`
|
||||
* `advisory/normalized/<schema_ver>/<hash>.json`
|
||||
* `vex/raw|normalized/...`
|
||||
* `sbom/raw|graph/...`
|
||||
|
||||
* **web API**
|
||||
|
||||
* CRUD for sources, runs, jobs, schedules, quotas.
|
||||
* Control actions (retry, cancel, pause, backfill).
|
||||
* Streaming updates via WebSocket/SSE for the Console.
|
||||
|
||||
* **console**
|
||||
|
||||
* React app consuming Orchestrator APIs, rendering DAGs, timelines, health charts and action panels with RBAC.
|
||||
|
||||
### 3.3 Data model (selected tables)
|
||||
|
||||
* `sources`
|
||||
|
||||
* `id`, `kind` (`advisory|vex|sbom|internal`), `subtype` (e.g., `osv`, `ghsa`, `csaf`, `vendor_pdf`), `display_name`, `owner_team`, `schedule_cron`, `rate_policy`, `enabled`, `secrets_ref`, `tags`, `schema_hint`, `created_at`, `updated_at`.
|
||||
|
||||
* `runs`
|
||||
|
||||
* `id`, `source_id`, `trigger` (`schedule|manual|event|backfill`), `window_start`, `window_end`, `state`, `started_at`, `finished_at`, `stats_json`.
|
||||
|
||||
* `jobs`
|
||||
|
||||
* `id`, `run_id`, `type`, `queue`, `priority`, `state` (`pending|running|succeeded|failed|canceled|deadletter`), `attempt`, `max_attempt`, `idempotency_key`, `input_artifact_id`, `output_artifact_id`, `worker_id`, `created_at`, `started_at`, `finished_at`, `error_class`, `error_message`, `metrics_json`.
|
||||
|
||||
* `dag_edges`
|
||||
|
||||
* `from_job_id`, `to_job_id`, `edge_kind` (`success_only|always`).
|
||||
|
||||
* `artifacts`
|
||||
|
||||
* `id`, `kind` (`raw|normalized|index|consensus`), `schema_ver`, `hash`, `uri`, `bytes`, `meta_json`, `created_at`.
|
||||
|
||||
* `quotas`
|
||||
|
||||
* `tenant_id`, `resource` (`requests_per_min`, `concurrent_jobs`), `limit`, `window_sec`.
|
||||
|
||||
* `schedules`
|
||||
|
||||
* Per‑source cron plus jitter, timezone, blackout windows.
|
||||
|
||||
### 3.4 Job lifecycle
|
||||
|
||||
1. **Plan**
|
||||
Scheduler creates a `run` for a source and plans a DAG: e.g., `fetch → parse → normalize → dedupe → index → policy_eval` (advisory) or `fetch → parse → normalize → consensus_compute` (VEX).
|
||||
|
||||
2. **Enqueue**
|
||||
Ready nodes become `jobs` with queue, priority, idempotency key and optional rate‑limit tokens reserved.
|
||||
|
||||
3. **Execute**
|
||||
Worker claims job, heartbeats every N seconds. Output artifacts are stored and linked. Failures are classified and retried with exponential backoff and jitter, up to `max_attempt`.
|
||||
|
||||
4. **Complete**
|
||||
Downstream nodes unblock. On run completion, orchestrator computes SLO deltas and emits run summary.
|
||||
|
||||
5. **Dead‑letter**
|
||||
Jobs exceeding attempts move to a DLQ with structured context and suggested remediation.
|
||||
|
||||
### 3.5 Scheduling, backpressure, rate‑limits
|
||||
|
||||
* **Token bucket** per `{tenant, source.host}` with adaptive refill if upstream 429/503 seen.
|
||||
* **Concurrency caps** per source and per job type to avoid thundering herd.
|
||||
* **Backpressure signals** from queue depth, worker CPU, and upstream error rates; scheduler reduces inflight issuance accordingly.
|
||||
* **Backfills** use event‑time windows with immutable watermarks to avoid re‑processing.
|
||||
* **Blackout windows** for vendor maintenance periods.
|
||||
|
||||
### 3.6 APIs
|
||||
|
||||
```
|
||||
POST /orchestrator/sources
|
||||
GET /orchestrator/sources?kind=&tag=&q=
|
||||
GET /orchestrator/sources/{id}
|
||||
PATCH /orchestrator/sources/{id}
|
||||
POST /orchestrator/sources/{id}/actions:test|pause|resume|sync-now
|
||||
POST /orchestrator/sources/{id}/backfill { "from":"2024-01-01", "to":"2024-03-01" }
|
||||
|
||||
GET /orchestrator/runs?source_id=&state=&from=&to=
|
||||
GET /orchestrator/runs/{run_id}
|
||||
GET /orchestrator/runs/{run_id}/dag
|
||||
POST /orchestrator/runs/{run_id}/cancel
|
||||
|
||||
GET /orchestrator/jobs?state=&type=&queue=&source_id=
|
||||
GET /orchestrator/jobs/{job_id}
|
||||
POST /orchestrator/jobs/{job_id}/actions:retry|cancel|prioritize
|
||||
|
||||
GET /orchestrator/metrics/overview
|
||||
GET /orchestrator/errors/top?window=1h
|
||||
GET /orchestrator/quotas
|
||||
PATCH /orchestrator/quotas/{tenant_id}
|
||||
WS /orchestrator/streams/updates
|
||||
```
|
||||
|
||||
### 3.7 Console (Web UI)
|
||||
|
||||
* **Overview**
|
||||
|
||||
* KPI tiles: sources healthy, runs in progress, queue depth, error rate, burn‑rate to SLO.
|
||||
* Heatmap of source health by last 24h success ratio.
|
||||
|
||||
* **Sources**
|
||||
|
||||
* Grid with filters, inline status (active, paused, throttled), next run eta, last error class.
|
||||
* Detail panel: config, secrets status (redacted), schedule, rate limits, ownership, run history, action buttons.
|
||||
|
||||
* **Runs**
|
||||
|
||||
* Timeline (Gantt) with critical path, duration distribution, and per‑stage breakdown.
|
||||
* Run detail: DAG view with node metrics, artifacts, logs, action menu (cancel).
|
||||
|
||||
* **Jobs**
|
||||
|
||||
* Live table with state filters and “tail” view.
|
||||
* Job detail: payload preview (redacted), worker, attempts, stack traces, linked artifacts.
|
||||
|
||||
* **Errors**
|
||||
|
||||
* Clusters by class and signature, suggested remediations (pause source, lower concurrency, patch parser).
|
||||
|
||||
* **Queues & Backpressure**
|
||||
|
||||
* Per‑queue depth, service rate, inflight, age percentiles.
|
||||
* Rate‑limit tokens graphs per source host.
|
||||
|
||||
* **Controls**
|
||||
|
||||
* Backfill wizard with event‑time preview and safety checks.
|
||||
* Canary routing: route 5% of next 100 runs to a new worker pool.
|
||||
|
||||
* **A11y**
|
||||
|
||||
* Keyboard nav, ARIA roles for DAG nodes, live regions for updates, color‑blind friendly graphs.
|
||||
|
||||
### 3.8 CLI
|
||||
|
||||
```
|
||||
stella orch sources list --kind advisory --tag prod
|
||||
stella orch sources add --file source.yaml
|
||||
stella orch sources test <source-id>
|
||||
stella orch sources pause <source-id> # or resume
|
||||
stella orch sources sync-now <source-id>
|
||||
stella orch sources backfill <source-id> --from 2024-01-01 --to 2024-03-01
|
||||
|
||||
stella orch runs list --source <id> --state running
|
||||
stella orch runs show <run-id> --dag
|
||||
stella orch runs cancel <run-id>
|
||||
|
||||
stella orch jobs list --state failed --type parse --limit 100
|
||||
stella orch jobs retry <job-id>
|
||||
stella orch jobs cancel <job-id>
|
||||
stella orch jobs tail --queue normalize --follow
|
||||
|
||||
stella orch quotas get --tenant default
|
||||
stella orch quotas set --tenant default --concurrent-jobs 50 --rpm 1200
|
||||
```
|
||||
|
||||
Exit codes: `0` success, `2` invalid args, `4` not found, `5` denied, `7` precondition failed, `8` rate‑limited.
|
||||
|
||||
### 3.9 RBAC & security
|
||||
|
||||
* **Roles**
|
||||
|
||||
* `Orch.Viewer`: read‑only sources/runs/jobs/metrics.
|
||||
* `Orch.Operator`: perform actions on sources and jobs, launch backfills.
|
||||
* `Orch.Admin`: manage quotas, schedules, connector versions, and delete sources.
|
||||
|
||||
* **Secrets**
|
||||
|
||||
* Stored only as references to your KMS; never persisted in cleartext.
|
||||
* Console shows redact badges and last rotated timestamp.
|
||||
|
||||
* **Tenancy**
|
||||
|
||||
* Source, run, job rows scoped by tenant id.
|
||||
* Queue names and token buckets namespaced per tenant.
|
||||
|
||||
* **Compliance**
|
||||
|
||||
* Full audit log for every operator action with “reason” and optional ticket link.
|
||||
* Exportable run ledger for audits.
|
||||
|
||||
### 3.10 Observability
|
||||
|
||||
* **Metrics (examples)**
|
||||
|
||||
* `orch_jobs_inflight{type,queue}`
|
||||
* `orch_jobs_latency_ms{type,percentile}`
|
||||
* `orch_rate_tokens_available{source}`
|
||||
* `orch_error_rate{source,error_class}`
|
||||
* `orch_slo_burn_rate{source,slo}`
|
||||
* `orch_deadletter_total{source,type}`
|
||||
|
||||
* **Traces**
|
||||
|
||||
* Span per job with baggage: `run_id`, `source_id`, `artifact_id`.
|
||||
* Links across services to Conseiller/Excitator/SBOM workers.
|
||||
|
||||
* **Logs**
|
||||
|
||||
* Structured JSON with correlation ids, attempt numbers and redacted payload previews.
|
||||
|
||||
### 3.11 Performance targets
|
||||
|
||||
* Job dispatch P95 < 150 ms after dependency satisfied.
|
||||
* Scheduler loop P95 < 500 ms for 10k pending jobs.
|
||||
* Console live updates sub‑second at 1k events/sec per tenant.
|
||||
* Backfill throughput ≥ 200 jobs/sec per worker pool with zero dupes.
|
||||
|
||||
### 3.12 Edge cases & behaviors
|
||||
|
||||
* **Upstream 429 storms:** auto‑throttle, pause optional, recommend extended jitter.
|
||||
* **Schema drift:** parser moves job to DLQ with `error_class=schema_mismatch` and opens a change ticket via webhook.
|
||||
* **Flapping source:** circuit breaker opens after N consecutive failures; requires human “resume”.
|
||||
* **Clock skew:** watermark logic uses upstream event time; large skews flagged.
|
||||
* **Idempotency collisions:** new attempt yields no‑op if artifact hash already exists.
|
||||
|
||||
---
|
||||
|
||||
## 4) Implementation plan
|
||||
|
||||
### 4.1 Modules (new and updated)
|
||||
|
||||
* New service: `src/Orchestrator/StellaOps.Orchestrator`
|
||||
|
||||
* `api/` REST + WS handlers
|
||||
* `scheduler/` run planner, DAG builder, watermark/backfill logic
|
||||
* `queues/` publisher and consumer abstractions
|
||||
* `ratelimit/` token bucket and adaptive controller
|
||||
* `state/` Postgres repositories and migrations
|
||||
* `audit/` action logging and export
|
||||
* `metrics/` Prometheus exporters
|
||||
* `security/` tenant scoping, KMS client, secret refs
|
||||
|
||||
* Worker SDKs:
|
||||
|
||||
* `src/Orchestrator/StellaOps.Orchestrator.WorkerSdk.Go` and `src/Orchestrator/StellaOps.Orchestrator.WorkerSdk.Python` with job claim, heartbeat, progress, artifact publish, and structured error reporting.
|
||||
|
||||
* Console:
|
||||
|
||||
* `console/apps/orch/` pages: Overview, Sources, Runs, Jobs, Errors, Queues.
|
||||
* `components/dag-view/`, `components/gantt/`, `components/health-heatmap/`.
|
||||
|
||||
* Updates to existing services:
|
||||
|
||||
* Conseiller/Excitator/SBOM workers adopt SDK and emit artifacts with schema/version/fingerprint.
|
||||
* VEX Lens exposes `consensus_compute` as a jobable operation.
|
||||
* Policy Engine exposes `policy_eval` as a job type for scheduled recalcs.
|
||||
|
||||
### 4.2 Packaging & deployment
|
||||
|
||||
* Containers:
|
||||
|
||||
* `stella/orchestrator:<ver>`
|
||||
* `stella/worker-sdk-examples:<ver>` for canary pools
|
||||
|
||||
* Helm values:
|
||||
|
||||
* Queues/topics, per‑tenant concurrency, rate‑limit defaults, WS replica count.
|
||||
* KMS integration secrets.
|
||||
|
||||
* Migrations:
|
||||
|
||||
* Flyway/Goose migrations for new tables and indexes.
|
||||
|
||||
### 4.3 Rollout strategy
|
||||
|
||||
* Phase 1: Read‑only dashboard fed by existing job tables; no controls.
|
||||
* Phase 2: Control actions enabled for non‑prod tenants.
|
||||
* Phase 3: Backfills and quota management, then GA.
|
||||
|
||||
---
|
||||
|
||||
## 5) Documentation changes
|
||||
|
||||
Create/update the following, each ending with the imposed rule statement.
|
||||
|
||||
1. `/docs/orchestrator/overview.md`
|
||||
Concepts, roles, responsibilities, AOC alignment.
|
||||
|
||||
2. `/docs/orchestrator/architecture.md`
|
||||
Scheduler, DAGs, watermarks, queues, rate‑limits, data model.
|
||||
|
||||
3. `/docs/orchestrator/api.md`
|
||||
Endpoints, WebSocket events, error codes, examples.
|
||||
|
||||
4. `/docs/orchestrator/console.md`
|
||||
Screens, actions, a11y, live updates.
|
||||
|
||||
5. `/docs/orchestrator/cli.md`
|
||||
Commands, examples, exit codes, scripting patterns.
|
||||
|
||||
6. `/docs/orchestrator/run‑ledger.md`
|
||||
Provenance and audit export format.
|
||||
|
||||
7. `/docs/security/secrets‑handling.md`
|
||||
KMS references, redaction rules, operator hygiene.
|
||||
|
||||
8. `/docs/operations/orchestrator‑runbook.md`
|
||||
Common failures, backfill guide, circuit breakers, tuning.
|
||||
|
||||
9. `/docs/schemas/artifacts.md`
|
||||
Artifact kinds, schema versions, hashing, storage layout.
|
||||
|
||||
10. `/docs/slo/orchestrator‑slo.md`
|
||||
SLO definitions, measurement, alerting.
|
||||
|
||||
---
|
||||
|
||||
## 6) Engineering tasks
|
||||
|
||||
### Backend (orchestrator)
|
||||
|
||||
* [ ] Stand up Postgres schemas and indices for sources, runs, jobs, dag_edges, artifacts, quotas, schedules.
|
||||
* [ ] Implement scheduler: DAG planner, dependency resolver, critical path computation.
|
||||
* [ ] Implement rate limiter with adaptive behavior on 429/503 and per‑tenant tokens.
|
||||
* [ ] Implement watermark/backfill manager with event‑time windows and idempotency keys.
|
||||
* [ ] Implement API endpoints + OpenAPI spec + request validation.
|
||||
* [ ] Implement WebSocket/SSE event stream for live updates.
|
||||
* [ ] Implement audit logging and export.
|
||||
* [ ] Implement dead‑letter store and replay.
|
||||
|
||||
### Worker SDKs and integrations
|
||||
|
||||
* [ ] Build Go/Python SDKs with claim/heartbeat/progress API.
|
||||
* [ ] Integrate SDK into Conseiller, Excitator, SBOM workers; ensure artifact emission with schema ver.
|
||||
* [ ] Add `consensus_compute` and `policy_eval` as job types with deterministic inputs/outputs.
|
||||
|
||||
### Console
|
||||
|
||||
* [ ] Overview tiles and health heatmap.
|
||||
* [ ] Source list/detail with actions and config view.
|
||||
* [ ] Runs timeline (Gantt) and DAG visualization with node inspector.
|
||||
* [ ] Jobs “tail” with live updates and filters.
|
||||
* [ ] Errors clustering and suggested remediations.
|
||||
* [ ] Queues/backpressure dashboard.
|
||||
* [ ] Backfill wizard with safety checks.
|
||||
|
||||
### Observability
|
||||
|
||||
* [ ] Emit metrics listed in §3.10 and wire traces across services.
|
||||
* [ ] Dashboards: health, queue depth, error classes, burn‑rate, dispatch latency.
|
||||
* [ ] Alerts for SLO burn and circuit breaker opens.
|
||||
|
||||
### Security & RBAC
|
||||
|
||||
* [ ] Enforce tenant scoping on all endpoints; test leakage.
|
||||
* [ ] Wire KMS for secret refs and redact everywhere.
|
||||
* [ ] Implement `Orch.Viewer|Operator|Admin` roles and check in Console and API.
|
||||
|
||||
### Docs
|
||||
|
||||
* [ ] Author all files in §5 with examples and screenshots.
|
||||
* [ ] Cross‑link from Conseiller/Excitator/SBOM pages to the dashboard docs.
|
||||
* [ ] Append imposed rule to each page.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
---
|
||||
|
||||
## 7) Acceptance criteria
|
||||
|
||||
* Operators can: pause/resume a source, run “sync‑now,” initiate a backfill for a date range, and retry/cancel individual jobs from Console and CLI.
|
||||
* DAG and timeline reflect reality within 1 second of job state changes at P95.
|
||||
* Backfills do not create duplicate artifacts; idempotency proven by hash equality.
|
||||
* Rate limiter reduces 429s by ≥80% under simulated throttle tests.
|
||||
* Audit log includes who/when/why for every operator action.
|
||||
* Provenance ledger exports a complete chain for any artifact.
|
||||
* RBAC prevents non‑admins from quota changes; tenancy isolation proven via automated tests.
|
||||
* SLO dashboard shows burn‑rate and triggers alerts under injected failure.
|
||||
|
||||
---
|
||||
|
||||
## 8) Risks & mitigations
|
||||
|
||||
* **Orchestrator becomes a single bottleneck.**
|
||||
Horizontal scale stateless workers; DB indexes tuned; job state updates batched; cache hot paths.
|
||||
|
||||
* **Secret spillage.**
|
||||
Only KMS references stored; aggressive redaction; log scrubbing in SDK.
|
||||
|
||||
* **Over‑eager backfills overwhelm upstream.**
|
||||
Enforce per‑source quotas and sandbox previews; dry‑run backfills first.
|
||||
|
||||
* **Schema drift silently corrupts normalization.**
|
||||
Hard‑fail on mismatch; DLQ with clear signatures; schema registry gating.
|
||||
|
||||
* **Flapping sources cause alert fatigue.**
|
||||
Circuit breaker with cool‑down and deduped alerts; error budget policy.
|
||||
|
||||
---
|
||||
|
||||
## 9) Test plan
|
||||
|
||||
* **Unit**
|
||||
Scheduler DAG building, topological sort, backoff math, token bucket, watermark math.
|
||||
|
||||
* **Integration**
|
||||
Orchestrator ↔ worker SDK, artifact store wiring, DLQ replay, audit pipeline.
|
||||
|
||||
* **Chaos**
|
||||
Inject 429 storms, packet loss, worker crashes; verify throttling and recovery.
|
||||
|
||||
* **Backfill**
|
||||
Simulate overlapping windows and verify idempotency and watermark correctness.
|
||||
|
||||
* **Perf**
|
||||
10k concurrent jobs: dispatch latency, DB contention, WebSocket fan‑out.
|
||||
|
||||
* **Security**
|
||||
Multi‑tenant isolation tests; KMS mock tests for secret access; RBAC matrix.
|
||||
|
||||
* **UX/A11y**
|
||||
Screen reader labels on DAG, keyboard navigation, live region updates.
|
||||
|
||||
---
|
||||
|
||||
## 10) Philosophy
|
||||
|
||||
* **Make the invisible visible.** Pipelines should be legible at a glance.
|
||||
* **Prefer reproducibility to heroics.** Idempotency and provenance over “we think it ran.”
|
||||
* **Safeguards before speed.** Throttle first, retry thoughtfully, never melt upstreams.
|
||||
* **No silent merges.** Evidence remains immutable; transformations are explicit, logged and reversible.
|
||||
|
||||
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**
|
||||
1831
docs/implplan/EXECPLAN.md
Normal file
1831
docs/implplan/EXECPLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
1096
docs/implplan/SPRINTS.md
Normal file
1096
docs/implplan/SPRINTS.md
Normal file
File diff suppressed because it is too large
Load Diff
208
docs/implplan/SPRINTS_PRIOR_20251019.md
Normal file
208
docs/implplan/SPRINTS_PRIOR_20251019.md
Normal file
@@ -0,0 +1,208 @@
|
||||
Closed sprint tasks archived from SPRINTS.md on 2025-10-19.
|
||||
|
||||
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-12) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-001 | SemVer primitive range-style metadata<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/__Libraries/StellaOps.Concelier.Models/AGENTS.md. This task lays the groundwork—complete the SemVer helper updates before teammates pick up FEEDMODELS-SCHEMA-01-002/003 and FEEDMODELS-SCHEMA-02-900. Use ./src/FASTER_MODELING_AND_NORMALIZATION.md for the target rule structure. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-11) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-002 | Provenance decision rationale field<br>Instructions to work:<br>AdvisoryProvenance now carries `decisionReason` and docs/tests were updated. Connectors and merge tasks should populate the field when applying precedence/freshness/tie-breaker logic; see src/Concelier/__Libraries/StellaOps.Concelier.Models/PROVENANCE_GUIDELINES.md for usage guidance. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-11) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-01-003 | Normalized version rules collection<br>Instructions to work:<br>`AffectedPackage.NormalizedVersions` and supporting comparer/docs/tests shipped. Connector owners must emit rule arrays per ./src/FASTER_MODELING_AND_NORMALIZATION.md and report progress via FEEDMERGE-COORD-02-900 so merge/storage backfills can proceed. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-12) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-02-900 | Range primitives for SemVer/EVR/NEVRA metadata<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/__Libraries/StellaOps.Concelier.Models/AGENTS.md before resuming this stalled effort. Confirm helpers align with the new `NormalizedVersions` representation so connectors finishing in Sprint 2 can emit consistent metadata. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDNORM-NORM-02-001 | SemVer normalized rule emitter<br>Shared `SemVerRangeRuleBuilder` now outputs primitives + normalized rules per `FASTER_MODELING_AND_NORMALIZATION.md`; CVE/GHSA connectors consuming the API have verified fixtures. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill<br>AdvisoryStore dual-writes flattened `normalizedVersions` when `concelier.storage.enableSemVerStyle` is set; migration `20251011-semver-style-backfill` updates historical records and docs outline the rollout. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-002 | Provenance decision reason persistence<br>Storage now persists `provenance.decisionReason` for advisories and merge events; tests cover round-trips. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-003 | Normalized versions indexing<br>Bootstrapper seeds compound/sparse indexes for flattened normalized rules and `docs/dev/mongo_indices.md` documents query guidance. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-TESTS-02-004 | Restore AdvisoryStore build after normalized versions refactor<br>Updated constructors/tests keep storage suites passing with the new feature flag defaults. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-ENGINE-01-002 | Plumb Authority client resilience options<br>WebService wires `authority.resilience.*` into `AddStellaOpsAuthClient` and adds binding coverage via `AuthorityClientResilienceOptionsAreBound`. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-003 | Author ops guidance for resilience tuning<br>Install/runbooks document connected vs air-gapped resilience profiles and monitoring hooks. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-004 | Document authority bypass logging patterns<br>Operator guides now call out `route/status/subject/clientId/scopes/bypass/remote` audit fields and SIEM triggers. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-005 | Update Concelier operator guide for enforcement cutoff<br>Install guide reiterates the 2025-12-31 cutoff and links audit signals to the rollout checklist. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | SEC3.HOST | Rate limiter policy binding<br>Authority host now applies configuration-driven fixed windows to `/token`, `/authorize`, and `/internal/*`; integration tests assert 429 + `Retry-After` headers; docs/config samples refreshed for Docs guild diagrams. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | SEC3.BUILD | Authority rate-limiter follow-through<br>`Security.RateLimiting` now fronts token/authorize/internal limiters; Authority + Configuration matrices (`dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.sln`, `dotnet test src/__Libraries/__Tests/StellaOps.Configuration.Tests/StellaOps.Configuration.Tests.csproj`) passed on 2025-10-11; awaiting #authority-core broadcast. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-14) | Team Authority Platform & Security Guild | AUTHCORE-BUILD-OPENIDDICT / AUTHCORE-STORAGE-DEVICE-TOKENS / AUTHCORE-BOOTSTRAP-INVITES | Address remaining Authority compile blockers (OpenIddict transaction shim, token device document, bootstrap invite cleanup) so `dotnet build src/Authority/StellaOps.Authority/StellaOps.Authority.sln` returns success. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | PLG6.DOC | Plugin developer guide polish<br>Section 9 now documents rate limiter metadata, config keys, and lockout interplay; YAML samples updated alongside Authority config templates. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-001 | Fetch pipeline & state tracking<br>Summary planner now drives monthly/yearly VINCE fetches, persists pending summaries/notes, and hydrates VINCE detail queue with telemetry.<br>Team instructions: Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/AGENTS.md. Coordinate daily with Models/Merge leads so new normalizedVersions output and provenance tags stay aligned with ./src/FASTER_MODELING_AND_NORMALIZATION.md. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-002 | VINCE note detail fetcher<br>Summary planner queues VINCE note detail endpoints, persists raw JSON with SHA/ETag metadata, and records retry/backoff metrics. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-003 | DTO & parser implementation<br>Added VINCE DTO aggregate, Markdown→text sanitizer, vendor/status/vulnerability parsers, and parser regression fixture. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-004 | Canonical mapping & range primitives<br>VINCE DTO aggregate flows through `CertCcMapper`, emitting vendor range primitives + normalized version rules that persist via `_advisoryStore`. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-005 | Deterministic fixtures/tests<br>Snapshot harness refreshed 2025-10-12; `certcc-*.snapshot.json` regenerated and regression suite green without UPDATE flag drift. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-006 | Telemetry & documentation<br>`CertCcDiagnostics` publishes summary/detail/parse/map metrics (meter `StellaOps.Concelier.Connector.CertCc`), README documents instruments, and log guidance captured for Ops on 2025-10-12. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-007 | Connector test harness remediation<br>Harness now wires `AddSourceCommon`, resets `FakeTimeProvider`, and passes canned-response regression run dated 2025-10-12. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-008 | Snapshot coverage handoff<br>Fixtures regenerated with normalized ranges + provenance fields on 2025-10-11; QA handoff notes published and merge backfill unblocked. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-012 | Schema sync & snapshot regen follow-up<br>Fixtures regenerated with normalizedVersions + provenance decision reasons; handoff notes updated for Merge backfill 2025-10-12. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-009 | Detail/map reintegration plan<br>Staged reintegration plan published in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/FEEDCONN-CERTCC-02-009_PLAN.md`; coordinates enablement with FEEDCONN-CERTCC-02-004. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertCc/TASKS.md | DONE (2025-10-12) | Team Connector Resumption – CERT/RedHat | FEEDCONN-CERTCC-02-010 | Partial-detail graceful degradation<br>Detail fetch now tolerates 404/403/410 responses and regression tests cover mixed endpoint availability. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md | DONE (2025-10-11) | Team Connector Resumption – CERT/RedHat | FEEDCONN-REDHAT-02-001 | Fixture validation sweep<br>Instructions to work:<br>Fixtures regenerated post-model-helper rollout; provenance ordering and normalizedVersions scaffolding verified via tests. Conflict resolver deltas logged in src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Distro.RedHat/CONFLICT_RESOLVER_NOTES.md for Sprint 3 consumers. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-001 | Canonical mapping & range primitives<br>Mapper emits SemVer rules (`scheme=apple:*`); fixtures regenerated with trimmed references + new RSR coverage, update tooling finalized. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-002 | Deterministic fixtures/tests<br>Sanitized live fixtures + regression snapshots wired into tests; normalized rule coverage asserted. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-003 | Telemetry & documentation<br>Apple meter metrics wired into Concelier WebService OpenTelemetry configuration; README and fixtures document normalizedVersions coverage. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-12) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-004 | Live HTML regression sweep<br>Sanitised HT125326/HT125328/HT106355/HT214108/HT215500 fixtures recorded and regression tests green on 2025-10-12. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md | DONE (2025-10-11) | Team Vendor Apple Specialists | FEEDCONN-APPLE-02-005 | Fixture regeneration tooling<br>`UPDATE_APPLE_FIXTURES=1` flow fetches & rewrites fixtures; README documents usage.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Apple/AGENTS.md. Resume stalled tasks, ensuring normalizedVersions output and fixtures align with ./src/FASTER_MODELING_AND_NORMALIZATION.md before handing data to the conflict sprint. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `tools/FixtureUpdater` updates across connectors. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-CVE-02-003 | CVE normalized versions uplift |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-KEV-02-003 | KEV normalized versions propagation |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-OSV-04-003 | OSV parity fixture refresh |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-10) | Team WebService & Authority | FEEDWEB-DOCS-01-001 | Document authority toggle & scope requirements<br>Quickstart carries toggle/scope guidance pending docs guild review (no change this sprint). |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-003 | Author ops guidance for resilience tuning<br>Operator docs now outline connected vs air-gapped resilience profiles and monitoring cues. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-004 | Document authority bypass logging patterns<br>Audit logging guidance highlights `route/status/subject/clientId/scopes/bypass/remote` fields and SIEM alerts. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-12) | Team WebService & Authority | FEEDWEB-DOCS-01-005 | Update Concelier operator guide for enforcement cutoff<br>Install guide reiterates the 2025-12-31 cutoff and ties audit signals to rollout checks. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs, tests, and docs now target `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-11) | Team WebService & Authority | FEEDWEB-OPS-01-007 | Authority resilience adoption<br>Deployment docs and CLI notes explain the LIB5 resilience knobs for rollout.<br>Instructions to work:<br>DONE Read ./AGENTS.md and src/Concelier/StellaOps.Concelier.WebService/AGENTS.md. These items were mid-flight; resume implementation ensuring docs/operators receive timely updates. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHCORE-ENGINE-01-001 | CORE8.RL — Rate limiter plumbing validated; integration tests green and docs handoff recorded for middleware ordering + Retry-After headers (see `docs/dev/authority-rate-limit-tuning-outline.md` for continuing guidance). |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHCRYPTO-ENGINE-01-001 | SEC3.A — Shared metadata resolver confirmed via host test run; SEC3.B now unblocked for tuning guidance (outline captured in `docs/dev/authority-rate-limit-tuning-outline.md`). |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-13) | Team Authority Platform & Security Guild | AUTHSEC-DOCS-01-002 | SEC3.B — Published `docs/security/rate-limits.md` with tuning matrix, alert thresholds, and lockout interplay guidance; Docs guild can lift copy into plugin guide. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-14) | Team Authority Platform & Security Guild | AUTHSEC-CRYPTO-02-001 | SEC5.B1 — Introduce libsodium signing provider and parity tests to unblock CLI verification enhancements. |
|
||||
| Sprint 1 | Bootstrap & Replay Hardening | src/__Libraries/StellaOps.Cryptography/TASKS.md | DONE (2025-10-14) | Security Guild | AUTHSEC-CRYPTO-02-004 | SEC5.D/E — Finish bootstrap invite lifecycle (API/store/cleanup) and token device heuristics; build currently red due to pending handler integration. |
|
||||
| Sprint 1 | Developer Tooling | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-15) | DevEx/CLI | AUTHCLI-DIAG-01-001 | Surface password policy diagnostics in CLI startup/output so operators see weakened overrides immediately.<br>CLI now loads Authority plug-ins at startup, logs weakened password policies (length/complexity), and regression coverage lives in `StellaOps.Cli.Tests/Services/AuthorityDiagnosticsReporterTests`. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md | DONE (2025-10-11) | Team Authority Platform & Security Guild | AUTHPLUG-DOCS-01-001 | PLG6.DOC — Developer guide copy + diagrams merged 2025-10-11; limiter guidance incorporated and handed to Docs guild for asset export. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md | DONE (2025-10-12) | Team Normalization & Storage Backbone | FEEDNORM-NORM-02-001 | SemVer normalized rule emitter<br>`SemVerRangeRuleBuilder` shipped 2025-10-12 with comparator/`||` support and fixtures aligning to `FASTER_MODELING_AND_NORMALIZATION.md`. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-002 | Provenance decision reason persistence |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-02-003 | Normalized versions indexing<br>Indexes seeded + docs updated 2025-10-11 to cover flattened normalized rules for connector adoption. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Normalization & Storage Backbone | FEEDMERGE-ENGINE-02-002 | Normalized versions union & dedupe<br>Affected package resolver unions/dedupes normalized rules, stamps merge provenance with `decisionReason`, and tests cover the rollout. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-004 | GHSA credits & ecosystem severity mapping |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-005 | GitHub quota monitoring & retries |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-006 | Production credential & scheduler rollout |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-GHSA-02-007 | Credit parity regression fixtures |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-002 | NVD normalized versions & timestamps |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-004 | NVD CVSS & CWE precedence payloads |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-NVD-02-005 | NVD merge/export parity regression |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-003 | OSV normalized versions & freshness |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-11) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-004 | OSV references & credits alignment |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-02-005 | Fixture updater workflow<br>Resolved 2025-10-12: OSV mapper now derives canonical PURLs for Go + scoped npm packages when raw payloads omit `purl`; conflict fixtures unchanged for invalid npm names. Verified via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests`, `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests`, and backbone normalization/storage suites. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Acsc/TASKS.md | DONE (2025-10-12) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ACSC-02-001 … 02-008 | Fetch→parse→map pipeline, fixtures, diagnostics, and README finished 2025-10-12; downstream export parity captured via FEEDEXPORT-JSON-04-001 / FEEDEXPORT-TRIVY-04-001 (completed). |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cccs/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CCCS-02-001 … 02-008 | Observability meter, historical harvest plan, and DOM sanitizer refinements wrapped; ops notes live under `docs/ops/concelier-cccs-operations.md` with fixtures validating EN/FR list handling. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertBund/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CERTBUND-02-001 … 02-008 | Telemetry/docs (02-006) and history/locale sweep (02-007) completed alongside pipeline; runbook `docs/ops/concelier-certbund-operations.md` captures locale guidance and offline packaging. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kisa/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-KISA-02-001 … 02-007 | Connector, tests, and telemetry/docs (02-006) finalized; localisation notes in `docs/dev/kisa_connector_notes.md` complete rollout. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-RUBDU-02-001 … 02-008 | Fetch/parser/mapper refinements, regression fixtures, telemetry/docs, access options, and trusted root packaging all landed; README documents offline access strategy. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md | DONE (2025-10-13) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-NKCKI-02-001 … 02-008 | Listing fetch, parser, mapper, fixtures, telemetry/docs, and archive plan finished; Mongo2Go/libcrypto dependency resolved via bundled OpenSSL noted in ops guide. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-ICSCISA-02-001 … 02-011 | Feed parser attachment fixes, SemVer exact values, regression suites, telemetry/docs updates, and handover complete; ops runbook now details attachment verification + proxy usage. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md | DONE (2025-10-14) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-CISCO-02-001 … 02-007 | OAuth fetch pipeline, DTO/mapping, tests, and telemetry/docs shipped; monitoring/export integration follow-ups recorded in Ops docs and exporter backlog (completed). |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md | DONE (2025-10-15) | Team Connector Expansion – Regional & Vendor Feeds | FEEDCONN-MSRC-02-001 … 02-008 | Azure AD onboarding (02-008) unblocked fetch/parse/map pipeline; fixtures, telemetry/docs, and Offline Kit guidance published in `docs/ops/concelier-msrc-operations.md`. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve/TASKS.md | DONE (2025-10-15) | Team Connector Support & Monitoring | FEEDCONN-CVE-02-001 … 02-002 | CVE data-source selection, fetch pipeline, and docs landed 2025-10-10. 2025-10-15: smoke verified using the seeded mirror fallback; connector now logs a warning and pulls from `seed-data/cve/` until live CVE Services credentials arrive. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev/TASKS.md | DONE (2025-10-12) | Team Connector Support & Monitoring | FEEDCONN-KEV-02-001 … 02-002 | KEV catalog ingestion, fixtures, telemetry, and schema validation completed 2025-10-12; ops dashboard published. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-01-001 | Canonical schema docs refresh<br>Updated canonical schema + provenance guides with SemVer style, normalized version rules, decision reason change log, and migration notes. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-02-001 | Concelier-SemVer Playbook<br>Published merge playbook covering mapper patterns, dedupe flow, indexes, and rollout checklist. |
|
||||
| Sprint 2 | Connector & Data Implementation Wave | docs/TASKS.md | DONE (2025-10-11) | Team Docs & Knowledge Base | FEEDDOCS-DOCS-02-002 | Normalized versions query guide<br>Delivered Mongo index/query addendum with `$unwind` recipes, dedupe checks, and operational checklist.<br>Instructions to work:<br>DONE Read ./AGENTS.md and docs/AGENTS.md. Document every schema/index/query change produced in Sprint 1-2 leveraging ./src/FASTER_MODELING_AND_NORMALIZATION.md. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-03-001 | Canonical merger implementation<br>`CanonicalMerger` ships with freshness/tie-breaker logic, provenance, and unit coverage feeding Merge. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-03-002 | Field precedence and tie-breaker map<br>Field precedence tables and tie-breaker metrics wired into the canonical merge flow; docs/tests updated.<br>Instructions to work:<br>Read ./AGENTS.md and core AGENTS. Implement the conflict resolver exactly as specified in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md, coordinating with Merge and Storage teammates. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-DATA-03-001 | Merge event provenance audit prep<br>Merge events now persist `fieldDecisions` and analytics-ready provenance snapshots. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-DATA-02-001 | Normalized range dual-write + backfill<br>Dual-write/backfill flag delivered; migration + options validated in tests. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-11) | Team Core Engine & Storage Analytics | FEEDSTORAGE-TESTS-02-004 | Restore AdvisoryStore build after normalized versions refactor<br>Storage tests adjusted for normalized versions/decision reasons.<br>Instructions to work:<br>Read ./AGENTS.md and storage AGENTS. Extend merge events with decision reasons and analytics views to support the conflict rules, and deliver the dual-write/backfill for `NormalizedVersions` + `decisionReason` so connectors can roll out safely. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-001 | GHSA/NVD/OSV conflict rules<br>Merge pipeline consumes `CanonicalMerger` output prior to precedence merge. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-002 | Override metrics instrumentation<br>Merge events capture per-field decisions; counters/logs align with conflict rules. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-003 | Reference & credit union pipeline<br>Canonical merge preserves unions with updated tests. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-11) | Team Merge & QA Enforcement | FEEDMERGE-QA-04-001 | End-to-end conflict regression suite<br>Added regression tests (`AdvisoryMergeServiceTests`) covering canonical + precedence flow.<br>Instructions to work:<br>Read ./AGENTS.md and merge AGENTS. Integrate the canonical merger, instrument metrics, and deliver comprehensive regression tests following ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-002 | GHSA conflict regression fixtures |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-NVD-04-002 | NVD conflict regression fixtures |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-12) | Team Connector Regression Fixtures | FEEDCONN-OSV-04-002 | OSV conflict regression fixtures<br>Instructions to work:<br>Read ./AGENTS.md and module AGENTS. Produce fixture triples supporting the precedence/tie-breaker paths defined in ./src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md and hand them to Merge QA. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-11) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-001 | Concelier Conflict Rules<br>Runbook published at `docs/ops/concelier-conflict-resolution.md`; metrics/log guidance aligned with Sprint 3 merge counters. |
|
||||
| Sprint 3 | Conflict Resolution Integration & Communications | docs/TASKS.md | DONE (2025-10-16) | Team Documentation Guild – Conflict Guidance | FEEDDOCS-DOCS-05-002 | Conflict runbook ops rollout<br>Ops review completed, alert thresholds applied, and change log appended in `docs/ops/concelier-conflict-resolution.md`; task closed after connector signals verified. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md | DONE (2025-10-15) | Team Models & Merge Leads | FEEDMODELS-SCHEMA-04-001 | Advisory schema parity (description/CWE/canonical metric)<br>Extend `Advisory` and related records with description text, CWE collection, and canonical metric pointer; refresh validation + serializer determinism tests. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-003 | Canonical merger parity for new fields<br>Teach `CanonicalMerger` to populate description, CWEResults, and canonical metric pointer with provenance + regression coverage. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-15) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-04-004 | Reference normalization & freshness instrumentation cleanup<br>Implement URL normalization for reference dedupe, align freshness-sensitive instrumentation, and add analytics tests. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-004 | Merge pipeline parity for new advisory fields<br>Ensure merge service + merge events surface description/CWE/canonical metric decisions with updated metrics/tests. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` validated canonical metric/CWE emission. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-16) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. |
|
||||
| Sprint 4 | Schema Parity & Freshness Alignment | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-15) | Team Excititor Core & Policy | EXCITITOR-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-15) | Team Excititor Policy | EXCITITOR-POLICY-01-001 | Established policy options & snapshot provider covering baseline weights/overrides. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-15) | Team Excititor Policy | EXCITITOR-POLICY-01-002 | Policy evaluator now feeds consensus resolver with immutable snapshots. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-003 | Author policy diagnostics, CLI/WebService surfacing, and documentation updates. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-16) | Team Excititor Policy | EXCITITOR-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Excititor Storage | EXCITITOR-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-16) | Team Excititor Storage | EXCITITOR-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-15) | Team Excititor Export | EXCITITOR-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-17) | Team Excititor Export | EXCITITOR-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors | EXCITITOR-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. |
|
||||
| Sprint 5 | Excititor Core Foundations | src/Excititor/StellaOps.Excititor.WebService/TASKS.md | DONE (2025-10-17) | Team Excititor WebService | EXCITITOR-WEB-01-001 | Scaffold minimal API host, DI, and `/excititor/status` endpoint integrating policy, storage, export, and attestation services. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/StellaOps.Excititor.Worker/TASKS.md | DONE (2025-10-17) | Team Excititor Worker | EXCITITOR-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md | DONE (2025-10-17) | Team Excititor Formats | EXCITITOR-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-002 | Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-003 | Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-004 | Persist resume cursors (last updated timestamp/document hashes) in storage and reload during fetch to avoid duplicates. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-005 | Register connector in Worker/WebService DI, add scheduled jobs, and document CLI triggers for Red Hat CSAF pulls. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Red Hat | EXCITITOR-CONN-RH-01-006 | Add CSAF normalization parity fixtures ensuring RHSA-specific metadata is preserved. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Cisco | EXCITITOR-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Cisco | EXCITITOR-CONN-CISCO-01-002 | Implement Cisco CSAF paginated fetch loop with dedupe and raw persistence support. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – SUSE | EXCITITOR-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – MSRC | EXCITITOR-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Oracle | EXCITITOR-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md | DONE (2025-10-17) | Team Excititor Connectors – Ubuntu | EXCITITOR-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-002 | Attestation fetch & verify loop – download DSSE attestations, trigger verification, handle retries/backoff, persist raw statements. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md | DONE (2025-10-18) | Team Excititor Connectors – OCI | EXCITITOR-CONN-OCI-01-003 | Provenance metadata & policy hooks – emit image, subject digest, issuer, and trust metadata for policy weighting/logging. |
|
||||
| Sprint 6 | Excititor Ingest & Formats | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-18) | DevEx/CLI | EXCITITOR-CLI-01-001 | Add `excititor` CLI verbs bridging to WebService with consistent auth and offline UX. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md | DONE (2025-10-19) | Team Excititor Core & Policy | EXCITITOR-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md | DONE (2025-10-19) | Team Excititor Policy | EXCITITOR-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md | DONE (2025-10-16) | Team Excititor Attestation | EXCITITOR-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-501 | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `ARCHITECTURE_SCANNER.md` §3–§4. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-502 | Observability helpers (correlation IDs, logging scopes, metric namespacing, deterministic hashes) consumed by WebService/Worker. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md | DONE (2025-10-18) | Team Scanner Core | SCANNER-CORE-09-503 | Security utilities: Authority client factory, OpTok caching, DPoP verifier, restart-time plug-in guardrails for scanner components. |
|
||||
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-001 | Buildx driver scaffold + handshake with Scanner.Emit (local CAS). |
|
||||
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-002 | OCI annotations + provenance hand-off to Attestor. |
|
||||
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-003 | CI demo: minimal SBOM push & backend report wiring. |
|
||||
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-004 | Stabilize descriptor nonce derivation so repeated builds emit deterministic placeholders. |
|
||||
| Sprint 9 | Scanner Build-time | src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-005 | Integrate determinism guard into GitHub/Gitea workflows and archive proof artifacts. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-18) | Team Scanner WebService | SCANNER-WEB-09-101 | Minimal API host with Authority enforcement, health/ready endpoints, and restart-time plug-in loader per architecture §1, §4. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-18) | Team Scanner WebService | SCANNER-WEB-09-102 | `/api/v1/scans` submission/status endpoints with deterministic IDs, validation, and cancellation support. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-WEB-09-104 | Configuration binding for Mongo, MinIO, queue, feature flags; startup diagnostics and fail-fast policy. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-201 | Worker host bootstrap with Authority auth, hosted services, and graceful shutdown semantics. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-202 | Lease/heartbeat loop with retry+jitter, poison-job quarantine, structured logging. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-203 | Analyzer dispatch skeleton emitting deterministic stage progress and honoring cancellation tokens. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-204 | Worker metrics (queue latency, stage duration, failure counts) with OpenTelemetry resource wiring. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.Worker/TASKS.md | DONE (2025-10-19) | Team Scanner Worker | SCANNER-WORKER-09-205 | Harden heartbeat jitter so lease safety margin stays ≥3× and cover with regression tests + optional live queue smoke run. |
|
||||
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-001 | Policy schema + binder + diagnostics. |
|
||||
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-002 | Policy snapshot store + revision digests. |
|
||||
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE | Policy Guild | POLICY-CORE-09-003 | `/policy/preview` API (image digest → projected verdict diff). |
|
||||
| Sprint 9 | DevOps Foundations | ops/devops/TASKS.md | DONE (2025-10-19) | DevOps Guild | DEVOPS-HELM-09-001 | Helm/Compose environment profiles (dev/staging/airgap) with deterministic digests. |
|
||||
| Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-19) | Docs Guild, DevEx | DOCS-ADR-09-001 | Establish ADR process and template. |
|
||||
| Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-19) | Docs Guild, Platform Events | DOCS-EVENTS-09-002 | Publish event schema catalog (`docs/events/`) for critical envelopes. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-301 | Mongo catalog schemas/indexes for images, layers, artifacts, jobs, lifecycle rules plus migrations. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-302 | MinIO layout, immutability policies, client abstraction, and configuration binding. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-19) | Team Scanner Storage | SCANNER-STORAGE-09-303 | Repositories/services with dual-write feature flag, deterministic digests, TTL enforcement tests. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-401 | Queue abstraction + Redis Streams adapter with ack/claim APIs and idempotency tokens. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-402 | Pluggable backend support (Redis, NATS) with configuration binding, health probes, failover docs. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md | DONE (2025-10-19) | Team Scanner Queue | SCANNER-QUEUE-09-403 | Retry + dead-letter strategy with structured logs/metrics for offline deployments. |
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa/TASKS.md | DONE (2025-10-12) | Team Connector Normalized Versions Rollout | FEEDCONN-GHSA-02-001 | GHSA normalized versions & provenance<br>Team instructions: Read ./AGENTS.md and each module's AGENTS file. Adopt the `NormalizedVersions` array emitted by the models sprint, wiring provenance `decisionReason` where merge overrides occur. Follow ./src/FASTER_MODELING_AND_NORMALIZATION.md; report via src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md (FEEDMERGE-COORD-02-900). Progress 2025-10-11: GHSA/OSV emit normalized arrays with refreshed fixtures; CVE mapper now surfaces SemVer normalized ranges; NVD/KEV adoption pending; outstanding follow-ups include FEEDSTORAGE-DATA-02-001, FEEDMERGE-ENGINE-02-002, and rolling `tools/FixtureUpdater` updates across connectors.<br>Progress 2025-10-20: Coordination matrix + rollout dashboard refreshed; upcoming deadlines tracked (Cccs/Cisco 2025-10-21, CertBund 2025-10-22, ICS-CISA 2025-10-23, KISA 2025-10-24) with escalation path documented in FEEDMERGE-COORD-02-900.|
|
||||
| Sprint 1 | Stabilize In-Progress Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Team WebService & Authority | FEEDWEB-OPS-01-006 | Rename plugin drop directory to namespaced path<br>Build outputs now point at `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`; defaults/docs/tests updated to reflect the new layout. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-02-001 | Statement events & scoring signals – immutable VEX statements store, consensus signal fields, and migration `20251019-consensus-signals-statements` with tests (`dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`, `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`). |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-19) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-19) | Concelier WebService Guild | FEEDWEB-EVENTS-07-001 | Advisory event replay API – expose `/concelier/advisories/{key}/replay` with `asOf` filter, hex hashes, and conflict data. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md | DONE (2025-10-20) | BE-Merge | FEEDMERGE-ENGINE-07-001 | Conflict sets & explainers – persist conflict materialization and replay hashes for merge decisions. |
|
||||
| Sprint 8 | Mongo strengthening | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Normalization & Storage Backbone | FEEDSTORAGE-MONGO-08-001 | Causal-consistent Concelier storage sessions<br>Scoped session facilitator registered, repositories accept optional session handles, and replica-set failover tests verify read-your-write + monotonic reads. |
|
||||
| Sprint 8 | Mongo strengthening | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-19) | Authority Core & Storage Guild | AUTHSTORAGE-MONGO-08-001 | Harden Authority Mongo usage<br>Scoped Mongo sessions with majority read/write concerns wired through stores and GraphQL/HTTP pipelines; replica-set election regression validated. |
|
||||
| Sprint 8 | Mongo strengthening | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-MONGO-08-001 | Causal consistency for Excititor repositories<br>Session-scoped repositories shipped with new Mongo records, orchestrators/workers now share scoped sessions, and replica-set failover coverage added via `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Storage.Mongo.Tests/StellaOps.Excititor.Storage.Mongo.Tests.csproj`. |
|
||||
| Sprint 8 | Platform Maintenance | src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Excititor Storage | EXCITITOR-STORAGE-03-001 | Statement backfill tooling – shipped admin backfill endpoint, CLI hook (`stellaops excititor backfill-statements`), integration tests, and operator runbook (`docs/dev/EXCITITOR_STATEMENT_BACKFILL.md`). |
|
||||
| Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json/TASKS.md | DONE (2025-10-19) | Concelier Export Guild | CONCELIER-EXPORT-08-201 | Mirror bundle + domain manifest – produce signed JSON aggregates for `*.stella-ops.org` mirrors. |
|
||||
| Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md | DONE (2025-10-19) | Concelier Export Guild | CONCELIER-EXPORT-08-202 | Mirror-ready Trivy DB bundles – mirror options emit per-domain manifests/metadata/db archives with deterministic digests for downstream sync. |
|
||||
| Sprint 8 | Mirror Distribution | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-20) | Concelier WebService Guild | CONCELIER-WEB-08-201 | Mirror distribution endpoints – expose domain-scoped index/download APIs with auth/quota. |
|
||||
| Sprint 8 | Mirror Distribution | ops/devops/TASKS.md | DONE (2025-10-19) | DevOps Guild | DEVOPS-MIRROR-08-001 | Managed mirror deployments for `*.stella-ops.org` – Helm/Compose overlays, CDN, runbooks. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-003 | Refactor Authority identity-provider registry to resolve scoped plugin services on-demand.<br>Introduce factory pattern aligned with scoped lifetimes decided in coordination workshop. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-004 | Update Authority plugin loader to activate registrars with DI support and scoped service awareness.<br>Add two-phase initialization allowing scoped dependencies post-container build. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-005 | Provide scoped-safe bootstrap execution for Authority plugins.<br>Implement scope-per-run pattern for hosted bootstrap tasks and document migration guidance. |
|
||||
| Sprint 10 | DevOps Security | ops/devops/TASKS.md | DONE (2025-10-20) | DevOps Guild | DEVOPS-SEC-10-301 | Address NU1902/NU1903 advisories for `MongoDB.Driver` 2.12.0 and `SharpCompress` 0.23.0; Wave 0A prerequisites confirmed complete before remediation work. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-20) | Authority Core & Security Guild | AUTH-DPOP-11-001 | Implement DPoP proof validation + nonce handling for high-value audiences per architecture. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/StellaOps.Notify.WebService/TASKS.md | DONE (2025-10-19) | Notify WebService Guild | NOTIFY-WEB-15-103 | Delivery history & test-send endpoints. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Slack/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-SLACK-15-502 | Slack health/test-send support. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-602 | Teams health/test-send support. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-604 | Teams health endpoint metadata alignment. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Slack/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-SLACK-15-503 | Package Slack connector as restart-time plug-in (manifest + host registration). |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Teams/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-TEAMS-15-603 | Package Teams connector as restart-time plug-in (manifest + host registration). |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Email/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-EMAIL-15-703 | Package Email connector as restart-time plug-in (manifest + host registration). |
|
||||
| Sprint 15 | Notify Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-20) | Scanner WebService Guild | SCANNER-EVENTS-15-201 | Emit `scanner.report.ready` + `scanner.scan.completed` events. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Connectors.Webhook/TASKS.md | DONE (2025-10-20) | Notify Connectors Guild | NOTIFY-CONN-WEBHOOK-15-803 | Package Webhook connector as restart-time plug-in (manifest + host registration). |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-103 | Versioning/migration helpers for schedules/runs. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Queue abstraction + Redis Streams adapter. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-402 | NATS JetStream adapter with health probes. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-300 | **STUB** ImpactIndex ingest/query using fixtures (to be removed by SP16 completion). |
|
||||
88
docs/implplan/SPRINTS_PRIOR_20251021.md
Normal file
88
docs/implplan/SPRINTS_PRIOR_20251021.md
Normal file
@@ -0,0 +1,88 @@
|
||||
This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not).
|
||||
|
||||
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| Sprint 7 | Contextual Truth Foundations | docs/TASKS.md | DONE (2025-10-22) | Docs Guild, Concelier WebService | DOCS-CONCELIER-07-201 | Final editorial review and publish pass for Concelier authority toggle documentation (Quickstart + operator guide). |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/StellaOps.Excititor.WebService/TASKS.md | DONE (2025-10-20) | Team Excititor WebService | EXCITITOR-WEB-01-002 | Ingest & reconcile endpoints – scope-enforced `/excititor/init`, `/excititor/ingest/run`, `/excititor/ingest/resume`, `/excititor/reconcile`; regression via `dotnet test … --filter FullyQualifiedName~IngestEndpointsTests`. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/StellaOps.Excititor.WebService/TASKS.md | DONE (2025-10-20) | Team Excititor WebService | EXCITITOR-WEB-01-004 | Resolve API & signed responses – expose `/excititor/resolve`, return signed consensus/score envelopes, document auth. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/StellaOps.Excititor.Worker/TASKS.md | DONE (2025-10-21) | Team Excititor Worker | EXCITITOR-WORKER-01-004 | TTL refresh & stability damper – schedule re-resolve loops and guard against status flapping. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-21) | Team Core Engine & Data Science | FEEDCORE-ENGINE-07-002 | Noise prior computation service – learn false-positive priors and expose deterministic summaries. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | DONE (2025-10-21) | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-003 | Unknown state ledger & confidence seeding – persist unknown flags, seed confidence bands, expose query surface. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/StellaOps.Excititor.WebService/TASKS.md | DONE (2025-10-19) | Team Excititor WebService | EXCITITOR-WEB-01-005 | Mirror distribution endpoints – expose download APIs for downstream Excititor instances. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-21) | Team Excititor Export | EXCITITOR-EXPORT-01-005 | Score & resolve envelope surfaces – include signed consensus/score artifacts in exports. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-21) | Team Excititor Export | EXCITITOR-EXPORT-01-006 | Quiet provenance packaging – attach quieted-by statement IDs, signers, justification codes to exports and attestations. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md | DONE (2025-10-21) | Team Excititor Export | EXCITITOR-EXPORT-01-007 | Mirror bundle + domain manifest – publish signed consensus bundles for mirrors. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.md | DONE (2025-10-21) | Excititor Connectors – Stella | EXCITITOR-CONN-STELLA-07-001 | Excititor mirror connector – ingest signed mirror bundles and map to VexClaims with resume handling. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Team Normalization & Storage Backbone | FEEDSTORAGE-DATA-07-001 | Advisory statement & conflict collections – provision Mongo schema/indexes for event-sourced merge. |
|
||||
| Sprint 7 | Contextual Truth Foundations | src/Web/StellaOps.Web/TASKS.md | DONE (2025-10-21) | UX Specialist, Angular Eng | WEB1.TRIVY-SETTINGS-TESTS | Add headless UI test run (`ng test --watch=false`) and document prerequisites once Angular tooling is chained up. |
|
||||
| Sprint 8 | Mirror Distribution | src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-001 | Concelier mirror connector – fetch mirror manifest, verify signatures, and hydrate canonical DTOs with resume support. |
|
||||
| Sprint 8 | Mirror Distribution | src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-002 | Map mirror payloads into canonical advisory DTOs with provenance referencing mirror domain + original source metadata. |
|
||||
| Sprint 8 | Mirror Distribution | src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-003 | Add incremental cursor + resume support (per-export fingerprint) and document configuration for downstream Concelier instances. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-21) | Plugin Platform Guild | PLUGIN-DI-08-001 | Scoped service support in plugin bootstrap – added dynamic plugin tests ensuring `[ServiceBinding]` metadata flows through plugin hosts and remains idempotent. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-002.COORD | Authority scoped-service integration handshake<br>Workshop concluded 2025-10-20 15:00–16:05 UTC; decisions + follow-ups recorded in `docs/dev/authority-plugin-di-coordination.md`. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/__Libraries/StellaOps.Plugin/TASKS.md | DONE (2025-10-20) | Plugin Platform Guild, Authority Core | PLUGIN-DI-08-002 | Authority plugin integration updates – scoped identity-provider services with registry handles; regression coverage via scoped registrar/unit tests. |
|
||||
| Sprint 8 | Plugin Infrastructure | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-20) | Authority Core, Plugin Platform Guild | AUTH-PLUGIN-COORD-08-002 | Coordinate scoped-service adoption for Authority plug-in registrars<br>Workshop notes and follow-up backlog captured 2025-10-20 in `docs/dev/authority-plugin-di-coordination.md`. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-WEB-09-103 | Progress streaming (SSE/JSONL) with correlation IDs and ISO-8601 UTC timestamps, documented in API reference. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-POLICY-09-105 | Policy snapshot loader + schema + OpenAPI (YAML ignore rules, VEX include/exclude, vendor precedence). |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-POLICY-09-106 | `/reports` verdict assembly (Feedser+Vexer+Policy) + signed response envelope. |
|
||||
| Sprint 9 | Scanner Core Foundations | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-19) | Team Scanner WebService | SCANNER-POLICY-09-107 | Expose score inputs, config version, and quiet provenance in `/reports` JSON and signed payload. |
|
||||
| Sprint 9 | DevOps Foundations | ops/devops/TASKS.md | DONE (2025-10-21) | DevOps Guild, Scanner WebService Guild | DEVOPS-SCANNER-09-204 | Surface `SCANNER__EVENTS__*` env config across Compose/Helm and document overrides. |
|
||||
| Sprint 9 | DevOps Foundations | ops/devops/TASKS.md | DONE (2025-10-21) | DevOps Guild, Notify Guild | DEVOPS-SCANNER-09-205 | Notify smoke job validates Redis stream + Notify deliveries after staging deploys. |
|
||||
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE (2025-10-19) | Policy Guild | POLICY-CORE-09-004 | Versioned scoring config with schema validation, trust table, and golden fixtures. |
|
||||
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE (2025-10-19) | Policy Guild | POLICY-CORE-09-005 | Scoring/quiet engine – compute score, enforce VEX-only quiet rules, emit inputs and provenance. |
|
||||
| Sprint 9 | Policy Foundations | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE (2025-10-19) | Policy Guild | POLICY-CORE-09-006 | Unknown state & confidence decay – deterministic bands surfaced in policy outputs. |
|
||||
| Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-21) | Platform Events Guild | PLATFORM-EVENTS-09-401 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. |
|
||||
| Sprint 10 | Benchmarks | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | BENCH-SCANNER-10-002 | Wire real language analyzers into bench harness & refresh baselines post-implementation. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-302 | Node analyzer handling workspaces/symlinks emitting `pkg:npm`. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-303 | Python analyzer reading `*.dist-info`, RECORD hashes, entry points. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-304 | Go analyzer leveraging buildinfo for `pkg:golang` components. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-304E | Plumb Go heuristic counter into Scanner metrics pipeline and alerting. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-305 | .NET analyzer parsing `*.deps.json`, assembly metadata, RID variants. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-306 | Rust analyzer detecting crates or falling back to `bin:{sha256}`. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Shared language evidence helpers + usage flag propagation. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-308 | Determinism + fixture harness for language analyzers. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-309 | Package language analyzers as restart-time plug-ins (manifest + host registration). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-601 | Compose inventory SBOM (CycloneDX JSON/Protobuf) from layer fragments. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-602 | Compose usage SBOM leveraging EntryTrace to flag actual usage. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-603 | Generate BOM index sidecar (purl table + roaring bitmap + usage flag). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-604 | Package artifacts for export + attestation with deterministic manifests. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-605 | Emit BOM-Index sidecar schema/fixtures (CRITICAL PATH for SP16). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-606 | Usage view bit flags integrated with EntryTrace. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-607 | Embed scoring inputs, confidence band, and quiet provenance in CycloneDX/DSSE artifacts. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Implement layer cache store keyed by layer digest with metadata retention per architecture §3.3. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-102 | Build file CAS with dedupe, TTL enforcement, and offline import/export hooks. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-103 | Expose cache metrics/logging and configuration toggles for warm/cold thresholds. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-104 | Implement cache invalidation workflows (layer delete, TTL expiry, diff invalidation). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-201 | Alpine/apk analyzer emitting deterministic components with provenance. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-202 | Debian/dpkg analyzer mapping packages to purl identity with evidence. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-203 | RPM analyzer capturing EVR, file listings, provenance. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-204 | Shared OS evidence helpers for package identity + provenance. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-205 | Vendor metadata enrichment (source packages, license, CVE hints). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-206 | Determinism harness + fixtures for OS analyzers. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md | DONE (2025-10-19) | OS Analyzer Guild | SCANNER-ANALYZERS-OS-10-207 | Package OS analyzers as restart-time plug-ins (manifest + host registration). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301 | Java analyzer emitting `pkg:maven` with provenance. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-401 | POSIX shell AST parser with deterministic output. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-402 | Command resolution across layered rootfs with evidence attribution. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-403 | Interpreter tracing for shell wrappers to Python/Node/Java launchers. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-404 | Python entry analyzer (venv shebang, module invocation, usage flag). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-405 | Node/Java launcher analyzer capturing script/jar targets. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-406 | Explainability + diagnostics for unresolved constructs with metrics. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-407 | Package EntryTrace analyzers as restart-time plug-ins (manifest + host registration). |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-501 | Build component differ tracking add/remove/version changes with deterministic ordering. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-502 | Attribute diffs to introducing/removing layers including provenance evidence. |
|
||||
| Sprint 10 | Scanner Analyzers & SBOM | src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-503 | Produce JSON diff output for inventory vs usage views aligned with API contract. |
|
||||
| Sprint 10 | Samples | samples/TASKS.md | DONE (2025-10-20) | Samples Guild, Scanner Team | SAMPLES-10-001 | Sample images with SBOM/BOM-Index sidecars. |
|
||||
| Sprint 10 | DevOps Perf | ops/devops/TASKS.md | DONE (2025-10-22) | DevOps Guild | DEVOPS-PERF-10-001 | Perf smoke job ensuring <5 s SBOM compose. |
|
||||
| Sprint 10 | DevOps Perf | ops/devops/TASKS.md | DONE (2025-10-23) | DevOps Guild | DEVOPS-PERF-10-002 | Publish analyzer bench metrics to Grafana/perf workbook and alarm on ≥20 % regressions. |
|
||||
| Sprint 10 | Policy Samples | samples/TASKS.md | DONE (2025-10-23) | Samples Guild, Policy Guild | SAMPLES-13-004 | Add policy preview/report fixtures showing confidence bands and unknown-age tags. |
|
||||
| Sprint 10 | Policy Samples | src/Web/StellaOps.Web/TASKS.md | DONE (2025-10-23) | UI Guild | WEB-POLICY-FIXTURES-10-001 | Wire policy preview/report doc fixtures into UI harness (test utility or Storybook substitute) with type bindings and validation guard so UI stays aligned with documented payloads. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Signer/StellaOps.Signer/TASKS.md | DONE (2025-10-21) | Signer Guild | SIGNER-API-11-101 | `/sign/dsse` pipeline with Authority auth, PoE introspection, release verification, DSSE signing. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Signer/StellaOps.Signer/TASKS.md | DONE (2025-10-21) | Signer Guild | SIGNER-REF-11-102 | `/verify/referrers` endpoint with OCI lookup, caching, and policy enforcement. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Signer/StellaOps.Signer/TASKS.md | DONE (2025-10-21) | Signer Guild | SIGNER-QUOTA-11-103 | Enforce plan quotas, concurrency/QPS limits, artifact size caps with metrics/audit logs. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-23) | Authority Core & Security Guild | AUTH-MTLS-11-002 | Add OAuth mTLS client credential support with certificate-bound tokens and introspection updates. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-20) | Scanner WebService Guild | SCANNER-RUNTIME-12-301 | `/runtime/events` ingestion endpoint with validation, batching, storage hooks. |
|
||||
| Sprint 13 | UX & CLI Experience | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-21) | DevEx/CLI | CLI-OFFLINE-13-006 | Implement offline kit pull/import/status commands with integrity checks. |
|
||||
| Sprint 13 | UX & CLI Experience | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-22) | DevEx/CLI | CLI-PLUGIN-13-007 | Package non-core CLI verbs as restart-time plug-ins (manifest + loader tests). |
|
||||
| Sprint 13 | UX & CLI Experience | src/Web/StellaOps.Web/TASKS.md | DONE (2025-10-21) | UX Specialist, Angular Eng, DevEx | WEB1.DEPS-13-001 | Stabilise Angular workspace dependencies for headless CI installs (`npm install`, Chromium handling, docs). |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-403 | Dead-letter handling + metrics. |
|
||||
| Sprint 18 | Launch Readiness | ops/offline-kit/TASKS.md | DONE (2025-10-22) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-004 | Rebuild Offline Kit bundle with Go analyzer plug-in and refreshed manifest/signature set. |
|
||||
34
docs/implplan/SPRINTS_PRIOR_20251025.md
Normal file
34
docs/implplan/SPRINTS_PRIOR_20251025.md
Normal file
@@ -0,0 +1,34 @@
|
||||
This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not).
|
||||
|
||||
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Attestor/StellaOps.Attestor/TASKS.md | DONE (2025-10-19) | Attestor Guild | ATTESTOR-API-11-201 | `/rekor/entries` submission pipeline with dedupe, proof acquisition, and persistence. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Attestor/StellaOps.Attestor/TASKS.md | DONE (2025-10-19) | Attestor Guild | ATTESTOR-VERIFY-11-202 | `/rekor/verify` + retrieval endpoints validating signatures and Merkle proofs. |
|
||||
| Sprint 11 | Signing Chain Bring-up | src/Attestor/StellaOps.Attestor/TASKS.md | DONE (2025-10-19) | Attestor Guild | ATTESTOR-OBS-11-203 | Telemetry, alerting, mTLS hardening, and archive workflow for Attestor. |
|
||||
| Sprint 11 | Storage Platform Hardening | src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-23) | Scanner Storage Guild | SCANNER-STORAGE-11-401 | Migrate scanner object storage integration from MinIO to RustFS with data migration plan. |
|
||||
| Sprint 11 | UI Integration | src/UI/StellaOps.UI/TASKS.md | DONE (2025-10-23) | UI Guild | UI-ATTEST-11-005 | Attestation visibility (Rekor id, status) on Scan Detail. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-CORE-12-201 | Define runtime event/admission DTOs, hashing helpers, and versioning strategy. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-CORE-12-202 | Provide configuration/logging/metrics utilities shared by Observer/Webhook. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-CORE-12-203 | Authority client helpers, OpTok caching, and security guardrails for runtime services. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-OPS-12-204 | Operational runbooks, alert rules, and dashboard exports for runtime plane. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-001 | Container lifecycle watcher emitting deterministic runtime events with buffering. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Capture entrypoint traces + loaded libraries, hashing binaries and linking to baseline SBOM. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-003 | Posture checks for signatures/SBOM/attestation with offline caching. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-004 | Batch `/runtime/events` submissions with disk-backed buffer and rate limits. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-101 | Admission controller host with TLS bootstrap and Authority auth. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-102 | Query Scanner `/policy/runtime`, resolve digests, enforce verdicts. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-103 | Caching, fail-open/closed toggles, metrics/logging for admission decisions. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Zastava/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-104 | Wire `/admission` endpoint to runtime policy client and emit allow/deny envelopes. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-302 | `/policy/runtime` endpoint joining SBOM baseline + policy verdict, returning admission guidance. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-303 | Align `/policy/runtime` verdicts with canonical policy evaluation (Feedser/Vexer). |
|
||||
| Sprint 12 | Runtime Guardrails | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-304 | Integrate attestation verification into runtime policy metadata. |
|
||||
| Sprint 12 | Runtime Guardrails | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-305 | Deliver shared fixtures + e2e validation with Zastava/CLI teams. |
|
||||
| Sprint 13 | UX & CLI Experience | src/UI/StellaOps.UI/TASKS.md | DONE (2025-10-23) | UI Guild | UI-AUTH-13-001 | Integrate Authority OIDC + DPoP flows with session management. |
|
||||
| Sprint 13 | UX & CLI Experience | src/UI/StellaOps.UI/TASKS.md | DONE (2025-10-25) | UI Guild | UI-NOTIFY-13-006 | Notify panel: channels/rules CRUD, deliveries view, test send. |
|
||||
| Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-25) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-001 | Wire up .NET 10 preview feeds/local mirrors so `dotnet restore` succeeds offline; document updated NuGet bootstrap. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Queue/TASKS.md | DONE (2025-10-23) | Notify Queue Guild | NOTIFY-QUEUE-15-401 | Bus abstraction + Redis Streams adapter with ordering/idempotency. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Queue/TASKS.md | DONE (2025-10-23) | Notify Queue Guild | NOTIFY-QUEUE-15-402 | NATS JetStream adapter with health probes and failover. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/__Libraries/StellaOps.Notify.Queue/TASKS.md | DONE (2025-10-23) | Notify Queue Guild | NOTIFY-QUEUE-15-403 | Delivery queue with retry/dead-letter + metrics. |
|
||||
| Sprint 15 | Notify Foundations | src/Notify/StellaOps.Notify.Worker/TASKS.md | DONE (2025-10-23) | Notify Worker Guild | NOTIFY-WORKER-15-201 | Bus subscription + leasing loop with backoff. |
|
||||
| Sprint 17 | Symbol Intelligence & Forensics | src/Zastava/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-25) | Zastava Observer Guild | ZASTAVA-OBS-17-005 | Collect GNU build-id during runtime observation and attach it to emitted events. |
|
||||
| Sprint 17 | Symbol Intelligence & Forensics | src/Scanner/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-25) | Scanner WebService Guild | SCANNER-RUNTIME-17-401 | Persist runtime build-id observations and expose them for debug-symbol correlation. |
|
||||
84
docs/implplan/SPRINTS_PRIOR_20251027.md
Normal file
84
docs/implplan/SPRINTS_PRIOR_20251027.md
Normal file
@@ -0,0 +1,84 @@
|
||||
This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not).
|
||||
|
||||
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-NUGET-13-002 | Ensure all solutions/projects prioritize `local-nuget` before public feeds and add restore-order validation. |
|
||||
| Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-003 | Upgrade `Microsoft.*` dependencies pinned to 8.* to their latest .NET 10 (or 9.x) releases and refresh guidance. |
|
||||
| Sprint 14 | Release & Offline Ops | ops/deployment/TASKS.md | DONE (2025-10-26) | Deployment Guild | DEVOPS-OPS-14-003 | Deployment/update/rollback automation and channel management documentation. |
|
||||
| Sprint 14 | Release & Offline Ops | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-14-001 | Deterministic build/release pipeline with SBOM/provenance, signing, and manifest generation. |
|
||||
| Sprint 14 | Release & Offline Ops | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, Scanner Guild | DEVOPS-REL-14-004 | Extend release/offline smoke jobs to cover Python analyzer plug-ins (warm/cold, determinism, signing). |
|
||||
| Sprint 14 | Release & Offline Ops | ops/licensing/TASKS.md | DONE (2025-10-26) | Licensing Guild | DEVOPS-LIC-14-004 | Registry token service tied to Authority, plan gating, revocation handling, monitoring. |
|
||||
| Sprint 14 | Release & Offline Ops | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild | DEVOPS-OFFLINE-14-002 | Offline kit packaging workflow with integrity verification and documentation. |
|
||||
| Sprint 15 | Benchmarks | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild, Notify Team | BENCH-NOTIFY-15-001 | Notify dispatch throughput bench with results CSV. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Define Scheduler DTOs & validation. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-102 | Publish schema docs/sample payloads. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Mongo schemas/indexes for Scheduler state. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-202 | Repositories with tenant scoping, TTL, causal consistency. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-203 | Audit/run stats materialization for UI. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-302 | Query APIs for ResolveByPurls/ResolveByVulns/ResolveAll. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Ingest BOM-Index into roaring bitmap store. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-102 | Schedules CRUD (cron validation, pause/resume, audit). |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-103 | Runs API (list/detail/cancel) + impact previews. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-WEB-16-104 | Feedser/Vexer webhook handlers with security enforcement. |
|
||||
| Sprint 17 | Symbol Intelligence & Forensics | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-RUNTIME-17-004 | Document build-id workflows for SBOMs, runtime events, and debug-store usage. |
|
||||
| Sprint 17 | Symbol Intelligence & Forensics | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-17-002 | Ship stripped debug artifacts organised by build-id within release/offline kits. |
|
||||
| Sprint 17 | Symbol Intelligence & Forensics | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild, DevOps Guild | DEVOPS-OFFLINE-17-003 | Mirror release debug-store artefacts into Offline Kit packaging and document validation. |
|
||||
| Sprint 17 | Symbol Intelligence & Forensics | src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md | DONE (2025-10-26) | Emit Guild | SCANNER-EMIT-17-701 | Record GNU build-id for ELF components and surface it in SBOM/diff outputs. |
|
||||
| Sprint 18 | Launch Readiness | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-LAUNCH-18-001 | Production launch cutover rehearsal and runbook publication. |
|
||||
| Sprint 18 | Launch Readiness | ops/offline-kit/TASKS.md | DONE (2025-10-26) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-005 | Rebuild Offline Kit with Python analyzer artefacts and refreshed manifest/signature pair. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-AOC-19-001 | Publish aggregation-only contract reference documentation. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-002 | Update architecture overview with AOC boundary diagrams. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-AOC-19-003 | Refresh policy engine doc with raw ingestion constraints. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-AOC-19-004 | Document console AOC dashboard and drill-down flow. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-AOC-19-005 | Document CLI AOC commands and exit codes. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-AOC-19-006 | Document new AOC metrics, traces, and logs. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-AOC-19-007 | Document new Authority scopes and tenancy enforcement. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, DevOps Guild | DOCS-AOC-19-008 | Update deployment guide with validator enablement and verify user guidance. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce new ingestion/auth scopes across Authority. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-POLICY-20-001 | Publish `/docs/policy/overview.md` with compliance checklist. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-POLICY-20-002 | Document DSL grammar + examples in `/docs/policy/dsl.md`. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Authority Core | DOCS-POLICY-20-003 | Write `/docs/policy/lifecycle.md` covering workflow + roles. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Scheduler Guild | DOCS-POLICY-20-004 | Document policy run modes + cursors in `/docs/policy/runs.md`. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Platform Guild | DOCS-POLICY-20-005 | Produce `/docs/api/policy.md` with endpoint schemas + errors. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, CLI Guild | DOCS-POLICY-20-006 | Author `/docs/cli/policy.md` with commands, exit codes, JSON output. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, UI Guild | DOCS-POLICY-20-007 | Create `/docs/ui/policy-editor.md` covering editor, simulation, approvals. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-POLICY-20-008 | Publish `/docs/architecture/policy-engine.md` with sequence diagrams. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Observability Guild | DOCS-POLICY-20-009 | Document metrics/traces/logs in `/docs/observability/policy.md`. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Security Guild | DOCS-POLICY-20-010 | Publish `/docs/security/policy-governance.md` for scopes + approvals. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Policy Guild | DOCS-POLICY-20-011 | Add example policies under `/docs/examples/policies/` with commentary. |
|
||||
| Sprint 20 | Policy Engine v2 | docs/TASKS.md | DONE (2025-10-26) | Docs Guild, Support Guild | DOCS-POLICY-20-012 | Draft `/docs/faq/policy-faq.md` covering conflicts, determinism, pitfalls. |
|
||||
| Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-POLICY-20-001 | Add DSL lint + compile checks to CI pipelines. |
|
||||
| Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild, QA Guild | DEVOPS-POLICY-20-003 | Add determinism CI job diffing repeated policy runs. |
|
||||
| Sprint 20 | Policy Engine v2 | samples/TASKS.md | DONE (2025-10-26) | Samples Guild, Policy Guild | SAMPLES-POLICY-20-001 | Commit baseline/serverless/internal-only policy samples + fixtures. |
|
||||
| Sprint 20 | Policy Engine v2 | samples/TASKS.md | DONE (2025-10-26) | Samples Guild, UI Guild | SAMPLES-POLICY-20-002 | Produce simulation diff fixtures for UI/CLI tests. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Add new policy scopes (`policy:*`, `findings:read`, `effective:write`). |
|
||||
| Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-002 | Enforce Policy Engine service identity and scope checks at gateway. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-POLICY-20-003 | Update Authority docs/config samples for policy scopes + workflows. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild, Policy Guild | BENCH-POLICY-20-001 | Create policy evaluation benchmark suite + baseline metrics. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-26) | Policy Guild, Platform Guild | POLICY-ENGINE-20-000 | Spin up new Policy Engine service host with DI bootstrap and Authority wiring. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-26) | Policy Guild | POLICY-ENGINE-20-001 | Deliver `stella-dsl@1` parser + IR compiler with diagnostics and checksums. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-20-001 | Define policy run/diff DTOs + validation helpers. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core Guild | AUTH-GRAPH-21-001 | Introduce graph scopes (`graph:*`) with configuration binding and defaults. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core Guild | AUTH-GRAPH-21-002 | Enforce graph scopes/identities at gateway with tenant propagation. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-GRAPH-21-003 | Update security docs/config samples for graph access and least privilege. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums; document in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-002 | Publish schema docs/sample payloads for graph job lifecycle. |
|
||||
| Sprint 22 | Link-Not-Merge v1 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild | BENCH-LNM-22-001 | Benchmark advisory observation ingest/correlation throughput. |
|
||||
| Sprint 22 | Link-Not-Merge v1 | src/Bench/StellaOps.Bench/TASKS.md | DONE (2025-10-26) | Bench Guild | BENCH-LNM-22-002 | Benchmark VEX ingest/correlation latency and event emission. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-001 | Publish `/docs/ui/console-overview.md` (IA, tenant model, filters, AOC alignment). |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-002 | Author `/docs/ui/navigation.md` with route map, filters, keyboard shortcuts, deep links. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-003 | Document `/docs/ui/sbom-explorer.md` covering catalog, graph, overlays, exports. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-004 | Produce `/docs/ui/advisories-and-vex.md` detailing aggregation-not-merge UX. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-005 | Write `/docs/ui/findings.md` with filters, explain, exports, CLI parity notes. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-006 | Publish `/docs/ui/policies.md` (editor, simulation, approvals, RBAC). |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-007 | Document `/docs/ui/runs.md` with SSE monitoring, diff, retries, evidence downloads. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-26) | Docs Guild | DOCS-CONSOLE-23-008 | Draft `/docs/ui/admin.md` covering tenants, roles, tokens, integrations, fresh-auth. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-27) | Docs Guild | DOCS-CONSOLE-23-009 | Publish `/docs/ui/downloads.md` aligning manifest with commands and offline flow. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DOCS-CONSOLE-23-010 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, env vars, health checks). |
|
||||
| Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-001 | Provide graph build/overlay job APIs; see `docs/SCHED-WEB-21-001-GRAPH-APIS.md`. |
|
||||
| Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-002 | Provide overlay lag metrics endpoint/webhook; see `docs/SCHED-WEB-21-001-GRAPH-APIS.md`. |
|
||||
| Sprint 28 | Graph Explorer | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild, Authority Core Guild | SCHED-WEB-21-003 | Replace header auth with Authority scopes using `StellaOpsScopes`; dev fallback only when `Scheduler:Authority:Enabled=false`. |
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-001 | Deploy default OpenTelemetry collector manifests with secure OTLP pipeline. |
|
||||
| Sprint 50 | Observability & Forensics Phase 1 – Baseline Telemetry | ops/devops/TASKS.md | DONE (2025-10-26) | DevOps Guild | DEVOPS-OBS-50-003 | Package telemetry stack configs for offline/air-gapped installs with signatures. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-WEB-16-101 | Minimal API host with Authority enforcement. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | ImpactIndex targeting and shard planning. |
|
||||
26
docs/implplan/SPRINTS_PRIOR_20251028.md
Normal file
26
docs/implplan/SPRINTS_PRIOR_20251028.md
Normal file
@@ -0,0 +1,26 @@
|
||||
This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not).
|
||||
|
||||
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-203 | Runner execution invoking Scanner analysis/content refresh. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-204 | Emit rescan/report events for Notify/UI. |
|
||||
| Sprint 16 | Scheduler Intelligence | src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-205 | Metrics/telemetry for Scheduler planners/runners. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-27) | Authority Core & Security Guild | AUTH-AOC-19-002 | Enforce tenant claim propagation and cross-tenant guardrails. |
|
||||
> AUTH-AOC-19-002: Tenant metadata now flows through rate limiter/audit/token persistence; password grant scope/tenant enforcement landed. Docs/stakeholder walkthrough pending.
|
||||
> 2025-10-27 Update: Ingestion scopes require tenant assignment; access tokens propagate tenant claims and reject cross-tenant mismatches with coverage.
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | src/Authority/StellaOps.Authority/TASKS.md | DONE (2025-10-27) | Authority Core & Docs Guild | AUTH-AOC-19-003 | Update Authority docs/config samples for new scopes. |
|
||||
> AUTH-AOC-19-003: Scope catalogue, console/CLI docs, and sample config updated to require `aoc:verify` plus read scopes; verification clients now explicitly include tenant hints. Authority test run remains blocked on Concelier build failure (`ImmutableHashSet<string?>`), previously noted under AUTH-AOC-19-002.
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | src/Concelier/StellaOps.Concelier.WebService/TASKS.md | DONE (2025-10-28) | Concelier WebService Guild | CONCELIER-WEB-AOC-19-001 | Implement raw advisory ingestion endpoints with AOC guard and verifier. |
|
||||
| Sprint 19 | Aggregation-Only Contract Enforcement | src/Excititor/StellaOps.Excititor.Worker/TASKS.md | DONE (2025-10-28) | QA Guild | EXCITITOR-WORKER-AOC-19-003 | Expand worker tests for deterministic batching and restart safety. |
|
||||
| Sprint 20 | Policy Engine v2 | ops/devops/TASKS.md | DONE (2025-10-27) | DevOps Guild, Scheduler Guild, CLI Guild | DEVOPS-POLICY-20-004 | Automate policy schema exports and change notifications for CLI consumers. |
|
||||
| Sprint 20 | Policy Engine v2 | src/Cli/StellaOps.Cli/TASKS.md | DONE (2025-10-27) | DevEx/CLI Guild | CLI-POLICY-20-002 | Implement `stella policy simulate` with diff outputs + exit codes. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Cartographer/StellaOps.Cartographer/TASKS.md | DONE (2025-10-27) | Cartographer Guild | CARTO-GRAPH-21-010 | Replace hard-coded `graph:*` scope strings with shared constants once graph services integrate. |
|
||||
| Sprint 21 | Graph Explorer v1 | src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-002 | Expose overlay lag metrics and job completion hooks for Cartographer. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-28) | Docs Guild | DOCS-CONSOLE-23-011 | Update `/docs/install/docker.md` to include console image, compose/Helm/offline examples. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-28) | Docs Guild | DOCS-CONSOLE-23-012 | Publish `/docs/security/console-security.md` covering OIDC, scopes, CSP, evidence handling. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-28) | Docs Guild | DOCS-CONSOLE-23-013 | Write `/docs/observability/ui-telemetry.md` cataloguing metrics/logs/dashboards/alerts. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-28) | Docs Guild | DOCS-CONSOLE-23-014 | Maintain `/docs/cli-vs-ui-parity.md` matrix with CI drift detection guidance. |
|
||||
| Sprint 23 | StellaOps Console | docs/TASKS.md | DONE (2025-10-28) | Docs Guild | DOCS-CONSOLE-23-016 | Refresh `/docs/accessibility.md` with console keyboard flows, tokens, testing tools. <br>2025-10-28: Published guide covering keyboard matrix, screen-reader behaviour, colour tokens, testing workflow, offline guidance, and compliance checklist. |
|
||||
| Sprint 25 | Exceptions v1 | docs/TASKS.md | DONE (2025-10-27) | Docs Guild | DOCS-EXC-25-004 | Document policy exception effects + simulation. |
|
||||
| Sprint 25 | Exceptions v1 | src/Policy/StellaOps.Policy.Engine/TASKS.md | DONE (2025-10-27) | Policy Guild | POLICY-ENGINE-70-001 | Add exception evaluation layer with specificity + effects. |
|
||||
| Sprint 25 | Exceptions v1 | src/Policy/__Libraries/StellaOps.Policy/TASKS.md | DONE (2025-10-27) | Policy Guild | POLICY-EXC-25-001 | Extend SPL schema to reference exception effects and routing. |
|
||||
Reference in New Issue
Block a user