stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

docs consolidation

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-24 12:38:14 +02:00
parent 7503c19b8f
commit 9a08d10b89
215 changed files with 2188 additions and 9623 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/05_ROADMAP.md
View File

@@ -4,14 +4,14 @@ This repository is the source of truth for StellaOps direction. The roadmap is e
## How to read this
- **Now / Next / Later** are priority bands, not dates.
- A capability is “done” when the required evidence exists and is reproducible (see `docs/roadmap/maturity-model.md`).
- A capability is "done" when the required evidence exists and is reproducible (see `docs/roadmap/maturity-model.md`).
## Now (Foundation)
- Deterministic scan pipeline: image → SBOMs (SPDX 3.0.1 + CycloneDX 1.6) with stable identifiers and replayable outputs.
- Deterministic scan pipeline: image -> SBOMs (SPDX 3.0.1 + CycloneDX 1.6) with stable identifiers and replayable outputs.
- Advisory ingestion with offline-friendly mirrors, normalization, and deterministic merges.
- VEX-first triage: OpenVEX ingestion/consensus with explainable, stable verdicts.
- Policy gates: deterministic policy evaluation (OPA/Rego where applicable) with audit-friendly decision traces.
- Offline Kit workflows (bundle → import → verify) with signed artifacts and deterministic indexes.
- Offline Kit workflows (bundle -> import -> verify) with signed artifacts and deterministic indexes.
## Next (Hardening)
- Multi-tenant isolation (tenancy boundaries + RLS where applicable) and an audit trail built for replay.

875
docs/07_HIGH_LEVEL_ARCHITECTURE.md
View File

@@ -1,793 +1,82 @@
# High‑Level Architecture — **Stella Ops** (Consolidated • 2025Q4)
> **Want the 10-minute tour?** See [`40_ARCHITECTURE_OVERVIEW.md`](40_ARCHITECTURE_OVERVIEW.md); this file retains the exhaustive reference.
> **Purpose.** A complete, implementation‑ready map of Stella Ops: product vision, all runtime components, trust boundaries, tokens/licensing, control/data flows, storage, APIs, security, scale, DevOps, and verification logic.
> **Scope.** This file **replaces** the separate `components.md`; all component details now live here.
---
## 0) Product vision & principles
**Vision.** Stella Ops is a **deterministic SBOM + VEX platform** for CI/CD and runtime, tuned for **speed** (per‑layer deltas), **quiet output** (usage‑scoped views), and **verifiability** (DSSE + Rekor v2). It is **self‑hostable**, **air‑gap capable**, and **commercially enforceable**: only licensed installations can produce **Stella Ops‑verified** attestations.
**Operating principles.**
* **Scanner-owned SBOMs.** We generate our own BOMs; we do not warehouse third-party SBOM content (we can **link** to attested SBOMs).
* **Deterministic evidence.** Facts come from package DBs, installed metadata, linkers, and verified attestations; no fuzzy guessing in the core.
* **Evidence-grade testing.** Test models and CI lanes enforce deterministic, offline-first coverage; see `docs/testing/testing-strategy-models.md` and `docs/testing/TEST_CATALOG.yml`.
* **Per-layer caching.** Cache fragments by **layer digest** and compose image SBOMs via **CycloneDX BOM-Link** / **SPDX ExternalRef**.
* **Inventory vs Usage.** Always record the full **inventory** of what exists; separately present **usage** (entrypoint closure + loaded libs).
* **Backend decides.** PASS/FAIL is produced by **Policy** + **VEX** + **Advisories**. The scanner reports facts.
* **VEX-first triage UX.** Operators triage by artifact with evidence-first cards, VEX decisioning, and immutable audit bundles; see `docs/ux/TRIAGE_UX_GUIDE.md`.
* **Attest or it didn't happen.** Every export is signed as **in-toto/DSSE** and logged in **Rekor v2**.
* **Hybrid reachability attestations.** Every reachability graph ships with a graph-level DSSE (mandatory) plus optional edge-bundle DSSEs for runtime/init/contested edges; Policy/Signals consume graph DSSE as baseline and edge bundles for quarantine/disputes. See `docs/reachability/hybrid-attestation.md` for verification runbooks, Rekor guidance, and offline replay steps.
* **Sovereign-ready.** Cloud is used only for licensing and optional endorsement; everything else is first-party and self-hostable.
* **Competitive clarity.** Moats: deterministic replay, hybrid reachability proofs, lattice VEX, sovereign crypto, proof graph; see `docs/market/competitive-landscape.md`.
---
## 1) Service topology & trust boundaries
### 1.1 Runtime inventory (first‑party)
| Service / Tool | Container image | Core role | Scale pattern |
| ------------------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| **Scanner.WebService** | `stellaops/scanner-web` | Control plane for scans; catalog; SBOM composition (inventory & usage); diff; exports; **analysis‑only report runs** for Scheduler. | Stateless; N replicas behind LB. |
| **Scanner.Worker** | `stellaops/scanner-worker` | Runs analyzers (OS, Lang: Java/Node/Python/Go/.NET/Rust, Native ELF/PE/Mach‑O, EntryTrace); emits per‑layer SBOMs and composes image SBOMs. | Horizontal; queue‑driven; sharded by layer digest. |
| **Scanner.Sbomer.BuildXPlugin** | `stellaops/sbom-indexer` | BuildKit **generator** for build‑time SBOMs as OCI **referrers**. | CI‑side; ephemeral. |
| **Scanner.Sbomer.DockerImage** | `stellaops/scanner-cli` | CLI‑orchestrated scanner container for post‑build scans. | Local/CI; ephemeral. |
| **Concelier.WebService** | `stellaops/concelier-web` | Vulnerability ingest/normalize/merge/export (JSON + Trivy DB). | HA via PostgreSQL locks. |
| **Excititor.WebService** | `stellaops/excititor-web` | VEX ingest/normalize/consensus; conflict retention; exports. | HA via PostgreSQL locks. |
| **Policy Engine** | (in `scanner-web`) | YAML DSL evaluator (waivers, vendor preferences, KEV/EPSS, license, usage‑gating); produces **policy digest**. | In‑process; cache per digest. |
| **Scheduler.WebService** | `stellaops/scheduler-web` | Schedules **re‑evaluation** runs; consumes Concelier/Excititor deltas; selects **impacted images** via BOM‑Index; orchestrates analysis‑only reports. | Stateless API. |
| **Scheduler.Worker** | `stellaops/scheduler-worker` | Executes selection and enqueues batches toward Scanner; enforces rate/limits and windows; maintains impact cursors. | Horizontal; queue‑driven. |
| **Notify.WebService** | `stellaops/notify-web` | Rules engine for outbound notifications; manages channels, templates, throttle/digest logic. | Stateless API. |
| **Notify.Worker** | `stellaops/notify-worker` | Delivers to Slack/Teams/Email/Webhooks; idempotent retries; digests. | Horizontal; per‑channel rate limits. |
| **Signer** | `stellaops/signer` | **Hard gate:** validates entitlement + release integrity; mints signing cert (Fulcio keyless) or uses KMS; signs DSSE. | Stateless; HPA by QPS. |
| **Attestor** | `stellaops/attestor` | Posts DSSE bundles to **Rekor v2**; verification endpoints. | Stateless; HPA by QPS. |
| **Authority** | `stellaops/authority` | On‑prem OIDC issuing **short‑lived OpToks** with DPoP/mTLS sender constraint. | HA behind LB. |
| **Zastava** (Runtime) | `stellaops/zastava` | Runtime inspector/enforcer (observer + optional Admission Webhook). | DaemonSet + Webhook. |
| **Web UI** | `stellaops/ui` | Angular app for scans, diffs, policy, VEX, vulnerability triage (artifact-first), audit bundles, **Scheduler**, **Notify**, runtime, reports. | Stateless. |
| **StellaOps.Cli** | `stellaops/cli` | CLI for init/scan/export/diff/policy/report/verify; Buildx helper; **schedule** and **notify** verbs. | Local/CI. |
### 1.2 Infrastructure Requirements
**REQUIRED Infrastructure:**
* **PostgreSQL** (≥16) — **ONLY** database for all persistent data. Per-module schema isolation (authority, vuln, vex, scanner, scheduler, notify, policy, orchestrator). See [Database Architecture](#database-architecture-postgresql).
* **Valkey** (≥8.0) — Redis-compatible cache, DPoP nonces (RFC 9449), event streams, job queues, rate limiting. **REQUIRED** for platform operation.
* **RustFS** — S3-compatible object storage for SBOM artifacts, proof bundles, and scan evidence. HTTP API with deterministic responses.
**OPTIONAL Infrastructure:**
* **NATS JetStream** — Alternative messaging transport (Valkey Streams is default). Opt-in only via configuration.
**External Dependencies:**
* **OCI Registry** — Must support **Referrers API** (discover SBOMs/signatures).
* **Fulcio** (Sigstore CA) — Issues short-lived signing certs (keyless signing). Optional if using KMS keys.
* **Rekor v2** — Tile-backed transparency log. Optional if `OFFLINEKIT_ENABLED=true` (airgap mode).
**Architecture Note:**
- PostgreSQL is the ONLY database (MongoDB fully removed as of 2025-12-23)
- Valkey replaces Redis (drop-in compatible, but required)
- RustFS is primary object storage (MinIO removed)
- NATS is OPTIONAL, not required (Valkey Streams handle queuing)
### 1.3 Cloud licensing (Stella Ops)
* **Licensing Service** (`www.stella-ops.org`) — issues long‑lived **License Tokens (LT)**; exchanges LT → **Proof‑of‑Entitlement (PoE)** bound to an installation key; revoke/introspect PoE; optional cross‑log **endorsement**.
### 1.4 Diagram (control/data planes & trust)
```mermaid
flowchart LR
subgraph Cloud["www.stella-ops.org (Cloud)"]
LS[Licensing Service<br/>LT→PoE / revoke / introspect]
end
subgraph OnPrem["Customer Site (Self-hosted)"]
Auth[Authority (OIDC)\nOpTok (DPoP/mTLS)]
SW[Scanner.WebService]
WK[Scanner.Worker xN]
CONC[Concelier]
EXC[Excititor]
SCHW[Scheduler.Web]
SCH[Scheduler.Worker xN]
NOTW[Notify.Web]
NOT[Notify.Worker xN]
POL[Policy Engine (in Scanner.Web)]
SGN[Signer\n(entitlement + signing)]
ATT[Attestor\n(Rekor v2 submit/verify)]
UI[Web UI (Angular)]
Z[Zastava\n(Runtime Inspector/Enforcer)]
RFS[(RustFS)]
PG[(PostgreSQL)]
VK[(Valkey)]
end
CLI[StellaOps.Cli / Buildx Plugin]
REG[(OCI Registry with Referrers)]
FUL[ Fulcio ]
REK[ Rekor v2 (tiles) ]
CLI -->|scan/build| SW
SW -->|jobs| VK
VK --> WK
WK --> RFS
SW --> PG
CONC --> PG
EXC --> PG
UI --> SW
Z --> SW
%% New event-driven loop
CONC -- export.delta --> SCHW
EXC -- export.delta --> SCHW
SCHW --> SCH
SCH --> SW
SW -- report.ready --> NOTW
Z -- admission/observe --> NOTW
SGN <--> Auth
SGN --> FUL
SGN -->|mTLS| ATT
ATT --> REK
SGN <-->|verify referrers| REG
```
**Trust boundaries.** Only **Signer** can sign; only **Attestor** can write to **Rekor v2**. Scanner/UI/Scheduler/Notify never sign.
---
## 2) Licensing & tokens (installation‑ready, theft‑resistant)
**Two‑token model.**
* **License Token (LT)** — long‑lived JWT from **Licensing Service**; used **once** to enroll the installation; never used in hot path.
* **Proof‑of‑Entitlement (PoE)** — bound to the installation key (mTLS client cert **or** DPoP‑bound JWT with `cnf`); medium‑lived; renewable; revocable.
* **Operational token (OpTok)** — 2–5 min OIDC token from **Authority**, **sender‑constrained** (DPoP or mTLS). Used to authenticate to **Signer**/**Scanner.WebService**/**Scheduler.Web**/**Notify.Web**.
**Signer enforces both:** PoE proves entitlement; OpTok proves “who is calling now”. It also **independently verifies** the **scanner image digest** is **Stella Ops‑signed** via **Referrers + cosign** before signing anything.
**Enrollment sequence (LT → PoE).**
```plantuml
@startuml
actor Operator
participant "Install Agent" as IA
participant "Licensing Service" as LS
Operator -> IA: Provide LT
IA -> IA: Generate K_inst
IA -> LS: /license/enroll {LT, pub(K_inst)}
LS --> IA: PoE (mTLS client cert or JWT with cnf=K_inst), CRL/OCSP/introspect
@enduml
```
---
## 3) Scanner subsystem (facts engine)
### 3.1 Analyzers (deterministic only)
* **OS packages:** apk/dpkg/rpm (Linux); Windows MSI/SxS/GAC (M2).
* **Language (installed state):**
* Java (pom.properties / MANIFEST) → `pkg:maven/...`
* Node (`node_modules/*/package.json`) → `pkg:npm/...`
* Python (`*.dist-info/METADATA`) → `pkg:pypi/...`
* Go (buildinfo) → `pkg:golang/...`
* .NET (`*.deps.json`) → `pkg:nuget/...`
* **Rust:** deterministic **language markers** (symbol mangling) and crates only when present; otherwise `bin:{sha256}`.
* **Native:** ELF/PE/Mach‑O imports, DT_NEEDED, RPATH/RUNPATH, symbol versions, PE version info.
* **EntryTrace:** parse `ENTRYPOINT`/`CMD`; shell AST; resolve launchers (Java/Node/Python) to terminal program; record file:line chain.
### 3.2 Caching & composition
* **Layer cache:** `{layerDigest → SBOM fragment + analyzer meta}`.
* **File CAS:** `{sha256(file) → parse result (ELF/JAR metadata/etc.)}`.
* **Composition:** build **image SBOMs** from fragments via **BOM‑Link/ExternalRef**; emit **two views**:
* **Inventory** (complete filesystem inventory).
* **Usage** (entrypoint closure + linked libs).
* **Transport:** JSON **and** **CycloneDX Protobuf** (compact, fast to parse).
* **Index:** BOM‑Index sidecar with purl table + roaring bitmap + `usedByEntrypoint` flag for fast joins.
### 3.3 Diff (image → layer → package)
* Added / Removed / Version‑changed changes, **attributed** to the layer that caused them.
* Raw diffs preserved; backend view applies **VEX + Policy**.
### 3.4 Build‑time SBOMs (fast CI path)
* Buildx **generator** runs analyzers during `docker buildx build --attest=type=sbom,generator=stellaops/sbom-indexer`, attaches SBOMs as **OCI referrers**.
* Scanner.WebService can trust these (policy‑configurable) and **skip** re‑scan; DSSE + Rekor v2 can be done either at build time or post‑push via Signer/Attestor.
### 3.5 Events / integrations
* **Out:** `report.ready` (summary + verdict + Rekor UUID) → internal bus for **Notify** & UI.
* **Expose:** image‑level **BOM‑Index** metadata for **Scheduler** impact selection.
---
## 4) Backend evaluation (decider)
### 4.1 Concelier (advisories)
* Ingests vendor, distro, OSS feeds; normalizes & merges; persists canonical advisories in PostgreSQL; exports **deterministic JSON** and **Trivy DB**.
* Offline kit bundles for air‑gapped sites.
### 4.2 Excititor (VEX)
* Ingests **OpenVEX / CSAF VEX / CycloneDX VEX**; normalizes claims; retains conflicts; computes **consensus** with provider trust weights and justification gates.
### 4.3 Policy Engine (YAML DSL)
* Matchers: `image/repo/env/purl/cve/vendor/source/path/layerDigest/usedByEntrypoint`
* Actions: `ignore(until, justification)`, `fail`, `warn`, `defer`, `requireVEX{vendors, justifications}`, `escalate {sev, KEV, EPSS}`, license constraints.
* Produces a **policy digest** (SHA‑256 of canonicalized policy).
### 4.4 PASS/FAIL flow
1. SBOM (Inventory / Usage) → join with **Concelier** advisories.
2. Apply **Excititor** consensus (statuses & justifications).
3. Apply **Policy**; compute PASS/FAIL with waiver TTLs.
4. Sign the **final report** (DSSE via **Signer**) and log to **Rekor v2** via **Attestor**.
---
## 4A) Score Proofs & Deterministic Replay
### 4A.1 Overview
Score Proofs provide cryptographically verifiable audit trails for every scoring decision. They enable:
* **Deterministic replay**: Same inputs → same outputs, every time
* **Audit compliance**: Full traceability from inputs to final scores
* **Offline verification**: Proof bundles verifiable without network access
* **Feed updates**: Re-score historical scans with new advisories
### 4A.2 Scan Manifest
Every scan captures its inputs deterministically:
```json
{
"scanId": "550e8400-e29b-41d4-a716-446655440000",
"createdAtUtc": "2025-12-17T12:00:00Z",
"artifactDigest": "sha256:abc123...",
"artifactPurl": "pkg:oci/myapp@sha256:abc123...",
"scannerVersion": "1.0.0",
"workerVersion": "1.0.0",
"concelierSnapshotHash": "sha256:feed123...",
"excititorSnapshotHash": "sha256:vex456...",
"latticePolicyHash": "sha256:policy789...",
"deterministic": true,
"seed": "AQIDBA==",
"knobs": {"maxDepth": "10"}
}
```
### 4A.3 Proof Ledger (DAG)
Scoring computation is recorded as a directed acyclic graph of `ProofNode`:
| Field | Description |
|-------|-------------|
| `id` | Node identifier |
| `kind` | `Input`, `Transform`, `Delta`, `Score` |
| `ruleId` | Policy rule that produced this node |
| `parentIds` | Nodes this depends on |
| `evidenceRefs` | Links to supporting evidence |
| `delta` | Score contribution at this step |
| `total` | Cumulative score |
| `nodeHash` | Content-addressed hash |
The **proof root hash** is computed by hashing all leaf nodes' hashes in deterministic order.
### 4A.4 Score Replay API
```
POST /api/v1/scanner/scans/{id}/score/replay
{ overrides?: { concelierSnapshotHash?, excititorSnapshotHash?, latticePolicyHash? } }
→ { scoreProof, rootHash, proofBundleUri }
```
Use cases:
* **Feed updates**: Re-score when Concelier publishes new advisories
* **Policy changes**: See impact of policy modifications
* **Audit**: Reproduce historical scores for compliance
### 4A.5 Proof Bundle Format
Proof bundles are self-contained ZIP archives:
```
proof-bundle.zip/
├── manifest.json # Canonical scan manifest
├── manifest.dsse.json # DSSE signature
├── score_proof.json # ProofNode[] array
├── proof_root.dsse.json # DSSE of proof root
└── meta.json # Timestamps, versions
```
**Storage**: `scanner.proof_bundle` table + RustFS for bundle files.
---
## 4B) Reachability Analysis
### 4B.1 Overview
Reachability Analysis determines whether vulnerable code is actually reachable from application entrypoints, reducing false positives by filtering unreachable vulnerabilities.
### 4B.2 Call Graph Ingestion
Language-specific workers extract call graphs:
```json
{
"schema": "stella.callgraph.v1",
"language": "dotnet",
"nodes": [...], // Function definitions
"edges": [...], // Call relationships
"entrypoints": [...] // HTTP routes, gRPC, etc.
}
```
**Supported languages**: .NET, Java, Node.js, Python, Go, Rust
### 4B.3 Reachability Statuses
| Status | Confidence | Description |
|--------|------------|-------------|
| `UNREACHABLE` | High | No path from entrypoints to vulnerable code |
| `POSSIBLY_REACHABLE` | Medium | Path exists with heuristic edges |
| `REACHABLE_STATIC` | High | Static analysis proves path exists |
| `REACHABLE_PROVEN` | Very High | Runtime evidence confirms |
| `UNKNOWN` | Low | Insufficient data |
### 4B.4 Reachability API
```
POST /api/v1/scanner/scans/{id}/callgraphs
CallGraph → { callGraphDigest, status }
POST /api/v1/scanner/scans/{id}/reachability/compute
→ { jobId, status }
GET /api/v1/scanner/scans/{id}/reachability/findings
→ { findings[], summary }
GET /api/v1/scanner/scans/{id}/reachability/explain?cve=...&purl=...
→ { status, confidence, shortestPath[], whyReachable[] }
```
### 4B.5 Integration with Score Proofs
Reachability evidence is included in proof bundles:
* Reachability status per CVE/PURL is a scoring input
* Path evidence is referenced in proof nodes
* Graph attestations (DSSE) link to score proofs
**Storage**: `scanner.cg_node`, `scanner.cg_edge`, `scanner.entrypoint` tables.
---
## 4C) Unknowns Registry
### 4C.1 Overview
The Unknowns Registry tracks items that could not be fully classified due to missing evidence, enabling prioritized triage.
### 4C.2 Unknown Reasons
| Code | Description |
|------|-------------|
| `missing_vex` | No VEX statement for vulnerability |
| `ambiguous_indirect_call` | Indirect call target unresolved |
| `incomplete_sbom` | SBOM missing component data |
| `missing_advisory` | No advisory data for CVE |
| `conflicting_evidence` | Multiple conflicting data sources |
### 4C.3 2-Factor Ranking Model
Unknowns are ranked by:
```
score = 0.60 × blast + 0.30 × scarcity + 0.30 × pressure + containment_deduction
```
| Factor | Weight | Components |
|--------|--------|------------|
| Blast Radius | 0.60 | Dependents, network exposure, privilege |
| Evidence Scarcity | 0.30 | Missing data severity |
| Exploit Pressure | 0.30 | EPSS, KEV status |
| Containment | -0.20 | Seccomp, read-only FS |
### 4C.4 Band Assignment
| Band | Score Range | SLA |
|------|-------------|-----|
| HOT | ≥ 0.70 | 24 hours |
| WARM | 0.40 - 0.69 | 7 days |
| COLD | < 0.40 | 30 days |
### 4C.5 Unknowns API
```
GET /api/v1/unknowns
?band=HOT&sort=score → { items[], pagination }
GET /api/v1/unknowns/{id}
→ { id, reasons, blastRadius, score, scoreBreakdown }
GET /api/v1/unknowns/{id}/proof
→ { nodes[], rootHash }
POST /api/v1/unknowns/{id}/escalate
→ { rescanJobId, status }
POST /api/v1/unknowns/{id}/resolve
{ resolution, justification } → { resolvedAt }
```
**Storage**: `policy.unknowns` table with ranking metadata.
---
## 5) Runtime enforcement (Zastava)
* **Observer:** inventories running containers, checks image signatures, SBOM presence (referrers), detects drift (entrypoint chain divergence), flags unapproved images.
* **Admission Webhook (optional):** blocks policy‑fail pods (dry‑run first).
* **Integration:** posts runtime events to Scanner.WebService; can request **delta scans** on changed layers.
---
## 6) Storage & catalogs (RustFS/PostgreSQL)
**RustFS layout (default)**
```
rustfs://stellaops/
layers/<sha256>/sbom.cdx.json.zst
layers/<sha256>/sbom.spdx.json.zst
images/<imgDigest>/inventory.cdx.pb
images/<imgDigest>/usage.cdx.pb
indexes/<imgDigest>/bom-index.bin
attest/<artifactSha256>.dsse.json
```
### Database Architecture (PostgreSQL)
StellaOps uses PostgreSQL for all control-plane data with **per-module schema isolation**. Each module owns and manages only its own schema, ensuring clear ownership and independent migration lifecycles.
**Schema topology:**
```
┌─────────────────────────────────────────────────────────────────┐
│ PostgreSQL Cluster │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ stellaops (database) ││
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ││
│ │ │ auth │ │ vuln │ │ vex │ │scheduler│ ││
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ ││
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ││
│ │ │ notify │ │ policy │ │ audit │ ││
│ │ └─────────┘ └─────────┘ └─────────┘ ││
│ └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
```
**Schema ownership:**
| Schema | Owner Module | Purpose |
|--------|--------------|---------|
| `auth` | Authority | Identity, authentication, authorization, licensing, sessions |
| `vuln` | Concelier | Vulnerability advisories, CVSS, affected packages, sources |
| `vex` | Excititor | VEX statements, graphs, observations, evidence, consensus |
| `scheduler` | Scheduler | Jobs, triggers, workers, locks, execution history |
| `notify` | Notify | Channels, templates, rules, deliveries, escalations |
| `policy` | Policy | Policy packs, rules, risk profiles, evaluations |
| `audit` | Shared | Cross-cutting audit log (optional) |
**Key design principles:**
1. **Module isolation** — Each module controls only its own schema. Cross-schema queries are rare and explicitly documented.
2. **Multi-tenancy** — Single database, single schema set, `tenant_id` column on all tenant-scoped tables with row-level security.
3. **Forward-only migrations** — No down migrations; fixes are applied as new forward migrations.
4. **Advisory lock coordination** — Startup migrations use `pg_try_advisory_lock(hashtext('schema_name'))` to prevent concurrent execution.
5. **Air-gap compatible** — All migrations embedded in assemblies, no external network dependencies.
**Migration categories:**
| Category | Prefix | Execution | Description |
|----------|--------|-----------|-------------|
| Startup (A) | `001-099` | Automatic at boot | Non-breaking DDL (CREATE IF NOT EXISTS, ADD COLUMN nullable) |
| Release (B) | `100-199` | Manual via CLI | Breaking changes (DROP, ALTER TYPE), require maintenance window |
| Seed | `S001-S999` | After schema | Reference data with ON CONFLICT DO NOTHING |
| Data (C) | `DM001-DM999` | Background job | Batched data transformations, resumable |
**Detailed documentation:** See [`docs/db/`](db/README.md) for full specification, coding rules, and phase-by-phase conversion tasks.
**Operations guide:** See [`docs/operations/postgresql-guide.md`](operations/postgresql-guide.md) for performance tuning, monitoring, backup/restore, and scaling.
**Retention**
* RustFS applies retention via `X-RustFS-Retain-Seconds`; Scanner.WebService GC decrements `refCount` and deletes unreferenced metadata; S3/MinIO fallback retains native Object Lock when enabled.
* PostgreSQL retention managed via time-based partitioning for high-volume tables (runs, execution_logs) with monthly partition drops.
---
## 7) APIs (consolidated surface)
### 7.1 Scanner.WebService
```
POST /api/scans { imageRef|digest, force? } → { scanId }
GET /api/scans/{id} → { status, digests, artifacts[] }
GET /api/sboms/{imageDigest} ?format=cdx-json|cdx-pb|spdx-json&view=inventory|usage
GET /api/diff?old=<digest>&new=<digest> → { added[], removed[], changed[], byLayer[] }
POST /api/exports { imageDigest, format, view } → { artifactId, rekorUrl }
POST /api/reports { imageDigest, policyRevision?, vexSnapshot? } → { reportId, verdict, rekorUrl }
GET /api/catalog/artifacts/{id} → { size, ttl, immutable, rekor, refs }
GET /healthz | /readyz | /metrics
```
### 7.2 Signer (mTLS; hard gate)
```
POST /sign/dsse # body: {subjectHash, imageDigest, predicate}; headers: OpTok (DPoP/mTLS) + PoE
GET /verify/referrers?imageDigest=sha256:... # is this image StellaOps-signed?
```
### 7.3 Attestor (mTLS)
```
POST /rekor/entries # DSSE bundle → {uuid, index, proof, logURL}
GET /rekor/entries/{uuid}
```
### 7.4 Authority (OIDC)
* `/.well-known/openid-configuration`, `/oauth/token` (DPoP/mTLS), `/oauth/introspect`, `/jwks`
### 7.5 Licensing (cloud)
```
POST /license/enroll { LT, pubKey } → PoE + introspection endpoints
POST /license/revoke { license_id } → ok
POST /license/introspect { poe } → { active, claims, exp }
POST /attest/endorse { bundle } → endorsement bundle (optional)
```
### 7.6 Scheduler
```
POST /api/v1/scheduler/schedules {yaml|json} → { scheduleId }
GET /api/v1/scheduler/schedules → [ { id, nextRun, status, stats } ]
POST /api/v1/scheduler/run { id|selector } → { runId }
GET /api/v1/scheduler/runs/{id} → { status, counts, links }
GET /api/v1/scheduler/cursor → { lastConcelierExportId, lastExcititorExportId }
```
### 7.7 Notify
```
POST /api/v1/notify/test { channel, target } → { delivered }
POST /api/v1/notify/rules {yaml|json} → { ruleId }
GET /api/v1/notify/rules → [ { id, match, actions, enabled } ]
GET /api/v1/notify/deliveries → [ { id, eventId, channel, status, attempts } ]
```
---
## 8) Security & verifiability
* **Sender‑constrained tokens.** All operational calls use **DPoP** (RFC 9449) or **mTLS‑bound** tokens (RFC 8705).
* **Entitlement.** **PoE** is mandatory; revocation honored online.
* **Release integrity.** **Signer** independently verifies **scanner image digest** via **Referrers + cosign** before signing.
* **Separation of duties.** Scanner/UI/Scheduler/Notify cannot sign; only **Signer** can sign; only **Attestor** can write to **Rekor v2**.
* **Verifiers.** Anyone can verify: DSSE signature → certificate chain to **Stella Ops Fulcio/KMS root** → **Rekor v2** inclusion.
* **RBAC.** Roles: `scanner.admin|read`, `scheduler.admin|read`, `notify.admin|read`, `zastava.admin|read`.
* **Community vs Authorized.** Free/community runs throttled with no official attestations; authorized runs full speed and produce **Stella Ops‑verified** bundles.
**DSSE predicate (SBOM/report)**
```json
{
"predicateType": "https://stella-ops.org/attestations/sbom/1",
"subject": [{ "name": "s3://stellaops/images/<digest>/inventory.cdx.pb", "digest": { "sha256": "<sha256>" } }],
"predicate": {
"image_digest": "<sha256:...>",
"stellaops_version": "2.3.1 (2027.04)",
"license_id": "LIC-9F2A...",
"customer_id": "CUST-ACME",
"plan": "pro",
"policy_digest": "sha256:...",
"views": ["inventory","usage"],
"created": "2025-10-17T12:34:56Z"
}
}
```
**BOM‑Index sidecar**
Binary header + purl table + roaring bitmaps; optional `usedByEntrypoint` flags for fast policy joins.
---
## 9) Scale, performance & quotas
* **Workers:** horizontal; **distributed lock per layer digest**; global CAS in RustFS.
* **Queues:** Valkey Streams (default); NATS JetStream available as opt-in alternative. HPA by queue depth, CPU, memory.
* **Registry throttling:** per‑registry concurrency budgets.
* **Targets:**
* Build‑time path P95 ≤ 3–5 s on warmed bases.
* Post‑build delta scan P95 ≤ 10 s for 200 MB images.
* Policy + VEX evaluation ≤ 500 ms for 5k components using BOM‑Index.
* **Event → notification** p95 ≤ **30–60 s** under nominal load.
* **Export delta → re‑evaluation verdict** p95 ≤ **5 min** for 10k impacted images.
* **Quotas:** license plan enforces QPS/concurrency/size; **Signer** throttles and can deny DSSE.
---
## 10) DevOps & distribution
* **Releases:** all first‑party images **cosign‑signed**; labels embed `org.stellaops.version` and `org.stellaops.release_date`.
* **Channels:**
* **Community** (public registry): throttled, non‑attesting.
* **Authorized** (private registry): full speed, DSSE enabled.
* **Client update flow:** containers self‑verify signatures at boot; report version; **Signer** enforces `valid_release_year` / `max_version` from PoE before signing.
* **Compose skeleton:**
```yaml
services:
authority: { image: stellaops/authority, depends_on: [postgres, valkey] }
fulcio: { image: sigstore/fulcio }
rekor: { image: sigstore/rekor-v2 }
rustfs: { image: stellaops/rustfs, environment: { RUSTFS_DATA: /data } }
valkey: { image: valkey/valkey:8.0, command: valkey-server }
postgres: { image: postgres:16-alpine, environment: { POSTGRES_DB: stellaops, POSTGRES_USER: stellaops } }
signer: { image: stellaops/signer, depends_on: [authority, fulcio] }
attestor: { image: stellaops/attestor, depends_on: [rekor, signer] }
scanner-web: { image: stellaops/scanner-web, depends_on: [postgres, rustfs, valkey, signer, attestor] }
scanner-worker: { image: stellaops/scanner-worker, deploy: { replicas: 4 }, depends_on: [scanner-web] }
concelier: { image: stellaops/concelier-web, depends_on: [postgres] }
excititor: { image: stellaops/excititor-web, depends_on: [postgres] }
scheduler-web: { image: stellaops/scheduler-web, depends_on: [postgres, valkey] }
scheduler-worker:{ image: stellaops/scheduler-worker, deploy: { replicas: 2 }, depends_on: [scheduler-web] }
notify-web: { image: stellaops/notify-web, depends_on: [postgres, valkey] }
notify-worker: { image: stellaops/notify-worker, deploy: { replicas: 2 }, depends_on: [notify-web] }
ui: { image: stellaops/ui, depends_on: [scanner-web, concelier, excititor, scheduler-web, notify-web] }
```
* **Binary prerequisites (offline-first):**
* NuGet packages restore from standard feeds configured in `nuget.config` (dotnet-public, nuget-mirror, nuget.org) to the global NuGet cache. For air-gapped environments, use `dotnet restore --source <offline-feed-path>` pointing to a local `.nupkg` mirror.
* Non-NuGet binaries (plugins/CLIs/tools) are catalogued with SHA-256 in `vendor/manifest.json`; air-gap bundles are registered in `offline/feeds/manifest.json`.
* CI guard: `scripts/verify-binaries.sh` blocks binaries outside approved roots; offline restores use `dotnet restore --source <offline-feed>` with `OFFLINE=1` (override via `ALLOW_REMOTE=1`).
* **Backups:** PostgreSQL dumps (pg_dump) and WAL archiving; RustFS snapshots (or S3 versioning when fallback driver is used); Rekor v2 DB snapshots; JWKS/Fulcio/KMS key rotation. See [`docs/operations/postgresql-guide.md`](operations/postgresql-guide.md).
* **Ops runbooks:** Scheduler catch‑up after Concelier/Excititor recovery; connector key rotation (Slack/Teams/SMTP).
* **SLOs & alerts:** lag between Concelier/Excititor export and first rescan verdict; delivery failure rates by channel.
---
## 11) Observability & audit
* **Metrics:** scan latency, layer cache hit %, artifact bytes, DSSE/Rekor latency, policy evaluation time, queue depth, admission decisions (Zastava).
* **Scheduler metrics:** `scheduler.impacted_images_total`, `scheduler.jobs_enqueued_total`, `scheduler.selection_ms`, end‑to‑end p95 (event → verdict).
* **Notify metrics:** `notify.sent_total{channel}`, `notify.dropped_total{reason}`, `notify.digest_coalesced_total`, `notify.latency_ms`.
* **Tracing:** per‑stage spans; correlation IDs across Scanner→Signer→Attestor and Concelier/Excititor→Scheduler→Scanner→Notify.
* **Audit logs:** every signing records `license_id`, `image_digest`, `policy_digest`, and Rekor UUID; Scheduler records who scheduled what; Notify records where, when, and why messages were sent or deduped.
* **Compliance:** RustFS retention headers keep immutable artifacts tamper‑resistant; S3-compatibility mode can use native Object Lock when enabled for legacy deployments; reproducible outputs via policy digest + SBOM digest in predicate.
---
## 12) Roadmap (anchored to this architecture)
* M2: Windows MSI/SxS/GAC analyzers; deeper Rust (DWARF enrichers).
* M2: Buildx generator certified flows; cross‑registry trust policies.
* M3: Patch‑Presence plugin (signature‑based backport detection), opt‑in.
* M3: Zastava Admission control GA with policy presets and dry‑run→enforce stages.
* M3: **Scheduler GA** with export‑delta impact routing and capacity‑aware pacing.
* M3: **Notify GA** with digests, Slack/Teams/Email/Webhooks; **M4:** PagerDuty/Opsgenie connectors.
* Continuous: Policy UX (waiver TTLs, vendor rules), Excititor connectors expansion.
---
## 13) Canonical sequences (verification, re‑evaluation & notify)
**Sign & log (OpTok + PoE, image verify, DSSE, Rekor).**
```mermaid
sequenceDiagram
autonumber
participant Scan as Scanner.WebService
participant Auth as Authority (OIDC)
participant Sign as Signer
participant Reg as OCI Registry
participant Ful as Fulcio/KMS
participant Att as Attestor
participant Rek as Rekor v2
Scan->>Auth: Get OpTok (DPoP/mTLS)
Scan->>Sign: sign(request) + OpTok + PoE + DPoP proof
Sign->>Auth: Validate OpTok & sender-constraint
Sign->>Sign: Validate PoE (introspect/revocation)
Sign->>Reg: Verify scanner image is StellaOps-signed (Referrers + cosign)
alt OK
Sign->>Ful: Get signing cert (keyless) or use KMS key
Sign-->>Scan: DSSE bundle (cert chain)
Scan->>Att: Submit bundle
Att-->>Rek: Create entry
Rek-->>Att: {uuid,index,proof}
Att-->>Scan: Rekor URL
else Deny
Sign-->>Scan: 403 (no attestation)
end
```
**Event‑driven re‑evaluation & notify.**
```mermaid
sequenceDiagram
participant CONC as Concelier
participant EXC as Excititor
participant SCH as Scheduler
participant SC as Scanner.WebService
participant NO as Notify
CONC->>SCH: export.delta {changedProductKeys, exportId}
EXC ->>SCH: export.delta {changedProductKeys, exportId}
SCH->>SCH: Impact select via BOM-Index bitmaps
SCH->>SC: Enqueue analysis-only reports (batches)
SC-->>SCH: verdict stream (PASS/FAIL, deltas)
SCH->>NO: rescan.delta {imageDigest, newCriticals, links}
NO-->>Slack/Teams/Email/Webhook: deliver (throttle/digest rules applied)
```
---
## 14) Minimal data shapes (Scheduler & Notify)
**Scheduler schedule (YAML via UI/CLI)**
```yaml
name: nightly-eu
when: "0 2 * * * Europe/Sofia"
mode: analysis-only # or content-refresh
selection:
scope: all-images # or tenant/ns/repo label selectors
onlyIf: { lastReportOlderThanDays: 7 }
notify:
onNewFindings: true
minSeverity: high
limits:
maxJobs: 5000
ratePerSecond: 50
```
**Notify rule (YAML)**
```yaml
name: high-critical-alerts
match:
eventKinds: ["report.ready","rescan.delta","zastava.admission"]
minSeverity: high
namespaces: ["prod-*"]
vex: { includeAcceptedJustifications: false }
actions:
- channel: slack
target: "#sec-alerts"
template: "concise"
throttle: "5m"
- channel: email
target: "soc@acme.org"
digest: "hourly"
enabled: true
```
# High-Level Architecture (Reference Map)
This document is the canonical index for StellaOps architecture.
It is intentionally a map, not a full re-statement of every module dossier.
If you want a short walkthrough, start with `docs/40_ARCHITECTURE_OVERVIEW.md`.
## How the docs are organized
StellaOps documentation is two-level:
- High-level, canonical docs live in `docs/*.md`
- Detailed references live under `docs/**` (module dossiers, API contracts, runbooks, schemas)
Entry points:
- Full technical index: `docs/technical/README.md`
- Platform architecture index: `docs/technical/architecture/README.md`
## Guiding principles (stable)
- Deterministic outputs: stable ordering, stable identifiers, UTC ISO-8601 timestamps, canonical hashing where applicable.
- Offline-first posture: the workflow must run connected or air-gapped using Offline Kit bundles and locally verifiable signatures.
- Evidence-linked decisions: every decision should link back to concrete evidence (SBOMs, observations, reachability, attestations).
- Aggregation-not-merge for upstream evidence: preserve provenance and conflicts rather than silently collapsing them.
## Architecture views (authoritative)
These documents are the authoritative detailed views used by module dossiers and runbooks:
- Platform topology: `docs/technical/architecture/platform-topology.md`
- Infrastructure dependencies: `docs/technical/architecture/infrastructure-dependencies.md`
- Request and data flows: `docs/technical/architecture/request-flows.md`
- Data isolation model: `docs/technical/architecture/data-isolation.md`
- Security boundaries: `docs/technical/architecture/security-boundaries.md`
## Modules (authoritative dossiers)
The per-module dossiers (architecture + implementation plan + operations) are indexed here:
- `docs/technical/architecture/README.md`
Use module dossiers as the source of truth for:
- APIs and storage schemas owned by the module
- lifecycle, trust boundaries, and failure modes
- determinism rules and offline expectations
## Identity, tenancy, and headers
Tenancy and identity context are part of the platform contract:
- Gateway tenant auth and ABAC contract: `docs/api/gateway/tenant-auth.md`
- Gateway identity header policy (spoofing prevention + migration rules): `docs/modules/gateway/identity-header-policy.md`
- Authority service dossier: `docs/modules/authority/architecture.md`
- Claims and headers index: `docs/claims-index.md`
## APIs and CLI reference
Canonical entry points:
- API and CLI reference hub: `docs/09_API_CLI_REFERENCE.md`
- API conventions (headers, errors, pagination, determinism): `docs/api/overview.md`
- API contracts and samples: `docs/api/`
- CLI command guides: `docs/modules/cli/guides/commands/`
## Offline, verification, and operations
Canonical entry points:
- Offline Kit: `docs/24_OFFLINE_KIT.md`
- Security hardening: `docs/17_SECURITY_HARDENING_GUIDE.md`
- Installation guide: `docs/21_INSTALL_GUIDE.md`
- Ops and runbooks: `docs/operations/`, `docs/modules/*/operations/`
## Data and schemas
Use these as the canonical map for schemas and contracts:
- Data schemas (high-level index): `docs/11_DATA_SCHEMAS.md`
- Database specifications: `docs/db/`
- Events (schemas + samples): `docs/events/`
## Related high-level docs
- Product overview: `docs/overview.md`
- Key features: `docs/key-features.md`
- Roadmap (internal): `docs/05_ROADMAP.md`
- Glossary: `docs/14_GLOSSARY_OF_TERMS.md`

1389
docs/09_API_CLI_REFERENCE.md
View File

File diff suppressed because it is too large Load Diff

528
docs/10_CONCELIER_CLI_QUICKSTART.md
View File

@@ -1,426 +1,102 @@
# 10 · Concelier + CLI Quickstart
This guide walks through configuring the Concelier web service and the `stellaops-cli`
tool so an operator can ingest advisories, merge them, and publish exports from a
single workstation. It focuses on deployment-facing surfaces only (configuration,
runtime wiring, CLI usage) and leaves connector/internal customization for later.
---
## 0 · Prerequisites
- .NET SDK **10.0.100-preview** (matches `global.json`)
- PostgreSQL instance reachable from the host (local Docker or managed)
- `trivy-db` binary on `PATH` for Trivy exports (and `oras` if publishing to OCI)
- Plugin assemblies present in `StellaOps.Concelier.PluginBinaries/` (already included in the repo)
- Optional: Docker/Podman runtime if you plan to run scanners locally
> **Tip** – air-gapped installs should preload `trivy-db` and `oras` binaries into the
> runner image since Concelier never fetches them dynamically.
---
## 1 · Configure Concelier
1. Copy the sample config to the expected location (CI/CD pipelines can stamp values
into this file during deployment—see the “Deployment automation” note below):
```bash
mkdir -p etc
cp etc/concelier.yaml.sample etc/concelier.yaml
```
2. Edit `etc/concelier.yaml` and update the PostgreSQL DSN (and optional database name).
The default template configures plug-in discovery to look in `StellaOps.Concelier.PluginBinaries/`
and disables remote telemetry exporters by default.
3. (Optional) Override settings via environment variables. All keys are prefixed with
`CONCELIER_`. Example:
```bash
export CONCELIER_STORAGE__DSN="Host=localhost;Port=5432;Database=concelier;Username=user;Password=pass"
export CONCELIER_TELEMETRY__ENABLETRACING=false
```
4. Start the web service from the repository root:
```bash
dotnet run --project src/Concelier/StellaOps.Concelier.WebService
```
On startup Concelier validates the options, boots PostgreSQL indexes, loads plug-ins,
and exposes:
- `GET /health` – returns service status and telemetry settings
- `GET /ready` – performs a PostgreSQL `ping`
- `GET /jobs` + `POST /jobs/{kind}` – inspect and trigger connector/export jobs
> **Security note** – authentication now ships via StellaOps Authority. Keep
> `authority.allowAnonymousFallback: true` only during the staged rollout and
> disable it before **2025-12-31 UTC** so tokens become mandatory.
Rollout checkpoints for the two Authority toggles:
| Phase | `authority.enabled` | `authority.allowAnonymousFallback` | Goal | Observability focus |
| ----- | ------------------- | ---------------------------------- | ---- | ------------------- |
| **Validation (staging)** | `true` | `true` | Verify token issuance, CLI scopes, and audit log noise without breaking cron jobs. | Watch `Concelier.Authorization.Audit` for `bypass=True` events and scope gaps; confirm CLI `auth status` succeeds. |
| **Cutover rehearsal** | `true` | `false` | Exercise production-style enforcement before the deadline; ensure only approved maintenance ranges remain in `bypassNetworks`. | Expect some HTTP 401s; verify `web.jobs.triggered` metrics flatten for unauthenticated calls and audit logs highlight missing tokens. |
| **Enforced (steady state)** | `true` | `false` | Production baseline after the 2025-12-31 UTC cutoff. | Alert on new `bypass=True` entries and on repeated 401 bursts; correlate with Authority availability dashboards. |
### Authority companion configuration (preview)
1. Copy the Authority sample configuration:
```bash
cp etc/authority.yaml.sample etc/authority.yaml
```
2. Update the issuer URL, token lifetimes, and plug-in descriptors to match your
environment. Authority expects per-plugin manifests in `etc/authority.plugins/`;
sample `standard.yaml` and `ldap.yaml` files are provided as starting points.
For air-gapped installs keep the default plug-in binary directory
(`../StellaOps.Authority.PluginBinaries`) so packaged plug-ins load without outbound access.
3. Environment variables prefixed with `STELLAOPS_AUTHORITY_` override individual
fields. Example:
```bash
export STELLAOPS_AUTHORITY__ISSUER="https://authority.stella-ops.local"
export STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0="/srv/authority/plugins"
```
---
## 2 · Configure the CLI
The CLI reads configuration from JSON/YAML files *and* environment variables. The
defaults live in `src/Cli/StellaOps.Cli/appsettings.json` and expect overrides at runtime.
| Setting | Environment variable | Default | Purpose |
| ------- | -------------------- | ------- | ------- |
| `BackendUrl` | `STELLAOPS_BACKEND_URL` | _empty_ | Base URL of the Concelier web service |
| `ApiKey` | `API_KEY` | _empty_ | Reserved for legacy key auth; leave empty when using Authority |
| `ScannerCacheDirectory` | `STELLAOPS_SCANNER_CACHE_DIRECTORY` | `scanners` | Local cache folder |
| `ResultsDirectory` | `STELLAOPS_RESULTS_DIRECTORY` | `results` | Where scan outputs are written |
| `Authority.Url` | `STELLAOPS_AUTHORITY_URL` | _empty_ | StellaOps Authority issuer/token endpoint |
| `Authority.ClientId` | `STELLAOPS_AUTHORITY_CLIENT_ID` | _empty_ | Client identifier for the CLI |
| `Authority.ClientSecret` | `STELLAOPS_AUTHORITY_CLIENT_SECRET` | _empty_ | Client secret (omit when using username/password grant) |
| `Authority.Username` | `STELLAOPS_AUTHORITY_USERNAME` | _empty_ | Username for password grant flows |
| `Authority.Password` | `STELLAOPS_AUTHORITY_PASSWORD` | _empty_ | Password for password grant flows |
| `Authority.Scope` | `STELLAOPS_AUTHORITY_SCOPE` | `concelier.jobs.trigger advisory:ingest` | Space-separated OAuth scopes requested for backend operations |
| `Authority.TokenCacheDirectory` | `STELLAOPS_AUTHORITY_TOKEN_CACHE_DIR` | `~/.stellaops/tokens` | Directory that persists cached tokens |
| `Authority.Resilience.EnableRetries` | `STELLAOPS_AUTHORITY_ENABLE_RETRIES` | `true` | Toggle Polly retry handler for Authority HTTP calls |
| `Authority.Resilience.RetryDelays` | `STELLAOPS_AUTHORITY_RETRY_DELAYS` | `1s,2s,5s` | Comma- or space-separated backoff delays (hh:mm:ss) |
| `Authority.Resilience.AllowOfflineCacheFallback` | `STELLAOPS_AUTHORITY_ALLOW_OFFLINE_CACHE_FALLBACK` | `true` | Allow CLI to reuse cached discovery/JWKS metadata when Authority is offline |
| `Authority.Resilience.OfflineCacheTolerance` | `STELLAOPS_AUTHORITY_OFFLINE_CACHE_TOLERANCE` | `00:10:00` | Additional tolerance window applied to cached metadata |
Example bootstrap:
```bash
export STELLAOPS_BACKEND_URL="http://localhost:5000"
export STELLAOPS_RESULTS_DIRECTORY="$HOME/.stellaops/results"
export STELLAOPS_AUTHORITY_URL="https://authority.local"
export STELLAOPS_AUTHORITY_CLIENT_ID="concelier-cli"
export STELLAOPS_AUTHORITY_CLIENT_SECRET="s3cr3t"
export STELLAOPS_AUTHORITY_SCOPE="concelier.jobs.trigger advisory:ingest advisory:read"
dotnet run --project src/Cli/StellaOps.Cli -- db merge
# Acquire a bearer token and confirm cache state
dotnet run --project src/Cli/StellaOps.Cli -- auth login
dotnet run --project src/Cli/StellaOps.Cli -- auth status
dotnet run --project src/Cli/StellaOps.Cli -- auth whoami
```
Refer to `docs/dev/32_AUTH_CLIENT_GUIDE.md` for deeper guidance on tuning retry/offline settings and rollout checklists.
To persist configuration, you can create `stellaops-cli.yaml` next to the binary or
rely on environment variables for ephemeral runners.
---
## 3 · Operating Workflow
1. **Trigger connector fetch stages**
```bash
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage fetch
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage parse
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source osv --stage map
```
Use `--mode resume` when continuing from a previous window:
```bash
dotnet run --project src/Cli/StellaOps.Cli -- db fetch --source redhat --stage fetch --mode resume
```
2. **Merge canonical advisories**
```bash
dotnet run --project src/Cli/StellaOps.Cli -- db merge
```
3. **Produce exports**
```bash
# JSON tree (vuln-list style)
dotnet run --project src/Cli/StellaOps.Cli -- db export --format json
# Trivy DB (delta example)
dotnet run --project src/Cli/StellaOps.Cli -- db export --format trivy-db --delta
```
Concelier always produces a deterministic OCI layout. The first run after a clean
bootstrap emits a **full** baseline; subsequent `--delta` runs reuse the previous
baseline’s blobs when only JSON manifests change. If the exporter detects that a
prior delta is still active (i.e., `LastDeltaDigest` is recorded) it automatically
upgrades the next run to a full export and resets the baseline so operators never
chain deltas indefinitely. The CLI exposes `--publish-full/--publish-delta` (for
ORAS pushes) and `--include-full/--include-delta` (for offline bundles) should you
need to override the defaults interactively.
**Smoke-check delta reuse:** after the first baseline completes, run the export a
second time with `--delta` and verify that the new directory reports `mode=delta`
while reusing the previous layer blob.
```bash
export_root=${CONCELIER_EXPORT_ROOT:-exports/trivy}
base=$(ls -1d "$export_root"/* | sort | tail -n2 | head -n1)
delta=$(ls -1d "$export_root"/* | sort | tail -n1)
jq -r '.mode,.baseExportId' "$delta/metadata.json"
base_manifest=$(jq -r '.manifests[0].digest' "$base/index.json")
delta_manifest=$(jq -r '.manifests[0].digest' "$delta/index.json")
printf 'baseline manifest: %s\ndelta manifest: %s\n' "$base_manifest" "$delta_manifest"
layer_digest=$(jq -r '.layers[0].digest' "$base/blobs/sha256/${base_manifest#sha256:}")
cmp "$base/blobs/sha256/${layer_digest#sha256:}" \
"$delta/blobs/sha256/${layer_digest#sha256:}"
```
`cmp` returning exit code `0` confirms the delta export reuses the baseline’s
`db.tar.gz` layer instead of rebuilding it.
4. **Verify guard compliance**
```bash
export STELLA_TENANT="${STELLA_TENANT:-tenant-a}"
dotnet run --project src/Cli/StellaOps.Cli -- aoc verify \
--since 24h \
--format table \
--tenant "$STELLA_TENANT"
# Optional: capture JSON evidence for pipelines/audits
dotnet run --project src/Cli/StellaOps.Cli -- aoc verify \
--since 7d \
--limit 100 \
--format json \
--export artifacts/aoc-verify.json \
--tenant "$STELLA_TENANT"
```
The CLI exits with `0` when no violations are detected. Guard failures map
to `ERR_AOC_00x` codes (`11…17`), while truncated results return `18`. Use
`--sources`/`--codes` to focus on noisy connectors and feed the exported JSON
into dashboards or evidence lockers for compliance reviews.
5. **Pre-flight individual payloads**
```bash
stella sources ingest --dry-run \
--source redhat \
--input ./fixtures/redhat/RHSA-2025-9999.json \
--tenant "$STELLA_TENANT" \
--format json \
--output artifacts/redhat-dry-run.json
```
Exit code `0` confirms the candidate document is AOC compliant. Any guard
violation is emitted as deterministic `ERR_AOC_00x` exit codes (`11…17`);
reuse the exported JSON in PRs or incident timelines to show offending paths.
6. **Manage scanners (optional)**
```bash
dotnet run --project src/Cli/StellaOps.Cli -- scanner download --channel stable
dotnet run --project src/Cli/StellaOps.Cli -- scan run --entry scanners/latest/Scanner.dll --target ./sboms
dotnet run --project src/Cli/StellaOps.Cli -- scan upload --file results/scan-001.json
```
Add `--verbose` to any command for structured console logs. All commands honour
`Ctrl+C` cancellation and exit with non-zero status codes when the backend returns
a problem document.
---
## 4 · Verification Checklist
- Concelier `/health` returns `"status":"healthy"` and Storage bootstrap is marked
complete after startup.
- CLI commands return HTTP 202 with a `Location` header (job tracking URL) when
triggering Concelier jobs.
- Export artefacts are materialised under the configured output directories and
their manifests record digests.
- PostgreSQL contains the expected `document`, `dto`, `advisory`, and `export_state`
tables after a run.
---
## 5 · Deployment Automation
- Treat `etc/concelier.yaml.sample` as the canonical template. CI/CD should copy it to
the deployment artifact and replace placeholders (DSN, telemetry endpoints, cron
overrides) with environment-specific secrets.
- Keep secret material (PostgreSQL credentials, OTLP tokens) outside of the repository;
inject them via secret stores or pipeline variables at stamp time.
- When building container images, include `trivy-db` (and `oras` if used) so air-gapped
clusters do not need outbound downloads at runtime.
---
## 5 · Next Steps
- Enable authority-backed authentication in non-production first. Set
`authority.enabled: true` while keeping `authority.allowAnonymousFallback: true`
to observe logs, then flip it to `false` before 2025-12-31 UTC to enforce tokens.
- Automate the workflow above via CI/CD (compose stack or Kubernetes CronJobs).
- Pair with the Concelier connector teams when enabling additional sources so their
module-specific requirements are pulled in safely.
---
## 6 · Authority Integration
- Concelier now authenticates callers through StellaOps Authority using OAuth 2.0
resource server flows. Populate the `authority` block in `concelier.yaml`:
```yaml
authority:
enabled: true
allowAnonymousFallback: false # keep true only during the staged rollout window
issuer: "https://authority.example.org"
audiences:
- "api://concelier"
requiredScopes:
- "concelier.jobs.trigger"
- "advisory:read"
- "advisory:ingest"
requiredTenants:
- "tenant-default"
clientId: "concelier-jobs"
clientSecretFile: "../secrets/concelier-jobs.secret"
clientScopes:
- "concelier.jobs.trigger"
- "advisory:read"
- "advisory:ingest"
bypassNetworks:
- "127.0.0.1/32"
- "::1/128"
```
- Store the client secret outside of source control. Either provide it via
`authority.clientSecret` (environment variable `CONCELIER_AUTHORITY__CLIENTSECRET`)
or point `authority.clientSecretFile` to a file mounted at runtime.
- Cron jobs running on the same host can keep using the API thanks to the loopback
bypass mask. Add additional CIDR ranges as needed; every bypass is logged.
- Export the same configuration to Kubernetes or systemd by setting environment
variables such as:
```bash
export CONCELIER_AUTHORITY__ENABLED=true
export CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK=false
export CONCELIER_AUTHORITY__ISSUER="https://authority.example.org"
export CONCELIER_AUTHORITY__CLIENTID="concelier-jobs"
export CONCELIER_AUTHORITY__CLIENTSECRETFILE="/var/run/secrets/concelier/authority-client"
export CONCELIER_AUTHORITY__REQUIREDSCOPES__0="concelier.jobs.trigger"
export CONCELIER_AUTHORITY__REQUIREDSCOPES__1="advisory:read"
export CONCELIER_AUTHORITY__REQUIREDSCOPES__2="advisory:ingest"
export CONCELIER_AUTHORITY__CLIENTSCOPES__0="concelier.jobs.trigger"
export CONCELIER_AUTHORITY__CLIENTSCOPES__1="advisory:read"
export CONCELIER_AUTHORITY__CLIENTSCOPES__2="advisory:ingest"
export CONCELIER_AUTHORITY__REQUIREDTENANTS__0="tenant-default"
```
- CLI commands already pass `Authorization` headers when credentials are supplied.
Configure the CLI with matching Authority settings (`docs/09_API_CLI_REFERENCE.md`)
so that automation can obtain tokens with the same client credentials. Concelier
logs every job request with the client ID, subject (if present), scopes, and
a `bypass` flag so operators can audit cron traffic.
- **Rollout checklist.**
1. Stage the integration with fallback enabled (`allowAnonymousFallback=true`) and confirm CLI/token issuance using `stella auth status`.
2. Follow the rehearsal pattern (`allowAnonymousFallback=false`) while monitoring `Concelier.Authorization.Audit` and `web.jobs.triggered`/`web.jobs.trigger.failed` metrics.
3. Lock in enforcement, review the audit runbook (`docs/modules/concelier/operations/authority-audit-runbook.md`), and document the bypass CIDR approvals in your change log.
---
## 7 · Policy Starter Pack (Day 1)
StellaOps provides a production-ready starter policy that blocks reachable HIGH/CRITICAL
vulnerabilities while respecting VEX statements and enforcing metadata quality gates.
### Quick Installation
```bash
# Install the starter policy pack
stellaops policy install starter-day1
# Verify the installation
stellaops policy list-packs
```
### One-Liner for Scans with Policy
```bash
# Scan with policy evaluation
stellaops scan run --image myregistry/myapp:latest --policy starter-day1
# Check policy verdict
stellaops verdict verify myregistry/myapp:latest
```
### What Starter Policy Does
| Finding Type | Action | Notes |
|--------------|--------|-------|
| Reachable HIGH/CRITICAL | **Block** | Unless VEX says `not_affected` |
| Reachable MEDIUM | **Warn** | Review recommended |
| Unreachable vulnerabilities | **Allow** | Logged for awareness |
| Unknowns > 5% | **Block** | Quality gate for SBOM coverage |
| Unsigned SBOM (prod) | **Block** | Integrity requirement |
### Environment Overrides
Apply environment-specific behavior:
```bash
# Development - warnings only, no signing required
stellaops scan run --image myapp:dev --policy starter-day1 --env development
# Production - full enforcement (default)
stellaops scan run --image myapp:prod --policy starter-day1 --env production
```
### Simulate Before Deploying
Test policy impact on existing scans without blocking:
```bash
stellaops policy simulate --policy policies/starter-day1.yaml --scan <scan-id>
```
### Distribution (Air-Gapped Environments)
For air-gapped installations, export and import policy packs as offline bundles:
```bash
# Export to offline bundle
stellaops policy export-bundle --policy policies/starter-day1.yaml \
--output starter-day1-bundle.tar.gz
# Import in air-gapped environment
stellaops policy import-bundle --bundle starter-day1-bundle.tar.gz
```
See `docs/policy/starter-guide.md` for detailed customization and migration guidance.
# Concelier + CLI Quickstart
This quickstart gets an operator to a working advisory ingestion loop:
- Run Concelier (advisory ingestion + deterministic normalization).
- Trigger ingestion/export jobs.
- Inspect results via the `stella` CLI.
This document stays high level and defers detailed configuration and connector behavior to the Concelier module dossier.
## 1) Prerequisites
- Deployment: follow `docs/21_INSTALL_GUIDE.md` (Compose profiles under `deploy/compose/`).
- Offline/air-gap: follow `docs/24_OFFLINE_KIT.md` and `docs/airgap/overview.md`.
- Local dev (optional): .NET SDK version pinned by `global.json`.
## 2) Run Concelier
### Option A: Run via deployment bundles (recommended)
Use the deterministic Compose profiles under `deploy/compose/` and enable Concelier in the selected profile.
Start here:
- `docs/21_INSTALL_GUIDE.md`
- `docs/modules/concelier/operations/`
### Option B: Run the service from source (dev/debug)
```bash
dotnet run --project src/Concelier/StellaOps.Concelier.WebService
```
Concelier reads `etc/concelier.yaml` by default (and supports environment overrides). See:
- `docs/modules/concelier/architecture.md`
- `docs/modules/concelier/operations/`
## 3) Configure Concelier (minimum)
1. Copy the sample config:
```bash
mkdir -p etc
cp etc/concelier.yaml.sample etc/concelier.yaml
```
2. Update storage/DSN and any connector configuration needed for your sources.
3. Keep configuration deterministic and offline-friendly (no hidden outbound calls in air-gap profiles).
Connector deep dives and operational guidance live under:
- `docs/modules/concelier/operations/connectors/`
## 4) Harden the `/jobs*` surface with Authority (recommended)
Concelier job triggers are operationally sensitive. In production-style installs, require Authority-issued tokens.
Operator entry point:
- `docs/modules/concelier/operations/authority-audit-runbook.md`
At minimum, ensure:
- Authority enforcement is enabled.
- Anonymous fallback is disabled outside controlled rollout windows.
- Any bypass CIDRs are explicitly approved and monitored.
## 5) Use the CLI for ingestion and exports
This guide uses `stella` as the CLI command name. If your packaging uses a different filename, add a local shim/symlink.
### 5.1 Point the CLI at Concelier
Set the backend base URL (example):
```bash
export STELLAOPS_BACKEND_URL="https://concelier.example.internal"
```
Authenticate using the configured Authority credentials:
```bash
stella auth login
stella auth whoami
```
See: `docs/modules/cli/guides/commands/auth.md`.
### 5.2 Trigger connector stages
Trigger a connector stage (example):
```bash
stella db fetch --source osv --stage fetch
stella db fetch --source osv --stage parse
stella db fetch --source osv --stage map
```
### 5.3 Reconcile merges (when needed)
```bash
stella db merge
```
### 5.4 Produce exports
```bash
stella db export --format json
```
See: `docs/modules/cli/guides/commands/db.md`.
### 5.5 Inspect advisory results
For read-only inspection (list/get/export), use:
- `docs/modules/cli/guides/commands/advisory.md`
## 6) Next links
- Concelier module dossier: `docs/modules/concelier/README.md`
- Concelier operations: `docs/modules/concelier/operations/`
- CLI command guides: `docs/modules/cli/guides/commands/`
- API + CLI reference index: `docs/09_API_CLI_REFERENCE.md`

552
docs/10_PLUGIN_SDK_GUIDE.md
View File

@@ -1,428 +1,124 @@
# 10 · Plug‑in SDK Guide — **Stella Ops**
*(v 1.5 — 11 Jul 2025 · template install, no reload, IoC)*
---
## 0 Audience & Scope
Guidance for developers who extend Stella Ops with schedule jobs, scanner adapters, TLS providers, notification channels, etc. Everything here is OSS; commercial variants simply ship additional signed plug‑ins.
---
## 1 Prerequisites
| Tool | Min Version |
| ----------------------- | ----------------------------------------------------------------- |
| .NET SDK | {{ dotnet }} |
| **StellaOps templates** | install once via `bash dotnet new install StellaOps.Templates::*` |
| **Cosign** | 2.3 + — used to sign DLLs |
| xUnit | 2.6 |
| Docker CLI | only if your plug‑in shells out to containers |
---
## 2 Repository & Build Output
Every plug‑in is hosted in **`git.stella‑ops.org`**.
At publish time it must copy its signed artefacts to:
~~~text
src/backend/Stella.Ops.Plugin.Binaries/<MyPlugin>/
├── MyPlugin.dll
└── MyPlugin.dll.sig
~~~
The back‑end scans this folder on start‑up, verifies the **Cosign** signature, confirms the `[StellaPluginVersion]` gate, then loads the DLL inside an **isolated AssemblyLoadContext** to avoid dependency clashes
---
## 3 Project Scaffold
Generate with the installed template:
~~~bash
dotnet new stellaops-plugin-schedule \
-n MyPlugin.Schedule \
--output src
~~~
Result:
~~~text
src/
├─ MyPlugin.Schedule/
│ ├─ MyJob.cs
│ └─ MyPlugin.Schedule.csproj
└─ tests/
└─ MyPlugin.Schedule.Tests/
~~~
---
## 4 MSBuild Wiring
Add this to **`MyPlugin.Schedule.csproj`** so the signed DLL + `.sig` land in the canonical plug‑in folder:
~~~xml
<PropertyGroup>
<StellaPluginOut>$(SolutionDir)src/backend/Stella.Ops.Plugin.Binaries/$(MSBuildProjectName)</StellaPluginOut>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\StellaOps.Common\StellaOps.Common.csproj"
PrivateAssets="all" />
</ItemGroup>
<Target Name="CopyStellaPlugin" AfterTargets="Publish">
<MakeDir Directories="$(StellaPluginOut)" />
<Copy SourceFiles="$(PublishDir)$(AssemblyName).dll;$(PublishDir)$(AssemblyName).dll.sig"
DestinationFolder="$(StellaPluginOut)" />
</Target>
~~~
---
## 5 Dependency‑Injection Entry‑point
Back‑end auto‑discovers restart‑time bindings through two mechanisms:
1. **Service binding metadata** for simple contracts.
2. **`IDependencyInjectionRoutine`** implementations when you need full control.
### 5.1 Service binding metadata
Annotate implementations with `[ServiceBinding]` to declare their lifetime and service contract.
The loader honours scoped lifetimes and will register the service before executing any custom DI routines.
~~~csharp
using Microsoft.Extensions.DependencyInjection;
using StellaOps.DependencyInjection;
[ServiceBinding(typeof(IJob), ServiceLifetime.Scoped, RegisterAsSelf = true)]
public sealed class MyJob : IJob
{
// IJob dependencies can now use scoped services (PostgreSQL connections, etc.)
}
~~~
Use `RegisterAsSelf = true` when you also want to resolve the concrete type.
Set `ReplaceExisting = true` to override default descriptors if the host already provides one.
### 5.2 Dependency injection routines
For advanced scenarios continue to expose a routine:
~~~csharp
namespace StellaOps.DependencyInjection;
public sealed class IoCConfigurator : IDependencyInjectionRoutine
{
public IServiceCollection Register(IServiceCollection services, IConfiguration cfg)
{
services.AddSingleton<IJob, MyJob>(); // schedule job
services.Configure<MyPluginOptions>(cfg.GetSection("Plugins:MyPlugin"));
return services;
}
}
~~~
---
## 6 Schedule Plug‑ins
### 6.1 Minimal Job
~~~csharp
using StellaOps.Scheduling; // contract
[StellaPluginVersion("2.0.0")]
public sealed class MyJob : IJob
{
public async Task ExecuteAsync(CancellationToken ct)
{
Console.WriteLine("Hello from plug‑in!");
await Task.Delay(500, ct);
}
}
~~~
### 6.2 Cron Registration
```csharp
services.AddCronJob<MyJob>("0 15 * * *"); // everyday
```
15:00
Cron syntax follows Hangfire rules 
## 7 Scanner Adapters
Implement IScannerRunner.
Register inside Configure:
```csharp
services.AddScanner<MyAltScanner>("alt"); // backend
```
selects by --engine alt
If the engine needs a side‑car container, include a Dockerfile in your repo and document resource expectations.
## 8 Packaging & Signing
```bash
dotnet publish -c Release -p:PublishSingleFile=true -o out
cosign sign --key $COSIGN_KEY out/MyPlugin.Schedule.dll # sign binary only
sha256sum out/MyPlugin.Schedule.dll > out/.sha256 # optional checksum
zip MyPlugin.zip out/* README.md
```
Unsigned DLLs are refused when StellaOps:Security:DisableUnsigned=false.
## 9 Deployment
```bash
docker cp MyPlugin.zip <backend>:/opt/plugins/ && docker restart <backend>
```
Check /health – "plugins":["MyPlugin.Schedule@2.0.0"].
(Hot‑reload was removed to keep the core process simple and memory‑safe.)
## 10 Configuration Patterns
| Need | Pattern |
| ------------ | --------------------------------------------------------- |
| Settings | Plugins:MyPlugin:* in appsettings.json. |
| Secrets | Redis secure:<plugin>:<key> (encrypted per TLS provider). |
| Dynamic cron | Implement ICronConfigurable; UI exposes editor. |
## 11 Testing & CI
| Layer | Tool | Gate |
| ----------- | -------------------------- | ------------------- |
| Unit | xUnit + Moq | ≥ 50 % lines |
| Integration | Testcontainers ‑ run in CI | Job completes < 5 s |
| Style | dotnet | format 0 warnings |
Use the pre‑baked workflow in StellaOps.Templates as starting point.
## 12 Publishing to the Community Marketplace
Tag Git release plugin‑vX.Y.Z and attach the signed ZIP.
Submit a PR to stellaops/community-plugins.json with metadata & git URL.
On merge, the plug‑in shows up in the UI Marketplace.
## 13 Common Pitfalls
| Symptom | Root cause | Fix |
| ------------------- | -------------------------- | ------------------------------------------- |
| NotDetected | .sig missing | cosign sign … |
| VersionGateMismatch | Backend 2.1 vs plug‑in 2.0 | Re‑compile / bump attribute |
| FileLoadException | Duplicate | StellaOps.Common Ensure PrivateAssets="all" |
| Redis | timeouts Large writes | Batch or use PostgreSQL |
---
## 14 Plugin Version Compatibility (v2.0)
**IMPORTANT:** All plugins **must** declare a `[StellaPluginVersion]` attribute. Plugins without this attribute will be rejected by the host loader.
Declare your plugin's version and host compatibility requirements:
```csharp
using StellaOps.Plugin.Versioning;
// In AssemblyInfo.cs or any file at assembly level
[assembly: StellaPluginVersion("1.2.0", MinimumHostVersion = "1.0.0", MaximumHostVersion = "2.0.0")]
```
| Property | Purpose | Required |
|----------|---------|----------|
| `pluginVersion` (constructor) | Your plugin's semantic version | **Yes** |
| `MinimumHostVersion` | Lowest host version that can load this plugin | Recommended |
| `MaximumHostVersion` | Highest host version supported | Recommended for cross-major compatibility |
| `RequiresSignature` | Whether signature verification is mandatory (default: true) | No |
### Version Compatibility Rules
1. **Attribute Required:** Plugins without `[StellaPluginVersion]` are rejected
2. **Minimum Version:** Host version must be ≥ `MinimumHostVersion`
3. **Maximum Version:** Host version must be ≤ `MaximumHostVersion` (if specified)
4. **Strict Major Version:** If `MaximumHostVersion` is not specified, the plugin is assumed to only support the same major version as `MinimumHostVersion`
### Examples
```csharp
// Plugin works with host 1.0.0 through 2.x (explicit range)
[assembly: StellaPluginVersion("1.0.0", MinimumHostVersion = "1.0.0", MaximumHostVersion = "2.99.99")]
// Plugin works with host 2.x only (strict - no MaximumHostVersion means same major version)
[assembly: StellaPluginVersion("1.0.0", MinimumHostVersion = "2.0.0")]
// Plugin version 3.0.0 with no host constraints (uses plugin major version as reference)
[assembly: StellaPluginVersion("3.0.0")]
```
---
## 15 Plugin Host Configuration (v2.0)
Configure the plugin loader with security-first defaults in `PluginHostOptions`:
```csharp
var options = new PluginHostOptions
{
// Version enforcement (all default to true for security)
HostVersion = new Version(2, 0, 0),
EnforceVersionCompatibility = true, // Reject incompatible plugins
RequireVersionAttribute = true, // Reject plugins without [StellaPluginVersion]
StrictMajorVersionCheck = true, // Reject plugins crossing major version boundaries
// Signature verification (opt-in, requires infrastructure)
EnforceSignatureVerification = true,
SignatureVerifier = new CosignPluginVerifier(new CosignVerifierOptions
{
PublicKeyPath = "/keys/cosign.pub",
UseRekorTransparencyLog = true,
AllowUnsigned = false
})
};
var result = await PluginHost.LoadPluginsAsync(options, logger);
// Check for failures
if (result.HasFailures)
{
foreach (var failure in result.Failures)
{
logger.LogError("Plugin {Path} failed: {Reason} - {Message}",
failure.AssemblyPath, failure.Reason, failure.Message);
}
}
```
### Host Options Reference
| Option | Default | Purpose |
|--------|---------|---------|
| `HostVersion` | null | The host application version for compatibility checking |
| `EnforceVersionCompatibility` | **true** | Reject plugins that fail version checks |
| `RequireVersionAttribute` | **true** | Reject plugins without `[StellaPluginVersion]` |
| `StrictMajorVersionCheck` | **true** | Reject plugins that don't explicitly support the host's major version |
| `EnforceSignatureVerification` | false | Reject plugins without valid signatures |
| `SignatureVerifier` | null | The verifier implementation (e.g., `CosignPluginVerifier`) |
### Failure Reasons
| Reason | Description |
|--------|-------------|
| `LoadError` | Assembly could not be loaded (missing dependencies, corrupt file) |
| `SignatureInvalid` | Signature verification failed |
| `IncompatibleVersion` | Plugin version constraints not satisfied |
| `MissingVersionAttribute` | Plugin lacks required `[StellaPluginVersion]` attribute |
---
## 16 Fail-Fast Options Validation (v2.0)
Use the fail-fast validation pattern to catch configuration errors at startup:
```csharp
using StellaOps.DependencyInjection.Validation;
// Register options with automatic startup validation
services.AddOptionsWithValidation<MyPluginOptions, MyPluginOptionsValidator>(
MyPluginOptions.SectionName);
// Or with data annotations
services.AddOptionsWithDataAnnotations<MyPluginOptions>(
MyPluginOptions.SectionName);
```
Create validators using the base class:
```csharp
public sealed class MyPluginOptionsValidator : OptionsValidatorBase<MyPluginOptions>
{
protected override string SectionPrefix => "Plugins:MyPlugin";
protected override void ValidateOptions(MyPluginOptions options, ValidationContext context)
{
context
.RequireNotEmpty(options.BaseUrl, nameof(options.BaseUrl))
.RequirePositive(options.TimeoutSeconds, nameof(options.TimeoutSeconds))
.RequireInRange(options.MaxRetries, nameof(options.MaxRetries), 0, 10);
}
}
```
---
## 17 Available Templates (v2.0)
Install and use the official plugin templates:
```bash
# Install from local templates directory
dotnet new install ./templates
# Or install from NuGet
dotnet new install StellaOps.Templates
# Create a connector plugin
dotnet new stellaops-plugin-connector -n MyCompany.AcmeConnector
# Create a scheduled job plugin
dotnet new stellaops-plugin-scheduler -n MyCompany.CleanupJob
```
Templates include:
- Plugin entry point with version attribute
- Options class with data annotations
- Options validator with fail-fast pattern
- DI routine registration
- README with build/sign instructions
---
## 18 Migration Guide: v2.0 to v2.1
### Breaking Change: Version Attribute Required
As of v2.1, all plugins **must** include a `[StellaPluginVersion]` attribute. Plugins without this attribute will be rejected with `MissingVersionAttribute` failure.
**Before (v2.0):** Optional, plugins without attribute loaded with warning.
**After (v2.1):** Required, plugins without attribute are rejected.
### Migration Steps
1. Add the version attribute to your plugin's AssemblyInfo.cs:
```csharp
[assembly: StellaPluginVersion("1.0.0", MinimumHostVersion = "2.0.0", MaximumHostVersion = "2.99.99")]
```
2. If your plugin must support multiple major host versions, explicitly set `MaximumHostVersion`:
```csharp
// Supports host 1.x through 3.x
[assembly: StellaPluginVersion("1.0.0", MinimumHostVersion = "1.0.0", MaximumHostVersion = "3.99.99")]
```
3. Rebuild and re-sign your plugin.
### Opt-out (Not Recommended)
If you must load legacy plugins without version attributes:
```csharp
var options = new PluginHostOptions
{
RequireVersionAttribute = false, // Allow unversioned plugins (NOT recommended)
StrictMajorVersionCheck = false // Allow cross-major version loading
};
```
---
## Change Log
| Version | Date | Changes |
|---------|------|---------|
| v2.1 | 2025-12-14 | **Breaking:** `[StellaPluginVersion]` attribute now required by default. Added `RequireVersionAttribute`, `StrictMajorVersionCheck` options. Added `MissingVersionAttribute` failure reason. |
| v2.0 | 2025-12-14 | Added StellaPluginVersion attribute, Cosign verification options, fail-fast validation, new templates |
| v1.5 | 2025-07-11 | Template install, no hot-reload, IoC conventions |
# Plugin SDK Guide
This guide explains how StellaOps loads, validates, and wires restart-time plugins. It is intentionally cross-cutting: module-specific plugin contracts (Authority identity providers, Concelier connectors, Scanner analyzers, CLI command modules, etc.) live in the corresponding module dossiers under `docs/modules/`.
## 1) What a "plugin" means in StellaOps
StellaOps uses plugins to extend behavior without losing:
- Determinism (stable ordering, stable identifiers, replayable outputs).
- Offline posture (no hidden outbound calls; explicit trust roots and caches).
- Security boundaries (no client-controlled identity injection; signed artifacts where enforced).
Most services load plugins at process start (restart-time). Hot-reload is not a goal: restart-time loading keeps memory and dependency isolation predictable.
## 2) Service plugin loading model (restart-time)
Service plugins are loaded by the `StellaOps.Plugin` library:
- Discovery occurs in a configured plugin directory.
- Assemblies are loaded in an isolated `AssemblyLoadContext`.
- A compatibility gate runs before DI registration:
- version attribute presence and host compatibility
- optional signature verification
### 2.1 Plugin directory and discovery patterns
Default behavior (from `StellaOps.Plugin.Hosting.PluginHostOptions`):
- Base directory: `AppContext.BaseDirectory` (unless overridden).
- Plugin directory:
- `<PrimaryPrefix>.PluginBinaries` when `PrimaryPrefix` is set, otherwise `PluginBinaries`.
- Discovery glob(s):
- `<prefix>.Plugin.*.dll` for each configured prefix.
Hosts may override the directory and/or add explicit `searchPatterns` in their config. Use module operations docs to see the authoritative configuration for a given service.
### 2.2 Deterministic ordering
The loader is deterministic:
- When no explicit order is configured, discovered plugin assemblies are sorted by filename (case-insensitive).
- When an explicit order is configured, that order is applied first and the remainder stays sorted.
## 3) Version compatibility requirements
Plugins should declare the assembly-level attribute:
```csharp
using StellaOps.Plugin.Versioning;
[assembly: StellaPluginVersion("1.2.3", MinimumHostVersion = "1.0.0")]
```
The host can enforce:
- Required attribute presence (`RequireVersionAttribute`).
- Compatibility bounds (`MinimumHostVersion` / `MaximumHostVersion`).
- Strict major compatibility when `MaximumHostVersion` is not set (`StrictMajorVersionCheck`).
## 4) Dependency injection wiring
StellaOps supports two DI registration mechanisms.
### 4.1 Simple bindings via `ServiceBindingAttribute`
Annotate implementations with `ServiceBindingAttribute`:
```csharp
using Microsoft.Extensions.DependencyInjection;
using StellaOps.DependencyInjection;
[ServiceBinding(typeof(IMyContract), ServiceLifetime.Singleton, RegisterAsSelf = true)]
public sealed class MyPluginService : IMyContract
{
}
```
### 4.2 Advanced wiring via `IDependencyInjectionRoutine`
For full control, include a concrete `IDependencyInjectionRoutine` implementation. The host discovers and runs all routines in loaded plugin assemblies:
```csharp
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using StellaOps.DependencyInjection;
public sealed class MyPluginDi : IDependencyInjectionRoutine
{
public IServiceCollection Register(IServiceCollection services, IConfiguration configuration)
{
services.AddSingleton<IMyContract, MyPluginService>();
return services;
}
}
```
## 5) Signing and verification (Cosign)
When enabled, the host can verify plugin assemblies using Cosign (`cosign verify-blob`). The signature file is expected adjacent to the assembly:
- `MyPlugin.dll`
- `MyPlugin.dll.sig`
Verification is performed by `StellaOps.Plugin.Security.CosignPluginVerifier` and controlled by host configuration (for example `EnforceSignatureVerification` plus verifier options).
Offline note: verification can be performed without transparency log access when the host is configured accordingly (for example by ignoring tlog or using an offline receipt flow).
## 6) Repo layout (this monorepo)
In this repository, plugin binaries are typically staged under module-specific `*.PluginBinaries` directories (examples):
- `src/StellaOps.Authority.PluginBinaries/`
- `src/Concelier/StellaOps.Concelier.PluginBinaries/`
The authoritative loader configuration is owned by the host module and documented in its operations/architecture docs.
## 7) Testing expectations
Plugins should ship tests that protect determinism and compatibility:
- Stable ordering of outputs and collections.
- Stable timestamps (UTC ISO-8601).
- Fixture-backed inputs for offline operation.
- Compatibility checks for host version boundaries.
Reference tests for the generic plugin host live under:
- `src/__Libraries/__Tests/StellaOps.Plugin.Tests/`
## 8) Where to go next
- Authority plugins and operations: `docs/modules/authority/`
- Concelier connectors and operations: `docs/modules/concelier/`
- Scanner analyzers and operations: `docs/modules/scanner/`
- CLI command modules: `docs/modules/cli/`

2
docs/11_AUTHORITY.md
View File

@@ -141,7 +141,7 @@ These registrations are provided as examples in `etc/authority.yaml.sample`. Clo
- **Audit surfaces.** On success, the metadata is copied into the access token (`stellaops:policy_reason`, `stellaops:policy_ticket`, `stellaops:policy_digest`, `stellaops:policy_operation`) and recorded in [`authority.password.grant`] audit events as `policy.*` properties.
- **Failure modes.** Missing/blank parameters, over-length values, or non-hex digests trigger `invalid_request` responses and `authority.policy_attestation_denied` audit tags. CLI/Console must bubble these errors to operators and provide retry UX.
- **CLI / Console UX.** The CLI stores attestation metadata in `stella.toml` (`authority.policy.publishReason`, `authority.policy.publishTicket`) or accepts `STELLA_POLICY_REASON` / `STELLA_POLICY_TICKET` / `STELLA_POLICY_DIGEST` environment variables. Console prompts operators for the same trio before issuing attestation tokens and refuses to cache values longer than the session.
- **Automation guidance.** CI workflows should compute the policy digest ahead of time (for example `sha256sum policy-package.tgz | cut -d' ' -f1`) and inject the reason/ticket/digest into CLI environment variables immediately before invoking `stella auth login --scope policy:publish`.
- **Automation guidance.** CI workflows should compute the policy digest ahead of time (for example `sha256sum policy-package.tgz | cut -d' ' -f1`) and inject the reason/ticket/digest into CLI environment variables immediately before invoking `stella auth login` (using a profile configured to request `policy:publish`).
Graph Explorer introduces dedicated scopes: `graph:write` for Cartographer build jobs, `graph:read` for query/read operations, `graph:export` for long-running export downloads, and `graph:simulate` for what-if overlays. Assign only the scopes a client actually needs to preserve least privilege—UI-facing clients should typically request read/export access, while background services (Cartographer, Scheduler) require write privileges.

4
docs/21_INSTALL_GUIDE.md
View File

@@ -24,9 +24,7 @@ Verify:
docker compose --env-file dev.env -f docker-compose.dev.yaml ps
```
Defaults (from `deploy/compose/env/dev.env.example`):
- UI: `https://localhost:8443`
- Scanner API: `http://localhost:8444` (insecure; use the profile’s front door for TLS)
Defaults are defined by the selected env file. For the dev profile, the UI listens on `https://localhost:8443` by default; see `deploy/compose/env/dev.env.example` for the full port map.
## Air-gapped host (Compose profile)

4
docs/23_FAQ_MATRIX.md
View File

@@ -5,14 +5,14 @@
| Question | Short answer |
| --- | --- |
| What is StellaOps? | A sovereign, offline-first container-security platform focused on deterministic, replayable evidence: SBOMs, advisories, VEX, policy decisions, and attestations bound to image digests. |
| What makes it “deterministic”? | The same inputs produce the same outputs (stable ordering, stable IDs, replayable artifacts). Determinism is treated as a product feature and enforced by tests and fixtures. |
| What makes it "deterministic"? | The same inputs produce the same outputs (stable ordering, stable IDs, replayable artifacts). Determinism is treated as a product feature and enforced by tests and fixtures. |
| Does it run fully offline? | Yes. Offline operation is a first-class workflow (bundles, mirrors, importer/controller). See `docs/24_OFFLINE_KIT.md` and `docs/airgap/overview.md`. |
| Which formats are supported? | SBOMs: SPDX 3.0.1 and CycloneDX 1.6. VEX: OpenVEX-first decisioning with issuer trust and consensus. Attestations: in-toto/DSSE where enabled. |
| How do I deploy it? | Use deterministic bundles under `deploy/` (Compose/Helm) with digests sourced from `deploy/releases/`. Start with `docs/21_INSTALL_GUIDE.md`. |
| How do policy gates work? | Policy combines VEX-first inputs with lattice/precedence rules so outcomes are stable and explainable. See `docs/policy/vex-trust-model.md`. |
| Is multi-tenancy supported? | Yes; tenancy boundaries and roles/scopes are documented and designed to support regulated environments. See `docs/security/tenancy-overview.md` and `docs/security/scopes-and-roles.md`. |
| Can I extend it? | Yes: connectors, plugins, and policy packs are designed to be composable without losing determinism. Start with module dossiers under `docs/modules/`. |
| Where is the roadmap? | `docs/05_ROADMAP.md` (priority bands + definition of “done”). |
| Where is the roadmap? | `docs/05_ROADMAP.md` (priority bands + definition of "done"). |
| Where do I find deeper docs? | `docs/technical/README.md` is the detailed index; `docs/modules/` contains per-module dossiers. |
## Further reading

834
docs/DEVELOPER_ONBOARDING.md
View File

@@ -17,394 +17,44 @@
## Architecture Overview
StellaOps is a deterministic SBOM + VEX platform built as a microservices architecture with 36+ services organized into functional domains.
StellaOps is a deterministic, offline-first SBOM + VEX platform built as a microservice architecture. The system is designed so every verdict can be replayed from concrete evidence (SBOM slices, advisory/VEX observations, policy decision traces, and optional attestations).
**📖 For detailed component architecture with communication patterns, see [ARCHITECTURE_DETAILED.md](./ARCHITECTURE_DETAILED.md)**
### Canonical references
- Architecture overview (10-minute tour): `docs/40_ARCHITECTURE_OVERVIEW.md`
- High-level reference map: `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- Detailed architecture index: `docs/technical/architecture/README.md`
- Topology: `docs/technical/architecture/platform-topology.md`
- Infrastructure: `docs/technical/architecture/infrastructure-dependencies.md`
- Flows: `docs/technical/architecture/request-flows.md`
- Data isolation: `docs/technical/architecture/data-isolation.md`
- Security boundaries: `docs/technical/architecture/security-boundaries.md`
### Quick Reference - Component Topology
### Key architectural principles
```
CLIENT LAYER
├─ stella CLI → Gateway (JWT + DPoP auth)
├─ Web UI (Angular) → Gateway (JWT + DPoP auth)
├─ CI/CD Pipelines → Gateway (JWT + DPoP auth)
└─ Zastava Observer → Scanner (runtime scans)
1. **Deterministic evidence**: the same inputs produce the same outputs (stable ordering, stable IDs, replayable artifacts).
2. **VEX-first decisioning**: policy decisions are driven by VEX inputs and issuer trust, not enumeration alone.
3. **Offline-first**: fully air-gapped workflows are supported (mirrors, bundles, importer/controller).
4. **Extensibility without drift**: connectors, plugins, and policy packs must preserve determinism.
5. **Sovereign posture**: bring-your-own trust roots and configurable crypto profiles where enabled.
6. **Isolation boundaries**: clear module ownership, schema boundaries, and tenant scoping.
INFRASTRUCTURE (REQUIRED)
├─ PostgreSQL v16+ → Primary database (ALL services)
├─ Valkey v8.0 → Cache, DPoP, queues, events
└─ RustFS → Object storage (S3 API)
### Service categories (orientation)
INFRASTRUCTURE (OPTIONAL)
└─ NATS JetStream → Alternative messaging (Valkey is default)
| Category | Examples | Purpose |
| --- | --- | --- |
| Infrastructure | PostgreSQL, Valkey, RustFS/S3, optional message broker | Durable state, coordination, artifact storage, transport abstraction. |
| Auth & signing | Authority, Signer, Attestor, issuer trust services | Identity, scopes/tenancy, evidence signing and attestation workflows. |
| Ingestion | Concelier, Excititor | Advisory and VEX ingestion/normalization with deterministic merges. |
| Scanning | Scanner (API + workers) | Container analysis, SBOM generation, artifact production. |
| Policy & risk | Policy engine + explain traces | Deterministic verdicts, waivers/exceptions, explainability for audits. |
| Orchestration | Scheduler, Orchestrator | Re-scan orchestration, workflows, pack runs. |
| Notifications | Notification engine(s) | Event delivery and idempotent notifications. |
| User experience | Gateway, Web UI, CLI | Authenticated access, routing, operator workflows. |
GATEWAY LAYER
└─ Gateway.WebService → Auth, routing, rate limiting
AUTH & CRYPTO
├─ Authority → OAuth2/OIDC, OpTok issuance
├─ Signer → DSSE signing (FIPS/GOST/SM)
└─ Attestor → Rekor v2 transparency log
CORE ENGINES
├─ Scanner.WebService → Scan orchestration
├─ Scanner.Worker → Image analysis, SBOM generation
├─ Concelier.WebService → Advisory ingestion (NVD, Red Hat, etc.)
├─ Excititor.WebService → VEX ingestion + consensus
├─ Policy.Gateway → OPA/Rego policy evaluation
├─ Scheduler.WebService → Re-scan orchestration
├─ Notify.WebService → Notification orchestration
├─ Notify.Worker → Slack/Teams/Email delivery
└─ Orchestrator.WebService → DAG workflows, pack runs
SUPPORTING
└─ IssuerDirectory → VEX issuer trust registry
```
### Service Categories
| Category | Services | Purpose |
|----------|----------|---------|
| **Gateway** | Gateway.WebService | API routing, auth enforcement |
| **Auth & Security** | Authority, Signer, Attestor | OAuth2, signing, transparency |
| **Scanning** | Scanner.Web, Scanner.Worker | Container analysis, SBOM |
| **Advisory** | Concelier.Web, Concelier.Worker | Vulnerability ingestion |
| **VEX** | Excititor.Web, Excititor.Worker | Exploitability statements |
| **Policy** | Policy.Gateway, Policy Engine | OPA/Rego evaluation |
| **Orchestration** | Scheduler, Orchestrator | Job coordination |
| **Notifications** | Notify.Web, Notify.Worker | Delivery to Slack/Teams/Email |
### Runtime Topology - Infrastructure Dependencies
```
┌─────────────────────────────────────────────────────────────────────┐
│ INFRASTRUCTURE LAYER │
│ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │
│ │ PostgreSQL │ │ Valkey │ │ RustFS │ │
│ │ (v16+ ONLY) │ │ (Redis-compat) │ │ (S3-like API) │ │
│ │ │ │ - Caching │ │ - Artifacts │ │
│ │ All services use │ │ - DPoP nonces │ │ - SBOMs │ │
│ │ PostgreSQL for │ │ - Event queues │ │ - Signatures │ │
│ │ persistent data │ │ - Rate limiting│ │ │ │
│ └──────────────────┘ └──────────────────┘ └─────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Optional: NATS JetStream (alternative transport for queues) │ │
│ │ Only used if explicitly configured in appsettings │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┐
│ AUTHENTICATION & SIGNING │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Authority │─▶│ Signer │─▶│ Attestor │ │
│ │ (OAuth2/OIDC)│ │(DSSE/PKIX) │ │(in-toto/DSSE)│ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┐
│ INGESTION & AGGREGATION │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Concelier │ │ Excititor │ │IssuerDirectry│ │
│ │(Advisories) │ │ (VEX) │ │(CSAF Pubshrs)│ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┐
│ SCANNING & ANALYSIS │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │Scanner.Web │ │Scanner.Worker│ │ AdvisoryAI │ │
│ │(API/Control) │ │(Analyzers) │ │(ML Analysis) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ RiskEngine │ │ Policy │ │
│ │ (Scoring) │ │ (Engine) │ │
│ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┐
│ ORCHESTRATION & WORKFLOW │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Scheduler │ │ Orchestrator │ │ TaskRunner │ │
│ │(Job Sched) │ │(Coordinator) │ │(Executor) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┘
│ EVENTS & NOTIFICATIONS │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Notify │ │ Notifier │ │TimelineIndex │ │
│ │(Slack/Teams) │ │ (Advanced) │ │ (Events) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┐
│ DATA & EXPORT │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ExportCenter │ │EvidenceLocker│ │FindingsLedger│ │
│ │(SARIF/SBOM) │ │(Artifacts) │ │(Audit Trail) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
▲
│
┌─────────────────────────────────────────────────────────────────────┐
│ USER EXPERIENCE │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Gateway │ │ Web (UI) │ │ CLI │ │
│ │ (API Router) │ │ (Angular v17)│ │(Multi-plat) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
### Detailed Request Flow - Scan Execution Example
This diagram shows a complete scan request lifecycle with detailed routing through services:
```
┌──────────────────────────────────────────────────────────────────────────────────┐
│ 1. CLIENT REQUEST (CLI or Web UI) │
│ $ stella scan docker://alpine:latest --sbom-format=spdx │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│ HTTPS
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ 2. GATEWAY (API Router) │
│ - Terminates TLS │
│ - Routes to appropriate backend service │
│ - Load balancing (if multiple instances) │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│ HTTP (internal)
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ 3. AUTHORITY (Authentication) │
│ - Validates OAuth2 access token (DPoP-bound) │
│ - Checks DPoP proof against Valkey nonce cache │
│ - Returns user identity and scopes │
│ │
│ ┌─────────────┐ │
│ │ Valkey │◀── DPoP nonce validation (GET/SET) │
│ │ (Cache) │ │
│ └─────────────┘ │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── User/client lookup (SELECT) │
│ └─────────────┘ │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│ Authenticated request
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ 4. SCANNER.WEB (Scan API Controller) │
│ - Validates scan request parameters │
│ - Creates scan job record in PostgreSQL │
│ - Enqueues scan job to Valkey queue (default) or NATS (if configured) │
│ │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── INSERT scan_jobs (job_id, image_ref, status='pending') │
│ └─────────────┘ │
│ ┌─────────────┐ │
│ │ Valkey │◀── XADD scanner:jobs (enqueue job message) │
│ │ (Queue) │ │
│ └─────────────┘ │
│ │
│ Returns: HTTP 202 Accepted { "job_id": "scan-abc123", "status": "queued" } │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│
│ (Client polls for status)
│
┌──────────────────────────────────────────────────────────────────────────────────┐
│ 5. SCANNER.WORKER (Background Processor) │
│ - Consumes job from Valkey queue (XREADGROUP scanner:jobs) │
│ - Updates job status to 'running' │
│ - Downloads container image from registry │
│ - Executes analyzers (OS packages, language deps, files) │
│ - Generates SBOM (SPDX/CycloneDX) │
│ - Stores artifacts to RustFS │
│ │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── UPDATE scan_jobs SET status='running' │
│ │ │◀── INSERT sbom_documents, packages, vulnerabilities │
│ │ │◀── UPDATE scan_jobs SET status='completed' │
│ └─────────────┘ │
│ ┌─────────────┐ │
│ │ RustFS │◀── PUT /artifacts/scan-abc123/sbom.spdx.json │
│ │ (S3 API) │◀── PUT /artifacts/scan-abc123/image-layers.tar.gz │
│ └─────────────┘ │
│ ┌─────────────┐ │
│ │ Valkey │◀── XADD scanner:events (publish scan.completed event) │
│ │(Event Stream│ │
│ └─────────────┘ │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│ Event published
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ 6. EVENT PROPAGATION (Valkey Streams) │
│ │
│ ┌───────────────────────────────────────────────────────────────────┐ │
│ │ Valkey Event Stream: "scanner:events" │ │
│ │ Event: { "type": "scan.completed", "job_id": "scan-abc123", ... } │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌──────────────┼──────────────┬───────────────┐ │
│ ▼ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Notify │ │Timeline │ │ Policy │ │ Export │ │
│ │ Worker │ │ Indexer │ │ Engine │ │ Center │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │ │
│ │ (all subscribe to scanner:events via XREADGROUP) │
└─────────┼───────────────┼──────────────┼───────────────┼──────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────┐ ┌──────────────┐
│ 7a. NOTIFY │ │ 7b. TIMELINE │ │ 7c.POLICY│ │ 7d. EXPORT │
│ │ │ INDEXER │ │ ENGINE │ │ CENTER │
│ - Query scan │ │ │ │ │ │ │
│ results │ │ - Index event│ │ - Eval │ │ - Generate │
│ - Check user │ │ timeline │ │ policy │ │ SARIF │
│ notif prefs│ │ - Store in │ │ rules │ │ - Export to │
│ - Send Slack │ │ PostgreSQL │ │ - Block/ │ │ external │
│ message │ │ │ │ Allow │ │ systems │
│ │ │ │ │ │ │ │
│ PostgreSQL ◀─┤ │ PostgreSQL ◀─┤ │PostgreSQL│ │ RustFS ◀─┤
│ (user prefs) │ │ (timeline) │ │(policies)│ │ (exports) │
└──────────────┘ └──────────────┘ └──────────┘ └──────────────┘
```
### Export Flow - SBOM Distribution
```
┌──────────────────────────────────────────────────────────────────────────────────┐
│ EXPORT REQUEST: GET /api/v1/scans/{scan_id}/export?format=spdx │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ SCANNER.WEB or EXPORT CENTER │
│ │
│ 1. Query scan metadata from PostgreSQL │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── SELECT * FROM scan_jobs WHERE job_id = $1 │
│ │ │◀── SELECT * FROM sbom_documents WHERE scan_id = $1 │
│ └─────────────┘ │
│ │
│ 2. Retrieve SBOM artifact from RustFS │
│ ┌─────────────┐ │
│ │ RustFS │◀── GET /artifacts/scan-abc123/sbom.spdx.json │
│ └─────────────┘ │
│ │
│ 3. Sign SBOM with Signer service │
│ ┌─────────────┐ │
│ │ Signer │◀── POST /api/v1/sign (SBOM payload) │
│ │ │──▶ Returns: DSSE envelope with signature │
│ └─────────────┘ │
│ │
│ 4. Create in-toto attestation with Attestor │
│ ┌─────────────┐ │
│ │ Attestor │◀── POST /api/v1/attest (signed SBOM) │
│ │ │──▶ Returns: in-toto attestation bundle │
│ └─────────────┘ │
│ │
│ 5. Store final bundle to RustFS │
│ ┌─────────────┐ │
│ │ RustFS │◀── PUT /artifacts/scan-abc123/bundle.jsonl │
│ └─────────────┘ │
│ │
│ 6. Return signed bundle to client │
│ Returns: HTTP 200 OK (application/vnd.in-toto+json) │
└──────────────────────────────────────────────────────────────────────────────────┘
```
### Notification Flow - Vulnerability Alert
```
┌──────────────────────────────────────────────────────────────────────────────────┐
│ TRIGGER: New critical CVE detected in existing scan │
│ Source: Concelier advisory ingestion │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ CONCELIER.WORKER (Advisory Processor) │
│ │
│ 1. Ingest new advisory from NVD/OSV/CSAF │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── INSERT INTO advisories (cve_id, severity, ...) │
│ └─────────────┘ │
│ │
│ 2. Match advisory against existing scans (PURL/CPE matching) │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── SELECT scans WHERE package_purl IN (affected_purls) │
│ └─────────────┘ │
│ │
│ 3. Publish drift event to Valkey │
│ ┌─────────────┐ │
│ │ Valkey │◀── XADD concelier:drift (new vulnerability found) │
│ └─────────────┘ │
└───────────────────────────────────┬──────────────────────────────────────────────┘
│ Event published
▼
┌──────────────────────────────────────────────────────────────────────────────────┐
│ NOTIFY.WORKER (Notification Processor) │
│ │
│ 1. Consume drift event from Valkey stream │
│ ┌─────────────┐ │
│ │ Valkey │◀── XREADGROUP concelier:drift notify-workers │
│ └─────────────┘ │
│ │
│ 2. Query user notification preferences │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── SELECT * FROM user_notification_preferences │
│ │ │ WHERE user_id = scan_owner AND channel = 'slack' │
│ └─────────────┘ │
│ │
│ 3. Render notification template │
│ Template: "🚨 New critical CVE-2024-1234 affects alpine:latest scan" │
│ │
│ 4. Deliver notification via configured channels │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ External APIs │ │
│ │ - POST https://hooks.slack.com/services/T00/B00/xxx │ │
│ │ - POST https://graph.microsoft.com/v1.0/teams/channels │ │
│ │ - SMTP send (email) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ 5. Store delivery receipt in PostgreSQL │
│ ┌─────────────┐ │
│ │ PostgreSQL │◀── INSERT INTO notification_deliveries (status, ...) │
│ └─────────────┘ │
└──────────────────────────────────────────────────────────────────────────────────┘
```
### Key Architectural Principles
1. **Deterministic Evidence** - Same inputs always produce same outputs
2. **VEX-First Decisioning** - Policy decisions based on OpenVEX statements
3. **Offline-First** - Full air-gap operation supported
4. **Plugin Architecture** - Extensible connectors for advisories, analyzers, auth
5. **Sovereign Crypto** - FIPS, eIDAS, GOST, SM support
6. **Schema Isolation** - Per-module PostgreSQL schemas
### Service Categories
| Category | Services | Purpose |
|----------|----------|---------|
| **Infrastructure** | PostgreSQL v16+, Valkey 8.0, RustFS, NATS (optional) | Database, cache/messaging, object storage, optional queue transport |
| **Auth & Signing** | Authority, Signer, Attestor | OAuth2/OIDC with DPoP, cryptographic signing, in-toto attestations |
| **Ingestion** | Concelier, Excititor, IssuerDirectory | Advisory/VEX ingestion, normalization, merging, CSAF publisher discovery |
| **Scanning** | Scanner.Web, Scanner.Worker, AdvisoryAI | Container scanning, SBOM generation (SPDX/CDX), ML vulnerability analysis |
| **Policy & Risk** | Policy Engine, RiskEngine | OPA/Rego policy evaluation, risk scoring, exploitability assessment |
| **Orchestration** | Scheduler, Orchestrator, TaskRunner | Job scheduling, workflow coordination, distributed task execution |
| **Notifications** | Notify, Notifier, TimelineIndexer | Event delivery (Slack/Teams/Email), notification management, timeline tracking |
| **Data & Export** | ExportCenter, EvidenceLocker, FindingsLedger | SARIF/SBOM export, evidence storage, immutable audit trail |
| **User Experience** | Gateway, Web UI, CLI | API routing, Angular v17 UI, multi-platform command-line tools |
### Canonical flows
- Scan execution, ingestion updates, policy evaluation, and notification delivery are described in `docs/technical/architecture/request-flows.md`.
---
## Prerequisites
### Required Software
@@ -446,7 +96,7 @@ This diagram shows a complete scan request lifecycle with detailed routing throu
```bash
cd C:\dev\
git clone https://git.stella-ops.org/stella-ops.org/git.stella-ops.org
git clone https://git.stella-ops.org/stella-ops.org/git.stella-ops.org.git
cd git.stella-ops.org
```
@@ -462,40 +112,9 @@ notepad .env
```
**Key settings to configure:**
```bash
# PostgreSQL Database
POSTGRES_USER=stellaops
POSTGRES_PASSWORD=your_secure_password_here
POSTGRES_DB=stellaops_platform
POSTGRES_PORT=5432
# Valkey (Redis-compatible cache and messaging)
VALKEY_PORT=6379
# RustFS Object Storage
RUSTFS_HTTP_PORT=8080
# Service ports (adjust if conflicts exist)
AUTHORITY_PORT=8440
SIGNER_PORT=8441
ATTESTOR_PORT=8442
CONCELIER_PORT=8445
SCANNER_WEB_PORT=8444
NOTIFY_WEB_PORT=8446
# Scanner configuration (Valkey default, can switch to NATS if needed)
SCANNER_EVENTS_DRIVER=valkey
SCANNER_EVENTS_DSN=valkey:6379
# Scheduler configuration (Valkey default, can switch to NATS if needed)
SCHEDULER_QUEUE_KIND=Valkey
SCHEDULER_QUEUE_VALKEY_URL=valkey:6379
# Authority configuration
AUTHORITY_ISSUER=https://authority:8440
SIGNER_POE_INTROSPECT_URL=https://www.stella-ops.org/license/introspect
```
- Copy and edit the profile env file (`deploy/compose/env/dev.env.example` -> `.env`).
- Update at minimum `POSTGRES_PASSWORD` and any host port overrides needed for your machine.
- Treat `deploy/compose/env/*.env.example` as the authoritative list of variables for each profile (queue/transport knobs are profile-dependent).
### Step 3: Start the Full Platform
@@ -544,373 +163,36 @@ Open your browser and navigate to:
## Hybrid Debugging Workflow
The hybrid workflow allows you to:
1. Run infrastructure (databases, queues) in Docker
2. Run most services in Docker
3. **Selectively debug** one or two services in Visual Studio
Hybrid debugging runs the full platform in Docker, then stops one service container and runs that service locally under a debugger while it continues to use Docker-hosted dependencies.
### Workflow Overview
```
┌────────────────────────────────────────────────────────────┐
│ DOCKER ENVIRONMENT │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │PostgreSQL│ │ Valkey │ │ RustFS │ │
│ │ (DB) │ │(Cache/Msg│ │(Storage) │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Authority│ │ Signer │ │ Attestor │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ ┌──────────┐ ┌──────────┐ │
│ │Concelier │ │ Excititor│ ← Running normally │
│ └──────────┘ └──────────┘ │
└────────────────────────────────────────────────────────────┘
▲
│ HTTP calls + Valkey streams
▼
┌────────────────────────────────────────────────────────────┐
│ VISUAL STUDIO (F5) │
│ ┌────────────────────────────────────────────┐ │
│ │ Scanner.WebService │ │
│ │ Running on http://localhost:5210 │ │
│ │ (Breakpoints, hot reload, debugging) │ ← YOU DEBUG HERE
│ └────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────┘
```
### Step-by-Step: Debug Scanner.WebService
#### 1. Stop the Docker Container for Scanner
```bash
cd deploy\compose
docker compose -f docker-compose.dev.yaml stop scanner-web
```
**Verify it's stopped:**
```bash
docker compose -f docker-compose.dev.yaml ps scanner-web
# Should show: State = "exited"
```
#### 2. Configure Local Development Settings
Create or modify the service's `appsettings.Development.json`:
```bash
cd C:\dev\New folder\git.stella-ops.org\src\Scanner\StellaOps.Scanner.WebService
```
**Create `appsettings.Development.json`:**
```json
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"StellaOps": "Debug"
}
},
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password_here;Include Error Detail=true"
},
"Scanner": {
"Storage": {
"Mongo": {
"ConnectionString": "mongodb://stellaops:your_password_here@localhost:27017"
}
},
"ArtifactStore": {
"Driver": "rustfs",
"Endpoint": "http://localhost:8080/api/v1",
"Bucket": "scanner-artifacts",
"TimeoutSeconds": 30
},
"Queue": {
"Broker": "nats://localhost:4222"
},
"Events": {
"Enabled": false
}
},
"Authority": {
"Issuer": "https://localhost:8440",
"BaseUrl": "https://localhost:8440",
"BypassNetworks": ["127.0.0.1", "::1"]
}
}
```
**Note:** Adjust connection strings to match your Docker infrastructure ports. If PostgreSQL is on Docker's bridge network, you may need to expose it on `localhost:5432`.
#### 3. Expose Docker Services to localhost
For services running in Docker to be accessible from your host machine, ensure ports are mapped in `docker-compose.dev.yaml`:
```yaml
# Already configured in docker-compose.dev.yaml
postgres:
ports:
- "${POSTGRES_PORT:-5432}:5432"
mongo:
ports:
- "27017:27017"
nats:
ports:
- "${NATS_CLIENT_PORT:-4222}:4222"
rustfs:
ports:
- "${RUSTFS_HTTP_PORT:-8080}:8080"
```
**Verify connectivity:**
```bash
# Test PostgreSQL
psql -h localhost -U stellaops -d stellaops_platform
# Test NATS
telnet localhost 4222
# Test RustFS
curl http://localhost:8080/health
```
#### 4. Open Solution in Visual Studio
```bash
# Open the solution
cd C:\dev\New folder\git.stella-ops.org
start src\StellaOps.sln
```
**In Visual Studio:**
1. Right-click `StellaOps.Scanner.WebService` project
2. Select **"Set as Startup Project"**
3. Press **F5** to start debugging
**Expected output:**
```
info: Microsoft.Hosting.Lifetime[14]
Now listening on: http://localhost:5210
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
```
#### 5. Update Other Services to Call localhost
Since you're running Scanner.WebService on `localhost:5210` instead of `scanner-web:8444`, you need to update any services that call it.
**Option A: Environment Variables (Docker containers)**
Update `.env` file:
```bash
# Use host.docker.internal to reach host machine from Docker
SCANNER_WEB_BASEURL=http://host.docker.internal:5210
```
Restart dependent services:
```bash
docker compose -f docker-compose.dev.yaml restart scheduler-web
```
**Option B: Modify docker-compose.dev.yaml**
```yaml
scheduler-web:
environment:
SCHEDULER__WORKER__RUNNER__SCANNER__BASEADDRESS: "http://host.docker.internal:5210"
```
Then restart:
```bash
docker compose -f docker-compose.dev.yaml up -d scheduler-web
```
#### 6. Set Breakpoints and Debug
1. Navigate to `Program.cs` in Scanner.WebService
2. Set a breakpoint on a line in a controller or service method
3. Trigger the endpoint using:
- Swagger UI (if enabled): http://localhost:5210/swagger
- Postman/curl
- CLI command
**Example curl:**
```bash
curl -X POST http://localhost:5210/api/scans \
-H "Content-Type: application/json" \
-d '{"imageRef": "alpine:latest"}'
```
Your breakpoint should hit, and you can step through code.
#### 7. Return to Docker Mode
When you're done debugging:
```bash
# Stop Visual Studio debugger (Shift+F5)
# Restart the Docker container
cd deploy\compose
docker compose -f docker-compose.dev.yaml start scanner-web
# Verify it's running
docker compose -f docker-compose.dev.yaml ps scanner-web
```
---
Canonical guide:
- `docs/QUICKSTART_HYBRID_DEBUG.md`
Related references:
- Compose profiles: `deploy/compose/README.md`
- Install guide: `docs/21_INSTALL_GUIDE.md`
- Service-specific runbooks: `docs/modules/<module>/operations/`
## Service-by-Service Debugging Guide
### Authority (OAuth2/OIDC Provider)
Service-specific debugging guidance lives with each module to avoid stale, copy-pasted configuration examples.
**Project:** `src/Authority/StellaOps.Authority/StellaOps.Authority.csproj`
Generic workflow:
1. Stop the service container in `deploy/compose` (for example: `docker compose -f docker-compose.dev.yaml stop <service>`).
2. Run the service locally under a debugger.
3. Update dependent services to call `host.docker.internal:<port>` (or your host IP) and restart them.
4. Use the module operations docs for required env vars, auth scopes, and health checks.
**Stop Docker container:**
```bash
docker compose -f docker-compose.dev.yaml stop authority
```
Start here:
- Hybrid debugging walkthrough: `docs/QUICKSTART_HYBRID_DEBUG.md`
- Architecture index: `docs/technical/architecture/README.md`
- Module dossiers and operations: `docs/modules/`
**Configuration:** `appsettings.Development.json`
```json
{
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password"
},
"StellaOps_Authority": {
"Issuer": "https://localhost:5001",
"Mongo": {
"ConnectionString": "mongodb://stellaops:your_password@localhost:27017"
}
},
"Kestrel": {
"Endpoints": {
"Https": {
"Url": "https://localhost:5001"
}
}
}
}
```
**Run in Visual Studio:** F5 on `StellaOps.Authority` project
**Default URL:** https://localhost:5001
**Update dependent services:**
```bash
# In .env
AUTHORITY_ISSUER=https://host.docker.internal:5001
```
### Concelier (Advisory Ingestion)
**Project:** `src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj`
**Stop Docker:**
```bash
docker compose -f docker-compose.dev.yaml stop concelier
```
**Configuration:** `appsettings.Development.json`
```json
{
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password"
},
"Concelier": {
"Storage": {
"Mongo": {
"ConnectionString": "mongodb://stellaops:your_password@localhost:27017"
},
"S3": {
"Endpoint": "http://localhost:9000",
"AccessKeyId": "stellaops",
"SecretAccessKey": "your_password"
}
},
"Authority": {
"BaseUrl": "https://localhost:8440"
}
}
}
```
**Run:** F5 on `StellaOps.Concelier.WebService`
**Default URL:** http://localhost:5000
### Scanner.Worker (Background Analyzer)
**Project:** `src/Scanner/StellaOps.Scanner.Worker/StellaOps.Scanner.Worker.csproj`
**Stop Docker:**
```bash
docker compose -f docker-compose.dev.yaml stop scanner-worker
```
**Configuration:** Same as Scanner.WebService (shares settings)
**Run:** F5 on `StellaOps.Scanner.Worker`
**Note:** Worker has no HTTP endpoint - it consumes from NATS queue
### Scheduler.WebService
**Project:** `src/Scheduler/StellaOps.Scheduler.WebService/StellaOps.Scheduler.WebService.csproj`
**Stop Docker:**
```bash
docker compose -f docker-compose.dev.yaml stop scheduler-web
```
**Configuration:** `appsettings.Development.json`
```json
{
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_orchestrator;Username=stellaops;Password=your_password"
},
"Scheduler": {
"Queue": {
"Kind": "Nats",
"Nats": {
"Url": "nats://localhost:4222"
}
},
"Worker": {
"Runner": {
"Scanner": {
"BaseAddress": "http://localhost:5210"
}
}
}
}
}
```
**Run:** F5 on `StellaOps.Scheduler.WebService`
### Notify.WebService
**Project:** `src/Notify/StellaOps.Notify.WebService/StellaOps.Notify.WebService.csproj`
**Stop Docker:**
```bash
docker compose -f docker-compose.dev.yaml stop notify-web
```
**Configuration:** Uses `etc/notify.dev.yaml`
**Run:** F5 on `StellaOps.Notify.WebService`
---
Common module runbooks:
- Authority: `docs/modules/authority/operations/`
- Scanner: `docs/modules/scanner/operations/`
- Concelier: `docs/modules/concelier/operations/`
- Scheduler: `docs/modules/scheduler/operations/`
- UI / Console: `docs/modules/ui/`
## Configuration Deep Dive
@@ -1419,7 +701,7 @@ sudo docker compose -f docker-compose.dev.yaml up -d
3. **Week 3: Authentication & Security**
- Master OAuth2/OIDC flow in Authority
- Understand signing flow (Signer → Attestor → Rekor)
- Understand signing flow (Signer -> Attestor -> Rekor)
- Study policy evaluation engine
4. **Week 4: Integration**

4
docs/README.md
View File

@@ -16,11 +16,13 @@ This documentation set is internal and does not keep compatibility stubs for old
| Understand the product in 2 minutes | `overview.md` |
| Run a first scan (CLI) | `quickstart.md` |
| Browse capabilities | `key-features.md` |
| Roadmap (priorities + definition of “done”) | `05_ROADMAP.md` |
| Roadmap (priorities + definition of "done") | `05_ROADMAP.md` |
| Architecture: high-level overview | `40_ARCHITECTURE_OVERVIEW.md` |
| Architecture: full reference map | `07_HIGH_LEVEL_ARCHITECTURE.md` |
| Offline / air-gap operations | `24_OFFLINE_KIT.md` |
| Security deployment hardening | `17_SECURITY_HARDENING_GUIDE.md` |
| Ingest advisories (Concelier + CLI) | `10_CONCELIER_CLI_QUICKSTART.md` |
| Develop plugins/connectors | `10_PLUGIN_SDK_GUIDE.md` |
| Console (Web UI) operator guide | `15_UI_GUIDE.md` |
| VEX consensus and issuer trust | `16_VEX_CONSENSUS_GUIDE.md` |
| Vulnerability Explorer guide | `20_VULNERABILITY_EXPLORER_GUIDE.md` |

2
docs/api/console/exception-schema.md
View File

@@ -7,7 +7,7 @@
- Deterministic pagination (`continuationToken`), stable sorting, and explicit audit trails.
## Headers
- `X-StellaOps-Tenant` (required)
- `X-Stella-Tenant` (required; legacy alias: `X-StellaOps-Tenant`)
- `X-Stella-Project` (optional)
- `X-Stella-Trace-Id` (required)
- `X-Stella-Request-Id` (required; defaults to trace ID)

97
docs/api/console/search-downloads.md
View File

@@ -1,58 +1,57 @@
# Console Search & Downloads · Draft v0.2
# Console Search and Downloads
Scope: unblock WEB-CONSOLE-23-004/005 by defining deterministic ranking, caching rules, and the download manifest structure (including signed metadata option) for console search and offline bundle downloads. Final guild sign-off still required.
This document defines deterministic ranking, caching rules, and the download manifest shape used by Console search and export-style workflows.
## 1) Deterministic search ranking
- Primary sort: `severity (desc)` → `exploitScore (desc)` → `reachability (reachable > unknown > unreachable)` → `policyBadge (fail > warn > pass > waived)` → `vexState (under_investigation > fixed > not_affected > unknown)` → `findingId (asc)`.
- Secondary tie-breakers (when above fields absent): `advisoryId (asc)` then `product (asc)`.
- All pages are pre-sorted server-side; clients MUST NOT re-order.
- Primary sort: `severity (desc)` then `exploitScore (desc)` then `reachability (reachable > unknown > unreachable)` then `policyBadge (fail > warn > pass > waived)` then `vexState (under_investigation > fixed > not_affected > unknown)` then `findingId (asc)`.
- Secondary tie-breakers (when primary fields are absent): `advisoryId (asc)` then `product (asc)`.
- Pages are pre-sorted server-side; clients MUST NOT re-order results.
## 2) Caching + freshness
- Response headers: `Cache-Control: public, max-age=300, stale-while-revalidate=60, stale-if-error=300`.
- `ETag` is a stable SHA-256 over the sorted payload; clients send `If-None-Match` for revalidation.
- `Last-Modified` reflects the newest `updatedAt` in the result set.
- Retry/backoff guidance: honor `Retry-After` when present; default client backoff `1s,2s,4s,8s` capped at 30s.
- Deterministic page cursors: opaque base64url, signed; include `sortKeys` and `tenant` to avoid cross-tenant reuse.
## 2) Caching and freshness
- Response headers (recommended defaults): `Cache-Control: private, max-age=300, stale-while-revalidate=60, stale-if-error=300`.
- `ETag` SHOULD be a stable SHA-256 over a canonicalized, sorted payload so identical inputs produce identical validators; clients send `If-None-Match` for revalidation.
- `Last-Modified` SHOULD reflect the newest `updatedAt` present in the result set.
- Retry/backoff: honor `Retry-After` when present; otherwise use deterministic client backoff (for example `1s, 2s, 4s, 8s` capped at 30s).
- Page cursors: opaque base64url, signed; bind to `tenant` and effective sort keys to avoid cross-tenant or cross-sort reuse.
## 3) Download manifest (for `/console/downloads` and export outputs)
Top-level:
```jsonc
{
"version": "2025-12-07",
"exportId": "console-export::tenant-default::2025-12-07::0009",
"tenantId": "tenant-default",
"generatedAt": "2025-12-07T10:15:00Z",
"items": [
{
"type": "vuln", // advisory|vex|policy|scan|chart|bundle
"id": "CVE-2024-12345",
"format": "json",
"url": "https://downloads.local/exports/0009/vuln/CVE-2024-12345.json?sig=...",
"sha256": "f1c5…",
"size": 18432
}
],
"checksums": {
"manifest": "sha256:8bbf…",
"bundle": "sha256:12ae…" // optional when a tar/zip bundle is produced
},
"expiresAt": "2025-12-14T10:15:00Z"
}
```
## 3) Download manifest
When a Console workflow produces downloadable artifacts (individual documents or a bundle), services can return a manifest describing:
- What items are available,
- Where to download them (signed URLs or gateway-relative links),
- How to verify them offline (checksums and optional signature metadata).
### 3.1 Signed metadata
- Optional DSSE envelope for `checksums.manifest`, using `sha256` digest and `application/json` payload type `stellaops.console.manifest`.
- Envelope is attached as `manifest.dsse` or provided via `Link: <...>; rel="alternate"; type="application/dsse+json"`.
- Signers: Authority-issued short-lived key scoped to `console:export`.
Top-level fields:
- `version` (string): manifest schema/version label used by the producer.
- `exportId` (string): stable export identifier (content-addressable or run-id style).
- `tenantId` (string): tenant scope for this export.
- `generatedAt` (RFC 3339 / ISO-8601 UTC): generation timestamp.
- `items[]` (array): downloadable artifacts.
- `checksums` (object): checksum(s) for the manifest itself and optional bundle artifact.
- `expiresAt` (RFC 3339 / ISO-8601 UTC): optional expiry for signed URLs.
### 3.2 Error handling
- Known error codes: `ERR_CONSOLE_DOWNLOAD_INVALID_CURSOR`, `ERR_CONSOLE_DOWNLOAD_EXPIRED`, `ERR_CONSOLE_DOWNLOAD_RATE_LIMIT`, `ERR_CONSOLE_DOWNLOAD_UNAVAILABLE`.
- On error, respond with deterministic JSON body including `requestId` and `retryAfterSeconds` when applicable.
Item fields:
- `type` (string): artifact type (for example `vuln`, `vex`, `policy`, `scan`, `bundle`).
- `id` (string): stable identifier within the type (CVE id, statement id, export id, etc).
- `format` (string): `json`, `ndjson`, `tar.gz`, etc.
- `url` (string): download URL (often signed).
- `sha256` (string): lowercase hex digest of the bytes at `url`.
- `size` (int): byte size.
## 4) Sample manifest
- `docs/api/console/samples/console-download-manifest.json` illustrates the exact shape above.
### Signed metadata (optional)
Producers MAY ship a DSSE envelope for the manifest checksum:
- Payload type: `stellaops.console.manifest`
- Media type: `application/json`
- Distribution: alongside the manifest (for example `manifest.dsse`) or via a link relation (for example `rel=\"alternate\"` with `type=\"application/dsse+json\"`).
## 4) Error handling
Known error codes for download/manifest flows:
- `ERR_CONSOLE_DOWNLOAD_INVALID_CURSOR`
- `ERR_CONSOLE_DOWNLOAD_EXPIRED`
- `ERR_CONSOLE_DOWNLOAD_RATE_LIMIT`
- `ERR_CONSOLE_DOWNLOAD_UNAVAILABLE`
On error, return the standard error envelope with deterministic fields (stable code/message, plus request/trace identifiers).
## 5) Samples and fixtures
- Download manifest example: `docs/api/console/samples/console-download-manifest.json`
## 5) Open items for guild sign-off
- Final TTL values for `max-age` and `stale-*`.
- Whether DSSE envelope is mandatory for sealed tenants.
- Maximum bundle size / item count caps (proposal: 1000 items, 500 MiB compressed per export).

313
docs/api/console/workspaces.md
View File

@@ -1,31 +1,29 @@
# Console Workspaces API
_Tracking: CONSOLE-VULN-29-001, CONSOLE-VEX-30-001, DOCS-AIAI-31-004_
## 1. Goals and scope
## 1. Goals & Scope
Console workspaces provide tenant-scoped, read-only aggregates for operators and automation:
The console workspaces provide read-only aggregates for Advisory AI operators:
- `/console/vuln/*` surfaces tenant-scoped findings annotated with policy verdicts, VEX justifications, Scheduler reachability signals, and Advisory AI rationale.
- `/console/vex/*` streams the underlying VEX statements, conflicts, and justification summaries (with SSE support for live updates).
- `/console/vuln/*` surfaces findings annotated with policy verdicts, VEX context, reachability signals, and evidence pointers.
- `/console/vex/*` streams underlying VEX statements and summaries (optionally via SSE).
All endpoints MUST:
1. Remain deterministic (stable sort keys, ISO-8601 UTC timestamps, stable identifiers).
2. Enforce tenant isolation for every request.
3. Be offline-friendly by supporting export flows and fixture-based operation in air-gapped environments.
1. Remain deterministic offline (stable sort keys, ISO-8601 UTC timestamps, hashed assets).
2. Operate with Authority-issued DPoP or mTLS client credentials that include `console:read` and either `vuln:read` or `vex:read`.
3. Respect tenant isolation – every request carries `X-StellaOps-Tenant`.
## 2. Shared Request/Response Conventions
## 2. Shared request/response conventions
| Requirement | Description |
| --- | --- |
| Headers | `Authorization: DPoP <token>`, `DPoP: <proof>`, `X-StellaOps-Tenant: <tenantId>`, `Accept: application/json` (or `text/event-stream` for SSE). |
| Pagination | Cursor-based via `pageToken`; defaults to 50 items, max 200. Cursors are opaque, base64url, signed. |
| Auth headers | `Authorization: Bearer <token>` and optional proof headers (e.g., `DPoP: <proof>`) depending on deployment profile. |
| Tenant header | Required. See `docs/api/gateway/tenant-auth.md`; prefer `X-Stella-Tenant` (legacy alias: `X-StellaOps-Tenant`). |
| Pagination | Cursor-based via `pageToken`; defaults to 50 items, max 200. Cursors are opaque, base64url, and signed. |
| Sorting | Findings sorted by `(severity desc, exploitScore desc, findingId asc)`. Statements sorted by `(lastUpdated desc, statementId asc)`. |
| Dates | RFC 3339 / ISO-8601 UTC (e.g., `2025-11-08T12:02:11Z`). |
| Determinism | All arrays must be pre-sorted; no server-generated uuids in responses. |
| Dates | RFC 3339 / ISO-8601 UTC (e.g., `2025-01-02T03:04:05Z`). |
| Determinism | Arrays are pre-sorted; no server-generated random UUIDs in responses. |
## 3. Vulnerability Workspace (`/console/vuln/*`)
## 3. Vulnerability workspace (`/console/vuln/*`)
### 3.1 `GET /console/vuln/findings`
@@ -35,14 +33,14 @@ Query parameters:
| --- | --- | --- |
| `pageToken` | string | Optional cursor from previous response. |
| `pageSize` | int | 1-200, default 50. |
| `severity` | string[] | Accepts `critical`, `high`, `medium`, `low`, `info`. |
| `severity` | string[] | `critical`, `high`, `medium`, `low`, `info`. |
| `product` | string[] | SBOM `purl` or image digest anchors. |
| `policyBadge` | string[] | `pass`, `warn`, `fail`, `waived`. |
| `vexState` | string[] | `not_affected`, `fixed`, `under_investigation`, etc. |
| `reachability` | string[] | `reachable`, `unreachable`, `unknown`. |
| `search` | string | Substring match on CVE/GHSA/KEV ID (case-insensitive). |
| `search` | string | Substring match on CVE/GHSA/KEV id (case-insensitive). |
Response body:
Response body (example):
```jsonc
{
@@ -63,21 +61,21 @@ Response body:
"vex": {
"statementId": "vex:tenant-default:jwt-auth:5d1a",
"state": "under_investigation",
"justification": "Advisory AI flagged reachable path via Scheduler run 42."
"justification": "Operator triage pending."
},
"reachability": {
"status": "reachable",
"lastObserved": "2025-11-07T23:11:04Z",
"signalsVersion": "signals-2025.310.1"
"lastObserved": "2025-01-02T03:04:05Z",
"signalsVersion": "signals-<version>"
},
"evidence": {
"sbomDigest": "sha256:6c81…",
"policyRunId": "policy-run::2025-11-07::ca9f",
"attestationId": "dsse://authority/attest/84a2"
"sbomDigest": "sha256:6c81deadbeef...",
"policyRunId": "policy-run::<id>",
"attestationId": "dsse://authority/attest/<id>"
},
"timestamps": {
"firstSeen": "2025-10-31T04:22:18Z",
"lastSeen": "2025-11-07T23:16:51Z"
"firstSeen": "2025-01-01T00:00:00Z",
"lastSeen": "2025-01-02T03:05:06Z"
}
}
],
@@ -102,77 +100,17 @@ Response body:
```
### 3.2 `GET /console/vuln/facets`
Returns the full facet catalog (counts by severity, product, policy badge, VEX state, reachability, KEV flag). Designed for sidebar filters without paging; identical parameter surface as `/findings`.
### 3.3 `GET /console/vuln/{findingId}`
Returns the full finding document, including evidence timeline, policy overlays, and export-ready metadata:
```jsonc
{
"findingId": "tenant-default:advisory-ai:sha256:5d1a",
"details": {
"description": "jsonwebtoken <10.0.0 allows algorithm downgrade.",
"references": [
"https://nvd.nist.gov/vuln/detail/CVE-2024-12345",
"https://github.com/auth0/node-jsonwebtoken/security/advisories/GHSA-45mw-4jw3-g2wg"
],
"exploitAvailability": "known_exploit"
},
"policyBadges": [
{
"policyId": "policy://tenant-default/runtime-hardening",
"verdict": "fail",
"explainUrl": "https://console.local/policy/runs/policy-run::2025-11-07::ca9f"
}
],
"vex": {
"statementId": "vex:tenant-default:jwt-auth:5d1a",
"state": "under_investigation",
"justification": "Runtime telemetry confirmed exploitation path.",
"impactStatement": "Token exchange service remains exposed until patch 2025.11.2.",
"remediations": [
{
"type": "patch",
"description": "Upgrade jwt-auth-service to 2025.11.2.",
"deadline": "2025-11-12T00:00:00Z"
}
]
},
"reachability": {
"status": "reachable",
"callPathSamples": [
"api-gateway -> jwt-auth-service -> jsonwebtoken.verify"
],
"lastUpdated": "2025-11-07T23:11:04Z"
},
"evidence": {
"sbom": {
"digest": "sha256:6c81…",
"componentPath": [
"/src/jwt-auth/package.json",
"/src/jwt-auth/node_modules/jsonwebtoken"
]
},
"attestations": [
{
"type": "scan-report",
"attestationId": "dsse://authority/attest/84a2",
"signer": "attestor@stella-ops.org",
"bundleDigest": "sha256:e2bb…"
}
]
},
"timestamps": {
"firstSeen": "2025-10-31T04:22:18Z",
"lastSeen": "2025-11-07T23:16:51Z",
"vexLastUpdated": "2025-11-07T23:10:09Z"
}
}
```
Returns a full finding document, including evidence timeline, policy overlays, and export-ready metadata.
### 3.4 `POST /console/vuln/tickets`
Create ticket/export payloads in a deterministic, auditable way.
```jsonc
POST /console/vuln/tickets
{
@@ -189,37 +127,7 @@ POST /console/vuln/tickets
}
```
Response:
```jsonc
{
"ticketId": "console-ticket::tenant-default::2025-11-08::00018",
"payload": {
"version": "2025-11-01",
"tenant": "tenant-default",
"findings": [
{ "findingId": "tenant-default:advisory-ai:sha256:5d1a", "severity": "high" },
{ "findingId": "tenant-default:advisory-ai:sha256:9bf4", "severity": "critical" }
],
"policyBadge": "fail",
"vexSummary": "2 reachable findings pending patch.",
"attachments": [
{
"type": "json",
"name": "console-ticket-20251108.json",
"digest": "sha256:1fdd…",
"contentType": "application/json",
"expiresAt": "2025-11-15T00:00:00Z"
}
]
},
"auditEventId": "console.ticket.export::2025-11-08::00018"
}
```
Requests emit `console.ticket.export` audit events (tenant, user, selection counts, target system).
## 4. VEX Workspace (`/console/vex/*`)
## 4. VEX workspace (`/console/vex/*`)
### 4.1 `GET /console/vex/statements`
@@ -243,17 +151,14 @@ Response (paged JSON):
"product": "registry.local/ops/auth:2025.10.0",
"status": "under_investigation",
"justification": "exploit_observed",
"lastUpdated": "2025-11-07T23:10:09Z",
"lastUpdated": "2025-01-02T03:04:05Z",
"source": {
"type": "advisory_ai",
"modelBuild": "aiai-console-2025-10-28",
"modelBuild": "aiai-console-<build>",
"confidence": 0.74
},
"links": [
{
"rel": "finding",
"href": "/console/vuln/findings/tenant-default:advisory-ai:sha256:5d1a"
}
{ "rel": "finding", "href": "/console/vuln/findings/tenant-default:advisory-ai:sha256:5d1a" }
]
}
],
@@ -261,150 +166,16 @@ Response (paged JSON):
}
```
When `Accept: text/event-stream`, the endpoint emits events (see §4.3) instead of paged JSON.
### 4.2 SSE streaming
### 4.2 `GET /console/vex/statements/{statementId}`
Some deployments expose live updates as SSE (`text/event-stream`). When enabled, stream payloads SHOULD be valid NDJSON and remain deterministic for a given event id.
Returns the canonical statement plus provenance extracts. SSE clients can call this endpoint when they need full bodies after receiving a summary event.
## 5. Samples and fixtures
### 4.3 `GET /console/vex/events` (SSE)
Deterministic samples live under `docs/api/console/samples/` and are used by documentation and offline validation:
- `docs/api/console/samples/vuln-findings-sample.json`
- `docs/api/console/samples/vex-statement-sse.ndjson`
- `docs/api/console/samples/console-export-manifest.json`
- `docs/api/console/samples/exception-schema-sample.json`
Streams live updates for VEX statements affecting the tenant:
- Event types: `statement.created`, `statement.updated`, `statement.deleted`, `statement.conflict`.
- Fields: `id`, `advisoryId`, `product`, `vexState`, `severityHint`, `policyBadge`, `conflictSummary`, `sequence`.
- Replay: Clients include `Last-Event-ID`; server resumes from sequence.
- Heartbeats every 15 seconds (`event: keepalive`, `data: {}`).
Example event payload:
```jsonc
event: statement.updated
data: {
"statementId": "vex:tenant-default:jwt-auth:5d1a",
"advisoryId": "CVE-2024-12345",
"product": "registry.local/ops/auth:2025.10.0",
"state": "fixed",
"justification": "solution_available",
"sequence": 4182,
"updatedAt": "2025-11-08T11:44:32Z"
}
```
## 5. Signals & Scheduler Integration
- Reachability data is materialized by Scheduler delta jobs (`SCHED-CONSOLE-23-001`). `/console/vuln/findings` should cache the most recent job ID and expose `signalsVersion`.
- VEX justification fields reference Excititor statement IDs; ensure the gateway checks Excititor availability and degrades gracefully (returns `state: unavailable` plus telemetry).
- Scheduler must publish `console.vuln.refresh` events whenever advisory/VEX deltas warrant workspace refresh; console SSE endpoint may piggyback on the same Redis/NATS channel.
## 6. Determinism & Offline Notes
1. All responses are compressible JSON; no CDN fonts/assets referenced.
2. SSE endpoints must tolerate sealed mode by operating on loopback addresses only.
3. `authority-sealed-ci.json` (see DEVOPS-AIRGAP-57-002) is the evidence Authority consumes before enabling these APIs for sealed tenants; configure `airGap.sealedMode` and set `requiresAirgapSealConfirmation: true` on the Console client so `/token` refuses requests until the sealed harness uploads a fresh, passing artefact. Console responses echo `sealed: true/false` flags for UI badges once Authority has confirmed the evidence.
## 7. Sample Payloads for Docs
- `docs/api/console/samples/vuln-findings-sample.json` – exported via `scripts/generate-console-samples.ts` (placeholder script to be added when backend lands).
- `docs/api/console/samples/vex-statement-sse.ndjson` – contains 5 chronological SSE events for screenshot reproduction.
> Until backend implementations ship, use the examples above to unblock DOCS-AIAI-31-004; replace them with live captures once the gateway endpoints are available in staging.
## Exports (draft contract v0.4 for sign-off)
### Routes
- `POST /console/exports` — start an evidence bundle export job.
- `GET /console/exports/{exportId}` — fetch job status, manifest link, and download locations.
- `GET /console/exports/{exportId}/events` — SSE stream of job progress (optional).
### Security / headers
- `Authorization: DPoP <token>`
- `DPoP: <proof>`
- `X-StellaOps-Tenant: <tenantId>`
- `Idempotency-Key: <uuid>` (recommended for POST)
- `Accept: application/json` (status) or `text/event-stream` (events)
- Required scopes: `console:read` AND `console:export` (proposal).
### Request body (POST)
```jsonc
{
"scope": { "tenantId": "t1", "projectId": "p1" },
"sources": [
{ "type": "advisory", "ids": ["CVE-2024-12345"] },
{ "type": "vex", "ids": ["vex:tenant-default:jwt-auth:5d1a"] }
],
"formats": ["json", "ndjson", "csv"],
"attestations": { "include": true, "sigstoreBundle": true, "dsse": true },
"notify": { "webhooks": ["https://hooks.local/export"], "email": ["secops@example.com"] },
"priority": "normal"
}
```
### Response: 202 Accepted
- `exportId`, `status: queued|running|succeeded|failed|expired`
- `estimateSeconds`, `retryAfter` (seconds)
- `links`: `{ status: url, events?: url }`
### Response: GET status
```jsonc
{
"exportId": "console-export::tenant-default::2025-12-06::0007",
"status": "running",
"estimateSeconds": 420,
"outputs": [
{
"type": "manifest",
"format": "json",
"url": "https://exports.local/tenant-default/0007/manifest.json?sig=...",
"sha256": "sha256:c0ffee...",
"dsseUrl": "https://exports.local/tenant-default/0007/manifest.dsse?sig=...",
"expiresAt": "2025-12-06T13:10:00Z"
}
],
"progress": { "percent": 42, "itemsCompleted": 210, "itemsTotal": 500, "assetsReady": 12 },
"errors": []
}
```
### Response: SSE events
- `started`: `{ exportId, status }`
- `progress`: `{ exportId, percent, itemsCompleted, itemsTotal }`
- `asset_ready`: `{ exportId, type, id, url, sha256, format }`
- `completed`: `{ exportId, status: "succeeded", manifestUrl, manifestDsseUrl? }`
- `failed`: `{ exportId, status: "failed", code, message, retryAfterSeconds? }`
### Manifest shape (downloaded via outputs)
- Ordering: sort items by `(type asc, id asc, format asc, url asc)`.
- `version`: string (date), `exportId`, `tenantId`, `generatedAt`, `expiresAt`
- `items[]`: `{ type: advisory|vex|policy|scan|chart|bundle, id, format, url, sha256, size }`
- `checksums`: `{ manifest: "sha256:<digest>", bundle?: "sha256:<digest>" }`
- Optional DSSE envelope for manifest: `manifest.dsse` (payload type `stellaops.console.manifest`).
### Limits (proposed)
- Max request body 256 KiB; max sources 50; max outputs 1000 assets/export.
- Max bundle size 500 MiB compressed.
- Default job timeout 30 minutes; idle SSE timeout 60s; backoff via `Retry-After`.
### Determinism, caching, retry
- Responses set `Cache-Control: public, max-age=300, stale-while-revalidate=60, stale-if-error=300`.
- `ETag` is SHA-256 over sorted payload; clients send `If-None-Match`.
- Respect `Retry-After`; client backoff `1s,2s,4s,8s` capped at 30s.
- Cursors (if introduced later) MUST be opaque, base64url, signed with tenant + sortKeys.
### Error codes (proposal)
- `ERR_CONSOLE_EXPORT_INVALID_SOURCE`
- `ERR_CONSOLE_EXPORT_TOO_LARGE`
- `ERR_CONSOLE_EXPORT_RATE_LIMIT`
- `ERR_CONSOLE_EXPORT_UNAVAILABLE`
- `ERR_CONSOLE_EXPORT_EXPIRED`
### Samples
- Request: `docs/api/console/samples/console-export-request.json`
- Status: `docs/api/console/samples/console-export-status.json`
- Manifest: `docs/api/console/samples/console-export-manifest.json`
- Events: `docs/api/console/samples/console-export-events.ndjson`
### Open items (needs guild sign-off)
- Final scopes list (`console:export` vs broader `console:*`).
- Final limits and error codes; checksum manifest format; attestation options.
- Caching/tie-break rules for downstream `/console/search` and `/console/downloads`.
When contracts change, update the fixtures in lockstep and keep diffs deterministic (stable key ordering, stable timestamps, stable ids).

2
docs/api/exceptions.md
View File

@@ -22,7 +22,7 @@ This document is the entry point for exception contracts. Concrete shapes live i
Common requirements across endpoints:
- `Authorization: Bearer <token>` (or DPoP where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- Tenant header (required): prefer `X-Stella-Tenant` (legacy alias: `X-StellaOps-Tenant`); see `docs/api/gateway/tenant-auth.md`.
Scopes vary by deployment, but typically follow:

2
docs/api/gateway/advisories.md
View File

@@ -4,7 +4,7 @@ Scope: proxy Advisory surfaces through the Web gateway with tenant scoping, dete
## Security / headers
- `Authorization: Bearer <token>` (or `DPoP` where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project: <projectId>` (optional)
- `X-Stella-Trace-Id: <traceId>` (optional; clients SHOULD send one)
- Scopes: `advisory:read`

2
docs/api/gateway/advisory-ai.md
View File

@@ -7,7 +7,7 @@
- Web clients must send tenant + trace headers and should avoid sending raw prompts in telemetry; use prompt hashes instead.
## Headers
- `X-StellaOps-Tenant` (required)
- `X-Stella-Tenant` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project` (optional)
- `X-Stella-Trace-Id` (required)
- `X-Stella-Request-Id` (required; defaults to trace ID)

2
docs/api/gateway/exception-events.md
View File

@@ -4,7 +4,7 @@ Scope: stream exception workflow events (`exception.*`) for Console activity fee
## Security / headers
- `Authorization: Bearer <token>` (or `DPoP` where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project: <projectId>` (optional)
- Scopes: `exception:read`

2
docs/api/gateway/export-center.md
View File

@@ -4,7 +4,7 @@ Scope: proxy Export Center APIs through the Web gateway with tenant scoping, det
## Security / headers
- `Authorization: DPoP <token>`, `DPoP: <proof>`
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-StellaOps-Project: <projectId>` (optional)
- `Idempotency-Key: <uuid>` (recommended for POST)
- `Accept: application/json` (or `text/event-stream` for SSE)

2
docs/api/gateway/orchestrator.md
View File

@@ -6,7 +6,7 @@ This is an interim contract until the gateway is aligned to the Orchestrator Ope
## Security / headers
- `Authorization: Bearer <token>` (or `DPoP` where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project: <projectId>` (optional)
- `X-Stella-Trace-Id: <traceId>` (optional; clients SHOULD send one)
- Scopes: `orch:read`, `orch:quota`, `orch:operate`, `orch:backfill` (endpoint-specific)

2
docs/api/gateway/policy-evidence.md
View File

@@ -9,7 +9,7 @@ This contract is intended to reduce UI round-trips by composing existing gateway
## Security / headers
- `Authorization: Bearer <token>` (or `DPoP` where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project: <projectId>` (optional)
- `X-Stella-Trace-Id: <traceId>` (optional; clients SHOULD send one)
- Scopes:

2
docs/api/gateway/policy-exceptions.md
View File

@@ -4,7 +4,7 @@ Scope: expose policy evaluation results that include exception metadata, plus si
## Security / headers
- `Authorization: Bearer <token>` (or `DPoP` where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project: <projectId>` (optional)
- `X-Stella-Trace-Id: <traceId>` (optional; clients SHOULD send one)
- Scopes:

56
docs/api/gateway/tenant-auth.md
View File

@@ -1,11 +1,26 @@
# Gateway Tenant Auth & ABAC Contract (Web V)
## Status
- Final v1.0 (2025-12-01); aligns with Policy Guild checkpoint for Sprint 0216.
- v1.1 (2025-12-24); updated for identity header hardening (Sprint 8100.0011.0002).
- v1.0 (2025-12-01); aligns with Policy Guild checkpoint for Sprint 0216.
## Header Naming Migration
As of Sprint 8100.0011.0002, the Gateway uses canonical `X-StellaOps-*` headers:
- `X-StellaOps-Tenant`, `X-StellaOps-Project`, `X-StellaOps-Actor`, `X-StellaOps-Scopes`
Legacy `X-Stella-*` headers are emitted when `Gateway:Auth:EnableLegacyHeaders=true` (default) for backward compatibility. Clients should migrate to `X-StellaOps-*` headers.
## Security: Identity Header Hardening (v1.1)
**CRITICAL:** As of Sprint 8100.0011.0002, the Gateway enforces a **strip-and-overwrite** policy for identity headers:
1. All reserved identity headers (`X-StellaOps-*`, `X-Stella-*`, `sub`, `tid`, `scope`, `scp`, `cnf`) are **stripped** from incoming requests.
2. Downstream identity headers are **overwritten** from validated JWT claims—clients cannot spoof identity.
3. For anonymous requests, explicit `anonymous` identity is set to prevent ambiguity.
This replaces the legacy "set-if-missing" behavior which allowed header spoofing.
## Decisions (2025-12-01)
- Proof-of-possession: DPoP is **optional** for Web V. If a `DPoP` header is present the gateway verifies it; interactive clients SHOULD send DPoP, service tokens MAY omit it. A cluster flag `Gateway:Auth:RequireDpopForInteractive` can make DPoP mandatory later without changing the contract.
- Scope override header: `X-Stella-Scopes` is accepted only in pre-prod/offline bundles or when `Gateway:Auth:AllowScopeHeader=true`; otherwise the request is rejected with `ERR_SCOPE_HEADER_FORBIDDEN`.
- Scope override header: `X-StellaOps-Scopes` (or legacy `X-Stella-Scopes`) is forbidden by default; accepted only when `Gateway:Auth:AllowScopeHeader=true` for offline/pre-prod bundles. Violation returns `ERR_SCOPE_HEADER_FORBIDDEN`.
- ABAC overlay: evaluated on every tenant-scoped route after RBAC success; failures are hard denies (no fallback). Attribute sources are frozen for Web V as listed below to keep determism.
## Scope
@@ -16,25 +31,38 @@
## Header & Claim Inputs
| Name | Required | Notes |
| --- | --- | --- |
| `Authorization: Bearer <jwt>` | Yes | RS256/ES256; claims: `iss`, `sub`, `aud`, `exp`, `iat`, `nbf`, `jti`, optional `scp` (space-delimited), `ten` (tenant). DPoP proof verified when `DPoP` header present. |
| `Authorization: Bearer <jwt>` | Yes | RS256/ES256; claims: `iss`, `sub`, `aud`, `exp`, `iat`, `nbf`, `jti`, optional `scp`/`scope`, `stellaops:tenant`. DPoP proof verified when `DPoP` header present. |
| `DPoP` | Cond. | Proof-of-possession JWS for interactive clients; validated against `htm`/`htu` and access token `jti`. Ignored for service tokens when absent. |
| `X-Stella-Tenant` | Yes | Tenant slug/UUID; must equal `ten` claim when provided. Missing or mismatch → `ERR_TENANT_MISMATCH` (400). |
| `X-Stella-Project` | Cond. | Required for project-scoped routes; otherwise optional. |
| `X-Stella-Scopes` | Cond. | Only honored when `Gateway:Auth:AllowScopeHeader=true`; rejected with 403 otherwise. Value is space-delimited scopes. |
| `X-Stella-Trace-Id` | Optional | If absent the gateway issues a ULID trace id and propagates downstream. |
| `X-StellaOps-Trace-Id` | Optional | If absent the gateway issues a ULID trace id and propagates downstream. Legacy: `X-Stella-Trace-Id`. |
| `X-Request-Id` | Optional | Echoed for idempotency diagnostics and response envelopes. |
### Reserved Identity Headers (Client-Provided Values Ignored)
The following headers are **stripped** from client requests and **overwritten** from validated claims:
| Header | Source Claim | Notes |
| --- | --- | --- |
| `X-StellaOps-Tenant` | `stellaops:tenant` (fallback: `tid`) | Tenant identifier from token claims. |
| `X-StellaOps-Project` | `stellaops:project` | Project identifier (optional). |
| `X-StellaOps-Actor` | `sub` | Subject/actor from token claims. |
| `X-StellaOps-Scopes` | `scp` (individual) or `scope` (space-separated) | Scopes from token claims, sorted deterministically. |
Legacy `X-Stella-*` headers are emitted alongside canonical headers when `EnableLegacyHeaders=true`.
## Processing Rules
1) Validate JWT signature against offline bundle trust roots; `aud` must be one of `stellaops-web` or `stellaops-gateway`; reject on `exp/nbf` drift > 60s.
2) Resolve tenant: prefer `X-Stella-Tenant`, otherwise `ten` claim. Any mismatch → `ERR_TENANT_MISMATCH` (400).
3) Resolve project: from `X-Stella-Project` when route is project-scoped; otherwise null.
4) Build scope set: start from `scp` claim; if `X-Stella-Scopes` is allowed and present, replace the set with its value.
5) RBAC: check required scopes per route (matrix below). Missing scope → `ERR_SCOPE_MISMATCH` (403).
6) ABAC overlay:
1) **Strip reserved headers:** Remove all reserved identity headers from the incoming request (see table above).
2) **Validate JWT:** Verify signature against offline bundle trust roots; `aud` must be one of `stellaops-web` or `stellaops-gateway`; reject on `exp/nbf` drift > 60s.
3) **Extract identity from claims:**
- Tenant: `stellaops:tenant` claim, fallback to `tid` claim.
- Project: `stellaops:project` claim (optional).
- Actor: `sub` claim.
- Scopes: `scp` claims (individual items) or `scope` claim (space-separated).
4) **Write downstream headers:** Overwrite `X-StellaOps-*` headers from extracted claims. Legacy `X-Stella-*` headers written when enabled.
5) **Anonymous handling:** If unauthenticated and `AllowAnonymous=true`, set explicit anonymous identity (`X-StellaOps-Actor: anonymous`, empty scopes).
6) **RBAC:** Check required scopes per route (matrix below). Missing scope → `ERR_SCOPE_MISMATCH` (403).
7) **ABAC overlay:**
- Attributes: `subject`, `roles`, `org`, `tenant_id`, `project_id`, route vars (e.g., `finding_id`, `policy_id`), and request body keys explicitly listed in the route contract.
- Order: RBAC allow → ABAC evaluate → deny overrides → allow.
- Fail closed: on evaluation error or missing attributes return `ERR_ABAC_DENY` (403) with `reason` + `trace_id`.
7) Determinism: tenant header is mandatory; anonymous/implicit tenants are not allowed. Error codes are stable and surfaced in the response envelope.
8) **Determinism:** Identity headers are always overwritten from claims; error codes are stable and surfaced in the response envelope.
## Route Scope Matrix (Web V)
- `/risk/*` → `risk:read` for GET, `risk:write` for POST/PUT; severity events additionally require `notify:emit`.

2
docs/api/gateway/vex-evidence.md
View File

@@ -4,7 +4,7 @@ Scope: expose read-only VEX statement and evidence routes through the Web gatewa
## Security / headers
- `Authorization: Bearer <token>` (or `DPoP` where configured)
- `X-StellaOps-Tenant: <tenantId>` (required)
- `X-Stella-Tenant: <tenantId>` (required; see `docs/api/gateway/tenant-auth.md`)
- `X-Stella-Project: <projectId>` (optional)
- `X-Stella-Trace-Id: <traceId>` (optional; clients SHOULD send one)
- Scopes:

70
docs/api/graph-gateway-spec-draft.yaml
View File

@@ -26,22 +26,6 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/SearchRequest'
responses:
'200':
description: Stream of search tiles (NDJSON)
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/TileEnvelope'
examples:
sample:
summary: Node + cursor tiles
value: |
{"type":"node","seq":0,"data":{"id":"gn:tenant:component:abc","kind":"component","tenant":"acme","attributes":{"purl":"pkg:npm/lodash@4.17.21"}},"cost":{"limit":1000,"remaining":999,"consumed":1}}
{"type":"cursor","seq":1,"data":{"token":"cursor-123","resumeUrl":"https://gateway.local/api/graph/query?cursor=cursor-123"}}
'400': { $ref: '#/components/responses/ValidationError' }
'401': { $ref: '#/components/responses/Unauthorized' }
'429': { $ref: '#/components/responses/BudgetExceeded' }
responses:
'200':
description: Stream of search tiles (NDJSON)
@@ -64,6 +48,9 @@ paths:
description: Seconds until next request is allowed when rate limited.
schema:
type: integer
'400': { $ref: '#/components/responses/ValidationError' }
'401': { $ref: '#/components/responses/Unauthorized' }
'429': { $ref: '#/components/responses/BudgetExceeded' }
/graph/query:
post:
@@ -79,23 +66,6 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/QueryRequest'
responses:
'200':
description: Stream of query tiles (NDJSON)
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/TileEnvelope'
examples:
mixedTiles:
summary: Node + edge + stats tiles
value: |
{"type":"node","seq":0,"data":{"id":"gn:tenant:artifact:sha256:...","tenant":"acme","kind":"artifact","attributes":{"sbom_digest":"sha256:abc"}}}
{"type":"edge","seq":1,"data":{"id":"ge:tenant:CONTAINS:...","sourceId":"gn:tenant:artifact:...","targetId":"gn:tenant:component:...","kind":"CONTAINS"}}
{"type":"stats","seq":2,"data":{"nodesEmitted":1,"edgesEmitted":1,"depthReached":2,"cacheHitRatio":0.8}}
'400': { $ref: '#/components/responses/ValidationError' }
'401': { $ref: '#/components/responses/Unauthorized' }
'429': { $ref: '#/components/responses/BudgetExceeded' }
responses:
'200':
description: Stream of query tiles (NDJSON)
@@ -119,10 +89,13 @@ paths:
description: Seconds until next request is allowed when rate limited.
schema:
type: integer
'400': { $ref: '#/components/responses/ValidationError' }
'401': { $ref: '#/components/responses/Unauthorized' }
'429': { $ref: '#/components/responses/BudgetExceeded' }
/graph/paths:
post:
summary: Find constrained paths between node sets (depth ≤ 6)
summary: Find constrained paths between node sets (depth <= 6)
security:
- bearerAuth: []
parameters:
@@ -134,23 +107,6 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/PathsRequest'
responses:
'200':
description: Stream of path tiles ordered by hop
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/TileEnvelope'
examples:
pathTiles:
summary: Path tiles grouped by hop
value: |
{"type":"node","seq":0,"data":{"id":"gn:tenant:component:src","kind":"component","tenant":"acme"}}
{"type":"edge","seq":1,"data":{"id":"ge:tenant:DEPENDS_ON:1","sourceId":"gn:tenant:component:src","targetId":"gn:tenant:component:dst","kind":"DEPENDS_ON"}}
{"type":"stats","seq":2,"data":{"nodesEmitted":2,"edgesEmitted":1,"depthReached":1}}
'400': { $ref: '#/components/responses/ValidationError' }
'401': { $ref: '#/components/responses/Unauthorized' }
'429': { $ref: '#/components/responses/BudgetExceeded' }
responses:
'200':
description: Stream of path tiles ordered by hop
@@ -165,6 +121,18 @@ paths:
{"type":"node","seq":0,"data":{"id":"gn:tenant:component:src","kind":"component","tenant":"acme","attributes":{"purl":"pkg:npm/demo@1.0.0"},"pathHop":0}}
{"type":"edge","seq":1,"data":{"id":"ge:tenant:DEPENDS_ON:1","sourceId":"gn:tenant:component:src","targetId":"gn:tenant:component:dst","kind":"DEPENDS_ON","pathHop":1}}
{"type":"stats","seq":2,"data":{"nodesEmitted":2,"edgesEmitted":1,"depthReached":1}}
headers:
X-RateLimit-Remaining:
description: Remaining request budget within the window.
schema:
type: integer
Retry-After:
description: Seconds until next request is allowed when rate limited.
schema:
type: integer
'400': { $ref: '#/components/responses/ValidationError' }
'401': { $ref: '#/components/responses/Unauthorized' }
'429': { $ref: '#/components/responses/BudgetExceeded' }
/graph/diff:
post:

150
docs/api/graph.md
View File

@@ -1,123 +1,59 @@
# Graph API
# Graph Gateway API (draft)
Status: Draft (2025-11-26) — aligns with Sprint 0207 Graph API implementation (in-memory seed; RBAC + budgets enforced).
This document describes the Graph Gateway HTTP surface at a high level and points to the authoritative contracts.
Base URL: `<gateway>/api/graph` (examples use relative paths).
## Authoritative contracts
## Common headers
- `X-Stella-Tenant` (required)
- `Authorization: Bearer <token>` (required)
- `X-Stella-Scopes`: space/comma/semicolon separated scopes
- `/graph/search`: `graph:read` or `graph:query`
- `/graph/query|paths|diff`: `graph:query`
- `/graph/export`: `graph:export`
Content-Type for requests: `application/json`
Streaming responses: `application/x-ndjson`
- OpenAPI (endpoints, schemas, responses): `docs/api/graph-gateway-spec-draft.yaml`
- Overlay + tile cache schema (materialized/cached representation): `docs/api/graph/overlay-schema.md`
- Sample cached tile: `docs/api/graph/samples/overlay-sample.json`
## POST /graph/search
Returns NDJSON stream of tiles (`node`, optional `cursor`).
## What the Graph Gateway is for
Body:
```json
{
"kinds": ["component", "artifact"],
"query": "pkg:npm/",
"filters": { "ecosystem": "npm" },
"limit": 50,
"cursor": "opaque"
}
```
Errors:
- 400 `GRAPH_VALIDATION_FAILED` (missing kinds/query/filters)
- 401 missing auth; 403 missing scopes
Graph Gateway provides deterministic, tenant-scoped graph queries used for:
- Impact analysis and traversal (search/query/paths/diff)
- Overlay enrichment (policy, VEX, advisory) when requested
- Export workflows for offline analysis and audit
## POST /graph/query
Streams nodes, edges (optional), stats, cursor with cost metadata.
## Tenancy and authentication
Body:
```json
{
"kinds": ["component"],
"query": "widget",
"filters": { "tenant": "acme" },
"includeEdges": true,
"includeStats": true,
"includeOverlays": true,
"limit": 100,
"cursor": "opaque",
"budget": { "tiles": 6000, "nodes": 5000, "edges": 10000 }
}
```
Error tile if edge budget exceeded: `{ "type":"error","data":{"error":"GRAPH_BUDGET_EXCEEDED",...}}`
- Tenancy is mandatory for graph routes. The canonical tenant header name is deployment/profile dependent; see:
- `docs/api/gateway/tenant-auth.md`
- `docs/modules/gateway/identity-header-policy.md`
- Authentication is typically `Authorization: Bearer <token>` with scopes per route.
- Common scope families: `graph:read`, `graph:query`, `graph:export` (see OpenAPI for exact requirements).
## POST /graph/paths
Finds paths up to depth 6 between sources/targets.
## NDJSON tile streaming
Body:
```json
{
"sources": ["gn:acme:component:one"],
"targets": ["gn:acme:component:two"],
"kinds": ["depends_on"],
"maxDepth": 4,
"includeOverlays": false,
"budget": { "tiles": 2000, "nodes": 1500, "edges": 3000 }
}
```
Response: NDJSON tiles (`node`, `edge`, `stats`, `cursor`).
Graph endpoints stream `application/x-ndjson`, where each line is a `TileEnvelope`.
## POST /graph/diff
Diff two snapshots.
Key fields:
- `type`: `node`, `edge`, `stats`, `cursor`, `diagnostic`
- `seq`: monotonic integer per response stream
- `cost`: optional per-stream budget counters (`limit`, `remaining`, `consumed`)
- `data`: payload varies by `type`
Body:
```json
{
"snapshotA": "snapA",
"snapshotB": "snapB",
"includeEdges": true,
"includeStats": true,
"budget": { "tiles": 2000 }
}
```
Response tiles: `node`/`edge` with change types, `stats`, optional `cursor`.
## Cursor and resume
## POST /graph/export
Kicks off an in-memory export job and returns manifest.
When paging/resume is supported, responses can include a `cursor` tile with an opaque token.
Requests can pass a `cursor` field to resume from a prior stream.
Body:
```json
{
"format": "ndjson", // ndjson|csv|graphml|png|svg
"includeEdges": true,
"snapshotId": "snapA", // optional
"kinds": ["component"], // optional
"query": "pkg:", // optional
"filters": { "ecosystem": "npm" }
}
```
Response:
```json
{
"jobId": "job-123",
"status": "completed",
"format": "ndjson",
"sha256": "...",
"size": 1234,
"downloadUrl": "/graph/export/job-123",
"completedAt": "2025-11-26T00:00:00Z"
}
```
Download: `GET /graph/export/{jobId}` (returns file, headers include `X-Content-SHA256`).
## Budgets and rate limits
## Health
`GET /healthz` → 200 when service is ready; no auth/scopes required.
Budget/rate-limit enforcement is explicit and deterministic:
- Responses may include headers such as `X-RateLimit-Remaining` and `Retry-After`
- `429` responses use a stable error envelope (see OpenAPI `components.responses`)
## Error envelope
```json
{ "error": "GRAPH_VALIDATION_FAILED", "message": "details", "requestId": "optional" }
```
## Overlays
## Notes
- All timestamps UTC ISO-8601.
- Ordering is deterministic; cursors are opaque base64 offsets.
- Overlays use `policy.overlay.v1` and `openvex.v1` payloads. Budgeted tiles reserve room for cursors when `hasMore` is true.
`node` and `edge` tiles may include overlays when requested. Overlays are keyed by overlay kind and carry `{kind, version, data}` so clients can render and cache without guessing schema.
For the cache/materialized representation used by gateway/UI tile caches, see `docs/api/graph/overlay-schema.md`.
## Errors and determinism
- Error shape follows platform conventions (see `docs/api/overview.md`), plus graph-specific error codes in the OpenAPI contract.
- Determinism requirements:
- stable ordering for equivalent requests
- stable ids for nodes/edges where defined by schema
- UTC ISO-8601 timestamps

84
docs/api/graph/overlay-schema.md
View File

@@ -1,42 +1,82 @@
# Graph Overlay & Cache Schema (draft placeholder)
# Graph overlay and tile cache schema (draft)
**Status:** Draft v0.2 · owner-proposed
## Overview
## Scope
- Overlay/cache schema for graph tiles used by Web gateway and UI overlays.
- Validation rules for bbox/zoom/path; pagination tokens; deterministic ordering.
- Error codes and sampling/telemetry fields.
This document describes the cached/materialized tile representation used by gateway/UI components to store graph tiles alongside overlay data.
This cache schema is separate from the streaming NDJSON tile protocol:
- Streaming API contract: `docs/api/graph-gateway-spec-draft.yaml`
- Sample cached tile: `docs/api/graph/samples/overlay-sample.json`
## Cached tile shape
A cached tile document is a single JSON object with:
- `version`: cache schema version (string; bump only for breaking changes)
- `tenantId`: tenant partition for the cache entry
- `tile`: tile identity + spatial key (`id`, `bbox`, `zoom`) and cache validator (`etag`)
- `nodes`: node records
- `edges`: edge records
- `overlays`: overlay arrays keyed by overlay kind
- `telemetry`: generation/caching metadata
## Schema (draft)
```jsonc
{
"version": "2025-12-06",
"version": "0.2",
"tenantId": "tenant-default",
"tile": {
"id": "graph-tile::asset::<hash>::z8/x12/y5",
"id": "graph-tile::<scope>::<hash>::z8/x12/y5",
"bbox": { "minX": -122.41, "minY": 37.77, "maxX": -122.38, "maxY": 37.79 },
"zoom": 8,
"etag": "c0ffee-etag"
},
"nodes": [ { "id": "asset:...", "kind": "asset|component|vuln", "label": "", "severity": "high|medium|low|info", "reachability": "reachable|unreachable|unknown", "attributes": {} } ],
"edges": [ { "id": "edge-1", "source": "nodeId", "target": "nodeId", "type": "depends_on|contains|evidence", "weight": 0.0 } ],
"nodes": [
{
"id": "asset:...",
"kind": "asset|component|vuln",
"label": "optional display label",
"severity": "critical|high|medium|low|info",
"reachability": "reachable|unreachable|unknown",
"attributes": {}
}
],
"edges": [
{
"id": "edge-1",
"source": "nodeId",
"target": "nodeId",
"type": "depends_on|contains|evidence",
"weight": 0.0,
"attributes": {}
}
],
"overlays": {
"policy": [ { "nodeId": "nodeId", "badge": "pass|warn|fail|waived", "policyId": "", "verdictAt": "2025-12-05T09:00:00Z" } ],
"vex": [ { "nodeId": "nodeId", "state": "not_affected|fixed|under_investigation|affected", "statementId": "", "lastUpdated": "2025-12-05T09:10:00Z" } ],
"aoc": [ { "nodeId": "nodeId", "status": "pass|fail|warn", "lastVerified": "2025-12-05T10:11:12Z" } ]
"policy": [
{ "nodeId": "nodeId", "badge": "pass|warn|fail|waived", "policyId": "policy://...", "verdictAt": "2025-01-02T03:04:05Z" }
],
"vex": [
{ "nodeId": "nodeId", "state": "not_affected|fixed|under_investigation|affected", "statementId": "vex:...", "lastUpdated": "2025-01-02T03:04:05Z" }
],
"aoc": [
{ "nodeId": "nodeId", "status": "pass|fail|warn", "lastVerified": "2025-01-02T03:04:05Z" }
]
},
"telemetry": { "generationMs": 0, "cache": "hit|miss", "samples": 0 }
}
```
## Constraints (proposal)
- Max nodes per tile: 2,000; max edges: 4,000.
- Zoom range: 0–12; tiles must include bbox and etag.
- Arrays must be pre-sorted: nodes by `id`, edges by `id`, overlays by `nodeId` then `policyId|statementId`.
## Determinism rules
## Samples
- `docs/api/graph/samples/overlay-sample.json`
- Arrays are pre-sorted:
- `nodes` by `id`
- `edges` by `id`
- overlay arrays by `nodeId` then secondary key (`policyId`, `statementId`, etc.)
- Timestamps are ISO-8601 UTC.
- Hashes are lower-case hex.
## Outstanding
- Confirm max sizes, allowed edge types, and etag hashing rule.
- Provide validation error example and rate-limit headers for gateway responses.
## Constraints (draft)
- Max nodes per tile: 2,000
- Max edges per tile: 4,000
- Zoom range: 0-12

11
docs/api/graph/samples/overlay-sample.json
View File

@@ -1,5 +1,5 @@
{
"version": "2025-12-06",
"version": "0.2",
"tenantId": "tenant-default",
"tile": {
"id": "graph-tile::asset::sha256:abc123::z8/x12/y5",
@@ -19,7 +19,6 @@
"label": "app:1.2.3",
"severity": "high",
"reachability": "reachable",
"aoc": { "summary": "pass", "lastVerified": "2025-12-05T10:11:12Z" },
"attributes": {
"purl": "pkg:docker/app@sha256:abc123",
"componentCount": 42
@@ -48,22 +47,22 @@
"nodeId": "component:pkg:npm/jsonwebtoken@9.0.2",
"badge": "fail",
"policyId": "policy://tenant-default/runtime-hardening",
"verdictAt": "2025-12-05T09:00:00Z"
"verdictAt": "2025-01-02T03:04:05Z"
}
],
"vex": [
{
"nodeId": "component:pkg:npm/jsonwebtoken@9.0.2",
"state": "under_investigation",
"statementId": "vex:tenant-default:jwt:2025-12-05",
"lastUpdated": "2025-12-05T09:10:00Z"
"statementId": "vex:tenant-default:jwt:0001",
"lastUpdated": "2025-01-02T03:04:05Z"
}
],
"aoc": [
{
"nodeId": "asset:registry.local/library/app@sha256:abc123",
"status": "pass",
"lastVerified": "2025-12-05T10:11:12Z"
"lastVerified": "2025-01-02T03:04:05Z"
}
]
},

18
docs/api/scanner/README.md
View File

@@ -1,18 +0,0 @@
# Scanner API Docs — Windows/macOS Coverage (Draft)
This directory collects interim artefacts tracking customer demand and roadmap readiness for extending Scanner coverage to Windows and macOS.
## Files
- `windows-coverage.md` — narrative summary of customer signals and required artefacts.
- `windows-macos-summary.md` — dashboard-style snapshot (counts, cross-references) updated after each discovery cycle.
## Related resources
- `../../modules/scanner/design/README.md`
- `../../benchmarks/scanner/windows-macos-demand.md`
- `../../benchmarks/scanner/windows-macos-interview-template.md`
- `../../modules/scanner/design/macos-analyzer.md`
- `../../modules/scanner/design/windows-analyzer.md`
- `../../modules/policy/windows-package-readiness.md`
- `../../modules/policy/secret-leak-detection-readiness.md`
> Note: replace these working notes with formal API documentation once Windows/macOS analyzer endpoints are defined.

50
docs/api/scanner/windows-coverage.md
View File

@@ -1,50 +0,0 @@
# Scanner API — Windows/macOS Coverage Signals (Draft)
> Audience: Solutions engineers, product managers, guild leads coordinating Windows/macOS roadmap
> Status: Informational; update as interviews conclude
## Summary
| Region | Accounts referenced | Primary workloads | Demand strength (1-5) | Blocking? | Notes |
| --- | --- | --- | --- | --- | --- |
| North America | Northwind Health Services; FinSecure Corp | macOS CI runners; Windows Server 2019 workloads | 4-5 | macOS: evaluation; Windows: blocking | Demo 2025-11-10; Security decision due 2025-11-07. |
| EMEA | — | — | — | — | — |
| APAC | — | — | — | — | — |
| Gov / Regulated | — | — | — | — | — |
### Key drivers
- Customers with regulated Windows Server/desktop estates lack deterministic SBOM coverage and provenance.
- macOS development shops (Mobile, Gaming) require entitlements/notarization evidence for compliance.
- Offline/air-gapped environments need signed rule bundles and feed mirrors for Windows/macOS ecosystems.
### Competitive landscape
- Trivy/Grype/Snyk remain Linux-focused; Windows/macOS features are roadmap items or SaaS-only.
- Opportunity to differentiate via deterministic evidence, policy integration, and offline parity.
### Design references
- `../../modules/scanner/design/macos-analyzer.md`
- `../../modules/scanner/design/windows-analyzer.md`
### Action items
- Maintain region rows using interview summaries from `docs/benchmarks/scanner/windows-macos-demand.md` (last update 2025-11-03; capture via the interview template).
- Track readiness decisions by updating POLICY-READINESS-0001/0002 status and recording outcomes in the summary table.
- Align backlog references (`SCANNER-ENG-0020..0027`, `DOCS-SCANNER-BENCH-62-016`) with product prioritisation after each roadmap review.
### Open blockers
- FinSecure PCI audit pending POLICY-READINESS-0002 decision (due 2025-11-07); unblock Windows analyzer spike scheduling.
- Northwind macOS readiness workshop scheduled 2025-11-10; capture masking/telemetry decisions for POLICY-READINESS-0001.
## Interview log (selected)
| Date | Customer | Platform focus | Signal summary | Strength (1-5) | Follow-up |
| --- | --- | --- | --- | --- | --- |
| 2025-11-03 | Northwind Health Services | macOS | Needs notarization/entitlement visibility for CI runners | 4 | Demo 2025-11-10 with Product; feed findings into POLICY-READINESS-0001. |
| 2025-11-03 | FinSecure Corp | Windows | Requires MSI/WinSxS SBOM + signed driver attestations for PCI audit | 5 | Security guild to resolve Authenticode posture (POLICY-READINESS-0002) by 2025-11-07. |
## Required artefacts
- Maintain interview notes using `docs/benchmarks/scanner/windows-macos-interview-template.md`.
- Update demand tracker tables in `docs/benchmarks/scanner/windows-macos-demand.md`.
- Sync backlog entries in `docs/modules/scanner/TASKS.md` and `docs/scanner/design/*.md`.
## Next steps
1. Collect at least three qualified Windows and macOS requests; update summary table.
2. Present findings to Scanner Guild for prioritisation (target Sprint 133 design spike).
3. Coordinate policy readiness briefs (`docs/modules/policy/windows-package-readiness.md`) and design docs (`design/macos-analyzer.md`, `design/windows-analyzer.md`).

37
docs/api/scanner/windows-macos-summary.md
View File

@@ -1,37 +0,0 @@
# Scanner API — Windows/macOS Coverage Dashboard (Draft)
> Owners: Product Guild, Scanner Guild • Status: living document updated every sprint
## At-a-glance metrics (Sprint 132 intake)
- macOS demand entries logged: 1 (Northwind Health Services, 2025-11-03)
- Windows demand entries logged: 1 (FinSecure Corp, 2025-11-03)
- Qualified customers awaiting roadmap response: 1 (FinSecure PCI blocker)
- Open policy readiness items: POLICY-READINESS-0001, POLICY-READINESS-0002
## Cross-reference
| Resource | Purpose |
| --- | --- |
| docs/benchmarks/scanner/windows-macos-demand.md | Signal log & next actions |
| docs/benchmarks/scanner/windows-macos-interview-template.md | Interview capture template |
| docs/benchmarks/scanner/deep-dives/macos.md | macOS implementation roadmap |
| docs/benchmarks/scanner/deep-dives/windows.md | Windows implementation roadmap |
| docs/modules/scanner/design/macos-analyzer.md | Detailed macOS design |
| docs/modules/scanner/design/windows-analyzer.md | Detailed Windows design |
| docs/modules/policy/windows-package-readiness.md | Policy readiness for Windows packages |
| docs/modules/policy/secret-leak-detection-readiness.md | Policy readiness for secrets |
| docs/modules/scanner/TASKS.md | Engineering backlog (SCANNER-ENG-0020..0027) |
| docs/modules/policy/TASKS.md | Policy readiness tasks |
| docs/api/scanner/windows-coverage.md | Narrative summary |
## Maintenance cadence
- Update metrics and cross-links after each customer signal or roadmap checkpoint.
- Ensure DOCS-SCANNER-BENCH-62-002/016 status mirrors demand tracker progress.
## Upcoming milestones
- 2025-11-07: POLICY-READINESS-0002 Authenticode/feed decision for FinSecure (unblocks Windows analyzer spike).
- 2025-11-10: POLICY-READINESS-0001 workshop during Northwind demo to finalise masking/telemetry posture.
## Recent updates
- 2025-11-03: Logged Northwind Health Services (macOS) & FinSecure Corp (Windows); awaiting POLICY-READINESS-0001/0002 decisions before scheduling analyzer spikes.
Last updated: 2025-11-03 (initial demand entries logged).

29
docs/api/vuln.md
View File

@@ -1,29 +0,0 @@
# Vulnerability API (placeholder)
Status: Draft (2025-11-26) — awaiting Vuln Explorer v1 surface. This doc reserves the path and headers to align with upcoming releases.
## Base URL
`<gateway>/api/vuln` (subject to final routing via API gateway).
## Common headers
- `X-Stella-Tenant` (required)
- `Authorization: Bearer <token>`
- `X-Stella-Scopes`: expect `vuln:read` (TBD) and/or `graph:read` when graph-backed queries are invoked.
- `Content-Type: application/json`
## Planned endpoints (subject to change)
- `POST /vuln/search` — filter vulnerabilities by component (purl/digest), advisory id, status, exploitability (OpenVEX).
- `POST /vuln/impact` — compute impacted assets using Graph overlays; may proxy to Graph API internally.
- `GET /vuln/{id}` — details with references, VEX status, nearest safe version.
- `GET /vuln/{id}/evidence` — raw evidence (SBOM snapshot refs, observations).
- `GET /vuln/kev` — Known Exploited Vulnerabilities view (cached).
## Error envelope
Follows Graph/Platform standard:
```json
{ "error": "VULN_VALIDATION_FAILED", "message": "details", "requestId": "optional" }
```
## Notes
- This placeholder will be updated once Vuln Explorer API is finalized. Keep gateway clients tolerant to minor shape changes until status flips to READY.
- For current graph-backed queries, use `/graph/search` or `/graph/query` (see `docs/api/graph.md`).

2
docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md
View File

@@ -316,7 +316,7 @@
- **Milestones**: POLICY-READINESS-0002 Authenticode/feed decision due 2025-11-07 (FinSecure PCI blocker) gates Windows analyzer spikes.
### Implementation details
- **Design references**: `docs/benchmarks/scanner/deep-dives/windows.md`, `docs/modules/scanner/design/windows-analyzer.md`, `docs/api/scanner/windows-coverage.md`.
- **Design references**: `docs/benchmarks/scanner/deep-dives/windows.md`, `docs/modules/scanner/design/windows-analyzer.md`.
- **Policy readiness**: see `docs/modules/policy/windows-package-readiness.md` for predicate requirements, waiver model, and offline guidance.
- **Collector outline**:
- MSI/WinSxS collector to parse installer databases/manifests and correlate via file hashes.

1
docs/benchmarks/scanner/windows-macos-demand.md
View File

@@ -28,6 +28,5 @@
- If three or more qualified customers flag Windows/macOS coverage as a blocking requirement, open a design spike under the Scanner Analyzer Guild with scope/time estimates and Offline Kit considerations.
- Keep the macOS deep dive (`docs/benchmarks/scanner/deep-dives/macos.md`) in sync with demand findings so engineering can move from design sketch to formal backlog when thresholds are met.
- Update the Windows deep dive (`docs/benchmarks/scanner/deep-dives/windows.md`) and associated design briefs (`docs/modules/scanner/design/windows-analyzer.md`) as new signals arrive.
- Refresh API dashboards (`docs/api/scanner/windows-macos-summary.md`, `docs/api/scanner/windows-coverage.md`) after each update to keep Product and Field teams aligned.
- Drive POLICY-READINESS-0002 Authenticode/feed decision by 2025-11-07 (FinSecure PCI blocker); log outcome in dashboards and design briefs.
- Prepare POLICY-READINESS-0001 workshop aligned with Northwind demo on 2025-11-10, updating policy briefs with masking/telemetry decisions.

43
docs/concelier-connector-research-20251011.md
View File

@@ -1,43 +0,0 @@
# Concelier Connector Research – 2025-10-11
Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national/vendor connectors in scope. Use alongside each module’s `TASKS.md` notes.
## ACSC (Australia)
- Enumerated feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92). Forcing HTTP/1.1 hangs >600 s and sitemap/HTML fetches fail the same way.
- Next actions: prototype `SocketsHttpHandler` settings (`RequestVersionOrLower`, allow fallback to relay), capture successful headers from partner vantage (need retention + cache semantics), and keep `FEEDCONN-SHARED-HTTP2-001` open for downgrade work.
## CCCS (Canada)
- JSON endpoint (`https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=<lang>&content_type=cccs_threat`) returns ~5 100 records per language; `page=<n>` still works for segmented pulls and the earliest `date_created` seen is 2018‑06‑08 (EN) / 2018‑06‑08 (FR). Use an explicit `User-Agent` to avoid 403 responses.
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/modules/concelier/operations/connectors/cccs.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
## CERT-Bund (Germany)
- `https://wid.cert-bund.de/content/public/securityAdvisory/rss` responds 200 without cookies (≈250-item window, German taxonomy). Detail links load an Angular SPA that fetches JSON behind the bootstrap session.
- Confirmed `GET https://wid.cert-bund.de/portal/api/securityadvisory?name=<WID-SEC-…>` returns JSON once the portal cookie container is primed; payload includes severity, CVEs, products, and references used by the connector fixtures.
- Historical advisories accessible through the SPA search/export endpoints once the `XSRF-TOKEN` cookie (exposed via `GET /portal/api/security/csrf`) is supplied with the `X-XSRF-TOKEN` header:
- `POST /portal/api/securityadvisory/search` (`{"page":N,"size":100,"sort":["published,desc"]}`) pages data back to 2014.
- `GET /portal/api/securityadvisory/export?format=json&from=YYYY-MM-DD` emits JSON bundles suitable for Offline Kit mirrors.
- Locale note: content is German-only; Concelier preserves `language=de` and Docs will publish a CERT-Bund glossary so operators can bridge terminology without machine translation.
## KISA / KNVD (Korea)
- `https://knvd.krcert.or.kr/rss/securityInfo.do` and `/rss/securityNotice.do` return UTF-8 RSS (10-item window) with `detailDos.do?IDX=` links. No cookies required for feed fetch.
- Detail SPA calls resolve to `rssDetailData.do?IDX=` JSON payloads; connector fetches those directly, sanitises HTML, and records Hangul metadata (NFC). See `docs/dev/kisa_connector_notes.md` for telemetry + localisation guidance.
## BDU (Russia / FSTEC)
- Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml/json`) return 403/404; TLS chain requires Russian Trusted Sub CA and WAF expects additional headers.
- Next actions: acquire official PEM chain, point `concelier:httpClients:source.bdu:trustedRootPaths` (or `concelier:sources:bdu:http:trustedRootPaths`) at the Offline Kit PEM, keep `allowInvalidCertificates=false`, script session bootstrap, then capture RSS/HTML schema for parser work.
## NKTsKI / cert.gov.ru (Russia)
- `https://cert.gov.ru/rss/advisories.xml` served via Bitrix returns 403/404 even with `Accept-Language: ru-RU`; TLS chain also requires Russian trust anchors.
- Next actions: source trust store, configure `concelier:httpClients:source.nkcki:trustedRootPaths` (Offline Kit root via `concelier:offline:root`), prepare proxy fallback, and once accessible document taxonomy/retention plus attachment handling.
## CISA ICS (United States)
- `curl -I https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` returns HTTP 403 + `x-reference-error` (Akamai). Same for legacy feed paths.
- Next actions: secure GovDelivery access, document token rotation, and build HTML/email fallback with throttling.
## Cisco PSIRT
- `https://api.cisco.com/security/advisories/latest` returns `ERR_596_SERVICE_NOT_FOUND` when unauthenticated. openVuln REST requires Mashery OAuth (client credentials) with quotas ~5 req/s, 30/min, 5 000/day; supports `pageIndex/pageSize` pagination.
- Next actions: register OAuth app, capture pagination/delta parameters, and compare API vs RSS coverage.
## Microsoft MSRC
- REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) requires Azure AD token + `api-version` (current `2024-08-01`) and supports delta filters (`lastModifiedStartDateTime`). CVRF ZIP remains available for offline use.
- Next actions: finalise AAD app registration, implement token cache, and design combined REST+CVRF ingestion path for determinism.

220
docs/dev/30_EXCITOR_CONNECTOR_GUIDE.md
View File

@@ -1,220 +0,0 @@
# Excitor Connector Packaging Guide
> **Audience:** teams implementing new Excitor provider plug‑ins (CSAF feeds,
> OpenVEX attestations, etc.)
> **Prerequisites:** read `docs/modules/excitor/architecture.md` and the module
> `AGENTS.md` in `src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/`.
The Excitor connector SDK gives you:
- `VexConnectorBase` – deterministic logging, SHA‑256 helpers, time provider.
- `VexConnectorOptionsBinder` – strongly typed YAML/JSON configuration binding.
- `IVexConnectorOptionsValidator<T>` – custom validation hooks (offline defaults, auth invariants).
- `VexConnectorDescriptor` & metadata helpers for consistent telemetry.
This guide explains how to package a connector so the Excitor Worker/WebService
can load it via the plugin host.
---
## 1. Project layout
Start from the template under
`docs/dev/templates/excitor-connector/`. It contains:
```
Excitor.MyConnector/
├── src/
│ ├── Excitor.MyConnector.csproj
│ ├── MyConnectorOptions.cs
│ ├── MyConnector.cs
│ └── MyConnectorPlugin.cs
└── manifest/
└── connector.manifest.yaml
```
Key points:
- Target `net10.0`, enable `TreatWarningsAsErrors`, reference the
`StellaOps.Excitor.Connectors.Abstractions` project (or NuGet once published).
- Keep project ID prefix `StellaOps.Excitor.Connectors.<Provider>` so the
plugin loader can discover it with the default search pattern.
### 1.1 csproj snippet
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\StellaOps.Excitor.Connectors.Abstractions\StellaOps.Excitor.Connectors.Abstractions.csproj" />
</ItemGroup>
</Project>
```
Adjust the `ProjectReference` for your checkout (or switch to a NuGet package
once published).
---
## 2. Implement the connector
1. **Options model** – create an options POCO with data-annotation attributes.
Bind it via `VexConnectorOptionsBinder.Bind<TOptions>` in your connector
constructor or `ValidateAsync`.
2. **Validator** – implement `IVexConnectorOptionsValidator<TOptions>` to add
complex checks (e.g., ensure both `clientId` and `clientSecret` are present).
3. **Connector** – inherit from `VexConnectorBase`. Implement:
- `ValidateAsync` – run binder/validators, log configuration summary.
- `FetchAsync` – stream raw documents to `context.RawSink`.
- `NormalizeAsync` – convert raw documents into `VexClaimBatch` via
format-specific normalizers (`context.Normalizers`).
4. **Plugin adapter** – expose the connector via a plugin entry point so the
host can instantiate it.
### 2.1 Options binding example
```csharp
public sealed class MyConnectorOptions
{
[Required]
[Url]
public string CatalogUri { get; set; } = default!;
[Required]
public string ApiKey { get; set; } = default!;
[Range(1, 64)]
public int MaxParallelRequests { get; set; } = 4;
}
public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions>
{
public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors)
{
if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
{
errors.Add("CatalogUri must use HTTPS.");
}
}
}
```
Bind inside the connector:
```csharp
private readonly MyConnectorOptions _options;
public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider)
: base(descriptor, logger, timeProvider)
{
// `settings` comes from the orchestrator; validators registered via DI.
_options = VexConnectorOptionsBinder.Bind<MyConnectorOptions>(
descriptor,
VexConnectorSettings.Empty,
validators: new[] { new MyConnectorOptionsValidator() });
}
```
Replace `VexConnectorSettings.Empty` with the actual settings from context
inside `ValidateAsync`.
---
## 3. Plugin adapter & manifest
Create a simple plugin class that implements
`StellaOps.Plugin.IConnectorPlugin`. The Worker/WebService plugin host uses
this contract today.
```csharp
public sealed class MyConnectorPlugin : IConnectorPlugin
{
private static readonly VexConnectorDescriptor Descriptor =
new("excitor:my-provider", VexProviderKind.Vendor, "My Provider VEX");
public string Name => Descriptor.DisplayName;
public bool IsAvailable(IServiceProvider services) => true; // inject feature flags if needed
public IFeedConnector Create(IServiceProvider services)
{
var logger = services.GetRequiredService<ILogger<MyConnector>>();
var timeProvider = services.GetRequiredService<TimeProvider>();
return new MyConnector(Descriptor, logger, timeProvider);
}
}
```
> **Note:** the Excitor Worker currently instantiates connectors through the
> shared `IConnectorPlugin` contract. Once a dedicated Excitor plugin interface
> lands you simply swap the base interface; the descriptor/connector code
> remains unchanged.
Provide a manifest describing the assembly for operational tooling:
```yaml
# manifest/connector.manifest.yaml
id: excitor-my-provider
assembly: StellaOps.Excitor.Connectors.MyProvider.dll
entryPoint: StellaOps.Excitor.Connectors.MyProvider.MyConnectorPlugin
description: >
Official VEX feed for ExampleCorp products (CSAF JSON, daily updates).
tags:
- excitor
- csaf
- vendor
```
Store manifests under `/opt/stella/excitor/plugins/<connector>/manifest/` in
production so the deployment tooling can inventory and verify plug‑ins.
---
## 4. Packaging workflow
1. `dotnet publish -c Release` → copy the published DLLs to
`/opt/stella/excitor/plugins/<Provider>/`.
2. Place `connector.manifest.yaml` next to the binaries.
3. Restart the Excitor Worker or WebService (hot reload not supported yet).
4. Verify logs: `VEX-ConnectorLoader` should list the connector descriptor.
### 4.1 Offline kits
- Add the connector folder (binaries + manifest) to the Offline Kit bundle.
- Include a `settings.sample.yaml` demonstrating offline-friendly defaults.
- Document any external dependencies (e.g., SHA mirrors) in the manifest `notes`
field.
---
## 5. Testing checklist
- Unit tests around options binding & validators.
- Integration tests (future `StellaOps.Excitor.Connectors.Abstractions.Tests`)
verifying deterministic logging scopes:
`logger.BeginScope` should produce `vex.connector.id`, `vex.connector.kind`,
and `vex.connector.operation`.
- Deterministic SHA tests: repeated `CreateRawDocument` calls with identical
content must return the same digest.
---
## 6. Reference template
See `docs/dev/templates/excitor-connector/` for the full quick‑start including:
- Sample options class + validator.
- Connector implementation inheriting from `VexConnectorBase`.
- Plugin adapter + manifest.
Copy the directory, rename namespaces/IDs, then iterate on provider-specific
logic.
---
*Last updated: 2025-10-17*

25
docs/dev/aoc-normalization-removal-notes.md
View File

@@ -1,25 +0,0 @@
# AOC Normalization Removal Notes
_Last updated: 2025-10-29_
## Goal
Document follow-up actions for CONCELIER-CORE-AOC-19-004 as we unwind the final pieces of normalization from the ingestion/runtime path.
## Current Findings
- `AdvisoryRawService` and `MongoAdvisoryRawRepository` already preserve upstream ordering and duplicate aliases (trim-only). No additional code changes required there.
- Observation layers (`AdvisoryObservationFactory`, `AdvisoryObservationQueryService`) still canonicalise aliases, PURLs, CPEs, and references. These need to be relaxed so Policy/overlays receive raw linksets and can own dedupe logic.
- Linkset mapper continues to emit deterministic hints. We will keep the mapper but ensure observation output can surface both raw and canonical views for downstream services.
## Next Steps
1. Introduce a raw linkset projection alongside the existing canonical mapper so Policy Engine can choose which flavour to consume. ✅ 2025-10-31: `AdvisoryObservation` now surfaces `RawLinkset`; Mongo documents store both canonical & raw shapes; tests/goldens updated.
2. Update observation factory/query tests to assert duplicate handling and ordering with the relaxed projection. ✅ 2025-10-31.
3. Refresh docs (`docs/ingestion/aggregation-only-contract.md`) once behaviour lands to explain the “raw vs canonical linkset” split. ✅ 2025-11-06: Added invariant notes and rollout guidance, linked to `docs/migration/no-merge.md` and `docs/dev/raw-linkset-backfill-plan.md`.
4. Coordinate with Policy Guild to validate consumers against the new raw projection before flipping defaults. ↺ Ongoing — see action items in `docs/dev/raw-linkset-backfill-plan.md` (2025-10-31 handshake with POLICY-ENGINE-20-003 owners).
- 2025-11-05: Catalogued residual normalization paths tied to the legacy Merge service and outlined `noMergeEnabled` feature-toggle work to keep AOC ingestion fully merge-free.
- 2025-11-05 19:20Z: Observation factory/linkset now preserve upstream ordering and duplicates; canonicalisation shifts to downstream services.
- 2025-11-06: Documented post-merge rollout plan and annotated sprint trackers with analyzer gating updates.
- 2025-11-06 23:30Z: Concelier core/linkset query paths now keep alias/reference casing & whitespace intact; alias filters switched to case-insensitive regex so raw data and lookups remain compatible.

77
docs/dev/authority-plugin-di-coordination.md
View File

@@ -1,77 +0,0 @@
# Authority Plug-in Scoped Service Coordination
> Created: 2025-10-19 — Plugin Platform Guild & Authority Core
> Status: Completed (workshop held 2025-10-20 15:00–16:05 UTC)
This document tracks preparation, agenda, and outcomes for the scoped-service workshop required before implementing PLUGIN-DI-08-002.
## Objectives
- Inventory Authority plug-in surfaces that need scoped service lifetimes.
- Confirm session/scope handling for identity-provider registrars and background jobs.
- Assign follow-up tasks/actions with owners and due dates.
## Scheduling Snapshot
- **Meeting time:** 2025-10-20 15:00–16:00 UTC (10:00–11:00 CDT / 08:00–09:00 PDT).
- **Facilitator:** Plugin Platform Guild — Alicia Rivera.
- **Attendees (confirmed):** Authority Core — Jasmin Patel; Authority Security Guild — Mohan Singh; Plugin Platform — Alicia Rivera, Leah Chen.
- **Optional invitees:** DevOps liaison — Sofia Ortega (accepted).
- **Logistics:** Invites sent via shared calendar on 2025-10-19 15:30 UTC with Teams bridge + offline dial-in. Meeting notes will be captured here.
- **Preparation deadline:** 2025-10-20 12:00 UTC — complete checklist below.
## Pre-work Checklist
- Review `ServiceBindingAttribute` contract introduced by PLUGIN-DI-08-001.
- Collect existing Authority plug-in registration code paths to evaluate.
- Audit background jobs that assume singleton lifetimes.
- Identify plug-in health checks/telemetry surfaces impacted by scoped lifetimes.
### Pre-work References
| Focus | Path | Notes |
|-------|------|-------|
| Host DI wiring | `src/Authority/StellaOps.Authority/StellaOps.Authority/Program.cs:159` | Startup registers `IAuthorityIdentityProviderRegistry` as a singleton and invokes `AuthorityPluginLoader.RegisterPlugins(...)` before the container is built. Any scoped plugin services will currently be captured in the singleton registry context. |
| Registrar discovery | `src/Authority/StellaOps.Authority/StellaOps.Authority/Plugins/AuthorityPluginLoader.cs:46` | Loader instantiates `IAuthorityPluginRegistrar` implementations via `Activator.CreateInstance`, so registrars cannot depend on host services yet. Need agreement on whether to move discovery post-build or introduce `ActivatorUtilities`. |
| Registry aggregation | `src/Authority/StellaOps.Authority/StellaOps.Authority/AuthorityIdentityProviderRegistry.cs:16` | Registry caches `IIdentityProviderPlugin` instances at construction time. With scoped lifetimes we must revisit how providers are resolved (factory vs accessor). |
| Standard registrar services | `src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/StandardPluginRegistrar.cs:21` | All plugin services are registered as singletons today (`StandardUserCredentialStore`, `StandardClientProvisioningStore`, hosted bootstrapper). This registrar is our baseline for migrating to scoped bindings. |
| Hosted bootstrapper | `src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/Bootstrap/StandardPluginBootstrapper.cs:17` | Background job directly consumes `StandardUserCredentialStore`. If the store becomes scoped we will need an `IServiceScopeFactory` bridge. |
| Password grant handler | `src/Authority/StellaOps.Authority/StellaOps.Authority/OpenIddict/Handlers/PasswordGrantHandlers.cs:26` | Password flow resolves `IIdentityProviderPlugin` during scoped requests. Scope semantics must ensure credential stores stay cancellation-aware. |
| Client credential handler | `src/Authority/StellaOps.Authority/StellaOps.Authority/OpenIddict/Handlers/ClientCredentialsHandlers.cs:21` | Handler fetches provider + `ClientProvisioning` store; confirms need for consistent scoping in both user and client flows. |
## Preliminary Findings — 2025-10-20
- `IAuthorityIdentityProviderRegistry` must stop materialising provider singletons when scoped lifetimes land. Options to evaluate: make the registry itself scoped, convert it to a factory over `IServiceProvider`, or cache lightweight descriptors and resolve implementations on-demand.
- `AuthorityPluginLoader` instantiates registrars without DI support. To let registrars request scoped helpers (e.g. `IServiceScopeFactory`) we may need a two-phase registration: discover types at build time, defer execution until the container is available.
- Hosted bootstrap tasks (e.g. `StandardPluginBootstrapper`) will break if their dependencies become scoped. Workshop should align on using scoped pipelines inside `StartAsync` or shifting bootstrap work to queued jobs.
- Standard plugin stores assume singleton access to Mongo collections and password hashing utilities. If we embrace scoped stores, document thread-safety expectations and reuse of Mongo clients across scopes.
- OpenIddict handlers already run as scoped services; once providers move to scoped lifetimes we must ensure the new resolution path stays cancellation-aware and avoids redundant service resolution per request.
- 2025-10-20 (PLUGIN-DI-08-003): Registry implementation updated to expose metadata + scoped handles; OpenIddict flows, bootstrap endpoints, and `/health` now resolve providers via scoped leases with accompanying test coverage.
- 2025-10-20 (PLUGIN-DI-08-004): Authority plugin loader now instantiates registrars via scoped DI activations and honours `[ServiceBinding]` metadata in plugin assemblies.
- 2025-10-20 (PLUGIN-DI-08-005): `StandardPluginBootstrapper` shifted to scope-per-run execution using `IServiceScopeFactory`, enabling future scoped stores without singleton leaks.
## Draft Agenda
1. Context recap (5 min) — why scoped DI is needed; summary of PLUGIN-DI-08-001 changes.
2. Authority plug-in surfaces (15 min) — registrars, background services, telemetry.
3. Session handling strategy (10 min) — scope creation semantics, cancellation propagation.
4. Action items & owners (10 min) — capture code/docs/test tasks with due dates.
5. Risks & follow-ups (5 min) — dependencies, rollout sequencing.
## Notes
- Session opened with recap of scoped-service goals and PLUGIN-DI-08-001 changes, confirming Authority readiness to adopt `[ServiceBinding]` metadata.
- Agreed to treat `IAuthorityIdentityProviderRegistry` as a scoped-factory facade rather than a singleton cache; registry will track descriptors and resolve implementations on-demand per request/worker scope.
- Standard plug-in bootstrap will create scopes via `IServiceScopeFactory` and pass cancellation tokens through to avoid lingering singleton references.
- Authority Plugin Loader will enumerate plug-in assemblies at startup but defer registrar activation until a scoped service provider is available, aligning with PLUGIN-DI-08-004 implementation.
- Follow-up engineering tasks assigned to land PLUGIN-DI-08-002 code path adjustments and Authority host updates before 2025-10-24.
## Action Item Log
| Item | Owner | Due | Status | Notes |
|------|-------|-----|--------|-------|
| Confirm meeting time | Alicia Rivera | 2025-10-19 15:30 UTC | DONE | Calendar invite sent; all required attendees accepted |
| Compile Authority plug-in DI entry points | Jasmin Patel | 2025-10-20 | DONE (2025-10-20) | Scoped-service touchpoints summarised in **Pre-work References** and **Preliminary Findings** ahead of the workshop. |
| Outline scoped-session pattern for background jobs | Leah Chen | 2025-10-21 | DONE (2025-10-20) | Pattern agreed: bootstrap services must open transient scopes per execution via `IServiceScopeFactory`; document update to follow in PLUGIN-DI-08-002 patch. |
| Update PLUGIN-DI-08-002 implementation plan | Alicia Rivera | 2025-10-21 | DONE (2025-10-20) | Task board + correspoding sprint file `../implplan/SPRINT_*.md` updated with scoped-integration delivery notes and test references. |
| Sync Authority host backlog | Mohan Singh | 2025-10-21 | DONE (2025-10-20) | Authority/Plugin TASKS.md and correspoding sprint file `../implplan/SPRINT_*.md` entries reflect scoped-service completion. |

53
docs/dev/cartographer-graph-handshake.md
View File

@@ -1,53 +0,0 @@
# Cartographer Graph Handshake Plan
> **Archived (2025-10-30).** Cartographer has been retired in favour of the Graph Indexer + Graph API platform (see `docs/devops/contracts-and-rules.md`). Keep this document only for historical reference; new work must reference `GRAPH-*` tasks and the Graph module docs.
>
> **2025-12-04 update:** The inspector contract requested here now lives at `docs/modules/graph/contracts/graph.inspect.v1.md` with schema `graph.inspect.v1.schema.json` and sample payloads under `docs/modules/graph/contracts/examples/`. Treat this file as historical background only.
_Status: 2025-10-29_
## Why this exists
The Concelier/Excititor graph enrichment work (CONCELIER-GRAPH-21-001/002, EXCITITOR-GRAPH-21-001/002/005) and the merge-side coordination tasks (FEEDMERGE-COORD-02-901/902) are blocked on a clear contract with Cartographer and the Policy Engine. This document captures the minimum artefacts each guild owes so we can unblock the graph pipeline and resume implementation without re-scoping every stand-up.
## Deliverables by guild
### Cartographer Guild
- **CARTO-GRAPH-21-002** (Inspector contract): publish the inspector payload schema (`graph.inspect.v1`) including the fields Cartographer needs from Concelier/Excititor (SBOM relationships, advisory/VEX linkouts, justification summaries). Target format: shared Proto/JSON schema stored under `src/Cartographer/Contracts/`.
- **CARTO-GRAPH-21-005** (Inspector access patterns): document the query shapes Cartographer will execute (PURL → advisory, PURL → VEX statement, policy scope filters) so storage can project the right indexes/materialized views. Include sample `mongosh` queries and desired TTL/limit behaviour.
- Provide a test harness (e.g., Postman collection or integration fixture) Cartographer will use to validate the Concelier/Excititor endpoints once they land.
### Concelier Core Guild
- Derive adjacency data from SBOM normalization as described in CONCELIER-GRAPH-21-001 (depends on `CONCELIER-POLICY-20-002`). Once Cartographer publishes the schema above, implement:
- Node payloads: component metadata, scopes, entrypoint annotations.
- Edge payloads: `contains`, `depends_on`, `provides`, provenance array.
- **Change events (CONCELIER-GRAPH-21-002)**: define `sbom.relationship.changed` event contract with tenant + context metadata, referencing Cartographer’s filter requirements. Include event samples and replay instructions in `docs/graph/concelier-events.md`.
- Coordinate with Cartographer on pagination/streaming expectations (page size, continuation token, retention window).
### Excititor Core & Storage Guilds
- **Inspector linkouts (EXCITITOR-GRAPH-21-001)**: expose Batched VEX/advisory lookup endpoint that accepts graph node PURLs and responds with raw document slices + justification metadata. Ensure Policy Engine scope enrichment (EXCITITOR-POLICY-20-002) feeds this response so Cartographer does not need to call multiple services.
- **Overlay enrichment (EXCITITOR-GRAPH-21-002)**: align the overlay metadata with Cartographer’s schema once it lands (include justification summaries, document versions, and provenance).
- **Indexes/materialized views (EXCITITOR-GRAPH-21-005)**: after Cartographer publishes query shapes, create the necessary indexes (PURL + tenant, policy scope) and document migrations in storage runbooks. Provide load testing evidence before enabling in production.
### Policy Guild
- **CONCELIER-POLICY-20-002**: publish the enriched linkset schema that powers both Concelier and Excititor payloads. Include enumerations for relationship types and scope tags.
- Share the Policy Engine timeline for policy overlay metadata (`POLICY-ENGINE-30-001`) so Excititor can plan the overlay enrichment delivery.
## Shared action items
| Owner | Task | Deadline | Notes |
|-------|------|----------|-------|
| Cartographer | Publish inspector schema + query patterns (`CARTO-GRAPH-21-002`/`21-005`) | 2025-11-04 | Attach schema files + examples to this doc once merged. |
| Concelier Core | Draft change-event payload with sample JSON | 2025-11-06 | Blocked until Cartographer schema lands; prepare skeleton PR in `docs/graph/concelier-events.md`. |
| Excititor Core/Storage | Prototype batch linkout API + index design doc | 2025-11-07 | Leverage Cartographer query patterns to size indexes; include perf targets. |
| Policy Guild | Confirm linkset enrichment fields + overlay timeline | 2025-11-05 | Needed to unblock both Concelier enrichment and Excititor overlay tasks. |
## Reporting
- Track progress in the `#cartographer-handshake` Slack thread (create once Cartographer posts the schema MR).
- During the twice-weekly graph sync, review outstanding checklist items above and update the task notes (`TASKS.md`) so the backlog reflects real-time status.
- Once the schema and query contracts are merged, the Concelier/Excititor teams can flip their tasks from **BLOCKED** to **DOING** and attach implementation plans referencing this document.
## Appendix: references
- `CONCELIER-GRAPH-21-001`, `CONCELIER-GRAPH-21-002` (Concelier Core task board)
- `EXCITITOR-GRAPH-21-001`, `EXCITITOR-GRAPH-21-002`, `EXCITITOR-GRAPH-21-005` (Excititor Core/Storage task boards)
- `CARTO-GRAPH-21-002`, `CARTO-GRAPH-21-005` (Cartographer task board)
- `POLICY-ENGINE-30-001`, `CONCELIER-POLICY-20-002`, `EXCITITOR-POLICY-20-002` (Policy Engine roadmap)

108
docs/dev/mongo_indices.md
View File

@@ -1,108 +0,0 @@
# Normalized Versions Query Guide
This guide complements the Sprint 1–2 normalized versions rollout. It documents recommended indexes and aggregation patterns for querying `AffectedPackage.normalizedVersions`.
For a field-by-field look at how normalized rules persist in MongoDB (including provenance metadata), see Section 8 of the [Concelier SemVer Merge Playbook](merge_semver_playbook.md).
## 1. Recommended indexes
When `concelier.storage.enableSemVerStyle` is enabled, advisories expose a flattened
`normalizedVersions` array at the document root. Create these indexes in `mongosh`
after the migration completes (adjust collection name if you use a prefix):
```javascript
db.advisories.createIndex(
{
"normalizedVersions.packageId": 1,
"normalizedVersions.scheme": 1,
"normalizedVersions.type": 1
},
{ name: "advisory_normalizedVersions_pkg_scheme_type" }
);
db.advisories.createIndex(
{ "normalizedVersions.value": 1 },
{ name: "advisory_normalizedVersions_value", sparse: true }
);
```
- The compound index accelerates `$match` stages that filter by package identifier and rule style without unwinding `affectedPackages`.
- The sparse index keeps storage costs low while supporting pure exact-version lookups (type `exact`).
The storage bootstrapper creates the same indexes automatically when the feature flag is enabled.
## 2. Query patterns
### 2.1 Determine if a specific version is affected
```javascript
db.advisories.aggregate([
{ $match: { "normalizedVersions.packageId": "pkg:npm/lodash" } },
{ $unwind: "$normalizedVersions" },
{ $match: {
$or: [
{ "normalizedVersions.type": "exact",
"normalizedVersions.value": "4.17.21" },
{ "normalizedVersions.type": "range",
"normalizedVersions.min": { $lte: "4.17.21" },
"normalizedVersions.max": { $gt: "4.17.21" } },
{ "normalizedVersions.type": "gte",
"normalizedVersions.min": { $lte: "4.17.21" } },
{ "normalizedVersions.type": "lte",
"normalizedVersions.max": { $gte: "4.17.21" } }
]
}},
{ $project: { advisoryKey: 1, title: 1, "normalizedVersions.packageId": 1 } }
]);
```
Use this pipeline during Sprint 2 staging validation runs. Invoke `explain("executionStats")` to confirm the compound index is selected.
### 2.2 Locate advisories missing normalized rules
```javascript
db.advisories.aggregate([
{ $match: { $or: [
{ "normalizedVersions": { $exists: false } },
{ "normalizedVersions": { $size: 0 } }
] } },
{ $project: { advisoryKey: 1, affectedPackages: 1 } }
]);
```
Run this query after backfill jobs to identify gaps that still rely solely on `rangeExpression`.
### 2.3 Deduplicate overlapping rules
```javascript
db.advisories.aggregate([
{ $unwind: "$normalizedVersions" },
{ $group: {
_id: {
identifier: "$normalizedVersions.packageId",
scheme: "$normalizedVersions.scheme",
type: "$normalizedVersions.type",
min: "$normalizedVersions.min",
minInclusive: "$normalizedVersions.minInclusive",
max: "$normalizedVersions.max",
maxInclusive: "$normalizedVersions.maxInclusive",
value: "$normalizedVersions.value"
},
advisories: { $addToSet: "$advisoryKey" },
notes: { $addToSet: "$normalizedVersions.notes" }
}},
{ $match: { "advisories.1": { $exists: true } } },
{ $sort: { "_id.identifier": 1, "_id.type": 1 } }
]);
```
Use this to confirm the merge dedupe logic keeps only one normalized rule per unique constraint.
## 3. Operational checklist
- [ ] Create the indexes in staging before toggling dual-write in production.
- [ ] Capture explain plans and attach them to the release notes.
- [ ] Notify downstream services that consume advisory snapshots about the new `normalizedVersions` array.
- [ ] Update export fixtures once dedupe verification passes.
Additional background and mapper examples live in [Concelier SemVer Merge Playbook](merge_semver_playbook.md).

55
docs/dev/normalized_versions_rollout.md
View File

@@ -1,55 +0,0 @@
# Normalized Versions Rollout Dashboard (Sprint 2 – Concelier)
_Status date: 2025-10-20 19:10 UTC_
This dashboard tracks connector readiness for emitting `AffectedPackage.NormalizedVersions` arrays and highlights upcoming coordination checkpoints. Use it alongside:
- [`src/Concelier/__Libraries/StellaOps.Concelier.Merge/RANGE_PRIMITIVES_COORDINATION.md`](../../src/Concelier/__Libraries/StellaOps.Concelier.Merge/RANGE_PRIMITIVES_COORDINATION.md) for detailed guidance and timelines.
- [Concelier SemVer Merge Playbook](merge_semver_playbook.md) §8 for persisted Mongo document shapes.
- [Normalized Versions Query Guide](mongo_indices.md) for index/query validation steps.
## Key milestones
- **2025-10-21** – Cccs and Cisco connectors finalize normalized rule emission and share merge-counter screenshots.
- **2025-10-22** – CertBund localisation translator reviewed; blockers escalated if localisation guidance slips.
- **2025-10-23** – ICS-CISA confirms SemVer reuse vs new firmware scheme and files Models ticket if needed.
- **2025-10-24** – KISA firmware scheme proposal due; Merge provides same-day review.
- **2025-10-25** – Merge runs cross-connector validation before enabling normalized-rule union logic by default.
## Connector readiness matrix
| Connector | Owner team | Normalized versions status | Last update | Next action / link |
|-----------|------------|---------------------------|-------------|--------------------|
| Acsc | BE-Conn-ACSC | ❌ Not started – normalized helper pending relay stability | 2025-10-20 | Prepare builder integration plan for 2025-10-24 kickoff; update `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Acsc/TASKS.md` once branch opens. |
| Cccs | BE-Conn-CCCS | ⚠️ DOING – trailing-version helper MR reviewing (due 2025-10-21) | 2025-10-20 | Land helper + fixture refresh, post merge-counter screenshot; `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cccs/TASKS.md`. |
| CertBund | BE-Conn-CERTBUND | ⚠️ In progress – localisation translator WIP (due 2025-10-22) | 2025-10-20 | Finish translator + provenance notes, regenerate fixtures; `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.CertBund/TASKS.md`. |
| CertCc | BE-Conn-CERTCC | ✅ Complete – `certcc.vendor` rules emitting | 2025-10-20 | Monitor VINCE payload changes; no action. |
| Kev | BE-Conn-KEV | ✅ Complete – catalog/due-date rules verified | 2025-10-20 | Routine monitoring only. |
| Cve | BE-Conn-CVE | ✅ Complete – SemVer normalized rules live | 2025-10-20 | Keep fixtures in sync as CVE schema evolves. |
| Ghsa | BE-Conn-GHSA | ✅ Complete – rollout merged 2025-10-11 | 2025-10-20 | Maintain parity with OSV ecosystems; no action. |
| Osv | BE-Conn-OSV | ✅ Complete – normalized rules shipping | 2025-10-20 | Watch for new ecosystems; refresh fixtures as needed. |
| Ics.Cisa | BE-Conn-ICS-CISA | ⚠️ Decision pending – exact SemVer promotion due 2025-10-23 | 2025-10-20 | Promote primitives or request new scheme; `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md`. |
| Kisa | BE-Conn-KISA | ⚠️ Proposal drafting – firmware scheme due 2025-10-24 | 2025-10-20 | Finalise `kisa.build` proposal with Models; update mapper/tests; `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kisa/TASKS.md`. |
| Ru.Bdu | BE-Conn-BDU | ✅ Complete – `ru-bdu.raw` rules live | 2025-10-20 | Continue monitoring UTF-8 handling; no action. |
| Ru.Nkcki | BE-Conn-Nkcki | ✅ Complete – normalized rules emitted | 2025-10-20 | Maintain transliteration guidance; no action. |
| Vndr.Apple | BE-Conn-Apple | ✅ Complete – normalized arrays emitting | 2025-10-20 | Add beta-channel coverage follow-up; see module README. |
| Vndr.Cisco | BE-Conn-Cisco | ⚠️ DOING – normalized promotion branch open (due 2025-10-21) | 2025-10-20 | Merge helper branch, refresh fixtures, post counters; `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md`. |
| Vndr.Msrc | BE-Conn-MSRC | ✅ Complete – `msrc.build` rules emitting | 2025-10-20 | Monitor monthly rollups; no action. |
| Nvd | BE-Conn-NVD | ✅ Complete – normalized SemVer output live | 2025-10-20 | Keep provenance aligned with CVE IDs; monitor export parity toggle. |
Legend: ✅ complete, ⚠️ in progress/partial, ❌ not started.
## Monitoring
- Merge now emits `concelier.merge.normalized_rules` (tags: `package_type`, `scheme`) and `concelier.merge.normalized_rules_missing` (tags: `package_type`). Track these counters to confirm normalized arrays land as connectors roll out.
- Expect `normalized_rules_missing` to trend toward zero as each connector flips on normalized output. Investigate any sustained counts by checking the corresponding module `TASKS.md`.
## Implementation tips
- When a connector only needs to populate `AffectedPackage.NormalizedVersions` (without reusing range primitives), call `SemVerRangeRuleBuilder.BuildNormalizedRules(rawRange, patchedVersion, note)` to project the normalized rule list directly. This avoids re-wrapping `SemVerRangeBuildResult` instances and keeps provenance notes consistent with the shared builder.
## How to use this dashboard
1. Before opening a connector PR, update the module `TASKS.md` entry and drop a short bullet here (status + timestamp).
2. When a connector lands normalized outputs, flip the status to ✅ and note any rollout toggles (feature flags, fixture regenerations).
3. If a dependency or blocker emerges, add it both in the module `TASKS.md` and in this matrix so merge/storage can escalate quickly.

56
docs/dev/raw-linkset-backfill-plan.md
View File

@@ -1,56 +0,0 @@
# Raw Linkset Backfill & Adoption Plan
_Last updated: 2025-11-06_
Owners: Concelier Storage Guild, DevOps Guild, Policy Guild
## Context
- Concelier observations now emit both a **canonical linkset** (deduped, normalised identifiers) and a **raw linkset** (`rawLinkset`) that preserves upstream ordering, duplicates, and original pointer metadata.
- Existing `concelier.advisory_observations` documents created before 2025-10-31 do **not** contain the `rawLinkset` field.
- Policy Engine selection joiners (`POLICY-ENGINE-20-003`) will switch to the raw projection once backfill completes and consumers validate fixtures.
## Objectives
1. Populate `rawLinkset` for historical observations across online clusters and Offline Kit bundles without breaking append-only guarantees.
2. Provide migration scripts + runbook so operators can rehearse in staging (and air-gapped deployments) before production rollout.
3. Unblock Policy Engine adoption by guaranteeing dual projections exist for all tenants.
## Deliverables
- [ ] **Migration script** (`20251104_advisory_observations_raw_linkset_backfill.csx`)
- Iterates observations lacking `rawLinkset`
- Rehydrates raw document via existing snapshot (or cached DTO)
- Reuses `AdvisoryObservationFactory.CreateRawLinkset`
- Writes using `$set` with optimistic retry; preserves `updatedAt` via `setOnInsert`
- [ ] **Offline Kit updater** (extend `ops/offline-kit/scripts/export_offline_bundle.py`) to patch bundles in-place
- [ ] **Runbook** covering:
- Pre-check query: `db.concelier.advisory_observations.countDocuments({ rawLinkset: { $exists: false } })`
- Backup procedure (`mongodump` or snapshot requirement)
- Dry-run mode limiting batches by tenant
- Metrics/telemetry expectations (`concelier.migrations.documents_processed_total`)
- Rollback (no-op because field addition; note to retain snapshot for verification)
- [ ] **Fixture updates** ensuring storage/CLI/Policy tests include `rawLinkset`
- [ ] **Policy Engine follow-up** to flip joiners once `rawLinkset` population reaches 100% (tracked via metrics).
## Timeline
| Date (UTC) | Milestone | Notes |
|------------|-----------|-------|
| 2025-10-31 | Handshake w/ Policy | Agreement to consume `rawLinkset`; this document created. |
| 2025-11-01 | Draft migration script | Validate against staging dataset snapshots. |
| 2025-11-04 | Storage task CONCELIER-STORE-AOC-19-005 due | Deliver script + runbook for review. |
| 2025-11-06 | Staging backfill rehearsal | Target < 30 min runtime on 5M observations; docs refreshed to highlight raw vs canonical invariants and analyzer guardrails. |
| 2025-11-08 | Policy fixtures updated | POL engine branch consumes `rawLinkset`. |
| 2025-11-11 | Production rollout window | Pending DevOps sign-off after rehearsals. |
## Open Questions
- Do we need archival of the canonical-only projection for backwards compatibility exports? (Policy to confirm.)
- Offline Kit delta: should we regenerate entire bundle or ship incremental patch? (DevOps reviewing.)
- Metrics: add `raw_linkset_missing_total` counter to detect regressions post-backfill?
## Next Actions
- [ ] Concelier Storage Guild: prototype migration script, share for review (`2025-11-01`).
- [ ] DevOps Guild: schedule staging rehearsal + update `docs/deploy/containers.md` with new runbook section.
- [ ] Policy Guild: prepare feature flag/branch to switch joiners once metrics show zero missing `rawLinkset`.

16
docs/events/prep/2025-11-20-advisoryai-orchestrator-followup.md
View File

@@ -1,16 +0,0 @@
# Escalation Follow-up Prep — PREP-ESCALATION-FOLLOW-UP-ADVISORYAI-ORCHESTR
Status: Draft (2025-11-20)
Owners: Planning · AdvisoryAI Guild · Orchestrator Service Guild · Notifications Guild
Scope: Track follow-up actions and ETAs for overdue AdvisoryAI evidence bundle schema and Orchestrator/Notifications envelopes.
## Follow-up actions
- Request revised ETA from AdvisoryAI for evidence bundle schema + payload notes; log hash placeholder and target drop date.
- Request revised ETA from Orchestrator/Notifications for capsule envelope and notification samples; confirm subject names and retention policy.
- If no ETA by 2025-11-21, escalate to Wave 150/140 leads and mark dependent sprint tasks BLOCKED explicitly with this reference.
## Recording commitments
- Capture responses (ETA/date + owner) in this file under a new “Commitments” section and mirror them into sprints 110/140/150/160/161/162/165.
## Handoff
This file is the published artefact for PREP-ESCALATION-FOLLOW-UP-ADVISORYAI-ORCHESTR. Update once responses arrive; if still silent by 2025-11-21, annotate with “No response” and keep dependents BLOCKED.

18
docs/events/prep/2025-11-20-orchestrator-notifications-schema-handoff.md
View File

@@ -1,18 +0,0 @@
# Orchestrator / Notifications Schema Handoff Prep — PREP-ORCHESTRATOR-NOTIFICATIONS-SCHEMA-HANDOF
Status: Draft (2025-11-20)
Owners: Orchestrator Service Guild · Notifications Guild · Planning
Scope: Capture the exact deliverables needed for the overdue schema handoff so downstream EvidenceLocker/ExportCenter/TimelineIndexer work can proceed.
## Expected deliverables
- **Capsule envelope schema** including `replay_id`, `dsse_envelope_hash`, `tenant_id`, `timeline_cursor`, `event_id`, `occurred_at`.
- **Transport bindings** for NATS/Redis topics with subject names and durable stream config; retention and dedupe requirements.
- **Samples**: at least one signed capsule example and one notification example referencing a replay record.
- **Versioning**: place canonical schema in `docs/events/orchestrator-scanner-events.md` and bump version tag; add samples under `docs/events/samples/`.
## Acceptance for unblock
- Schema + samples merged and checksummed; subject naming confirmed.
- Hash and version recorded back into sprint trackers (160, 161, 162, 165) under Decisions & Risks.
## Handoff
Use this document as the published prep artefact for PREP-ORCHESTRATOR-NOTIFICATIONS-SCHEMA-HANDOF.

5
docs/implplan/SPRINT_5100_0009_0001_scanner_tests.md
View File

@@ -53,7 +53,7 @@
| **PERF** | | | | | |
| 23 | SCANNER-5100-023 | DONE | None | Scanner Guild | Add perf smoke tests for reachability calculation (2× regression gate). |
| 24 | SCANNER-5100-024 | DONE | None | Scanner Guild | Add perf smoke tests for smart diff (2× regression gate). |
| 25 | SCANNER-5100-025 | DOING | None | Scanner Guild | Add perf smoke tests for canonical serialization (2× regression gate). |
| 25 | SCANNER-5100-025 | DONE | None | Scanner Guild | Add perf smoke tests for canonical serialization (2× regression gate). |
## Wave Coordination
- **Wave 1 (L0 + Determinism):** Tasks 1-10.
@@ -121,3 +121,6 @@
| 2025-12-24 | Task 20 (SCANNER-5100-020) DONE: Created `src/Scanner/__Tests/StellaOps.Scanner.Worker.Tests/Integration/EndToEndJobFlowTests.cs` with 5 test methods. Tests cover: basic job flow (enqueue → process → complete), multiple sequential jobs, stage event emission, telemetry recording (job_duration_ms), heartbeat renewal during long-running jobs. Uses in-memory mocks (FakeTimeProvider, ControlledDelayScheduler, RecordingAnalyzerDispatcher, EventRecorder). Also fixed pre-existing build error in WorkerEndToEndJobTests.cs (StartedAtUtc → StartUtc). All 5 tests passing. | Implementer |
| 2025-12-24 | Task 21 (SCANNER-5100-021) DONE: Created `src/Scanner/__Tests/StellaOps.Scanner.Worker.Tests/Integration/WorkerRetryTests.cs` with 8 test methods. Tests cover: transient failure on first attempt abandons for retry, permanent failure after max attempts poisons, second attempt under max abandons, maxAttempts=1 immediately poisons, host stopping abandons gracefully, successful job completes normally, retry boundary theory tests (5 variations). Uses TrackingJobLease with WasCompleted/WasAbandoned/WasPoisoned tracking. | Implementer |
| 2025-12-24 | Task 22 (SCANNER-5100-022) DONE: Created `src/Scanner/__Tests/StellaOps.Scanner.Worker.Tests/Integration/WorkerIdempotencyTests.cs` with 6 test methods. Tests cover: same job ID processed twice no duplicate results, different job IDs same scan ID single evidence, concurrent jobs same scan ID only one stored, exact same job ID second is no-op, distinct scan IDs each gets own evidence, idempotency with deterministic hash verification. Uses IdempotentEvidenceStore with processing count tracking and HashTrackingEvidenceStore for hash verification. | Implementer |
| 2025-12-24 | Task 23 (SCANNER-5100-023) DONE: Created `src/Scanner/__Tests/StellaOps.Scanner.Reachability.Tests/Perf/ReachabilityPerfSmokeTests.cs` with 12 test methods implementing 2× regression gate. Tests cover: graph construction (small/medium/large), graph ordering (deterministic, idempotent), subgraph extraction (single vuln, batch), path finding (entry-to-sink, scaling), memory efficiency (<100MB for 10K nodes). Uses FakeTimeProvider-like synthetic graphs with configurable node/edge counts. Baseline thresholds defined for each test. | Implementer |
| 2025-12-24 | Task 24 (SCANNER-5100-024) DONE: Created `src/Scanner/__Tests/StellaOps.Scanner.SmartDiff.Tests/Benchmarks/SmartDiffPerfSmokeTests.cs` with 12 test methods implementing 2× regression gate. Tests cover: diff computation (small/medium/large/XLarge), SARIF generation, scoring (single/batch), scaling behavior (linear), reachability flip handling, memory efficiency (<50MB). Baseline thresholds: small=25ms, medium=100ms, large=500ms, XLarge=2000ms. | Implementer |
| 2025-12-24 | Task 25 (SCANNER-5100-025) DONE: Created `src/Scanner/__Tests/StellaOps.Scanner.Core.Tests/Perf/CanonicalSerializationPerfSmokeTests.cs` with 14 test methods implementing 2× regression gate. Tests cover: serialization (small/medium/large/XLarge objects), digest computation (SHA-256), serialize+digest combined, batch operations (100 objects), dictionary ordering determinism, scaling behavior (linear), memory efficiency (<20MB), determinism verification (same input→same output, parallel safety). Baseline thresholds: small=1ms, medium=5ms, large=20ms, XLarge=100ms. | Implementer |

37
docs/implplan/SPRINT_5100_0009_0003_excititor_tests.md
View File

@@ -24,33 +24,33 @@
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| **C1 Connectors (CSAF/OpenVEX)** | | | | | |
| 1 | EXCITITOR-5100-001 | TODO | Connector fixtures | Excititor Guild | Set up fixture folders for CSAF connector: `Fixtures/csaf/<case>.json` (raw), `Expected/<case>.canonical.json` (normalized VEX claim). |
| 2 | EXCITITOR-5100-002 | TODO | Task 1 | Excititor Guild | Add parser tests for CSAF connector: fixture → parse → assert canonical JSON snapshot. |
| 3 | EXCITITOR-5100-003 | TODO | Task 1 | Excititor Guild | Add resilience tests: multiple product branches, status transitions, "not affected" with justification evidence. |
| 4 | EXCITITOR-5100-004 | TODO | Task 1 | Excititor Guild | Add security tests: URL allowlist, redirect handling, max payload size. |
| 5 | EXCITITOR-5100-005 | TODO | Connector fixtures | Excititor Guild | Repeat fixture setup for OpenVEX connector (Tasks 1-4 pattern). |
| 1 | EXCITITOR-5100-001 | DONE | Connector fixtures | Excititor Guild | Set up fixture folders for CSAF connector: `Fixtures/csaf/<case>.json` (raw), `Expected/<case>.canonical.json` (normalized VEX claim). |
| 2 | EXCITITOR-5100-002 | DONE | Task 1 | Excititor Guild | Add parser tests for CSAF connector: fixture → parse → assert canonical JSON snapshot. |
| 3 | EXCITITOR-5100-003 | DONE | Task 1 | Excititor Guild | Add resilience tests: multiple product branches, status transitions, "not affected" with justification evidence. |
| 4 | EXCITITOR-5100-004 | DONE | Task 1 | Excititor Guild | Add security tests: URL allowlist, redirect handling, max payload size. |
| 5 | EXCITITOR-5100-005 | DONE | Connector fixtures | Excititor Guild | Repeat fixture setup for OpenVEX connector (Tasks 1-4 pattern). |
| **L0 Formats/Export** | | | | | |
| 6 | EXCITITOR-5100-006 | TODO | TestKit | Excititor Guild | Add snapshot tests for OpenVEX export (Formats.OpenVEX) — canonical JSON. |
| 7 | EXCITITOR-5100-007 | TODO | TestKit | Excititor Guild | Add snapshot tests for CSAF export (Formats.CSAF) — canonical JSON. |
| 8 | EXCITITOR-5100-008 | TODO | TestKit | Excititor Guild | Add snapshot tests for CycloneDX VEX export (Formats.CycloneDX) — canonical JSON. |
| 6 | EXCITITOR-5100-006 | DONE | TestKit | Excititor Guild | Add snapshot tests for OpenVEX export (Formats.OpenVEX) — canonical JSON. |
| 7 | EXCITITOR-5100-007 | DONE | TestKit | Excititor Guild | Add snapshot tests for CSAF export (Formats.CSAF) — canonical JSON. |
| 8 | EXCITITOR-5100-008 | DONE | TestKit | Excititor Guild | Add snapshot tests for CycloneDX VEX export (Formats.CycloneDX) — canonical JSON. |
| **"Preserve Prune Source" Tests (Mandatory)** | | | | | |
| 9 | EXCITITOR-5100-009 | TODO | TestKit | Excititor Guild | Add preserve-prune test: input VEX with prune markers → output preserves source references. |
| 10 | EXCITITOR-5100-010 | TODO | TestKit | Excititor Guild | Add preserve-prune test: input VEX with pruning rationale → output preserves rationale. |
| 11 | EXCITITOR-5100-011 | TODO | TestKit | Excititor Guild | Add negative test: Excititor does not compute lattice decisions (only preserves and transports). |
| 9 | EXCITITOR-5100-009 | DONE | TestKit | Excititor Guild | Add preserve-prune test: input VEX with prune markers → output preserves source references. |
| 10 | EXCITITOR-5100-010 | DONE | TestKit | Excititor Guild | Add preserve-prune test: input VEX with pruning rationale → output preserves rationale. |
| 11 | EXCITITOR-5100-011 | DONE | TestKit | Excititor Guild | Add negative test: Excititor does not compute lattice decisions (only preserves and transports). |
| **S1 Storage** | | | | | |
| 12 | EXCITITOR-5100-012 | DONE | Storage harness | Excititor Guild | Add migration tests for Excititor.Storage (apply from scratch, apply from N-1). |
| 13 | EXCITITOR-5100-013 | DONE | Storage harness | Excititor Guild | Add idempotency tests: same VEX claim ID, same source snapshot → no duplicates. |
| 14 | EXCITITOR-5100-014 | DONE | Storage harness | Excititor Guild | Add query determinism tests (explicit ORDER BY checks). |
| **W1 WebService** | | | | | |
| 15 | EXCITITOR-5100-015 | TODO | WebService fixture | Excititor Guild | Add contract tests for Excititor.WebService endpoints (VEX ingest, export) — OpenAPI snapshot. |
| 16 | EXCITITOR-5100-016 | TODO | WebService fixture | Excititor Guild | Add auth tests (deny-by-default, token expiry, scope enforcement). |
| 17 | EXCITITOR-5100-017 | TODO | WebService fixture | Excititor Guild | Add OTel trace assertions (verify vex_claim_id, source_id tags). |
| 15 | EXCITITOR-5100-015 | DONE | WebService fixture | Excititor Guild | Add contract tests for Excititor.WebService endpoints (VEX ingest, export) — OpenAPI snapshot. |
| 16 | EXCITITOR-5100-016 | DONE | WebService fixture | Excititor Guild | Add auth tests (deny-by-default, token expiry, scope enforcement). |
| 17 | EXCITITOR-5100-017 | DONE | WebService fixture | Excititor Guild | Add OTel trace assertions (verify vex_claim_id, source_id tags). |
| **WK1 Worker** | | | | | |
| 18 | EXCITITOR-5100-018 | TODO | Storage harness | Excititor Guild | Add end-to-end ingest job test: enqueue VEX ingest → worker processes → claim stored → events emitted. |
| 19 | EXCITITOR-5100-019 | TODO | Storage harness | Excititor Guild | Add retry tests: transient failure uses backoff; permanent failure routes to poison. |
| 20 | EXCITITOR-5100-020 | TODO | Storage harness | Excititor Guild | Add OTel correlation tests: verify trace spans across job lifecycle. |
| 18 | EXCITITOR-5100-018 | DONE | Storage harness | Excititor Guild | Add end-to-end ingest job test: enqueue VEX ingest → worker processes → claim stored → events emitted. |
| 19 | EXCITITOR-5100-019 | DONE | Storage harness | Excititor Guild | Add retry tests: transient failure uses backoff; permanent failure routes to poison. |
| 20 | EXCITITOR-5100-020 | DONE | Storage harness | Excititor Guild | Add OTel correlation tests: verify trace spans across job lifecycle. |
| **Architecture Enforcement** | | | | | |
| 21 | EXCITITOR-5100-021 | TODO | Architecture tests | Excititor Guild | Add architecture test: Excititor assemblies must not reference Scanner lattice engine assemblies. |
| 21 | EXCITITOR-5100-021 | DONE | Architecture tests | Excititor Guild | Add architecture test: Excititor assemblies must not reference Scanner lattice engine assemblies. |
## Wave Coordination
- **Wave 1 (Connectors):** Tasks 1-5.
@@ -105,3 +105,4 @@
| --- | --- | --- |
| 2025-12-23 | Sprint created for Excititor module test implementation based on advisory Section 3.3 and TEST_CATALOG.yml. | Project Mgmt |
| 2025-12-24 | Tasks 12-14 DONE: Added S1 Storage tests. Task 12: `ExcititorMigrationTests.cs` (7 tests: from scratch, N-1, idempotency, schema integrity, FK constraints, VEX tables). Task 13: `VexStatementIdempotencyTests.cs` (8 tests: append dedupe, batch dedupe, disagreement idempotency, tenant isolation). Task 14: `VexQueryDeterminismTests.cs` (9 tests: mutation log ordering, conflict queries, observation ordering, concurrent queries). | Implementer |
| 2025-12-24 | Tasks 18-21 DONE: Added Worker and Architecture tests. Task 18: `EndToEndIngestJobTests.cs` (5 E2E tests). Task 19: `WorkerRetryPolicyTests.cs` (8 retry/backoff tests). Task 20: `WorkerOTelCorrelationTests.cs` (10 OTel correlation tests). Task 21: `ExcititorAssemblyDependencyTests.cs` (9 architecture constraint tests). Sprint 5100.0009.0003 COMPLETE. | Implementer |

28
docs/implplan/SPRINT_5100_0009_0005_authority_tests.md
View File

@@ -29,20 +29,20 @@
| 4 | AUTHORITY-5100-004 | DONE | TestKit | Authority Guild | Add unit tests for tenant isolation: token for tenant A cannot access tenant B resources. |
| 5 | AUTHORITY-5100-005 | DONE | TestKit | Authority Guild | Add unit tests for role-based access: role permissions correctly enforced. |
| **C1 Auth Provider Connectors** | | | | | |
| 6 | AUTHORITY-5100-006 | TODO | Connector fixtures | Authority Guild | Set up fixture folders for OIDC connector: `Fixtures/oidc/<case>.json` (raw), `Expected/<case>.canonical.json` (normalized). |
| 7 | AUTHORITY-5100-007 | TODO | Task 6 | Authority Guild | Add parser tests for OIDC connector: fixture → parse → assert canonical JSON snapshot. |
| 8 | AUTHORITY-5100-008 | TODO | Task 6 | Authority Guild | Add resilience tests: missing fields, invalid token formats, malformed claims. |
| 9 | AUTHORITY-5100-009 | TODO | Task 6 | Authority Guild | Add security tests: token replay protection, CSRF protection, redirect URI validation. |
| 10 | AUTHORITY-5100-010 | TODO | Connector fixtures | Authority Guild | Repeat fixture setup for SAML connector (Tasks 6-9 pattern). |
| 11 | AUTHORITY-5100-011 | TODO | Connector fixtures | Authority Guild | Repeat fixture setup for LDAP connector (Tasks 6-9 pattern). |
| 6 | AUTHORITY-5100-006 | BLOCKED | Connector fixtures | Authority Guild | Set up fixture folders for OIDC connector: `Fixtures/oidc/<case>.json` (raw), `Expected/<case>.canonical.json` (normalized). **BLOCKED: No OIDC plugin exists in Authority module. Need StellaOps.Authority.Plugin.Oidc implementation first.** |
| 7 | AUTHORITY-5100-007 | BLOCKED | Task 6 | Authority Guild | Add parser tests for OIDC connector: fixture → parse → assert canonical JSON snapshot. **BLOCKED: Depends on Task 6.** |
| 8 | AUTHORITY-5100-008 | BLOCKED | Task 6 | Authority Guild | Add resilience tests: missing fields, invalid token formats, malformed claims. **BLOCKED: Depends on Task 6.** |
| 9 | AUTHORITY-5100-009 | BLOCKED | Task 6 | Authority Guild | Add security tests: token replay protection, CSRF protection, redirect URI validation. **BLOCKED: Depends on Task 6.** |
| 10 | AUTHORITY-5100-010 | BLOCKED | Connector fixtures | Authority Guild | Repeat fixture setup for SAML connector (Tasks 6-9 pattern). **BLOCKED: No SAML plugin exists in Authority module.** |
| 11 | AUTHORITY-5100-011 | DONE | Connector fixtures | Authority Guild | Repeat fixture setup for LDAP connector (Tasks 6-9 pattern). **LDAP plugin exists; can proceed.** |
| **W1 WebService** | | | | | |
| 12 | AUTHORITY-5100-012 | TODO | WebService fixture | Authority Guild | Add contract tests for Authority.WebService endpoints (token issuance, token validation, user management) — OpenAPI snapshot. |
| 13 | AUTHORITY-5100-013 | TODO | WebService fixture | Authority Guild | Add auth tests: test auth bypass attempts (missing tokens, invalid signatures, expired tokens). |
| 14 | AUTHORITY-5100-014 | TODO | WebService fixture | Authority Guild | Add OTel trace assertions (verify user_id, tenant_id, scope tags). |
| 15 | AUTHORITY-5100-015 | TODO | WebService fixture | Authority Guild | Add negative tests: unsupported grant types, malformed requests, rate limiting. |
| 12 | AUTHORITY-5100-012 | DONE | WebService fixture | Authority Guild | Add contract tests for Authority.WebService endpoints (token issuance, token validation, user management) — OpenAPI snapshot. |
| 13 | AUTHORITY-5100-013 | DONE | WebService fixture | Authority Guild | Add auth tests: test auth bypass attempts (missing tokens, invalid signatures, expired tokens). |
| 14 | AUTHORITY-5100-014 | DONE | WebService fixture | Authority Guild | Add OTel trace assertions (verify user_id, tenant_id, scope tags). |
| 15 | AUTHORITY-5100-015 | DONE | WebService fixture | Authority Guild | Add negative tests: unsupported grant types, malformed requests, rate limiting. |
| **Sign/Verify Integration** | | | | | |
| 16 | AUTHORITY-5100-016 | TODO | TestKit | Authority Guild | Add sign/verify roundtrip tests: token signed with private key → verified with public key. |
| 17 | AUTHORITY-5100-017 | TODO | TestKit | Authority Guild | Add error classification tests: key not present, provider unavailable → deterministic error codes. |
| 16 | AUTHORITY-5100-016 | DONE | TestKit | Authority Guild | Add sign/verify roundtrip tests: token signed with private key → verified with public key. |
| 17 | AUTHORITY-5100-017 | DONE | TestKit | Authority Guild | Add error classification tests: key not present, provider unavailable → deterministic error codes. |
## Wave Coordination
- **Wave 1 (L0 Core Logic):** Tasks 1-5.
@@ -90,3 +90,7 @@
| 2025-12-23 | Sprint created for Authority module test implementation based on advisory Section 3.5 (partial) and TEST_CATALOG.yml. | Project Mgmt |
| 2025-12-24 | Tasks 1-4 DONE: Added L0 Core Auth Logic tests. Task 1: Added 5 token issuance tests to `StellaOpsTokenClientTests.cs` (client credentials flow, custom scopes, missing client ID, additional parameters). Task 2: Added 4 token validation tests (server error handling, missing access_token, default token type, default expiry). Tasks 3-4: Existing `StellaOpsScopeAuthorizationHandlerTests.cs` already covers scope enforcement (15+ tests) and tenant isolation (`HandleRequirement_Fails_WhenTenantMismatch`). | Implementer |
| 2025-12-24 | Task 5 DONE: Created `RoleBasedAccessTests.cs` with 13 comprehensive RBAC tests covering: user-role assignment (5 tests: permissions via roles, deny-by-default, expired roles, future expiry, permanent roles), multiple roles (4 tests: accumulated permissions, overlapping permissions, partial expiry), role removal (2 tests: removing role removes permissions, removing permission affects all users), and role permission enforcement (2 tests: assigned-only permissions, system roles). Wave 1 complete. | Implementer |
| 2025-12-24 | Tasks 6-10 BLOCKED: OIDC and SAML plugins do not exist in Authority module. Cannot create connector fixtures until `StellaOps.Authority.Plugin.Oidc` and `StellaOps.Authority.Plugin.Saml` are implemented. | Implementer |
| 2025-12-24 | Task 11 DONE: Created LDAP connector fixture tests. Added: `Fixtures/ldap/` folder with 5 fixtures (basic-user, minimal-user, multi-valued-user, service-account, user-not-found). Added `Expected/ldap/` with matching canonical JSON outputs. Created `LdapConnectorSnapshotTests.cs` (fixture-based snapshot tests), `LdapConnectorResilienceTests.cs` (12 resilience tests: missing attrs, invalid formats, connection failures, Unicode), `LdapConnectorSecurityTests.cs` (12 security tests: LDAP injection prevention, bind DN security, TLS enforcement, credential exposure prevention). | Implementer |
| 2025-12-24 | Tasks 12-15 DONE: Created W1 WebService tests. `AuthorityContractSnapshotTests.cs` (OpenAPI contract tests for token endpoints, security schemes, /.well-known). `AuthorityAuthBypassTests.cs` (15+ auth bypass prevention tests: missing tokens, invalid signatures, expired tokens, alg:none attacks). `AuthorityOTelTraceTests.cs` (OTel trace assertion tests for user_id, tenant_id, scope tags). `AuthorityNegativeTests.cs` (negative tests: unsupported grant types, malformed requests, size limits, method mismatch, error response format). | Implementer |
| 2025-12-24 | Tasks 16-17 DONE: Created Sign/Verify Integration tests. `TokenSignVerifyRoundtripTests.cs` (11 tests: RSA sign/verify, ECDSA sign/verify, HMAC sign/verify, multiple algorithms RS256/RS384/RS512, claims preservation, wrong public key rejection, tampered payload rejection, key rotation scenarios). `KeyErrorClassificationTests.cs` (12+ error classification tests: missing signing key, empty key collection, key ID mismatch, expired token, not-yet-valid token, issuer/audience mismatch, deterministic error code mapping). Wave 3 complete. **SPRINT COMPLETE** (all unblocked tasks done; Tasks 6-10 remain BLOCKED pending OIDC/SAML plugin implementations). | Implementer |

2
docs/implplan/SPRINT_5100_0009_0006_signer_tests.md
View File

@@ -28,7 +28,7 @@
| 2 | SIGNER-5100-002 | DONE | TestKit | Crypto Guild | Add stable digest computation tests: same input → same SHA-256 hash. |
| 3 | SIGNER-5100-003 | DONE | Determinism gate | Crypto Guild | Add determinism test: canonical payload hash stable across runs. |
| **C1 Crypto Plugin Tests** | | | | | |
| 4 | SIGNER-5100-004 | TODO | Connector fixtures | Crypto Guild | Add capability detection tests for BouncyCastle plugin: enumerate supported algorithms. |
| 4 | SIGNER-5100-004 | DOING | Connector fixtures | Crypto Guild | Add capability detection tests for BouncyCastle plugin: enumerate supported algorithms. |
| 5 | SIGNER-5100-005 | TODO | Task 4 | Crypto Guild | Add sign/verify roundtrip tests for BouncyCastle: sign with private key → verify with public key. |
| 6 | SIGNER-5100-006 | TODO | Task 4 | Crypto Guild | Add error classification tests for BouncyCastle: key not present → deterministic error code. |
| 7 | SIGNER-5100-007 | TODO | Connector fixtures | Crypto Guild | Repeat plugin tests for CryptoPro (GOST) plugin (Tasks 4-6 pattern). |

123
docs/license-jwt-quota.md
View File

@@ -1,123 +0,0 @@
---
title: Offline JWT licence & daily‑run quota
description: How Stella‑Ops enforces a **runs‑per‑day** limit in fully air‑gapped deployments.
nav:
order: 36
---
# JWT‑based daily‑run licence (offline‑capable)
When *Stella‑Ops* scanners operate entirely **offline**, they cannot phone home
for metering.
Instead, the backend accepts a **signed JSON Web Token (JWT)** that states the
**maximum number of scans per UTC day**.
If no token is supplied, a _grace quota_ of **33 runs/24 h** applies.
---
## 1  Token contents
| Claim | Purpose | Example |
|-------|---------|---------|
| `sub` | Customer / licensee identifier | `"f47ac10b…"` |
| `iat` | Issued‑at timestamp | `1722566400` |
| `exp` | Absolute licence expiry | `2025‑12‑31T23:59:59Z` |
| `tier` | **Max scans per UTC day** | `{{ quota_token }}` |
| `tid` | Token identifier (32‑byte) | `"7d2285..."` |
| `pkg` | Product SKU / edition | `"stella‑core"` |
Tokens are signed with **RS256** and verified locally using the bundled public key.
Only the public key ships inside the container; the private key never leaves
the build pipeline.
---
## 2  Obtaining a token
1. **Request** → `POST /​register { email:"alice@example.org" }`
2. Service hashes the e‑mail (SHA‑256), stores it, and issues a JWT (60 days by default).
3. Token is e‑mailed to you.
A new request for the same e‑mail returns the **same** token until it nears
expiry, avoiding quota “top‑ups” by re‑registration.
---
## 3  Supplying the token to an air‑gapped stack
```bash
# recommended
docker run \
-v /opt/stella/license/alice.jwt:/run/secrets/stella_license.jwt:ro \
stella‑ops
````
Other supported paths:
| Method | Mount point | Hot‑reload |
| ------------- | ------------------------ | ----------- |
| Docker secret | `/run/secrets/…` | ✓ (inotify) |
| Bind‑mounted | user‑chosen path (above) | ✓ |
| Env variable | `STELLA_LICENSE_JWT` | ✗ restart |
---
## 4  Quota‑enforcement algorithm
```mermaid
flowchart TD
Start --> Verify[Verify JWT signature]
Verify -->|Invalid| Deny1[Run in non licensed mode]
Verify --> Load[load today's counter UTC]
Load -->|SUM of last 24h scans < daily_quota| Permit[allow scan, add scan]
Permit --> End
Load -->|SUM of last 24h scans ≥ daily_quota| Deny1
```
## 5  Renewal procedure
| Scenario | Action |
| -------------- | --------------------------------------------------------------------------------- |
| More capacity | Request new token with higher `daily_quota`; replace file – **no restart needed** |
| Licence expiry | Same as above; new `exp` date |
| Key rotation | Container image ships new public key(s); older tokens still verify |
---
## 6  Fallback limits
| Situation | Daily quota |
| ----------------------- | ----------------------------------- |
| Valid JWT present | value of `daily_quota` claim ({{ quota_token }}) |
| No JWT | **33** |
| JWT expired (if used) | treated as **anonymous** unless policy enforces hard‑fail |
| Token signature invalid | **0** (reject) |
---
## 7  Threat‑model highlights (future work / optional hardening)
| Threat | Mitigation |
| --------------------------- | ---------------------------------------------------------------------- |
| Copy token & DB to 2nd node | Bind `sub`/`tid` to host fingerprint (TPM EK) – optional enterprise control |
| Counter DB rollback | Hash‑chain + monotonic clock – optional enterprise control |
| Flooding single node | Redis‑backed cluster rate‑limit (30 hits / 60 s) + edge Nginx (20 r/s) |
| Key compromise | Rotate RS256 key‑pair, ship new pubkey, re‑sign tokens |
---
## 8  Anonymous (33 runs) mode
Offline PoCs without registration still work:
```bash
docker compose exec stella-ops stella-jwt reload # reloads, discovers no token
```
…but **production deployments *must* register** to unlock real‑world quotas and
receive security advisories via e‑mail.
---
*Last updated: 2025‑08‑02*

19
docs/modules/attestor/prep/2025-11-24-attest-plan-2001.md
View File

@@ -1,19 +0,0 @@
# Attestation Plan 2001 · Evidence Locker contract handoff (2025-11-24)
Owners: Evidence Locker Guild · Excititor Guild
Status: Published (unblocks ATTEST-PLAN-2001)
## Inputs
- Sealed bundle contract: `docs/modules/evidence-locker/prep/2025-11-24-evidence-locker-contract.md`
- Bundle schema: `docs/modules/evidence-locker/schemas/bundle.schema.json`
- Sample bundle + hash: `docs/modules/evidence-locker/samples/evidence-bundle-sample.tgz` (+ `.sha256`)
## Plan
1) Align attestation payloads with sealed bundle contract (subjects, DSSE layout, manifest fields).
2) Produce CLI/Export Center consumer notes: expected file layout, required hashes, validation steps.
3) Add verification harness reference for Excititor/Attestor (reuse sample bundle + DSSE public key from contract note).
4) Update downstream sprints (Excititor airgap/export, Export Center) with contract link and hash.
## Next actions
- Evidence Locker Guild: confirm final schema hash matches sample bundle (track in contract note).
- Excititor Guild: wire contract path into airgap/attestation tests; report readiness in respective sprints.

1
docs/modules/authority/README.md
View File

@@ -34,6 +34,7 @@ Authority is the platform OIDC/OAuth2 control plane that mints short-lived, send
- ./operations/key-rotation.md
- ./operations/monitoring.md
- ./operations/grafana-dashboard.json
- ./crypto-provider-contract.md
- ./gaps/2025-12-04-auth-gaps-au1-au10.md
- ./gaps/2025-12-04-rekor-receipt-gaps-rr1-rr10.md
- Sprint/status mirrors: `docs/implplan/SPRINT_0314_0001_0001_docs_modules_authority.md`, `docs/modules/authority/TASKS.md`

68
docs/modules/authority/crypto-provider-contract.md Normal file
View File

@@ -0,0 +1,68 @@
# Authority Crypto Provider and JWKS Contract
This document defines the minimum contract Authority must satisfy to support sovereign crypto providers (FIPS, GOST, SM, PQ-ready modes where enabled) while keeping exports deterministic and offline-verifiable.
## 1) Scope
- Provider registry binding for Authority signing keys.
- JWKS export rules (which keys are exposed and how `kid` is computed).
- Signing profile selection rules (how Authority chooses keys/algorithms for different operations).
- Determinism requirements for JWKS JSON and key identifiers.
This contract is written so downstream consumers (Gateway, services, CLI, Offline Kit tooling) can validate tokens and rotate trust roots without depending on Authority's internal database schema.
## 2) Terms
- Provider: a crypto implementation or backend (software, HSM, KMS) identified by `provider_id`.
- Key: a signing key (or key handle) identified by `key_id` within a provider.
- Profile: a named signing policy that constrains algorithm choices and key selection.
- `kid`: the key identifier exposed in JWT headers and JWKS entries.
## 3) Provider registry (required fields)
Each registered Authority signing key MUST have, at minimum:
- `provider_id`: stable provider identifier (string).
- `key_id`: stable key identifier within the provider (string).
- `alg`: signing algorithm identifier as used by JWT/JWS (string).
- `usage`: at least `sig` (signing). If other usages are supported, they must be explicit.
- `tenant_scope` (optional): when set, the key is only valid for the specified tenant(s); when absent, it is platform-wide.
## 4) JWKS export rules
### 4.1 Which keys are exported
- JWKS MUST include only keys usable for token verification by downstream services.
- Exported keys MUST be filtered by:
- profile rules (do not export keys that cannot be selected by any active profile), and
- tenant scope (do not export tenant-scoped keys into other tenants' JWKS views if per-tenant JWKS is supported).
### 4.2 `kid` composition (deterministic)
`kid` MUST be stable across restarts and across equivalent deployments.
Recommended minimum rule:
- Compute `kid` from the public key material plus the profile discriminator:
- `kid = sha256(public_key_spki_bytes || ":" || profile_id)`
If Authority exposes both an all-profiles JWKS and a per-profile JWKS, the `kid` value must remain stable in both views.
### 4.3 Canonical JWKS JSON (deterministic)
- JWKS JSON MUST be canonicalized for reproducible hashing and offline verification:
- stable key ordering (sort by `kid` ascending),
- stable field ordering within a key object,
- no incidental whitespace dependence for hashing or signatures.
## 5) Signing profiles
Authority SHOULD define named profiles (examples: `default`, `ru-gost`, `pq-experimental`) that map:
- allowed algorithms (`alg` allowlist),
- allowed providers (provider allowlist),
- selection precedence when multiple keys match.
Profile selection MUST be deterministic given the same registry and request context.
## 6) Rotation and revocation expectations
- Rotation MUST preserve the ability to verify previously issued (unexpired) tokens.
- Revocation exports and JWKS exports MUST be consumable offline (cacheable and distributable in Offline Kit bundles).
## 7) Related docs
- Authority module dossier: `docs/modules/authority/architecture.md`
- Authority operations (key rotation, backup/restore): `docs/modules/authority/operations/`
- Gateway tenant/auth header contract: `docs/api/gateway/tenant-auth.md`

19
docs/modules/authority/prep/2025-11-20-auth-crypto-provider-prep.md
View File

@@ -1,19 +0,0 @@
# Authority Crypto Provider Contract Prep — PREP-AUTH-CRYPTO-90-001-NEEDS-AUTHORITY-PROVI
Status: Draft (2025-11-20)
Owners: Authority Core Guild · Security Guild
Scope: Capture the provider/key/JWKS contract Authority must publish to unblock sovereign crypto enablement.
## Required contract elements
- Provider registry binding for Authority signing keys (FIPS, GOST, PQ optional): fields `provider_id`, `key_id`, `alg`, `kid`, `usage`, `tenant_scope?`.
- JWKS export requirements: which keys exposed, `x5u`/`x5c` handling, `kid` format, and rotation cadence.
- Signing profiles: mapping of Authority API operations to provider profiles (default, ru-gost, pq-experimental).
- Determinism: canonical JSON for JWKS; stable `kid` composition (hash of public key + profile).
## Acceptance / unblock criteria
- Publish provider contract in `docs/modules/authority/crypto-provider-contract.md` (or update existing doc) with sample JWKS and provider config snippet.
- Record schema hash/kid composition rule here and in Sprint 0514 Decisions/Risks.
- Notify downstream consumers (Scanner, Attestor, Concelier) via sprint links once frozen.
## Handoff
Use this doc as the prep artefact for PREP-AUTH-CRYPTO-90-001-NEEDS-AUTHORITY-PROVI. Update with the final contract and samples; then set the sprint task to DONE and unblock AUTH-CRYPTO-90-001 implementation.

18
docs/modules/cli/guides/authentication.md
View File

@@ -1,18 +0,0 @@
# CLI Authentication — Draft Skeleton (2025-12-05 UTC)
Status: draft placeholder. Inputs pending: DVDO0110 env vars, token formats, monitoring plan.
## Supported Flows
- Device/code, PAT, workload identity (to confirm).
## Configuration
- Env vars and flags (to be filled once finalized).
## Multi-Tenant Considerations
- Scope selection and defaults.
## Troubleshooting
- Common errors; log paths; retry/backoff guidance.
## Open TODOs
- Insert definitive env var list and examples when available.

94
docs/modules/cli/guides/commands/auth.md
View File

@@ -1,19 +1,85 @@
# stella auth — Command Guide
# stella auth - Command Guide
The `stella auth` command group manages Authority-backed authentication and token operations used by other CLI commands.
## Commands
- `stella auth login --token <token> [--url <baseUrl>]`
- `stella auth status`
- `stella auth logout`
## Flags
- `--url`: API base URL; defaults to config/env.
- `--token`: bearer token or OIDC device code (future); stored in config if allowed.
### auth login
## Behaviour
- Login writes token to config file or keyring (where supported) with deterministic permissions; never echoes secrets.
- Status prints current user/tenant scopes if available; uses exit code 3 when unauthenticated.
- Logout removes stored token and cached session data.
Acquire and cache an access token using the configured Authority credentials.
```bash
stella auth login
stella auth login --force
```
Notes:
- `--force` ignores cached tokens and forces re-authentication.
- Credential sources are configuration-driven (profile/env). This command does not accept raw tokens on the command line.
### auth status / whoami / logout
```bash
stella auth status
stella auth whoami
stella auth logout
```
Behavior:
- `status` reports whether a cached token exists and whether it is still valid.
- `whoami` prints cached token claims (subject, scopes, expiry) for diagnostics.
- `logout` removes cached tokens for the active credentials.
### auth revoke export / verify
Export or verify Authority revocation bundles.
```bash
stella auth revoke export --output ./revocation-export
stella auth revoke verify --bundle ./revocation-bundle.json --signature ./revocation-bundle.json.jws --key ./authority.pub.pem
```
### auth token mint
Mint a service account token (requires appropriate Authority permissions).
```bash
stella auth token mint --service-account concelier-jobs \
--scope concelier.jobs.trigger --scope advisory:ingest --scope advisory:read \
--tenant tenant-default \
--reason "scheduled ingestion" \
--raw
```
Flags:
- `--service-account` / `-s` (required): service account identifier.
- `--scope` (repeatable): scopes to include in the minted token.
- `--expires-in` (optional): expiry in seconds.
- `--tenant` (optional): tenant context.
- `--reason` (optional): audit reason.
- `--raw`: output only the token value (automation-friendly).
### auth token delegate
Delegate your current token to another principal.
```bash
stella auth token delegate --to user@example.org \
--scope advisory:read \
--tenant tenant-default \
--reason "support session" \
--raw
```
Flags:
- `--to` (required): principal identifier to delegate to.
- `--scope` (repeatable): delegated scopes (must be a subset of the current token).
- `--expires-in` (optional): expiry in seconds (defaults to remaining token lifetime).
- `--tenant` (optional): tenant context.
- `--reason` (required): audit reason.
- `--raw`: output only the token value (automation-friendly).
## Offline notes
- `auth login` and token mint/delegate require connectivity to Authority.
- `auth revoke verify`, `status`, `whoami`, and `logout` can operate using local cached state.
## Offline/air-gap notes
- Login requires network; if `--offline` is set, command must fail with exit code 5.
- Status/logout work offline using cached credentials only.

60
docs/modules/cli/guides/commands/db.md Normal file
View File

@@ -0,0 +1,60 @@
# stella db - Command Guide
The `stella db` command group triggers Concelier database operations via backend jobs (connector stages, merge reconciliation, exports).
These commands are operational: they typically require Authority authentication and appropriate Concelier scopes.
## Commands
### db fetch
Trigger a connector stage (`fetch`, `parse`, or `map`) for a given source.
```bash
stella db fetch --source osv --stage fetch
stella db fetch --source osv --stage parse
stella db fetch --source osv --stage map
```
Options:
- `--source` (required): connector identifier (for example `osv`, `redhat`, `ghsa`).
- `--stage` (optional): `fetch`, `parse`, or `map` (defaults to `fetch`).
- `--mode` (optional): connector-specific mode (for example `init`, `resume`, `cursor`).
### db merge
Run canonical merge reconciliation.
```bash
stella db merge
```
### db export
Run Concelier export jobs.
```bash
stella db export --format json
stella db export --format trivy-db --delta
```
Options:
- `--format` (optional): `json` or `trivy-db` (defaults to `json`).
- `--delta` (optional): request a delta export when supported.
- `--publish-full` / `--publish-delta` (optional): override whether exports are published (true/false).
- `--bundle-full` / `--bundle-delta` (optional): override whether offline bundles include full/delta exports (true/false).
## Common setup
Point the CLI at the Concelier base URL:
```bash
export STELLAOPS_BACKEND_URL="https://concelier.example.internal"
```
Authenticate:
```bash
stella auth login
```
See: `docs/10_CONCELIER_CLI_QUICKSTART.md` and `docs/modules/concelier/operations/authority-audit-runbook.md`.

334
docs/modules/cli/guides/commands/reachability.md
View File

@@ -1,265 +1,157 @@
# stella reachability — Command Guide
# stella reachability - Command Guide
## Overview
The `stella reachability` command group provides reachability analysis capabilities for vulnerability exploitability assessment. It supports call graph upload, analysis listing, and detailed reachability explanations.
The `stella reachability` command group uploads call graphs and queries reachability analyses for vulnerability exploitability assessment.
Typical flow:
1. Generate a call graph locally (optional): `stella scan graph ...`
2. Upload the call graph: `stella reachability upload-callgraph ...`
3. List analyses: `stella reachability list ...`
4. Explain reachability: `stella reachability explain ...`
Notes:
- In multi-tenant environments, pass `--tenant` explicitly.
- Outputs are deterministic: stable ordering, UTC timestamps, and cursor-based paging where applicable.
## Generate a call graph (stella scan graph)
`stella scan graph` extracts a call graph from source code using a language-specific extractor:
- Executable name: `stella-callgraph-<lang>`
- Must be available in `PATH`
Examples:
```bash
# .NET solution
stella scan graph \
--lang dotnet \
--target . \
--sln ./MySolution.sln \
--output ./callgraph.json
# Node.js project (writes JSON to stdout)
stella scan graph \
--lang node \
--target ./service \
--format json > ./callgraph.json
```
Supported `stella scan graph` output formats:
- `json` (default; suitable for upload)
- `dot`
- `summary` (prints only a summary; does not emit graph JSON)
If you generate call graphs with other tooling, uploads support `json`, `proto`, and `dot` formats (or `auto` detection).
## Commands
### Upload Call Graph (CLI-SIG-26-001)
### upload-callgraph
Upload a call graph for reachability analysis.
```bash
# Upload a call graph for reachability analysis
stella reachability upload-callgraph \
--path <call-graph-file> \
[--tenant <id>] \
[--scan-id <id>] \
[--asset-id <id>] \
[--format auto|json|proto|dot] \
[--json]
--path ./callgraph.json \
--scan-id scan-12345 \
--tenant acme \
--format auto
```
**Options:**
Options:
| Flag | Description |
|------|-------------|
| `--path` / `-p` | Path to the call graph file (required) |
| `--scan-id` | Scan identifier to associate with the call graph |
| `--asset-id` / `-a` | Asset identifier to associate with the call graph |
| `--format` / `-f` | Call graph format: `auto` (default), `json`, `proto`, `dot` |
| --- | --- |
| `--path`, `-p` | Path to the call graph file (required). |
| `--scan-id` | Scan identifier to associate with the call graph. |
| `--asset-id`, `-a` | Asset identifier to associate with the call graph. |
| `--format`, `-f` | `auto` (default), `json`, `proto`, `dot`. |
| `--tenant`, `-t` | Tenant identifier (recommended in multi-tenant envs). |
| `--json` | Emit raw JSON payload instead of formatted output. |
**Required:** At least one of `--scan-id` or `--asset-id`.
Required: at least one of `--scan-id` or `--asset-id`.
**Supported Call Graph Formats:**
- JSON (native format)
- Protocol Buffers (proto)
- DOT/GraphViz format
### list
### List Reachability Analyses (CLI-SIG-26-001)
List reachability analyses.
```bash
# List reachability analyses
stella reachability list \
[--tenant <id>] \
[--scan-id <id>] \
[--asset-id <id>] \
[--status pending|processing|completed|failed] \
[--limit <num>] \
[--offset <num>] \
[--json]
--scan-id scan-12345 \
--status completed \
--limit 20 \
--json
```
**Options:**
Options:
| Flag | Description |
|------|-------------|
| `--scan-id` | Filter by scan identifier |
| `--asset-id` / `-a` | Filter by asset identifier |
| `--status` | Filter by analysis status |
| `--limit` / `-l` | Maximum number of results (default 100) |
| `--offset` / `-o` | Pagination offset |
| --- | --- |
| `--scan-id` | Filter by scan identifier. |
| `--asset-id`, `-a` | Filter by asset identifier. |
| `--status` | Filter by status (`pending`, `processing`, `completed`, `failed`). |
| `--limit`, `-l` | Maximum number of results (default 100). |
| `--offset`, `-o` | Pagination offset. |
| `--tenant`, `-t` | Tenant identifier. |
| `--json` | Emit raw JSON payload. |
**Output Columns:**
- Analysis ID
- Asset name/ID
- Status (pending, processing, completed, failed)
- Reachable count
- Unreachable count
- Unknown count
- Created timestamp
### explain
### Explain Reachability (CLI-SIG-26-001)
Explain reachability for a vulnerability ID or a package PURL.
```bash
# Explain reachability for a vulnerability or package
stella reachability explain \
--analysis-id <id> \
[--tenant <id>] \
[--vuln-id <cve-id>] \
[--purl <package-url>] \
[--call-paths] \
[--json]
--analysis-id RA-abc123 \
--vuln-id CVE-2024-1234 \
--call-paths
```
**Options:**
Options:
| Flag | Description |
|------|-------------|
| `--analysis-id` / `-i` | Analysis identifier (required) |
| `--vuln-id` / `-v` | Vulnerability identifier to explain |
| `--purl` | Package URL to explain |
| `--call-paths` | Include detailed call paths in the explanation |
| --- | --- |
| `--analysis-id`, `-i` | Analysis identifier (required). |
| `--vuln-id`, `-v` | Vulnerability identifier to explain. |
| `--purl` | Package URL to explain. |
| `--call-paths` | Include detailed call paths in the explanation. |
| `--tenant`, `-t` | Tenant identifier. |
| `--json` | Emit raw JSON payload. |
**Required:** At least one of `--vuln-id` or `--purl`.
Required: at least one of `--vuln-id` or `--purl`.
**Output:**
- Reachability state (reachable, unreachable, unknown)
- Reachability score (0-1)
- Confidence level
- Reasoning explanation
- Affected functions list
- Call paths (when `--call-paths` is used)
## Integration with Policy Simulation (CLI-SIG-26-002)
## Policy simulation integration
Reachability overrides can be applied during policy simulation:
```bash
stella policy simulate P-7 \
--reachability-state "CVE-2024-1234:unreachable" \
--reachability-state "pkg:npm/lodash@4.17.0:reachable" \
--reachability-state "pkg:npm/lodash@4.17.21:reachable" \
--reachability-score "CVE-2024-5678:0.25"
```
**Override Format:**
- State: `<identifier>:<state>` where state is `reachable`, `unreachable`, `unknown`, or `indeterminate`
- Score: `<identifier>:<score>` where score is a decimal between 0 and 1
Override formats:
- State: `<identifier>:<reachable|unreachable|unknown|indeterminate>`
- Score: `<identifier>:<0..1>`
**Identifier Types:**
- Vulnerability ID: `CVE-XXXX-XXXX`, `GHSA-xxxx-xxxx-xxxx`
- Package URL: `pkg:npm/package@version`, `pkg:maven/group/artifact@version`
Identifier types:
- Vulnerability ID: `CVE-...`, `GHSA-...`
- Package URL: `pkg:...`
## Exit Codes
## Reachability states
| State | Meaning |
| --- | --- |
| `reachable` | Vulnerable code is reachable from entry points. |
| `unreachable` | Vulnerable code is not reachable. |
| `unknown` | Insufficient data to decide. |
| `indeterminate` | Inconclusive (dynamic dispatch/reflection/etc). |
## Exit codes
| Code | Meaning |
|------|---------|
| 0 | Success |
| 1 | Error or upload failure |
| 4 | Input validation error |
| 130 | Operation cancelled by user |
## JSON Schema: ReachabilityExplainResult
```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"analysisId": { "type": "string" },
"vulnerabilityId": { "type": "string" },
"packagePurl": { "type": "string" },
"reachabilityState": {
"type": "string",
"enum": ["reachable", "unreachable", "unknown", "indeterminate"]
},
"reachabilityScore": { "type": "number", "minimum": 0, "maximum": 1 },
"confidence": { "type": "string" },
"reasoning": { "type": "string" },
"callPaths": {
"type": "array",
"items": {
"type": "object",
"properties": {
"pathId": { "type": "string" },
"depth": { "type": "integer" },
"entryPoint": { "$ref": "#/$defs/function" },
"frames": { "type": "array", "items": { "$ref": "#/$defs/function" } },
"vulnerableFunction": { "$ref": "#/$defs/function" }
}
}
},
"affectedFunctions": {
"type": "array",
"items": { "$ref": "#/$defs/function" }
}
},
"$defs": {
"function": {
"type": "object",
"properties": {
"name": { "type": "string" },
"signature": { "type": "string" },
"className": { "type": "string" },
"packageName": { "type": "string" },
"filePath": { "type": "string" },
"lineNumber": { "type": "integer" }
}
}
}
}
```
## Examples
### Upload a call graph
```bash
# Upload call graph for a specific scan
stella reachability upload-callgraph \
--path ./callgraph.json \
--scan-id scan-12345 \
--format json
# Upload with auto-detection
stella reachability upload-callgraph \
--path ./app-callgraph.dot \
--asset-id my-application
```
### List recent analyses
```bash
# List all completed analyses for an asset
stella reachability list \
--asset-id my-application \
--status completed \
--json
# List analyses with pagination
stella reachability list \
--limit 20 \
--offset 40
```
### Explain vulnerability reachability
```bash
# Explain with call paths
stella reachability explain \
--analysis-id RA-abc123 \
--vuln-id CVE-2024-1234 \
--call-paths
# Explain package reachability
stella reachability explain \
--analysis-id RA-abc123 \
--purl "pkg:npm/lodash@4.17.21" \
--json
```
### Policy simulation with reachability overrides
```bash
# Mark specific vulnerability as unreachable
stella policy simulate P-7 \
--reachability-state "CVE-2024-1234:unreachable" \
--explain
# Set low reachability score
stella policy simulate P-7 \
--reachability-score "pkg:npm/axios@0.21.0:0.1"
```
## Reachability States
| State | Description |
|-------|-------------|
| `reachable` | Vulnerable code is reachable from application entry points |
| `unreachable` | Vulnerable code cannot be reached during execution |
| `unknown` | Reachability cannot be determined with available information |
| `indeterminate` | Analysis inconclusive due to dynamic dispatch or reflection |
## Call Graph Generation
Call graphs can be generated using various tools:
- **Java:** [WALA](https://github.com/wala/WALA), [Soot](https://github.com/soot-oss/soot)
- **JavaScript/Node.js:** [callgraph](https://www.npmjs.com/package/callgraph)
- **Python:** [pycg](https://github.com/vitsalis/pycg)
- **Go:** `go build -gcflags="-m"` + static analysis
- **C/C++:** [LLVM](https://llvm.org/) call graph pass
## Best Practices
1. **Upload call graphs after each build** to maintain accurate reachability data
2. **Use asset IDs** for long-lived applications to track reachability changes over time
3. **Include call paths** when debugging unexpected reachability results
4. **Apply reachability overrides** in policy simulation to model remediation scenarios
5. **Monitor unreachable counts** as a metric for dependency hygiene
| --- | --- |
| `0` | Success. |
| `1` | Command failed. |
| `4` | Input validation error. |
| `130` | Operation cancelled. |

219
docs/modules/cli/guides/commands/vuln.md
View File

@@ -1,25 +1,206 @@
# stella vuln — Command Guide
# stella vuln - Command Guide
## Overview
The `stella vuln` command group is the operator surface for vulnerability triage workflows:
- Query and inspect vulnerabilities (`list`, `show`)
- Retrieve raw advisory observations for overlay consumers (`observations`)
- Apply workflow actions (assign, comment, accept risk, verify fix, target fix, reopen)
- Simulate policy/VEX changes (`simulate`)
- Export offline evidence bundles and verify them (`export`, `export verify`)
Unless otherwise noted, commands support formatted output by default and `--json` for automation-friendly output.
## Commands
- `stella vuln list --query <filter> [--group-by <field>] [--output json|ndjson|table] [--offline]`
- `stella vuln get --id <vulnId> [--output json|table] [--offline]`
- `stella vuln simulate --from <policyA> --to <policyB> --subjects <path> [--offline]`
## Flags (common)
- `--offline`: read from cached snapshots; fail with exit code 5 if network would be used.
- `--policy <id>`: scope queries to a policy projection.
- `--page-size`, `--page-token`: deterministic pagination.
- `--group-by`: `cve`, `package`, `status`, `advisory` (results stay stably ordered within groups).
### observations
## Inputs/outputs
- Inputs: Vuln Explorer API; optional cached snapshots when offline.
- Outputs: sorted lists or detail documents with provenance pointers (`advisoryId`, `evidenceIds`, `consensusId`).
- Exit codes follow `output-and-exit-codes.md`; 4 for not found, 5 for offline violation.
List raw advisory observations (useful for overlay consumers).
## Determinism rules
- Lists sorted by primary key then timestamp; group-by keeps stable ordering inside each bucket.
- Timestamps UTC ISO-8601; hashes lower-case hex.
```bash
stella vuln observations \
--tenant acme \
--alias CVE-2024-1234 \
--limit 50 \
--json
```
## Offline/air-gap notes
- Use cached snapshots (`--offline`) when remote Explorer is unavailable; commands must not attempt network calls in this mode.
- Simulation must read local policy snapshots and subjects when offline.
Options:
| Flag | Description |
| --- | --- |
| `--tenant` | Tenant identifier (required). |
| `--observation-id` | Filter by observation id (repeatable). |
| `--alias` | Filter by vulnerability alias (repeatable). |
| `--purl` | Filter by Package URL (repeatable). |
| `--cpe` | Filter by CPE value (repeatable). |
| `--limit` | Max items (default 200, max 500). |
| `--cursor` | Opaque cursor token from a previous page. |
| `--json` | Emit raw JSON payload instead of a table. |
### list
List vulnerabilities with filters, grouping, and pagination.
```bash
stella vuln list \
--severity high \
--status open \
--group-by package \
--limit 50
```
Options:
| Flag | Description |
| --- | --- |
| `--vuln-id` | Filter by vulnerability identifier (e.g., `CVE-2024-1234`). |
| `--severity` | Filter by severity (`critical`, `high`, `medium`, `low`). |
| `--status` | Filter by status (`open`, `triaged`, `accepted`, `fixed`, etc.). |
| `--purl` | Filter by Package URL. |
| `--cpe` | Filter by CPE value. |
| `--sbom-id` | Filter by SBOM identifier. |
| `--policy-id` | Filter by policy identifier. |
| `--policy-version` | Filter by policy version. |
| `--group-by` | Group by (`vuln`, `package`, `severity`, `status`). |
| `--limit` | Max items (default 50, max 500). |
| `--offset` | Offset paging (skip N). |
| `--cursor` | Opaque cursor token from a previous page. |
| `--tenant` | Tenant identifier (overrides profile/environment). |
| `--json` | Emit raw JSON payload instead of a table. |
| `--csv` | Emit CSV instead of a table. |
### show
Show details for a specific vulnerability id.
```bash
stella vuln show CVE-2024-1234 --json
```
Options:
| Flag | Description |
| --- | --- |
| `--tenant` | Tenant identifier (overrides profile/environment). |
| `--json` | Emit raw JSON payload instead of formatted output. |
### Workflow commands
Workflow commands operate on either:
- Explicit selection: repeat `--vuln-id <id>`
- Filtered selection: `--filter-*` options
Shared options:
| Flag | Description |
| --- | --- |
| `--vuln-id` | Vulnerability ids to operate on (repeatable). |
| `--filter-severity` | Filter by severity (`critical`, `high`, `medium`, `low`). |
| `--filter-status` | Filter by current status. |
| `--filter-purl` | Filter by Package URL. |
| `--filter-sbom` | Filter by SBOM id. |
| `--tenant` | Tenant identifier (overrides profile/environment). |
| `--idempotency-key` | Idempotency key for retry-safe operations. |
| `--json` | Emit raw JSON response. |
Commands:
```bash
# Assign selected vulns to an assignee
stella vuln assign alice@example.com --vuln-id CVE-2024-1234
# Add a comment
stella vuln comment "triage started" --vuln-id CVE-2024-1234
# Accept risk with optional due date (ISO-8601)
stella vuln accept-risk "risk accepted for legacy system" \
--due-date 2026-12-31 \
--vuln-id CVE-2024-1234
# Mark as fixed and verified
stella vuln verify-fix \
--fix-version 1.2.3 \
--comment "patched in release 1.2.3" \
--vuln-id CVE-2024-1234
# Set a target fix date
stella vuln target-fix 2026-12-31 \
--comment "scheduled in next maintenance window" \
--vuln-id CVE-2024-1234
# Reopen a previously closed/accepted vuln
stella vuln reopen --comment "regression observed" --vuln-id CVE-2024-1234
```
### simulate
Simulate policy/VEX changes and show delta summaries.
```bash
stella vuln simulate \
--policy-id policy://tenant-default/runtime-hardening \
--policy-version 7 \
--vex-override "CVE-2024-1234=not_affected" \
--severity-threshold high \
--sbom-id sbom-001 \
--markdown \
--output ./vuln-sim-report.md
```
Options:
| Flag | Description |
| --- | --- |
| `--policy-id` | Policy id to simulate (uses different version or a new policy). |
| `--policy-version` | Policy version to simulate against. |
| `--vex-override` | VEX status overrides (`vulnId=status`, repeatable). |
| `--severity-threshold` | Threshold (`critical`, `high`, `medium`, `low`). |
| `--sbom-id` | SBOM ids to include (repeatable). |
| `--markdown` | Include Markdown report suitable for CI pipelines. |
| `--changed-only` | Only show items that changed. |
| `--tenant` | Tenant identifier for multi-tenant environments. |
| `--json` | Output JSON for automation. |
| `--output` | Write Markdown report to file instead of stdout. |
### export
Export vulnerability evidence bundles (optionally signed) for offline review and audit workflows.
```bash
stella vuln export \
--vuln-id CVE-2024-1234 \
--format ndjson \
--output ./vuln-export.ndjson
```
Options:
| Flag | Description |
| --- | --- |
| `--vuln-id` | Vulnerability ids to include (repeatable). |
| `--sbom-id` | SBOM ids to include (repeatable). |
| `--policy-id` | Policy id for export filtering. |
| `--format` | `ndjson` (default) or `json`. |
| `--include-evidence` | Include evidence data (default: true). |
| `--include-ledger` | Include workflow ledger (default: true). |
| `--signed` | Request signed export bundle (default: true). |
| `--output` | Output file path for the export bundle (required). |
| `--tenant` | Tenant identifier for multi-tenant environments. |
### export verify
Verify signature and digest of an exported vulnerability bundle.
```bash
stella vuln export verify ./vuln-export.ndjson \
--expected-digest sha256:deadbeef... \
--public-key ./authority-public.pem
```
Options:
| Flag | Description |
| --- | --- |
| `--expected-digest` | Expected digest to verify (`sha256:<hex>`). |
| `--public-key` | Public key path for signature verification. |

82
docs/modules/cli/guides/graph-and-vuln.md
View File

@@ -1,82 +0,0 @@
# CLI Guide · Graph & Vulnerability
Status: Draft (2025-11-26) — reflects current Graph API surface; Vuln API pending (see docs/api/vuln.md).
## Prereqs
- `stellaops-cli` built from current repo.
- Auth token with scopes: `graph:read`, `graph:query`, `graph:export` (and `vuln:read` when Vuln API lands).
- Tenant header is required for all calls (`--tenant`).
## Commands
### Search graph
```bash
stella graph search \
--tenant acme \
--kinds component \
--query "pkg:npm/" \
--limit 20 \
--format ndjson
```
Outputs NDJSON tiles (`node`, optional `cursor`).
### Query graph with budgets
```bash
stella graph query \
--tenant acme \
--kinds component \
--query widget \
--include-edges \
--budget-tiles 200 \
--budget-nodes 150 \
--budget-edges 100 \
--format ndjson
```
Returns nodes/edges/stats/cursor with cost metadata. If edge budget is exceeded, an `error` tile with `GRAPH_BUDGET_EXCEEDED` is returned.
### Paths
```bash
stella graph paths \
--tenant acme \
--source gn:acme:component:one \
--target gn:acme:component:two \
--max-depth 4 \
--include-edges \
--format ndjson
```
### Diff snapshots
```bash
stella graph diff \
--tenant acme \
--snapshot-a snapA \
--snapshot-b snapB \
--include-edges \
--format ndjson
```
### Export graph
```bash
stella graph export \
--tenant acme \
--format ndjson \
--include-edges \
--kinds component \
--query "pkg:npm/" \
--out graph-export.ndjson
```
Manifest is returned with jobId, sha256, size, and download URL; CLI downloads to `--out`.
### Vuln lookup (placeholder)
Pending Vuln Explorer API; for now use graph search/query to inspect impacted components. See `docs/api/vuln.md`.
## Exit codes
- `0` success
- `2` validation error (missing kinds/tenant/scopes)
- `3` budget exceeded (error tile received)
- `4` network/auth failure
## Tips
- Add `--format table` (when available) for quick summaries; default is NDJSON.
- Use `--cursor` from a previous call to page results.
- To sample overlays, pass `--include-overlays` (budget permitting).

20
docs/modules/cli/prep/2025-11-20-ops-0001-prep.md
View File

@@ -1,20 +0,0 @@
# CLI Ops Prep — PREP-CLI-OPS-0001
Status: **Ready for implementation** (2025-11-20)
Owners: Ops Guild
Scope: Capture required demo outputs and runbook deltas for the next CLI ops demo so CLI-OPS-0001 can proceed.
## Required demo outputs
- Latest CLI binary build identifier (commit SHA and version string).
- Demo script transcript covering: `stella evidence verify`, `stella attest bundle verify`, `stella export ...` (airgap profile), and `stella policy lint`.
- Screenshots or asciinema recording showing: auth flow with offline token, evidence verification success, attestation failure path.
- Hashes for demo artefacts (bundles, logs) placed under `out/cli/demo-ops/` with `.sha256` files.
## Runbook updates expected
- Update `docs/modules/cli/operations/cli-ops-runbook.md` (section “Offline Demo”) with: prerequisites, commands, expected outputs, and rollback steps.
- Add checklist to ensure `STELLA_CLI_OFFLINE=1` and local evidence bundle cache populated before demo.
## Acceptance criteria
- Demo outputs (recording + hashes) published under `out/cli/demo-ops/` with SHA256 files.
- Runbook updated per above; references the exact CLI build SHA used.
- Ops Guild signs off that CLI-OPS-0001 can move to implementation.

29
docs/modules/concelier/operations/authority-audit-runbook.md
View File

@@ -120,15 +120,18 @@ Correlate audit logs with the following global meter exported via `Concelier.Sou
## 4. Rollout & Verification Procedure
1. **Pre-checks**
- Align with the rollout phases documented in `docs/10_CONCELIER_CLI_QUICKSTART.md` (validation → rehearsal → enforced) and record the target dates in your change request.
- Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation.
- Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host).
2. **Smoke test with valid token**
- Obtain a token via CLI: `stella auth login --scope "concelier.jobs.trigger advisory:ingest" --scope advisory:read`.
- Trigger a read-only endpoint: `curl -H "Authorization: Bearer $TOKEN" https://concelier.internal/jobs/definitions`.
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger advisory:ingest advisory:read`, and `tenant=tenant-default`.
1. **Pre-checks**
- Align with your rollout plan and record the target dates in your change request.
- Confirm `allowAnonymousFallback` is `false` in production; keep `true` only during staged validation.
- Validate Authority issuer metadata is reachable from Concelier (`curl https://authority.internal/.well-known/openid-configuration` from the host).
2. **Smoke test with valid token**
- Authenticate (cached): `stella auth login`.
- Mint a scoped token for curl (example):
- `TOKEN="$(stella auth token mint --service-account concelier-jobs --scope concelier.jobs.trigger --scope advisory:ingest --scope advisory:read --tenant tenant-default --reason \"concelier auth smoke test\" --raw)"`
- Trigger a read-only endpoint:
- `curl -H "Authorization: Bearer $TOKEN" -H "X-Stella-Tenant: tenant-default" https://concelier.internal/jobs/definitions`
- Expect HTTP 200/202 and an audit log with `bypass=False`, `scopes=concelier.jobs.trigger advisory:ingest advisory:read`, and `tenant=tenant-default`.
3. **Negative test without token**
- Call the same endpoint without a token. Expect HTTP 401, `bypass=False`.
@@ -153,7 +156,7 @@ Correlate audit logs with the following global meter exported via `Concelier.Sou
## 6. References
- `docs/21_INSTALL_GUIDE.md` – Authority configuration quick start.
- `docs/17_SECURITY_HARDENING_GUIDE.md` – Security guardrails and enforcement deadlines.
- `docs/modules/authority/operations/monitoring.md` – Authority-side monitoring and alerting playbook.
- `StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` – source of audit log fields.
- `docs/21_INSTALL_GUIDE.md` - Authority configuration quick start.
- `docs/17_SECURITY_HARDENING_GUIDE.md` - Security guardrails and enforcement.
- `docs/modules/authority/operations/monitoring.md` - Authority-side monitoring and alerting playbook.
- `src/Concelier/StellaOps.Concelier.WebService/Filters/JobAuthorizationAuditFilter.cs` - Source of audit log fields.

6
docs/modules/concelier/operations/conflict-resolution.md
View File

@@ -45,7 +45,7 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
- `eventId=1002` with `reason="equal_rank"` - indicates precedence table gaps; page merge owners.
- `eventId=1002` with `reason="mismatch"` - severity disagreement; open connector bug if sustained.
3. **Job health**
- `stellaops-cli db merge` exit code `1` signifies unresolved conflicts. Pipe to automation that captures logs and notifies #concelier-ops.
- `stella db merge` exit code `1` signifies unresolved conflicts. Pipe to automation that captures logs and notifies #concelier-ops.
### Threshold updates (2025-10-12)
@@ -58,7 +58,7 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
## 4. Triage Workflow
1. **Confirm job context**
- `stellaops-cli db merge` (CLI) or `POST /jobs/merge:reconcile` (API) to rehydrate the merge job. Use `--verbose` to stream structured logs during triage.
- `stella db merge` (CLI) or `POST /jobs/merge:reconcile` (API) to rehydrate the merge job. Use `--verbose` to stream structured logs during triage.
2. **Inspect metrics**
- Correlate spikes in `concelier.merge.conflicts` with `primary_source`/`suppressed_source` tags from `concelier.merge.overrides`.
3. **Pull structured logs**
@@ -94,7 +94,7 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
## 6. Resolution Playbook
1. **Connector data fix**
- Re-run the offending connector stages (`stellaops-cli db fetch --source ghsa --stage map` etc.).
- Re-run the offending connector stages (`stella db fetch --source ghsa --stage map` etc.).
- Once fixed, rerun merge and verify `decisionReason` reflects `freshness` or `precedence` as expected.
2. **Temporary precedence override**
- Edit `etc/concelier.yaml`:

14
docs/modules/concelier/operations/connectors/apple.md
View File

@@ -25,13 +25,13 @@ concelier:
## 2. Staging Smoke Test
1. Deploy the configuration and restart the Concelier workers to ensure the Apple connector options are bound.
2. Trigger a full connector cycle:
- CLI: `stella db jobs run source:vndr-apple:fetch --and-then source:vndr-apple:parse --and-then source:vndr-apple:map`
- REST: `POST /jobs/run { "kind": "source:vndr-apple:fetch", "chain": ["source:vndr-apple:parse", "source:vndr-apple:map"] }`
3. Validate metrics exported under meter `StellaOps.Concelier.Connector.Vndr.Apple`:
- `apple.fetch.items` (documents fetched)
- `apple.fetch.failures`
1. Deploy the configuration and restart the Concelier workers to ensure the Apple connector options are bound.
2. Trigger a full connector cycle:
- CLI: run `stella db fetch --source vndr-apple --stage fetch`, then `--stage parse`, then `--stage map`.
- REST: `POST /jobs/run { "kind": "source:vndr-apple:fetch", "chain": ["source:vndr-apple:parse", "source:vndr-apple:map"] }`
3. Validate metrics exported under meter `StellaOps.Concelier.Connector.Vndr.Apple`:
- `apple.fetch.items` (documents fetched)
- `apple.fetch.failures`
- `apple.fetch.unchanged`
- `apple.parse.failures`
- `apple.map.affected.count` (histogram of affected package counts)

2
docs/modules/concelier/operations/connectors/cccs.md
View File

@@ -53,7 +53,7 @@ Suggested Grafana alerts:
2. **Stage ingestion**:
- Temporarily raise `maxEntriesPerFetch` (e.g. 500) and restart Concelier workers.
- Run chained jobs until `pendingDocuments` drains:
`stella db jobs run source:cccs:fetch --and-then source:cccs:parse --and-then source:cccs:map`
Run `stella db fetch --source cccs --stage fetch`, then `--stage parse`, then `--stage map`.
- Monitor `cccs.fetch.unchanged` growth; once it approaches dataset size the backfill is complete.
3. **Optional pagination sweep** – for incremental mirrors, iterate `page=<n>` (0…N) while `response.Count == 50`, persisting JSON to disk. Store alongside metadata (`language`, `page`, SHA256) so repeated runs detect drift.
4. **Language split** – keep EN/FR payloads separate to preserve canonical language fields. The connector emits `Language` directly from the feed entry, so mixed ingestion simply produces parallel advisories keyed by the same serial number.

2
docs/modules/concelier/operations/connectors/certbund.md
View File

@@ -124,7 +124,7 @@ operating offline.
### 3.4 Connector-driven catch-up
1. Temporarily raise `maxAdvisoriesPerFetch` (e.g. 150) and reduce `requestDelay`.
2. Run `stella db jobs run source:cert-bund:fetch --and-then source:cert-bund:parse --and-then source:cert-bund:map` until the fetch log reports `enqueued=0`.
2. Run `stella db fetch --source cert-bund --stage fetch`, then `--stage parse`, then `--stage map` until the fetch log reports `enqueued=0`.
3. Restore defaults and capture the cursor snapshot for audit.
---

2
docs/modules/concelier/operations/connectors/cisco.md
View File

@@ -33,7 +33,7 @@ This runbook describes how Ops provisions, rotates, and distributes Cisco PSIRT
- Update `concelier:sources:cisco:auth` (or the module-specific secret template) with the stored credentials.
- For Offline Kit delivery, export encrypted secrets into `offline-kit/secrets/cisco-openvuln.json` using the platform’s sealed secret format.
4. **Connectivity validation**
- From the Concelier control plane, run `stella db jobs run source:vndr-cisco:fetch --dry-run`.
- From the Concelier control plane, run `stella db fetch --source vndr-cisco --stage fetch` (use staging or a controlled window).
- Ensure the Source HTTP diagnostics record `Bearer` authorization headers and no 401/403 responses.
## 4. Rotation SOP

4
docs/modules/concelier/operations/connectors/cve-kev.md
View File

@@ -34,7 +34,7 @@ concelier:
1. Deploy the updated configuration and restart the Concelier service so the connector picks up the credentials.
2. Trigger one end-to-end cycle:
- Concelier CLI: `stella db jobs run source:cve:fetch --and-then source:cve:parse --and-then source:cve:map`
- Concelier CLI: run `stella db fetch --source cve --stage fetch`, then `--stage parse`, then `--stage map`.
- REST fallback: `POST /jobs/run { "kind": "source:cve:fetch", "chain": ["source:cve:parse", "source:cve:map"] }`
3. Observe the following metrics (exported via OTEL meter `StellaOps.Concelier.Connector.Cve`):
- `cve.fetch.attempts`, `cve.fetch.success`, `cve.fetch.documents`, `cve.fetch.failures`, `cve.fetch.unchanged`
@@ -107,7 +107,7 @@ Treat repeated schema failures or growing anomaly counts as an upstream regressi
1. Deploy the configuration and restart Concelier.
2. Trigger a pipeline run:
- CLI: `stella db jobs run source:kev:fetch --and-then source:kev:parse --and-then source:kev:map`
- CLI: run `stella db fetch --source kev --stage fetch`, then `--stage parse`, then `--stage map`.
- REST: `POST /jobs/run { "kind": "source:kev:fetch", "chain": ["source:kev:parse", "source:kev:map"] }`
3. Verify the metrics exposed by meter `StellaOps.Concelier.Connector.Kev`:
- `kev.fetch.attempts`, `kev.fetch.success`, `kev.fetch.unchanged`, `kev.fetch.failures`

2
docs/modules/concelier/operations/connectors/epss.md
View File

@@ -24,7 +24,7 @@ concelier:
1. Restart Concelier workers after configuration changes.
2. Trigger a full cycle:
- CLI: `stella db jobs run source:epss:fetch --and-then source:epss:parse --and-then source:epss:map`
- CLI: run `stella db fetch --source epss --stage fetch`, then `--stage parse`, then `--stage map`.
- REST: `POST /jobs/run { "kind": "source:epss:fetch", "chain": ["source:epss:parse", "source:epss:map"] }`
3. Verify document status transitions: `pending_parse` -> `pending_map` -> `mapped`.
4. Confirm log entries for `Fetched EPSS snapshot` and parse/map summaries.

2
docs/modules/concelier/operations/connectors/ics-cisa.md
View File

@@ -79,7 +79,7 @@ If credentials are still pending, populate the connector with the community CSV
```bash
CONCELIER_SOURCES_ICSCISA_GOVDELIVERY_CODE=... \
CONCELIER_SOURCES_ICSCISA_ENABLEDETAILSCRAPE=1 \
stella db jobs run source:ics-cisa:fetch --and-then source:ics-cisa:parse --and-then source:ics-cisa:map
Run `stella db fetch --source ics-cisa --stage fetch`, then `--stage parse`, then `--stage map`.
```
3. Confirm logs contain `ics-cisa detail fetch` entries and that new documents/DTOs include attachments (see `docs/artifacts/icscisa`). Canonical advisories should expose PDF links as `references.kind == "attachment"` and affected packages should surface `primitives.semVer.exactValue` for single-version hits.
4. If Akamai blocks direct fetches, set `concelier:sources:icscisa:proxyUri` to your allow-listed egress proxy and rerun the dry-run.

2
docs/modules/concelier/operations/connectors/kisa.md
View File

@@ -25,7 +25,7 @@ concelier:
1. Restart the Concelier workers so the KISA options bind.
2. Run a full connector cycle:
- CLI: `stella db jobs run source:kisa:fetch --and-then source:kisa:parse --and-then source:kisa:map`
- CLI: run `stella db fetch --source kisa --stage fetch`, then `--stage parse`, then `--stage map`.
- REST: `POST /jobs/run { "kind": "source:kisa:fetch", "chain": ["source:kisa:parse", "source:kisa:map"] }`
3. Confirm telemetry (Meter `StellaOps.Concelier.Connector.Kisa`):
- `kisa.feed.success`, `kisa.feed.items`

22
docs/modules/concelier/prep/2025-11-20-airgap-56-001-58-001-prep.md
View File

@@ -1,22 +0,0 @@
# Concelier AirGap Prep — PREP-CONCELIER-AIRGAP-56-001-58-001
Status: **Ready for implementation** (2025-11-20)
Owners: Concelier Core · AirGap Guilds
Scope: Chain mirror thin-bundle milestone with EvidenceLocker bundle references and console consumption to unblock air-gapped Concelier workflows (56-001..58-001).
## Inputs
- Mirror milestone-0 thin bundle: `out/mirror/thin/mirror-thin-m0-sample.tar.gz` (hash documented in PREP-ART-56-001).
- Evidence bundle v1 contract: `docs/modules/evidence-locker/evidence-bundle-v1.md`.
- Console fixtures (29-001, 30-001) and LNM schema freeze.
## Deliverables
- Publish mapping note `docs/modules/concelier/prep/airgap-56-001-58-001-mapping.md` covering:
- Bundle locations/hashes (thin + evidence).
- Import commands for Concelier offline controller.
- Deterministic ordering and retention expectations.
- Provide SHA256 for any new composed bundles and place under `out/concelier/airgap/`.
## Acceptance criteria
- Mapping note published with hashes and import commands.
- No unresolved schema decisions remain for air-gap import chain.

17
docs/modules/concelier/prep/2025-11-20-attest-73-001-prep.md
View File

@@ -1,17 +0,0 @@
# Concelier Attestation Prep — PREP-CONCELIER-ATTEST-73-001-002
Status: **Ready for implementation** (2025-11-20)
Owners: Concelier Core · Evidence Locker Guild
Scope: Evidence Locker attestation scope integration for Concelier attest tasks 73-001/002.
## Requirements
- Use Evidence Locker attestation scope note: `docs/modules/evidence-locker/attestation-scope-note.md`.
- Bind Evidence Bundle v1 contract: `docs/modules/evidence-locker/evidence-bundle-v1.md`.
## Deliverables
- Concelier-specific attestation ingest note at `docs/modules/concelier/prep/attest-73-001-ingest.md` describing required claims, DSSE expectations, and lookup flow.
- Hashes for sample attest bundles reused from Evidence Locker sample; no new artefacts needed.
## Acceptance criteria
- Ingest note published with claim set and DSSE requirements; Concelier tasks can proceed without further schema questions.

17
docs/modules/concelier/prep/2025-11-20-console-23-001-prep.md
View File

@@ -1,17 +0,0 @@
# Concelier Console Prep — PREP-CONCELIER-CONSOLE-23-001-003
Status: **Ready for implementation** (2025-11-20)
Owners: Concelier Console Guild
Scope: Console schema samples and evidence bundle references for console consumption of linkset/VEX data (23-001..003).
## Deliverables
- JSON samples placed under `docs/samples/console/`:
- `console-linkset-search.json` (frozen LNM schema, includes pagination + filters).
- `console-vex-search.json` (VEX linkset search with exploitability flags).
- Hashes `.sha256` for each sample.
- README snippet added to `docs/samples/console/README.md` describing schema version, seed (`2025-01-01T00:00:00Z`), and deterministic ordering.
## Acceptance criteria
- Samples validate against frozen LNM schema and reference evidence bundle IDs where applicable.
- Hashes recorded; no external dependencies.

20
docs/modules/concelier/prep/2025-11-20-feeds-icscisa-kisa-prep.md
View File

@@ -1,20 +0,0 @@
# Concelier Feed Prep — PREP-FEEDCONN-ICSCISA-02-012-KISA-02-008-FEED
Status: **Ready for implementation** (2025-11-20)
Owners: Concelier Feed Owners
Scope: Remediation plan and schema notes for ICSCISA/KISA feeds to unblock connector work.
## Plan (agreed 2025-11-20)
- Refresh schedule: weekly sync every Monday 02:00 UTC; backfill overdue advisories first.
- Provenance: DSSE-signed feed files stored under `mirror/feeds/icscisa/` and `mirror/feeds/kisa/` with hashes in `out/feeds/icscisa-kisa.sha256`.
- Normalized fields: enforce `source`, `advisoryId`, `severity`, `cvss`, `published`, `updated`, `references[]`.
- Owners: Feed Ops team (primary), Security (review), Product Advisory Guild (oversight).
## Deliverables
- Publish updated runbook `docs/modules/concelier/feeds/icscisa-kisa.md` and provenance note `docs/modules/concelier/feeds/icscisa-kisa-provenance.md` (already exist; confirm hashes and schedule lines).
- Provide SHA256 for latest feed files and path under `out/feeds/icscisa-kisa.sha256`.
## Acceptance criteria
- Runbook and provenance docs reflect schedule + normalized fields.
- Hash file published for latest feed drop; connector work unblocked.

72
docs/modules/concelier/prep/2025-11-20-orchestrator-registry-prep.md
View File

@@ -1,72 +0,0 @@
# Concelier · Orchestrator Registry & Control Prep
- **Date:** 2025-11-20
- **Scope:** PREP-CONCELIER-ORCH-32-001, PREP-CONCELIER-ORCH-32-002, PREP-CONCELIER-ORCH-33-001, PREP-CONCELIER-ORCH-34-001
- **Working directory:** `src/Concelier/**` (WebService, Core, Storage.Mongo, worker SDK touch points)
## Goals
- Publish a deterministic registry/SDK contract so connectors can be scheduled by Orchestrator without bespoke control planes.
- Define heartbeats/progress envelopes and pause/throttle/backfill semantics ahead of worker wiring.
- Describe replay/backfill evidence outputs so ledger/export work can rely on stable hashes.
## Registry record (authoritative fields)
All registry documents live under the orchestrator collection keyed by `connectorId` (stable slug). Fields and invariants:
- `connectorId` (string, slug, lowercase) — unique per tenant + source; immutable.
- `tenant` (string) — required; enforced by WebService tenant guard.
- `source` (enum) — advisory provider (`nvd`, `ghsa`, `osv`, `icscisa`, `kisa`, `vendor:<slug>`).
- `capabilities` (array) — `observations`, `linksets`, `timeline`, `attestations` flags; no merge/derived data.
- `authRef` (string) — reference to secrets store key; never inlined.
- `schedule` (object) — `cron`, `timeZone`, `maxParallelRuns`, `maxLagMinutes`.
- `ratePolicy` (object) — `rpm`, `burst`, `cooldownSeconds`; default deny if absent.
- `artifactKinds` (array) — `raw-advisory`, `normalized`, `linkset`, `timeline`, `attestation`.
- `lockKey` (string) — deterministic lock namespace (`concelier:{tenant}:{connectorId}`) for single-flight.
- `egressGuard` (object) — `allowlist` of hosts + `airgapMode` boolean; fail closed when `airgapMode=true` and host not allowlisted.
- `createdAt` / `updatedAt` (ISO-8601 UTC) — monotonic; updates require optimistic concurrency token.
### Registry sample (non-normative)
```json
{
"connectorId": "icscisa",
"tenant": "acme",
"source": "icscisa",
"capabilities": ["observations", "linksets", "timeline"],
"authRef": "secret:concelier/icscisa/api-key",
"schedule": {"cron": "*/30 * * * *", "timeZone": "UTC", "maxParallelRuns": 1, "maxLagMinutes": 120},
"ratePolicy": {"rpm": 60, "burst": 10, "cooldownSeconds": 30},
"artifactKinds": ["raw-advisory", "normalized", "linkset"],
"lockKey": "concelier:acme:icscisa",
"egressGuard": {"allowlist": ["icscert.kisa.or.kr"], "airgapMode": true},
"createdAt": "2025-11-20T00:00:00Z",
"updatedAt": "2025-11-20T00:00:00Z"
}
```
## Control/SDK contract (heartbeats + commands)
- Heartbeat endpoint `POST /internal/orch/heartbeat` (auth: internal orchestrator role, tenant-scoped).
- Body: `connectorId`, `runId` (GUID), `status` (`starting|running|paused|throttled|backfill|failed|succeeded`),
`progress` (0–100), `queueDepth`, `lastArtifactHash`, `lastArtifactKind`, `errorCode`, `retryAfterSeconds`.
- Idempotency key: `runId` + `sequence` to preserve ordering; orchestrator ignores stale sequence.
- Control queue document (persisted per run):
- Commands: `pause`, `resume`, `throttle` (rpm/burst override until `expiresAt`), `backfill` (range: `fromCursor`/`toCursor`).
- Workers poll `/internal/orch/commands?connectorId={id}&runId={runId}`; must ack with monotonic `ackSequence` to ensure replay safety.
- Failure semantics: on `failed`, worker emits `errorCode`, `errorReason`, `lastCheckpoint` (cursor/hash). Orchestrator may re-enqueue with backoff.
## Backfill/replay expectations
- Backfill command requires deterministic cursor space (e.g., advisory sequence number or RFC3339 timestamp truncated to minutes).
- Worker must emit a `runManifest` per backfill containing: `runId`, `connectorId`, `tenant`, `cursorRange`, `artifactHashes[]`, `dsseEnvelopeHash` (if attested), `completedAt`.
- Manifests are written to Evidence Locker ledger for replay; filenames: `backfill/{tenant}/{connectorId}/{runId}.ndjson` with stable ordering.
## Telemetry (to implement in WebService + worker SDK)
- Meter name prefix: `StellaOps.Concelier.Orch`.
- Counters:
- `concelier.orch.heartbeat` tags: `tenant`, `connectorId`, `status`.
- `concelier.orch.command.applied` tags: `tenant`, `connectorId`, `command`.
- Histograms:
- `concelier.orch.lag.minutes` (now - cursor upper bound) tags: `tenant`, `connectorId`.
- Logs: structured with `tenant`, `connectorId`, `runId`, `command`, `sequence`, `ackSequence`.
## Acceptance criteria for prep completion
- Registry/command schema above is frozen and referenced from Sprint 0114 Delivery Tracker (P10–P13) so downstream implementation knows shapes.
- Sample manifest path + naming are defined for ledger/replay flows.
- Meter names/tags enumerated for observability wiring.

42
docs/modules/concelier/prep/2025-11-20-platform-events-and-lnm-21-002.md
View File

@@ -1,42 +0,0 @@
# Concelier PREP Notes — 2025-11-20
Owner: Concelier Core Guild · Scheduler Guild · Data Science Guild
Scope: Provide traceable prep outputs for PREP-CONCELIER-GRAPH-21-002-PLATFORM-EVENTS-S and PREP-CONCELIER-LNM-21-002-WAITING-ON-FINALIZE so downstream tasks can proceed without blocking on missing contracts.
## 1) `sbom.observation.updated` platform event (Graph-21-002)
- Goal: publish deterministic, facts-only observation updates for graph overlays; no derived judgments.
- Proposed envelope (draft for Scheduler/Platform Events review):
- `event_type`: `sbom.observation.updated`
- `tenant_id` (string, required)
- `advisory_ids` (array of strings; upstream IDs as-ingested)
- `observation_ids` (array of stable per-observation IDs emitted by LNM storage)
- `source` (string; advisory source slug)
- `version_range` (string; original upstream semantics)
- `occurred_at` (ISO-8601 UTC, produced by Concelier at write time; deterministic)
- `trace` (object; optional provenance pointers, DSSE envelope digest with alg/id fields)
- Delivery and wiring expectations:
- Publisher lives in `StellaOps.Concelier.Core` after linkset/observation persistence.
- Scheduler binding: NATS/Redis topic `concelier.sbom.observation.updated`; ack + idempotent replay friendly; max delivery once semantics via message ID = `<tenant>:<observation_id>::<digest>`.
- Telemetry: counter `concelier_events_observation_updated_total{tenant,source,result}`; log template includes `tenant`, `advisory_id`, `observation_id`, `event_id`.
- Offline posture: allow emitting into local bus, enqueue to file-backed spool when offline; retry with deterministic ordering by `(tenant, observation_id)`.
- Open questions to resolve in impl task:
- Final topic naming and DSSE requirement (optional vs required per deployment).
- Whether to include component alias list in the event payload or expect consumers to join via API.
## 2) LNM fixtures + precedence markers (LNM-21-002)
- Goal: unblock correlation pipelines and downstream linkset tasks by defining required fixture shape and precedence rules.
- Fixture requirements (additive to frozen LNM v1 schema):
- Provide at least three sources with conflicting severity/CVSS to exercise conflict markers.
- Include overlapping version ranges to validate precedence tie-breakers.
- Each fixture must include `provenance` (source, fetch_time, collector) and `confidence` hints.
- Precedence rule proposal for review:
1. Prefer explicit source ranking table (to be agreed) over recency.
2. If ranking ties, prefer narrower version ranges, then higher confidence, then stable lexical order of `(source, advisory_id)`.
3. Never collapse conflicting fields; emit `conflicts[]` entries with reason codes `severity-disagree`, `cvss-disagree`, `reference-disagree`.
- Delivery path for fixtures once agreed: `src/Concelier/seed-data/lnm/v1/fixtures/*.json` with deterministic ordering; wire into `StellaOps.Concelier.Core.Tests` harness.
- Next actions captured for implementation task:
- Confirm ranking table and conflict reason code list with Cartographer/Data Science.
- Drop initial fixtures into the above path and reference them from the implementation tasks’ tests.
## Handoff
- This document is the published prep artefact requested by PREP-CONCELIER-GRAPH-21-002-PLATFORM-EVENTS-S and PREP-CONCELIER-LNM-21-002-WAITING-ON-FINALIZE. Downstream tasks should cite this file until the final schemas/fixtures are merged.

37
docs/modules/concelier/prep/2025-11-20-policy-linkset-prep.md
View File

@@ -1,37 +0,0 @@
# Concelier · Policy Engine Linkset API Prep
- **Date:** 2025-11-20
- **Scope:** PREP-CONCELIER-POLICY-20-001 (LNM APIs not exposed via OpenAPI)
- **Working directory:** `src/Concelier/StellaOps.Concelier.WebService`
## Goal
Freeze the contract Policy Engine will consume for advisory lookups without inference/merges, and locate where the OpenAPI surface must be updated so downstream Policy tasks can begin.
## API surface to expose
- **Endpoint:** `GET /v1/lnm/linksets`
- **Query params:**
- `purl` (repeatable), `cpe`, `ghsa`, `cve`, `advisoryId`, `source` (nvd|ghsa|osv|vendor:<slug>), `severityMin`, `severityMax`, `publishedSince`, `modifiedSince`, `tenant` (header enforced, not query), `page` (default 1), `pageSize` (default 50, max 200), `sort` (publishedAt|modifiedAt|severity desc|source|advisoryId; default modifiedAt desc).
- **Response:** deterministic ordering; body fields = `advisoryId`, `source`, `purl[]`, `cpe[]`, `summary`, `publishedAt`, `modifiedAt`, `severity` (source-native), `status` (facts only), `provenance` (`ingestedAt`, `connectorId`, `evidenceHash`, `dsseEnvelopeHash?`), `conflicts[]` (raw disagreements, no merged verdicts), `timeline[]` (raw timestamps + hashes), `remarks[]` (human notes, optional).
- **Endpoint:** `GET /v1/lnm/linksets/{advisoryId}`
- Mirrors above fields; adds `normalized` block for any canonicalized IDs; `cached` flag already added in Sprint 110.B endpoint work.
- **Endpoint:** `POST /v1/lnm/linksets/search`
- Accepts body with same filters as query params plus boolean `includeTimeline`, `includeObservations` (default false). Must respect tenant guard and AOC (no inferred verdicts or merges).
## OpenAPI tasks
- Source file location: `src/Concelier/StellaOps.Concelier.WebService/openapi/concelier-lnm.yaml` (to be created / updated alongside code) and published copy under `docs/api/concelier/`.
- Add components:
- `LinksetProvenance` object (ingestedAt, connectorId, evidenceHash, dsseEnvelopeHash?).
- `LinksetConflict` object (source, field, observedValue, observedAt, evidenceHash).
- `LinksetTimeline` object (event, at, evidenceHash, dsseEnvelopeHash?).
- Pagination envelope: `{ "items": [...], "page": 1, "pageSize": 50, "total": <int> }` with stable ordering guarantees quoted above.
- Security: `Tenant` header required; bearer/mtls unchanged from existing WebService.
## Determinism & AOC guards
- Responses must never include merged severity/state; surface only source-provided facts and conflicts.
- Sorting: primary `modifiedAt desc`, tie-breaker `advisoryId asc`, then `source asc` for deterministic pagination.
- Cache: the `/linksets/{advisoryId}` endpoint may serve cached entries but must include `cached: true|false` and `provenance.evidenceHash` so Policy Engine can verify integrity.
## Deliverable
- This prep note is the canonical contract for policy-facing LNM APIs until the OpenAPI source is committed at the path above.
- Downstream tasks (POLICY-ENGINE-20-001 and linked Policy Engine sprints) should bind to these fields; any deviations must update this prep note and the sprint’s Decisions & Risks.

44
docs/modules/concelier/prep/2025-11-20-web-airgap-57-001-prep.md
View File

@@ -1,44 +0,0 @@
# Concelier Web AirGap Prep — PREP-CONCELIER-WEB-AIRGAP-57-001
Status: Draft (2025-11-20)
Owners: Concelier WebService Guild · AirGap Policy Guild
Scope: Define remediation payloads and staleness plumbing for sealed-mode violations, dependent on WEB-AIRGAP-56-002.
## Dependencies
- WEB-AIRGAP-56-001: mirror bundle registration + sealed-mode enforcement.
- WEB-AIRGAP-56-002: staleness + bundle provenance metadata surfaces.
- AirGap controller scopes (seal/unseal) and time anchor semantics from AirGap Controller/Time guilds.
## Proposed payload mapping (EGRESS blocked)
- Error code: `AIRGAP_EGRESS_BLOCKED`.
- Shape:
```json
{
"error": "AIRGAP_EGRESS_BLOCKED",
"message": "Direct internet fetches disabled in sealed mode; use mirror bundle sources only.",
"bundle_required": true,
"staleness_seconds": 0,
"remediation": [
"Import mirror bundle via /airgap/import or offline kit",
"Ensure sealed mode is set with valid time anchor",
"Retry with cached/mirrored sources enabled"
]
}
```
- Determinism: fixed ordering of fields, remediation list sorted.
## Staleness surfacing
- Staleness derived from bundle metadata supplied by 56-002 (`bundle_id`, `provenance`, `staleness_budget_seconds`).
- Responses include `staleness_seconds_remaining` and `bundle_id` when available.
## Observability
- Emit timeline event `concelier.airgap.egress_blocked` with `{tenant_id, bundle_id?, endpoint, request_id}`.
- Metric: `concelier_airgap_egress_blocked_total` (counter) tagged by endpoint.
## Open decisions
- Final error envelope format (depends on WEB-OAS-61-002 standard envelope).
- Exact header name for staleness metadata (suggest `x-concelier-bundle-staleness`).
- Whether to include advisory key/linkset ids in the blocked response.
## Handoff
Use this as the PREP artefact for WEB-AIRGAP-57-001. Update once 56-002 and error envelope standard are finalized.

29
docs/modules/concelier/prep/2025-11-22-oas-obs-prep.md
View File

@@ -1,29 +0,0 @@
# Concelier OAS & Observability Prep (61-001..63-001, 51-001..55-001)
Status: **Ready for implementation** (2025-11-22)
Owners: Concelier Core Guild · API Contracts Guild · DevOps/Observability Guilds
Scope: Freeze the API/SDK contracts and observability envelopes for LNM search/timeline APIs so downstream SDK, governance, and incident flows can proceed without schema churn.
## Inputs
- Frozen LNM payload schema: `docs/modules/concelier/link-not-merge-schema.md` (2025-11-17).
- Event contract: `docs/modules/concelier/events/advisory.observation.updated@1.md`.
- Registry/worker orchestration contract: `docs/modules/concelier/prep/2025-11-20-orchestrator-registry-prep.md`.
## Deliverables
- OpenAPI source stub for LNM + timeline surfaces recorded at `docs/modules/concelier/openapi/lnm-api.yaml` (paths enumerated; examples outlined below).
- SDK example library checklist covering `searchAdvisories`, `searchLinksets`, `getTimeline`, `getObservationById`; response bodies aligned to frozen schema; no consensus/merge fields.
- Observability contract (metrics/logs/traces):
- Metrics: `concelier_ingest_latency_seconds`, `concelier_linkset_conflicts_total`, `concelier_timeline_emit_lag_seconds`, `concelier_api_requests_total{route,tenant,status}` with burn-rate alert examples.
- Logs: structured fields `tenantId`, `advisoryKey`, `linksetId`, `timelineCursor`, `egressPolicy`.
- Traces: span names for `lnm.search`, `lnm.timeline`, `lnm.linkset-resolve` with baggage keys `tenant-id`, `request-id`.
- Incident/observability hooks: timeline/attestation enrichment notes for OBS-54/55 including DSSE envelope hash field and sealed-mode redaction rules.
## Acceptance Criteria
- Request/response shapes for `/api/v1/lnm/advisories`, `/api/v1/lnm/linksets`, `/api/v1/lnm/timeline` documented with required query params (`tenantId`, `productKey`, `offset`, `limit`, `sort`, `includeTimeline=true|false`).
- All responses MUST include `provenance` block (source, fetchedAt, digest, evidenceBundleId) and forbid consensus/merge fields.
- Metrics/logs names and labels are deterministic and lowercase; alert examples reference burn-rate SLOs.
- File path above is referenced from sprint trackers; any future schema edits require bumping version/comment in this prep doc.
## Notes
- This prep satisfies PREP-CONCELIER-OAS-61-001/002/62-001/63-001 and PREP-CONCELIER-OBS-51-001/52-001/53-001/54-001/55-001.
- No external dependencies remaining; downstream tasks may proceed using the stubbed OpenAPI and observability contracts here.

82
docs/modules/concelier/prep/store-aoc-19-005-dev.md
View File

@@ -1,82 +0,0 @@
# Concelier Backfill & Rollback Plan (STORE-AOC-19-005-DEV, Postgres)
## Objective
Prepare and rehearse the raw Link-Not-Merge backfill/rollback so Concelier Postgres reflects the dataset deterministically across dev/stage. This replaces the prior Mongo workflow.
## Inputs
- Dataset tarball: `out/linksets/linksets-stage-backfill.tar.zst`
- Files expected inside: `linksets.ndjson`, `advisory_chunks.ndjson`, `manifest.json`
- Record SHA-256 of the tarball here when staged:
```
$ sha256sum out/linksets/linksets-stage-backfill.tar.zst
2b43ef9b5694f59be8c1d513893c506b8d1b8de152d820937178070bfc00d0c0 out/linksets/linksets-stage-backfill.tar.zst
```
- To regenerate the tarball deterministically from repo seeds: `./scripts/concelier/build-store-aoc-19-005-dataset.sh`
- To validate a tarball locally (counts + hashes): `./scripts/concelier/test-store-aoc-19-005-dataset.sh out/linksets/linksets-stage-backfill.tar.zst`
## Preflight
- Env:
- `PGURI` (or `CONCELIER_PG_URI`) pointing to the target Postgres instance.
- `PGSCHEMA` (default `lnm_raw`) for staging tables.
- Ensure maintenance window for bulk import; no concurrent writers to staging tables.
## Backfill steps (CI-ready)
### Preferred: CI/manual script
- `scripts/concelier/backfill-store-aoc-19-005.sh /path/to/linksets-stage-backfill.tar.zst`
- Env: `PGURI` (or `CONCELIER_PG_URI`), optional `PGSCHEMA` (default `lnm_raw`), optional `DRY_RUN=1` for extraction-only.
- The script:
- Extracts and validates required files.
- Creates/clears staging tables (`<schema>.linksets_raw`, `<schema>.advisory_chunks_raw`).
- Imports via `\copy` from TSV derived with `jq -rc '[._id, .] | @tsv'`.
- Prints counts and echoes the manifest.
### Manual steps (fallback)
1) Extract dataset:
```
mkdir -p out/linksets/extracted
tar -xf out/linksets/linksets-stage-backfill.tar.zst -C out/linksets/extracted
```
2) Create/truncate staging tables and import:
```
psql "$PGURI" <<SQL
create schema if not exists lnm_raw;
create table if not exists lnm_raw.linksets_raw (id text primary key, raw jsonb not null);
create table if not exists lnm_raw.advisory_chunks_raw (id text primary key, raw jsonb not null);
truncate table lnm_raw.linksets_raw;
truncate table lnm_raw.advisory_chunks_raw;
\copy lnm_raw.linksets_raw (id, raw) from program 'jq -rc ''[._id, .] | @tsv'' out/linksets/extracted/linksets.ndjson' with (format csv, delimiter E'\\t', quote '\"', escape '\"');
\copy lnm_raw.advisory_chunks_raw (id, raw) from program 'jq -rc ''[._id, .] | @tsv'' out/linksets/extracted/advisory_chunks.ndjson' with (format csv, delimiter E'\\t', quote '\"', escape '\"');
SQL
```
3) Verify counts vs manifest:
```
jq '.' out/linksets/extracted/manifest.json
psql -tA "$PGURI" -c "select 'linksets_raw='||count(*) from lnm_raw.linksets_raw;"
psql -tA "$PGURI" -c "select 'advisory_chunks_raw='||count(*) from lnm_raw.advisory_chunks_raw;"
```
## Rollback procedure
- If validation fails: `truncate table lnm_raw.linksets_raw; truncate table lnm_raw.advisory_chunks_raw;` then rerun import.
- Promotion to production tables should be gated by a separate migration/ETL step; keep staging isolated.
## Validation checklist
- Tarball SHA-256 recorded above.
- Counts align with `manifest.json`.
- API smoke test (Postgres-backed): `dotnet test src/Concelier/StellaOps.Concelier.WebService.Tests --filter LinksetsEndpoint_SupportsCursorPagination` (against Postgres config).
- Optional: compare sample rows between staging and expected downstream tables.
## Artefacts to record
- Tarball SHA-256 and size.
- `manifest.json` copy alongside tarball.
- Import log (capture script output) and validation results.
- Decision: maintenance window and rollback outcome.
## How to produce the tarball (export from Postgres)
- Use `scripts/concelier/export-linksets-tarball.sh out/linksets/linksets-stage-backfill.tar.zst`.
- Env: `PGURI` (or `CONCELIER_PG_URI`), optional `PGSCHEMA`, `LINKSETS_TABLE`, `CHUNKS_TABLE`.
- The script exports `linksets` and `advisory_chunks` tables to NDJSON, generates `manifest.json`, builds the tarball, and prints the SHA-256.
## Owners
- Concelier Storage Guild (Postgres)
- AirGap/Backfill reviewers for sign-off

26
docs/modules/evidence-locker/prep/2025-11-20-replay-delivery-sync.md
View File

@@ -1,26 +0,0 @@
# Replay Delivery Coordination Prep — PREP-EVIDENCE-LOCKER-GUILD-REPLAY-DELIVERY-GU
Status: Draft (2025-11-20)
Owners: Planning · Evidence Locker Guild · Replay Delivery Guild · CLI Guild
Scope: Define minimum contract notes for replay delivery so EVID-REPLAY-187-001/002 and RUNBOOK-REPLAY-187-004 can move once schemas freeze.
## Ledger & delivery contract (draft)
- **Ingress API stub**: `POST /replay/records` (internal) accepting NDJSON of replay record envelopes (see `docs/modules/evidence-locker/replay-payload-contract.md`).
- **Indexing**: Mongo collection `replay_records` indexed on `{tenant_id, record_id, scan_id, created_at}`; TTL disabled until retention policy lands.
- **Delivery targets**:
- Evidence Locker storage CAS path `cas://replay/{tenant_id}/{record_id}/record.ndjson`
- Optional mirror to ExportCenter bundle queue once export contracts freeze (Sprint 162).
- **Retention knobs (placeholders)**: `max_records_per_tenant`, `max_age_days`, `max_bytes_per_tenant`. Defaults to be supplied by Replay Delivery Guild once ledger policy lands.
## Coordination points
- Replay Delivery Guild to publish retention defaults + eviction order alongside ledger spec; reference back here once available.
- CLI Guild to validate that CAS path + schema version are sufficient for `stella replay|verify|diff` flows (see `docs/modules/cli/guides/replay-cli-prep.md`).
- Ops/Runbook owners to mirror delivery + retention behaviour in `docs/runbooks/replay_ops.md` when promoted.
## Open questions to close before DOING
- Final subject keys for CAS path (include `source` or keep tenant/record only?).
- Whether exports to TimelineIndexer need additional fan-out event (likely tied to Orchestrator envelope once defined).
- Required observability signals: proposal is counter `evidence_replay_records_ingested_total{tenant,source}` and gauge `evidence_replay_storage_bytes{tenant}`.
## Handoff
Treat this as the PREP artefact for PREP-EVIDENCE-LOCKER-GUILD-REPLAY-DELIVERY-GU. Update with concrete retention values and event/fan-out decisions once the Replay Ledger spec is published.

29
docs/modules/evidence-locker/prep/2025-11-20-schema-readiness-blockers.md
View File

@@ -1,29 +0,0 @@
# Evidence Locker Schema Readiness Prep — PREP-EVIDENCE-LOCKER-GUILD-BLOCKED-SCHEMAS-NO
Status: Draft (2025-11-20)
Owners: Planning · Evidence Locker Guild · AdvisoryAI Guild · Orchestrator/Notifications Guild
Scope: Capture the exact signals still missing to unfreeze Evidence Locker replay/bundle schemas, so downstream implementation can proceed without ambiguity.
## Outstanding upstream artefacts (must land before new DOING status)
- **AdvisoryAI evidence bundle schema + payload notes** (Sprint 110.A)
- Need: JSON schema and at least one signed sample bundle covering SBOM + VEX + reachability attachments.
- Acceptance: versioned under `docs/modules/advisory-ai/schemas/evidence-bundle-v1.json` with hash and sample at `docs/samples/advisory-ai/evidence-bundle-v1.json`.
- **Orchestrator + Notifications capsule envelopes** (Sprint 150.A / 140)
- Need: capsule envelope schema carrying replay IDs and DSSE metadata used by ExportCenter/TimelineIndexer.
- Acceptance: schema at `docs/events/orchestrator-scanner-events.md` updated with `replay_id`, `dsse_envelope_hash`, and `tenant_id` fields plus sample message.
- **Replay Ledger retention policy** (shared with Replay Delivery Guild)
- Need: retention limits (days / count), eviction order, and required indexes for `{tenant_id, record_id, scan_id}` in Mongo.
- Acceptance: recorded in `docs/replay/DETERMINISTIC_REPLAY.md` section 8 with deterministic eviction rules.
## Ready-to-start criteria for Evidence Locker tasks
- Both schemas above are versioned and checksummed.
- Sample payloads are placed under `docs/samples/{advisory-ai,orchestrator}/` and referenced from this sprint.
- Recorded hashes are copied into `docs/modules/evidence-locker/replay-payload-contract.md` (section 5 once available).
## Temporary guidance until freeze
- Keep Evidence Locker tasks BLOCKED for code changes; only doc prep allowed.
- Use the draft schema hash from AdvisoryAI if provided, but mark it "unstable" in dependent docs.
- Prefer canonical JSON ordering and UTC RFC3339 timestamps in any provisional samples.
## Handoff
Use this document as the prep artefact for PREP-EVIDENCE-LOCKER-GUILD-BLOCKED-SCHEMAS-NO. Update or retire once the upstream schema hashes are frozen and recorded in this sprint’s Decisions & Risks.

25
docs/modules/evidence-locker/prep/2025-11-20-security-coordination.md
View File

@@ -1,25 +0,0 @@
# Security & Evidence Coordination Prep — PREP-EVIDENCE-LOCKER-GUILD-SECURITY-GUILD-DOC
Status: Draft (2025-11-20)
Owners: Evidence Locker Guild · Security Guild · Docs Guild · Exporter Service Guild · Mirror Creator Guild · DevOps Guild · Timeline Indexer Guild
Scope: Enumerate security-critical deliverables that must be frozen before EvidenceLocker/ExportCenter/TimelineIndexer move to DOING.
## Required artefacts (to freeze)
- **RootPack & crypto profiles**: confirm `ICryptoProviderRegistry` defaults and RootPack publication flow per `docs/security/crypto-routing-audit-2025-11-07.md`; publish profile matrix for FIPS/eIDAS/GOST.
- **Evidence bundle trust**: DSSE signing policy, Rekor optional segment, checksum publication location; hash-record table to be mirrored in DevPortal bundle verification CLI (DVOFF-64-002).
- **Air-gapped import**: mirror bundle path, checksum & signature publication steps for offline kits; rollback checklist for failed imports.
- **Audit & RLS**: required audit fields for EvidenceLocker/Postgres (TimelineIndexer) with tenant scoping; indexes to enforce retention caps once ledger policy lands.
## Deliverables & locations
- `docs/modules/evidence-locker/bundle-packaging.md` — add DSSE + checksum publication matrix (owner: Evidence Locker Guild).
- `docs/modules/export-center/profiles.md` — mirror bundle signing/verifier defaults (owner: Exporter Service Guild).
- `docs/modules/timelineindexer/architecture.md` — include RLS/audit fields for evidence linkage (owner: Timeline Indexer Guild).
- `docs/security/crypto-registry-decision-2025-11-18.md` — referenced as normative source for crypto provider defaults.
## Ready-to-start checklist (for downstream tasks)
- Above docs updated with hashes and profile matrix.
- Sample signed bundle + manifest published under `docs/samples/export-center/bundles/` with SHA256 + DSSE envelope.
- TimelineIndexer RLS/audit fields reviewed by Security.
## Handoff
Treat this file as the published prep artefact for PREP-EVIDENCE-LOCKER-GUILD-SECURITY-GUILD-DOC. Once the four bullets in “Required artefacts” are frozen, flip the sprint task to DONE and unblock downstream implementation tasks.

23
docs/modules/evidence-locker/prep/2025-11-24-evidence-locker-contract.md
View File

@@ -1,23 +0,0 @@
# Evidence Locker sealed bundle contract · 2025-11-24
Owners: Evidence Locker Guild · Security Guild
Status: Published 2025-11-24 (source for ELOCKER-CONTRACT-2001)
## Deliverables
- Bundle schema: `bundle.schema.json` (sealed DSSE envelope + manifest) — stored under `docs/modules/evidence-locker/schemas/bundle.schema.json`.
- DSSE layout: subject digests, payload (`evidence_bundle.json`), and signatures recorded; transparency optional; canonical hash: `SHA256:6f51d7a5c9d0c5db8a1f6e9d4a0af13e3e7eb5bcb4fa8457de99d8b1c2b3b8ff`.
- Sample bundle: `docs/modules/evidence-locker/samples/evidence-bundle-sample.tgz` with accompanying `.sha256` file.
## Scope and guarantees
- Sealed, offline-friendly; deterministic ordering of files in the tarball; UTC timestamps fixed to `1970-01-01T00:00:00Z` for reproducibility.
- Payload includes: `manifest.json`, `evidence_bundle.json`, `signatures/` (DSSE), `checksums.txt`.
- No network dependencies; validation and hashing performed locally.
## Validation
- `docs/modules/evidence-locker/schemas/bundle.schema.json` validated via `ajv` offline run (see `prep/validate.sh`).
- DSSE signature verifies with sample keypair; transparency step skipped (optional).
## Next steps
- Publish NuGet contract (if needed) referencing the schema path.
- Provide CLI/Export Center consumers with manifest path and hash above.
- Unblock ATTEST-PLAN-2001; keep downstream sprints updated.

27
docs/modules/excititor/prep/2025-11-20-consensus-removal-prep.md
View File

@@ -1,27 +0,0 @@
# Excititor · Consensus Removal Prep (AOC-19-004)
- **Date:** 2025-11-20
- **Working directory:** `src/Excititor/__Libraries/StellaOps.Excititor.Core` + `src/Excititor/StellaOps.Excititor.WebService`
- **Scope:** PREP-EXCITITOR-CORE-AOC-19-004-REMOVE-CONSENS
## Objective
Define the cutover plan to remove legacy consensus/severity merge logic so Excititor remains aggregation-only and emits raw facts for downstream Policy/Concelier consumers.
## Required changes (contract)
- **API/Storage:**
- Deprecate/disable any fields representing merged severity/status (`mergedSeverity`, `consensusScore`, `computedStatus`).
- Retain raw source fields: `status`, `justification`, `impact`, `affects`, `references`, `notes`, `provenance`, `reconciledFrom`.
- Add boolean `consensusDisabled: true` to existing documents during migration for audit.
- **Ingestion pipeline:**
- When dual/conflicting statuses arrive, store both observations; no reconciliation beyond stable ordering.
- Maintain deterministic ordering when multiple observations share `(tenant, advisoryId, component)` — sort by `ingestedAt`, then `source`, then `evidenceHash`.
- **Feature flag:** `excititor:aoc:disableConsensus` default `true`; only temporary `false` allowed for rollback during migration.
- **Telemetry:** counter `excititor.ingest.consensus.disabled` tagged by `tenant`, `source`, `connectorId`; increment once per batch after flag applied.
## Migration outline
- Backfill step sets `consensusDisabled=true` where merged fields exist, and clears merged fields without touching raw observations.
- Tests must assert merged fields are absent/null after migration and ingestion flows do not write them.
## Acceptance for prep completion
- Cutover rules, telemetry, and migration outline frozen here; implementation tasks must follow or update this note and sprint risks.

24
docs/modules/excititor/prep/2025-11-20-console-cache-rbac-prep.md
View File

@@ -1,24 +0,0 @@
# Console Cache & RBAC Prep — PREP-EXCITITOR-CONSOLE-23-003-DEPENDS-ON-23-001
Status: Draft (2025-11-20)
Owners: Excititor WebService Guild
Scope: Capture caching, RBAC, and precedence-context requirements for console VEX lookups once the base contract (23-001) is defined.
## Pending decisions
- Tenant scoping contract from Authority (AUTH-TEN-47-001) alignment: whether to propagate `tenant_ids[]` or single `tenant_id` per request.
- Caching TTLs and cache key shape: proposed key = hash of `(tenant_id, advisory_id, component_purl, version_range, include_precedence)`; TTL to follow Policy overlay freshness once defined.
- Precedence trace payload (links to Policy Engine overlays) depends on POLICY-ENGINE-30-001/002.
## Proposed endpoints (draft)
- `GET /console/vex/cache/entries?tenant_id=&component_purl=&advisory_id=` → returns cache metadata (`ttl_seconds`, `hits`, `last_refresh_at`, `materialization_version`).
- `DELETE /console/vex/cache/entries/{materialization_version}` → force eviction for specific tenant/advisory/component.
## RBAC sketch
- Roles: `console.viewer`, `console.operator`, `console.admin`.
- Permissions:
- viewer: read-only to `/console/vex` + counters.
- operator: can invalidate cache and request refresh.
- admin: can set cache policy per tenant/project.
## Handoff
This document is the prep artefact for PREP-EXCITITOR-CONSOLE-23-003-DEPENDS-ON-23-001. Fill in TTLs, cache key fields, and precedence trace format once 23-001 and Policy overlay schemas land, then finalize and move task to DONE.

23
docs/modules/excititor/prep/2025-11-20-console-counters-prep.md
View File

@@ -1,23 +0,0 @@
# Console Counters Prep — PREP-EXCITITOR-CONSOLE-23-002-DEPENDS-ON-23-001
Status: Draft (2025-11-20)
Owners: Excititor WebService Guild
Scope: Define the counter surfaces required for console delta cards, pending the `/console/vex` contract.
## Inputs still pending
- Final `/console/vex` contract (23-001) including status buckets and justification categories.
- Source-of-truth metrics/telemetry names from Policy Engine overlays (POLICY-ENGINE-30-001 once available).
## Proposed counter contract (to validate once 23-001 lands)
- Endpoint: `GET /console/vex/counters?tenant_id=&component_purl=&advisory_id=&since=`
- Response fields:
- `total`, `affected`, `not_affected`, `under_investigation`, `mitigated`, `unknown`
- `delta_since` (ISO-8601) and `window_seconds`
- `evidence_refs[]` (DSSE hashes or linkset ids) optional
- Metrics to emit:
- Gauge `console_vex_active_total{tenant,status}`
- Counter `console_vex_delta_total{tenant,status}` with `delta_since` label
- Determinism: counters computed from immutable materialized views keyed by `(tenant, advisory_id, component_purl)`; avoid wall-clock beyond `since` parameter.
## Handoff
Treat this as the prep artefact for PREP-EXCITITOR-CONSOLE-23-002-DEPENDS-ON-23-001. Update once status buckets are frozen in 23-001 and Policy metrics are published; then finalize endpoints and samples.

25
docs/modules/excititor/prep/2025-11-20-console-vex-contract-prep.md
View File

@@ -1,25 +0,0 @@
# Console / VEX Contract Prep — PREP-EXCITITOR-CONSOLE-23-001-AWAITING-CONCRE
Status: Draft (2025-11-20)
Owners: Excititor WebService Guild · BE-Base Platform Guild
Scope: Capture the required `/console/vex` API contract inputs so downstream tasks can proceed once the concrete spec lands.
## Missing inputs blocking final contract
- LNM 21-* view specification (grouping, sorting, pagination) to align with Console UI cards.
- Final status chip taxonomy and precedence rules from Policy/Concelier overlays.
- SSE channel naming + retry/heartbeat semantics shared with Scheduler/Policy streams.
## Expectations for the final artefact
- OpenAPI snippet covering endpoints:
- `GET /console/vex` with filters: `component_purl`, `advisory_id`, `tenant_id`, `status`, `justification`, `page`, `page_size`, `sort` (stable ordering by `(tenant_id, component_purl, advisory_id, status, updated_at)`).
- `GET /console/vex/{advisory_id}` returning grouped statements, precedence trace pointer, provenance links (DSSE hash + linkset id), and tenant scoping.
- Response envelope: standard console error schema once WEB-OAS-61-002 is frozen; until then use draft shape with `error`, `message`, `trace_id`.
- Determinism: results ordered by `(tenant_id, advisory_id, component_purl, version_range)`; pagination stable under new data.
- Counters: return aggregate status counters `{status -> count}` in the response to power delta chips without extra queries.
- Caching: allow short-lived (≤30s) in-memory cache keyed by tenant+filters for Console views; include `hasMore`+`cursor` to keep pagination stable.
## Placeholder samples to be replaced
- Add samples under `docs/events/samples/console.vex@draft.json` once view spec is provided.
## Handoff
Use this document as the prep artefact for PREP-EXCITITOR-CONSOLE-23-001-AWAITING-CONCRE. Update once LNM view spec and SSE envelope land; then freeze the OpenAPI excerpt and move the sprint task to DONE. (Initial implementation now live with caching + counters; SSE still pending.)

32
docs/modules/excititor/prep/2025-11-20-graph-21-001-prep.md
View File

@@ -1,32 +0,0 @@
# Excititor · Graph Linkouts Prep (GRAPH-21-001)
- **Date:** 2025-11-20
- **Scope:** PREP-EXCITITOR-GRAPH-21-001-NEEDS-CARTOGRAPHE
- **Working directory:** `src/Excititor/__Libraries/StellaOps.Excititor.Core` + `src/Excititor/StellaOps.Excititor.WebService`
## Goal
Define the Cartographer-facing contract for batched VEX/advisory reference fetches by PURL to unblock inspector linkouts.
## Batch request
- Endpoint (to be hosted in Excititor WebService): `POST /internal/graph/linkouts`
- Body:
- `tenant` (string, required)
- `purls` (array, required, max 500) — normalized PURL strings.
- `includeJustifications` (bool, default false)
- `includeProvenance` (bool, default true)
- Idempotency key: `tenant` + SHA256 of sorted `purls` list.
## Response shape
- `items[]` ordered by input PURL list:
- `purl`
- `advisories[]` — entries with `advisoryId`, `source`, `status`, `justification?`, `modifiedAt`, `evidenceHash`, `connectorId`, `dsseEnvelopeHash?`.
- `conflicts[]` — optional disagreements (status/justification) with `source`, `observedAt`, `evidenceHash`.
- `notFound[]` — PURLs with no VEX observations.
## Determinism & limits
- Response ordering stable: by input PURL order, then `advisoryId`, then `source`.
- Max rows: cap `advisories` to 200 per PURL; truncate with `truncated: true` flag and `nextCursor` (advisoryId, source).
## Acceptance for prep completion
- Request/response contract frozen; Cartographer can stub to this interface. Downstream GRAPH-21-001 implementation must adhere or update doc + sprint risks.

23
docs/modules/excititor/prep/2025-11-20-graph-21-002-prep.md
View File

@@ -1,23 +0,0 @@
# Excititor · Graph Overlay Prep (GRAPH-21-002)
- **Date:** 2025-11-20
- **Depends on:** GRAPH-21-001 linkout contract
- **Working directory:** `src/Excititor/StellaOps.Excititor.WebService`
## Overlay payload
- Aggregates output of GRAPH-21-001 into overlay items for inspectors:
- `purl`
- `summary`: `open`, `not_affected`, `under_investigation`, `no_statement` counts
- `latestModifiedAt` (ISO-8601 UTC)
- `justifications[]` (optional) — unique justification codes present for the PURL
- `provenance` — `sources[]` (unique source IDs), `lastEvidenceHash`
- Endpoint: `GET /v1/graph/overlays?purl=<purl>[&purl=...]&includeJustifications=true|false`
- Sorting: results ordered by input PURL list; within overlays, `justifications` sorted ascending.
## Caching
- Cache key: tenant + sorted PURL list + `includeJustifications` flag; ttl 5 minutes default, configurable `excititor:graph:overlayTtlSeconds`.
- Cache metadata returned: `cached: true|false`, `cacheAgeMs`.
## Acceptance for prep completion
- Overlay shape and caching contract defined; implementation can proceed once GRAPH-21-001 is available.

21
docs/modules/excititor/prep/2025-11-20-graph-21-005-prep.md
View File

@@ -1,21 +0,0 @@
# Excititor · Graph Indexes Prep (GRAPH-21-005)
- **Date:** 2025-11-20
- **Depends on:** GRAPH-21-002 overlays
- **Working directory:** `src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo`
## Index plan
- Collection: `vex_observations`
- Compound index `{ tenant: 1, component.purl: 1, advisoryId: 1, source: 1, modifiedAt: -1 }` (supports overlay queries and truncation cursor).
- Sparse index `{ tenant: 1, component.purl: 1, status: 1 }` for summary counts.
- Collection: `vex_overlays` (materialized cache, optional)
- Index `{ tenant: 1, purl: 1 }` unique.
- TTL index on `cachedAt` configurable via `excititor:graph:overlayTtlSeconds`.
## Determinism
- Materialization job must sort observations as per GRAPH-21-001 ordering before writing overlays so pagination/cursors align.
- TTL applied identically across tenants; default 300s, override allowed via config but must be documented.
## Acceptance for prep completion
- Index keys and TTL knobs defined; downstream storage tasks can implement without further contract churn.

18
docs/modules/excititor/prep/2025-11-20-linkset-extraction-prep.md
View File

@@ -1,18 +0,0 @@
# Linkset Extraction Prep — PREP-EXCITITOR-CORE-AOC-19-002-LINKSET-EXTRAC
Status: Draft (2025-11-20)
Owners: Excititor Core Guild
Scope: Identify the extraction rules and ordering needed to produce linksets from VEX/advisory inputs before idempotent raw upsert work starts.
## Required content to unblock
- Canonical linkset schema version (pending Cartographer/Concelier alignment); need field list and conflict markers.
- Source ranking/precedence table shared with Concelier LNM 21-002 fixtures.
## Proposed extraction rules (draft)
- Inputs: advisory documents (component PURLs, version ranges, references, severities, CVSS vectors); output: linkset entries with `advisory_id`, `component_purl`, `version_range`, `references[]`, `severity`, `cvss`.
- Ordering: sort entries by `(component_purl, advisory_id, version_range)`; within references, sort lexicographically.
- Conflict handling: if multiple sources disagree, emit `conflicts[]` with `source`, `field`, `reason`; never collapse values.
- Determinism: no wall-clock; timestamps only from source payloads (UTC ISO-8601) and preserved as-is.
## Handoff
Treat this as the prep artefact for PREP-EXCITITOR-CORE-AOC-19-002-LINKSET-EXTRAC. Once the shared linkset schema and precedence table land, finalize the rules and move the sprint task to DONE.

18
docs/modules/excititor/prep/2025-11-20-raw-upsert-idempotency-prep.md
View File

@@ -1,18 +0,0 @@
# Raw Upsert Idempotency Prep — PREP-EXCITITOR-CORE-AOC-19-003-BLOCKED-ON-19
Status: Draft (2025-11-20)
Owners: Excititor Core Guild
Scope: Document the idempotent raw upsert and versioning requirements once linkset extraction (19-002) is defined.
## Pending inputs
- Linkset schema and conflict markers from 19-002.
- Storage model choice (Mongo vs Postgres) and required unique keys per tenant/advisory/component/version_range.
## Proposed rules (draft)
- Unique key: `(tenant_id, advisory_id, component_purl, version_range, source)`; store a monotonic `revision` and `ingested_at` (UTC) for traceability.
- Idempotency: compute content hash over canonicalized payload; if identical, no-op; otherwise append new revision with `supersedes` pointer.
- Append-only log: keep prior revisions for audit; consumers read latest by hash or highest revision per key.
- Determinism: canonical JSON ordering; stable sorting by `(tenant_id, advisory_id, component_purl, version_range, revision)`.
## Handoff
Use this as the prep artefact for PREP-EXCITITOR-CORE-AOC-19-003-BLOCKED-ON-19. Finalize once 19-002 freezes schema and storage choice; then wire migrations/indexes accordingly.

23
docs/modules/excititor/prep/2025-11-20-tenant-authority-prep.md
View File

@@ -1,23 +0,0 @@
# Excititor · Tenant-Aware Authority Prep (AOC-19-013)
- **Date:** 2025-11-20
- **Scope:** PREP-EXCITITOR-CORE-AOC-19-013-SEED-TENANT-AW
- **Working directory:** `src/Excititor/StellaOps.Excititor.WebService`, `src/Excititor/StellaOps.Excititor.Worker`, `src/Excititor/__Libraries/StellaOps.Excititor.Core`
## Goals
- Enforce tenant-scoped Authority clients for all WebService/Worker actions to prevent cross-tenant leakage when consensus is removed.
- Provide deterministic fixture/seed guidance for e2e tests.
## Contract
- All Authority calls must be created through `IAuthorityClientFactory.Create(tenantId)`; factories that lack tenant must throw.
- Configuration: `excititor:authority:baseUrl`, `excititor:authority:audience`, per-tenant `clientId/clientSecret` retrieved via internal secret resolver (no cross-tenant cache).
- Headers: include `X-Tenant` on every outbound request; reject response lacking matching `tenant` claim.
- Telemetry: meter `StellaOps.Excititor.Auth` counters `authority.call` tagged `tenant`, `operation`, `result` (`ok|unauthorized|forbidden|error`).
## Testing seeds
- Provide seeded tenants `alpha`, `bravo` with stub secrets in test settings; integration tests must assert cross-tenant requests are rejected (401/403) when header mismatch or missing client mapping.
- Fake Authority server returns tenant claim; tests validate enforcement and logs.
## Acceptance for prep completion
- Tenant-scoped client contract, config keys, and test seeds documented; downstream tasks 19-013 can proceed using this as authority.

31
docs/modules/excititor/prep/2025-11-22-airgap-56-58-prep.md
View File

@@ -1,31 +0,0 @@
# Excititor Air-Gap Prep (56-001, 57-001, 58-001)
Status: **Ready for implementation** (2025-11-22)
Owners: Excititor Core Guild · AirGap Policy Guild · Evidence Locker Guild
Scope: Define ingestion/egress contracts for Excititor when operating in sealed/offline environments and align with mirror bundle + Evidence Locker artifacts.
## Inputs
- Mirror bundle schema (thin) from `docs/modules/mirror/assembler.md`.
- Evidence Locker attestation contract: `docs/modules/evidence-locker/attestation-contract.md`.
- Link-Not-Merge schema for advisory evidence: `docs/modules/concelier/link-not-merge-schema.md`.
## Deliverables
- Ingestion envelope for `POST /airgap/vex/import`:
- Fields: `bundleId`, `mirrorGeneration`, `signedAt`, `publisher`, `payloadHash`, `payloadUrl?` (offline tar path), `signature`, `transparencyLog?`.
- Validation: deterministic hash of NDJSON payloads; must reject mixed tenants; clock-skew tolerance ±5s.
- Idempotency: duplicate `(bundleId, mirrorGeneration)` must return HTTP 409 `AIRGAP_IMPORT_DUPLICATE` and not write a new record.
- Sealed-mode error catalog (57-001): `AIRGAP_EGRESS_BLOCKED`, `AIRGAP_PAYLOAD_STALE`, `AIRGAP_SIGNATURE_MISSING`, `AIRGAP_SOURCE_UNTRUSTED`; each with HTTP 4xx mapping and remediation text.
- Notification hooks (58-001): timeline events `airgap.import.started/completed/failed` with attributes `{tenantId,bundleId,generation,stalenessSeconds}`; link to Evidence Locker bundle ID for audit.
- Determinism rules: sort imported observations by `advisoryKey` then `productKey`; write timeline events in the same order; all timestamps UTC ISO-8601.
- Connector trust (CONN-TRUST-01-001):
- Trusted signer manifests reuse `docs/modules/excititor/schemas/connector-signer-metadata.schema.json`; require `fingerprint`, `issuer`, `validFrom/To`, `allowedProfiles`, `bundleHash`.
- Validation: fail import with `AIRGAP_SOURCE_UNTRUSTED` when signer fingerprint not in manifest, signature algorithm not in `{rsa-pss-sha256, ecdsa-p256-sha256, gost-r3410-2012-256}`, or bundle hash mismatch.
- Offline parity: store signer manifests alongside mirror bundle under `mirror/signers/` and include SHA256 in `SHA256SUMS.dsse`.
## Acceptance Criteria
- API shapes captured in this prep are referenced from Sprint 0119 Delivery Tracker; no further blockers for Excititor AirGap tasks.
- Error catalog and timeline events documented and consumed by downstream Policy/AirGap controller work.
- Import path validated against mirror bundle schema; mismatch should raise `AIRGAP_PAYLOAD_STALE`.
## Notes
- Satisfies PREP-EXCITITOR-AIRGAP-56-001, PREP-EXCITITOR-AIRGAP-57-001, and PREP-EXCITITOR-AIRGAP-58-001.

27
docs/modules/excititor/prep/2025-11-22-attestation-rehearsal-prep.md
View File

@@ -1,27 +0,0 @@
# Attestation Verifier Rehearsal — Excititor
Status: **Ready for implementation** (2025-11-22)
Owners: Excititor Attestation Guild · Evidence Locker Guild
Scope: Dry-run `IVexAttestationVerifier` against current Evidence Locker bundles to ensure Excititor attestation endpoints ship with deterministic verification.
## Test Matrix
- Inputs: Evidence Bundle v1 sample (`docs/samples/evidence-bundle/*`), mirror bundle thin sample (`out/mirror/thin/mirror-thin-m0-sample.tar.gz`).
- Verification steps:
1. Validate DSSE envelope signature and Rekor entry (if present); offline mode skips transparency but records `rekorSkipped=true`.
2. Verify manifest hash tree against payload NDJSON files; fail on first mismatch.
3. Assert policy hash matches Policy Engine overlay hash (placeholder `policyHash` captured for now).
4. Emit structured result JSON: `{bundleId, verified, dsseVerified, transparencyChecked, manifestRoot, failures[]}`.
- Determinism: sorted failure list, timestamps set to supplied `--as-of` flag.
## Deliverables
- Harness entry point: `tools/attestation/verifier-rehearsal.sh` (script stub path reserved).
- Sample output recorded at `docs/modules/excititor/prep/artifacts/2025-11-22-attestation-rehearsal.json` (to be produced in implementation).
- Logging fields to surface in Excititor: `attestationBundleId`, `evidenceBundleId`, `verified`, `failureCode`, `tenantId`.
## Acceptance Criteria
- Rehearsal script runs offline using bundled samples and exits non-zero on any verification failure.
- Output schema above is referenced by Excititor API tests and Policy attest replay tasks.
- Downstream tasks EXCITITOR-GRAPH-21-00x and attestation endpoints can rely on this contract.
## Notes
- Satisfies PREP-ATTESTATION-VERIFIER-REHEARSAL-EXCITITOR.

17
docs/modules/export-center/prep/2025-11-20-crypto-90-001-prep.md
View File

@@ -1,17 +0,0 @@
# Export Crypto Prep — PREP-EXPORT-CRYPTO-90-001
Status: Draft (2025-11-20)
Owners: Exporter Service · Security Guild
Scope: Capture crypto requirements pending Nov-18 review and reference implementation.
## Needs
- Mapping of signing/encryption algorithms per export profile.
- Integration with `ICryptoProviderRegistry` (same as Evidence Locker) for provider selection.
- Hashing defaults (sha256) and optional sha512/sha3 for high-assurance paths.
## Open decisions
- Final provider list and key storage (KMS/HSM) per profile.
- Whether to sign both manifest and per-artifact hashes.
## Handoff
Use as prep artefact for EXPORT-CRYPTO-90-001; fill once Security delivers profile list and reference implementation.

42
docs/modules/export-center/prep/2025-11-20-dvoff-64-002-prep.md
View File

@@ -1,42 +0,0 @@
# DevPortal Offline Prep — PREP-DVOFF-64-002
Status: **Ready for implementation** (2025-11-20)
Owners: DevPortal Offline Guild · AirGap Controller Guild
Scope: Define sealed bundle sample + CLI verify flow for DevPortal offline verification (`stella devportal verify bundle.tgz`).
## Required inputs
- EvidenceLocker sealed bundle contract: `docs/modules/evidence-locker/bundle-packaging.md` (bundle.tgz layout, determinism).
- Portable bundle guidance: `docs/airgap/portable-evidence.md` (for redacted flow).
## Sample artefacts to publish
- `out/devportal/samples/bundle.tgz` — copy of EvidenceLocker sealed bundle (write-once).
- `out/devportal/samples/bundle.tgz.sha256` — `sha256 bundle.tgz` line.
- `out/devportal/samples/verify-report.json` — expected CLI JSON output after verification (see below).
## CLI verification flow (contract)
- Command: `stella devportal verify --bundle bundle.tgz --offline`
- Steps performed:
1) Validate SHA-256 against `.sha256` file.
2) Extract `manifest.json`, `signature.json`, `bundle.json`, `checksums.txt` (no rewrite).
3) Run DSSE verification (offline) using embedded signature; if TSA token present, report but do not fail when `--offline` is set.
4) Emit JSON output:
```json
{
"status": "verified",
"bundleId": "<bundleId>",
"rootHash": "sha256:0123deadbeef",
"entries": 4,
"createdAt": "2025-01-01T00:00:00Z",
"portable": false
}
```
- Exit codes: 0 success, 2 checksum mismatch, 3 signature failure, 4 TSA missing (when not offline), 5 unexpected.
- Determinism: no network calls when `--offline`; output JSON keys sorted.
## Acceptance criteria
- Sample bundle and .sha256 published under `out/devportal/samples/` with hashes listed in this sprint.
- CLI flow documented above; exit codes and sample JSON provided.
- Prep doc linked from Sprint 0162 P1 and DevPortal docs when implemented.
## Next steps
- Publish the sample bundle + hashes; update sprint Delivery Tracker to DONE once artifacts exist.

12
docs/modules/export-center/prep/2025-11-20-export-airgap-56-001-prep.md
View File

@@ -1,12 +0,0 @@
# Export AirGap Prep — PREP-EXPORT-AIRGAP-56-001
Status: Draft (2025-11-20)
Owners: Exporter Service Guild · Mirror Creator Guild
Scope: EvidenceLocker contract + advisory schema to finalize DSSE contents for air-gapped exports.
## Needs
- EvidenceLocker contract (bundle schema, retention).
- Advisory schema alignment for DSSE contents.
## Handoff
Use as prep artefact; update when EvidenceLocker spec is available.

36
docs/modules/export-center/prep/2025-11-20-export-airgap-56-002-prep.md
View File

@@ -1,36 +0,0 @@
# Export AirGap Prep — PREP-EXPORT-AIRGAP-56-002
Status: **Ready for implementation** (2025-11-20)
Owners: Exporter Service Guild · DevOps Guild
Scope: Bootstrap pack (images + charts) packaging for air-gap deploys, dependent on 56-001 evidence/mirror bundle inputs.
## Dependencies
- Sealed bundle schema + advisory contents from 56-001 prep (`docs/modules/export-center/prep/2025-11-20-export-airgap-56-001-prep.md`).
- Mirror/DevOps deployment expectations (values-airgap.yaml) to place bootstrap packs.
## Packaging contract
- Produce deterministic OCI archive `bootstrap-pack-v1.tar` containing:
- `charts/` Helm charts with pinned template timestamps (SOURCE_DATE_EPOCH=2025-01-01T00:00:00Z).
- `images/` directory with referenced container layers/blobs; `manifest.json` aligning with `index.json` (OCI image layout).
- `signatures/` optional DSSE/TUF metadata if provided by 56-001.
- Tarball is gzip-compressed with mtime pinned to `2025-01-01T00:00:00Z`, `0644` perms, uid/gid 0.
- Checksums: `bootstrap-pack-v1.tar.sha256` with `sha256 bootstrap-pack-v1.tar` exactly.
## API/endpoints
- `POST /v1/exports/airgap/bootstrap` → stages pack build; returns `exportId` and profile `bootstrap`.
- `GET /v1/exports/airgap/bootstrap/{exportId}` → status + `downloadUri`, `rootHash`, `artifactSha256`.
- `GET /v1/exports/airgap/bootstrap/{exportId}/download` → serves `application/gzip` tarball; `ETag` = SHA-256.
- Auth scopes: `export:write` for POST; `export:read` for GET/Download.
## Determinism & observability
- Single build timestamp derived from SOURCE_DATE_EPOCH; no wall-clock elsewhere.
- Structured logs `{exportId, profile:"bootstrap", rootHash, artifactSha256}`; metrics `export.bootstrap.completed`, `export.bootstrap.duration_ms`.
## Acceptance criteria
- Tarball is byte-stable across reruns for same inputs; checksum file matches.
- Status/download endpoints documented with headers (`ETag`, `Last-Modified`, quota headers).
- Bootstrap pack content references evidence/mirror bundles from 56-001 (by digest/URL) without re-signing.
## Handoff
- Implement pack build and endpoints in ExportCenter Worker/WebService; use same storage layout as evidence export (`exports/{tenant}/{exportId}/bootstrap-pack-v1.tar`).
- Update Sprint 0162 Delivery Tracker entry P3 to DONE when contract is published.

61
docs/modules/export-center/prep/2025-11-20-export-airgap-57-001-prep.md
View File

@@ -1,61 +0,0 @@
# Export AirGap Prep — PREP-EXPORT-AIRGAP-57-001
Status: **Ready for implementation** (2025-11-20)
Owners: Exporter Service Guild · Evidence Locker Guild
Scope: Portable evidence export mode (air-gap) that reuses EvidenceLocker sealed/portable bundles and packages them for ExportCenter delivery.
## Dependencies (must remain green before coding)
- EvidenceLocker packaging contract (sealed + portable): `docs/modules/evidence-locker/bundle-packaging.md`, `docs/airgap/portable-evidence.md`.
- Upstream sealed bundle export readiness (56-001) and bootstrap pack alignment (56-002) — inputs are reused verbatim, no re-signing.
- Orchestrator/Notifications envelopes for emission events remain pending; not required to start packaging but block notification wiring.
## Contract for EXPORT-AIRGAP-57-001
1) **Input**: bundle id (`bundleId`) that is already sealed. Export service fetches the portable archive `portable-bundle-v1.tgz` via the EvidenceLocker portable endpoint (write-once cache).
2) **Packaging**: create deterministic gzip/tar (`export-portable-bundle-v1.tgz`) with fixed mtime `2025-01-01T00:00:00Z`, PAX headers, and sorted entries:
```
export-portable-bundle-v1.tgz
├── export.json # Export job metadata (bundleId, exportId, tenant, createdAtUtc, rootHash, sourceUri, portableVersion)
├── portable-bundle-v1.tgz # Bit-for-bit copy from EvidenceLocker (no re-signing)
├── checksums.txt # SHA256 for all files (portable bundle included) + Merkle root
├── verify-export.sh # POSIX script: checksum portable bundle, call `stella evidence verify --bundle portable-bundle-v1.tgz`
└── README.md # Operator instructions (ingress/egress steps, expected headers, schema links)
```
- Gzip header mtime and tar mtimes are pinned; permissions `0644`; owner/group `0`.
- `checksums.txt` lists files in lexical order; first line `root <sha256(export-portable-bundle-v1.tgz)>`.
- `verify-export.sh` uses only `tar` + `sha256sum`/`shasum`; no network calls.
3) **API surface (ExportCenter)**
- `POST /v1/exports/airgap/evidence/{bundleId}`: stages the export; responds `202 Accepted` with `exportId` and link to poll.
- `GET /v1/exports/airgap/evidence/{exportId}`: returns status + download link when ready; includes `rootHash`, `portableVersion`, `bundleId`.
- `GET /v1/exports/airgap/evidence/{exportId}/download`: `application/gzip`, filename `export-portable-bundle-v1.tgz`, ETag = SHA256.
- Auth: `export:read` for GET, `export:write` for POST; support tenant scoping identical to EvidenceLocker.
4) **Determinism & observability**
- No wall-clock usage beyond the already fixed `createdAtUtc` written once per export job.
- Emit structured log `{exportId,bundleId,portableVersion,rootHash}` on completion.
- Metrics: counter `export.airgap.portable.completed`, histogram `export.airgap.portable.duration_ms`, gauge `export.airgap.portable.queue_depth`.
5) **Error handling**
- If bundle not sealed → `409 SealedRequired` with `retryAfter`.
- If portable artefact missing → trigger fetch from EvidenceLocker; return `202` with `pendingReason=PortableMaterialising`.
- Verification failures of copied bundle (hash mismatch) → mark export `FAILED` and keep artefact; require operator acknowledgement.
## Acceptance criteria
- Deterministic archive bytes for a given (`bundleId`, `exportId`) across reruns; gzip/tar timestamps and ordering pinned.
- Export archive contains the unmodified EvidenceLocker portable bundle and top-level instructions for offline operators.
- CLI verification path documented in README and script; succeeds with no network access using current `stella evidence verify`.
- Status/Download endpoints documented and cover `202/404/409/500` cases; ETag and `Last-Modified` set.
## Implementation notes for developers
- Reuse EvidenceLocker’s `PortableBundleVersion` constant to avoid drift; do not unzip/repack the inner portable archive.
- Populate `export.json` using UTC ISO-8601; include `sourceUri` referencing the original EvidenceLocker portable endpoint used.
- Store artefacts under object key `exports/{tenant}/{bundleId}/{exportId}/export-portable-bundle-v1.tgz` with write-once semantics.
- Mirror logging/metrics naming with 56-001/56-002 to ease dashboard reuse.
## Open items / risks
- Notifications/timeline emission remains pending on Wave 150/140 envelope drop; add once schemas land (tracked separately).
- If portable bundle version bumps to v2, archive filename and `portableVersion` must be updated in tandem.
## Handoff
- This prep artefact is ready to implement in `src/ExportCenter/StellaOps.ExportCenter.WebService` job + `StellaOps.ExportCenter.Worker` for queue processing.
- Link back to this document from Sprint 0162 Delivery Tracker entry P4.

Some files were not shown because too many files have changed in this diff Show More