feat(scanner): Implement Deno analyzer and associated tests
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Added Deno analyzer with comprehensive metadata and evidence structure. - Created a detailed implementation plan for Sprint 130 focusing on Deno analyzer. - Introduced AdvisoryAiGuardrailOptions for managing guardrail configurations. - Developed GuardrailPhraseLoader for loading blocked phrases from JSON files. - Implemented tests for AdvisoryGuardrailOptions binding and phrase loading. - Enhanced telemetry for Advisory AI with metrics tracking. - Added VexObservationProjectionService for querying VEX observations. - Created extensive tests for VexObservationProjectionService functionality. - Introduced Ruby language analyzer with tests for simple and complex workspaces. - Added Ruby application fixtures for testing purposes.
This commit is contained in:
master
@@ -672,7 +672,7 @@ See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-
|
||||
| `stellaops-cli scanner download` | Fetch and install scanner container | `--channel <stable\|beta\|nightly>` (default `stable`)<br>`--output <path>`<br>`--overwrite`<br>`--no-install` | Saves artefact under `ScannerCacheDirectory`, verifies digest/signature, and executes `docker load` unless `--no-install` is supplied. |
|
||||
| `stellaops-cli scan run` | Execute scanner container against a directory (auto-upload) | `--target <directory>` (required)<br>`--runner <docker\|dotnet\|self>` (default from config)<br>`--entry <image-or-entrypoint>`<br>`[scanner-args...]` | Runs the scanner, writes results into `ResultsDirectory`, emits a structured `scan-run-*.json` metadata file, and automatically uploads the artefact when the exit code is `0`. |
|
||||
| `stellaops-cli scan upload` | Re-upload existing scan artefact | `--file <path>` | Useful for retries when automatic upload fails or when operating offline. |
|
||||
| `stellaops-cli ruby inspect` | Offline Ruby workspace inspection (Gemfile / lock + runtime signals) | `--root <directory>` (default current directory)<br>`--format <table\|json>` (default `table`) | Runs the bundled `RubyLanguageAnalyzer`, renders Package/Version/Group/Source/Lockfile/Runtime columns, or emits JSON `{ packages: [...] }`. Exit codes: `0` success, `64` invalid format, `70` unexpected failure, `71` missing directory. |
|
||||
| `stellaops-cli ruby inspect` | Offline Ruby workspace inspection (Gemfile / lock + runtime signals) | `--root <directory>` (default current directory)<br>`--format <table\|json>` (default `table`) | Runs the bundled `RubyLanguageAnalyzer`, renders Observation summary (bundler/runtime/capabilities) plus Package/Version/Group/Source/Lockfile/Runtime columns, or emits JSON `{ packages: [...], observation: {...} }`. Exit codes: `0` success, `64` invalid format, `70` unexpected failure, `71` missing directory. |
|
||||
| `stellaops-cli ruby resolve` | Fetch Ruby package inventory for a completed scan | `--image <registry-ref>` *or* `--scan-id <id>` (one required)<br>`--format <table\|json>` (default `table`) | Calls `GetRubyPackagesAsync` to download `ruby_packages.json`, groups entries by bundle/platform, and shows runtime entrypoints/usage. Table output mirrors `inspect`; JSON returns `{ scanId, groups: [...] }`. Exit codes: `0` success, `64` invalid args, `70` backend failure. |
|
||||
| `stellaops-cli db fetch` | Trigger connector jobs | `--source <id>` (e.g. `redhat`, `osv`)<br>`--stage <fetch\|parse\|map>` (default `fetch`)<br>`--mode <resume|init|cursor>` | Translates to `POST /jobs/source:{source}:{stage}` with `trigger=cli` |
|
||||
| `stellaops-cli db merge` | Run canonical merge reconcile | — | Calls `POST /jobs/merge:reconcile`; exit code `0` on acceptance, `1` on failures/conflicts |
|
||||
@@ -684,14 +684,14 @@ See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-
|
||||
|
||||
### Ruby dependency verbs (`stellaops-cli ruby …`)
|
||||
|
||||
`ruby inspect` runs the same deterministic `RubyLanguageAnalyzer` bundled with Scanner.Worker against the local working tree—no backend calls—so operators can sanity-check Gemfile / Gemfile.lock pairs before shipping. `ruby resolve` downloads the `ruby_packages.json` artifact that Scanner creates for each scan (via `GetRubyPackagesAsync`) and reshapes it for operators who need to reason about groups/platforms/runtime usage after the fact.
|
||||
`ruby inspect` runs the same deterministic `RubyLanguageAnalyzer` bundled with Scanner.Worker against the local working tree—no backend calls—so operators can sanity-check Gemfile / Gemfile.lock pairs before shipping. The command now renders an observation banner (bundler version, package/runtime counts, capability flags, scheduler names) before the package table so air-gapped users can prove what evidence was collected. `ruby resolve` downloads the `ruby_packages.json` artifact that Scanner creates for each scan (via `GetRubyPackagesAsync`) and reshapes it for operators who need to reason about groups/platforms/runtime usage after the fact.
|
||||
|
||||
**`ruby inspect` flags**
|
||||
|
||||
| Flag | Default | Description |
|
||||
| ---- | ------- | ----------- |
|
||||
| `--root <dir>` | current working directory | Directory containing `Gemfile`, `Gemfile.lock`, and runtime sources. Missing paths set exit code **71**. |
|
||||
| `--format <table\|json>` | `table` | `table` renders Package/Version/Groups/Platform/Source/Lockfile/Runtime columns; `json` emits `{ "packages": [...] }` with the analyzer metadata. |
|
||||
| `--format <table\|json>` | `table` | `table` renders Observation summary + Package/Version/Groups/Platform/Source/Lockfile/Runtime columns; `json` emits `{ "packages": [...], "observation": {...} }` with the analyzer metadata. |
|
||||
| `--verbose` / `-v` | `false` | Surfaces analyzer trace logging while keeping deterministic output. |
|
||||
|
||||
Successful runs exit `0`; invalid formats raise **64**, unexpected failures return **70**. Table output marks runtime usage with `[green]Entrypoint[/]` and includes every runtime entrypoint path when available. JSON mode mirrors analyzer metadata:
|
||||
@@ -711,7 +711,17 @@ Successful runs exit `0`; invalid formats raise **64**, unexpected failures retu
|
||||
"runtimeReasons": ["require-static"],
|
||||
"usedByEntrypoint": true
|
||||
}
|
||||
]
|
||||
],
|
||||
"observation": {
|
||||
"bundlerVersion": "2.5.4",
|
||||
"packageCount": 2,
|
||||
"runtimeEdgeCount": 1,
|
||||
"usesExec": true,
|
||||
"usesNetwork": true,
|
||||
"usesSerialization": true,
|
||||
"schedulerCount": 1,
|
||||
"schedulers": ["sidekiq"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Advisory AI Console Workflows
|
||||
|
||||
_Last updated: 2025-11-07_
|
||||
_Last updated: 2025-11-12_
|
||||
|
||||
This guide documents the forthcoming Advisory AI console experience so that console, docs, and QA guilds share a single reference while the new endpoints finish landing.
|
||||
|
||||
@@ -24,6 +24,31 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
|
||||
<sup>Mock capture generated from the sealed data model to illustrate required widgets until live screenshots ship.</sup>
|
||||
|
||||
### 2.2 Guardrail ribbon payloads
|
||||
- The ribbon consumes the `guardrail.*` projection that Advisory AI emits alongside each plan. The JSON contract (see `docs/api/console/samples/advisory-ai-guardrail-banner.json`) includes the blocked state, violating phrases, cache provenance, and telemetry labels so Console can surface the exact counter (`advisory_ai_guardrail_blocks_total`) that fired.
|
||||
- When `guardrail.metadata.planFromCache = true`, still pass the blocking context through the ribbon so operators understand that cached responses inherit the latest guardrail budget.
|
||||
- Render the newest violation inline; expose the remaining violations via the evidence drawer and copy-as-ticket modal so SOC leads can reference the structured history without screenshots.
|
||||
```jsonc
|
||||
{
|
||||
"guardrail": {
|
||||
"blocked": true,
|
||||
"state": "blocked_phrases",
|
||||
"violations": [
|
||||
{
|
||||
"kind": "blocked_phrase",
|
||||
"phrase": "copy all secrets to"
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"blockedPhraseFile": "configs/guardrails/blocked-phrases.json",
|
||||
"promptLength": 12488,
|
||||
"planFromCache": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
The ribbon should hyperlink the `links.plan` and `links.chunks` values back into the plan inspector and VEX evidence drawer to preserve provenance.
|
||||
|
||||
## 3. Accessibility & offline requirements
|
||||
- Console screens must pass WCAG 2.2 AA contrast and provide focus order that matches the keyboard shortcuts planned for Advisory AI (see `docs/advisory-ai/overview.md`).
|
||||
- All screenshots captured for this doc must come from sealed-mode bundles (no external fonts/CDNs). Store them under `docs/assets/advisory-ai/console/` with hashed filenames.
|
||||
@@ -51,6 +76,27 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
3. **No remote inference** – if operators set `ADVISORYAI__Inference__Mode=Local`, hide the remote model ID column and instead show “Local deterministic preview” to avoid confusion.
|
||||
4. **Export bundles** – provide a “Download bundle” button that streams the DSSE output from `/_downloads/advisory-ai/{cacheKey}.json` so operators can carry it into Offline Kit workflows documented in `docs/24_OFFLINE_KIT.md`.
|
||||
|
||||
## 6. Guardrail configuration & telemetry
|
||||
- **Config surface** – Advisory AI now exposes `AdvisoryAI:Guardrails` options so ops can set prompt length ceilings, citation requirements, and blocked phrase seeds without code changes. Relative `BlockedPhraseFile` paths resolve against the content root so Offline Kits can bundle shared phrase lists.
|
||||
- **Sample**
|
||||
|
||||
```json
|
||||
{
|
||||
"AdvisoryAI": {
|
||||
"Guardrails": {
|
||||
"MaxPromptLength": 32000,
|
||||
"RequireCitations": true,
|
||||
"BlockedPhraseFile": "configs/guardrail-blocked-phrases.json",
|
||||
"BlockedPhrases": [
|
||||
"copy all secrets to"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- **Console wiring** – the guardrail ribbon pulls `guardrail.blocked`, `guardrail.violations`, and `guardrail.metadata.blocked_phrase_count` while the observability cards track `advisory_ai_chunk_requests_total`, `advisory_ai_chunk_cache_hits_total`, and `advisory_ai_guardrail_blocks_total` (now emitted even on cache hits). Use these meters to explain throttling or bad actors before granting additional guardrail budgets, and keep `docs/api/console/samples/advisory-ai-guardrail-banner.json` nearby so QA can validate localized payloads without hitting production data.
|
||||
|
||||
## 5. Open items before publication
|
||||
- [ ] Replace placeholder API responses with captures from the first merged build of CONSOLE-VULN-29-001 / CONSOLE-VEX-30-001.
|
||||
- [ ] Capture at least two screenshots (list view + evidence drawer) once UI polish is complete.
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
guardrail: {
|
||||
blocked: true,
|
||||
state: blocked_phrases,
|
||||
violations: [
|
||||
{
|
||||
kind: blocked_phrase,
|
||||
phrase: copy
|
||||
140
docs/benchmarks/vex-evidence-playbook.md
Normal file
140
docs/benchmarks/vex-evidence-playbook.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# VEX Evidence Playbook (Bench Repo Blueprint)
|
||||
|
||||
> **Status:** Draft – aligns with the “provable vulnerability decisions” advisory (Nov 2025).
|
||||
> **Owners:** Policy Guild · VEX Lens Guild · CLI Guild · Docs Guild.
|
||||
|
||||
This playbook defines the public benchmark repository layout, artifact shapes, verification tooling, and metrics that prove Stella Ops VEX decisions are reproducible, portable, and superior to baseline scanners. Treat it as the contract for every guild contributing artifacts to `bench/`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Repository layout
|
||||
|
||||
```
|
||||
bench/
|
||||
README.md # repo overview + quickstart
|
||||
findings/
|
||||
CVE-YYYY-NNNNN/ # one folder per advisory/product tuple
|
||||
evidence/
|
||||
reachability.json # static+runtime call graph for the finding
|
||||
sbom.cdx.json # CycloneDX slice containing the involved components
|
||||
decision.openvex.json # OpenVEX statement (status + justification)
|
||||
decision.dsse.json # DSSE envelope wrapping the OpenVEX payload
|
||||
rekor.txt # optional Rekor UUID/index/checkpoint
|
||||
metadata.json # producer info (policy rev, analyzer digests, CAS URIs)
|
||||
tools/
|
||||
verify.sh # shell helper: dsse verify + optional rekor verification
|
||||
verify.py # python verifier (offline) that recomputes digests
|
||||
compare.py # baseline diff against Trivy/Syft/Grype/Snyk/Xray outputs
|
||||
replay.sh # reruns reachability graphs via `stella replay`
|
||||
results/
|
||||
summary.csv # FP reduction, MTTD, reproducibility metrics
|
||||
runs/2025-11-10/ # pinned scanner/policy versions + raw outputs
|
||||
stella/
|
||||
findings.json
|
||||
runtime-facts.ndjson
|
||||
reachability.manifest.json
|
||||
trivy/
|
||||
findings.json
|
||||
...
|
||||
```
|
||||
|
||||
### File contracts
|
||||
|
||||
- `reachability.json` is the canonical export from `cas://reachability/graphs/...` with symbol IDs, call edges, runtime hits, analyzer fingerprints, and CAS references.
|
||||
- `decision.openvex.json` follows OpenVEX v1 with Stella Ops-specific `status_notes`, `justification`, `impact_statement`, and `action_statement` text.
|
||||
- `decision.dsse.json` is the DSSE envelope returned by Signer (see §3). Always include the PEM cert chain (keyless) or KMS key id.
|
||||
- `rekor.txt` captures `{uuid, logIndex, checkpoint}` from Attestor when the decision is logged to Rekor.
|
||||
- `metadata.json` binds the DSSE payload back to internal evidence: `{policy_revision, reachability_graph_sha256, runtime_trace_sha256, evidence_cas_uri[], analyzer_versions[], createdBy, createdAt}`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Evidence production flow
|
||||
|
||||
1. **Scanner Worker**
|
||||
- Generate `reachability.json` + `sbom.cdx.json` per prioritized CVE.
|
||||
- Store artifacts under CAS and surface URIs via `ReachabilityReplayWriter`.
|
||||
2. **Policy Engine / VEXer**
|
||||
- Evaluate reachability states + policy lattice to produce an OpenVEX statement.
|
||||
- Persist `decision.openvex.json` and forward it to Signer.
|
||||
3. **Signer & Attestor**
|
||||
- Sign the OpenVEX payload via DSSE (`payloadType: application/vnd.in-toto+json`) and return `decision.dsse.json`.
|
||||
- Optionally call Attestor to log the DSSE bundle to Rekor; write `{uuid, logIndex, checkpoint}` to `rekor.txt`.
|
||||
4. **Bench harness**
|
||||
- Collect SBOM slice, reachability proof, OpenVEX, DSSE, Rekor metadata, and companion metrics into `bench/findings/CVE-...`.
|
||||
- Record tool versions + CAS digests under `metadata.json`.
|
||||
|
||||
All steps must be deterministic: repeated scans with the same inputs produce identical artifacts and digests.
|
||||
|
||||
---
|
||||
|
||||
## 3. Signing & transparency requirements
|
||||
|
||||
| Artifact | Producer | Format | Notes |
|
||||
|-------------------------|---------------|----------------------------------------|-------|
|
||||
| Reachability evidence | Scanner | Canonical JSON (sorted keys) | CAS URI recorded in metadata. |
|
||||
| SBOM slice | Scanner | CycloneDX 1.6 JSON | Keep only components relevant to the finding. |
|
||||
| OpenVEX decision | Policy/VEXer | OpenVEX v1 | One statement per `(CVE, product)` tuple. |
|
||||
| DSSE bundle | Signer | DSSE envelope over OpenVEX payload | Include Fulcio cert or KMS key id. |
|
||||
| Rekor record (optional) | Attestor | Rekor UUID/index/checkpoint | Store alongside DSSE for offline verification. |
|
||||
|
||||
Signer must expose a predicate alias `stella.ops/vexDecision@v1` (see Sprint task `SIGN-VEX-401-018`). Payload = OpenVEX JSON. Rekor logging reuses the existing Attestor `/rekor/entries` pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 4. Verification tooling
|
||||
|
||||
The repo ships two verifiers:
|
||||
|
||||
1. `tools/verify.sh` (bash) — wraps `cosign verify-attestation`/`in-toto verify`, Rekor inclusion checks (`rekor-cli logproof`), and digest comparison.
|
||||
2. `tools/verify.py` — pure-Python offline verifier for air-gapped environments:
|
||||
- Validates DSSE signature using the embedded Fulcio cert or configured root.
|
||||
- Recomputes `sha256` over `reachability.json`, `sbom.cdx.json`, and `decision.openvex.json` to ensure the DSSE payload matches.
|
||||
- Optionally replays reachability by invoking `stella replay --manifest ... --finding CVE-...`.
|
||||
|
||||
CLI addition (`stella decision verify`) should shell out to these helpers when `--from bench` is provided.
|
||||
|
||||
---
|
||||
|
||||
## 5. Metrics & comparison harness
|
||||
|
||||
`tools/compare.py` ingests raw outputs from Stella Ops and baseline scanners (Trivy, Syft, Grype, Snyk, Xray) stored under `results/runs/<date>/<scanner>/findings.json`. For each target:
|
||||
|
||||
- **False-positive reduction (FPR)** = `1 - (# of findings confirmed true positives / # of baseline findings)`.
|
||||
- **Mean time to decision (MTTD)** = average wall-clock time between scan start and DSSE-signed OpenVEX emission.
|
||||
- **Reproducibility score** = `1` if re-running reachability produces identical digests for all artifacts, else `0`; aggregated per run.
|
||||
|
||||
`results/summary.csv` columns:
|
||||
|
||||
```
|
||||
target,cve,baseline_scanner,baseline_hits,stella_hits,fp_reduction,mttd_seconds,reproducible,rekor_uuid
|
||||
```
|
||||
|
||||
Automate collection via `Makefile` or `bench/run.sh` pipeline (task `BENCH-AUTO-401-019`).
|
||||
|
||||
---
|
||||
|
||||
## 6. Publication & README checklist
|
||||
|
||||
`bench/README.md` must include:
|
||||
|
||||
- High-level workflow diagram (scan → reachability → OpenVEX → DSSE → Rekor → bench).
|
||||
- Prerequisites (`cosign`, `rekor-cli`, `stella` CLI).
|
||||
- Quickstart commands:
|
||||
```bash
|
||||
./tools/verify.sh CVE-2023-12345 pkg:purl/example@1.2.3
|
||||
./tools/compare.py --target sample/nginx --baseline trivy --run 2025-11-10
|
||||
```
|
||||
- How to recreate a finding: `stella replay --manifest results/runs/.../replay.yaml --finding CVE-...`.
|
||||
- Contribution guide (where to place new findings, how to update metrics, required metadata).
|
||||
|
||||
---
|
||||
|
||||
## 7. Implementation tasks (see Sprint 401+)
|
||||
|
||||
- `POLICY-VEX-401-010` — emit OpenVEX per finding and publish to bench repo.
|
||||
- `SIGN-VEX-401-018` — add DSSE predicate + Rekor logging for decision payloads.
|
||||
- `CLI-VEX-401-011` — new `stella decision` verbs (`export`, `verify`, `compare`).
|
||||
- `BENCH-AUTO-401-019` — automation to populate `bench/findings/**`, run baseline scanners, and update `results/summary.csv`.
|
||||
- `DOCS-VEX-401-012` — maintain this playbook + README templates, document verification workflow.
|
||||
|
||||
Update `docs/implplan/SPRINT_401_reachability_evidence_chain.md` whenever these tasks move state.
|
||||
@@ -6,9 +6,9 @@ Active items only. Completed/historic work now resides in docs/implplan/archived
|
||||
|
||||
| Wave | Guild owners | Shared prerequisites | Status | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 110.A AdvisoryAI | Advisory AI Guild · Docs Guild · SBOM Service Guild | Sprint 100.A – Attestor (closed 2025-11-09 per `docs/implplan/archived/SPRINT_100_identity_signing.md`) | DOING | Regression/perf suite (AIAI-31-009) and console doc (DOCS-AIAI-31-004) remain DOING; SBOM (SBOM-AIAI-31-001/003), CLI (CLI-VULN-29-001/CLI-VEX-30-001), Policy (POLICY-ENGINE-31-001), and DevOps (DEVOPS-AIAI-31-001) owners owe delivery ETA updates on 2025-11-10 so the CLI/policy/runbook docs can unblock. |
|
||||
| 110.B Concelier | Concelier Core & WebService Guilds · Observability Guild · AirGap Guilds (Importer/Policy/Time) | Sprint 100.A – Attestor | DOING | Paragraph chunk API shipped 2025-11-07; structured field/caching (CONCELIER-AIAI-31-002) is still TODO, telemetry (CONCELIER-AIAI-31-003) DOING, and air-gap/console/attestation tracks remain gated on Link-Not-Merge + Cartographer schema. |
|
||||
| 110.C Excititor | Excititor WebService/Core Guilds · Observability Guild · Evidence Locker Guild | Sprint 100.A – Attestor | DOING | Normalized justification projections (EXCITITOR-AIAI-31-001) are DOING; chunk API, telemetry, docs, attestation, and mirror backlog stay queued behind that work plus Link-Not-Merge / Cartographer prerequisites. |
|
||||
| 110.A AdvisoryAI | Advisory AI Guild · Docs Guild · SBOM Service Guild | Sprint 100.A – Attestor (closed 2025-11-09 per `docs/implplan/archived/SPRINT_100_identity_signing.md`) | DOING | Guardrail regression suite (AIAI-31-009) closed 2025-11-12 with the new `AdvisoryAI:Guardrails` configuration; console doc (DOCS-AIAI-31-004) remains DOING while SBOM/CLI/Policy/DevOps dependencies finish unblocking the screenshots/runbook work. |
|
||||
| 110.B Concelier | Concelier Core & WebService Guilds · Observability Guild · AirGap Guilds (Importer/Policy/Time) | Sprint 100.A – Attestor | DOING | Paragraph chunk API shipped 2025-11-07; structured field/caching (CONCELIER-AIAI-31-002) is still TODO, telemetry (CONCELIER-AIAI-31-003) closed 2025-11-12 with tenant/result/cache counters for Advisory AI, and air-gap/console/attestation tracks remain gated on Link-Not-Merge + Cartographer schema. |
|
||||
| 110.C Excititor | Excititor WebService/Core Guilds · Observability Guild · Evidence Locker Guild | Sprint 100.A – Attestor | DOING | Normalized justification projections (EXCITITOR-AIAI-31-001) landed via `/v1/vex/observations/{vulnerabilityId}/{productKey}`; chunk API, telemetry, docs, attestation, and mirror backlog stay queued behind Link-Not-Merge / Cartographer prerequisites. |
|
||||
| 110.D Mirror | Mirror Creator Guild · Exporter Guild · CLI Guild · AirGap Time Guild | Sprint 100.A – Attestor | TODO | Wave remains TODO—MIRROR-CRT-56-001 has not started, so DSSE/TUF, OCI/time-anchor, CLI, and scheduling integrations cannot proceed. |
|
||||
|
||||
## Status Snapshot (2025-11-09)
|
||||
@@ -18,11 +18,13 @@ Active items only. Completed/historic work now resides in docs/implplan/archived
|
||||
- 2025-11-09: DOCS-AIAI-31-004 continues DOING—guardrail/offline sections are drafted, but screenshots plus copy blocks wait on CONSOLE-VULN-29-001, CONSOLE-VEX-30-001, and EXCITITOR-CONSOLE-23-001.
|
||||
- SBOM-AIAI-31-003 and DOCS-AIAI-31-005/006/008/009 remain BLOCKED pending SBOM-AIAI-31-001, CLI-VULN-29-001, CLI-VEX-30-001, POLICY-ENGINE-31-001, and DEVOPS-AIAI-31-001.
|
||||
- 2025-11-10: AIAI-31-009 performance suite doubled dataset coverage (blocked phrase seed + perf scenarios) and now enforces sub-400 ms guardrail batches so Advisory AI can cite deterministic budgets.
|
||||
- **Concelier (110.B)** – `/advisories/{advisoryKey}/chunks` shipped on 2025-11-07 with tenant enforcement, chunk tuning knobs, and regression fixtures; structured field/caching work (CONCELIER-AIAI-31-002) is still TODO while telemetry/guardrail instrumentation (CONCELIER-AIAI-31-003) is DOING.
|
||||
- 2025-11-12: AIAI-31-009 shipped guardrail configuration binding (`AdvisoryAI:Guardrails`) plus the expanded perf suite; only DOCS-AIAI-31-004 remains DOING while SBOM/CLI/Policy dependencies unblock.
|
||||
- 2025-11-12: DOCS-AIAI-31-004 now documents the guardrail ribbon payload contract (sample + telemetry hooks) so Console/QA can exercise blocked/cached states without waiting for staging captures.
|
||||
- **Concelier (110.B)** – `/advisories/{advisoryKey}/chunks` shipped on 2025-11-07 with tenant enforcement, chunk tuning knobs, and regression fixtures; structured field/caching work (CONCELIER-AIAI-31-002) is still TODO while telemetry/guardrail instrumentation (CONCELIER-AIAI-31-003) closed 2025-11-12 with OTEL counters for tenant/result/cache tags.
|
||||
- Air-gap provenance/staleness bundles (`CONCELIER-AIRGAP-56-001` → `CONCELIER-AIRGAP-58-001`), console views/deltas (`CONCELIER-CONSOLE-23-001..003`), and attestation metadata (`CONCELIER-ATTEST-73-001/002`) remain TODO pending Link-Not-Merge plus Cartographer schema delivery.
|
||||
- Connector provenance refreshes `FEEDCONN-ICSCISA-02-012` and `FEEDCONN-KISA-02-008` are still overdue, leaving evidence parity gaps for those feeds.
|
||||
- 2025-11-10: CONCELIER-AIAI-31-003 shipped cache/request histograms + guardrail counters/log scopes; docs now map the new metrics for Advisory AI dashboards.
|
||||
- **Excititor (110.C)** – Normalized VEX justification projections (EXCITITOR-AIAI-31-001) are DOING as of 2025-11-09; the downstream chunk API (EXCITITOR-AIAI-31-002), telemetry/guardrails (EXCITITOR-AIAI-31-003), docs/OpenAPI alignment (EXCITITOR-AIAI-31-004), and attestation payload work (`EXCITITOR-ATTEST-*`) stay TODO until that projection work plus Link-Not-Merge schema land.
|
||||
- **Excititor (110.C)** – Normalized VEX justification projections (EXCITITOR-AIAI-31-001) landed via `/v1/vex/observations/{vulnerabilityId}/{productKey}`; the downstream chunk API (EXCITITOR-AIAI-31-002), telemetry/guardrails (EXCITITOR-AIAI-31-003), docs/OpenAPI alignment (EXCITITOR-AIAI-31-004), and attestation payload work (`EXCITITOR-ATTEST-*`) stay TODO until Link-Not-Merge schema land.
|
||||
- Mirror/air-gap backlog (`EXCITITOR-AIRGAP-56-001` .. `EXCITITOR-AIRGAP-58-001`) and connector provenance parity (`EXCITITOR-CONN-TRUST-01-001`) remain unscheduled, so Advisory AI cannot yet hydrate sealed VEX evidence or cite connector signatures.
|
||||
- **Mirror (110.D)** – MIRROR-CRT-56-001 (deterministic bundle assembler) has not kicked off, so DSSE/TUF (MIRROR-CRT-56-002), OCI exports (MIRROR-CRT-57-001), time anchors (MIRROR-CRT-57-002), CLI verbs (MIRROR-CRT-58-001), and Export Center automation (MIRROR-CRT-58-002) are all blocked.
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ DOCS-AIAI-31-008 | BLOCKED (2025-11-03) | Publish `/docs/sbom/remediation-heuris
|
||||
DOCS-AIAI-31-009 | BLOCKED (2025-11-03) | Create `/docs/runbooks/assistant-ops.md` for warmup, cache priming, model outages, scaling. Dependencies: DEVOPS-AIAI-31-001. | Docs Guild, DevOps Guild (docs)
|
||||
SBOM-AIAI-31-003 | TODO (2025-11-03) | Publish the Advisory AI hand-off kit for `/v1/sbom/context`, share base URL/API key + tenant header contract, and run a joint end-to-end retrieval smoke test with Advisory AI. Dependencies: SBOM-AIAI-31-001. | SBOM Service Guild, Advisory AI Guild (src/SbomService/StellaOps.SbomService)
|
||||
AIAI-31-008 | TODO | Package inference on-prem container, remote inference toggle, Helm/Compose manifests, scaling guidance, offline kit instructions. Dependencies: AIAI-31-006..007. | Advisory AI Guild, DevOps Guild (src/AdvisoryAI/StellaOps.AdvisoryAI)
|
||||
AIAI-31-009 | DOING (2025-11-09) | Develop unit/golden/property/perf tests, injection harness, and regression suite; ensure determinism with seeded caches. Dependencies: AIAI-31-001..006. | Advisory AI Guild, QA Guild (src/AdvisoryAI/StellaOps.AdvisoryAI) |
|
||||
AIAI-31-009 | DONE (2025-11-12) | Develop unit/golden/property/perf tests, injection harness, and regression suite; ensure determinism with seeded caches. Dependencies: AIAI-31-001..006. | Advisory AI Guild, QA Guild (src/AdvisoryAI/StellaOps.AdvisoryAI) |
|
||||
|
||||
|
||||
|
||||
@@ -39,6 +39,7 @@ DOCS-AIAI-31-005 | BLOCKED (2025-11-03) | Publish `/docs/advisory-ai/cli.md` cov
|
||||
> 2025-11-06: AIAI-31-007 completed – Advisory AI WebService/Worker emit latency histograms, guardrail/validation counters, citation coverage ratios, and OTEL spans; Grafana dashboard + burn-rate alerts refreshed.
|
||||
|
||||
> 2025-11-09: Guardrail harness converted to JSON fixtures + legacy payloads, property-style plan cache load tests added, and file-system cache/output suites cover seeded/offline scenarios.
|
||||
> 2025-11-12: Guardrail/perf suite now enforces sub-400 ms budgets and binds `AdvisoryAI:Guardrails` configuration (prompt length, citation toggle, blocked phrase files) so Console surfaces can reflect ops-tuned budgets.
|
||||
> 2025-11-02: AIAI-31-004 kicked off orchestration pipeline design – establishing deterministic task sequence (summary/conflict/remediation) and cache key strategy.
|
||||
> 2025-11-02: AIAI-31-004 orchestration prerequisites documented in docs/modules/advisory-ai/orchestration-pipeline.md (tasks 004A/004B/004C).
|
||||
> 2025-11-02: AIAI-31-003 moved to DOING – beginning deterministic tooling (comparators, dependency analysis) while awaiting SBOM context client. Semantic & EVR comparators shipped; toolset interface published for orchestrator adoption.
|
||||
|
||||
@@ -8,7 +8,7 @@ Summary: Ingestion & Evidence focus on Concelier (phase I).
|
||||
Task ID | State | Task description | Owners (Source)
|
||||
--- | --- | --- | ---
|
||||
CONCELIER-AIAI-31-002 `Structured fields` | TODO | Ship chunked advisory observation responses (workaround/fix notes, CVSS, affected range) where every field is traced back to the upstream document via provenance anchors; enforce deterministic sorting/pagination and add read-through caching so Advisory AI can hydrate RAG contexts without recomputing severity. | Concelier WebService Guild (src/Concelier/StellaOps.Concelier.WebService)
|
||||
CONCELIER-AIAI-31-003 `Advisory AI telemetry` | DOING | Instrument the new chunk endpoints with request/tenant metrics, cache-hit ratios, and guardrail violation counters so we can prove Concelier is serving raw evidence safely (no merges, no derived fields). | Concelier WebService Guild, Observability Guild (src/Concelier/StellaOps.Concelier.WebService)
|
||||
CONCELIER-AIAI-31-003 `Advisory AI telemetry` | DONE (2025-11-12) | Instrument the new chunk endpoints with request/tenant metrics, cache-hit ratios, and guardrail violation counters so we can prove Concelier is serving raw evidence safely (no merges, no derived fields). | Concelier WebService Guild, Observability Guild (src/Concelier/StellaOps.Concelier.WebService)
|
||||
CONCELIER-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Add mirror ingestion paths that read advisory bundles, persist bundle IDs/merkle roots unchanged, and assert append-only semantics so sealed deployments ingest the same raw facts as online clusters. | Concelier Core Guild (src/Concelier/__Libraries/StellaOps.Concelier.Core)
|
||||
CONCELIER-AIRGAP-56-002 `Bundle catalog linking` | TODO | Record `bundle_id`, `merkle_root`, and time-anchor metadata on every observation/linkset so provenance survives exports; document how Offline Kit verifiers replay the references. Depends on CONCELIER-AIRGAP-56-001. | Concelier Core Guild, AirGap Importer Guild (src/Concelier/__Libraries/StellaOps.Concelier.Core)
|
||||
CONCELIER-AIRGAP-57-001 `Sealed-mode source restrictions` | TODO | Enforce sealed-mode policies that disable non-mirror connectors, emit actionable remediation errors, and log attempts without touching advisory content. Depends on CONCELIER-AIRGAP-56-001. | Concelier Core Guild, AirGap Policy Guild (src/Concelier/__Libraries/StellaOps.Concelier.Core)
|
||||
@@ -20,3 +20,5 @@ CONCELIER-CONSOLE-23-001 `Advisory aggregation views` | TODO | Provide `/console
|
||||
CONCELIER-CONSOLE-23-002 `Dashboard deltas API` | TODO | Calculate deterministic advisory deltas (new, modified, conflicting) for Console dashboards, referencing linkset IDs and timestamps rather than computed verdicts. Depends on CONCELIER-CONSOLE-23-001. | Concelier WebService Guild (src/Concelier/StellaOps.Concelier.WebService)
|
||||
CONCELIER-CONSOLE-23-003 `Search fan-out helpers` | TODO | Implement CVE/GHSA/PURL lookup helpers that return observation/linkset excerpts plus provenance pointers so global search can preview raw evidence safely; include caching + tenant guards. | Concelier WebService Guild (src/Concelier/StellaOps.Concelier.WebService)
|
||||
CONCELIER-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Expand smoke/e2e suites so Authority tokens + tenant headers are required for every ingest/read path, proving that aggregation stays tenant-scoped and merge-free. | Concelier Core Guild (src/Concelier/__Libraries/StellaOps.Concelier.Core)
|
||||
|
||||
> 2025-11-12: CONCELIER-AIAI-31-003 shipped OTEL counters (`advisory_ai_chunk_requests_total`, `advisory_ai_chunk_cache_hits_total`, `advisory_ai_guardrail_blocks_total`) with tenant/result/cache tags so Advisory AI dashboards can see guardrail hits even when Concelier serves cached chunk responses.
|
||||
|
||||
@@ -8,7 +8,7 @@ Summary: Ingestion & Evidence focus on Excititor (phase I).
|
||||
> **Prep:** Read `docs/modules/excititor/architecture.md` and the relevant Excititor `AGENTS.md` files (per component directory) before working any tasks below; this preserves the guidance that previously lived in the component boards.
|
||||
Task ID | State | Task description | Owners (Source)
|
||||
--- | --- | --- | ---
|
||||
EXCITITOR-AIAI-31-001 `Justification enrichment` | DOING (2025-11-09) | Expose normalized VEX justifications, product scope trees, and paragraph/JSON-pointer anchors via `VexObservation` projections so Advisory AI can cite raw evidence without invoking any consensus logic. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
|
||||
EXCITITOR-AIAI-31-001 `Justification enrichment` | DONE (2025-11-12) | Expose normalized VEX justifications, product scope trees, and paragraph/JSON-pointer anchors via `VexObservation` projections so Advisory AI can cite raw evidence without invoking any consensus logic. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
|
||||
EXCITITOR-AIAI-31-002 `VEX chunk API` | TODO | Ship `/vex/evidence/chunks` with tenant/policy filters that streams raw statements, signature metadata, and scope scores for Retrieval-Augmented Generation clients; response must stay aggregation-only and reference observation/linkset IDs. Depends on EXCITITOR-AIAI-31-001. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
|
||||
EXCITITOR-AIAI-31-003 `Telemetry & guardrails` | TODO | Instrument the new evidence APIs with request counters, chunk sizes, signature verification failure meters, and AOC guard violations so Lens/Advisory AI teams can detect misuse quickly. Depends on EXCITITOR-AIAI-31-002. | Excititor WebService Guild, Observability Guild (src/Excititor/StellaOps.Excititor.WebService)
|
||||
EXCITITOR-AIAI-31-004 `Schema & docs alignment` | TODO | Update OpenAPI/SDK/docs to codify the Advisory-AI evidence contract (fields, determinism guarantees, pagination) and describe how consumers map observation IDs back to raw storage. | Excititor WebService Guild, Docs Guild (src/Excititor/StellaOps.Excititor.WebService)
|
||||
@@ -19,3 +19,5 @@ EXCITITOR-ATTEST-01-003 `Verification suite & observability` | TODO (2025-11-06)
|
||||
EXCITITOR-ATTEST-73-001 `VEX attestation payloads` | TODO | Emit attestation payloads that capture supplier identity, justification summary, and scope metadata so downstream Lens/Policy jobs can chain trust without Excititor interpreting the evidence. Depends on EXCITITOR-ATTEST-01-003. | Excititor Core Guild, Attestation Payloads Guild (src/Excititor/__Libraries/StellaOps.Excititor.Core)
|
||||
EXCITITOR-ATTEST-73-002 `Chain provenance` | TODO | Provide APIs that link attestation IDs back to observation/linkset/product tuples, enabling Advisory AI to cite provenance without any derived verdict. Depends on EXCITITOR-ATTEST-73-001. | Excititor Core Guild (src/Excititor/__Libraries/StellaOps.Excititor.Core)
|
||||
EXCITITOR-CONN-TRUST-01-001 `Connector provenance parity` | TODO | Update MSRC, Oracle, Ubuntu, and Stella mirror connectors to emit signer fingerprints, issuer tiers, and bundle references while remaining aggregation-only; document how Lens consumers should interpret these hints. | Excititor Connectors Guild (src/Excititor/__Libraries/StellaOps.Excititor.Connectors.*)
|
||||
|
||||
> 2025-11-12: EXCITITOR-AIAI-31-001 delivered `/v1/vex/observations/{vulnerabilityId}/{productKey}` backed by the new `IVexObservationProjectionService`, returning normalized statements (scope tree, anchors, document metadata) so Advisory AI and Console can cite raw VEX evidence without touching consensus logic.
|
||||
|
||||
@@ -29,3 +29,4 @@
|
||||
- `SCANNER-CLI-0001`: Added CLI unit tests (`CommandFactoryTests`, Ruby inspect JSON assertions) to guard the new verbs and runtime metadata output; `dotnet test src/Cli/__Tests/StellaOps.Cli.Tests/StellaOps.Cli.Tests.csproj --filter "CommandFactoryTests|Ruby"` now covers the CLI surface.
|
||||
- `SCANNER-ENG-0016`: 2025-11-10 — Completed Ruby lock collector and vendor ingestion work: honour `.bundle/config` overrides, fold workspace lockfiles, emit bundler groups, add Ruby analyzer fixtures/goldens (including new git/path offline kit mirror), and `dotnet test ... --filter Ruby` passes.
|
||||
- `SCANNER-ENG-0009`: Emitted observation payload + `ruby-observation` component summarising packages, runtime edges, and capability flags for Policy/Surface exports; fixtures updated for determinism and Offline Kit now ships the observation JSON.
|
||||
- `SCANNER-ENG-0009`: 2025-11-12 — Added bundler-version metadata to observation payloads, introduced the `complex-app` fixture to cover vendor caches/BUNDLE_PATH overrides, and taught `stellaops-cli ruby inspect` to print the observation banner (bundler/runtime/capabilities) alongside JSON `observation` blocks.
|
||||
|
||||
@@ -14,6 +14,7 @@ _Theme:_ Finish the provable reachability pipeline (graph CAS → replay → DSS
|
||||
| GRAPH-CAS-401-001 | TODO | Finalize richgraph schema (`richgraph-v1`), emit canonical SymbolIDs, compute graph hash (BLAKE3), and store CAS manifests under `cas://reachability/graphs/{sha256}`. Update Scanner Worker adapters + fixtures. | Scanner Worker Guild (`src/Scanner/StellaOps.Scanner.Worker`) |
|
||||
| GAP-SYM-007 | TODO | Extend reachability evidence schema/DTOs with demangled symbol hints, `symbol.source`, confidence, and optional `code_block_hash`; ensure Scanner SBOM/evidence writers and CLI serializers emit the new fields deterministically. | Scanner Worker Guild & Docs Guild (`src/Scanner/StellaOps.Scanner.Models`, `docs/modules/scanner/architecture.md`, `docs/reachability/function-level-evidence.md`) |
|
||||
| SCAN-REACH-401-009 | TODO | Ship .NET/JVM symbolizers and call-graph generators (roots, edges, framework adapters), merge results into component-level reachability manifests, and back them with golden fixtures. | Scanner Worker Guild (`src/Scanner/StellaOps.Scanner.Worker`, `src/Scanner/__Libraries`) |
|
||||
| SCANNER-NATIVE-401-015 | TODO | Stand up `StellaOps.Scanner.Symbols.Native` + `StellaOps.Scanner.CallGraph.Native` (ELF/PE readers, demanglers, probabilistic carving) and publish `FuncNode`/`CallEdge` CAS bundles consumed by reachability graphs. | Scanner Worker Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Symbols.Native`, `src/Scanner/__Libraries/StellaOps.Scanner.CallGraph.Native`) |
|
||||
| SYMS-SERVER-401-011 | TODO | Deliver `StellaOps.Symbols.Server` (REST+gRPC) with DSSE-verified uploads, Mongo/MinIO storage, tenant isolation, and deterministic debugId indexing; publish health/manifest APIs (spec: `docs/specs/SYMBOL_MANIFEST_v1.md`). | Symbols Guild (`src/Symbols/StellaOps.Symbols.Server`) |
|
||||
| SYMS-CLIENT-401-012 | TODO | Ship `StellaOps.Symbols.Client` SDK (resolve/upload APIs, platform key derivation for ELF/PDB/Mach-O/JVM/Node, disk LRU cache) and integrate with Scanner.Symbolizer/runtime probes (ref. `docs/specs/SYMBOL_MANIFEST_v1.md`). | Symbols Guild (`src/Symbols/StellaOps.Symbols.Client`, `src/Scanner/StellaOps.Scanner.Symbolizer`) |
|
||||
| SYMS-INGEST-401-013 | TODO | Build `symbols ingest` CLI to emit DSSE-signed `SymbolManifest v1`, upload blobs, and register Rekor entries; document GitLab/Gitea pipeline usage. | Symbols Guild, DevOps Guild (`src/Symbols/StellaOps.Symbols.Ingestor.Cli`, `docs/specs/SYMBOL_MANIFEST_v1.md`) |
|
||||
@@ -23,13 +24,20 @@ _Theme:_ Finish the provable reachability pipeline (graph CAS → replay → DSS
|
||||
| REPLAY-401-004 | TODO | Bump replay manifest to v2 (feeds, analyzers, policies), have `ReachabilityReplayWriter` enforce CAS registration + hash sorting, and add deterministic tests to `tests/reachability/StellaOps.Reachability.FixtureTests`. | BE-Base Platform Guild (`src/__Libraries/StellaOps.Replay.Core`) |
|
||||
| AUTH-REACH-401-005 | TODO | Introduce DSSE predicate types for SBOM/Graph/VEX/Replay, plumb signing through Authority + Signer, and mirror statements to Rekor (including PQ variants where required). | Authority & Signer Guilds (`src/Authority/StellaOps.Authority`, `src/Signer/StellaOps.Signer`) |
|
||||
| POLICY-VEX-401-006 | TODO | Policy Engine consumes reachability facts, applies the deterministic score/label buckets (≥0.80 reachable, 0.30–0.79 conditional, <0.30 unreachable), emits OpenVEX with call-path proofs, and updates SPL schema with `reachability.state/confidence` predicates and suppression gates. | Policy Guild (`src/Policy/StellaOps.Policy.Engine`, `src/Policy/__Libraries/StellaOps.Policy`) |
|
||||
| POLICY-VEX-401-010 | TODO | Implement `VexDecisionEmitter` to serialize per-finding OpenVEX, attach evidence hashes, request DSSE signatures, capture Rekor metadata, and publish artifacts following the bench playbook. | Policy Guild (`src/Policy/StellaOps.Policy.Engine/Vex`, `docs/modules/policy/architecture.md`, `docs/benchmarks/vex-evidence-playbook.md`) |
|
||||
| UI-CLI-401-007 | TODO | Implement CLI `stella graph explain` + UI explain drawer showing signed call-path, predicates, runtime hits, and DSSE pointers; include counterfactual controls. | UI & CLI Guilds (`src/Cli/StellaOps.Cli`, `src/UI/StellaOps.UI`) |
|
||||
| QA-DOCS-401-008 | TODO | Wire `reachbench-2025-expanded` fixtures into CI, document CAS layouts + replay steps in `docs/reachability/DELIVERY_GUIDE.md`, and publish operator runbook for runtime ingestion. | QA & Docs Guilds (`docs`, `tests/README.md`) |
|
||||
| GAP-SIG-003 | TODO | Finish `/signals/runtime-facts` ingestion, add CAS-backed runtime storage, extend scoring to lattice states (`Unknown/NotPresent/Unreachable/Conditional/Reachable/Observed`), and emit `signals.fact.updated` events. Document retention/RBAC. | Signals Guild (`src/Signals/StellaOps.Signals`, `docs/reachability/function-level-evidence.md`) |
|
||||
| SIG-STORE-401-016 | TODO | Introduce shared reachability store collections (`func_nodes`, `call_edges`, `cve_func_hits`), indexes, and repository APIs so Scanner/Signals/Policy can reuse canonical function data. | Signals Guild · BE-Base Platform Guild (`src/Signals/StellaOps.Signals`, `src/__Libraries/StellaOps.Replay.Core`) |
|
||||
| GAP-REP-004 | TODO | Enforce BLAKE3 hashing + CAS registration for graphs/traces before manifest writes, upgrade replay manifest v2 with analyzer versions/policy thresholds, and add deterministic tests. | BE-Base Platform Guild (`src/__Libraries/StellaOps.Replay.Core`, `docs/replay/DETERMINISTIC_REPLAY.md`) |
|
||||
| GAP-POL-005 | TODO | Ingest reachability facts into Policy Engine, expose `reachability.state/confidence` in SPL/API, enforce auto-suppress (<0.30) rules, and generate OpenVEX evidence blocks referencing graph hashes + runtime facts with policy thresholds. | Policy Guild (`src/Policy/StellaOps.Policy.Engine`, `docs/modules/policy/architecture.md`, `docs/reachability/function-level-evidence.md`) |
|
||||
| GAP-VEX-006 | TODO | Wire Policy/Excititor/UI/CLI surfaces so VEX emission and explain drawers show call paths, graph hashes, and runtime hits; add CLI `--evidence=graph`/`--threshold` plus Notify template updates. | Policy, Excititor, UI, CLI & Notify Guilds (`docs/modules/excititor/architecture.md`, `src/Cli/StellaOps.Cli`, `src/UI/StellaOps.UI`, `docs/09_API_CLI_REFERENCE.md`) |
|
||||
| GAP-DOC-008 | TODO | Publish the cross-module function-level evidence guide, update API/CLI references with the new `code_id` fields, and add OpenVEX/replay samples under `samples/reachability/**`. | Docs Guild (`docs/reachability/function-level-evidence.md`, `docs/09_API_CLI_REFERENCE.md`, `docs/api/policy.md`) |
|
||||
| CLI-VEX-401-011 | TODO | Add `stella decision export|verify|compare` verbs, integrate with Policy/Signer APIs, and ship local verifier wrappers for bench artifacts. | CLI Guild (`src/Cli/StellaOps.Cli`, `docs/modules/cli/architecture.md`, `docs/benchmarks/vex-evidence-playbook.md`) |
|
||||
| SIGN-VEX-401-018 | TODO | Extend Signer predicate catalog with `stella.ops/vexDecision@v1`, enforce payload policy, and plumb DSSE/Rekor integration for policy decisions. | Signing Guild (`src/Signer/StellaOps.Signer`, `docs/modules/signer/architecture.md`) |
|
||||
| BENCH-AUTO-401-019 | TODO | Create automation to populate `bench/findings/**`, run baseline scanners (Trivy/Syft/Grype/Snyk/Xray), compute FP/MTTD/repro metrics, and update `results/summary.csv`. | Benchmarks Guild (`docs/benchmarks/vex-evidence-playbook.md`, `scripts/bench/**`) |
|
||||
| DOCS-VEX-401-012 | TODO | Maintain the VEX Evidence Playbook, publish repo templates/README, and document verification workflows for operators. | Docs Guild (`docs/benchmarks/vex-evidence-playbook.md`, `bench/README.md`) |
|
||||
| SYMS-BUNDLE-401-014 | TODO | Produce deterministic symbol bundles for air-gapped installs (`symbols bundle create|verify|load`), including DSSE manifests and Rekor checkpoints, and document offline workflows (`docs/specs/SYMBOL_MANIFEST_v1.md`). | Symbols Guild, Ops Guild (`src/Symbols/StellaOps.Symbols.Bundle`, `ops`) |
|
||||
| DOCS-RUNBOOK-401-017 | TODO | Publish the reachability runtime ingestion runbook, link it from delivery guides, and keep Ops/Signals troubleshooting steps current. | Docs Guild · Ops Guild (`docs/runbooks/reachability-runtime.md`, `docs/reachability/DELIVERY_GUIDE.md`) |
|
||||
|
||||
> Use `docs/reachability/DELIVERY_GUIDE.md` for architecture context, dependencies, and acceptance tests.
|
||||
|
||||
@@ -16,3 +16,5 @@ Execute the tasks below strictly in order; each artifact unblocks the next analy
|
||||
| 6 | `SCANNER-ANALYZERS-DENO-26-006` | DONE | Implement the OCI/container adapter that stitches per-layer Deno caches, vendor trees, and compiled binaries back into provenance-aware analyzer inputs. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-005 |
|
||||
| 7 | `SCANNER-ANALYZERS-DENO-26-007` | DONE | Produce AOC-compliant observation writers (entrypoints, modules, capability edges, workers, warnings, binaries) with deterministic reason codes. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-006 |
|
||||
| 8 | `SCANNER-ANALYZERS-DENO-26-008` | DOING | Finalize fixture + benchmark suite (vendor/npm/FFI/worker/dynamic import/bundle/cache/container cases) validating analyzer determinism and performance. | Deno Analyzer Guild, QA Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-007 |
|
||||
|
||||
_Status 2025-11-12:_ Task `SCANNER-ANALYZERS-DENO-26-008` reopened to diagnose missing vendor-cache edges in the Deno analyzer golden fixture; Codex now DOING to stabilize graph + fixtures before finalizing the sprint.
|
||||
@@ -138,7 +138,27 @@ Both subcommands honour offline-first expectations (no network access) and norma
|
||||
* Uses `STELLAOPS_ADVISORYAI_URL` when configured; otherwise it reuses the backend base address and adds `X-StellaOps-Scopes` (`advisory:run` + task scope) per request.
|
||||
* `--timeout 0` performs a single cache lookup (for CI flows that only want cached artefacts).
|
||||
|
||||
### 2.12 Air-gap guard
|
||||
### 2.12 Decision evidence (new)
|
||||
|
||||
- `decision export`
|
||||
|
||||
* Parameters: `--cve`, `--product <purl or digest>`, `--scan-id <optional>`, `--output-dir`.
|
||||
* Pulls `decision.openvex.json`, `decision.dsse.json`, `rekor.txt`, and evidence metadata from Policy Engine and writes them into the `bench/findings/<CVE>/` layout defined in [docs/benchmarks/vex-evidence-playbook.md](../benchmarks/vex-evidence-playbook.md).
|
||||
* When `--sync` is set, uploads the bundle to Git (bench repo) with deterministic commit messages.
|
||||
|
||||
- `decision verify`
|
||||
|
||||
* Offline verifier that wraps `tools/verify.sh`/`verify.py` from the bench repo. Checks DSSE signature, optional Rekor inclusion, and recomputes digests for reachability/SBOM artifacts.
|
||||
* Supports `--from bench` (local path) and `--remote` (fetch via API). Exit codes align with `verify.sh` (0 success, 3 signature failure, 18 truncated evidence).
|
||||
|
||||
- `decision compare`
|
||||
|
||||
* Executes the benchmark harness against baseline scanners (Trivy/Syft/Grype/Snyk/Xray), capturing false-positive reduction, mean-time-to-decision, and reproducibility metrics into `results/summary.csv`.
|
||||
* Flags regressions when Stella Ops produces more false positives or slower MTTD than the configured target.
|
||||
|
||||
All verbs require scopes `policy.findings:read`, `signer.verify`, and (for Rekor lookups) `attestor.read`. They honour sealed-mode rules by falling back to offline verification only when Rekor/Signer endpoints are unreachable.
|
||||
|
||||
### 2.13 Air-gap guard
|
||||
|
||||
- CLI outbound HTTP flows (Authority auth, backend APIs, advisory downloads) route through `StellaOps.AirGap.Policy`. When sealed mode is active the CLI refuses commands that would require external egress and surfaces the shared `AIRGAP_EGRESS_BLOCKED` remediation guidance instead of attempting the request.
|
||||
|
||||
|
||||
@@ -556,12 +556,13 @@ concelier:
|
||||
* `concelier.fetch.docs_total{source}`
|
||||
* `concelier.fetch.bytes_total{source}`
|
||||
* `concelier.parse.failures_total{source}`
|
||||
* `concelier.map.statements_total{source}`
|
||||
* `concelier.observations.write_total{result=ok|noop|error}`
|
||||
* `concelier.linksets.updated_total{result=ok|skip|error}`
|
||||
* `concelier.linksets.conflicts_total{type}`
|
||||
* `concelier.export.bytes{kind}`
|
||||
* `concelier.export.duration_seconds{kind}`
|
||||
* `concelier.map.statements_total{source}`
|
||||
* `concelier.observations.write_total{result=ok|noop|error}`
|
||||
* `concelier.linksets.updated_total{result=ok|skip|error}`
|
||||
* `concelier.linksets.conflicts_total{type}`
|
||||
* `concelier.export.bytes{kind}`
|
||||
* `concelier.export.duration_seconds{kind}`
|
||||
* `advisory_ai_chunk_requests_total{tenant,result,cache}` and `advisory_ai_guardrail_blocks_total{tenant,reason,cache}` instrument the `/advisories/{key}/chunks` surfaces that Advisory AI consumes. Cache hits now emit the same guardrail counters so operators can see blocked segments even when responses are served from cache.
|
||||
* **Tracing** around fetch/parse/map/observe/linkset/export.
|
||||
* **Logs**: structured with `source`, `uri`, `docDigest`, `advisoryKey`, `exportId`.
|
||||
|
||||
|
||||
@@ -15,12 +15,13 @@ The service operates strictly downstream of the **Aggregation-Only Contract (AOC
|
||||
|
||||
## 1 · Responsibilities & Constraints
|
||||
|
||||
- Compile and evaluate `stella-dsl@1` policy packs into deterministic verdicts.
|
||||
- Join SBOM inventory, Concelier advisories, and Excititor VEX evidence via canonical linksets and equivalence tables.
|
||||
- Materialise effective findings (`effective_finding_{policyId}`) with append-only history and produce explain traces.
|
||||
- Operate incrementally: react to change streams (advisory/vex/SBOM deltas) with ≤ 5 min SLA.
|
||||
- Provide simulations with diff summaries for UI/CLI workflows without modifying state.
|
||||
- Enforce strict determinism guard (no wall-clock, RNG, network beyond allow-listed services) and RBAC + tenancy via Authority scopes.
|
||||
- Compile and evaluate `stella-dsl@1` policy packs into deterministic verdicts.
|
||||
- Join SBOM inventory, Concelier advisories, and Excititor VEX evidence via canonical linksets and equivalence tables.
|
||||
- Materialise effective findings (`effective_finding_{policyId}`) with append-only history and produce explain traces.
|
||||
- Emit per-finding OpenVEX decisions anchored to reachability evidence, forward them to Signer/Attestor for DSSE/Rekor, and publish the resulting artifacts for bench/verification consumers.
|
||||
- Operate incrementally: react to change streams (advisory/vex/SBOM deltas) with ≤ 5 min SLA.
|
||||
- Provide simulations with diff summaries for UI/CLI workflows without modifying state.
|
||||
- Enforce strict determinism guard (no wall-clock, RNG, network beyond allow-listed services) and RBAC + tenancy via Authority scopes.
|
||||
- Support sealed/air-gapped deployments with offline bundles and sealed-mode hints.
|
||||
|
||||
Non-goals: policy authoring UI (handled by Console), ingestion or advisory normalisation (Concelier), VEX consensus (Excititor), runtime enforcement (Zastava).
|
||||
@@ -110,8 +111,9 @@ Key notes:
|
||||
| **Materialiser** (`Materialization/`) | Upsert effective findings, append history, manage explain bundle exports. | Mongo transactions per SBOM chunk. |
|
||||
| **Orchestrator** (`Runs/`) | Change-stream ingestion, fairness, retry/backoff, queue writer. | Works with Scheduler Models DTOs. |
|
||||
| **API** (`Api/`) | Minimal API endpoints, DTO validation, problem responses, idempotency. | Generated clients for CLI/UI. |
|
||||
| **Observability** (`Telemetry/`) | Metrics (`policy_run_seconds`, `rules_fired_total`), traces, structured logs. | Sampled rule-hit logs with redaction. |
|
||||
| **Offline Adapter** (`Offline/`) | Bundle export/import (policies, simulations, runs), sealed-mode enforcement. | Uses DSSE signing via Signer service. |
|
||||
| **Observability** (`Telemetry/`) | Metrics (`policy_run_seconds`, `rules_fired_total`), traces, structured logs. | Sampled rule-hit logs with redaction. |
|
||||
| **Offline Adapter** (`Offline/`) | Bundle export/import (policies, simulations, runs), sealed-mode enforcement. | Uses DSSE signing via Signer service. |
|
||||
| **VEX Decision Emitter** (`Vex/Emitter/`) | Build OpenVEX statements, attach reachability evidence hashes, request DSSE signing, and persist artifacts for Export Center / bench repo. | New (Sprint 401); integrates with Signer predicate `stella.ops/vexDecision@v1` and Attestor Rekor logging. |
|
||||
|
||||
---
|
||||
|
||||
@@ -172,22 +174,36 @@ Determinism guard instrumentation wraps the evaluator, rejecting access to forbi
|
||||
|
||||
---
|
||||
|
||||
## 6 · Run Orchestration & Incremental Flow
|
||||
|
||||
- **Change streams:** Concelier and Excititor publish document changes to the scheduler queue (`policy.trigger.delta`). Payload includes `tenant`, `source`, `linkset digests`, `cursor`.
|
||||
- **Orchestrator:** Maintains per-tenant backlog; merges deltas until time/size thresholds met, then enqueues `PolicyRunRequest`.
|
||||
- **Queue:** Mongo queue with lease; each job assigned `leaseDuration`, `maxAttempts`.
|
||||
## 6 · Run Orchestration & Incremental Flow
|
||||
|
||||
- **Change streams:** Concelier and Excititor publish document changes to the scheduler queue (`policy.trigger.delta`). Payload includes `tenant`, `source`, `linkset digests`, `cursor`.
|
||||
- **Orchestrator:** Maintains per-tenant backlog; merges deltas until time/size thresholds met, then enqueues `PolicyRunRequest`.
|
||||
- **Queue:** Mongo queue with lease; each job assigned `leaseDuration`, `maxAttempts`.
|
||||
- **Workers:** Lease jobs, execute evaluation pipeline, report status (success/failure/canceled). Failures with recoverable errors requeue with backoff; determinism or schema violations mark job `failed` and raise incident event.
|
||||
- **Fairness:** Round-robin per `{tenant, policyId}`; emergency jobs (`priority=emergency`) jump queue but limited via circuit breaker.
|
||||
- **Replay:** On demand, orchestrator rehydrates run via stored cursors and exports sealed bundle for audit/CI determinism checks.
|
||||
- **Batch evaluation service (`/api/policy/eval/batch`):** Stateless evaluator powering Findings Ledger and replay/offline workflows. Requests contain canonical ledger events plus optional current projection; responses return status/severity/labels/rationale without mutating state. Policy Engine enforces per-tenant cost budgets, caches results by `(tenantId, policyVersion, eventHash, projectionHash)`, and falls back to inline evaluation when the remote service is disabled.
|
||||
|
||||
---
|
||||
|
||||
## 7 · Security & Tenancy
|
||||
|
||||
- **Auth:** All API calls pass through Authority gateway; DPoP tokens enforced for service-to-service (Policy Engine service principal). CLI/UI tokens include scope claims.
|
||||
- **Scopes:** Mutations require `policy:*` scopes corresponding to action; `effective:write` restricted to service identity.
|
||||
|
||||
### 6.1 · VEX decision attestation pipeline
|
||||
|
||||
1. **Verdict capture.** Each evaluation result contains `{findingId, cve, productKey, reachabilityState, evidenceRefs}` plus SBOM and runtime CAS hashes.
|
||||
2. **OpenVEX serialization.** `VexDecisionEmitter` builds an OpenVEX document with one statement per `(cve, productKey)` and fills:
|
||||
- `status`, `justification`, `status_notes`, `impact_statement`, `action_statement`.
|
||||
- `products` (purl) and `evidence` array referencing `reachability.json`, `sbom.cdx.json`, `runtimeFacts`.
|
||||
3. **DSSE signing.** The emitter calls Signer `POST /api/v1/signer/sign/dsse` with predicate `stella.ops/vexDecision@v1`. Signer verifies PoE + scanner integrity and returns a DSSE envelope (`decision.dsse.json`).
|
||||
4. **Transparency (optional).** When Rekor integration is enabled, Attestor logs the DSSE payload and returns `{uuid, logIndex, checkpoint}` which we persist next to the decision.
|
||||
5. **Export.** API/CLI endpoints expose `decision.openvex.json`, `decision.dsse.json`, `rekor.txt`, and evidence metadata so Export Center + bench automation can mirror them into `bench/findings/**` as defined in the [VEX Evidence Playbook](../../benchmarks/vex-evidence-playbook.md).
|
||||
|
||||
All payloads are immutable and include analyzer fingerprints (`scanner.native@sha256:...`, `policyEngine@sha256:...`) so replay tooling can recompute identical digests. Determinism tests cover both the OpenVEX JSON and the DSSE payload bytes.
|
||||
|
||||
---
|
||||
|
||||
## 7 · Security & Tenancy
|
||||
|
||||
- **Auth:** All API calls pass through Authority gateway; DPoP tokens enforced for service-to-service (Policy Engine service principal). CLI/UI tokens include scope claims.
|
||||
- **Scopes:** Mutations require `policy:*` scopes corresponding to action; `effective:write` restricted to service identity.
|
||||
- **Tenancy:** All queries filter by `tenant`. Service identity uses `tenant-global` for shared policies; cross-tenant reads prohibited unless `policy:tenant-admin` scope present.
|
||||
- **Secrets:** Configuration loaded via environment variables or sealed secrets; runtime avoids writing secrets to logs.
|
||||
- **Determinism guard:** Static analyzer prevents referencing forbidden namespaces; runtime guard intercepts `DateTime.Now`, `Random`, `Guid`, HTTP clients beyond allow-list.
|
||||
|
||||
@@ -31,7 +31,9 @@ src/
|
||||
├─ StellaOps.Scanner.EntryTrace/ # ENTRYPOINT/CMD → terminal program resolver (shell AST)
|
||||
├─ StellaOps.Scanner.Analyzers.OS.[Apk|Dpkg|Rpm]/
|
||||
├─ StellaOps.Scanner.Analyzers.Lang.[Java|Node|Python|Go|DotNet|Rust]/
|
||||
├─ StellaOps.Scanner.Analyzers.Native.[ELF|PE|MachO]/ # PE/Mach-O planned (M2)
|
||||
├─ StellaOps.Scanner.Analyzers.Native.[ELF|PE|MachO]/ # PE/Mach-O planned (M2)
|
||||
├─ StellaOps.Scanner.Symbols.Native/ # NEW – native symbol reader/demangler (Sprint 401)
|
||||
├─ StellaOps.Scanner.CallGraph.Native/ # NEW – function/call-edge builder + CAS emitter
|
||||
├─ StellaOps.Scanner.Emit.CDX/ # CycloneDX (JSON + Protobuf)
|
||||
├─ StellaOps.Scanner.Emit.SPDX/ # SPDX 3.0.1 JSON
|
||||
├─ StellaOps.Scanner.Diff/ # image→layer→component three‑way diff
|
||||
@@ -216,14 +218,17 @@ When `scanner.events.enabled = true`, the WebService serialises the signed repor
|
||||
|
||||
> **Rule:** We only report components proven **on disk** with authoritative metadata. Lockfiles are evidence only.
|
||||
|
||||
**C) Native link graph**
|
||||
|
||||
* **ELF**: parse `PT_INTERP`, `DT_NEEDED`, RPATH/RUNPATH, **GNU symbol versions**; map **SONAMEs** to file paths; link executables → libs.
|
||||
* **PE/Mach‑O** (planned M2): import table, delay‑imports; version resources; code signatures.
|
||||
* Map libs back to **OS packages** if possible (via file lists); else emit `bin:{sha256}` components.
|
||||
* The exported metadata (`stellaops.os.*` properties, license list, source package) feeds policy scoring and export pipelines
|
||||
directly – Policy evaluates quiet rules against package provenance while Exporters forward the enriched fields into
|
||||
downstream JSON/Trivy payloads.
|
||||
**C) Native link graph**
|
||||
|
||||
* **ELF**: parse `PT_INTERP`, `DT_NEEDED`, RPATH/RUNPATH, **GNU symbol versions**; map **SONAMEs** to file paths; link executables → libs.
|
||||
* **PE/Mach‑O** (planned M2): import table, delay‑imports; version resources; code signatures.
|
||||
* Map libs back to **OS packages** if possible (via file lists); else emit `bin:{sha256}` components.
|
||||
* The exported metadata (`stellaops.os.*` properties, license list, source package) feeds policy scoring and export pipelines
|
||||
directly – Policy evaluates quiet rules against package provenance while Exporters forward the enriched fields into
|
||||
downstream JSON/Trivy payloads.
|
||||
* Sprint 401 introduces `StellaOps.Scanner.Symbols.Native` (DWARF/PDB reader + demangler) and `StellaOps.Scanner.CallGraph.Native`
|
||||
(function boundary detector + call-edge builder). These libraries feed `FuncNode`/`CallEdge` CAS bundles and enrich reachability
|
||||
graphs with `{code_id, confidence, evidence}` so Signals/Policy/UI can cite function-level justifications.
|
||||
|
||||
**D) EntryTrace (ENTRYPOINT/CMD → terminal program)**
|
||||
|
||||
|
||||
@@ -20,17 +20,17 @@
|
||||
|
||||
## 1) Responsibilities (contract)
|
||||
|
||||
1. **Authenticate** caller with **OpTok** (Authority OIDC, DPoP or mTLS‑bound).
|
||||
2. **Authorize** scopes (`signer.sign`) + audience (`aud=signer`) + tenant/installation.
|
||||
3. **Validate entitlement** via **PoE** (Proof‑of‑Entitlement) against Cloud Licensing `/license/introspect`.
|
||||
4. **Verify release integrity** of the **scanner** image digest presented in the request: must be **cosign‑signed** by Stella Ops release key, discoverable via **OCI Referrers API**.
|
||||
5. **Enforce plan & quotas** (concurrency/QPS/artifact size/rate caps).
|
||||
6. **Mint signing identity**:
|
||||
1. **Authenticate** caller with **OpTok** (Authority OIDC, DPoP or mTLS‑bound).
|
||||
2. **Authorize** scopes (`signer.sign`) + audience (`aud=signer`) + tenant/installation.
|
||||
3. **Validate entitlement** via **PoE** (Proof‑of‑Entitlement) against Cloud Licensing `/license/introspect`.
|
||||
4. **Verify release integrity** of the **scanner** image digest presented in the request: must be **cosign‑signed** by Stella Ops release key, discoverable via **OCI Referrers API**.
|
||||
5. **Enforce plan & quotas** (concurrency/QPS/artifact size/rate caps).
|
||||
6. **Mint signing identity**:
|
||||
|
||||
* **Keyless** (default): get a short‑lived X.509 cert from **Fulcio** using the Signer’s OIDC identity and sign the DSSE.
|
||||
* **Keyful** (optional): sign with an HSM/KMS key.
|
||||
7. **Return DSSE bundle** (subject digests + predicate + cert chain or KMS key id).
|
||||
8. **Audit** every decision; expose metrics.
|
||||
7. **Return DSSE bundle** (subject digests + predicate + cert chain or KMS key id).
|
||||
8. **Audit** every decision; expose metrics.
|
||||
|
||||
---
|
||||
|
||||
@@ -115,18 +115,30 @@ Errors (RFC 7807):
|
||||
* `400 invalid_request` (schema/predicate/type invalid)
|
||||
* `500 signing_unavailable` (Fulcio/KMS outage)
|
||||
|
||||
### 3.2 `GET /verify/referrers?imageDigest=<sha256>`
|
||||
|
||||
Checks whether the **image** at digest is signed by **Stella Ops release key**.
|
||||
|
||||
Response:
|
||||
### 3.2 `GET /verify/referrers?imageDigest=<sha256>`
|
||||
|
||||
Checks whether the **image** at digest is signed by **Stella Ops release key**.
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{ "trusted": true, "signatures": [ { "type": "cosign", "digest": "sha256:...", "signedBy": "StellaOps Release 2027 Q2" } ] }
|
||||
```
|
||||
|
||||
{ "trusted": true, "signatures": [ { "type": "cosign", "digest": "sha256:...", "signedBy": "StellaOps Release 2027 Q2" } ] }
|
||||
```
|
||||
|
||||
> **Note:** This endpoint is also used internally by Signer before issuing signatures.
|
||||
|
||||
### 3.3 Predicate catalog (Sprint 401 update)
|
||||
|
||||
Signer now enforces an allowlist of predicate identifiers:
|
||||
|
||||
| Predicate | Description | Producer |
|
||||
|-----------|-------------|----------|
|
||||
| `stella.ops/sbom@v1` | SBOM/report attestation (existing). | Scanner WebService. |
|
||||
| `stella.ops/promotion@v1` | Promotion evidence (see `docs/release/promotion-attestations.md`). | DevOps/Export Center. |
|
||||
| `stella.ops/vexDecision@v1` | OpenVEX decision for a single `(cve, product)` pair, including reachability evidence references. | Policy Engine / VEXer. |
|
||||
|
||||
Requests with unknown predicates receive `400 predicate_not_allowed`. Policy Engine must supply the OpenVEX JSON as the `predicate` body; Signer preserves payload bytes verbatim so DSSE digest = OpenVEX digest.
|
||||
|
||||
---
|
||||
|
||||
### KMS drivers (keyful mode)
|
||||
|
||||
@@ -36,6 +36,8 @@ This guide translates the deterministic reachability blueprint into concrete wor
|
||||
|
||||
| Stream | Owner Guild(s) | Key deliverables |
|
||||
|--------|----------------|------------------|
|
||||
| **Native symbols & callgraphs** | Scanner Worker · Symbols Guild | Ship `Scanner.Symbols.Native` + `Scanner.CallGraph.Native`, integrate Symbol Manifest v1, demangle Itanium/MSVC names, emit `FuncNode`/`CallEdge` CAS bundles (task `SCANNER-NATIVE-401-015`). |
|
||||
| **Reachability store** | Signals · BE-Base Platform | Provision shared Mongo collections (`func_nodes`, `call_edges`, `cve_func_hits`), indexes, and repositories plus REST hooks for reuse (task `SIG-STORE-401-016`). |
|
||||
| **Language lifters** | Scanner Worker | CLI/hosted lifters for DotNet, Go, Node/Deno, JVM, Rust, Swift, Binary, Shell with CAS uploads and richgraph output |
|
||||
| **Signals ingestion & scoring** | Signals | `/callgraphs`, `/runtime-facts` (JSON + NDJSON/gzip), `/graphs/{id}`, `/reachability/recompute` GA; CAS-backed storage, runtime dedupe, BFS+predicates scoring |
|
||||
| **Runtime capture** | Zastava + Runtime Guild | EntryTrace/eBPF samplers, NDJSON batches (symbol IDs + timestamps + counts) |
|
||||
@@ -104,7 +106,8 @@ Each sprint is two weeks; refer to `docs/implplan/SPRINT_401_reachability_eviden
|
||||
|
||||
- Place developer-facing updates here (`docs/reachability`).
|
||||
- [Function-level evidence guide](function-level-evidence.md) captures the Nov 2025 advisory scope, task references, and schema expectations; keep it in lockstep with sprint status.
|
||||
- Operator runbooks (`docs/runbooks/reachability-runtime.md`) – TODO reference to be added when runtime pipeline lands.
|
||||
- [Reachability runtime runbook](../runbooks/reachability-runtime.md) now documents ingestion, CAS staging, air-gap handling, and troubleshooting—link every runtime feature PR to this guide.
|
||||
- [VEX Evidence Playbook](../benchmarks/vex-evidence-playbook.md) defines the bench repo layout, artifact shapes, verifier tooling, and metrics; keep it updated when Policy/Signer/CLI features land.
|
||||
- Update module dossiers (Scanner, Signals, Replay, Authority, Policy, UI) once each guild lands work.
|
||||
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Function-Level Evidence Readiness (Nov 2025 Advisory)
|
||||
|
||||
_Last updated: 2025-11-09. Owner: Business Analysis Guild._
|
||||
_Last updated: 2025-11-12. Owner: Business Analysis Guild._
|
||||
|
||||
This memo captures the outstanding work required to make Stella Ops scanners emit stable, function-level evidence that matches the November 2025 advisory. It does **not** implement any code; instead it enumerates requirements, links them to sprint tasks, and spells out the schema/API updates that the next agent must land.
|
||||
|
||||
@@ -62,6 +62,18 @@ Out of scope: implementing disassemblers or symbol servers; those will be handle
|
||||
* Write CLI/API walkthroughs in `docs/09_API_CLI_REFERENCE.md` and `docs/api/policy.md` showing how to request reachability evidence and verify DSSE chains.
|
||||
* Produce OpenVEX + replay samples under `samples/reachability/` showing `facts.type = "stella.reachability"` with `graph_hash` and `code_id` arrays.
|
||||
|
||||
### 3.6 Native lifter & Reachability Store (SCANNER-NATIVE-401-015 / SIG-STORE-401-016)
|
||||
|
||||
* Stand up `Scanner.Symbols.Native` + `Scanner.CallGraph.Native` libraries that:
|
||||
* parse ELF (DWARF + `.symtab`/`.dynsym`), PE/COFF (CodeView/PDB), and stripped binaries via probabilistic carving;
|
||||
* emit deterministic `FuncNode` + `CallEdge` records with demangled names, language hints, and `{confidence,evidence}` arrays; and
|
||||
* attach analyzer + toolchain identifiers consumed by `richgraph-v1`.
|
||||
* Introduce `Reachability.Store` collections in Mongo:
|
||||
* `func_nodes` – keyed by `func:<format>:<sha256>:<va>` with `{binDigest,name,addr,size,lang,confidence,sym}`.
|
||||
* `call_edges` – `{from,to,kind,confidence,evidence[]}` linking internal/external nodes.
|
||||
* `cve_func_hits` – `{cve,purl,func_id,match_kind,confidence,source}` for advisory alignment.
|
||||
* Build indexes (`binDigest+name`, `from→to`, `cve+func_id`) and expose repository interfaces so Scanner, Signals, and Policy can reuse the same canonical data without duplicating queries.
|
||||
|
||||
---
|
||||
|
||||
## 4. Schema & API Touchpoints
|
||||
@@ -86,6 +98,50 @@ API contracts to amend:
|
||||
- Signals dedupes events, merges metadata, and persists the aggregated `RuntimeFacts` onto `ReachabilityFactDocument`. These facts now feed reachability scoring (SIGNALS-24-004/005) as part of the runtime bonus lattice.
|
||||
- Outstanding work: record CAS URIs for runtime traces, emit provenance events, and expose the enriched context to Policy/Replay consumers.
|
||||
|
||||
### 4.2 Reachability store layout (SIG-STORE-401-016)
|
||||
|
||||
All producers **must** persist native function evidence using the shared collections below (names are advisory; exact names live in Mongo options):
|
||||
|
||||
```json
|
||||
// func_nodes
|
||||
{
|
||||
"_id": "func:ELF:sha256:4012a0",
|
||||
"binDigest": "sha256:deadbeef...",
|
||||
"name": "ssl3_read_bytes",
|
||||
"addr": "0x4012a0",
|
||||
"size": 312,
|
||||
"lang": "c",
|
||||
"confidence": 0.92,
|
||||
"symbol": { "mangled": "_Z15ssl3_read_bytes", "demangled": "ssl3_read_bytes", "source": "DWARF" },
|
||||
"sym": "present"
|
||||
}
|
||||
|
||||
// call_edges
|
||||
{
|
||||
"from": "func:ELF:sha256:4012a0",
|
||||
"to": "func:ELF:sha256:40f0ff",
|
||||
"kind": "static",
|
||||
"confidence": 0.88,
|
||||
"evidence": ["reloc:.plt.got", "bb-target:0x40f0ff"]
|
||||
}
|
||||
|
||||
// cve_func_hits
|
||||
{
|
||||
"cve": "CVE-2023-XXXX",
|
||||
"purl": "pkg:generic/openssl@1.1.1u",
|
||||
"func_id": "func:ELF:sha256:4012a0",
|
||||
"match": "name+version",
|
||||
"confidence": 0.77,
|
||||
"source": "concelier:openssl-advisory"
|
||||
}
|
||||
```
|
||||
|
||||
Writers **must**:
|
||||
|
||||
1. Upsert `func_nodes` before emitting edges/hits to ensure `_id` lookups remain stable.
|
||||
2. Serialize evidence arrays in deterministic order (`reloc`, `bb-target`, `import`, …) and normalise hex casing.
|
||||
3. Attach analyzer fingerprints (`scanner.native@sha256:...`) so Replay/Policy can enforce provenance.
|
||||
|
||||
---
|
||||
|
||||
## 5. Test & Fixture Expectations
|
||||
|
||||
80
docs/runbooks/reachability-runtime.md
Normal file
80
docs/runbooks/reachability-runtime.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# Runbook — Reachability Runtime Ingestion
|
||||
|
||||
> **Audience:** Signals Guild · Zastava Guild · Scanner Guild · Ops Guild
|
||||
> **Prereqs:** `docs/reachability/DELIVERY_GUIDE.md`, `docs/reachability/function-level-evidence.md`, `docs/modules/platform/architecture-overview.md` §5
|
||||
|
||||
This runbook documents how to stage, ingest, and troubleshoot runtime evidence (`/signals/runtime-facts`) so function-level reachability data remains provable across online and air-gapped environments.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Runtime capture pipeline
|
||||
|
||||
1. **Zastava Observer / runtime probes**
|
||||
- Emit NDJSON lines with `symbolId`, `codeId`, `loaderBase`, `hitCount`, `process{Id,Name}`, `socketAddress`, `containerId`, optional `evidenceUri`, and `metadata` map.
|
||||
- Compress large batches with gzip (`.ndjson.gz`), max 10 MiB per chunk, monotonic timestamps.
|
||||
- Attach subject context via HTTP query (`scanId`, `imageDigest`, `component`, `version`) when using the streaming endpoint.
|
||||
2. **CAS staging (optional but recommended)**
|
||||
- Upload raw batches to `cas://reachability/runtime/<sha256>` before ingestion.
|
||||
- Store CAS URIs alongside probe metadata so Signals can echo them in `ReachabilityFactDocument.Metadata`.
|
||||
3. **Signals ingestion**
|
||||
- POST `/signals/runtime-facts` (JSON) for one-off uploads or stream NDJSON to `/signals/runtime-facts/ndjson` (set `Content-Encoding: gzip` when applicable).
|
||||
- Signals validates schema, dedupes events by `(symbolId, codeId, loaderBase)`, and updates `runtimeFacts` with cumulative `hitCount`.
|
||||
4. **Reachability scoring**
|
||||
- `ReachabilityScoringService` recomputes lattice states (`Unknown → Observed`), persists references to runtime CAS artifacts, and emits `signals.fact.updated` once `GAP-SIG-003` lands.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Operator checklist
|
||||
|
||||
| Step | Action | Owner | Notes |
|
||||
|------|--------|-------|-------|
|
||||
| 1 | Verify probe health (`zastava observer status`) and confirm NDJSON batches include `symbolId` + `codeId`. | Runtime Guild | Reject batches missing `symbolId`; restart probe with debug logging. |
|
||||
| 2 | Stage batches in CAS (`stella cas put reachability/runtime ...`) and record the returned URI. | Ops Guild | Required for replay-grade evidence. |
|
||||
| 3 | Call `/signals/runtime-facts/ndjson` with `tenant` and `callgraphId` headers, streaming the gzip payload. | Signals Guild | Use service identity with `signals.runtime:write`. |
|
||||
| 4 | Monitor ingestion metrics: `signals_runtime_events_total`, `signals_runtime_ingest_failures_total`. | Observability | Alert if failures exceed 1% over 5 min. |
|
||||
| 5 | Trigger recompute (`POST /signals/reachability/recompute`) when new runtime batches arrive for an active scan. | Signals Guild | Provide `callgraphId` + subject tuple. |
|
||||
| 6 | Validate Policy/UI surfaces by requesting `/policy/findings?includeReachability=true` and checking `reachability.evidence`. | Policy + UI Guilds | Ensure evidence references the CAS URIs from Step 2. |
|
||||
|
||||
---
|
||||
|
||||
## 3 · Air-gapped workflow
|
||||
|
||||
1. Export runtime NDJSON batches via Offline Kit: `offline/reachability/runtime/<scan-id>/<timestamp>.ndjson.gz` + manifest.
|
||||
2. On the secure network, load CAS entries locally (`stella cas load ...`) and invoke `stella signals runtime-facts ingest --from offline/...`.
|
||||
3. Re-run `stella replay manifest.json --section reachability` to ensure manifests cite the imported runtime digests.
|
||||
4. Sync ingestion receipts (`signals-runtime-ingest.log`) back to the air-gapped environment for audit.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Troubleshooting
|
||||
|
||||
| Symptom | Cause | Resolution |
|
||||
|---------|-------|------------|
|
||||
| `422 Unprocessable Entity: missing symbolId` | Probe emitted incomplete JSON. | Restart probe with `--include-symbols`, confirm symbol server availability, regenerate batch. |
|
||||
| `403 Forbidden: sealed-mode evidence invalid` | Signals sealed-mode verifier rejected payload (likely missing CAS proof). | Upload batch to CAS first, include `X-Reachability-Cas-Uri` header, or disable sealed-mode in non-prod. |
|
||||
| Runtime facts missing from Policy/UI | Recompute not triggered or `callgraphId` mismatch. | List facts via `/signals/reachability/facts?subject=...`, confirm `callgraphId`, then POST recompute. |
|
||||
| CAS hash mismatch during replay | Batch mutated post-ingestion. | Re-stage from original gzip, invalidate old CAS entry, rerun ingestion to regenerate manifest references. |
|
||||
|
||||
---
|
||||
|
||||
## 5 · Retention & observability
|
||||
|
||||
- Default retention: 30 days hot in Signals Mongo, 180 days in CAS (match replay policy). Configure via `signals.runtimeFacts.retentionDays`.
|
||||
- Metrics to alert on:
|
||||
- `signals_runtime_ingest_latency_seconds` (P95 < 2 s).
|
||||
- `signals_runtime_cas_miss_total` (should be 0 once CAS is mandatory).
|
||||
- Logs/traces:
|
||||
- Category `Reachability.Runtime` records ingestion batches and CAS URIs.
|
||||
- Trace attributes: `callgraphId`, `subjectKey`, `casUri`, `eventCount`.
|
||||
|
||||
---
|
||||
|
||||
## 6 · References
|
||||
|
||||
- `docs/reachability/DELIVERY_GUIDE.md`
|
||||
- `docs/reachability/function-level-evidence.md`
|
||||
- `docs/replay/DETERMINISTIC_REPLAY.md`
|
||||
- `docs/modules/platform/architecture-overview.md` §5 (Replay CAS)
|
||||
- `docs/runbooks/replay_ops.md`
|
||||
|
||||
Update this runbook whenever endpoints, retention knobs, or CAS layouts change.
|
||||
Reference in New Issue
Block a user