Add execution waves documentation and function-level evidence readiness memo
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Created `execution-waves.md` to outline the execution waves for sprints, detailing shared prerequisites, parallelism guidance, and specific sprints involved in each wave. - Added `function-level-evidence.md` to capture the requirements for stable function-level evidence in Stella Ops scanners, including goals, scope, advisory requirements, workstreams, schema/API touchpoints, and a handoff checklist for the next agent.
This commit is contained in:
master
@@ -103,6 +103,7 @@ Each sprint is two weeks; refer to `docs/implplan/SPRINT_401_reachability_eviden
|
||||
## 7. Documentation & Runbooks
|
||||
|
||||
- Place developer-facing updates here (`docs/reachability`).
|
||||
- [Function-level evidence guide](function-level-evidence.md) captures the Nov 2025 advisory scope, task references, and schema expectations; keep it in lockstep with sprint status.
|
||||
- Operator runbooks (`docs/runbooks/reachability-runtime.md`) – TODO reference to be added when runtime pipeline lands.
|
||||
- Update module dossiers (Scanner, Signals, Replay, Authority, Policy, UI) once each guild lands work.
|
||||
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
# Reachability Evidence – Gap Analysis & Task References
|
||||
|
||||
_Last updated: 2025-11-09 (Business Analysis role)._
|
||||
_Scope:_ outline the missing functionality required to make binary-level reachability evidence first-class across Scanner, Signals, Policy, Replay, and VEX emission.
|
||||
|
||||
## 1. Source Materials
|
||||
|
||||
| Area | Reference |
|
||||
|------|-----------|
|
||||
| Architecture vision | `docs/reachability/DELIVERY_GUIDE.md`, `docs/modules/platform/architecture-overview.md:145` |
|
||||
| Active sprints | `docs/implplan/SPRINT_400_runtime_facts_static_callgraph_union.md`, `docs/implplan/SPRINT_401_reachability_evidence_chain.md` |
|
||||
| Current implementations | `src/Signals/StellaOps.Signals/Program.cs:214-287`, `src/Signals/StellaOps.Signals/Services/CallgraphIngestionService.cs`, `src/Signals/StellaOps.Signals/Services/ReachabilityScoringService.cs`, `src/Scanner/__Libraries/StellaOps.Scanner.Reachability`, `tests/reachability/*` |
|
||||
|
||||
Use this document to break down outstanding work into actionable tasks and to keep documentation links synchronized.
|
||||
|
||||
## 2. Current Snapshot (11 Nov 2025)
|
||||
|
||||
1. **Callgraph ingestion exists** – Signals exposes `/signals/callgraphs` and stores graphs + CAS metadata (`Program.cs`, `CallgraphIngestionService`).
|
||||
2. **Reachability recompute API exists but is simplistic** – BFS scoring with static confidences, no lattice states, no CAS evidence linking.
|
||||
3. **Runtime ingestion is a stub** – `/signals/runtime-facts` returns HTTP 501.
|
||||
4. **Scanner Worker doesn’t emit canonical SymbolIDs/graphs** – `StellaOps.Scanner.Reachability` library exists, yet Worker binaries do not reference it.
|
||||
5. **Replay manifests record reachability via helpers** – `ReachabilityReplayWriter` can add graph/trace refs, but manifests don’t enforce CAS registration/hashing.
|
||||
6. **Policy/UI still consume coarse `reachability:*` tags** – no OpenVEX evidence blocks or graph hashes attached to statements/events.
|
||||
|
||||
## 3. Gap Breakdown & Tasks
|
||||
|
||||
Canonical sprint tracking for these tasks now lives in `docs/implplan/SPRINT_400_runtime_facts_static_callgraph_union.md` and `docs/implplan/SPRINT_401_reachability_evidence_chain.md`. Use the table below as a consolidated reference when planning cross-guild work.
|
||||
|
||||
| Task ID | Module / Doc anchor | Description | Dependencies | Deliverables |
|
||||
|---------|--------------------|-------------|--------------|--------------|
|
||||
| GAP-SCAN-001 | `src/Scanner/StellaOps.Scanner.Worker`, `docs/modules/scanner/architecture.md` | Implement binary/language Symbolizers that emit `richgraph-v1` payloads with canonical `SymbolID = {file:hash, section, addr, name, linkage}`. Persist graphs to CAS and register them via `ReachabilityGraphBuilder`. | Sprint 400 `SCAN-REACH-201-002` | Analyzer services + config docs updated, sample graph fixtures, regression tests under `tests/reachability/StellaOps.ScannerSignals.IntegrationTests`. |
|
||||
| GAP-ZAS-002 | `src/Zastava/StellaOps.Zastava.Observer`, `docs/modules/zastava/architecture.md` | Stream runtime NDJSON batches with `SymbolID`, hit counts, CAS URIs to `/signals/runtime-facts`. Capture build-ids + entrypoint context per sprint spec. | Sprint 400 `ZASTAVA-REACH-201-001` | Observer implementation, operator runbook `docs/runbooks/reachability-runtime.md`, fixture updates. |
|
||||
| GAP-SIG-003 | `src/Signals/StellaOps.Signals/Program.cs`, `ReachabilityScoringService.cs`, `docs/reachability/DELIVERY_GUIDE.md#5.2` | Finish `/signals/runtime-facts`, introduce CAS-backed runtime storage, extend scoring to lattice states (`Unknown/NotPresent/Unreachable/Conditional/Reachable/Observed`) with per-path confidence accumulation. Emit `signals.fact.updated` events. | Sprint 401 `SIGNALS-RUNTIME-401-002`, `SIGNALS-SCORING-401-003` | API schema, Mongo indices, deterministic scoring tests (`tests/reachability/StellaOps.Signals.Reachability.Tests`). |
|
||||
| GAP-REP-004 | `src/__Libraries/StellaOps.Replay.Core`, `docs/replay/DETERMINISTIC_REPLAY.md` | Enforce CAS registration + BLAKE3 hashing for graphs/traces before manifest writes. Upgrade manifest schema v2 to include analyzer versions + policy thresholds. | Sprint 400 `REPLAY-REACH-201-005`, Sprint 401 `REPLAY-401-004` | Updated schema docs, fixture pack coverage (`tests/reachability/StellaOps.Replay.Core.Tests`). |
|
||||
| GAP-POL-005 | `src/Policy/StellaOps.Policy.Engine`, `docs/modules/policy/architecture.md` | Ingest Signals reachability facts, expose `reachability.state/confidence` in SPL, and generate OpenVEX evidence blocks referencing graph hashes + runtime facts. Implement policy threshold (e.g., affected if `max_path_conf ≥ 0.6`). | Sprint 401 `POLICY-VEX-401-006` | Updated policy schemas (`policy-scoring-schema@1.json`), OpenVEX templates, backend tests.
|
||||
| GAP-VEX-006 | `docs/modules/excititor/architecture.md`, `docs/modules/ui/architecture.md`, `docs/implplan/SPRINT_401_reachability_evidence_chain.md` | Wire VEX emission/UI surfaces: CLI/UI explain drawer with call-path visualization, DSSE evidence attachments, `--threshold` and `--evidence=graph` flags. | Sprint 401 `UI-CLI-401-007` | CLI documentation, UI walkthrough, Notify templates referencing reachability evidence. |
|
||||
|
||||
## 4. Documentation Actions
|
||||
|
||||
1. **Module dossiers** – Once each GAP task lands, update the matching module architecture doc to reflect binary reachability specifics (symbol schema, APIs, thresholds).
|
||||
2. **Runbooks** – Create `docs/runbooks/reachability-runtime.md` for operators (Zastava deployment, retention, troubleshooting) and extend `docs/runbooks/replay_ops.md` with reachability CAS sections.
|
||||
3. **API references** – Add `/signals/runtime-facts` and explain reachability fields to `docs/09_API_CLI_REFERENCE.md` and `docs/api/policy.md`.
|
||||
4. **Sample payloads** – Under `samples/`, add OpenVEX examples that include `facts.type = stella.reachability` with `graph_hash`, entrypoints, and analyzer versions.
|
||||
|
||||
## 5. Next Steps for Business Analysis
|
||||
|
||||
- Socialize this gap list with module owners; confirm task ownership aligns with the sprint trackers.
|
||||
- Link this document from `docs/reachability/DELIVERY_GUIDE.md` so engineers can reference the gap tasks quickly.
|
||||
- Revisit after Sprint 401 midpoint to mark completed tasks and add any newly discovered blockers.
|
||||
102
docs/reachability/function-level-evidence.md
Normal file
102
docs/reachability/function-level-evidence.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# Function-Level Evidence Readiness (Nov 2025 Advisory)
|
||||
|
||||
_Last updated: 2025-11-09. Owner: Business Analysis Guild._
|
||||
|
||||
This memo captures the outstanding work required to make Stella Ops scanners emit stable, function-level evidence that matches the November 2025 advisory. It does **not** implement any code; instead it enumerates requirements, links them to sprint tasks, and spells out the schema/API updates that the next agent must land.
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal & Scope
|
||||
|
||||
**Goal.** Anchor every vulnerability finding to an immutable `{artifact_digest, code_id}` tuple plus optional symbol hints so replayers can prove reachability against stripped binaries.
|
||||
|
||||
**Scope.** Scanner analyzers, runtime ingestion, Signals scoring, Replay manifests, Policy/VEX emission, CLI/UI explainers, and documentation/runbooks needed to operationalise the advisory.
|
||||
|
||||
Out of scope: implementing disassemblers or symbol servers; those will be handled inside the module-specific backlog tasks referenced below.
|
||||
|
||||
---
|
||||
|
||||
## 2. Advisory Requirements vs. System Gaps
|
||||
|
||||
| Requirement | Current gap | Task references | Notes |
|
||||
|-------------|-------------|-----------------|-------|
|
||||
| Immutable code identity (`code_id` = `{format, build_id, start, length}` + optional `code_block_hash`) | Callgraph nodes are opaque strings with no address metadata. | Sprint 401 `GRAPH-CAS-401-001`, `GAP-SCAN-001`, `GAP-SYM-007` | `code_id` should live alongside existing `SymbolID` helpers so analyzers can emit it without duplicating logic. |
|
||||
| Symbol hints (demangled name, source, confidence) | No schema fields for symbol metadata; demangling is ad-hoc per analyzer. | `GAP-SYM-007` | Require deterministic casing + `symbol.source ∈ {DWARF,PDB,SYM,none}`. |
|
||||
| Runtime facts mapped to code anchors | `/signals/runtime-facts` is a stub; Zastava streams only Build-IDs. | Sprint 400 `ZASTAVA-REACH-201-001`, Sprint 401 `SIGNALS-RUNTIME-401-002`, `GAP-ZAS-002`, `GAP-SIG-003` | Need NDJSON schema documenting `code_id`, `symbol.sid`, `hit_count`, `loader_base`. |
|
||||
| Replay/DSSE coverage | Replay manifests don’t enforce hash/CAS registration for graphs/traces. | Sprint 400 `REPLAY-REACH-201-005`, Sprint 401 `REPLAY-401-004`, `GAP-REP-004` | Extend manifest v2 with analyzer versions + BLAKE3 digests; add DSSE predicate types. |
|
||||
| Policy/VEX/UI explainability | Policy uses coarse `reachability:*` tags; UI/CLI cannot show call paths or evidence hashes. | Sprint 401 `POLICY-VEX-401-006`, `UI-CLI-401-007`, `GAP-POL-005`, `GAP-VEX-006`, `EXPERIENCE-GAP-401-012` | Evidence blocks must cite `code_id`, graph hash, runtime CAS URI, analyzer version. |
|
||||
| Operator documentation & samples | No guide shows how to replay `{build_id,start,len}` across CLI/API. | Sprint 401 `QA-DOCS-401-008`, `GAP-DOC-008` | Produce samples under `samples/reachability/**` plus CLI walkthroughs. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Workstreams & Expectations
|
||||
|
||||
### 3.1 Scanner Symbolization (GAP-SCAN-001 / GAP-SYM-007)
|
||||
|
||||
* Define `SymbolID` helpers that glue together `{artifact_digest, file`, optional `section`, `addr`, `length`, `code_block_hash`}.
|
||||
* Update analyzer contracts so every analyzer returns both `symbol_id` and `code_id`, with demangled names stored under the new `symbol` block.
|
||||
* Persist the data into `richgraph-v1` payloads and attach CAS URIs via `StellaOps.Scanner.Reachability`.
|
||||
* Deliver fixtures in `tests/reachability/StellaOps.ScannerSignals.IntegrationTests` that prove determinism (same hash when analyzer flags reorder).
|
||||
|
||||
### 3.2 Runtime + Signals (GAP-ZAS-002 / GAP-SIG-003)
|
||||
|
||||
* Extend Zastava Observer NDJSON schema to emit: `symbol_id`, `code_id`, `hit_count`, `observed_at`, `loader_base`, `process.buildId`.
|
||||
* Implement `/signals/runtime-facts` ingestion (gzip + NDJSON) with CAS-backed storage under `cas://reachability/runtime/{sha256}`.
|
||||
* Update `ReachabilityScoringService` to lattice states and include runtime evidence references plus CAS URIs in `ReachabilityFactDocument.Metadata`.
|
||||
|
||||
### 3.3 Replay & Evidence (GAP-REP-004)
|
||||
|
||||
* Enforce CAS registration + BLAKE3 hashing before manifest writes (graphs and traces).
|
||||
* Teach `ReachabilityReplayWriter` to require analyzer name/version, graph kind, `code_id` coverage summary.
|
||||
* Update `docs/replay/DETERMINISTIC_REPLAY.md` once schema v2 is finalized.
|
||||
|
||||
### 3.4 Policy, VEX, CLI/UI (GAP-POL-005 / GAP-VEX-006)
|
||||
|
||||
* Policy Engine: ingest new reachability facts, expose `reachability.state`, `max_path_conf`, and `evidence.graph_hash` via SPL + API.
|
||||
* CLI/UI: add `stella graph explain` and explain drawer showing call path (`SymbolID` list), code anchors, runtime hits, DSSE references.
|
||||
* Notify templates: include short evidence summary (first hop + truncated `code_id`).
|
||||
|
||||
### 3.5 Documentation & Samples (GAP-DOC-008)
|
||||
|
||||
* Publish schema diffs in `docs/data/evidence-schema.md` (new file) covering SBOM evidence nodes, runtime NDJSON, and API responses.
|
||||
* Write CLI/API walkthroughs in `docs/09_API_CLI_REFERENCE.md` and `docs/api/policy.md` showing how to request reachability evidence and verify DSSE chains.
|
||||
* Produce OpenVEX + replay samples under `samples/reachability/` showing `facts.type = "stella.reachability"` with `graph_hash` and `code_id` arrays.
|
||||
|
||||
---
|
||||
|
||||
## 4. Schema & API Touchpoints
|
||||
|
||||
The next implementation pass must cover the following documents/files (create them if missing):
|
||||
|
||||
1. `docs/data/evidence-schema.md` – authoritative schema for `{code_id, symbol, tool}` blocks.
|
||||
2. `docs/runbooks/reachability-runtime.md` – operator steps for staging runtime ingestion bundles, retention, and troubleshooting.
|
||||
3. `docs/runbooks/replay_ops.md` – add section detailing replay verification using the new graph/runtime CAS entries.
|
||||
|
||||
API contracts to amend:
|
||||
|
||||
- `POST /signals/callgraphs` response should include `graphHash` (BLAKE3) once `GRAPH-CAS-401-001` lands.
|
||||
- `POST /signals/runtime-facts` request body schema (NDJSON) with `symbol_id`, `code_id`, `hit_count`, `loader_base`.
|
||||
- `GET /policy/findings` payload must surface `reachability.evidence[]` objects.
|
||||
|
||||
---
|
||||
|
||||
## 5. Test & Fixture Expectations
|
||||
|
||||
- **Reachbench fixtures**: update golden cases with `code_id` + `symbol` metadata. Ensure both reachable/unreachable variants still pass once graphs contain the richer IDs.
|
||||
- **Signals unit tests**: add deterministic tests for lattice scoring + runtime evidence linking (`tests/reachability/StellaOps.Signals.Reachability.Tests`).
|
||||
- **Replay tests**: extend `tests/reachability/StellaOps.Replay.Core.Tests` to assert manifest v2 serialization and hash enforcement.
|
||||
|
||||
All fixtures must remain deterministic: sort nodes/edges, normalise casing, and freeze timestamps in test data.
|
||||
|
||||
---
|
||||
|
||||
## 6. Handoff Checklist for the Next Agent
|
||||
|
||||
1. Confirm sprint entries (`SPRINT_400` and `SPRINT_401`) remain in sync when moving `GAP-*` tasks to DOING/DONE.
|
||||
2. Start with `GAP-SYM-007` (schema/helper implementation) because downstream work depends on the new `code_id` payload shape.
|
||||
3. Once schema PR merges, coordinate with Signals + Policy guilds to align on CAS naming and DSSE predicates before wiring APIs.
|
||||
4. Update the docs listed in §4 as each component lands; keep this file current with statuses and links to PRs/ADRs.
|
||||
5. Before shipping, run the reachbench fixtures end-to-end and capture hashes for inclusion in replay docs.
|
||||
|
||||
Keep this document updated as tasks change state; it is the authoritative hand-off note for the advisory.
|
||||
|
||||
Reference in New Issue
Block a user