Add execution waves documentation and function-level evidence readiness memo
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Created `execution-waves.md` to outline the execution waves for sprints, detailing shared prerequisites, parallelism guidance, and specific sprints involved in each wave.
- Added `function-level-evidence.md` to capture the requirements for stable function-level evidence in Stella Ops scanners, including goals, scope, advisory requirements, workstreams, schema/API touchpoints, and a handoff checklist for the next agent.
This commit is contained in:
master
2025-11-09 23:06:33 +02:00
parent cef4cb2c5a
commit 9df52d84aa
15 changed files with 751 additions and 56 deletions

View File

@@ -0,0 +1,102 @@
# Function-Level Evidence Readiness (Nov 2025 Advisory)
_Last updated: 2025-11-09. Owner: Business Analysis Guild._
This memo captures the outstanding work required to make StellaOps scanners emit stable, function-level evidence that matches the November2025 advisory. It does **not** implement any code; instead it enumerates requirements, links them to sprint tasks, and spells out the schema/API updates that the next agent must land.
---
## 1. Goal & Scope
**Goal.** Anchor every vulnerability finding to an immutable `{artifact_digest, code_id}` tuple plus optional symbol hints so replayers can prove reachability against stripped binaries.
**Scope.** Scanner analyzers, runtime ingestion, Signals scoring, Replay manifests, Policy/VEX emission, CLI/UI explainers, and documentation/runbooks needed to operationalise the advisory.
Out of scope: implementing disassemblers or symbol servers; those will be handled inside the module-specific backlog tasks referenced below.
---
## 2. Advisory Requirements vs. System Gaps
| Requirement | Current gap | Task references | Notes |
|-------------|-------------|-----------------|-------|
| Immutable code identity (`code_id` = `{format, build_id, start, length}` + optional `code_block_hash`) | Callgraph nodes are opaque strings with no address metadata. | Sprint401 `GRAPH-CAS-401-001`, `GAP-SCAN-001`, `GAP-SYM-007` | `code_id` should live alongside existing `SymbolID` helpers so analyzers can emit it without duplicating logic. |
| Symbol hints (demangled name, source, confidence) | No schema fields for symbol metadata; demangling is ad-hoc per analyzer. | `GAP-SYM-007` | Require deterministic casing + `symbol.source ∈ {DWARF,PDB,SYM,none}`. |
| Runtime facts mapped to code anchors | `/signals/runtime-facts` is a stub; Zastava streams only Build-IDs. | Sprint400 `ZASTAVA-REACH-201-001`, Sprint401 `SIGNALS-RUNTIME-401-002`, `GAP-ZAS-002`, `GAP-SIG-003` | Need NDJSON schema documenting `code_id`, `symbol.sid`, `hit_count`, `loader_base`. |
| Replay/DSSE coverage | Replay manifests dont enforce hash/CAS registration for graphs/traces. | Sprint400 `REPLAY-REACH-201-005`, Sprint401 `REPLAY-401-004`, `GAP-REP-004` | Extend manifest v2 with analyzer versions + BLAKE3 digests; add DSSE predicate types. |
| Policy/VEX/UI explainability | Policy uses coarse `reachability:*` tags; UI/CLI cannot show call paths or evidence hashes. | Sprint401 `POLICY-VEX-401-006`, `UI-CLI-401-007`, `GAP-POL-005`, `GAP-VEX-006`, `EXPERIENCE-GAP-401-012` | Evidence blocks must cite `code_id`, graph hash, runtime CAS URI, analyzer version. |
| Operator documentation & samples | No guide shows how to replay `{build_id,start,len}` across CLI/API. | Sprint401 `QA-DOCS-401-008`, `GAP-DOC-008` | Produce samples under `samples/reachability/**` plus CLI walkthroughs. |
---
## 3. Workstreams & Expectations
### 3.1 Scanner Symbolization (GAP-SCAN-001 / GAP-SYM-007)
* Define `SymbolID` helpers that glue together `{artifact_digest, file`, optional `section`, `addr`, `length`, `code_block_hash`}.
* Update analyzer contracts so every analyzer returns both `symbol_id` and `code_id`, with demangled names stored under the new `symbol` block.
* Persist the data into `richgraph-v1` payloads and attach CAS URIs via `StellaOps.Scanner.Reachability`.
* Deliver fixtures in `tests/reachability/StellaOps.ScannerSignals.IntegrationTests` that prove determinism (same hash when analyzer flags reorder).
### 3.2 Runtime + Signals (GAP-ZAS-002 / GAP-SIG-003)
* Extend Zastava Observer NDJSON schema to emit: `symbol_id`, `code_id`, `hit_count`, `observed_at`, `loader_base`, `process.buildId`.
* Implement `/signals/runtime-facts` ingestion (gzip + NDJSON) with CAS-backed storage under `cas://reachability/runtime/{sha256}`.
* Update `ReachabilityScoringService` to lattice states and include runtime evidence references plus CAS URIs in `ReachabilityFactDocument.Metadata`.
### 3.3 Replay & Evidence (GAP-REP-004)
* Enforce CAS registration + BLAKE3 hashing before manifest writes (graphs and traces).
* Teach `ReachabilityReplayWriter` to require analyzer name/version, graph kind, `code_id` coverage summary.
* Update `docs/replay/DETERMINISTIC_REPLAY.md` once schema v2 is finalized.
### 3.4 Policy, VEX, CLI/UI (GAP-POL-005 / GAP-VEX-006)
* Policy Engine: ingest new reachability facts, expose `reachability.state`, `max_path_conf`, and `evidence.graph_hash` via SPL + API.
* CLI/UI: add `stella graph explain` and explain drawer showing call path (`SymbolID` list), code anchors, runtime hits, DSSE references.
* Notify templates: include short evidence summary (first hop + truncated `code_id`).
### 3.5 Documentation & Samples (GAP-DOC-008)
* Publish schema diffs in `docs/data/evidence-schema.md` (new file) covering SBOM evidence nodes, runtime NDJSON, and API responses.
* Write CLI/API walkthroughs in `docs/09_API_CLI_REFERENCE.md` and `docs/api/policy.md` showing how to request reachability evidence and verify DSSE chains.
* Produce OpenVEX + replay samples under `samples/reachability/` showing `facts.type = "stella.reachability"` with `graph_hash` and `code_id` arrays.
---
## 4. Schema & API Touchpoints
The next implementation pass must cover the following documents/files (create them if missing):
1. `docs/data/evidence-schema.md` authoritative schema for `{code_id, symbol, tool}` blocks.
2. `docs/runbooks/reachability-runtime.md` operator steps for staging runtime ingestion bundles, retention, and troubleshooting.
3. `docs/runbooks/replay_ops.md` add section detailing replay verification using the new graph/runtime CAS entries.
API contracts to amend:
- `POST /signals/callgraphs` response should include `graphHash` (BLAKE3) once `GRAPH-CAS-401-001` lands.
- `POST /signals/runtime-facts` request body schema (NDJSON) with `symbol_id`, `code_id`, `hit_count`, `loader_base`.
- `GET /policy/findings` payload must surface `reachability.evidence[]` objects.
---
## 5. Test & Fixture Expectations
- **Reachbench fixtures**: update golden cases with `code_id` + `symbol` metadata. Ensure both reachable/unreachable variants still pass once graphs contain the richer IDs.
- **Signals unit tests**: add deterministic tests for lattice scoring + runtime evidence linking (`tests/reachability/StellaOps.Signals.Reachability.Tests`).
- **Replay tests**: extend `tests/reachability/StellaOps.Replay.Core.Tests` to assert manifest v2 serialization and hash enforcement.
All fixtures must remain deterministic: sort nodes/edges, normalise casing, and freeze timestamps in test data.
---
## 6. Handoff Checklist for the Next Agent
1. Confirm sprint entries (`SPRINT_400` and `SPRINT_401`) remain in sync when moving `GAP-*` tasks to DOING/DONE.
2. Start with `GAP-SYM-007` (schema/helper implementation) because downstream work depends on the new `code_id` payload shape.
3. Once schema PR merges, coordinate with Signals + Policy guilds to align on CAS naming and DSSE predicates before wiring APIs.
4. Update the docs listed in §4 as each component lands; keep this file current with statuses and links to PRs/ADRs.
5. Before shipping, run the reachbench fixtures end-to-end and capture hashes for inclusion in replay docs.
Keep this document updated as tasks change state; it is the authoritative hand-off note for the advisory.