prep docs and service updates
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

This commit is contained in:
master
2025-11-21 06:56:36 +00:00
parent ca35db9ef4
commit d519782a8f
242 changed files with 17293 additions and 13367 deletions

View File

@@ -0,0 +1,69 @@
# Patch-Oracles QA Pattern (Nov 2026)
Patch oracles are paired vulnerable/fixed binaries that prove our analyzers can see the function and call-edge deltas introduced by real CVE fixes. This file replaces earlier advisory text; use it directly when adding tests.
## 1. Workflow (per CVE)
1) Pick a CVE with a small, clean fix (e.g., OpenSSL, zlib, BusyBox). Identify vulnerable commit `A` and fixed commit `B`.
2) Build two stripped binaries (`vuln`, `fixed`) with identical toolchains/flags; keep a tiny harness that exercises the affected path.
3) Run Scanner binary analyzers to emit `richgraph-v1` for each binary.
4) Diff graphs: expect new/removed functions and edges to match the patch (e.g., `foo_parse -> validate_len` added; `foo_parse -> memcpy` removed).
5) Fail the test if expected functions/edges are absent or unchanged.
## 2. Oracle manifest (YAML)
```yaml
cve: CVE-YYYY-XXXX
target: libfoo 1.2.3
build:
cc: clang
cflags: [-O2, -fno-omit-frame-pointer]
ldflags: []
strip: true
expect:
functions_added: [validate_len]
functions_removed: [unsafe_copy]
edges_added:
- { caller: foo_parse, callee: validate_len }
edges_removed:
- { caller: foo_parse, callee: memcpy }
tolerances:
allow_unresolved_symbols: 0
allow_extra_funcs: 2
```
Place manifests under `tests/reachability/patch-oracles/<cve>/oracle.yml` next to the sources/build scripts.
## 3. Repository layout
```
tests/reachability/patch-oracles/
CVE-YYYY-XXXX-foo/
src/ # vuln + fixed sources + harness
build.sh # produces ./out/vuln ./out/fixed
oracle.yml
```
## 4. Harness rules
- Output binaries to `out/vuln` and `out/fixed` with deterministic flags and stripped symbols.
- Record toolchain version in a sidecar `build-meta.json` so Replay captures provenance.
- Never download from the internet during CI; vendor tiny sources into the fixture folder.
## 5. Test runner expectations
- Runs Scanner binary analyzers on both binaries; emits `richgraph-v1` CAS entries.
- Compares graphs against `oracle.yml` expectations (functions/edges added/removed, tolerances).
- Fails when deltas are missing; succeeds when expected guards/edges are present.
## 6. Integration points
- **Scanner**: add fixture runner under `tests/reachability/StellaOps.Scanner.Binary.PatchOracleTests`.
- **CI**: wire into reachbench/patch-oracles job; ensure artifacts are small and deterministic.
- **Docs**: link this file from reachability delivery guide once tests are live.
## 7. Acceptance criteria
- At least three seed oracles (e.g., zlib overflow, OpenSSL length guard, BusyBox ash fix) committed with passing expectations.
- CI job proves deterministic hashes across reruns.
- Failures emit clear diffs (`expected edge foo->validate_len missing`).