save changes
This commit is contained in:
@@ -119,6 +119,7 @@ This documentation set is intentionally consolidated and does not maintain compa
|
||||
- Security deep dives: `docs/security/`
|
||||
- Benchmarks and fixtures: `docs/benchmarks/`, `docs/assets/`
|
||||
- Product advisories: `docs/product/advisories/`
|
||||
- Hybrid diff patching blueprint: `docs/hybrid-diff-patching.md`
|
||||
|
||||
---
|
||||
|
||||
@@ -140,3 +141,4 @@ This documentation set is intentionally consolidated and does not maintain compa
|
||||
- **Evidence-linked decisions:** every decision links to concrete evidence artifacts.
|
||||
- **Digest-first identity:** releases are immutable OCI digests, not mutable tags.
|
||||
- **Pluggable integrations:** connectors and steps are extensible; the core evidence chain stays stable.
|
||||
|
||||
|
||||
45
docs/hybrid-diff-patching.md
Normal file
45
docs/hybrid-diff-patching.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Hybrid Diff Patching (Source + Symbols + Binary)
|
||||
|
||||
## Purpose
|
||||
|
||||
This document captures the product-level blueprint for hybrid diff patching:
|
||||
|
||||
- Source semantic edits (AST-level intent).
|
||||
- Build-time symbol mapping (source ranges to binary symbols and addresses).
|
||||
- Normalized binary deltas (stable and compact byte patches).
|
||||
- Signed evidence bundle for policy gating and replay.
|
||||
|
||||
The goal is to make release decisions auditable at function granularity while
|
||||
remaining deterministic and offline-capable.
|
||||
|
||||
## Review outcome (2026-02-16)
|
||||
|
||||
The advisory is directionally aligned with existing Stella Ops work but not
|
||||
fully implemented end-to-end.
|
||||
|
||||
Already present:
|
||||
|
||||
- ELF normalization and delta hashing pipeline in BinaryIndex.
|
||||
- DeltaSig attestation models and CLI flows for extract/author/sign/verify.
|
||||
- Symbol manifest model with debug/code identifiers and source path metadata.
|
||||
|
||||
Missing or incomplete for the full hybrid stack:
|
||||
|
||||
- AST semantic edit-script generation and stable source anchors.
|
||||
- Build artifact contract that emits canonical `symbol_map.json` from DWARF/PDB
|
||||
during build.
|
||||
- Deterministic source-edit -> symbol patch plan artifact.
|
||||
- Verifier workflow that reconciles AST anchors with symbol boundaries and
|
||||
normalized per-symbol deltas in one attested contract.
|
||||
|
||||
## Canonical module dossier
|
||||
|
||||
Detailed contracts, phased implementation, and policy hooks are defined in:
|
||||
|
||||
- `docs/modules/binary-index/hybrid-diff-stack.md`
|
||||
|
||||
## Execution sprint
|
||||
|
||||
Implementation planning for this advisory is tracked in:
|
||||
|
||||
- `docs/implplan/SPRINT_20260216_001_BinaryIndex_hybrid_diff_patch_pipeline.md`
|
||||
@@ -0,0 +1,126 @@
|
||||
# Sprint 20260216-001 - Hybrid Diff Patch Pipeline
|
||||
|
||||
## Topic & Scope
|
||||
- Translate advisory guidance into an executable cross-module delivery plan for source-to-binary patch evidence.
|
||||
- Define deterministic contracts for semantic edit scripts, symbol maps, symbol patch plans, and normalized per-symbol deltas.
|
||||
- Wire policy and verification expectations so Release Orchestrator can gate on function-level change intent and byte-level proof.
|
||||
- Working directory: `src/BinaryIndex/`.
|
||||
- Expected evidence: targeted unit/integration tests, deterministic fixture artifacts, DSSE predicate samples, updated module docs.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- Depends on existing DeltaSig v2 predicate baseline in `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.DeltaSig/`.
|
||||
- Safe parallel workstreams:
|
||||
- source semantic edit artifact generation (`src/Tools/` or `src/ReleaseOrchestrator/` integration)
|
||||
- symbol map extraction contracts (`src/Symbols/`)
|
||||
- normalized delta and verifier integration (`src/BinaryIndex/`, `src/Attestor/`, `src/Doctor/`)
|
||||
- Cross-module edits are explicitly allowed for this sprint in:
|
||||
- `src/Symbols/`
|
||||
- `src/EvidenceLocker/`
|
||||
- `src/Policy/`
|
||||
- `src/ReleaseOrchestrator/`
|
||||
- `src/Attestor/`
|
||||
- `src/Doctor/`
|
||||
- `src/Web/`
|
||||
- `docs/modules/**`
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/hybrid-diff-patching.md`
|
||||
- `docs/modules/binary-index/hybrid-diff-stack.md`
|
||||
- `docs/modules/binary-index/semantic-diffing.md`
|
||||
- `docs/modules/binary-index/deltasig-v2-schema.md`
|
||||
- `docs/modules/evidence-locker/guides/evidence-pack-schema.md`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
### BHP-01 - Source semantic edit script artifact
|
||||
Status: TODO
|
||||
Dependency: none
|
||||
Owners: Developer, Documentation author
|
||||
Task description:
|
||||
- Add deterministic source semantic edit artifact generation that emits stable
|
||||
node identifiers and symbol anchors for changed code elements.
|
||||
- Integrate artifact emission into release comparison flow and persist into
|
||||
evidence pipelines.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] A `semantic_edit_script.json` contract is implemented and validated with tests.
|
||||
- [ ] Artifact generation is deterministic across repeated runs with identical inputs.
|
||||
- [ ] Documentation for schema and limits is added to module dossier docs.
|
||||
|
||||
### BHP-02 - Build symbol map contract and build-id binding
|
||||
Status: TODO
|
||||
Dependency: BHP-01
|
||||
Owners: Developer
|
||||
Task description:
|
||||
- Emit canonical `symbol_map.json` with source ranges, symbol boundaries, and
|
||||
build-id metadata from DWARF/PDB capable pipelines.
|
||||
- Ensure map digests and build-id values are linked into DeltaSig/attestation
|
||||
subjects for replay validation.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Symbol map generation is implemented for supported binary formats in scope.
|
||||
- [ ] Build-id and map digest are bound in emitted attestation payloads.
|
||||
- [ ] Tests cover mapping correctness and deterministic ordering.
|
||||
|
||||
### BHP-03 - Symbol patch plan and normalized per-symbol delta manifests
|
||||
Status: TODO
|
||||
Dependency: BHP-02
|
||||
Owners: Developer
|
||||
Task description:
|
||||
- Join semantic edits and symbol maps into `symbol_patch_plan.json` and
|
||||
generate normalized per-symbol deltas and `patch_manifest.json` outputs.
|
||||
- Remove placeholder function address/size derivation in DeltaSig generation
|
||||
where exact boundaries are required for audit claims.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Symbol patch plan artifact exists and links to AST anchors and symbol ids.
|
||||
- [ ] Patch manifest includes pre/post hashes, address ranges, and delta digests.
|
||||
- [ ] DeltaSig function-level outputs use real boundaries and sizes in covered paths.
|
||||
|
||||
### BHP-04 - Verifier and attestation enforcement
|
||||
Status: TODO
|
||||
Dependency: BHP-03
|
||||
Owners: Developer, Test Automation
|
||||
Task description:
|
||||
- Add verifier flow for build-id matching, re-normalization checks, dry-run delta
|
||||
application, and boundary/hash reconciliation.
|
||||
- Extend attestation validation logic in Attestor/Doctor and produce actionable
|
||||
verification evidence for release decisions.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Verifier checks fail closed on build-id mismatch, boundary mismatch, or hash mismatch.
|
||||
- [ ] DSSE validation and replay checks are captured in test evidence.
|
||||
- [ ] CLI/API surfaces expose verification outcome details for operators.
|
||||
|
||||
### BHP-05 - Policy and Evidence Locker integration
|
||||
Status: TODO
|
||||
Dependency: BHP-04
|
||||
Owners: Developer, Product Manager
|
||||
Task description:
|
||||
- Add policy gate inputs for symbol-count change budgets, namespace restrictions,
|
||||
API-surface invariants, and byte budget thresholds.
|
||||
- Store hybrid diff artifacts in Evidence Locker and expose summary/read paths in
|
||||
UI and release records.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Policy rules can gate promotions using hybrid diff metrics.
|
||||
- [ ] Evidence Locker stores and retrieves the full hybrid artifact chain.
|
||||
- [ ] UI/CLI render concise "what changed" summaries with links to signed evidence.
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2026-02-16 | Sprint created from product advisory review for hybrid source-symbol-binary diff pipeline. | Product Manager |
|
||||
|
||||
## Decisions & Risks
|
||||
- Advisory overlap confirmed with archived advisories:
|
||||
- `docs-archived/product/advisories/30-Dec-2025 - Binary Diff Signatures for Patch Detection.md`
|
||||
- `docs-archived/product/advisories/18-Dec-2025 - Building Better Binary Mapping and Call-Stack Reachability.md`
|
||||
- Decision: treat this advisory as an extension that unifies source intent and binary proof in one contract chain, not as a duplicate effort.
|
||||
- Risk: multi-module coordination can drift schemas; mitigation is to keep canonical contracts in BinaryIndex dossier and require digest-linked schema versions in attestations.
|
||||
- Risk: AST differencing backend choice may vary by language; mitigation is a language-agnostic output schema with adapter-specific provenance fields.
|
||||
|
||||
## Next Checkpoints
|
||||
- 2026-02-18: Contract freeze review for artifact schemas (`semantic_edit_script`, `symbol_map`, `symbol_patch_plan`, `patch_manifest`).
|
||||
- 2026-02-22: First end-to-end dry run in CI with signed evidence and verifier replay.
|
||||
- 2026-02-26: Policy gate integration demo with allow/deny examples on symbol namespaces.
|
||||
@@ -0,0 +1,102 @@
|
||||
# Sprint SPRINT_20260216_001_Signals_ebpf_micro_witness_determinism_profile - eBPF Micro-Witness Determinism
|
||||
|
||||
## Topic & Scope
|
||||
- Translate the eBPF micro-witness advisory into implementation-ready contracts and sprint tasks.
|
||||
- Close determinism gaps for runtime witness replay across kernel/distro/toolchain variance.
|
||||
- Define one portable evidence profile for DSSE + Sigstore bundle based offline replay.
|
||||
- Working directory: `docs/`.
|
||||
- Cross-module edits explicitly allowed for implementation tasks: `src/Signals/`, `src/Scanner/`, `src/Attestor/`, `src/EvidenceLocker/`.
|
||||
- Expected evidence: contract docs, schema/API updates, targeted module tests, offline verification artifacts.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- Upstream contracts: `docs/contracts/witness-v1.md`, `docs/modules/attestor/repro-bundle-profile.md`, `docs/modules/evidence/unified-model.md`.
|
||||
- Safe parallelism:
|
||||
- Signals loader/BTF work can run in parallel with Attestor/Evidence Locker bundle contract work.
|
||||
- Scanner witness model updates should run after profile fields are frozen.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/product/ebpf-micro-witness-determinism.md`
|
||||
- `docs/modules/signals/contracts/ebpf-micro-witness-determinism-profile.md`
|
||||
- `docs/reachability/deployment-guide.md`
|
||||
- `docs/contracts/witness-v1.md`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
### MWD-001 - Signals BTF fallback contract and metadata emission
|
||||
Status: TODO
|
||||
Dependency: none
|
||||
Owners: Product Manager, Developer
|
||||
Task description:
|
||||
- Implement deterministic BTF selection order in the runtime collector and emit selected source metadata (`source_kind`, `source_path`, `source_digest`, `selection_reason`) into runtime evidence/witness context.
|
||||
- Ensure behavior is explicit for kernel BTF, external vmlinux BTF, and split-BTF fallback.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Collector no longer fails solely on missing `/sys/kernel/btf/vmlinux` when configured fallback BTF exists.
|
||||
- [ ] Runtime evidence includes immutable BTF selection metadata required for replay.
|
||||
|
||||
### MWD-002 - Runtime witness schema extensions for deterministic symbolization
|
||||
Status: TODO
|
||||
Dependency: MWD-001
|
||||
Owners: Developer, Documentation author
|
||||
Task description:
|
||||
- Extend runtime witness payload schema to include deterministic symbolization tuple: `build_id`, debug/symbol pointer(s), symbolizer identity/version/digest, libc variant, and sysroot digest.
|
||||
- Update witness contracts and validation rules in docs and implementation.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Witness schema and code models carry required symbolization fields.
|
||||
- [ ] Validation rejects witnesses missing required deterministic symbolization inputs.
|
||||
|
||||
### MWD-003 - Implement Scanner runtime witness generation pipeline
|
||||
Status: TODO
|
||||
Dependency: MWD-002
|
||||
Owners: Developer, Test Automation
|
||||
Task description:
|
||||
- Deliver concrete `IRuntimeWitnessGenerator` implementation, integrating runtime observations, witness building, DSSE signing, and storage.
|
||||
- Ensure deterministic ordering/canonicalization for runtime observation payloads.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Runtime witness generation is implemented (not interface-only) and wired into runtime instrumentation flow.
|
||||
- [ ] Determinism tests show stable witness bytes for fixed inputs.
|
||||
|
||||
### MWD-004 - DSSE plus Sigstore bundle witness packaging
|
||||
Status: TODO
|
||||
Dependency: MWD-003
|
||||
Owners: Developer, Documentation author
|
||||
Task description:
|
||||
- Standardize and implement per-witness artifact triplet: `trace.json`, `trace.dsse.json`, `trace.sigstore.json`.
|
||||
- Store and export this profile through Evidence Locker with offline verification compatibility.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Evidence Locker manifest/index model supports the Sigstore bundle artifact and links it to witness identity.
|
||||
- [ ] Offline verify workflow succeeds using bundle-contained material only.
|
||||
|
||||
### MWD-005 - Cross-distro deterministic replay test matrix
|
||||
Status: TODO
|
||||
Dependency: MWD-004
|
||||
Owners: Test Automation, QA
|
||||
Task description:
|
||||
- Add targeted replay verification across kernel/libc matrix (minimum 3 kernels, glibc + musl), asserting byte-identical replay frames for fixed witness artifacts.
|
||||
- Capture command output and evidence artifacts for deterministic QA sign-off.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Matrix tests run against targeted projects (not solution filters) and show deterministic replay output.
|
||||
- [ ] Execution evidence is recorded with artifact hashes and replay verification logs.
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2026-02-16 | Sprint created from eBPF micro-witness advisory review; gaps confirmed and translated to implementation tasks. | Project Manager |
|
||||
|
||||
## Decisions & Risks
|
||||
- Decision: Adopt a single micro-witness determinism profile defined in `docs/modules/signals/contracts/ebpf-micro-witness-determinism-profile.md`.
|
||||
- Decision: Product-level promise and current baseline are captured in `docs/product/ebpf-micro-witness-determinism.md`.
|
||||
- Decision: Advisory translation record archived at `docs-archived/product/advisories/16-Feb-2026 - eBPF micro-witness deterministic replay across distros.md`.
|
||||
- Risk: Existing runtime collector hard dependency on kernel BTF may block non-BTF kernels until fallback path is implemented.
|
||||
- Risk: Runtime witness generation remains incomplete without a concrete generator implementation; downstream attestation/export is blocked.
|
||||
- Risk: Absence of standardized Sigstore witness bundle may produce non-portable replay evidence across environments.
|
||||
- External web fetches: none.
|
||||
|
||||
## Next Checkpoints
|
||||
- 2026-02-18: Contract review sign-off (Signals/Scanner/Attestor/Evidence Locker owners).
|
||||
- 2026-02-21: MWD-001 and MWD-002 implementation readiness checkpoint.
|
||||
- 2026-02-25: First end-to-end deterministic replay demo with DSSE + Sigstore witness bundle.
|
||||
@@ -37,6 +37,7 @@ Key features:
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Hybrid Diff Stack: `./hybrid-diff-stack.md`
|
||||
- High-Level Architecture: `../../ARCHITECTURE_OVERVIEW.md`
|
||||
- Scanner Architecture: `../scanner/architecture.md`
|
||||
- Concelier Architecture: `../concelier/architecture.md`
|
||||
@@ -63,7 +64,7 @@ A major enhancement to BinaryIndex is planned to enable **semantic-level binary
|
||||
| **Phase 1** | IR-Level Semantic Analysis | +15% accuracy on optimized binaries | Planned |
|
||||
| **Phase 2** | Function Behavior Corpus | +10% coverage on stripped binaries | Planned |
|
||||
| **Phase 3** | Ghidra Integration | +5% edge case handling | Planned |
|
||||
| **Phase 4** | Decompiler & ML Similarity | +10% obfuscation resilience | Planned |
|
||||
| **Phase 4** | Decompiler and ML Similarity | +10% obfuscation resilience | Planned |
|
||||
|
||||
### New Libraries (Planned)
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
> **Ownership:** Scanner Guild + Concelier Guild
|
||||
> **Status:** DRAFT
|
||||
> **Version:** 1.0.0
|
||||
> **Related:** [High-Level Architecture](../../ARCHITECTURE_OVERVIEW.md), [Scanner Architecture](../scanner/architecture.md), [Concelier Architecture](../concelier/architecture.md)
|
||||
> **Related:** [High-Level Architecture](../../ARCHITECTURE_OVERVIEW.md), [Scanner Architecture](../scanner/architecture.md), [Concelier Architecture](../concelier/architecture.md), [Hybrid Diff Stack](./hybrid-diff-stack.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -1774,3 +1774,4 @@ inside `AddNormalizationPipelines()` in `ServiceCollectionExtensions.cs`.
|
||||
|
||||
*Document Version: 1.5.0*
|
||||
*Last Updated: 2026-02-12*
|
||||
|
||||
|
||||
163
docs/modules/binary-index/hybrid-diff-stack.md
Normal file
163
docs/modules/binary-index/hybrid-diff-stack.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# Hybrid Diff Stack Architecture (Source -> Symbols -> Normalized Bytes)
|
||||
|
||||
> Status: Planned (advisory translation, 2026-02-16)
|
||||
> Module: BinaryIndex with cross-module contracts (Symbols, EvidenceLocker, Policy, Attestor, ReleaseOrchestrator)
|
||||
|
||||
## 1. Objective
|
||||
|
||||
Produce compact, auditable patch artifacts that preserve developer intent and
|
||||
binary truth at the same time:
|
||||
|
||||
- Source-level intent: semantic edit scripts anchored to classes/functions.
|
||||
- Build-level mapping: symbol map linked to immutable build identity.
|
||||
- Binary-level patching: normalization-first per-symbol deltas.
|
||||
- Release evidence: DSSE-signed contract consumed by policy and replay.
|
||||
|
||||
## 2. Current implementation baseline
|
||||
|
||||
Implemented today:
|
||||
|
||||
- ELF normalization passes and deterministic delta hash generation.
|
||||
- DeltaSig predicate contracts (v1 and v2) with CLI author/sign/verify flows.
|
||||
- Symbol manifest model with debug id, code id, source paths, and line data.
|
||||
|
||||
Gaps for full advisory scope:
|
||||
|
||||
- No AST semantic edit script artifact pipeline in current release workflow.
|
||||
- No canonical builder output for source-range to symbol-address map as a
|
||||
first-class build artifact contract.
|
||||
- No end-to-end "source edits -> symbol patch plan -> normalized deltas"
|
||||
bundle schema consumed by release policy.
|
||||
- Existing function delta composition still contains placeholder address/size
|
||||
behavior in parts of DeltaSig generation.
|
||||
|
||||
## 3. Target contracts
|
||||
|
||||
### 3.1 Source semantic edit script (`semantic_edit_script.json`)
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `sourceTreeDigest`
|
||||
- `edits[]` where each edit includes:
|
||||
- `editType`: `add|remove|move|update|rename`
|
||||
- `nodeKind`: `class|method|field|import|statement`
|
||||
- `nodePath`: stable language-specific path
|
||||
- `anchor`: symbol-like identifier (for example `Namespace.Type.Method`)
|
||||
- `pre` and `post` source spans and digests
|
||||
|
||||
Determinism rules:
|
||||
|
||||
- Stable sort by file path, then node path.
|
||||
- Stable source digests and normalized paths.
|
||||
|
||||
### 3.2 Symbol map (`symbol_map.json`)
|
||||
|
||||
Produced during build from DWARF/PDB + build metadata.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `buildId`
|
||||
- `binaryDigest`
|
||||
- `symbols[]`:
|
||||
- `name`
|
||||
- `kind` (`function|object|section`)
|
||||
- `addressStart` and `addressEnd`
|
||||
- `section`
|
||||
- `sourceRanges[]` (`file`, `lineStart`, `lineEnd`)
|
||||
|
||||
Determinism rules:
|
||||
|
||||
- Symbol ordering by address then name.
|
||||
- Build id must match attestation subject.
|
||||
|
||||
### 3.3 Symbol patch plan (`symbol_patch_plan.json`)
|
||||
|
||||
Joins source edits with concrete symbols.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `buildIdBefore` and `buildIdAfter`
|
||||
- `editsDigest`
|
||||
- `symbolMapDigestBefore` and `symbolMapDigestAfter`
|
||||
- `changes[]`:
|
||||
- `symbol`
|
||||
- `changeType` (`added|removed|modified|moved`)
|
||||
- `astAnchors[]`
|
||||
- `preHash` and `postHash`
|
||||
- `deltaRef`
|
||||
|
||||
### 3.4 Patch manifest (`patch_manifest.json`)
|
||||
|
||||
Binds per-symbol normalized deltas to evidence and policy.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `buildId`
|
||||
- `normalizationRecipeId`
|
||||
- `patches[]`:
|
||||
- `symbol`
|
||||
- `addressRange`
|
||||
- `deltaDigest`
|
||||
- `pre` (`size`, `hash`)
|
||||
- `post` (`size`, `hash`)
|
||||
- `attestation` (`predicateType`, `dsseDigest`)
|
||||
|
||||
## 4. Evidence and policy integration
|
||||
|
||||
EvidenceLocker stores four linked artifacts per release comparison:
|
||||
|
||||
1. semantic edit script
|
||||
2. symbol maps (before/after)
|
||||
3. symbol patch plan
|
||||
4. normalized patch manifest + delta blobs
|
||||
|
||||
Policy hooks:
|
||||
|
||||
- Allowlist/denylist by namespace or symbol path.
|
||||
- Max function-count and max byte budget controls.
|
||||
- API surface change checks.
|
||||
- Hot-path and cryptography namespace protection rules.
|
||||
|
||||
## 5. Verifier contract (Attestor/Doctor)
|
||||
|
||||
Verifier must prove all of the following before promotion:
|
||||
|
||||
- Build-id and subject digest alignment.
|
||||
- Re-normalization of target binary with matching recipe id.
|
||||
- Dry-run delta application succeeds within declared symbol boundaries.
|
||||
- Resulting hashes equal manifest `post` values.
|
||||
- AST anchors reconcile to changed symbols in symbol patch plan.
|
||||
- DSSE signatures and transparency references validate per policy.
|
||||
|
||||
## 6. Integration boundaries
|
||||
|
||||
Builder step (CI): emit symbol map and normalized segments.
|
||||
|
||||
ReleaseOrchestrator step: combine source edits, symbol maps, and normalized
|
||||
bytes into patch plan and manifest.
|
||||
|
||||
BinaryIndex/DeltaSig: own normalization and per-symbol diff generation.
|
||||
|
||||
Attestor/Doctor: own verification and attestation checks.
|
||||
|
||||
EvidenceLocker: own storage schema and query surfaces.
|
||||
|
||||
Policy: consume summarized patch-plan metrics and rule evaluations.
|
||||
|
||||
## 7. Implementation tracker
|
||||
|
||||
Execution is tracked in:
|
||||
|
||||
- `docs/implplan/SPRINT_20260216_001_BinaryIndex_hybrid_diff_patch_pipeline.md`
|
||||
|
||||
## 8. Related documents
|
||||
|
||||
- `docs/hybrid-diff-patching.md`
|
||||
- `docs/modules/binary-index/semantic-diffing.md`
|
||||
- `docs/modules/binary-index/deltasig-v2-schema.md`
|
||||
- `docs/modules/scanner/binary-diff-attestation.md`
|
||||
- `docs/modules/evidence-locker/guides/evidence-pack-schema.md`
|
||||
@@ -49,6 +49,7 @@ Key settings:
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Contract: `./contracts/ebpf-micro-witness-determinism-profile.md`
|
||||
- Policy Engine: `../policy/`
|
||||
- VexLens: `../vex-lens/`
|
||||
- High-Level Architecture: `../../ARCHITECTURE_OVERVIEW.md`
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
# eBPF Micro-Witness Determinism Profile v1.0.0
|
||||
|
||||
**Status:** PLANNED
|
||||
**Version:** 1.0.0
|
||||
**Effective:** 2026-02-16
|
||||
**Owner:** Signals Guild + Scanner Guild + Attestor Guild + Evidence Locker Guild
|
||||
**Sprint:** `docs/implplan/SPRINT_20260216_001_Signals_ebpf_micro_witness_determinism_profile.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This profile defines the minimum deterministic contract for runtime eBPF "micro-witnesses" so replay yields the same symbolized result across distros/toolchains and in offline environments.
|
||||
|
||||
---
|
||||
|
||||
## 2. Contract Scope
|
||||
|
||||
- Runtime collection and BTF selection (`Signals`).
|
||||
- Runtime witness payload schema and signing (`Scanner`).
|
||||
- DSSE and transparency evidence shape (`Attestor`).
|
||||
- Portable storage/export/indexing (`Evidence Locker`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Runtime Loader Contract (BTF Selection)
|
||||
|
||||
### 3.1 Selection order (mandatory)
|
||||
1. `/sys/kernel/btf/vmlinux`
|
||||
2. configured full-kernel BTF path (for example distro debug package path)
|
||||
3. split-BTF selected by `{kernel_release, arch}`
|
||||
|
||||
### 3.2 Required emitted metadata
|
||||
|
||||
```json
|
||||
{
|
||||
"kernel_release": "6.8.0-45-generic",
|
||||
"kernel_arch": "x86_64",
|
||||
"btf": {
|
||||
"source_kind": "kernel|external-vmlinux|split-btf",
|
||||
"source_path": "/sys/kernel/btf/vmlinux",
|
||||
"source_digest": "sha256:...",
|
||||
"selection_reason": "kernel_btf_present"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`source_path` and `source_digest` are mandatory for deterministic replay.
|
||||
|
||||
---
|
||||
|
||||
## 4. Deterministic Symbolization Contract
|
||||
|
||||
Each runtime witness must carry deterministic symbolization inputs:
|
||||
|
||||
```json
|
||||
{
|
||||
"symbolization": {
|
||||
"build_id": "gnu-build-id:...",
|
||||
"debug_artifact_uri": "cas://symbols/by-build-id/gnu-build-id:.../artifact.debug",
|
||||
"symbol_table_uri": "cas://symbols/by-build-id/gnu-build-id:.../symtab.json",
|
||||
"symbolizer": {
|
||||
"name": "llvm-symbolizer",
|
||||
"version": "18.1.7",
|
||||
"digest": "sha256:..."
|
||||
},
|
||||
"libc_variant": "glibc|musl",
|
||||
"sysroot_digest": "sha256:..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
At least one of `debug_artifact_uri` or `symbol_table_uri` must be present.
|
||||
|
||||
---
|
||||
|
||||
## 5. Witness Packaging Contract
|
||||
|
||||
Each micro-witness must be exportable as:
|
||||
|
||||
1. `trace.json` (canonical payload)
|
||||
2. `trace.dsse.json` (DSSE envelope)
|
||||
3. `trace.sigstore.json` (Sigstore bundle with signature/cert/transparency proof)
|
||||
|
||||
Offline verification must use only bundle-contained material (no network dependency).
|
||||
|
||||
---
|
||||
|
||||
## 6. Evidence Locker Index Contract
|
||||
|
||||
Evidence Locker must index runtime witness artifacts by:
|
||||
|
||||
- `build_id`
|
||||
- `kernel_release`
|
||||
- `probe_id`
|
||||
- `policy_run_id`
|
||||
|
||||
These keys are required for deterministic replay lookup and audit search.
|
||||
|
||||
---
|
||||
|
||||
## 7. Validation Matrix (minimum)
|
||||
|
||||
- Kernel matrix: at least 3 supported kernel lines.
|
||||
- libc matrix: glibc + musl.
|
||||
- Verification modes: online + offline.
|
||||
- Determinism check: byte-identical replayed frame output for fixed input evidence.
|
||||
|
||||
---
|
||||
|
||||
## 8. Confirmed Gaps (2026-02-16 Baseline)
|
||||
|
||||
- Hard BTF dependency with no split-BTF fallback metadata contract in collector:
|
||||
- `src/Signals/__Libraries/StellaOps.Signals.Ebpf/Services/RuntimeSignalCollector.cs`
|
||||
- Probe load path is simulated and does not record selected BTF source:
|
||||
- `src/Signals/__Libraries/StellaOps.Signals.Ebpf/Probes/CoreProbeLoader.cs`
|
||||
- Runtime witness payload lacks required symbolization tuple fields:
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/PathWitness.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/RuntimeObservation.cs`
|
||||
- Runtime witness generator implementation is missing:
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/IRuntimeWitnessGenerator.cs`
|
||||
- Sigstore bundle (`trace.sigstore.json`) is not yet standardized in witness storage/export:
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/013_witness_storage.sql`
|
||||
- `src/EvidenceLocker/__Libraries/StellaOps.EvidenceLocker.Export/Models/BundleManifest.cs`
|
||||
@@ -12,6 +12,7 @@ Product strategy, competitive analysis, and marketing bridge documents.
|
||||
| [decision-capsules.md](decision-capsules.md) | Decision Capsules concept (audit-grade evidence bundles) |
|
||||
| [evidence-linked-vex.md](evidence-linked-vex.md) | Evidence-linked VEX technical bridge |
|
||||
| [hybrid-reachability.md](hybrid-reachability.md) | Hybrid reachability feature positioning |
|
||||
| [ebpf-micro-witness-determinism.md](ebpf-micro-witness-determinism.md) | eBPF micro-witness deterministic replay profile and current implementation gaps |
|
||||
| [portable-audit-pack-plan.md](portable-audit-pack-plan.md) | Portable supply-chain audit pack rollout plan |
|
||||
| [reachability-benchmark-launch.md](reachability-benchmark-launch.md) | Reachability benchmark launch materials |
|
||||
|
||||
|
||||
36
docs/product/ebpf-micro-witness-determinism.md
Normal file
36
docs/product/ebpf-micro-witness-determinism.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# eBPF Micro-Witness Determinism Profile
|
||||
|
||||
## Status
|
||||
- Advisory translated: 2026-02-16 (UTC)
|
||||
- Current implementation status: gaps confirmed
|
||||
- Implementation sprint: `docs/implplan/SPRINT_20260216_001_Signals_ebpf_micro_witness_determinism_profile.md`
|
||||
|
||||
## Purpose
|
||||
- Define what "replayable and deterministic micro-witnesses" means for Stella Ops runtime evidence.
|
||||
- Align Signals, Scanner, Attestor, and Evidence Locker on one verifiable output profile.
|
||||
- Ensure the same incident replay result across distros/toolchains and in offline mode.
|
||||
|
||||
## Required product behavior
|
||||
1. One CO-RE probe object must run unchanged across supported kernels when BTF is available.
|
||||
2. If kernel BTF is missing, the loader must use deterministic fallback selection and record exactly what BTF source was used.
|
||||
3. Runtime witnesses must include deterministic symbolization inputs (build identity + symbol/debug material + toolchain tuple).
|
||||
4. Witness evidence must be portable as DSSE plus a Sigstore bundle that can be verified offline.
|
||||
|
||||
## Verified current state (2026-02-16)
|
||||
- eBPF support check currently hard-requires `/sys/kernel/btf/vmlinux` with no split-BTF fallback path selection metadata in collector output.
|
||||
- `src/Signals/__Libraries/StellaOps.Signals.Ebpf/Services/RuntimeSignalCollector.cs`
|
||||
- Probe loader path is simulated for runtime attachment lifecycle and does not implement deterministic BTF source recording.
|
||||
- `src/Signals/__Libraries/StellaOps.Signals.Ebpf/Probes/CoreProbeLoader.cs`
|
||||
- Runtime witness model includes `build_id` but does not include symbol bundle pointers or symbolizer/libc/sysroot tuple required for cross-distro deterministic symbolization.
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/PathWitness.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/RuntimeObservation.cs`
|
||||
- Runtime witness generator is interface-defined but has no production implementation in Scanner.
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/IRuntimeWitnessGenerator.cs`
|
||||
- DSSE envelope support exists; end-to-end per-witness Sigstore bundle contract (`trace.sigstore.json`) is not standardized in witness storage/indexing.
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/WitnessDsseSigner.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/013_witness_storage.sql`
|
||||
- `src/EvidenceLocker/__Libraries/StellaOps.EvidenceLocker.Export/Models/BundleManifest.cs`
|
||||
|
||||
## Decision
|
||||
- Advisory is accepted as implementation-required.
|
||||
- Contract and sprint tasks are created to close deterministic replay gaps.
|
||||
Reference in New Issue
Block a user