save changes
This commit is contained in:
master
@@ -37,6 +37,7 @@ Key features:
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Hybrid Diff Stack: `./hybrid-diff-stack.md`
|
||||
- High-Level Architecture: `../../ARCHITECTURE_OVERVIEW.md`
|
||||
- Scanner Architecture: `../scanner/architecture.md`
|
||||
- Concelier Architecture: `../concelier/architecture.md`
|
||||
@@ -63,7 +64,7 @@ A major enhancement to BinaryIndex is planned to enable **semantic-level binary
|
||||
| **Phase 1** | IR-Level Semantic Analysis | +15% accuracy on optimized binaries | Planned |
|
||||
| **Phase 2** | Function Behavior Corpus | +10% coverage on stripped binaries | Planned |
|
||||
| **Phase 3** | Ghidra Integration | +5% edge case handling | Planned |
|
||||
| **Phase 4** | Decompiler & ML Similarity | +10% obfuscation resilience | Planned |
|
||||
| **Phase 4** | Decompiler and ML Similarity | +10% obfuscation resilience | Planned |
|
||||
|
||||
### New Libraries (Planned)
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
> **Ownership:** Scanner Guild + Concelier Guild
|
||||
> **Status:** DRAFT
|
||||
> **Version:** 1.0.0
|
||||
> **Related:** [High-Level Architecture](../../ARCHITECTURE_OVERVIEW.md), [Scanner Architecture](../scanner/architecture.md), [Concelier Architecture](../concelier/architecture.md)
|
||||
> **Related:** [High-Level Architecture](../../ARCHITECTURE_OVERVIEW.md), [Scanner Architecture](../scanner/architecture.md), [Concelier Architecture](../concelier/architecture.md), [Hybrid Diff Stack](./hybrid-diff-stack.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -1774,3 +1774,4 @@ inside `AddNormalizationPipelines()` in `ServiceCollectionExtensions.cs`.
|
||||
|
||||
*Document Version: 1.5.0*
|
||||
*Last Updated: 2026-02-12*
|
||||
|
||||
|
||||
163
docs/modules/binary-index/hybrid-diff-stack.md
Normal file
163
docs/modules/binary-index/hybrid-diff-stack.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# Hybrid Diff Stack Architecture (Source -> Symbols -> Normalized Bytes)
|
||||
|
||||
> Status: Planned (advisory translation, 2026-02-16)
|
||||
> Module: BinaryIndex with cross-module contracts (Symbols, EvidenceLocker, Policy, Attestor, ReleaseOrchestrator)
|
||||
|
||||
## 1. Objective
|
||||
|
||||
Produce compact, auditable patch artifacts that preserve developer intent and
|
||||
binary truth at the same time:
|
||||
|
||||
- Source-level intent: semantic edit scripts anchored to classes/functions.
|
||||
- Build-level mapping: symbol map linked to immutable build identity.
|
||||
- Binary-level patching: normalization-first per-symbol deltas.
|
||||
- Release evidence: DSSE-signed contract consumed by policy and replay.
|
||||
|
||||
## 2. Current implementation baseline
|
||||
|
||||
Implemented today:
|
||||
|
||||
- ELF normalization passes and deterministic delta hash generation.
|
||||
- DeltaSig predicate contracts (v1 and v2) with CLI author/sign/verify flows.
|
||||
- Symbol manifest model with debug id, code id, source paths, and line data.
|
||||
|
||||
Gaps for full advisory scope:
|
||||
|
||||
- No AST semantic edit script artifact pipeline in current release workflow.
|
||||
- No canonical builder output for source-range to symbol-address map as a
|
||||
first-class build artifact contract.
|
||||
- No end-to-end "source edits -> symbol patch plan -> normalized deltas"
|
||||
bundle schema consumed by release policy.
|
||||
- Existing function delta composition still contains placeholder address/size
|
||||
behavior in parts of DeltaSig generation.
|
||||
|
||||
## 3. Target contracts
|
||||
|
||||
### 3.1 Source semantic edit script (`semantic_edit_script.json`)
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `sourceTreeDigest`
|
||||
- `edits[]` where each edit includes:
|
||||
- `editType`: `add|remove|move|update|rename`
|
||||
- `nodeKind`: `class|method|field|import|statement`
|
||||
- `nodePath`: stable language-specific path
|
||||
- `anchor`: symbol-like identifier (for example `Namespace.Type.Method`)
|
||||
- `pre` and `post` source spans and digests
|
||||
|
||||
Determinism rules:
|
||||
|
||||
- Stable sort by file path, then node path.
|
||||
- Stable source digests and normalized paths.
|
||||
|
||||
### 3.2 Symbol map (`symbol_map.json`)
|
||||
|
||||
Produced during build from DWARF/PDB + build metadata.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `buildId`
|
||||
- `binaryDigest`
|
||||
- `symbols[]`:
|
||||
- `name`
|
||||
- `kind` (`function|object|section`)
|
||||
- `addressStart` and `addressEnd`
|
||||
- `section`
|
||||
- `sourceRanges[]` (`file`, `lineStart`, `lineEnd`)
|
||||
|
||||
Determinism rules:
|
||||
|
||||
- Symbol ordering by address then name.
|
||||
- Build id must match attestation subject.
|
||||
|
||||
### 3.3 Symbol patch plan (`symbol_patch_plan.json`)
|
||||
|
||||
Joins source edits with concrete symbols.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `buildIdBefore` and `buildIdAfter`
|
||||
- `editsDigest`
|
||||
- `symbolMapDigestBefore` and `symbolMapDigestAfter`
|
||||
- `changes[]`:
|
||||
- `symbol`
|
||||
- `changeType` (`added|removed|modified|moved`)
|
||||
- `astAnchors[]`
|
||||
- `preHash` and `postHash`
|
||||
- `deltaRef`
|
||||
|
||||
### 3.4 Patch manifest (`patch_manifest.json`)
|
||||
|
||||
Binds per-symbol normalized deltas to evidence and policy.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schemaVersion`
|
||||
- `buildId`
|
||||
- `normalizationRecipeId`
|
||||
- `patches[]`:
|
||||
- `symbol`
|
||||
- `addressRange`
|
||||
- `deltaDigest`
|
||||
- `pre` (`size`, `hash`)
|
||||
- `post` (`size`, `hash`)
|
||||
- `attestation` (`predicateType`, `dsseDigest`)
|
||||
|
||||
## 4. Evidence and policy integration
|
||||
|
||||
EvidenceLocker stores four linked artifacts per release comparison:
|
||||
|
||||
1. semantic edit script
|
||||
2. symbol maps (before/after)
|
||||
3. symbol patch plan
|
||||
4. normalized patch manifest + delta blobs
|
||||
|
||||
Policy hooks:
|
||||
|
||||
- Allowlist/denylist by namespace or symbol path.
|
||||
- Max function-count and max byte budget controls.
|
||||
- API surface change checks.
|
||||
- Hot-path and cryptography namespace protection rules.
|
||||
|
||||
## 5. Verifier contract (Attestor/Doctor)
|
||||
|
||||
Verifier must prove all of the following before promotion:
|
||||
|
||||
- Build-id and subject digest alignment.
|
||||
- Re-normalization of target binary with matching recipe id.
|
||||
- Dry-run delta application succeeds within declared symbol boundaries.
|
||||
- Resulting hashes equal manifest `post` values.
|
||||
- AST anchors reconcile to changed symbols in symbol patch plan.
|
||||
- DSSE signatures and transparency references validate per policy.
|
||||
|
||||
## 6. Integration boundaries
|
||||
|
||||
Builder step (CI): emit symbol map and normalized segments.
|
||||
|
||||
ReleaseOrchestrator step: combine source edits, symbol maps, and normalized
|
||||
bytes into patch plan and manifest.
|
||||
|
||||
BinaryIndex/DeltaSig: own normalization and per-symbol diff generation.
|
||||
|
||||
Attestor/Doctor: own verification and attestation checks.
|
||||
|
||||
EvidenceLocker: own storage schema and query surfaces.
|
||||
|
||||
Policy: consume summarized patch-plan metrics and rule evaluations.
|
||||
|
||||
## 7. Implementation tracker
|
||||
|
||||
Execution is tracked in:
|
||||
|
||||
- `docs/implplan/SPRINT_20260216_001_BinaryIndex_hybrid_diff_patch_pipeline.md`
|
||||
|
||||
## 8. Related documents
|
||||
|
||||
- `docs/hybrid-diff-patching.md`
|
||||
- `docs/modules/binary-index/semantic-diffing.md`
|
||||
- `docs/modules/binary-index/deltasig-v2-schema.md`
|
||||
- `docs/modules/scanner/binary-diff-attestation.md`
|
||||
- `docs/modules/evidence-locker/guides/evidence-pack-schema.md`
|
||||
Reference in New Issue
Block a user