c2c6b58b41
- Introduced a new document for promotion-time attestations, detailing the purpose, predicate schema, producer workflow, verification flow, APIs, and security considerations. - Implemented the `stella.ops/promotion@v1` predicate schema to capture promotion evidence including image digest, SBOM/VEX artifacts, and Rekor proof. - Defined producer responsibilities and workflows for CLI orchestration, signer responsibilities, and Export Center integration. - Added verification steps for auditors to validate promotion attestations offline. feat: Create Symbol Manifest v1 Specification - Developed a specification for Symbol Manifest v1 to provide a deterministic format for publishing debug symbols and source maps. - Defined the manifest structure, including schema, entries, source maps, toolchain, and provenance. - Outlined upload and verification processes, resolve APIs, runtime proxy, caching, and offline bundle generation. - Included security considerations and related tasks for implementation. chore: Add Ruby Analyzer with Git Sources - Created a Gemfile and Gemfile.lock for Ruby analyzer with dependencies on git-gem, httparty, and path-gem. - Implemented main application logic to utilize the defined gems and output their versions. - Added expected JSON output for the Ruby analyzer to validate the integration of the new gems and their functionalities. - Developed internal observation classes for Ruby packages, runtime edges, and capabilities, including serialization logic for observations. test: Add tests for Ruby Analyzer - Created test fixtures for Ruby analyzer, including Gemfile, Gemfile.lock, main application, and expected JSON output. - Ensured that the tests validate the correct integration and functionality of the Ruby analyzer with the specified gems.
127 lines
4.5 KiB
Markdown
127 lines
4.5 KiB
Markdown
# Entropy Analysis for Executable Layers
|
||
|
||
> **Status:** Draft – Sprint 186/209
|
||
> **Owners:** Scanner Guild · Policy Guild · UI Guild · Docs Guild
|
||
|
||
## 1. Overview
|
||
|
||
Entropy analysis highlights opaque regions inside container layers (packed binaries, stripped blobs, embedded firmware) so Stella Ops can prioritise artefacts that are hard to audit. The scanner computes per-file entropy metrics, reports opaque ratios per layer, and feeds penalties into the trust algebra.
|
||
|
||
## 2. Scanner pipeline (`SCAN-ENTROPY-186-011/012`)
|
||
|
||
* **Target files:** ELF, PE/COFF, Mach-O executables and large raw blobs (>16 KB). Archive formats (zip/tar) are unpacked by existing analyzers before entropy processing.
|
||
* **Section analysis:**
|
||
* ELF – `.text`, `.rodata`, `.data`, custom sections.
|
||
* PE – section table entries (`IMAGE_SECTION_HEADER`).
|
||
* Mach-O – LC_SEGMENT/LC_SEGMENT_64 sections.
|
||
* **Sliding window:** 4 KB window with 1 KB stride. Entropy calculated using Shannon entropy:
|
||
|
||
\[
|
||
H = -\sum_{i=0}^{255} p_i \log_2 p_i
|
||
\]
|
||
|
||
Windows with `H ≥ 7.2` bits/byte are marked “opaque”.
|
||
* **Heuristics & hints:**
|
||
* Flag entire files with no symbols or stripped debug info.
|
||
* Detect known packer section names (`.UPX*`, `.aspack`, etc.).
|
||
* Record offsets, window sizes, and entropy values to support explainability.
|
||
* **Outputs:**
|
||
* `entropy.report.json` (per-file details, windows, hints).
|
||
* `layer_summary.json` (opaque byte ratios per layer and overall image).
|
||
* Penalty score contributed to the trust algebra (`entropy_penalty`).
|
||
|
||
All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestations/replay bundles.
|
||
|
||
## 3. JSON Schemas
|
||
|
||
### 3.1 `entropy.report.json`
|
||
|
||
```jsonc
|
||
{
|
||
"schema": "stellaops.entropy/report@1",
|
||
"imageDigest": "sha256:…",
|
||
"layerDigest": "sha256:…",
|
||
"files": [
|
||
{
|
||
"path": "/opt/app/libblob.so",
|
||
"size": 5242880,
|
||
"opaqueBytes": 1342177,
|
||
"opaqueRatio": 0.25,
|
||
"flags": ["stripped", "section:.UPX0"],
|
||
"windows": [
|
||
{ "offset": 0, "length": 4096, "entropy": 7.45 },
|
||
{ "offset": 1024, "length": 4096, "entropy": 7.38 }
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
### 3.2 `layer_summary.json`
|
||
|
||
```jsonc
|
||
{
|
||
"schema": "stellaops.entropy/layer-summary@1",
|
||
"imageDigest": "sha256:…",
|
||
"layers": [
|
||
{
|
||
"digest": "sha256:layer4…",
|
||
"opaqueBytes": 2306867,
|
||
"totalBytes": 10485760,
|
||
"opaqueRatio": 0.22,
|
||
"indicators": ["packed", "no-symbols"]
|
||
}
|
||
],
|
||
"imageOpaqueRatio": 0.18,
|
||
"entropyPenalty": 0.12
|
||
}
|
||
```
|
||
|
||
## 4. Policy integration (`POLICY-RISK-90-001`)
|
||
|
||
* Policy Engine receives `entropy_penalty` and per-layer ratios via scan evidence.
|
||
* Default thresholds:
|
||
* Block when `imageOpaqueRatio > 0.15` and provenance unknown.
|
||
* Warn when any executable has `opaqueRatio > 0.30`.
|
||
* Penalty weights are configurable per tenant. Policy explanations include:
|
||
* Highest-entropy files and offsets.
|
||
* Reason code (packed, no symbols, runtime reachable).
|
||
|
||
## 5. UI experience (`UI-ENTROPY-40-001/002`)
|
||
|
||
* **Heatmaps:** render entropy along the file timeline (green → red).
|
||
* **Layer donut:** show opaque % per layer with tooltips linking to file list.
|
||
* **“Why risky?” chips:** highlight triggers such as *Packed-like*, *Stripped*, *No symbols*.
|
||
* Policy banners explain configured thresholds and mitigation (add provenance, unpack, or accept risk).
|
||
* Provide direct download links to `entropy.report.json` for audits.
|
||
|
||
## 6. CLI / API hooks
|
||
|
||
* CLI – `stella scan artifacts --entropy` option prints top opaque files and penalties.
|
||
* API – `GET /api/v1/scans/{id}/entropy` serves summary + evidence references.
|
||
* Notify templates can include entropy penalties to escalate opaque images.
|
||
|
||
## 7. Trust algebra
|
||
|
||
The penalty is computed as:
|
||
|
||
\[
|
||
\text{entropyPenalty} = K \sum_{\text{layers}} \left( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \right)
|
||
\]
|
||
|
||
* Default `K = 0.5`.
|
||
* Cap penalty at 0.3 to avoid over-weighting tiny blobs.
|
||
* Combine with other trust signals (reachability, provenance) to prioritise audits.
|
||
|
||
## 8. Implementation checklist
|
||
|
||
| Area | Task ID | Notes |
|
||
|------|---------|-------|
|
||
| Scanner analysis | `SCAN-ENTROPY-186-011` | Sliding window entropy & heuristics |
|
||
| Evidence output | `SCAN-ENTROPY-186-012` | JSON reports + DSSE |
|
||
| Policy integration | `POLICY-RISK-90-001` | Trust weight + explanations |
|
||
| UI | `UI-ENTROPY-40-001/002` | Visualisation & messaging |
|
||
| Docs | `DOCS-ENTROPY-70-004` | (this guide) |
|
||
|
||
Update this document as thresholds change or additional packer signatures are introduced.
|