git.stella-ops.org/docs/modules/scanner/entropy.md

# Entropy Analysis for Executable Layers

> **Imposed rule:** Entropy evidence must be included in scan exports and DSSE attestations; opaque regions without provenance cannot be whitelisted without an explicit policy waiver.
> **Status:** Stable (2025-11)  
> **Owners:** Scanner Guild · Policy Guild · UI Guild · Docs Guild

## 1. Overview

Entropy analysis highlights opaque regions inside container layers (packed binaries, stripped blobs, embedded firmware) so Stella Ops can prioritise artefacts that are hard to audit. The scanner computes per-file entropy metrics, reports opaque ratios per layer, and feeds penalties into the trust algebra.

## 2. Scanner pipeline (`SCAN-ENTROPY-186-011/012`)

* **Target files:** ELF, PE/COFF, Mach-O executables and large raw blobs (>16 KB). Archive formats (zip/tar) are unpacked by existing analyzers before entropy processing.
* **Section analysis:**  
  * ELF – `.text`, `.rodata`, `.data`, custom sections.  
  * PE – section table entries (`IMAGE_SECTION_HEADER`).  
  * Mach-O – LC_SEGMENT/LC_SEGMENT_64 sections.
* **Sliding window:** 4 KB window with 1 KB stride. Entropy calculated using Shannon entropy:

  \[
  H = -\sum_{i=0}^{255} p_i \log_2 p_i
  \]

  Windows with `H ≥ 7.2` bits/byte are marked “opaque”.
* **Heuristics & hints:**
  * Flag entire files with no symbols or stripped debug info.
  * Detect known packer section names (`.UPX*`, `.aspack`, etc.).
  * Record offsets, window sizes, and entropy values to support explainability.
* **Outputs (all canonical, UTF-8, sorted keys):**
  * `entropy.report.json` (per-file details, windows, hints; schema `stellaops.entropy/report@1`).
  * `layer_summary.json` (opaque byte ratios per layer and overall image; schema `stellaops.entropy/layer-summary@1`).
  * `entropy_penalty` scalar injected into trust lattice inputs.
* All outputs are signed within the scan DSSE bundle and exported in Offline/Replay kits.

All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestations/replay bundles.

## 3. JSON Schemas

### 3.1 `entropy.report.json`

```jsonc
{
  "schema": "stellaops.entropy/report@1",
  "generatedAt": "2025-11-26T12:00:00Z",
  "imageDigest": "sha256:…",
  "layerDigest": "sha256:…",
  "files": [
    {
      "path": "/opt/app/libblob.so",
      "size": 5242880,
      "opaqueBytes": 1342177,
      "opaqueRatio": 0.25,
      "flags": ["stripped", "section:.UPX0"],
      "windows": [
        { "offset": 0, "length": 4096, "entropy": 7.45 },
        { "offset": 1024, "length": 4096, "entropy": 7.38 }
      ]
    }
  ]
}
```

### 3.2 `layer_summary.json`

```jsonc
{
  "schema": "stellaops.entropy/layer-summary@1",
  "generatedAt": "2025-11-26T12:00:00Z",
  "imageDigest": "sha256:…",
  "layers": [
    {
      "digest": "sha256:layer4…",
      "opaqueBytes": 2306867,
      "totalBytes": 10485760,
      "opaqueRatio": 0.22,
      "indicators": ["packed", "no-symbols"]
    }
  ],
  "imageOpaqueRatio": 0.18,
  "entropyPenalty": 0.12
}
```

## 4. Policy integration (`POLICY-RISK-90-001`)

* Policy Engine receives `entropy_penalty` and per-layer ratios via scan evidence.
* Default thresholds (tenant-overridable):
  * Block when `imageOpaqueRatio > 0.15` **and** provenance unknown.
  * Warn when any executable has `opaqueRatio > 0.30`.
  * Suppress penalty when symbols are present **and** provenance attested.
* Trust lattice mapping:
  * `entropy_penalty` feeds the risk lattice alongside reachability, provenance, and exploitability signals; capped at 0.3.
  * Policy explanations include highest-entropy files, offsets, and reason codes (packed, no symbols, runtime reachable).

## 5. UI experience (`UI-ENTROPY-40-001/002`)

* **Heatmaps:** render entropy along the file timeline (green → red).
* **Layer donut:** show opaque % per layer with tooltips linking to file list.
* **“Why risky?” chips:** highlight triggers such as *Packed-like*, *Stripped*, *No symbols*.
* Policy banners explain configured thresholds and mitigation (add provenance, unpack, or accept risk).
* Provide direct download links to `entropy.report.json` for audits.

## 6. CLI / API hooks

* CLI – `stella scan artifacts --entropy --threshold 0.15 --top 10` prints top opaque files and penalty; exits non-zero when penalty exceeds threshold.
* CLI – `stella scan export --include entropy` bundles entropy reports with SBOM/VEX for Offline kits.
* API – `GET /api/v1/scans/{id}/entropy` serves summary + evidence references; supports `Accept: application/json` or NDJSON stream.
* Notify templates can include entropy penalties to escalate opaque images (channel: Ops/Sec).

## 7. Trust algebra

The penalty is computed as:

\[
\text{entropyPenalty} = \min\Bigg(0.3,\; K \sum_{\text{layers}} \big( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \big)\Bigg)
\]

* Default `K = 0.5`; tenants can override via policy lattice config.
* If symbols are present and attested, apply a 0.5 multiplier to the per-layer contribution.
* Combine with reachability and provenance weights before final risk verdict.

## 8. Implementation checklist

| Area | Task ID | Notes |
|------|---------|-------|
| Scanner analysis | `SCAN-ENTROPY-186-011` | Sliding window entropy & heuristics |
| Evidence output | `SCAN-ENTROPY-186-012` | JSON reports + DSSE |
| Policy integration | `POLICY-RISK-90-001` | Trust weight + explanations |
| UI | `UI-ENTROPY-40-001/002` | Visualisation & messaging |
| Docs | `DOCS-ENTROPY-70-004` | (this guide) |

Update this document as thresholds change or additional packer signatures are introduced.