git.stella-ops.org/docs/modules/scanner/entropy.md

Entropy Analysis for Executable Layers

Imposed rule: Entropy evidence must be included in scan exports and DSSE attestations; opaque regions without provenance cannot be whitelisted without an explicit policy waiver. Status: Stable (2025-11)
Owners: Scanner Guild · Policy Guild · UI Guild · Docs Guild

1. Overview

Entropy analysis highlights opaque regions inside container layers (packed binaries, stripped blobs, embedded firmware) so Stella Ops can prioritise artefacts that are hard to audit. The scanner computes per-file entropy metrics, reports opaque ratios per layer, and feeds penalties into the trust algebra.

2. Scanner pipeline (SCAN-ENTROPY-186-011/012)

• Target files: ELF, PE/COFF, Mach-O executables and large raw blobs (>16 KB). Archive formats (zip/tar) are unpacked by existing analyzers before entropy processing.

• Section analysis:

  • ELF – .text, .rodata, .data, custom sections.
  • PE – section table entries (IMAGE_SECTION_HEADER).
  • Mach-O – LC_SEGMENT/LC_SEGMENT_64 sections.

• Sliding window: 4 KB window with 1 KB stride. Entropy calculated using Shannon entropy:

  [ H = -\sum_{i=0}^{255} p_i \log_2 p_i ]

  Windows with H ≥ 7.2 bits/byte are marked “opaque”.

• Heuristics & hints:

  • Flag entire files with no symbols or stripped debug info.
  • Detect known packer section names (.UPX*, .aspack, etc.).
  • Record offsets, window sizes, and entropy values to support explainability.

• Outputs (all canonical, UTF-8, sorted keys):

  • entropy.report.json (per-file details, windows, hints; schema stellaops.entropy/report@1).
  • layer_summary.json (opaque byte ratios per layer and overall image; schema stellaops.entropy/layer-summary@1).
  • entropy_penalty scalar injected into trust lattice inputs.

• All outputs are signed within the scan DSSE bundle and exported in Offline/Replay kits.

All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestations/replay bundles.

3. JSON Schemas

3.1 entropy.report.json

{
  "schema": "stellaops.entropy/report@1",
  "generatedAt": "2025-11-26T12:00:00Z",
  "imageDigest": "sha256:…",
  "layerDigest": "sha256:…",
  "files": [
    {
      "path": "/opt/app/libblob.so",
      "size": 5242880,
      "opaqueBytes": 1342177,
      "opaqueRatio": 0.25,
      "flags": ["stripped", "section:.UPX0"],
      "windows": [
        { "offset": 0, "length": 4096, "entropy": 7.45 },
        { "offset": 1024, "length": 4096, "entropy": 7.38 }
      ]
    }
  ]
}

3.2 layer_summary.json

{
  "schema": "stellaops.entropy/layer-summary@1",
  "generatedAt": "2025-11-26T12:00:00Z",
  "imageDigest": "sha256:…",
  "layers": [
    {
      "digest": "sha256:layer4…",
      "opaqueBytes": 2306867,
      "totalBytes": 10485760,
      "opaqueRatio": 0.22,
      "indicators": ["packed", "no-symbols"]
    }
  ],
  "imageOpaqueRatio": 0.18,
  "entropyPenalty": 0.12
}

4. Policy integration (POLICY-RISK-90-001)

• Policy Engine receives entropy_penalty and per-layer ratios via scan evidence.
• Default thresholds (tenant-overridable):
  • Block when imageOpaqueRatio > 0.15 and provenance unknown.
  • Warn when any executable has opaqueRatio > 0.30.
  • Suppress penalty when symbols are present and provenance attested.
• Trust lattice mapping:
  • entropy_penalty feeds the risk lattice alongside reachability, provenance, and exploitability signals; capped at 0.3.
  • Policy explanations include highest-entropy files, offsets, and reason codes (packed, no symbols, runtime reachable).

5. UI experience (UI-ENTROPY-40-001/002)

• Heatmaps: render entropy along the file timeline (green → red).
• Layer donut: show opaque % per layer with tooltips linking to file list.
• “Why risky?” chips: highlight triggers such as Packed-like, Stripped, No symbols.
• Policy banners explain configured thresholds and mitigation (add provenance, unpack, or accept risk).
• Provide direct download links to entropy.report.json for audits.

6. CLI / API hooks

• CLI – stella scan artifacts --entropy --threshold 0.15 --top 10 prints top opaque files and penalty; exits non-zero when penalty exceeds threshold.
• CLI – stella scan export --include entropy bundles entropy reports with SBOM/VEX for Offline kits.
• API – GET /api/v1/scans/{id}/entropy serves summary + evidence references; supports Accept: application/json or NDJSON stream.
• Notify templates can include entropy penalties to escalate opaque images (channel: Ops/Sec).

7. Trust algebra

The penalty is computed as:

[ \text{entropyPenalty} = \min\Bigg(0.3,; K \sum_{\text{layers}} \big( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \big)\Bigg) ]

• Default K = 0.5; tenants can override via policy lattice config.
• If symbols are present and attested, apply a 0.5 multiplier to the per-layer contribution.
• Combine with reachability and provenance weights before final risk verdict.

8. Implementation checklist

Area	Task ID	Notes
Scanner analysis	SCAN-ENTROPY-186-011	Sliding window entropy & heuristics
Evidence output	SCAN-ENTROPY-186-012	JSON reports + DSSE
Policy integration	POLICY-RISK-90-001	Trust weight + explanations
UI	UI-ENTROPY-40-001/002	Visualisation & messaging
Docs	DOCS-ENTROPY-70-004	(this guide)

Update this document as thresholds change or additional packer signatures are introduced.