advisories update
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,972 @@
|
|||||||
|
Here’s a compact, ready‑to‑run plan to benchmark how consistently different vulnerability scanners score the *same* SBOM/VEX—so we can quantify Stella Ops’ determinism advantage.
|
||||||
|
|
||||||
|
# Why this matters (quickly)
|
||||||
|
|
||||||
|
Scanners often disagree on CVSS and “severity after VEX.” Measuring variance under identical inputs lets us prove scoring stability (a Stella Ops moat: deterministic, replayable scans).
|
||||||
|
|
||||||
|
# What we’ll measure
|
||||||
|
|
||||||
|
* **Determinism rate**: % of runs yielding identical (hash‑equal) results per scanner.
|
||||||
|
* **CVSS delta σ**: standard deviation of (scanner_score − reference_score) across vulns.
|
||||||
|
* **Order‑invariance**: re-feed inputs in randomized orders; expect identical outputs.
|
||||||
|
* **VEX application stability**: variance before vs. after applying VEX justifications.
|
||||||
|
* **Drift vs. feeds**: pin feeds to content hashes; any change must be attributable.
|
||||||
|
|
||||||
|
# Inputs (frozen & hashed)
|
||||||
|
|
||||||
|
* 3–5 **SBOMs** (CycloneDX 1.6 + SPDX 3.0.1) from well‑known images (e.g., nginx, keycloak, alpine‑glibc, a Java app, a Node app).
|
||||||
|
* Matching **VEX** docs (CycloneDX VEX) covering “not affected,” “affected,” and “fixed.”
|
||||||
|
* **Feeds bundle**: vendor DBs (NVD, GHSA, distro OVAL), all vendored and hashed.
|
||||||
|
* **Policy**: identical normalization rules (CVSS v3.1 only, prefer vendor over NVD, etc.).
|
||||||
|
|
||||||
|
# Scanners (example set)
|
||||||
|
|
||||||
|
* Anchore/Grype, Trivy, Snyk CLI, osv‑scanner, Dependency‑Track API (server mode), plus **Stella Ops Scanner**.
|
||||||
|
|
||||||
|
# Protocol (10 runs × 2 orders)
|
||||||
|
|
||||||
|
1. **Pin environment** (Docker images + air‑gapped tarballs). Record:
|
||||||
|
|
||||||
|
* tool version, container digest, feed bundle SHA‑256, SBOM/VEX SHA‑256.
|
||||||
|
2. **Run matrix**: for each SBOM/VEX, per scanner:
|
||||||
|
|
||||||
|
* 10 runs with canonical file order.
|
||||||
|
* 10 runs with randomized SBOM component order + shuffled VEX statements.
|
||||||
|
3. **Capture** normalized JSON: `{purl, vuln_id, base_cvss, effective_severity, vex_applied, notes}`.
|
||||||
|
4. **Hash** each run’s full result (SHA‑256 over canonical JSON).
|
||||||
|
5. **Compute** metrics per scanner:
|
||||||
|
|
||||||
|
* Determinism rate = identical_hash_runs / total_runs.
|
||||||
|
* σ(CVSS delta) vs. reference (choose NVD base as reference, or Stella policy).
|
||||||
|
* Order‑invariance failures (# of distinct hashes between canonical vs. shuffled).
|
||||||
|
* VEX stability: σ before vs. after VEX; Δσ should shrink, not grow.
|
||||||
|
|
||||||
|
# Minimal harness (Python outline)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# run_bench.py
|
||||||
|
# prerequisites: docker CLI, Python 3.10+, numpy, pandas
|
||||||
|
from pathlib import Path
|
||||||
|
import json, hashlib, random, subprocess
|
||||||
|
import numpy as np
|
||||||
|
SBOMS = ["sboms/nginx.cdx.json", "sboms/keycloak.spdx.json", ...]
|
||||||
|
VEXES = ["vex/nginx.vex.json", "vex/keycloak.vex.json", ...]
|
||||||
|
SCANNERS = {
|
||||||
|
"grype": ["docker","run","--rm","-v","$PWD:/w","grype:TAG","--input","/w/{sbom}","--vex","/w/{vex}","--output","json"],
|
||||||
|
"trivy": ["docker","run","--rm","-v","$PWD:/w","aquasec/trivy:TAG","sbom","/w/{sbom}","--vex","/w/{vex}","--format","json"],
|
||||||
|
# add more…
|
||||||
|
"stella": ["docker","run","--rm","-v","$PWD:/w","stellaops/scanner:TAG","scan","--sbom","/w/{sbom}","--vex","/w/{vex}","--normalize","json"]
|
||||||
|
}
|
||||||
|
def canon(obj): return json.dumps(obj, sort_keys=True, separators=(",",":")).encode()
|
||||||
|
def shas(b): return hashlib.sha256(b).hexdigest()
|
||||||
|
def shuffle_file(src, dst): # implement component/VEX statement shuffle preserving semantics
|
||||||
|
data = json.load(open(src))
|
||||||
|
for k in ("components","vulnerabilities","vex","statements"):
|
||||||
|
if isinstance(data, dict) and k in data and isinstance(data[k], list):
|
||||||
|
random.shuffle(data[k])
|
||||||
|
json.dump(data, open(dst,"w"), indent=0, separators=(",",":"))
|
||||||
|
def run(cmd): return subprocess.check_output(cmd, text=True)
|
||||||
|
|
||||||
|
results=[]
|
||||||
|
for sbom, vex in zip(SBOMS, VEXES):
|
||||||
|
for scanner, tmpl in SCANNERS.items():
|
||||||
|
for mode in ("canonical","shuffled"):
|
||||||
|
for i in range(10):
|
||||||
|
sb, vx = sbom, vex
|
||||||
|
if mode=="shuffled":
|
||||||
|
sb, vx = f"tmp/{Path(sbom).stem}.json", f"tmp/{Path(vex).stem}.json"
|
||||||
|
shuffle_file(sbom, sb); shuffle_file(vex, vx)
|
||||||
|
out = run([c.format(sbom=sb, vex=vx) for c in tmpl])
|
||||||
|
j = json.loads(out)
|
||||||
|
# normalize to minimal tuple per finding (purl,id,base_cvss,effective)
|
||||||
|
norm = [{"purl":x["purl"],"id":x["id"],"base":x.get("cvss","NA"),
|
||||||
|
"eff":x.get("effectiveSeverity","NA")} for x in j.get("findings",[])]
|
||||||
|
blob = canon({"scanner":scanner,"sbom":sbom,"vex":vex,"findings":norm})
|
||||||
|
results.append({
|
||||||
|
"scanner":scanner,"sbom":sbom,"mode":mode,"run":i,
|
||||||
|
"hash":shas(blob),"norm":norm
|
||||||
|
})
|
||||||
|
# compute stats (pandas groupby): determinism %, std dev of (eff - ref) per (scanner,sbom)
|
||||||
|
```
|
||||||
|
|
||||||
|
# Pass/Fail gates (suggested)
|
||||||
|
|
||||||
|
* **Determinism ≥ 99%** across 20 runs per (scanner, SBOM).
|
||||||
|
* **Order‑invariance = 100%** identical hashes.
|
||||||
|
* **VEX stability**: σ_after ≤ σ_before (VEX reduces variance).
|
||||||
|
* **Provenance**: any change must correlate to a different feed bundle hash.
|
||||||
|
|
||||||
|
# Deliverables
|
||||||
|
|
||||||
|
* `bench/` with SBOMs, VEX, feeds bundle manifest (hashes).
|
||||||
|
* `run_bench.py` + `analyze.ipynb` (charts: determinism%, σ by scanner).
|
||||||
|
* One‑page **Stella Ops Differentiator**: “Provable Scoring Stability” with the above metrics and reproducibility recipe.
|
||||||
|
|
||||||
|
# Next step
|
||||||
|
|
||||||
|
If you want, I’ll generate the folder skeleton, example SBOM/VEX, and the analysis notebook stub so you can drop in your scanners and hit run.
|
||||||
|
Here’s a concrete, .NET‑friendly implementation plan you can actually build, not just admire in a doc.
|
||||||
|
|
||||||
|
I’ll assume:
|
||||||
|
|
||||||
|
* .NET 8 (or 6) SDK
|
||||||
|
* Windows or Linux dev machine with Docker installed
|
||||||
|
* You’re comfortable with basic C#, CLI, and JSON
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Project structure
|
||||||
|
|
||||||
|
Create a simple solution with two projects:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet new sln -n ScannerBench
|
||||||
|
cd ScannerBench
|
||||||
|
|
||||||
|
dotnet new console -n ScannerBench.Runner
|
||||||
|
dotnet new xunit -n ScannerBench.Tests
|
||||||
|
|
||||||
|
dotnet sln add ScannerBench.Runner/ScannerBench.Runner.csproj
|
||||||
|
dotnet sln add ScannerBench.Tests/ScannerBench.Tests.csproj
|
||||||
|
dotnet add ScannerBench.Tests reference ScannerBench.Runner
|
||||||
|
```
|
||||||
|
|
||||||
|
Inside `ScannerBench.Runner` create folders:
|
||||||
|
|
||||||
|
* `Inputs/` – SBOM & VEX files
|
||||||
|
|
||||||
|
* `Inputs/Sboms/nginx.cdx.json`
|
||||||
|
* `Inputs/Vex/nginx.vex.json`
|
||||||
|
* (and a few more pairs)
|
||||||
|
* `Config/` – scanner config JSON or YAML later if you want
|
||||||
|
* `Results/` – captured run outputs (for debugging / manual inspection)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Define core domain models (C#)
|
||||||
|
|
||||||
|
In `ScannerBench.Runner` add a file `Models.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System.Collections.Generic;
|
||||||
|
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public sealed record ScannerConfig(
|
||||||
|
string Name,
|
||||||
|
string DockerImage,
|
||||||
|
string[] CommandTemplate // tokens; use {sbom} and {vex} placeholders
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record BenchInput(
|
||||||
|
string Id, // e.g. "nginx-cdx"
|
||||||
|
string SbomPath,
|
||||||
|
string VexPath
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record NormalizedFinding(
|
||||||
|
string Purl,
|
||||||
|
string VulnerabilityId, // CVE-2021‑1234, GHSA‑xxx, etc.
|
||||||
|
string BaseCvss, // normalized to string for simplicity
|
||||||
|
string EffectiveSeverity // e.g. "LOW", "MEDIUM", "HIGH"
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record ScanRun(
|
||||||
|
string ScannerName,
|
||||||
|
string InputId,
|
||||||
|
int RunIndex,
|
||||||
|
string Mode, // "canonical" | "shuffled"
|
||||||
|
string ResultHash,
|
||||||
|
IReadOnlyList<NormalizedFinding> Findings
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record DeterminismStats(
|
||||||
|
string ScannerName,
|
||||||
|
string InputId,
|
||||||
|
string Mode,
|
||||||
|
int TotalRuns,
|
||||||
|
int DistinctHashes
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record CvssDeltaStats(
|
||||||
|
string ScannerName,
|
||||||
|
string InputId,
|
||||||
|
double MeanDelta,
|
||||||
|
double StdDevDelta
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
You can grow this later, but this is enough to get the first version working.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Hard‑code scanner configs (first pass)
|
||||||
|
|
||||||
|
In `ScannerConfigs.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class ScannerConfigs
|
||||||
|
{
|
||||||
|
public static readonly ScannerConfig[] All =
|
||||||
|
{
|
||||||
|
new(
|
||||||
|
Name: "grype",
|
||||||
|
DockerImage: "anchore/grype:v0.79.0",
|
||||||
|
CommandTemplate: new[]
|
||||||
|
{
|
||||||
|
"grype",
|
||||||
|
"--input", "/work/{sbom}",
|
||||||
|
"--output", "json"
|
||||||
|
// add flags like --vex when supported
|
||||||
|
}
|
||||||
|
),
|
||||||
|
new(
|
||||||
|
Name: "trivy",
|
||||||
|
DockerImage: "aquasec/trivy:0.55.0",
|
||||||
|
CommandTemplate: new[]
|
||||||
|
{
|
||||||
|
"trivy", "sbom", "/work/{sbom}",
|
||||||
|
"--format", "json"
|
||||||
|
}
|
||||||
|
),
|
||||||
|
new(
|
||||||
|
Name: "stella",
|
||||||
|
DockerImage: "stellaops/scanner:latest",
|
||||||
|
CommandTemplate: new[]
|
||||||
|
{
|
||||||
|
"scanner", "scan",
|
||||||
|
"--sbom", "/work/{sbom}",
|
||||||
|
"--vex", "/work/{vex}",
|
||||||
|
"--output-format", "json"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You can tweak command templates once you wire up actual tools.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Input set (SBOM + VEX pairs)
|
||||||
|
|
||||||
|
In `BenchInputs.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class BenchInputs
|
||||||
|
{
|
||||||
|
public static readonly BenchInput[] All =
|
||||||
|
{
|
||||||
|
new("nginx-cdx", "Inputs/Sboms/nginx.cdx.json", "Inputs/Vex/nginx.vex.json"),
|
||||||
|
new("keycloak-spdx", "Inputs/Sboms/keycloak.spdx.json", "Inputs/Vex/keycloak.vex.json")
|
||||||
|
// add more as needed
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Populate `Inputs/Sboms` and `Inputs/Vex` manually or with a script (doesn’t need to be .NET).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Utility: JSON shuffle to test order‑invariance
|
||||||
|
|
||||||
|
You want to randomize component/vulnerability/VEX statement order to confirm that scanners don’t change results based on input ordering.
|
||||||
|
|
||||||
|
Create `JsonShuffler.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.IO;
|
||||||
|
using System.Linq;
|
||||||
|
using System.Text.Json;
|
||||||
|
using System.Text.Json.Nodes;
|
||||||
|
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class JsonShuffler
|
||||||
|
{
|
||||||
|
private static readonly string[] ListKeysToShuffle =
|
||||||
|
{
|
||||||
|
"components",
|
||||||
|
"vulnerabilities",
|
||||||
|
"statements",
|
||||||
|
"vex"
|
||||||
|
};
|
||||||
|
|
||||||
|
public static string CreateShuffledCopy(string sourcePath, string tmpDir)
|
||||||
|
{
|
||||||
|
Directory.CreateDirectory(tmpDir);
|
||||||
|
|
||||||
|
var jsonText = File.ReadAllText(sourcePath);
|
||||||
|
var node = JsonNode.Parse(jsonText);
|
||||||
|
if (node is null)
|
||||||
|
throw new InvalidOperationException($"Could not parse JSON: {sourcePath}");
|
||||||
|
|
||||||
|
ShuffleLists(node);
|
||||||
|
|
||||||
|
var fileName = Path.GetFileName(sourcePath);
|
||||||
|
var destPath = Path.Combine(tmpDir, fileName);
|
||||||
|
File.WriteAllText(destPath, node.ToJsonString(new JsonSerializerOptions
|
||||||
|
{
|
||||||
|
WriteIndented = false
|
||||||
|
}));
|
||||||
|
return destPath;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void ShuffleLists(JsonNode node)
|
||||||
|
{
|
||||||
|
if (node is JsonObject obj)
|
||||||
|
{
|
||||||
|
foreach (var kvp in obj.ToList())
|
||||||
|
{
|
||||||
|
if (kvp.Value is JsonArray arr && ListKeysToShuffle.Contains(kvp.Key))
|
||||||
|
{
|
||||||
|
ShuffleInPlace(arr);
|
||||||
|
}
|
||||||
|
else if (kvp.Value is not null)
|
||||||
|
{
|
||||||
|
ShuffleLists(kvp.Value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (node is JsonArray arr)
|
||||||
|
{
|
||||||
|
foreach (var child in arr)
|
||||||
|
{
|
||||||
|
if (child is not null)
|
||||||
|
ShuffleLists(child);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void ShuffleInPlace(JsonArray arr)
|
||||||
|
{
|
||||||
|
var rnd = new Random();
|
||||||
|
var list = arr.ToList();
|
||||||
|
arr.Clear();
|
||||||
|
foreach (var item in list.OrderBy(_ => rnd.Next()))
|
||||||
|
{
|
||||||
|
arr.Add(item);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Utility: run Dockerized scanner from C#
|
||||||
|
|
||||||
|
Create `DockerRunner.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System;
|
||||||
|
using System.Diagnostics;
|
||||||
|
using System.IO;
|
||||||
|
using System.Linq;
|
||||||
|
using System.Text;
|
||||||
|
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class DockerRunner
|
||||||
|
{
|
||||||
|
public static string RunScanner(
|
||||||
|
ScannerConfig scanner,
|
||||||
|
string sbomPath,
|
||||||
|
string? vexPath,
|
||||||
|
string workDir)
|
||||||
|
{
|
||||||
|
// Build the container command (inside container)
|
||||||
|
var innerCmdTokens = scanner.CommandTemplate
|
||||||
|
.Select(t => t.Replace("{sbom}", sbomPath.Replace("\\", "/"))
|
||||||
|
.Replace("{vex}", vexPath?.Replace("\\", "/") ?? ""))
|
||||||
|
.ToArray();
|
||||||
|
|
||||||
|
// We run: docker run --rm -v <fullpath>:/work <image> <innerCmdTokens...>
|
||||||
|
var dockerArgs = new StringBuilder();
|
||||||
|
dockerArgs.Append("run --rm ");
|
||||||
|
dockerArgs.Append($"-v \"{workDir}:/work\" ");
|
||||||
|
dockerArgs.Append(scanner.DockerImage);
|
||||||
|
dockerArgs.Append(' ');
|
||||||
|
dockerArgs.Append(string.Join(' ', innerCmdTokens.Select(Escape)));
|
||||||
|
|
||||||
|
return RunProcess("docker", dockerArgs.ToString());
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string RunProcess(string fileName, string arguments)
|
||||||
|
{
|
||||||
|
var psi = new ProcessStartInfo
|
||||||
|
{
|
||||||
|
FileName = fileName,
|
||||||
|
Arguments = arguments,
|
||||||
|
RedirectStandardOutput = true,
|
||||||
|
RedirectStandardError = true,
|
||||||
|
UseShellExecute = false,
|
||||||
|
CreateNoWindow = true
|
||||||
|
};
|
||||||
|
|
||||||
|
using var process = Process.Start(psi)
|
||||||
|
?? throw new InvalidOperationException("Failed to start process");
|
||||||
|
var stdout = process.StandardOutput.ReadToEnd();
|
||||||
|
var stderr = process.StandardError.ReadToEnd();
|
||||||
|
process.WaitForExit();
|
||||||
|
|
||||||
|
if (process.ExitCode != 0)
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"Process failed ({fileName} {arguments}): {stderr}");
|
||||||
|
}
|
||||||
|
|
||||||
|
return stdout;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string Escape(string arg)
|
||||||
|
{
|
||||||
|
if (string.IsNullOrEmpty(arg)) return "\"\"";
|
||||||
|
if (arg.Contains(' ') || arg.Contains('"'))
|
||||||
|
{
|
||||||
|
return "\"" + arg.Replace("\"", "\\\"") + "\"";
|
||||||
|
}
|
||||||
|
return arg;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
* `workDir` will be your project directory (so `/work/Inputs/...` inside the container).
|
||||||
|
* For simplicity, I’m not handling Windows vs Linux nuances heavily; adjust path escaping if needed on Windows.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Utility: Normalize scanner JSON output
|
||||||
|
|
||||||
|
Different scanners have different JSON; you just need a **mapping** from each scanner to the `NormalizedFinding` shape.
|
||||||
|
|
||||||
|
Create `Normalizer.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Text.Json;
|
||||||
|
using System.Text.Json.Nodes;
|
||||||
|
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class Normalizer
|
||||||
|
{
|
||||||
|
public static IReadOnlyList<NormalizedFinding> Normalize(
|
||||||
|
string scannerName,
|
||||||
|
string rawJson)
|
||||||
|
{
|
||||||
|
var node = JsonNode.Parse(rawJson)
|
||||||
|
?? throw new InvalidOperationException("Cannot parse scanner JSON");
|
||||||
|
|
||||||
|
return scannerName switch
|
||||||
|
{
|
||||||
|
"grype" => NormalizeGrype(node),
|
||||||
|
"trivy" => NormalizeTrivy(node),
|
||||||
|
"stella" => NormalizeStella(node),
|
||||||
|
_ => throw new NotSupportedException($"Unknown scanner: {scannerName}")
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private static IReadOnlyList<NormalizedFinding> NormalizeGrype(JsonNode root)
|
||||||
|
{
|
||||||
|
// Adjust based on actual Grype JSON
|
||||||
|
var findings = new List<NormalizedFinding>();
|
||||||
|
var matches = root["matches"] as JsonArray;
|
||||||
|
if (matches is null) return findings;
|
||||||
|
|
||||||
|
foreach (var m in matches)
|
||||||
|
{
|
||||||
|
if (m is null) continue;
|
||||||
|
var artifact = m["artifact"];
|
||||||
|
var vuln = m["vulnerability"];
|
||||||
|
|
||||||
|
var purl = artifact?["purl"]?.ToString() ?? "";
|
||||||
|
var id = vuln?["id"]?.ToString() ?? "";
|
||||||
|
var cvss = vuln?["cvss"]?[0]?["metrics"]?["baseScore"]?.ToString() ?? "NA";
|
||||||
|
var severity = vuln?["severity"]?.ToString() ?? "UNKNOWN";
|
||||||
|
|
||||||
|
findings.Add(new NormalizedFinding(
|
||||||
|
Purl: purl,
|
||||||
|
VulnerabilityId: id,
|
||||||
|
BaseCvss: cvss,
|
||||||
|
EffectiveSeverity: severity.ToUpperInvariant()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
return findings;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static IReadOnlyList<NormalizedFinding> NormalizeTrivy(JsonNode root)
|
||||||
|
{
|
||||||
|
var list = new List<NormalizedFinding>();
|
||||||
|
|
||||||
|
var results = root["Results"] as JsonArray;
|
||||||
|
if (results is null) return list;
|
||||||
|
|
||||||
|
foreach (var r in results)
|
||||||
|
{
|
||||||
|
var vulnerabilities = r?["Vulnerabilities"] as JsonArray;
|
||||||
|
if (vulnerabilities is null) continue;
|
||||||
|
|
||||||
|
foreach (var v in vulnerabilities)
|
||||||
|
{
|
||||||
|
if (v is null) continue;
|
||||||
|
var pkgName = v["PkgName"]?.ToString() ?? "";
|
||||||
|
var purl = v["Purl"]?.ToString() ?? pkgName;
|
||||||
|
var id = v["VulnerabilityID"]?.ToString() ?? "";
|
||||||
|
var cvss = v["CVSS"]?["nvd"]?["V3Score"]?.ToString()
|
||||||
|
?? v["CVSS"]?["nvd"]?["V2Score"]?.ToString()
|
||||||
|
?? "NA";
|
||||||
|
var severity = v["Severity"]?.ToString() ?? "UNKNOWN";
|
||||||
|
|
||||||
|
list.Add(new NormalizedFinding(
|
||||||
|
Purl: purl,
|
||||||
|
VulnerabilityId: id,
|
||||||
|
BaseCvss: cvss,
|
||||||
|
EffectiveSeverity: severity.ToUpperInvariant()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return list;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static IReadOnlyList<NormalizedFinding> NormalizeStella(JsonNode root)
|
||||||
|
{
|
||||||
|
// Adjust to match Stella Ops output schema
|
||||||
|
var list = new List<NormalizedFinding>();
|
||||||
|
var findings = root["findings"] as JsonArray;
|
||||||
|
if (findings is null) return list;
|
||||||
|
|
||||||
|
foreach (var f in findings)
|
||||||
|
{
|
||||||
|
if (f is null) continue;
|
||||||
|
var purl = f["purl"]?.ToString() ?? "";
|
||||||
|
var id = f["id"]?.ToString() ?? "";
|
||||||
|
var cvss = f["baseCvss"]?.ToString()
|
||||||
|
?? f["cvss"]?.ToString()
|
||||||
|
?? "NA";
|
||||||
|
var severity = f["effectiveSeverity"]?.ToString()
|
||||||
|
?? f["severity"]?.ToString()
|
||||||
|
?? "UNKNOWN";
|
||||||
|
|
||||||
|
list.Add(new NormalizedFinding(
|
||||||
|
Purl: purl,
|
||||||
|
VulnerabilityId: id,
|
||||||
|
BaseCvss: cvss,
|
||||||
|
EffectiveSeverity: severity.ToUpperInvariant()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
return list;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You’ll need to tweak the JSON paths once you inspect real outputs, but the pattern is clear.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Utility: Hashing & canonicalization
|
||||||
|
|
||||||
|
Create `Hashing.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Linq;
|
||||||
|
using System.Security.Cryptography;
|
||||||
|
using System.Text;
|
||||||
|
using System.Text.Json;
|
||||||
|
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class Hashing
|
||||||
|
{
|
||||||
|
public static string ComputeResultHash(
|
||||||
|
string scannerName,
|
||||||
|
string inputId,
|
||||||
|
IReadOnlyList<NormalizedFinding> findings)
|
||||||
|
{
|
||||||
|
// Ensure deterministic ordering before hashing
|
||||||
|
var ordered = findings
|
||||||
|
.OrderBy(f => f.Purl)
|
||||||
|
.ThenBy(f => f.VulnerabilityId)
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
var payload = new
|
||||||
|
{
|
||||||
|
scanner = scannerName,
|
||||||
|
input = inputId,
|
||||||
|
findings = ordered
|
||||||
|
};
|
||||||
|
|
||||||
|
var json = JsonSerializer.Serialize(payload,
|
||||||
|
new JsonSerializerOptions
|
||||||
|
{
|
||||||
|
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
|
||||||
|
});
|
||||||
|
|
||||||
|
using var sha = SHA256.Create();
|
||||||
|
var bytes = Encoding.UTF8.GetBytes(json);
|
||||||
|
var hashBytes = sha.ComputeHash(bytes);
|
||||||
|
return ConvertToHex(hashBytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string ConvertToHex(byte[] bytes)
|
||||||
|
{
|
||||||
|
var sb = new StringBuilder(bytes.Length * 2);
|
||||||
|
foreach (var b in bytes)
|
||||||
|
sb.Append(b.ToString("x2"));
|
||||||
|
return sb.ToString();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Metric computation (determinism & CVSS deltas)
|
||||||
|
|
||||||
|
Create `StatsCalculator.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Globalization;
|
||||||
|
using System.Linq;
|
||||||
|
|
||||||
|
namespace ScannerBench.Runner;
|
||||||
|
|
||||||
|
public static class StatsCalculator
|
||||||
|
{
|
||||||
|
public static DeterminismStats ComputeDeterminism(
|
||||||
|
string scannerName,
|
||||||
|
string inputId,
|
||||||
|
string mode,
|
||||||
|
IReadOnlyList<ScanRun> runs)
|
||||||
|
{
|
||||||
|
var hashes = runs.Select(r => r.ResultHash).Distinct().Count();
|
||||||
|
return new DeterminismStats(
|
||||||
|
ScannerName: scannerName,
|
||||||
|
InputId: inputId,
|
||||||
|
Mode: mode,
|
||||||
|
TotalRuns: runs.Count,
|
||||||
|
DistinctHashes: hashes
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public static CvssDeltaStats ComputeCvssDeltas(
|
||||||
|
string scannerName,
|
||||||
|
string inputId,
|
||||||
|
IReadOnlyList<ScanRun> scannerRuns,
|
||||||
|
IReadOnlyList<ScanRun> referenceRuns)
|
||||||
|
{
|
||||||
|
// Use the *first* run of each as baseline (assuming deterministic)
|
||||||
|
var scannerFindings = scannerRuns.First().Findings;
|
||||||
|
var refFindings = referenceRuns.First().Findings;
|
||||||
|
|
||||||
|
// Map by (purl,id)
|
||||||
|
var refMap = refFindings.ToDictionary(
|
||||||
|
f => (f.Purl, f.VulnerabilityId),
|
||||||
|
f => ParseCvss(f.BaseCvss)
|
||||||
|
);
|
||||||
|
|
||||||
|
var deltas = new List<double>();
|
||||||
|
|
||||||
|
foreach (var f in scannerFindings)
|
||||||
|
{
|
||||||
|
if (!refMap.TryGetValue((f.Purl, f.VulnerabilityId), out var refScore))
|
||||||
|
continue;
|
||||||
|
|
||||||
|
var score = ParseCvss(f.BaseCvss);
|
||||||
|
if (double.IsNaN(score) || double.IsNaN(refScore))
|
||||||
|
continue;
|
||||||
|
|
||||||
|
deltas.Add(score - refScore);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (deltas.Count == 0)
|
||||||
|
{
|
||||||
|
return new CvssDeltaStats(scannerName, inputId, double.NaN, double.NaN);
|
||||||
|
}
|
||||||
|
|
||||||
|
var mean = deltas.Average();
|
||||||
|
var variance = deltas.Sum(d => Math.Pow(d - mean, 2)) / deltas.Count;
|
||||||
|
var stdDev = Math.Sqrt(variance);
|
||||||
|
|
||||||
|
return new CvssDeltaStats(scannerName, inputId, mean, stdDev);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static double ParseCvss(string value)
|
||||||
|
{
|
||||||
|
if (double.TryParse(value, NumberStyles.Float, CultureInfo.InvariantCulture, out var v))
|
||||||
|
return v;
|
||||||
|
return double.NaN;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Pick your “reference” scanner (e.g., NVD‑aligned policy or Stella) when you call this method.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Main runner: orchestrate everything
|
||||||
|
|
||||||
|
In `Program.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.IO;
|
||||||
|
using System.Linq;
|
||||||
|
using ScannerBench.Runner;
|
||||||
|
|
||||||
|
class Program
|
||||||
|
{
|
||||||
|
static void Main(string[] args)
|
||||||
|
{
|
||||||
|
var projectRoot = GetProjectRoot();
|
||||||
|
var tmpDir = Path.Combine(projectRoot, "Tmp");
|
||||||
|
Directory.CreateDirectory(tmpDir);
|
||||||
|
|
||||||
|
const int runsPerMode = 10;
|
||||||
|
|
||||||
|
var allRuns = new List<ScanRun>();
|
||||||
|
|
||||||
|
foreach (var input in BenchInputs.All)
|
||||||
|
{
|
||||||
|
Console.WriteLine($"=== Input: {input.Id} ===");
|
||||||
|
|
||||||
|
foreach (var scanner in ScannerConfigs.All)
|
||||||
|
{
|
||||||
|
Console.WriteLine($" Scanner: {scanner.Name}");
|
||||||
|
|
||||||
|
// Canonical runs
|
||||||
|
var canonicalRuns = RunMultiple(
|
||||||
|
scanner, input, projectRoot, tmpDir,
|
||||||
|
mode: "canonical", runsPerMode);
|
||||||
|
|
||||||
|
// Shuffled runs
|
||||||
|
var shuffledRuns = RunMultiple(
|
||||||
|
scanner, input, projectRoot, tmpDir,
|
||||||
|
mode: "shuffled", runsPerMode);
|
||||||
|
|
||||||
|
allRuns.AddRange(canonicalRuns);
|
||||||
|
allRuns.AddRange(shuffledRuns);
|
||||||
|
|
||||||
|
// Determinism stats
|
||||||
|
var canonStats = StatsCalculator.ComputeDeterminism(
|
||||||
|
scanner.Name, input.Id, "canonical", canonicalRuns);
|
||||||
|
var shuffleStats = StatsCalculator.ComputeDeterminism(
|
||||||
|
scanner.Name, input.Id, "shuffled", shuffledRuns);
|
||||||
|
|
||||||
|
Console.WriteLine($" Canonical: {canonStats.DistinctHashes}/{canonStats.TotalRuns} distinct hashes");
|
||||||
|
Console.WriteLine($" Shuffled: {shuffleStats.DistinctHashes}/{shuffleStats.TotalRuns} distinct hashes");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Example: compute CVSS deltas vs Stella
|
||||||
|
var stellaByInput = allRuns.Where(r => r.ScannerName == "stella")
|
||||||
|
.GroupBy(r => r.InputId)
|
||||||
|
.ToDictionary(g => g.Key, g => g.ToList());
|
||||||
|
|
||||||
|
foreach (var scanner in ScannerConfigs.All.Where(s => s.Name != "stella"))
|
||||||
|
{
|
||||||
|
foreach (var input in BenchInputs.All)
|
||||||
|
{
|
||||||
|
var scannerRuns = allRuns
|
||||||
|
.Where(r => r.ScannerName == scanner.Name &&
|
||||||
|
r.InputId == input.Id &&
|
||||||
|
r.Mode == "canonical")
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
if (scannerRuns.Count == 0 || !stellaByInput.TryGetValue(input.Id, out var stellaRuns))
|
||||||
|
continue;
|
||||||
|
|
||||||
|
var stats = StatsCalculator.ComputeCvssDeltas(
|
||||||
|
scanner.Name,
|
||||||
|
input.Id,
|
||||||
|
scannerRuns,
|
||||||
|
stellaRuns.Where(r => r.Mode == "canonical").ToList());
|
||||||
|
|
||||||
|
Console.WriteLine(
|
||||||
|
$"CVSS delta vs Stella [{scanner.Name}, {input.Id}]: mean={stats.MeanDelta:F2}, stddev={stats.StdDevDelta:F2}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Console.WriteLine("Done.");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static List<ScanRun> RunMultiple(
|
||||||
|
ScannerConfig scanner,
|
||||||
|
BenchInput input,
|
||||||
|
string projectRoot,
|
||||||
|
string tmpDir,
|
||||||
|
string mode,
|
||||||
|
int runsPerMode)
|
||||||
|
{
|
||||||
|
var list = new List<ScanRun>();
|
||||||
|
var inputSbomFull = Path.Combine(projectRoot, input.SbomPath);
|
||||||
|
var inputVexFull = Path.Combine(projectRoot, input.VexPath);
|
||||||
|
|
||||||
|
for (int i = 0; i < runsPerMode; i++)
|
||||||
|
{
|
||||||
|
string sbomPath;
|
||||||
|
string vexPath;
|
||||||
|
|
||||||
|
if (mode == "canonical")
|
||||||
|
{
|
||||||
|
sbomPath = input.SbomPath; // path relative to /work
|
||||||
|
vexPath = input.VexPath;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
sbomPath = Path.GetRelativePath(
|
||||||
|
projectRoot,
|
||||||
|
JsonShuffler.CreateShuffledCopy(inputSbomFull, tmpDir));
|
||||||
|
|
||||||
|
vexPath = Path.GetRelativePath(
|
||||||
|
projectRoot,
|
||||||
|
JsonShuffler.CreateShuffledCopy(inputVexFull, tmpDir));
|
||||||
|
}
|
||||||
|
|
||||||
|
var rawJson = DockerRunner.RunScanner(
|
||||||
|
scanner,
|
||||||
|
sbomPath,
|
||||||
|
vexPath,
|
||||||
|
projectRoot);
|
||||||
|
|
||||||
|
var findings = Normalizer.Normalize(scanner.Name, rawJson);
|
||||||
|
var hash = Hashing.ComputeResultHash(scanner.Name, input.Id, findings);
|
||||||
|
|
||||||
|
list.Add(new ScanRun(
|
||||||
|
ScannerName: scanner.Name,
|
||||||
|
InputId: input.Id,
|
||||||
|
RunIndex: i,
|
||||||
|
Mode: mode,
|
||||||
|
ResultHash: hash,
|
||||||
|
Findings: findings));
|
||||||
|
|
||||||
|
Console.WriteLine($" {mode} run {i + 1}/{runsPerMode}: hash={hash[..8]}...");
|
||||||
|
}
|
||||||
|
|
||||||
|
return list;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string GetProjectRoot()
|
||||||
|
{
|
||||||
|
var dir = Directory.GetCurrentDirectory();
|
||||||
|
// If you run from bin/Debug, go up until we find .sln or .git, or just go two levels up
|
||||||
|
return dir;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This is intentionally straightforward: run all scanners × inputs × modes, gather runs, print determinism stats and CVSS deltas vs Stella.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Add a couple of automated tests (xUnit)
|
||||||
|
|
||||||
|
In `ScannerBench.Tests`, create `StatsTests.cs`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using ScannerBench.Runner;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
public class StatsTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void Determinism_Is_One_When_All_Hashes_Equal()
|
||||||
|
{
|
||||||
|
var runs = new List<ScanRun>
|
||||||
|
{
|
||||||
|
new("s", "i", 0, "canonical", "aaa", new List<NormalizedFinding>()),
|
||||||
|
new("s", "i", 1, "canonical", "aaa", new List<NormalizedFinding>()),
|
||||||
|
};
|
||||||
|
|
||||||
|
var stats = StatsCalculator.ComputeDeterminism("s", "i", "canonical", runs);
|
||||||
|
Assert.Equal(1, stats.DistinctHashes);
|
||||||
|
Assert.Equal(2, stats.TotalRuns);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void CvssDelta_Computes_Mean_And_StdDev()
|
||||||
|
{
|
||||||
|
var refRuns = new List<ScanRun>
|
||||||
|
{
|
||||||
|
new("ref", "i", 0, "canonical", "h1", new List<NormalizedFinding>
|
||||||
|
{
|
||||||
|
new("pkg1","CVE-1","5.0","HIGH"),
|
||||||
|
new("pkg2","CVE-2","7.0","HIGH")
|
||||||
|
})
|
||||||
|
};
|
||||||
|
|
||||||
|
var scannerRuns = new List<ScanRun>
|
||||||
|
{
|
||||||
|
new("scan", "i", 0, "canonical", "h2", new List<NormalizedFinding>
|
||||||
|
{
|
||||||
|
new("pkg1","CVE-1","6.0","HIGH"), // +1
|
||||||
|
new("pkg2","CVE-2","8.0","HIGH") // +1
|
||||||
|
})
|
||||||
|
};
|
||||||
|
|
||||||
|
var stats = StatsCalculator.ComputeCvssDeltas("scan", "i", scannerRuns, refRuns);
|
||||||
|
Assert.Equal(1.0, stats.MeanDelta, 3);
|
||||||
|
Assert.Equal(0.0, stats.StdDevDelta, 3);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Run tests:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet test
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. How you’ll use this in practice
|
||||||
|
|
||||||
|
1. **Drop SBOM & VEX files** into `Inputs/Sboms` and `Inputs/Vex`.
|
||||||
|
|
||||||
|
2. **Install Docker** and make sure CLI works.
|
||||||
|
|
||||||
|
3. Pull scanner images (optional but nice):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker pull anchore/grype:v0.79.0
|
||||||
|
docker pull aquasec/trivy:0.55.0
|
||||||
|
docker pull stellaops/scanner:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
4. `cd ScannerBench.Runner` and run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Inspect console output:
|
||||||
|
|
||||||
|
* For each scanner & SBOM:
|
||||||
|
|
||||||
|
* Determinism: `distinct hashes / total runs` (expect 1 / N).
|
||||||
|
* Order‑invariance: compare canonical vs shuffled determinism.
|
||||||
|
* CVSS deltas vs Stella: look at standard deviation (lower = more aligned).
|
||||||
|
|
||||||
|
6. Optional: serialize `allRuns` and metrics to `Results/*.json` and plot them in whatever you like.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
If you’d like, next step I can help you:
|
||||||
|
|
||||||
|
* tighten the JSON normalization against real scanner outputs, or
|
||||||
|
* add a small HTML/Blazor or minimal API endpoint that renders the stats as a web dashboard instead of console output.
|
||||||
File diff suppressed because it is too large
Load Diff
1044
docs/product-advisories/23-Nov-2025 - Stella Ops vs Competitors.md
Normal file
1044
docs/product-advisories/23-Nov-2025 - Stella Ops vs Competitors.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,830 @@
|
|||||||
|
Here’s a crisp idea I think you’ll like: **attested, offline‑verifiable call graphs** for binaries.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
### The gist
|
||||||
|
|
||||||
|
* **Goal:** Make binary reachability (who calls whom) something an auditor can replay **deterministically**, even air‑gapped.
|
||||||
|
* **How:**
|
||||||
|
|
||||||
|
1. Build the call graph for ELF/PE/Mach‑O.
|
||||||
|
2. **Seal each edge (caller → callee) as its own artifact** and sign it in a **DSSE** (in‑toto envelope).
|
||||||
|
3. Bundle a **reachability graph manifest** listing all edge‑artifacts + hashes of the inputs (binary, debug info, decompiler version, lattice/policy config).
|
||||||
|
4. Upload edge‑attestations to a **transparency log** (e.g., Rekor v2).
|
||||||
|
5. Anyone can later fetch/verifiy the envelopes and **replay the analysis identically** (same inputs ⇒ same graph).
|
||||||
|
|
||||||
|
### Why this matters
|
||||||
|
|
||||||
|
* **Deterministic audits:** “Prove this edge existed at analysis time.” No hand‑wavy “our tool said so last week.”
|
||||||
|
* **Granular trust:** You can quarantine or dispute **just one edge** without invalidating the whole graph.
|
||||||
|
* **Supply‑chain fit:** Edge‑artifacts compose nicely with SBOM/VEX; you can say “CVE‑123 is reachable via these signed edges.”
|
||||||
|
|
||||||
|
### Minimal vocabulary
|
||||||
|
|
||||||
|
* **DSSE:** A standard envelope that signs the *statement* (here: an edge) and its *subject* (binary, build‑ID, PURLs).
|
||||||
|
* **Rekor (v2):** An append‑only public log for attestations. Inclusion proofs = tamper‑evidence.
|
||||||
|
* **Reachability graph:** Nodes are functions/symbols; edges are possible calls; roots are entrypoints (exports, handlers, ctors, etc.).
|
||||||
|
|
||||||
|
### What “best‑in‑class” looks like in Stella Ops
|
||||||
|
|
||||||
|
* **Edge schema (per envelope):**
|
||||||
|
|
||||||
|
* `subject`: binary digest + **build‑id**, container image digest (if relevant)
|
||||||
|
* `caller`: {binary‑offset | symbol | demangled | PURL, version}
|
||||||
|
* `callee`: same structure
|
||||||
|
* `reason`: static pattern (PLT/JMP, thunk), **init_array/ctors**, EH frames, import table, or **dynamic witness** (trace sample ID)
|
||||||
|
* `provenance`: tool name + version, pipeline run ID, OS, container digest
|
||||||
|
* `policy-hash`: hash of lattice/policy/rules used
|
||||||
|
* `evidence`: (optional) byte slice, CFG snippet hash, or trace excerpt hash
|
||||||
|
* **Graph manifest (DSSE too):**
|
||||||
|
|
||||||
|
* list of edge envelope digests, **roots set**, toolchain hashes, input feeds, **PURL map** (component/function ↔ PURL).
|
||||||
|
* **Verification flow:**
|
||||||
|
|
||||||
|
* Verify envelopes → verify Rekor inclusion → recompute edges from inputs (or check cached proofs) → compare manifest hash.
|
||||||
|
* **Roots you must include:** exports, syscalls, signal handlers, **.init_array / .ctors**, TLS callbacks, exception trampolines, plugin entrypoints, registered callbacks.
|
||||||
|
|
||||||
|
### Quick implementation plan (C#/.NET 10, fits your stack)
|
||||||
|
|
||||||
|
1. **Parsers**: ELF/PE/Mach‑O loaders (SymbolTable, DynSym, Reloc/Relr, Import/Export, Sections, Build‑ID), plus DWARF/PDB stubs when present.
|
||||||
|
2. **Normalizer**: stable symbol IDs (image base + RVA) and **PURL resolver** (package → function namespace).
|
||||||
|
3. **Edge extractors** (pluggable):
|
||||||
|
|
||||||
|
* Static: import thunks, PLT/JMP, reloc‑targets, vtable patterns, .init_array, EH tables, jump tables.
|
||||||
|
* Dynamic (optional): eBPF/ETW/Perf trace ingester → produce **witness edges**.
|
||||||
|
4. **Edge attestation**: one DSSE per edge + signer (FIPS/SM/GOST/EIDAS as needed).
|
||||||
|
5. **Manifest builder**: emit graph manifest + policy/lattice hash; store in your **Ledger**.
|
||||||
|
6. **Transparency client**: Rekor v2 submit/query; cache inclusion proofs for offline bundles.
|
||||||
|
7. **Verifier**: deterministic replay runner; diff engine (edge‑set, roots, policy changes).
|
||||||
|
8. **UI**: “Edge provenance” panel; click an edge → see DSSE, Rekor proof, extraction reason.
|
||||||
|
|
||||||
|
### Practical guardrails
|
||||||
|
|
||||||
|
* **Idempotence:** Edge IDs = `hash(callerID, calleeID, reason, tool-version)`. Re‑runs don’t duplicate.
|
||||||
|
* **Explainability:** Every edge must say *why it exists* (pattern or witness).
|
||||||
|
* **Stripped binaries:** fall back to pattern heuristics + patch oracles; mark edges **probabilistic** with separate attestation type.
|
||||||
|
* **Hybrid truth:** Keep static and dynamic edges distinct; policies can require both for “reachable”.
|
||||||
|
|
||||||
|
### How this helps your day‑to‑day
|
||||||
|
|
||||||
|
* **Compliance**: Ship an SBOM/VEX plus a **proof pack**; auditors can verify offline.
|
||||||
|
* **Triage**: For a CVE, show **the exact signed path** from entrypoint → vulnerable function; suppresses noisy “maybe‑reachable” claims.
|
||||||
|
* **Vendor claims**: Accept third‑party edges only if they come with DSSE + Rekor inclusion.
|
||||||
|
|
||||||
|
If you want, I can draft the **DSSE edge schema (JSON)**, the **manifest format**, and the **.NET 10 interfaces** (`IEdgeExtractor`, `IAttestor`, `IReplayer`, `ITransparencyClient`) so your mid‑level dev can start coding today.
|
||||||
|
Here’s a concrete, “give this to a mid‑level .NET dev” implementation plan for the attested, offline‑verifiable call graph.
|
||||||
|
|
||||||
|
I’ll assume:
|
||||||
|
|
||||||
|
* Recent .NET (your “.NET 10”)
|
||||||
|
* C#
|
||||||
|
* You can add NuGet packages
|
||||||
|
* You already have (or will have) an “Authority Signer” for DSSE signatures (file key, KMS, etc.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. Solution layout (what projects to create)
|
||||||
|
|
||||||
|
Create a new solution, e.g. `StellaOps.CallGraph.sln` with:
|
||||||
|
|
||||||
|
1. **`StellaOps.CallGraph.Core`** (Class Library)
|
||||||
|
|
||||||
|
* Domain models (functions, edges, manifests)
|
||||||
|
* Interfaces (`IBinaryParser`, `IEdgeExtractor`, `IAttestor`, `IRekorClient`, etc.)
|
||||||
|
* DSSE envelope and helpers
|
||||||
|
|
||||||
|
2. **`StellaOps.CallGraph.BinaryParsers`** (Class Library)
|
||||||
|
|
||||||
|
* Implementations of `IBinaryParser` for:
|
||||||
|
|
||||||
|
* **PE/.NET assemblies** using `System.Reflection.Metadata` / `PEReader`([NuGet][1])
|
||||||
|
* Optionally native PE / ELF using `Microsoft.Binary.Parsers`([NuGet][2]) or `ELFSharp`([NuGet][3])
|
||||||
|
|
||||||
|
3. **`StellaOps.CallGraph.EdgeExtraction`** (Class Library)
|
||||||
|
|
||||||
|
* Call‑graph builder / edge extractors (import table, IL call instructions, .ctors, etc.)
|
||||||
|
|
||||||
|
4. **`StellaOps.CallGraph.Attestation`** (Class Library)
|
||||||
|
|
||||||
|
* DSSE helpers
|
||||||
|
* Attestation logic for edges + graph manifest
|
||||||
|
* Transparency log (Rekor) client
|
||||||
|
|
||||||
|
5. **`StellaOps.CallGraph.Cli`** (Console app)
|
||||||
|
|
||||||
|
* Developer entrypoint: `callgraph analyze <binary>`
|
||||||
|
* Outputs:
|
||||||
|
|
||||||
|
* Edge DSSE envelopes (one per edge, or batched)
|
||||||
|
* Graph manifest DSSE
|
||||||
|
* Human‑readable summary
|
||||||
|
|
||||||
|
6. **`StellaOps.CallGraph.Tests`** (xUnit / NUnit)
|
||||||
|
|
||||||
|
* Unit tests per layer
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Define the core domain (Core project)
|
||||||
|
|
||||||
|
### 1.1 Records and enums
|
||||||
|
|
||||||
|
Create these in `StellaOps.CallGraph.Core`:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed record BinaryIdentity(
|
||||||
|
string LogicalId, // e.g. build-id or image digest
|
||||||
|
string Path, // local path used during analysis
|
||||||
|
string? BuildId,
|
||||||
|
string? ImageDigest, // e.g. OCI digest
|
||||||
|
IReadOnlyDictionary<string, string> Digests // sha256, sha512, etc.
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record FunctionRef(
|
||||||
|
string BinaryLogicalId, // link to BinaryIdentity.LogicalId
|
||||||
|
ulong Rva, // Relative virtual address (for native) or metadata token for managed
|
||||||
|
string? SymbolName, // raw symbol if available
|
||||||
|
string? DisplayName, // demangled, user-facing
|
||||||
|
string? Purl // optional: pkg/function mapping
|
||||||
|
);
|
||||||
|
|
||||||
|
public enum EdgeReasonKind
|
||||||
|
{
|
||||||
|
ImportTable,
|
||||||
|
StaticCall, // direct call instruction
|
||||||
|
VirtualDispatch, // via vtable / callvirt
|
||||||
|
InitArrayOrCtor,
|
||||||
|
ExceptionHandler,
|
||||||
|
DynamicWitness // from traces
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed record EdgeReason(
|
||||||
|
EdgeReasonKind Kind,
|
||||||
|
string Detail // e.g. ".text: call 0x401234", "import: kernel32!CreateFileW"
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record ReachabilityEdge(
|
||||||
|
FunctionRef Caller,
|
||||||
|
FunctionRef Callee,
|
||||||
|
EdgeReason Reason,
|
||||||
|
string ToolVersion,
|
||||||
|
string PolicyHash, // hash of lattice/policy
|
||||||
|
string EvidenceHash // hash of raw evidence blob (CFG snippet, trace, etc.)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
Graph manifest:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed record CallGraphManifest(
|
||||||
|
string SchemaVersion,
|
||||||
|
BinaryIdentity Binary,
|
||||||
|
IReadOnlyList<FunctionRef> Roots,
|
||||||
|
IReadOnlyList<string> EdgeEnvelopeDigests, // sha256 of DSSE envelopes
|
||||||
|
string PolicyHash,
|
||||||
|
IReadOnlyDictionary<string, string> ToolMetadata
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 Core interfaces
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public interface IBinaryParser
|
||||||
|
{
|
||||||
|
BinaryIdentity Identify(string path);
|
||||||
|
IReadOnlyList<FunctionRef> GetFunctions(BinaryIdentity binary);
|
||||||
|
IReadOnlyList<FunctionRef> GetRoots(BinaryIdentity binary); // exports, entrypoint, handlers, etc.
|
||||||
|
BinaryCodeRegion GetCodeRegion(BinaryIdentity binary); // raw bytes + mappings, see below
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed record BinaryCodeRegion(
|
||||||
|
byte[] Bytes,
|
||||||
|
ulong ImageBase,
|
||||||
|
IReadOnlyList<SectionInfo> Sections
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record SectionInfo(
|
||||||
|
string Name,
|
||||||
|
ulong Rva,
|
||||||
|
uint Size
|
||||||
|
);
|
||||||
|
|
||||||
|
public interface IEdgeExtractor
|
||||||
|
{
|
||||||
|
IReadOnlyList<ReachabilityEdge> Extract(
|
||||||
|
BinaryIdentity binary,
|
||||||
|
IReadOnlyList<FunctionRef> functions,
|
||||||
|
BinaryCodeRegion code);
|
||||||
|
}
|
||||||
|
|
||||||
|
public interface IAttestor
|
||||||
|
{
|
||||||
|
Task<DsseEnvelope> SignEdgeAsync(
|
||||||
|
ReachabilityEdge edge,
|
||||||
|
BinaryIdentity binary,
|
||||||
|
CancellationToken ct = default);
|
||||||
|
|
||||||
|
Task<DsseEnvelope> SignManifestAsync(
|
||||||
|
CallGraphManifest manifest,
|
||||||
|
CancellationToken ct = default);
|
||||||
|
}
|
||||||
|
|
||||||
|
public interface IRekorClient
|
||||||
|
{
|
||||||
|
Task<RekorEntryRef> UploadAsync(DsseEnvelope envelope, CancellationToken ct = default);
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed record RekorEntryRef(string LogId, long Index, string Uuid);
|
||||||
|
```
|
||||||
|
|
||||||
|
(We’ll define `DsseEnvelope` in section 3.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Implement minimal PE parser (BinaryParsers project)
|
||||||
|
|
||||||
|
Start with **PE/.NET** only; expand later.
|
||||||
|
|
||||||
|
### 2.1 Add NuGet packages
|
||||||
|
|
||||||
|
* `System.Reflection.Metadata` (if you’re not already on a shared framework that has it)([NuGet][1])
|
||||||
|
* Optionally `Microsoft.Binary.Parsers` for native PE & ELF; it already knows how to parse PE headers and ELF.([NuGet][2])
|
||||||
|
|
||||||
|
### 2.2 Implement `PeBinaryParser` (managed assemblies)
|
||||||
|
|
||||||
|
In `StellaOps.CallGraph.BinaryParsers`:
|
||||||
|
|
||||||
|
* `BinaryIdentity Identify(string path)`
|
||||||
|
|
||||||
|
* Open file, compute SHA‑256 (streaming).
|
||||||
|
* Use `PEReader` and `MetadataReader` to pull:
|
||||||
|
|
||||||
|
* MVID (`ModuleDefinition`).
|
||||||
|
* Assembly name, version.
|
||||||
|
* Derive `LogicalId`, e.g. `"dotnet:<AssemblyName>/<Mvid>"`.
|
||||||
|
|
||||||
|
* `IReadOnlyList<FunctionRef> GetFunctions(...)`
|
||||||
|
|
||||||
|
* Use `PEReader` → `GetMetadataReader()` to enumerate methods:
|
||||||
|
|
||||||
|
* `reader.TypeDefinitions` → methods in each type.
|
||||||
|
* For each `MethodDefinition`, compute:
|
||||||
|
|
||||||
|
* `BinaryLogicalId = binary.LogicalId`
|
||||||
|
* `Rva = methodDef.RelativeVirtualAddress`
|
||||||
|
* `SymbolName = reader.GetString(methodDef.Name)`
|
||||||
|
* `DisplayName = typeFullName + "::" + methodName + signature`
|
||||||
|
* `Purl` optional mapping (you can fill later from SBOM).
|
||||||
|
|
||||||
|
* `IReadOnlyList<FunctionRef> GetRoots(...)`
|
||||||
|
|
||||||
|
* Roots for .NET:
|
||||||
|
|
||||||
|
* `Main` methods in entry assembly.
|
||||||
|
* Public exported API if you want (public methods in public types).
|
||||||
|
* Static constructors (.cctor) for public types (init roots).
|
||||||
|
* Keep it simple for v1: treat `Main` as only root.
|
||||||
|
|
||||||
|
* `BinaryCodeRegion GetCodeRegion(...)`
|
||||||
|
|
||||||
|
* For managed assemblies, you only need IL for now:
|
||||||
|
|
||||||
|
* Use `PEReader.GetMethodBody(rva)` to get `MethodBodyBlock`.([Microsoft Learn][4])
|
||||||
|
* For v1, you can assemble per‑method IL as you go in the extractor instead of pre‑building a whole region.
|
||||||
|
|
||||||
|
Implementation trick: have `PeBinaryParser` expose a helper:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public MethodBodyBlock? TryGetMethodBody(BinaryIdentity binary, uint rva);
|
||||||
|
```
|
||||||
|
|
||||||
|
You’ll pass this down to the edge extractor.
|
||||||
|
|
||||||
|
### 2.3 (Optional) native PE/ELF
|
||||||
|
|
||||||
|
Once managed assemblies work:
|
||||||
|
|
||||||
|
* Add `Microsoft.Binary.Parsers` for PE + ELF.([NuGet][2])
|
||||||
|
* Or `ELFSharp` if you prefer.([NuGet][3])
|
||||||
|
|
||||||
|
You can then:
|
||||||
|
|
||||||
|
* Parse import table → edges from “import stub” → imported function.
|
||||||
|
* Parse export table → roots (exports).
|
||||||
|
* Parse `.pdata`, `.xdata` → exception handlers.
|
||||||
|
* Parse `.init_array` (ELF) / TLS callbacks, C runtime init functions.
|
||||||
|
|
||||||
|
For an “average dev” first iteration, you can **skip native** and get a lot of value from .NET assemblies only.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. DSSE attestation primitives (Attestation project)
|
||||||
|
|
||||||
|
You already use DSSE elsewhere, but here’s a self‑contained minimal version.
|
||||||
|
|
||||||
|
### 3.1 Envelope models
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed record DsseSignature(
|
||||||
|
string KeyId,
|
||||||
|
string Sig // base64 signature
|
||||||
|
);
|
||||||
|
|
||||||
|
public sealed record DsseEnvelope(
|
||||||
|
string PayloadType, // e.g. "application/vnd.stella.call-edge+json"
|
||||||
|
string Payload, // base64-encoded JSON statement
|
||||||
|
IReadOnlyList<DsseSignature> Signatures
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
Statement for a **single edge**:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed record EdgeStatement(
|
||||||
|
string _type, // e.g. "https://stella.ops/Statement/CallEdge/v1"
|
||||||
|
object subject, // Binary info + maybe PURLs
|
||||||
|
ReachabilityEdge edge
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
You can loosely follow the DSSE / in‑toto style: Google’s Grafeas `Envelope` type also matches DSSE’s `envelope.proto`.([Google Cloud][5])
|
||||||
|
|
||||||
|
### 3.2 Pre‑authentication encoding (PAE)
|
||||||
|
|
||||||
|
Implement DSSE PAE once:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public static class Dsse
|
||||||
|
{
|
||||||
|
public static byte[] PreAuthEncode(string payloadType, byte[] payload)
|
||||||
|
{
|
||||||
|
static byte[] Cat(params byte[][] parts)
|
||||||
|
{
|
||||||
|
var total = parts.Sum(p => p.Length);
|
||||||
|
var buf = new byte[total];
|
||||||
|
var offset = 0;
|
||||||
|
foreach (var part in parts)
|
||||||
|
{
|
||||||
|
Buffer.BlockCopy(part, 0, buf, offset, part.Length);
|
||||||
|
offset += part.Length;
|
||||||
|
}
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
static byte[] Utf8(string s) => Encoding.UTF8.GetBytes(s);
|
||||||
|
|
||||||
|
var header = Utf8("DSSEv1");
|
||||||
|
var pt = Utf8(payloadType);
|
||||||
|
var lenPt = Utf8(pt.Length.ToString(CultureInfo.InvariantCulture));
|
||||||
|
var lenPayload = Utf8(payload.Length.ToString(CultureInfo.InvariantCulture));
|
||||||
|
var space = Utf8(" ");
|
||||||
|
|
||||||
|
return Cat(header, space, lenPt, space, pt, space, lenPayload, space, payload);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.3 Implement `IAttestor`
|
||||||
|
|
||||||
|
Assume you already have some `IAuthoritySigner` that can sign arbitrary byte arrays (Ed25519, RSA, etc.).
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed class DsseAttestor : IAttestor
|
||||||
|
{
|
||||||
|
private readonly IAuthoritySigner _signer;
|
||||||
|
private readonly JsonSerializerOptions _jsonOptions = new()
|
||||||
|
{
|
||||||
|
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
|
||||||
|
};
|
||||||
|
|
||||||
|
public DsseAttestor(IAuthoritySigner signer) => _signer = signer;
|
||||||
|
|
||||||
|
public async Task<DsseEnvelope> SignEdgeAsync(
|
||||||
|
ReachabilityEdge edge,
|
||||||
|
BinaryIdentity binary,
|
||||||
|
CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
var stmt = new EdgeStatement(
|
||||||
|
_type: "https://stella.ops/Statement/CallEdge/v1",
|
||||||
|
subject: new
|
||||||
|
{
|
||||||
|
type = "file",
|
||||||
|
name = binary.Path,
|
||||||
|
digest = binary.Digests
|
||||||
|
},
|
||||||
|
edge: edge
|
||||||
|
);
|
||||||
|
|
||||||
|
return await SignStatementAsync(
|
||||||
|
stmt,
|
||||||
|
payloadType: "application/vnd.stella.call-edge+json",
|
||||||
|
ct);
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task<DsseEnvelope> SignManifestAsync(
|
||||||
|
CallGraphManifest manifest,
|
||||||
|
CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
var stmt = new
|
||||||
|
{
|
||||||
|
_type = "https://stella.ops/Statement/CallGraphManifest/v1",
|
||||||
|
subject = new
|
||||||
|
{
|
||||||
|
type = "file",
|
||||||
|
name = manifest.Binary.Path,
|
||||||
|
digest = manifest.Binary.Digests
|
||||||
|
},
|
||||||
|
manifest
|
||||||
|
};
|
||||||
|
|
||||||
|
return await SignStatementAsync(
|
||||||
|
stmt,
|
||||||
|
payloadType: "application/vnd.stella.call-manifest+json",
|
||||||
|
ct);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task<DsseEnvelope> SignStatementAsync(
|
||||||
|
object statement,
|
||||||
|
string payloadType,
|
||||||
|
CancellationToken ct)
|
||||||
|
{
|
||||||
|
var payloadBytes = JsonSerializer.SerializeToUtf8Bytes(statement, _jsonOptions);
|
||||||
|
var pae = Dsse.PreAuthEncode(payloadType, payloadBytes);
|
||||||
|
|
||||||
|
var signatureBytes = await _signer.SignAsync(pae, ct).ConfigureAwait(false);
|
||||||
|
var keyId = await _signer.GetKeyIdAsync(ct).ConfigureAwait(false);
|
||||||
|
|
||||||
|
return new DsseEnvelope(
|
||||||
|
PayloadType: payloadType,
|
||||||
|
Payload: Convert.ToBase64String(payloadBytes),
|
||||||
|
Signatures: new[]
|
||||||
|
{
|
||||||
|
new DsseSignature(keyId, Convert.ToBase64String(signatureBytes))
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You can plug in:
|
||||||
|
|
||||||
|
* `IAuthoritySigner` using `System.Security.Cryptography.Ed25519` on .NET (or BouncyCastle) for signatures.([Stack Overflow][6])
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Edge extraction (EdgeExtraction project)
|
||||||
|
|
||||||
|
### 4.1 Choose strategy per binary type
|
||||||
|
|
||||||
|
For **managed .NET assemblies** the easiest route is to use `Mono.Cecil` to read IL opcodes.([NuGet][7])
|
||||||
|
|
||||||
|
Add package: `Mono.Cecil`.
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed class ManagedIlEdgeExtractor : IEdgeExtractor
|
||||||
|
{
|
||||||
|
public IReadOnlyList<ReachabilityEdge> Extract(
|
||||||
|
BinaryIdentity binary,
|
||||||
|
IReadOnlyList<FunctionRef> functions,
|
||||||
|
BinaryCodeRegion code)
|
||||||
|
{
|
||||||
|
// For managed we won't use BinaryCodeRegion; we’ll re-open file with Cecil.
|
||||||
|
var result = new List<ReachabilityEdge>();
|
||||||
|
var filePath = binary.Path;
|
||||||
|
|
||||||
|
var module = ModuleDefinition.ReadModule(filePath, new ReaderParameters
|
||||||
|
{
|
||||||
|
ReadSymbols = false
|
||||||
|
});
|
||||||
|
|
||||||
|
foreach (var type in module.Types)
|
||||||
|
foreach (var method in type.Methods.Where(m => m.HasBody))
|
||||||
|
{
|
||||||
|
var callerRef = ToFunctionRef(binary, method);
|
||||||
|
|
||||||
|
foreach (var instr in method.Body.Instructions)
|
||||||
|
{
|
||||||
|
if (instr.OpCode.FlowControl != FlowControl.Call)
|
||||||
|
continue;
|
||||||
|
|
||||||
|
if (instr.Operand is not MethodReference calleeMethod)
|
||||||
|
continue;
|
||||||
|
|
||||||
|
var calleeRef = ToFunctionRef(binary, calleeMethod);
|
||||||
|
|
||||||
|
var edge = new ReachabilityEdge(
|
||||||
|
Caller: callerRef,
|
||||||
|
Callee: calleeRef,
|
||||||
|
Reason: new EdgeReason(
|
||||||
|
EdgeReasonKind.StaticCall,
|
||||||
|
Detail: $"IL {instr.OpCode} {calleeMethod.FullName}"
|
||||||
|
),
|
||||||
|
ToolVersion: "stella-callgraph/0.1.0",
|
||||||
|
PolicyHash: "TODO",
|
||||||
|
EvidenceHash: "TODO" // later: hash of snippet
|
||||||
|
);
|
||||||
|
|
||||||
|
result.Add(edge);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static FunctionRef ToFunctionRef(BinaryIdentity binary, MethodReference method)
|
||||||
|
{
|
||||||
|
var displayName = $"{method.DeclaringType.FullName}::{method.Name}";
|
||||||
|
return new FunctionRef(
|
||||||
|
BinaryLogicalId: binary.LogicalId,
|
||||||
|
Rva: (ulong)method.MetadataToken.ToInt32(),
|
||||||
|
SymbolName: method.FullName,
|
||||||
|
DisplayName: displayName,
|
||||||
|
Purl: null
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Later, you can add:
|
||||||
|
|
||||||
|
* Import table edges (`EdgeReasonKind.ImportTable`).
|
||||||
|
* Virtual dispatch edges, heuristics, etc.
|
||||||
|
* Dynamic edges from trace logs (`EdgeReasonKind.DynamicWitness`).
|
||||||
|
|
||||||
|
### 4.2 Call‑graph builder
|
||||||
|
|
||||||
|
Add a thin orchestration service:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed class CallGraphBuilder
|
||||||
|
{
|
||||||
|
private readonly IBinaryParser _parser;
|
||||||
|
private readonly IReadOnlyList<IEdgeExtractor> _extractors;
|
||||||
|
|
||||||
|
public CallGraphBuilder(
|
||||||
|
IBinaryParser parser,
|
||||||
|
IEnumerable<IEdgeExtractor> extractors)
|
||||||
|
{
|
||||||
|
_parser = parser;
|
||||||
|
_extractors = extractors.ToList();
|
||||||
|
}
|
||||||
|
|
||||||
|
public (BinaryIdentity binary,
|
||||||
|
IReadOnlyList<FunctionRef> functions,
|
||||||
|
IReadOnlyList<FunctionRef> roots,
|
||||||
|
IReadOnlyList<ReachabilityEdge> edges) Build(string path)
|
||||||
|
{
|
||||||
|
var binary = _parser.Identify(path);
|
||||||
|
var functions = _parser.GetFunctions(binary);
|
||||||
|
var roots = _parser.GetRoots(binary);
|
||||||
|
|
||||||
|
// Optionally, pack code region if needed
|
||||||
|
var code = new BinaryCodeRegion(Array.Empty<byte>(), 0, Array.Empty<SectionInfo>());
|
||||||
|
|
||||||
|
var edges = _extractors
|
||||||
|
.SelectMany(e => e.Extract(binary, functions, code))
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
return (binary, functions, roots, edges);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Edge→DSSE and manifest→DSSE wiring
|
||||||
|
|
||||||
|
In `StellaOps.CallGraph.Attestation`, create a coordinator:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed class CallGraphAttestationService
|
||||||
|
{
|
||||||
|
private readonly CallGraphBuilder _builder;
|
||||||
|
private readonly IAttestor _attestor;
|
||||||
|
private readonly IRekorClient _rekor;
|
||||||
|
|
||||||
|
public CallGraphAttestationService(
|
||||||
|
CallGraphBuilder builder,
|
||||||
|
IAttestor attestor,
|
||||||
|
IRekorClient rekor)
|
||||||
|
{
|
||||||
|
_builder = builder;
|
||||||
|
_attestor = attestor;
|
||||||
|
_rekor = rekor;
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task<CallGraphAttestationResult> AnalyzeAndAttestAsync(
|
||||||
|
string path,
|
||||||
|
CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
var (binary, functions, roots, edges) = _builder.Build(path);
|
||||||
|
|
||||||
|
// 1) Sign each edge
|
||||||
|
var edgeEnvelopes = new List<DsseEnvelope>();
|
||||||
|
foreach (var edge in edges)
|
||||||
|
{
|
||||||
|
var env = await _attestor.SignEdgeAsync(edge, binary, ct);
|
||||||
|
edgeEnvelopes.Add(env);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2) Compute digests for manifest
|
||||||
|
var edgeEnvelopeDigests = edgeEnvelopes
|
||||||
|
.Select(e => Crypto.HashSha256(JsonSerializer.SerializeToUtf8Bytes(e)))
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
var manifest = new CallGraphManifest(
|
||||||
|
SchemaVersion: "1.0",
|
||||||
|
Binary: binary,
|
||||||
|
Roots: roots,
|
||||||
|
EdgeEnvelopeDigests: edgeEnvelopeDigests,
|
||||||
|
PolicyHash: edges.FirstOrDefault()?.PolicyHash ?? "",
|
||||||
|
ToolMetadata: new Dictionary<string, string>
|
||||||
|
{
|
||||||
|
["builder"] = "stella-callgraph/0.1.0",
|
||||||
|
["created-at"] = DateTimeOffset.UtcNow.ToString("O")
|
||||||
|
});
|
||||||
|
|
||||||
|
var manifestEnvelope = await _attestor.SignManifestAsync(manifest, ct);
|
||||||
|
|
||||||
|
// 3) Publish DSSE envelopes to Rekor (if configured)
|
||||||
|
var rekorRefs = new List<RekorEntryRef>();
|
||||||
|
foreach (var env in edgeEnvelopes.Append(manifestEnvelope))
|
||||||
|
{
|
||||||
|
var entry = await _rekor.UploadAsync(env, ct);
|
||||||
|
rekorRefs.Add(entry);
|
||||||
|
}
|
||||||
|
|
||||||
|
return new CallGraphAttestationResult(
|
||||||
|
Manifest: manifest,
|
||||||
|
ManifestEnvelope: manifestEnvelope,
|
||||||
|
EdgeEnvelopes: edgeEnvelopes,
|
||||||
|
RekorEntries: rekorRefs);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed record CallGraphAttestationResult(
|
||||||
|
CallGraphManifest Manifest,
|
||||||
|
DsseEnvelope ManifestEnvelope,
|
||||||
|
IReadOnlyList<DsseEnvelope> EdgeEnvelopes,
|
||||||
|
IReadOnlyList<RekorEntryRef> RekorEntries);
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Rekor v2 client (transparency log)
|
||||||
|
|
||||||
|
Rekor is a REST‑based transparency log (part of Sigstore).([Sigstore][8])
|
||||||
|
|
||||||
|
For an average dev, keep it **simple**:
|
||||||
|
|
||||||
|
1. Add `HttpClient`‑based `RekorClient`:
|
||||||
|
|
||||||
|
* `UploadAsync(DsseEnvelope)`:
|
||||||
|
|
||||||
|
* POST to your Rekor server’s `/api/v1/log/entries` (v1 today; v2 is under active development, but the pattern is similar).
|
||||||
|
* Store returned `logID`, `logIndex`, `uuid` in `RekorEntryRef`.
|
||||||
|
|
||||||
|
2. For offline replay you’ll want to store:
|
||||||
|
|
||||||
|
* The DSSE envelopes.
|
||||||
|
* Rekor entry references (and ideally inclusion proofs, but that can come later).
|
||||||
|
|
||||||
|
You don’t need to fully implement Merkle tree verification in v1; you can add that when you harden the verifier.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. CLI for developers (Cli project)
|
||||||
|
|
||||||
|
A simple console app gives you fast feedback:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
stella-callgraph analyze myapp.dll \
|
||||||
|
--output-dir artifacts/callgraph
|
||||||
|
```
|
||||||
|
|
||||||
|
Implementation sketch:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
static async Task<int> Main(string[] args)
|
||||||
|
{
|
||||||
|
var input = args[1]; // TODO: proper parser
|
||||||
|
|
||||||
|
var services = Bootstrap(); // DI container
|
||||||
|
|
||||||
|
var svc = services.GetRequiredService<CallGraphAttestationService>();
|
||||||
|
var result = await svc.AnalyzeAndAttestAsync(input);
|
||||||
|
|
||||||
|
// Write DSSE envelopes & manifest as JSON files
|
||||||
|
var outDir = Path.Combine("artifacts", "callgraph");
|
||||||
|
Directory.CreateDirectory(outDir);
|
||||||
|
|
||||||
|
await File.WriteAllTextAsync(
|
||||||
|
Path.Combine(outDir, "manifest.dsse.json"),
|
||||||
|
JsonSerializer.Serialize(result.ManifestEnvelope, new JsonSerializerOptions { WriteIndented = true }));
|
||||||
|
|
||||||
|
for (var i = 0; i < result.EdgeEnvelopes.Count; i++)
|
||||||
|
{
|
||||||
|
var path = Path.Combine(outDir, $"edge-{i:D6}.dsse.json");
|
||||||
|
await File.WriteAllTextAsync(path,
|
||||||
|
JsonSerializer.Serialize(result.EdgeEnvelopes[i], new JsonSerializerOptions { WriteIndented = true }));
|
||||||
|
}
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Verifier (same libraries, different flow)
|
||||||
|
|
||||||
|
Later (or in parallel), add a **verification** mode:
|
||||||
|
|
||||||
|
1. Inputs:
|
||||||
|
|
||||||
|
* Binary file.
|
||||||
|
* Manifest DSSE file.
|
||||||
|
* Edge DSSE files.
|
||||||
|
* (Optionally) Rekor log inclusion proof bundle.
|
||||||
|
|
||||||
|
2. Steps (same dev can implement):
|
||||||
|
|
||||||
|
* Verify DSSE signatures for manifest and edges (using `IAuthoritySigner.VerifyAsync`).
|
||||||
|
* Check:
|
||||||
|
|
||||||
|
* Manifest’s binary digest matches the current file.
|
||||||
|
* Manifest’s edge‑envelope digests match hashes of the provided DSSE edge files.
|
||||||
|
* Rebuild call graph using the same tool & policy version and diff against attested edges:
|
||||||
|
|
||||||
|
* For deterministic replay, their differences should be zero.
|
||||||
|
* Optionally:
|
||||||
|
|
||||||
|
* Ask Rekor for current log info and verify inclusion proof (advanced).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Order of work for a mid‑level .NET dev
|
||||||
|
|
||||||
|
If you hand this as a sequence of tasks:
|
||||||
|
|
||||||
|
1. **Core models & interfaces**
|
||||||
|
|
||||||
|
* Add domain records (`BinaryIdentity`, `FunctionRef`, `ReachabilityEdge`, `CallGraphManifest`).
|
||||||
|
* Add `IBinaryParser`, `IEdgeExtractor`, `IAttestor`, `IRekorClient`.
|
||||||
|
|
||||||
|
2. **Managed PE parser**
|
||||||
|
|
||||||
|
* Implement `PeBinaryParser` using `System.Reflection.Metadata` (`PEReader`, `MetadataReader`).([NuGet][1])
|
||||||
|
* Return `BinaryIdentity`, a list of methods as `FunctionRef`, and roots (`Main`).
|
||||||
|
|
||||||
|
3. **IL edge extractor**
|
||||||
|
|
||||||
|
* Add `Mono.Cecil`.
|
||||||
|
* Implement `ManagedIlEdgeExtractor` that:
|
||||||
|
|
||||||
|
* Iterates methods and IL instructions.
|
||||||
|
* Emits edges for `call` and `callvirt`.
|
||||||
|
|
||||||
|
4. **CallGraphBuilder**
|
||||||
|
|
||||||
|
* Wire `.Build(path)` to use `PeBinaryParser` + `ManagedIlEdgeExtractor`.
|
||||||
|
|
||||||
|
5. **DSSE library**
|
||||||
|
|
||||||
|
* Add `DsseEnvelope`, `DsseSignature`, `Dsse.PreAuthEncode`.
|
||||||
|
* Implement `DsseAttestor` that wraps `ReachabilityEdge` and `CallGraphManifest` into DSSE envelopes using an `IAuthoritySigner`.
|
||||||
|
|
||||||
|
6. **Rekor client (stub, then real)**
|
||||||
|
|
||||||
|
* First: `DummyRekorClient` that just returns fake IDs.
|
||||||
|
* Then: `HttpRekorClient` that POSTs to your Rekor server.
|
||||||
|
|
||||||
|
7. **CallGraphAttestationService + CLI**
|
||||||
|
|
||||||
|
* Implement `CallGraphAttestationService`.
|
||||||
|
* CLI command to:
|
||||||
|
|
||||||
|
* Run analysis.
|
||||||
|
* Write DSSE files + a human readable summary.
|
||||||
|
|
||||||
|
8. **Verifier**
|
||||||
|
|
||||||
|
* Implement basic “offline verify” command:
|
||||||
|
|
||||||
|
* Verify DSSE signatures on manifest + edges.
|
||||||
|
* Verify manifest ↔ edge digest linkage.
|
||||||
|
* (Later) compare re‑analyzed graph with attested one.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
If you want, I can next:
|
||||||
|
|
||||||
|
* Propose the **exact JSON schema** for `EdgeStatement` and `CallGraphManifest` (with sample instances).
|
||||||
|
* Or help turn this into a **Jira/Linear ticket breakdown** ready for your team.
|
||||||
|
|
||||||
|
[1]: https://www.nuget.org/packages/system.reflection.metadata/?utm_source=chatgpt.com "System.Reflection.Metadata 10.0.0"
|
||||||
|
[2]: https://www.nuget.org/packages/Microsoft.Binary.Parsers?utm_source=chatgpt.com "Microsoft.Binary.Parsers 4.4.8"
|
||||||
|
[3]: https://www.nuget.org/packages/ELFSharp?utm_source=chatgpt.com "ELFSharp 2.17.3"
|
||||||
|
[4]: https://learn.microsoft.com/en-us/dotnet/api/system.reflection.portableexecutable.pereader?view=net-10.0&utm_source=chatgpt.com "PEReader Class (System.Reflection.PortableExecutable)"
|
||||||
|
[5]: https://cloud.google.com/dotnet/docs/reference/Grafeas.V1/latest/Grafeas.V1.Envelope?utm_source=chatgpt.com "Grafeas v1 API - Class Envelope (3.10.0) | .NET client library"
|
||||||
|
[6]: https://stackoverflow.com/questions/72152837/get-public-and-private-key-from-pem-ed25519-in-c-sharp?utm_source=chatgpt.com "Get public and private key from PEM ed25519 in C#"
|
||||||
|
[7]: https://www.nuget.org/packages/mono.cecil/?utm_source=chatgpt.com "Mono.Cecil 0.11.6"
|
||||||
|
[8]: https://docs.sigstore.dev/logging/overview/?utm_source=chatgpt.com "Rekor"
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user