Offline ready

Stella Ops · Reachability Benchmark

Deterministic, reproducible cases and scoring harness for reachability analysis tools.

Quick Start

Build cases deterministically: python tools/build/build_all.py --cases cases
Run your analyzer and emit submission.json in schemas/submission.schema.json format.
Score: tools/scorer/rb_score.py --truth benchmark/truth/<aggregate>.json --submission submission.json
Compare: tools/scorer/rb_compare.py --truth ... --submissions submission.json baselines/*/submission.json --output leaderboard.json

All tooling is offline-friendly; no network calls or external fonts.

Downloads

Cases: cases/
Schemas: schemas/
Truth: benchmark/truth/
Baselines: baselines/ (Semgrep, Stella, CodeQL)
CI script: ci/run-ci.sh

Determinism Checklist

Set SOURCE_DATE_EPOCH in builds.
Disable tool telemetry/version checks.
Sort cases and sinks before emission.
Keep outputs local; no registry or network pulls.

Leaderboard

Looking for leaderboard.json in this directory…