git.stella-ops.org/bench/reachability-benchmark/CONTRIBUTING.md

Contributing Guidelines

Determinism First

• Pin all dependencies (lockfiles, hashes, image digests).
• Set SOURCE_DATE_EPOCH and fixed seeds in build/test scripts.
• No network during builds/tests; use vendored toolchains.

Cases

• Place cases under cases/<lang>/<project>/.
• Include:
  • Dockerfile (deterministic build, no network after context stage)
  • Locked dependency file (e.g., package-lock.json, requirements.txt, pom.xml with exact versions)
  • Oracle tests proving reachability label
  • Coverage/artifact outputs for verification
  • README.md with case description, expected sink(s), build/run instructions
• Add SPDX license headers where required; attribute third-party code in THIRD_PARTY.md inside the case folder.

Schemas

• Keep schemas in schemas/; update scorer tests when schemas change.
• Provide JSON Schema drafts with $id and versioning.

Scorer

• tools/scorer: add unit tests for scoring math, schema validation, determinism (same input -> same output).
• No network, no telemetry.

Baselines

• Normalize outputs to submission schema.
• Document tool versions and invocation commands.

CI

• Workflows must be deterministic; avoid latest tags; prefer cached toolchains.

Submitting Changes

• Run relevant tests (rb-score tests, schema validation, case oracles) before opening a PR.
• Update docs/implplan/SPRINT_0513_0001_0001_public_reachability_benchmark.md statuses.
• Add Execution Log entry if scope or contracts change.