Files
StellaOps Bot e1262eb916 Add receipt input JSON and SHA256 hash for CVSS policy scoring tests
- Introduced a new JSON fixture `receipt-input.json` containing base, environmental, and threat metrics for CVSS scoring.
- Added corresponding SHA256 hash file `receipt-input.sha256` to ensure integrity of the JSON fixture.
2025-12-04 07:30:42 +02:00
..
up
2025-11-29 11:37:00 +02:00
up
2025-11-29 11:37:00 +02:00
up
2025-11-29 11:37:00 +02:00

Schemas

  • case.schema.yaml — case descriptor (language, sinks, deterministic build/test, environment, optional inline ground truth summary).
  • entrypoints.schema.yaml — declared entrypoints grouped by type (http, cli, scheduled, etc.).
  • truth.schema.json — ground-truth labels + evidence per sink (reachable/unreachable/unknown).
  • coverage.schema.json — coverage artifact emitted by oracle tests (lines covered or function-level coverage).
  • trace.schema.json — dynamic trace artifact (structured path or event log) proving reachability/unreachability.
  • submission.schema.json — submission format (tool/run metadata, per-sink predictions, explanations).
  • examples/ — minimal valid examples for each schema.
  • benchmark/schemas/benchmark-manifest.schema.json — kit manifest with hashed artifacts, resource limits, and optional DSSE signatures.

Validate quickly

# install minimal deps (offline-friendly, pinned)
python -m pip install -r ../tools/requirements.txt

# validate individual files
python ../tools/validate.py case examples/case.sample.yaml
python ../tools/validate.py entrypoints examples/entrypoints.sample.yaml
python ../tools/validate.py truth examples/truth.sample.json
python ../tools/validate.py coverage ../cases/js/unsafe-eval/outputs/coverage.json
python ../tools/validate.py trace ../cases/js/unsafe-eval/outputs/traces/traces.json
python ../tools/validate.py submission examples/submission.sample.json

# or validate everything in one shot
python ../tools/validate.py all examples