# Stella Ops Bench Repository

> **Status:** Active · Last updated: 2025-12-13
> **Purpose:** Host reproducible VEX decisions, reachability evidence, and comparison data proving Stella Ops' signal quality vs. baseline scanners.

## Layout

```
bench/
  README.md                 # this file
  findings/                 # per CVE/product bundles
    CVE-YYYY-NNNNN/
      evidence/
        reachability.json   # richgraph-v1 excerpt
        sbom.cdx.json       # CycloneDX SBOM
      decision.openvex.json # OpenVEX decision
      decision.dsse.json    # DSSE envelope
      rekor.txt             # Rekor log index + inclusion proof
      metadata.json         # finding metadata (purl, CVE, version)
  tools/
    verify.sh               # DSSE + Rekor verifier (online)
    verify.py               # offline verifier
    compare.py              # baseline comparison script
    replay.sh               # runs reachability replay manifests
  results/
    summary.csv             # aggregated metrics
    runs/<date>/...         # raw outputs + replay manifests
  reachability-benchmark/   # reachability benchmark with JDK fixtures
```

## Related Documentation

| Document | Purpose |
|----------|---------|
| [VEX Evidence Playbook](../docs/benchmarks/vex-evidence-playbook.md) | Proof bundle schema, justification catalog, verification workflow |
| [Hybrid Attestation](../docs/reachability/hybrid-attestation.md) | Graph-level and edge-bundle DSSE decisions |
| [Function-Level Evidence](../docs/reachability/function-level-evidence.md) | Cross-module evidence chain guide |
| [Deterministic Replay](../docs/replay/DETERMINISTIC_REPLAY.md) | Replay manifest specification |

## Verification Workflows

### Quick Verification (Online)

```bash
# Verify a VEX proof bundle with DSSE and Rekor
./tools/verify.sh findings/CVE-2021-44228/decision.dsse.json

# Output:
# ✓ DSSE signature valid
# ✓ Rekor inclusion verified (log index: 12345678)
# ✓ Evidence hashes match
# ✓ Justification catalog membership confirmed
```

### Offline Verification

```bash
# Verify without network access
python tools/verify.py \
  --bundle findings/CVE-2021-44228/decision.dsse.json \
  --cas-root ./findings/CVE-2021-44228/evidence/ \
  --catalog ../docs/benchmarks/vex-justifications.catalog.json

# Or use the VEX proof bundle verifier
python ../scripts/vex/verify_proof_bundle.py \
  --bundle ../tests/Vex/ProofBundles/sample-proof-bundle.json \
  --cas-root ../tests/Vex/ProofBundles/cas/
```

### Reachability Graph Verification

```bash
# Verify graph DSSE
stella graph verify --hash blake3:a1b2c3d4...

# Verify with edge bundles
stella graph verify --hash blake3:a1b2c3d4... --include-bundles

# Offline with local CAS
stella graph verify --hash blake3:a1b2c3d4... --cas-root ./offline-cas/
```

### Baseline Comparison

```bash
# Compare Stella Ops findings against baseline scanners
python tools/compare.py \
  --stellaops results/runs/2025-12-13/findings.json \
  --baseline results/baselines/trivy-latest.json \
  --output results/comparison-2025-12-13.csv

# Metrics generated:
# - True positives (reachability-confirmed)
# - False positives (unreachable code paths)
# - MTTD (mean time to detect)
# - Reproducibility score
```

## Artifact Contracts

All bench artifacts must comply with:

1. **VEX Proof Bundle Schema** (`docs/benchmarks/vex-evidence-playbook.schema.json`)
   - BLAKE3-256 primary hash, SHA-256 secondary
   - Canonical JSON with sorted keys
   - DSSE envelope with Rekor-ready digest

2. **Justification Catalog** (`docs/benchmarks/vex-justifications.catalog.json`)
   - VEX1-VEX10 justification codes
   - Required evidence types per justification
   - Expiry and re-evaluation rules

3. **Reachability Graph** (`docs/contracts/richgraph-v1.md`)
   - BLAKE3 graph_hash for content addressing
   - Deterministic node/edge ordering
   - SymbolID/EdgeID format compliance

## CI Integration

The bench directory is validated by:

- `.gitea/workflows/vex-proof-bundles.yml` - Verifies all proof bundles
- `.gitea/workflows/bench-determinism.yml` - Runs determinism benchmarks
- `.gitea/workflows/hybrid-attestation.yml` - Verifies graph/edge-bundle fixtures

## Contributing

1. Add new findings under `findings/CVE-YYYY-NNNNN/`
2. Include all required evidence artifacts
3. Generate DSSE envelope and Rekor proof
4. Update `results/summary.csv`
5. Run verification: `./tools/verify.sh findings/CVE-YYYY-NNNNN/decision.dsse.json`