git.stella-ops.org/docs/claims-index.md

Stella Ops Claims Index

This document provides a verifiable index of competitive claims. Each claim is linked to evidence and can be verified using the provided commands.

Integrity: This index is updated automatically by the benchmark CI workflow. Manual edits require PR approval.

---

How to Verify Claims

# Verify a specific claim
stella benchmark verify <CLAIM_ID>

# Run full benchmark suite
stella benchmark run --competitors trivy,grype,syft

# Generate updated claims from latest benchmark
stella benchmark claims --output docs/claims-index.md

---

Claim Categories

Category	Prefix	Description
Determinism	DET-*	Reproducible, bit-identical outputs
Reachability	REACH-*	Call-path and exploitability analysis
Proofs	PROOF-*	Attestation and cryptographic evidence
Unknowns	UNK-*	Explicit uncertainty tracking
VEX	VEX-*	VEX handling and conflict resolution
Offline	OFFLINE-*	Air-gapped operation
SBOM	SBOM-*	SBOM fidelity and formats
Performance	PERF-*	Speed and scalability

---

Active Claims

Determinism

Claim ID	Claim	Evidence	Status
DET-001	Stella Ops produces bit-identical scan results across runs	bench/determinism/results.json	PENDING
DET-002	Score replay produces identical verdicts from manifest	bench/replay/results.json	PENDING
DET-003	SBOM generation is deterministic (stable ordering, canonical JSON)	bench/sbom/determinism.json	PENDING

Reachability

Claim ID	Claim	Evidence	Status
REACH-001	Stella Ops detects reachable vulnerabilities missed by Trivy	bench/competitors/trivy-comparison.json	PENDING
REACH-002	Stella Ops eliminates X% false positives via unreachable path detection	bench/reachability/fp-elimination.json	PENDING
REACH-003	Three-layer reachability (static + binary + runtime) provides higher confidence	bench/reachability/3layer-corpus.json	PENDING

Proofs & Attestations

Claim ID	Claim	Evidence	Status
PROOF-001	Every scan produces DSSE-signed attestation	bench/attestation/coverage.json	PENDING
PROOF-002	Score proofs enable third-party verification	bench/proofs/verification.json	PENDING
PROOF-003	Evidence chain links finding to source material	bench/proofs/chain-integrity.json	PENDING

Unknowns

Claim ID	Claim	Evidence	Status
UNK-001	Unknowns are first-class, not suppressed	bench/unknowns/tracking.json	PENDING
UNK-002	Unknowns have budgets and decay policies	bench/unknowns/budgets.json	PENDING
UNK-003	Unknowns escalate based on blast radius and age	bench/unknowns/escalation.json	PENDING

VEX Handling

Claim ID	Claim	Evidence	Status
VEX-001	Native VEX ingestion with formal reasoning	bench/vex/ingestion.json	PENDING
VEX-002	Lattice merge resolves conflicting VEX from multiple sources	bench/vex/conflict-resolution.json	PENDING
VEX-003	VEX status affects scoring deterministically	bench/vex/scoring-impact.json	PENDING

Offline / Air-Gapped

Claim ID	Claim	Evidence	Status
OFFLINE-001	Full scan + attest + verify with no network	bench/offline/e2e.json	PENDING
OFFLINE-002	Knowledge snapshots are cryptographically bound to scans	bench/offline/snapshots.json	PENDING
OFFLINE-003	Offline bundles include all required feeds	bench/offline/bundle-completeness.json	PENDING

SBOM Fidelity

Claim ID	Claim	Evidence	Status
SBOM-001	CycloneDX 1.6+ and SPDX 3.0.1 export	bench/sbom/format-coverage.json	PENDING
SBOM-002	Binary provenance tracked (Build-ID, PE hash)	bench/sbom/binary-provenance.json	PENDING
SBOM-003	Layer attribution for all components	bench/sbom/layer-attribution.json	PENDING
SBOM-004	SBOM lineage DAG with semantic diffing	bench/sbom/lineage.json	PENDING

Performance

Claim ID	Claim	Evidence	Status
PERF-001	Scan latency < 30s p95 for 100k LOC	bench/performance/latency.json	PENDING
PERF-002	10k scans/day without degradation	bench/performance/scale.json	PENDING
PERF-003	Incremental scans < 5s for minor changes	bench/performance/incremental.json	PENDING

---

Competitor Comparison Matrix

Capability	Stella Ops	Trivy	Grype	Snyk	Anchore
SBOM Fidelity	HIGH	MEDIUM	MEDIUM	MEDIUM	HIGH
VEX Handling	NATIVE	PARTIAL	NONE	UNKNOWN	PARTIAL
Explainability	HIGH (with falsifiability)	LOW	LOW	MEDIUM	MEDIUM
Smart-Diff	SEMANTIC	NONE	NONE	NONE	POLICY
Call-Stack Reachability	3-LAYER	NONE	NONE	NONE	NONE
Deterministic Scoring	PROVEN	MODERATE	MODERATE	PROPRIETARY	MODERATE
Unknowns State	FIRST-CLASS	NONE	NONE	PARTIAL	PARTIAL
Offline/Air-Gapped	FULL	AD-HOC	AD-HOC	UNKNOWN	ENTERPRISE
Provenance/Attestations	DSSE/in-toto	SBOM ONLY	SBOM ONLY	UNKNOWN	SBOM+in-toto

Note

: Comparison based on public documentation and benchmarks. Updated: PENDING

---

Evidence Links

All evidence files are stored in bench/ directory and versioned in Git.

Evidence Type	Location	Format
Benchmark results	bench/competitors/	JSON
Determinism tests	bench/determinism/	JSON
Reachability corpus	bench/reachability/	JSON + ground truth
Performance baselines	bench/performance/	JSON
Attestation samples	bench/attestation/	DSSE envelopes

---

Updating Claims

Claims are updated via:

1. Automated: benchmark-vs-competitors.yml workflow runs weekly
2. Manual: PRs updating evidence require benchmark re-run
3. Release: All claims verified before release

Claim Lifecycle

PENDING → VERIFIED → PUBLISHED
           ↓
        DISPUTED → RESOLVED

• PENDING: Claim defined, evidence not yet generated
• VERIFIED: Evidence generated and validated
• PUBLISHED: Included in marketing materials
• DISPUTED: External challenge received
• RESOLVED: Dispute addressed with updated evidence

---

Related Documentation

• Benchmark Architecture
• Sprint 7000.0001.0001 - Competitive Benchmarking
• Testing Strategy

---

Last Updated: 2025-12-22 Next Review: After Sprint 7000.0001.0001 completion