stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity
Files
634233dfed12f52e50cf2846e72bb92e600f41ef
git.stella-ops.org/docs/claims-index.md
StellaOps Bot 634233dfed feat: Implement distro-native version comparison for RPM, Debian, and Alpine packages 
- Add RpmVersionComparer for RPM version comparison with epoch, version, and release handling.
- Introduce DebianVersion for parsing Debian EVR (Epoch:Version-Release) strings.
- Create ApkVersion for parsing Alpine APK version strings with suffix support.
- Define IVersionComparator interface for version comparison with proof-line generation.
- Implement VersionComparisonResult struct to encapsulate comparison results and proof lines.
- Add tests for Debian and RPM version comparers to ensure correct functionality and edge case handling.
- Create project files for the version comparison library and its tests.
2025-12-22 09:50:12 +02:00

174 lines
6.8 KiB
Markdown
Raw Blame History

# Stella Ops Claims Index
This document provides a verifiable index of competitive claims. Each claim is linked to evidence and can be verified using the provided commands.
> **Integrity**: This index is updated automatically by the benchmark CI workflow. Manual edits require PR approval.
---
## How to Verify Claims
```bash
# Verify a specific claim
stella benchmark verify <CLAIM_ID>
# Run full benchmark suite
stella benchmark run --competitors trivy,grype,syft
# Generate updated claims from latest benchmark
stella benchmark claims --output docs/claims-index.md
```
---
## Claim Categories
| Category | Prefix | Description |
|----------|--------|-------------|
| Determinism | DET-* | Reproducible, bit-identical outputs |
| Reachability | REACH-* | Call-path and exploitability analysis |
| Proofs | PROOF-* | Attestation and cryptographic evidence |
| Unknowns | UNK-* | Explicit uncertainty tracking |
| VEX | VEX-* | VEX handling and conflict resolution |
| Offline | OFFLINE-* | Air-gapped operation |
| SBOM | SBOM-* | SBOM fidelity and formats |
| Performance | PERF-* | Speed and scalability |
---
## Active Claims
### Determinism
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| DET-001 | Stella Ops produces bit-identical scan results across runs | `bench/determinism/results.json` | PENDING |
| DET-002 | Score replay produces identical verdicts from manifest | `bench/replay/results.json` | PENDING |
| DET-003 | SBOM generation is deterministic (stable ordering, canonical JSON) | `bench/sbom/determinism.json` | PENDING |
### Reachability
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| REACH-001 | Stella Ops detects reachable vulnerabilities missed by Trivy | `bench/competitors/trivy-comparison.json` | PENDING |
| REACH-002 | Stella Ops eliminates X% false positives via unreachable path detection | `bench/reachability/fp-elimination.json` | PENDING |
| REACH-003 | Three-layer reachability (static + binary + runtime) provides higher confidence | `bench/reachability/3layer-corpus.json` | PENDING |
### Proofs & Attestations
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| PROOF-001 | Every scan produces DSSE-signed attestation | `bench/attestation/coverage.json` | PENDING |
| PROOF-002 | Score proofs enable third-party verification | `bench/proofs/verification.json` | PENDING |
| PROOF-003 | Evidence chain links finding to source material | `bench/proofs/chain-integrity.json` | PENDING |
### Unknowns
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| UNK-001 | Unknowns are first-class, not suppressed | `bench/unknowns/tracking.json` | PENDING |
| UNK-002 | Unknowns have budgets and decay policies | `bench/unknowns/budgets.json` | PENDING |
| UNK-003 | Unknowns escalate based on blast radius and age | `bench/unknowns/escalation.json` | PENDING |
### VEX Handling
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| VEX-001 | Native VEX ingestion with formal reasoning | `bench/vex/ingestion.json` | PENDING |
| VEX-002 | Lattice merge resolves conflicting VEX from multiple sources | `bench/vex/conflict-resolution.json` | PENDING |
| VEX-003 | VEX status affects scoring deterministically | `bench/vex/scoring-impact.json` | PENDING |
### Offline / Air-Gapped
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| OFFLINE-001 | Full scan + attest + verify with no network | `bench/offline/e2e.json` | PENDING |
| OFFLINE-002 | Knowledge snapshots are cryptographically bound to scans | `bench/offline/snapshots.json` | PENDING |
| OFFLINE-003 | Offline bundles include all required feeds | `bench/offline/bundle-completeness.json` | PENDING |
### SBOM Fidelity
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| SBOM-001 | CycloneDX 1.6+ and SPDX 3.0.1 export | `bench/sbom/format-coverage.json` | PENDING |
| SBOM-002 | Binary provenance tracked (Build-ID, PE hash) | `bench/sbom/binary-provenance.json` | PENDING |
| SBOM-003 | Layer attribution for all components | `bench/sbom/layer-attribution.json` | PENDING |
| SBOM-004 | SBOM lineage DAG with semantic diffing | `bench/sbom/lineage.json` | PENDING |
### Performance
| Claim ID | Claim | Evidence | Status |
|----------|-------|----------|--------|
| PERF-001 | Scan latency < 30s p95 for 100k LOC | `bench/performance/latency.json` | PENDING |
| PERF-002 | 10k scans/day without degradation | `bench/performance/scale.json` | PENDING |
| PERF-003 | Incremental scans < 5s for minor changes | `bench/performance/incremental.json` | PENDING |
---
## Competitor Comparison Matrix
| Capability | Stella Ops | Trivy | Grype | Snyk | Anchore |
|------------|-----------|-------|-------|------|---------|
| SBOM Fidelity | HIGH | MEDIUM | MEDIUM | MEDIUM | HIGH |
| VEX Handling | NATIVE | PARTIAL | NONE | UNKNOWN | PARTIAL |
| Explainability | HIGH (with falsifiability) | LOW | LOW | MEDIUM | MEDIUM |
| Smart-Diff | SEMANTIC | NONE | NONE | NONE | POLICY |
| Call-Stack Reachability | 3-LAYER | NONE | NONE | NONE | NONE |
| Deterministic Scoring | PROVEN | MODERATE | MODERATE | PROPRIETARY | MODERATE |
| Unknowns State | FIRST-CLASS | NONE | NONE | PARTIAL | PARTIAL |
| Offline/Air-Gapped | FULL | AD-HOC | AD-HOC | UNKNOWN | ENTERPRISE |
| Provenance/Attestations | DSSE/in-toto | SBOM ONLY | SBOM ONLY | UNKNOWN | SBOM+in-toto |
> **Note**: Comparison based on public documentation and benchmarks. Updated: PENDING
---
## Evidence Links
All evidence files are stored in `bench/` directory and versioned in Git.
| Evidence Type | Location | Format |
|---------------|----------|--------|
| Benchmark results | `bench/competitors/` | JSON |
| Determinism tests | `bench/determinism/` | JSON |
| Reachability corpus | `bench/reachability/` | JSON + ground truth |
| Performance baselines | `bench/performance/` | JSON |
| Attestation samples | `bench/attestation/` | DSSE envelopes |
---
## Updating Claims
Claims are updated via:
1. **Automated**: `benchmark-vs-competitors.yml` workflow runs weekly
2. **Manual**: PRs updating evidence require benchmark re-run
3. **Release**: All claims verified before release
### Claim Lifecycle
```
PENDING → VERIFIED → PUBLISHED
↓
DISPUTED → RESOLVED
```
- **PENDING**: Claim defined, evidence not yet generated
- **VERIFIED**: Evidence generated and validated
- **PUBLISHED**: Included in marketing materials
- **DISPUTED**: External challenge received
- **RESOLVED**: Dispute addressed with updated evidence
---
## Related Documentation
- [Benchmark Architecture](modules/benchmark/architecture.md)
- [Sprint 7000.0001.0001 - Competitive Benchmarking](implplan/SPRINT_7000_0001_0001_competitive_benchmarking.md)
- [Testing Strategy](implplan/SPRINT_5100_SUMMARY.md)
---
*Last Updated*: 2025-12-22
*Next Review*: After Sprint 7000.0001.0001 completion
Reference in New Issue View Git Blame Copy Permalink