Files
git.stella-ops.org/docs/benchmarks/scanner-feature-comparison-grype.md
master 3a2100aa78 Add unit and integration tests for VexCandidateEmitter and SmartDiff repositories
- Implemented comprehensive unit tests for VexCandidateEmitter to validate candidate emission logic based on various scenarios including absent and present APIs, confidence thresholds, and rate limiting.
- Added integration tests for SmartDiff PostgreSQL repositories, covering snapshot storage and retrieval, candidate storage, and material risk change handling.
- Ensured tests validate correct behavior for storing, retrieving, and querying snapshots and candidates, including edge cases and expected outcomes.
2025-12-16 19:00:43 +02:00

8.0 KiB
Raw Permalink Blame History

Scanner Feature Comparison — StellaOps vs Grype

Reference snapshot: Grype commit 6e746a546ecca3e2456316551673357e4a166d77 cloned 2025-11-02.

Verification Metadata

Field Value
Last Updated 2025-12-15
Last Verified 2025-12-14
Next Review 2026-03-14
Claims Index docs/market/claims-citation-index.md
Claim IDs COMP-GRYPE-001, COMP-GRYPE-002, COMP-GRYPE-003
Verification Method Source code audit (OSS), documentation review, feature testing

Confidence Levels:

  • High (80-100%): Verified against source code or authoritative documentation
  • Medium (50-80%): Based on documentation or limited testing; needs deeper verification
  • Low (<50%): Unverified or based on indirect evidence; requires validation

TL;DR

  • StellaOps runs as a multi-service platform with deterministic SBOM generation, attestation (DSSE + Rekor), and tenant-aware controls, whereas Grype is a single Go CLI that leans on Syft to build SBOMs before vulnerability matching.1g1
  • Grype covers a broad OS and language matrix via Syft catalogers and Anchores aggregated vulnerability database, but it lacks attestation, runtime usage context, and secret management features found in StellaOps Surface/Policy ecosystem.1g2g3
  • Opportunity: position StellaOps strengths (evidence provenance, policy orchestration, offline kits) while tracking high-demand ecosystems that Grype already supports (e.g., Ruby, PHP) and could inform StellaOps backlog priorities.

Comparison Matrix

Dimension StellaOps Scanner Grype
Architecture & deployment WebService + Worker services, queue backbones, RustFS/S3 artifact store, PostgreSQL catalog, Authority-issued OpToks, Surface libraries, restart-only analyzers.1345 Go CLI that invokes Syft to construct an SBOM from images/filesystems and feeds Syft's packages into Anchore matchers; optional SBOM ingest via syft/sbom inputs.g1
Scan targets & coverage Container images & filesystem captures; analyzers for APK/DPKG/RPM, Java/Node/Python/Go/.NET/Rust, native ELF, EntryTrace usage graph (PE/Mach-O roadmap).1 Images, directories, archives, and SBOMs; OS feeds include Alpine, Ubuntu, RHEL, SUSE, Wolfi, etc., and language support spans Ruby, Java, JavaScript, Python, .NET, Go, PHP, Rust.g2
Evidence & outputs CycloneDX JSON/Protobuf, SPDX 3.0.1, deterministic diffs, BOM-index sidecar, explain traces, DSSE-ready report metadata.12 Outputs table, JSON, CycloneDX (XML/JSON), SARIF, and templated formats; evidence tied to Syft SBOM and JSON report (no deterministic replay artifacts).g4
Attestation & supply chain DSSE signing via Signer → Attestor → Rekor v2, OpenVEX-first modelling, policy overlays, provenance digests.1 Supports ingesting OpenVEX for filtering but ships no signing/attestation workflow; relies on external tooling for provenance.g2
Policy & decisioning Central Policy Engine (stella-dsl), VEX-first decisioning, lattice logic, API streaming of policy previews, fail-fast validation pipelines.17 CLI gating via --fail-on, ignore rules, and allow/deny lists; no multi-tenant policy service.g4
Offline & air-gap Offline kits bundle artifacts, manifests, secrets; Surface.Env/Validation enforce prerequisites; RustFS supports air-gapped object storage.346 Local SQLite vulnerability DB auto-managed; supports offline by disabling auto-update and importing Anchore DB archives manually.g3
Caching & performance Layer CAS caches, queue leasing, EntryTrace reuse, deterministic sorting for replay.14 SBOM caching handled by Syft; vulnerability DB stored under $XDG_CACHE_HOME; no layer caches or queue orchestration.g3
Security & tenancy OpTok enforcement (DPoP/mTLS), tenant-aware storage prefixes, Surface.Secrets providers, validation pipeline for misconfiguration, DSSE for tamper evidence.156 Single-user CLI; registry credentials handled via config secrets; no tenant isolation or secret provider abstraction.g3
Extensibility & ecosystem Analyzer plug-ins, BuildX SBOM generator, CLI/Worker integration, Surface libraries, attested exports.12 Template-based reporters, external Maven lookup, GitHub Actions integrations; cataloging delegated to Syft (extensible separately).g2g4
Observability & ops Structured logs, metrics, explain traces, offline manifests, runbooks.146 CLI logging and exit codes; no built-in metrics/traces beyond verbose logs.g4

Ecosystem Deep Dives

Observations

  • Grypes reliance on Syft provides broad package coverage quickly, but it inherits Syfts metadata limits (no runtime usage or deterministic replay). StellaOps can continue emphasizing provenance and policy orchestration while considering backlog for ecosystems where Grype today has parity (Ruby/PHP).g1g5
  • Anchores aggregated vulnerability DB offers fast setup, though it introduces dependency on Anchore services; StellaOps approach keeps attestation and evidence self-hostable, which remains a differentiator for regulated tenants.1g3
  • Secret detection, SAST, and runtime attestation remain gaps for both Trivy and Grype—opportunity for StellaOps to lead with Surface policies, recommend SAST partners, and emphasise DSSE/EntryTrace integration.

Opportunities for StellaOps

  1. Prioritize analyzers for ecosystems currently supported by Grype (Ruby, PHP, Composer) to close parity gaps without sacrificing determinism.g5
  2. Publish guidance on integrating Anchore DB feeds (where allowed) while keeping StellaOps provenance guarantees, or offer curated feeds with DSSE-backed provenance for air-gapped users.g3
  3. Evaluate optional syft/grype compatibility layers (importing Syft SBOMs) so customers can transition while retaining StellaOps attestation benefits.

Sources

  1. docs/modules/scanner/architecture.md
  2. docs/modules/scanner/implementation_plan.md
  3. docs/modules/scanner/design/surface-env.md
  4. docs/modules/scanner/design/surface-fs.md
  5. docs/modules/scanner/design/surface-secrets.md
  6. docs/modules/scanner/design/surface-validation.md
  7. docs/modules/platform/architecture-overview.md

Grype sources

  • [g1] grype/pkg/syft_provider.go
  • [g2] grype/README.md (Features list: OS & language coverage, external sources, integrations)
  • [g3] grype/README.md (Vulnerability DB management, offline workflow)
  • [g4] grype/README.md (Output formats, CLI controls)
  • [g5] grype/grype/matcher/javascript/matcher.go