Files
git.stella-ops.org/docs/benchmarks/scanner-feature-comparison-snyk.md
master 3a2100aa78 Add unit and integration tests for VexCandidateEmitter and SmartDiff repositories
- Implemented comprehensive unit tests for VexCandidateEmitter to validate candidate emission logic based on various scenarios including absent and present APIs, confidence thresholds, and rate limiting.
- Added integration tests for SmartDiff PostgreSQL repositories, covering snapshot storage and retrieval, candidate storage, and material risk change handling.
- Ensured tests validate correct behavior for storing, retrieving, and querying snapshots and candidates, including edge cases and expected outcomes.
2025-12-16 19:00:43 +02:00

7.5 KiB
Raw Permalink Blame History

Scanner Feature Comparison — StellaOps vs Snyk CLI

Reference snapshot: Snyk CLI commit 7ae3b11642d143b588016d4daef0a6ddaddb792b cloned 2025-11-02.

Verification Metadata

Field Value
Last Updated 2025-12-15
Last Verified 2025-12-14
Next Review 2026-03-14
Claims Index docs/market/claims-citation-index.md
Claim IDs COMP-SNYK-001, COMP-SNYK-002, COMP-SNYK-003
Verification Method Source code audit (OSS), documentation review, feature testing

Confidence Levels:

  • High (80-100%): Verified against source code or authoritative documentation
  • Medium (50-80%): Based on documentation or limited testing; needs deeper verification
  • Low (<50%): Unverified or based on indirect evidence; requires validation

TL;DR

  • StellaOps delivers a self-hosted, multi-service scanning plane with deterministic SBOMs, attestation (DSSE + Rekor), and tenant-aware Surface controls, while the Snyk CLI is a Node.js tool that authenticates against Snyks SaaS to analyse dependency graphs, containers, IaC, and code.1s1
  • Snyks plugin ecosystem covers many package managers (npm, yarn, pnpm, Maven, Gradle, NuGet, Go modules, Composer, etc.) and routes scans through Snyks cloud for policy, reporting, and fix advice; however it lacks offline operation, deterministic evidence, and attestation workflows that StellaOps provides out of the box.1s1s2
  • Opportunity: Lean on StellaOps strengths (offline parity, provenance, policy) while tracking Snyk-only ecosystems (for example, SwiftPM, CocoaPods) and SaaS conveniences (IaC, Snyk Code) that may influence backlog priorities.

Comparison Matrix

Dimension StellaOps Scanner Snyk CLI
Architecture & deployment WebService + Worker services, queue backbone, RustFS/S3 artifact store, PostgreSQL catalog, Authority-issued OpToks, Surface libs, restart-only analyzers.1345 Node.js CLI; users authenticate (snyk auth) and run commands (snyk test, snyk monitor, snyk container test) that upload project metadata to Snyk's SaaS for analysis.s2
Scan targets & coverage Container images/filesystems, analyzers for APK/DPKG/RPM, Java/Node/Python/Go/.NET/Rust, native ELF, EntryTrace usage graph.1 Supports Snyk Open Source, Container, Code (SAST), and IaC; plugin loader dispatches npm/yarn/pnpm, Maven/Gradle/SBT, pip/poetry, Go modules, NuGet/Paket, Composer, CocoaPods, Hex, SwiftPM.s1s2
Evidence & outputs CycloneDX JSON/Protobuf, SPDX 3.0.1, deterministic diffs, BOM-index sidecar, explain traces, DSSE-ready report metadata.12 CLI prints human-readable tables and supports JSON/SARIF outputs for Snyk Open Source/Snyk Code; results originate from cloud analysis, not deterministic SBOM fragments.s3
Attestation & supply chain DSSE signing via Signer → Attestor → Rekor v2, OpenVEX-first modelling, policy overlays, provenance digests.1 No DSSE/attestation workflow; remediation guidance and monitors live in Snyk SaaS.s2
Policy & decisioning Central Policy Engine (stella-dsl), lattice logic, VEX-first decisioning, API streaming of policy previews.17 Policy controls managed in Snyk platform (org/project settings); CLI can gate on severity (--severity-threshold) and push projects for monitoring.s2
Offline & air-gap Offline kits with Surface manifests, secrets bundles, RustFS; no external connectivity required after provisioning.346 Requires internet connectivity for authentication and analysis; no offline mode documented.s2
Caching & performance Layer CAS caches, queue leasing, EntryTrace reuse, deterministic ordering.14 Dependency graphs are resolved locally, but vulnerability analysis happens in the cloud; no local cache beyond CLI conveniences.s1s2
Security & tenancy OpTok enforcement (DPoP/mTLS), tenant-aware storage, Surface.Secrets providers, validation pipeline.156 Authentication scoped per Snyk org; registry credentials handled via config/secret stores but no tenant isolation inside the CLI itself.s2
Extensibility & ecosystem Analyzer plug-ins, BuildX SBOM generator, CLI/Worker integration, attested exports.12 Plugin architecture for package managers (Node.js legacy + external plugins); integrations for CI/IDE rely on Snyk platform APIs.s1
Observability & ops Structured logs, metrics, explain traces, offline manifests, runbooks.146 CLI logs to stdout/stderr; deeper analytics available via Snyk SaaS dashboards rather than local instrumentation.s2

Ecosystem Deep Dives

Observations

  • Snyks cloud-first workflow simplifies policy management and monitoring for hosted users, but it prevents fully offline operation—StellaOps should continue emphasising sovereign/offline parity while documenting bridge options (e.g., exporting SBOMs for Snyk ingestion).s2
  • Plugin breadth (SwiftPM, CocoaPods, Hex) exceeds current StellaOps coverage; backlog items for these ecosystems may become higher priority if customer demand aligns.s1
  • Secret detection and SAST are available via Snyk Code, yet require uploading code; StellaOps can differentiate with deterministic, self-hosted evidence plus attestation.

Opportunities for StellaOps

  1. Evaluate demand for additional package managers (SwiftPM, CocoaPods, Hex) supported by Snyk plugins and scope analyzer roadmap accordingly.s1
  2. Provide guidance on integrating with external SaaS tools (including Snyk) using StellaOps SBOM exports for hybrid workflows.
  3. Continue to highlight DSSE/attestation, offline kits, and tenant isolation as differentiators versus cloud-only scanners.

Sources

  1. docs/modules/scanner/architecture.md
  2. docs/modules/scanner/implementation_plan.md
  3. docs/modules/scanner/design/surface-env.md
  4. docs/modules/scanner/design/surface-fs.md
  5. docs/modules/scanner/design/surface-secrets.md
  6. docs/modules/scanner/design/surface-validation.md
  7. docs/modules/platform/architecture-overview.md

Snyk sources

  • [s1] /tmp/snyk-cli/src/lib/plugins/index.ts
  • [s2] /tmp/snyk-cli/README.md
  • [s3] /tmp/snyk-cli/src/lib/plugins/sast/format/output-format.ts