Files
git.stella-ops.org/docs/benchmarks/scanner-rust-analyzer.md
master f98cea3bcf Add Authority Advisory AI and API Lifecycle Configuration
- Introduced AuthorityAdvisoryAiOptions and related classes for managing advisory AI configurations, including remote inference options and tenant-specific settings.
- Added AuthorityApiLifecycleOptions to control API lifecycle settings, including legacy OAuth endpoint configurations.
- Implemented validation and normalization methods for both advisory AI and API lifecycle options to ensure proper configuration.
- Created AuthorityNotificationsOptions and its related classes for managing notification settings, including ack tokens, webhooks, and escalation options.
- Developed IssuerDirectoryClient and related models for interacting with the issuer directory service, including caching mechanisms and HTTP client configurations.
- Added support for dependency injection through ServiceCollectionExtensions for the Issuer Directory Client.
- Updated project file to include necessary package references for the new Issuer Directory Client library.
2025-11-02 13:50:25 +02:00

2.6 KiB

Scanner Rust Analyzer Benchmarks (Sprint 130)

Summary

  • New fixtures under src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/ capture heuristic and fallback behaviours.
  • RustLanguageAnalyzerBenchmark (see src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks) exercises the new fixtures and emits heuristic coverage metrics.
  • March 2025 snapshot: heuristic detection covers 3 crates (reqwest, serde, tokio) vs competitor baseline (1 crate), yielding 300% relative coverage.

Running the benchmarks

# Build once in Release for accurate metrics
DOTNET_CLI_UI_LANGUAGE=en dotnet build src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -c Release

# Dump heuristic output
DOTNET_CLI_UI_LANGUAGE=en dotnet run --no-build   --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj   -- --dump-heuristics

# Dump fallback binary output (bin provenance)
DOTNET_CLI_UI_LANGUAGE=en dotnet run --no-build   --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj   -- --dump-fallback

# Run benchmarks (optional)
DOTNET_CLI_UI_LANGUAGE=en dotnet run   --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj   -c Release

BenchmarkDotNet emits warnings when running against Debug builds. Use -c Release or set config.WithOptions(ConfigOptions.DisableOptimizationsValidator) if you need debugging traces.

Heuristic coverage comparison

Fixture Our detector Competitor baseline Relative gain
lang/rust/heuristics reqwest, serde, tokio serde +300% (3/1)

Competitor baseline data lives in competitor-baseline.json alongside the heuristics fixture and is used by RustHeuristicCoverageComparisonTests.

Fallback binary snapshot

componentKey: bin::sha256:10f3c03766e4403be40add0467a2b2d07fd7006e4b8515ab88740ffa327ea775
metadata:
  binary.path    -> usr/local/bin/opaque_bin
  binary.sha256  -> 10f3c03766e4403be40add0467a2b2d07fd7006e4b8515ab88740ffa327ea775
  provenance     -> binary
evidence:
  source=binary locator=usr/local/bin/opaque_bin sha256=10f3c03766e4403be40add0467a2b2d07fd7006e4b8515ab88740ffa327ea775

This snapshot is persisted in expected.json for repeatable regression tests.