48 lines
1.8 KiB
Markdown
48 lines
1.8 KiB
Markdown
# Bench (Performance Benchmarks)
|
|
|
|
**Status:** Implemented
|
|
**Source:** `src/Bench/`
|
|
**Owner:** Platform Team
|
|
|
|
> **Note:** This folder documents **performance benchmarks**. For **competitive benchmarking** (accuracy comparison with other scanners), see [`../benchmark/`](../benchmark/).
|
|
|
|
## Purpose
|
|
|
|
Bench provides performance benchmark infrastructure for StellaOps modules. Measures throughput, latency, and resource usage to detect regressions and validate performance targets.
|
|
|
|
## Components
|
|
|
|
**Benchmark Projects:**
|
|
- `StellaOps.Bench.LinkNotMerge` - Link-Not-Merge correlation performance
|
|
- `StellaOps.Bench.LinkNotMerge.Vex` - LNM VEX statement performance
|
|
- `StellaOps.Bench.Notify` - Notification delivery throughput
|
|
- `StellaOps.Bench.PolicyEngine` - Policy evaluation performance
|
|
- `StellaOps.Bench.ScannerAnalyzers` - Language analyzer performance
|
|
|
|
## Scanner Vendor Parity Tracking
|
|
|
|
`StellaOps.Bench.ScannerAnalyzers` now supports vendor parity tracking for offline benchmark runs:
|
|
- Scenario-level vendor ingestion from JSON or SARIF artifacts (`vendorResults[]` in benchmark config).
|
|
- Optional Stella finding ingestion (`stellaFindingsPath`) for exact overlap comparisons.
|
|
- Deterministic parity outputs in benchmark JSON and Prometheus exports:
|
|
- overlap counts and percentages
|
|
- scanner-only / vendor-only counts
|
|
- parity score (Jaccard-style overlap over union)
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Run all benchmarks
|
|
dotnet run -c Release --project src/Bench/StellaOps.Bench/LinkNotMerge/StellaOps.Bench.LinkNotMerge
|
|
|
|
# Run with specific runtime
|
|
dotnet run -c Release --project src/Bench/StellaOps.Bench/Notify/StellaOps.Bench.Notify
|
|
```
|
|
|
|
## Related Documentation
|
|
|
|
- Competitive Benchmark: `../benchmark/architecture.md`
|
|
- Scanner: `../scanner/architecture.md`
|
|
- Policy: `../policy/architecture.md`
|
|
- Notify: `../notify/architecture.md`
|