stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
3ecafc49a3805e5aa7fc89507a389359baa0db88
git.stella-ops.org/docs/features/checked/bench/benchmark-harness.md
master 5bca406787 save checkpoint: save features
2026-02-12 10:27:23 +02:00

2.4 KiB
Raw Blame History

Benchmark harness (reachability, scanner analyzers, policy engine, determinism)

Module

Bench

Status

VERIFIED

Description

Comprehensive benchmark harness code exists across LinkNotMerge, LinkNotMerge.Vex, Notify, PolicyEngine, and Scanner.Analyzers modules with deterministic benchmark/reporting support.

Implementation Details

  • LinkNotMerge Benchmark: src/Bench/StellaOps.Bench/LinkNotMerge/StellaOps.Bench.LinkNotMerge/ -- benchmark scenarios for linkset aggregation performance.
  • LinkNotMerge VEX Benchmark: src/Bench/StellaOps.Bench/LinkNotMerge.Vex/StellaOps.Bench.LinkNotMerge.Vex/ -- VEX-specific linkset benchmarks.
  • Notify Benchmark: src/Bench/StellaOps.Bench/Notify/StellaOps.Bench.Notify/ -- notification dispatch benchmarks.
  • PolicyEngine Benchmark: src/Bench/StellaOps.Bench/PolicyEngine/StellaOps.Bench.PolicyEngine/ -- policy evaluation benchmarks.
  • PolicyEngine Benchmark Policy: src/Bench/StellaOps.Bench/PolicyEngine/policies/benchmark-default.yaml -- benchmark policy fixture compatible with current StellaOps.Policy binder schema.
  • Scanner.Analyzers Benchmark: src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/ -- scanner analyzer benchmarks.
  • Baseline Infrastructure: benchmark modules include Baseline/BaselineEntry.cs and Baseline/BaselineLoader.cs for ground-truth comparison.
  • Reporting Infrastructure: benchmark modules include JSON and Prometheus writers for machine-consumable artifacts.
  • Tests: link-not-merge, vex, notify, and scanner-analyzer benchmark test suites under src/Bench/StellaOps.Bench/**.Tests/.

E2E Test Plan

  • Run LinkNotMerge benchmark harness and verify scenario table output is generated.
  • Verify JSON report output is produced and non-empty.
  • Verify Prometheus metrics output is produced and non-empty.
  • Verify CSV result output is produced and non-empty.
  • Verify negative-path CLI behavior (--config missing path) exits non-zero.

Verification

  • Verified on 2026-02-11 via FLOW Tier 0/1/2 replay in run-005.
  • Tier 0: docs/qa/feature-checks/runs/bench/benchmark-harness/run-005/tier0-source-check.json
  • Tier 1: docs/qa/feature-checks/runs/bench/benchmark-harness/run-005/tier1-build-check.json
  • Tier 2: docs/qa/feature-checks/runs/bench/benchmark-harness/run-005/tier2-integration-check.json
  • Tier 2 evidence: docs/qa/feature-checks/runs/bench/benchmark-harness/run-005/evidence/