git.stella-ops.org/bench/Scanner.Analyzers/README.md

Scanner Analyzer Microbench Harness

The bench harness exercises the language analyzers against representative filesystem layouts so that regressions are caught before they ship.

Layout

• run-bench.js – Node.js script that traverses the sample node_modules/ and site-packages/ trees, replicating the package discovery work performed by the upcoming analyzers.
• config.json – Declarative list of scenarios the harness executes. Each scenario points at a directory in samples/.
• baseline.csv – Reference numbers captured on the 4 vCPU warm rig described in docs/12_PERFORMANCE_WORKBOOK.md. CI publishes fresh CSVs so perf trends stay visible.

Running locally

cd bench/Scanner.Analyzers
node run-bench.js --out baseline.csv --samples ../..

The harness prints a table to stdout and writes the CSV (if --out is specified) with the following headers:

scenario,iterations,sample_count,mean_ms,p95_ms,max_ms

Use --iterations to override the default (5 passes per scenario) and --threshold-ms to customize the failure budget. Budgets default to 5 000 ms, aligned with the SBOM compose objective.

Adding scenarios

1. Drop the fixture tree under samples/<area>/....
2. Append a new scenario entry to config.json describing:
   • id – snake_case scenario name (also used in CSV).
   • label – human-friendly description shown in logs.
   • root – path to the directory that will be scanned.
   • matcher – glob describing files that will be parsed (POSIX ** patterns).
   • parser – node or python to choose the metadata reader.
3. Re-run node run-bench.js --out baseline.csv.
4. Commit both the fixture and updated baseline.

The harness is intentionally dependency-free to remain runnable inside minimal CI runners.