37 lines
1.8 KiB
Markdown
37 lines
1.8 KiB
Markdown
# Scanner Analyzer Microbench Harness
|
||
|
||
The bench harness exercises the language analyzers against representative filesystem layouts so that regressions are caught before they ship.
|
||
|
||
## Layout
|
||
- `run-bench.js` – Node.js script that traverses the sample `node_modules/` and `site-packages/` trees, replicating the package discovery work performed by the upcoming analyzers.
|
||
- `config.json` – Declarative list of scenarios the harness executes. Each scenario points at a directory in `samples/`.
|
||
- `baseline.csv` – Reference numbers captured on the 4 vCPU warm rig described in `docs/12_PERFORMANCE_WORKBOOK.md`. CI publishes fresh CSVs so perf trends stay visible.
|
||
|
||
## Running locally
|
||
|
||
```bash
|
||
cd bench/Scanner.Analyzers
|
||
node run-bench.js --out baseline.csv --samples ../..
|
||
```
|
||
|
||
The harness prints a table to stdout and writes the CSV (if `--out` is specified) with the following headers:
|
||
|
||
```
|
||
scenario,iterations,sample_count,mean_ms,p95_ms,max_ms
|
||
```
|
||
|
||
Use `--iterations` to override the default (5 passes per scenario) and `--threshold-ms` to customize the failure budget. Budgets default to 5 000 ms, aligned with the SBOM compose objective.
|
||
|
||
## Adding scenarios
|
||
1. Drop the fixture tree under `samples/<area>/...`.
|
||
2. Append a new scenario entry to `config.json` describing:
|
||
- `id` – snake_case scenario name (also used in CSV).
|
||
- `label` – human-friendly description shown in logs.
|
||
- `root` – path to the directory that will be scanned.
|
||
- `matcher` – glob describing files that will be parsed (POSIX `**` patterns).
|
||
- `parser` – `node` or `python` to choose the metadata reader.
|
||
3. Re-run `node run-bench.js --out baseline.csv`.
|
||
4. Commit both the fixture and updated baseline.
|
||
|
||
The harness is intentionally dependency-free to remain runnable inside minimal CI runners.
|