git.stella-ops.org/docs/qa/feature-checks/runs/cli/cli-e2e-tests/batch-d-results.md

CLI E2E Test Results -- Batch D

Date: 2026-02-15 Runner: CLI E2E subagent (batch-d) CLI project: src/Cli/StellaOps.Cli/StellaOps.Cli.csproj (Release, --no-build)

Summary

• Total commands tested: 24
• All --help pass: 24/24
• Behavioral tests run: 4 (doctor list, ci list, golden list, fmap alias)
• Behavioral passes: 3/4 (golden list exits 1 -- expected, no corpus dir)
• Crashes / hangs: 0

All commands exhibit the expected ~4s SM remote probe timeout on startup (localhost:56080 refused). This is benign and does not affect command functionality.

Results Table

#	Command	Subcommands	--help OK	Behavioral Test	Exit Code	Notes
1	replay	verify, diff, batch, snapshot, export	Yes (exit 0)	--help only (requires --manifest)	0	Has REQUIRED --manifest option
2	delta	compute, check, attach, verify, push	Yes (exit 0)	--help only	0
3	budget	status, consume, check, history, list	Yes (exit 0)	--help only	0
4	reachability	show, export, trace, explain, witness, guards, graph, slice, witness-ops	Yes (exit 0)	--help only	0	9 subcommands; graph/slice/witness-ops from plugins
5	witness	generate, verify, bundle	Yes (exit 0)	--help only	0	generate/verify require args
6	watchlist	add, list, get, update, remove, test, alerts	Yes (exit 0)	--help only	0	7 subcommands
7	function-map	generate, verify	Yes (exit 0)	--help only	0	Alias: fmap
8	fmap (alias)	generate, verify	Yes (exit 0)	fmap --help	0	Alias works, shows same as function-map
9	observations	query	Yes (exit 0)	--help only	0	Single subcommand
10	gate	evaluate, status, score	Yes (exit 0)	--help only	0	score uses EWS
11	ci	init, list, validate	Yes (exit 0)	ci list	0	Lists 12 templates (github/gitlab/gitea x gate/scan/verify/full)
12	github	upload-sarif, list-alerts, get-alert, update-alert, upload-status	Yes (exit 0)	--help only	0	5 subcommands
13	exception	request, approve, reject, list, status	Yes (exit 0)	--help only	0	Full CRUD workflow
14	feedser	bundle, sites	Yes (exit 0)	--help only	0	Federation bundle ops
15	prove	(none -- leaf command)	Yes (exit 0)	--help only	0	Requires --image; supports --bundle for offline
16	evidence	export, verify, store, status, card, reindex, verify-continuity, migrate, holds, audit, replay, proof, provenance, seal, push-referrer, list-referrers	Yes (exit 0)	--help only	0	16 subcommands
17	seal	(none -- leaf with <image> arg)	Yes (exit 0)	--help only	0	Requires <image> argument
18	drift	(none -- leaf with <image> arg)	Yes (exit 0)	--help only	0	Requires <image> argument; has --fail-on-breach
19	golden	init, validate, import, list, show, build-index	Yes (exit 0)	golden list	1	Expected: "Corpus directory not found: ./golden-corpus"
20	verify-fix	(none -- leaf with <vuln-id> arg)	Yes (exit 0)	--help only	0	Requires <vuln-id>, --pre, --post; supports --attest
21	change-trace	build, export, verify	Yes (exit 0)	--help only	0
22	doctor	run, list, export, fix	Yes (exit 0)	doctor list	0	Lists 23 checks (Core/Database/Security categories)
23	ts	rfc3161, verify, info	Yes (exit 0)	--help only	0	RFC-3161 timestamp ops
24	explain	block	Yes (exit 0)	--help only	0	block requires <digest> arg
25	setup	run, resume, status, reset, validate	Yes (exit 0)	--help only (interactive)	0	Has --non-interactive flag; skipped interactive run

Behavioral Test Details

doctor list (exit 0)

Lists 23 diagnostic checks across 3 categories:

• Core (9 checks): auth.config, config.loaded, config.required, crypto.available, env.diskspace, env.memory, env.variables, services.dependencies, services.health
• Database (8 checks): connection, latency, migrations.failed, migrations.pending, permissions, pool.health, pool.size, schema.version
• Security (6 checks): binaryanalysis.buildinfo.cache, corpus.kpi.baseline, corpus.mirror.freshness, ddeb.enabled, debuginfod.available, symbol.recovery.fallback

ci list (exit 0)

Outputs formatted table with 12 CI/CD templates:

• Platforms: github, gitlab, gitea
• Templates per platform: gate, scan, verify, full

golden list (exit 1)

Expected error: "Corpus directory not found: ./golden-corpus" This is correct behavior -- no golden corpus exists in the working directory.

fmap --help (exit 0)

Alias for function-map works correctly, shows identical help output.

Notes

1. SM Remote Probe: All commands show a ~4s timeout connecting to localhost:56080 (SM remote crypto service). This is expected in dev environments without SM remote running.
2. No crashes or hangs: All 24 commands completed within timeout.
3. setup was tested with --help only to avoid interactive mode. It supports --non-interactive and --config for automated runs.
4. doctor was tested with list subcommand (safe, non-destructive) rather than run to avoid executing actual diagnostic checks.
5. prove is a leaf command (no subcommands) that requires --image flag.
6. evidence has the most subcommands (16) of any command in this batch.