Stabilize U

docs/qa/feature-checks/runs/cli/cli-e2e-tests/batch-a-results.md

@@ -0,0 +1,115 @@
+# CLI Batch A -- E2E Test Results
+
+**Date:** 2026-02-15
+**Agent:** batch-a
+**CLI Project:** `src/Cli/StellaOps.Cli/StellaOps.Cli.csproj`
+**Configuration:** Release (pre-built, `--no-build`)
+**Environment note:** SM remote probe fails (expected -- no SM remote service running). Adds ~4s startup latency per invocation.
+
+---
+
+## Top-Level Command Summary
+
+| # | Command | Description | Subcommands | --help OK | Behavioral Test | Exit Code | Notes |
+|---|---------|-------------|-------------|-----------|-----------------|-----------|-------|
+| 1 | `scanner` | Manage scanner artifacts and lifecycle | `download`, `workers` | YES | N/A (container-dependent) | 0 | 2 subcommands |
+| 2 | `scan` | Execute scanners and manage scan outputs | `entrytrace`, `sarif`, `replay`, `gate-policy`, `gate-results`, `layers`, `layer-sbom`, `recipe`, `diff`, `delta`, `verify-patches`, `download`, `workers`, `secrets`, `image`, `run`, `upload`, `graph` | YES | N/A (requires scan data) | 0 | 18 subcommands -- richest command |
+| 3 | `image` | OCI image operations | `inspect` | YES | N/A (requires registry) | 0 | 1 subcommand |
+| 4 | `ruby` | Work with Ruby analyzer outputs | `inspect`, `resolve` | YES | `ruby inspect --help` OK | 0 | 2 subcommands |
+| 5 | `php` | Work with PHP analyzer outputs | `inspect` | YES | N/A | 0 | 1 subcommand |
+| 6 | `python` | Work with Python analyzer outputs | `inspect` | YES | N/A | 0 | 1 subcommand |
+| 7 | `bun` | Work with Bun analyzer outputs | `inspect`, `resolve` | YES | N/A | 0 | 2 subcommands |
+| 8 | `db` | Trigger Concelier database operations | `fetch`, `merge`, `export` | YES | N/A (requires backend) | 0 | 3 subcommands |
+| 9 | `sources` | Interact with source ingestion workflows | `ingest`, `list`, `check`, `enable`, `disable`, `status` | YES | `sources list` CRASH (exit 1), `sources status` CRASH (exit 1) | 0 (help) / 1 (run) | **BUG: ISourceRegistry not registered in DI** |
+| 10 | `aoc` | Aggregation-Only Contract verification | `verify` | YES | `aoc verify` exits 71 (tenant required) | 0 (help) / 71 (run) | Correct error: requires `--tenant` |
+| 11 | `auth` | Manage authentication | `login`, `logout`, `status`, `whoami`, `revoke`, `token` | YES | `auth status` exits 1 (authority not configured) | 0 (help) / 1 (run) | Expected: no Authority URL configured |
+| 12 | `tenants` | Manage tenant contexts | `list`, `use`, `current`, `clear` | YES | `tenants current` exits 0: "No active tenant configured." | 0 | Correct offline behavior |
+| 13 | `policy` | Interact with Policy Engine | `simulate`, `activate`, `lint`, `edit`, `test`, `new`, `history`, `explain`, `init`, `compile`, `version`, `submit`, `review`, `publish`, `rollback`, `sign`, `verify-signature`, `lattice`, `verdicts`, `promote`, `validate-yaml`, `install`, `list-packs`, `export`, `import`, `validate`, `evaluate` | YES | `policy lint /nonexistent.stella` exits 4 (file not found) | 0 (help) / 4 (lint) | 27 subcommands; correct error for missing file |
+| 14 | `tools` | Local policy tooling | `policy-dsl-validate`, `policy-schema-export`, `policy-simulation-smoke`, `lint`, `benchmark`, `migrate` | YES | N/A | 0 | 6 subcommands; benchmark has sub-subs (policy/scan/crypto) |
+| 15 | `task-runner` | Interact with Task Runner | `simulate` | YES | N/A | 0 | 1 subcommand |
+| 16 | `findings` | Inspect policy findings | `ls`, `get`, `explain` | YES | `findings ls` exits 1 (--policy required) | 0 (help) / 1 (run) | Correct: shows required option hint |
+| 17 | `advise` | Advisory AI pipelines | `run`, `summarize`, `explain`, `remediate`, `batch`, `open-pr`, `ask`, `chat-doctor`, `chat-settings`, `export` | YES | `advise run --help` OK | 0 | 10 subcommands |
+| 18 | `config` | Manage configuration | `show`, `list`, `notify`, `integrations`, `feeds`, `registry`, `sources`, `signals` | YES | `config show` exits 0 (shows defaults), `config list` exits 0 (lists paths) | 0 | 8 subcommands; behavioral tests pass |
+| 19 | `kms` | Manage signing keys | `export`, `import` | YES | Both `--help` OK | 0 | 2 subcommands |
+| 20 | `key` | Key management | `list`, `add`, `revoke`, `rotate`, `status`, `history`, `verify` | YES | N/A (requires anchorId) | 0 | 7 subcommands |
+| 21 | `issuer` | Issuer key management | `keys` (sub: `list`, `create`, `rotate`, `revoke`) | YES | `issuer keys --help` OK | 0 | Nested: keys has 4 sub-subcommands |
+
+---
+
+## Subcommand --help Verification
+
+| Parent | Subcommand | --help OK | Exit Code | Notes |
+|--------|------------|-----------|-----------|-------|
+| `scanner` | `download` | YES | 0 | Options: --channel, --output, --overwrite, --no-install |
+| `scanner` | `workers` | YES | 0 | Sub-subcommands: get, set |
+| `scan` | `entrytrace` | YES | 0 | Options: --scan-id (required), --include-ndjson, --semantic |
+| `scan` | `sarif` | YES | 0 | Options: --scan-id (required), -o, --pretty, --include-hardening, --include-reachability, --min-severity |
+| `scan` | `replay` | YES | 0 | Options: --artifact (req), --manifest (req), --feeds (req), --policy (req), --offline, --verify-inputs |
+| `scan` | `secrets` | YES | 0 | Sub-subcommand: bundle |
+| `scan` | `graph` | YES | 0 | Options: --lang (req), --target (req), --format, --upload, --include-tests |
+| `image` | `inspect` | YES | 0 | Options: -r, -l, -p platform, -o format, --timeout |
+| `auth` | `login` | YES | 0 | Options: --force |
+| `auth` | `status` | YES | 0 | No extra options |
+| `auth` | `whoami` | YES | 0 | No extra options |
+| `db` | `fetch` | YES | 0 | Options: --source (req), --stage, --mode |
+| `db` | `merge` | YES | 0 | No extra options |
+| `db` | `export` | YES | 0 | Options: --format, --delta, --publish-full, --publish-delta, --bundle-full, --bundle-delta |
+| `policy` | `lint` | YES | 0 | Args: file; Options: -f, -o |
+| `policy` | `new` | YES | 0 | Args: name; Options: -t template, -o, -d, --tag, --shadow, --fixtures, --git-init |
+| `policy` | `compile` | YES | 0 | Args: file; Options: -o, --no-ir, --no-digest, --optimize, --strict |
+| `policy` | `validate-yaml` | YES | 0 | Args: path; Options: --schema, --strict |
+| `policy` | `list-packs` | YES | 0 | Options: --source |
+| `policy` | `evaluate` | YES | 0 | Options: -p policy (req), -i input (req), --format, -e environment, --include-remediation |
+| `tenants` | `list` | YES | 0 | Options: --tenant, --json |
+| `tenants` | `use` | YES | 0 | Args: tenant-id |
+| `tenants` | `clear` | YES | 0 | No extra options |
+| `tools` | `lint` | YES | 0 | Options: -i input (req), --fix, --strict, -f format |
+| `tools` | `benchmark` | YES | 0 | Sub-subcommands: policy, scan, crypto |
+| `tools` | `migrate` | YES | 0 | Sub-subcommands: config, data |
+| `task-runner` | `simulate` | YES | 0 | Options: --manifest, --inputs, --format, --output |
+| `kms` | `export` | YES | 0 | Options: --root, --key-id (req), --version, --output (req), --force, --passphrase |
+| `kms` | `import` | YES | 0 | Options: --root, --key-id (req), --input (req), --version, --passphrase |
+| `issuer` | `keys` | YES | 0 | Sub-subcommands: list, create, rotate, revoke |
+| `advise` | `run` | YES | 0 | Args: task; Options: --advisory-key (req), many more |
+| `findings` | `ls` | YES (via error) | 1 | Shows help with required --policy hint |
+| `config` | `show` | YES | 0 | No extra options |
+
+---
+
+## Behavioral Test Results
+
+| Command | Invocation | Exit Code | Behavior | Verdict |
+|---------|------------|-----------|----------|---------|
+| `auth status` | `auth status` | 1 | "Authority URL not configured. Set STELLAOPS_AUTHORITY_URL and run 'auth login'." | PASS -- correct error |
+| `tenants current` | `tenants current` | 0 | "No active tenant configured. Use 'stella tenants use <tenant-id>' to set one." | PASS -- correct offline |
+| `config show` | `config show` | 0 | Shows all config keys with defaults (Backend URL, Concelier URL, API Key, etc.) | PASS -- works offline |
+| `config list` | `config list` | 0 | Lists all config paths grouped by section (notify, feeds, integrations, etc.) | PASS -- works offline |
+| `sources list` | `sources list` | 1 | **CRASH: `InvalidOperationException: No service for type 'ISourceRegistry' has been registered.`** | FAIL -- DI bug |
+| `sources status` | `sources status` | 1 | **CRASH: Same `ISourceRegistry` DI exception** | FAIL -- DI bug |
+| `aoc verify` | `aoc verify` | 71 | "Tenant must be provided via --tenant or STELLA_TENANT." | PASS -- correct validation |
+| `policy lint` | `policy lint /nonexistent.stella` | 4 | "Error: Policy file not found: .../nonexistent.stella" | PASS -- correct file-not-found |
+| `findings ls` | `findings ls` | 1 | "Option '--policy' is required." + help text | PASS -- correct validation |
+
+---
+
+## Bugs Found
+
+### BUG-001: `sources list` and `sources status` crash with DI exception
+
+**Severity:** Medium
+**Commands affected:** `sources list`, `sources status`
+**Error:** `System.InvalidOperationException: No service for type 'StellaOps.Concelier.Core.Sources.ISourceRegistry' has been registered.`
+**Location:** `src/Cli/StellaOps.Cli/Commands/Sources/SourcesCommandHandlers.cs:line 35` (list), `line 332` (status)
+**Root cause:** The `ISourceRegistry` service is not registered in the CLI's DI container. The `sources --help` works fine, but actual invocation fails.
+**Impact:** Users cannot list or check status of advisory sources via CLI without backend connectivity.
+
+---
+
+## Summary
+
+- **21/21 commands** have working `--help` (exit 0)
+- **All subcommand --help** tests pass (30+ subcommands tested)
+- **9 behavioral tests** run: 7 PASS, 2 FAIL
+- **1 bug found:** `sources list`/`sources status` DI registration missing for `ISourceRegistry`
+- **Total subcommands discovered:** 100+ across all 21 top-level commands
+- **Richest commands:** `policy` (27 subcmds), `scan` (18 subcmds), `advise` (10 subcmds), `config` (8 subcmds)