doctor enhancements, setup, enhancements, ui functionality and design consolidation and , test projects fixes , product advisory attestation/rekor and delta verfications enhancements

2026-01-19 09:02:59 +02:00
parent 8c4bf54aed
commit 17419ba7c4
809 changed files with 170738 additions and 12244 deletions
--- a/docs/modules/cli/AGENTS.md
+++ b/docs/modules/cli/AGENTS.md
@@ -3,11 +3,45 @@
 ## Mission
 The `stella` CLI is the operator-facing Swiss army knife for scans, exports, policy management, offline kit operations, and automation scripting.

+## Active Work: CLI Consolidation (v2.x → v3.0)
+
+The CLI is undergoing a major consolidation to improve discoverability and consistency. See:
+
+- **Advisory:** `docs-archived/product/advisories/CLI_CONSOLIDATION_PROPOSAL.md`
+- **Command Mapping:** `docs-archived/product/advisories/CLI_COMMAND_MAPPING.md`
+- **Migration Guide:** `docs/modules/cli/guides/migration-v3.md`
+
+### Consolidation Sprints
+
+| Sprint | Scope | Status |
+|--------|-------|--------|
+| `SPRINT_20260118_010_CLI_consolidation_foundation` | Routing infrastructure, deprecation system | **DONE** |
+| `SPRINT_20260118_011_CLI_settings_consolidation` | `stella config` unified settings | **DONE** |
+| `SPRINT_20260118_012_CLI_verification_consolidation` | `stella verify` unified verification | **DONE** |
+| `SPRINT_20260118_013_CLI_scanning_consolidation` | `stella scan` unified scanning | **DONE** |
+| `SPRINT_20260118_014_CLI_evidence_remaining_consolidation` | Evidence, reachability, SBOM, crypto, etc. | TODO |
+
+### Key Changes
+
+- **81+ → 18 top-level commands** for discoverability
+- **Unified settings under `stella config`** (notify, feeds, registry, integrations)
+- **Unified verification under `stella verify`** (attestation, vex, patch, sbom)
+- **Compound commands split** (`scangraph` → `scan graph`)
+- **Backward compatibility** via deprecated aliases
+
+### Implementation Priorities
+
+1. Foundation (routing, deprecation) must complete first
+2. Sprints 011-014 can run in parallel after foundation
+3. All old commands kept as deprecated aliases until v3.0
+4. Tests must verify both old and new paths
+
 ## Key docs
 - [Module README](./README.md)
 - [Architecture](./architecture.md)
 - [Implementation plan](./implementation_plan.md)
 - [Task board](./TASKS.md)
+- [Migration Guide v3](./guides/migration-v3.md)

 ## How to get started
 1. Open sprint file `/docs/implplan/SPRINT_*.md` and locate the stories referencing this module.
--- a/docs/modules/cli/architecture.md
+++ b/docs/modules/cli/architecture.md
@@ -41,7 +41,72 @@ src/

 **Plug-in verbs.** Non-core verbs (Excititor, runtime helpers, future integrations) ship as restart-time plug-ins under `plugins/cli/**` with manifest descriptors. The launcher loads plug-ins on startup; hot reloading is intentionally unsupported. The inaugural bundle, `StellaOps.Cli.Plugins.NonCore`, packages the Excititor, runtime, and offline-kit command groups and publishes its manifest at `plugins/cli/StellaOps.Cli.Plugins.NonCore/`.

-**OS targets**: linuxâ€‘x64/arm64, windowsâ€‘x64/arm64, macOSâ€‘x64/arm64.
+**OS targets**: linuxâ€'x64/arm64, windowsâ€'x64/arm64, macOSâ€'x64/arm64.
+
+---
+
+## 1.1) Command Routing Infrastructure (v2.x→v3.0 Migration)
+
+> Sprint: SPRINT_20260118_010_CLI_consolidation_foundation
+
+The CLI includes a **command routing infrastructure** to support backward-compatible command migration. This enables consolidating 81+ top-level commands into ~18 organized command groups while maintaining backward compatibility.
+
+### Routing Components
+
+```
+src/Cli/StellaOps.Cli/Infrastructure/
+├── ICommandRouter.cs           # Router interface
+├── CommandRouter.cs            # Route registration and lookup
+├── CommandRoute.cs             # Route model (old→new path mapping)
+├── CommandGroupBuilder.cs      # Fluent builder for command groups
+├── DeprecationWarningService.cs # Warning display on stderr
+├── RouteMappingConfiguration.cs # JSON config model + loader
+
+src/Cli/StellaOps.Cli/
+└── cli-routes.json             # Embedded route mappings (60+ entries)
+```
+
+### How Routing Works
+
+1. **At startup**, `CommandFactory.RegisterDeprecatedAliases()` loads `cli-routes.json` (embedded resource)
+2. **For each deprecated route**, creates a hidden alias command that:
+   - Delegates to the canonical command
+   - Shows a deprecation warning on stderr (once per session)
+3. **Warnings** include the old path, new path, removal version, and suppression instructions
+
+### Route Configuration Schema
+
+```json
+{
+  "version": "1.0",
+  "mappings": [
+    {
+      "old": "scangraph",
+      "new": "scan graph",
+      "type": "deprecated",
+      "removeIn": "3.0",
+      "reason": "Consolidated under scan command"
+    }
+  ]
+}
+```
+
+### Deprecation Warning Format
+
+```
+WARNING: 'stella scangraph' is deprecated and will be removed in v3.0.
+         Use 'stella scan graph' instead.
+         Set STELLA_SUPPRESS_DEPRECATION_WARNINGS=1 to hide this message.
+```
+
+### Timeline
+
+- **v2.x**: Both old and new command paths work; old paths show deprecation warnings
+- **v3.0**: Old command paths removed
+
+### Migration Guide
+
+See [migration-v3.md](./guides/migration-v3.md) for user-facing migration instructions and command mappings.

 ---

@@ -174,12 +239,12 @@ Both subcommands honour offline-first expectations (no network access) and norma
  * Uses `STELLAOPS_ADVISORYAI_URL` when configured; otherwise it reuses the backend base address and adds `X-StellaOps-Scopes` (`advisory:run` + task scope) per request.
  * `--timeout 0` performs a single cache lookup (for CI flows that only want cached artefacts).

-* `advise ask "<question>" [--evidence] [--no-action] [--conversation-id <id>] [--context <cve|scan|image>]`
-
-  * Calls advisory chat endpoints, returns a cited answer with evidence refs.
-  * `--no-action` disables action proposals; `--evidence` forces evidence chips in output.
-
-### 2.12 Decision evidence (new)
+* `advise ask "<question>" [--evidence] [--no-action] [--conversation-id <id>] [--context <cve|scan|image>]`
+
+  * Calls advisory chat endpoints, returns a cited answer with evidence refs.
+  * `--no-action` disables action proposals; `--evidence` forces evidence chips in output.
+
+### 2.12 Decision evidence (new)

 - `decision export`

--- a/docs/modules/cli/guides/migration-v3.md
+++ b/docs/modules/cli/guides/migration-v3.md
@@ -0,0 +1,350 @@
+# CLI Migration Guide: v2.x to v3.0
+
+This guide documents the CLI command consolidation that begins in v2.x (with deprecation warnings) and completes in v3.0 (old commands removed).
+
+---
+
+## Overview
+
+The Stella CLI has been reorganized for better discoverability and consistency:
+
+| Change | Reason |
+|--------|--------|
+| 81+ top-level commands → 18 | Easier to discover and remember |
+| Scattered settings → `stella config` | Unified configuration management |
+| Multiple verify commands → `stella verify` | Consistent verification interface |
+| Compound names → proper hierarchy | `scangraph` → `scan graph` |
+
+## Deprecation Timeline
+
+- **v2.x**: Old commands work but show deprecation warnings
+- **v3.0**: Old commands removed
+
+To suppress deprecation warnings during transition:
+```bash
+export STELLA_SUPPRESS_DEPRECATION_WARNINGS=1
+```
+
+---
+
+## Quick Migration Reference
+
+### Settings & Configuration
+
+```bash
+# Before (deprecated)
+stella notify channels list
+stella admin feeds status
+stella registry list
+
+# After
+stella config notify channels list
+stella config feeds status
+stella config registry list
+```
+
+### Verification
+
+```bash
+# Before (deprecated)
+stella attest verify <artifact>
+stella vex verify <artifact>
+stella patchverify <artifact>
+
+# After
+stella verify attestation <artifact>
+stella verify vex <artifact>
+stella verify patch <artifact>
+```
+
+### Scanning
+
+```bash
+# Before (deprecated)
+stella scangraph list
+stella secrets bundle create <dir>
+stella image inspect <ref>
+
+# After
+stella scan graph list
+stella scan secrets bundle create <dir>
+stella scan image inspect <ref>
+```
+
+### Evidence & Audit
+
+```bash
+# Before (deprecated)
+stella evidenceholds list
+stella audit export
+stella prove --artifact <ref>
+stella replay run
+
+# After
+stella evidence holds list
+stella evidence audit export
+stella evidence proof generate --artifact <ref>
+stella evidence replay run
+```
+
+### Reachability
+
+```bash
+# Before (deprecated)
+stella reachgraph list
+stella slice create
+stella witness show <path>
+
+# After
+stella reachability graph list
+stella reachability slice create
+stella reachability witness show <path>
+```
+
+### SBOM
+
+```bash
+# Before (deprecated)
+stella sbomer compose
+stella layersbom show <digest>
+
+# After
+stella sbom compose
+stella sbom layer show <digest>
+```
+
+### Cryptography
+
+```bash
+# Before (deprecated)
+stella keys list
+stella issuerkeys list
+stella sign image <ref>
+
+# After
+stella crypto keys list
+stella crypto keys issuer list
+stella crypto sign image <ref>
+```
+
+### Administration
+
+```bash
+# Before (deprecated)
+stella doctor run
+stella db migrate
+stella admin users list
+
+# After
+stella admin doctor run
+stella admin db migrate
+stella auth users list
+```
+
+### CI/CD
+
+```bash
+# Before (deprecated)
+stella gate evaluate
+stella github upload
+
+# After (either works)
+stella release gate evaluate
+stella ci gate evaluate       # shortcut for CI pipelines
+stella ci github upload
+```
+
+### Utilities
+
+```bash
+# Before (deprecated)
+stella binary diff
+stella hlc show
+stella timeline query
+
+# After
+stella tools binary diff
+stella tools hlc show
+stella tools timeline query
+```
+
+---
+
+## New Command Structure
+
+### Primary Commands
+
+```
+stella scan           # Scanning operations
+stella release        # Release management
+stella verify         # All verification
+stella attest         # Create attestations
+stella evidence       # Evidence management
+stella policy         # Policy management
+stella vex            # VEX operations
+stella reachability   # Reachability analysis
+stella sbom           # SBOM operations
+stella crypto         # Cryptography
+stella config         # Settings & configuration
+stella auth           # Authentication
+stella admin          # Administration
+stella ci             # CI/CD integration
+stella setup          # Initial setup
+stella explain        # Explain decisions
+stella tools          # Utility commands
+```
+
+### `stella config` - Unified Settings
+
+All configuration is now under `stella config`:
+
+```
+stella config
+├── list [--category <cat>]     # List config paths
+├── show <path>                  # Show config value
+├── set <path> <value>           # Set config value
+├── export                       # Export all config
+├── import <file>                # Import config
+├── notify/                      # Notification settings
+│   ├── channels list/test
+│   ├── templates list/render
+│   └── preferences export/import
+├── feeds/                       # Feed configuration
+│   ├── list
+│   ├── status
+│   └── refresh
+├── integrations/                # Integration settings
+│   ├── list
+│   └── test
+├── registry/                    # Registry settings
+└── sources/                     # Data sources
+```
+
+### `stella verify` - Unified Verification
+
+All verification under one command:
+
+```
+stella verify
+├── image <ref>              # Image attestation
+├── bundle <path>            # Evidence bundle
+├── offline <artifact>       # Offline verification
+├── attestation <artifact>   # Attestation verification
+├── vex <artifact>           # VEX verification
+├── patch <artifact>         # Patch verification
+└── sbom <file>              # SBOM verification
+```
+
+### `stella scan` - Unified Scanning
+
+All scanning under one command:
+
+```
+stella scan
+├── run <ref>                # Run a scan
+├── status <id>              # Check status
+├── results <id>             # View results
+├── download                 # Download scanner bundle
+├── workers                  # Configure workers
+├── graph/                   # Scan graph operations
+├── secrets/                 # Secret detection
+│   └── bundle create/verify/info
+└── image/                   # Image analysis
+    ├── inspect
+    └── layers
+```
+
+---
+
+## CI/CD Script Updates
+
+### GitHub Actions
+
+```yaml
+# Before
+- run: stella gate evaluate --artifact ${{ env.IMAGE_SHA }}
+
+# After (either works)
+- run: stella ci gate evaluate --artifact ${{ env.IMAGE_SHA }}
+# or
+- run: stella release gate evaluate --artifact ${{ env.IMAGE_SHA }}
+```
+
+### GitLab CI
+
+```yaml
+# Before
+script:
+  - stella notify channels test --channel slack-alerts
+
+# After
+script:
+  - stella config notify channels test --channel slack-alerts
+```
+
+### Jenkins
+
+```groovy
+// Before
+sh 'stella scangraph list --format json'
+
+// After
+sh 'stella scan graph list --format json'
+```
+
+---
+
+## Common Errors and Solutions
+
+### "Command not found" in v3.0
+
+If upgrading to v3.0 and a command fails:
+
+```bash
+$ stella scangraph list
+Error: Unknown command 'scangraph'. Did you mean 'scan graph'?
+```
+
+Update your script to use the new path.
+
+### "Deprecated command" warnings
+
+```
+WARNING: 'stella notify' is deprecated and will be removed in v3.0.
+         Use 'stella config notify' instead.
+```
+
+This is informational. The command still works but should be updated.
+
+### Suppressing warnings in CI
+
+```bash
+export STELLA_SUPPRESS_DEPRECATION_WARNINGS=1
+stella notify channels list  # No warning
+```
+
+---
+
+## Getting Help
+
+```bash
+# See all commands
+stella --help
+
+# See subcommands
+stella config --help
+stella verify --help
+
+# See command details
+stella config notify channels list --help
+```
+
+---
+
+## Migration Checklist
+
+- [ ] Update CI/CD pipelines to use new command paths
+- [ ] Update documentation referencing CLI commands
+- [ ] Update automation scripts
+- [ ] Test with `STELLA_SUPPRESS_DEPRECATION_WARNINGS=0` to find deprecated usage
+- [ ] Plan upgrade to v3.0 before end-of-support for v2.x
--- a/docs/modules/cli/guides/setup-guide.md
+++ b/docs/modules/cli/guides/setup-guide.md
@@ -0,0 +1,269 @@
+# Setup Wizard Guide
+
+This guide covers the `stella setup` command for initial configuration of Stella Ops.
+
+## Overview
+
+The setup wizard guides you through configuring all required and optional components. Both CLI and UI setup wizards follow the same **Infrastructure-First** order and provide identical capabilities.
+
+## Quick Start
+
+```bash
+# Interactive setup
+stella setup run
+
+# Non-interactive with config file
+stella setup run --config setup.yaml --non-interactive
+
+# Dry-run mode (validate without applying)
+stella setup run --dry-run
+
+# Resume interrupted setup
+stella setup resume
+
+# Reconfigure a specific step
+stella setup --step vault
+```
+
+## Setup Steps
+
+Steps are organized in phases. Required steps must be completed; optional steps can be skipped.
+
+### Phase 1: Core Infrastructure (Required)
+
+| Step | Description |
+|------|-------------|
+| **database** | PostgreSQL connection for persistent storage |
+| **cache** | Valkey/Redis connection for caching and distributed locks |
+| **migrations** | Apply database schema migrations |
+
+### Phase 2: Security Foundation (Required)
+
+| Step | Description |
+|------|-------------|
+| **authority** | Authentication provider (Standard or LDAP) |
+| **users** | Initial super user account (skipped if LDAP selected) |
+| **crypto** | Cryptographic provider for signing/encryption (Default, FIPS, GOST, SM2/SM3) |
+
+### Phase 3: Secrets Management (Optional)
+
+| Step | Description | Configure Later |
+|------|-------------|-----------------|
+| **vault** | External secrets vault (HashiCorp Vault, Azure Key Vault, AWS Secrets Manager, GCP Secret Manager) | Settings > Trust & Signing, or `stella config set vault.*` |
+
+### Phase 4: Integrations (Optional)
+
+| Step | Description | Configure Later |
+|------|-------------|-----------------|
+| **registry** | Container registry for image scanning | Settings > Integrations, or `stella config set registry.*` |
+| **scm** | Source control integration (GitHub, GitLab, Gitea, Bitbucket, Azure DevOps) | Settings > Integrations, or `stella config set scm.*` |
+| **sources** | Advisory data sources (NVD, GHSA, OSV, distribution feeds) | Settings > Security Data, or `stella config set sources.*` |
+
+### Phase 5: Observability (Optional)
+
+| Step | Description | Configure Later |
+|------|-------------|-----------------|
+| **telemetry** | OpenTelemetry configuration for tracing, metrics, and logging | Settings > System > Telemetry, or `stella config set telemetry.*` |
+| **notify** | Notification channels (Email, Slack, Teams, Webhook) | Settings > Notifications, or `stella config set notify.*` |
+
+### Phase 6: AI Features (Optional)
+
+| Step | Description | Configure Later |
+|------|-------------|-----------------|
+| **llm** | AI/LLM provider for AdvisoryAI (OpenAI, Claude, Gemini, Ollama) | Settings > Integrations > AdvisoryAI, or `stella config set llm.*` |
+
+### Phase 7: Configuration Store (Optional)
+
+| Step | Description | Configure Later |
+|------|-------------|-----------------|
+| **settingsStore** | External configuration store (Consul, etcd, Azure App Config, AWS Parameter Store) | Settings > System, or `stella config set settingsStore.*` |
+
+### Phase 8: Release Orchestration (Optional)
+
+| Step | Description | Configure Later |
+|------|-------------|-----------------|
+| **environments** | Define deployment environments (dev, staging, production) | Settings > Environments, or `stella env create` |
+| **agents** | Register deployment agents for release execution | Settings > Agents, or `stella agent register` |
+
+## Multiple Integrations
+
+The **registry**, **scm**, and **notify** steps support configuring multiple instances. For example:
+
+```bash
+# Add multiple container registries
+stella config set registry.instances.0.name "Production ECR"
+stella config set registry.instances.0.provider "ecr"
+stella config set registry.instances.0.isPrimary "true"
+
+stella config set registry.instances.1.name "Docker Hub"
+stella config set registry.instances.1.provider "docker"
+
+# Add multiple SCM connections
+stella config set scm.instances.0.name "GitHub Main"
+stella config set scm.instances.0.provider "github"
+
+# Add multiple notification channels
+stella config set notify.instances.0.name "Ops Slack"
+stella config set notify.instances.0.provider "slack"
+
+stella config set notify.instances.1.name "Security Email"
+stella config set notify.instances.1.provider "email"
+```
+
+## Skip Warnings
+
+When skipping optional steps, the wizard displays warnings about implications:
+
+| Skipped Step | Warning |
+|--------------|---------|
+| vault | Secrets stored in configuration files (less secure for production) |
+| registry | Container scanning capabilities limited |
+| scm | Pipeline integration and automated workflows unavailable |
+| sources | CVE/VEX advisory feeds require manual updates |
+| telemetry | System observability limited; tracing and metrics unavailable |
+| llm | AdvisoryAI features unavailable |
+| environments | Manual deployment tracking only |
+| agents | Release orchestration unavailable without registered agents |
+
+## Cryptographic Provider Selection
+
+The **crypto** step allows selecting regional cryptographic standards:
+
+| Provider | Standards | Use Case |
+|----------|-----------|----------|
+| **Default** | AES-256-GCM, SHA-256/512, Ed25519, ECDSA P-256 | General use |
+| **FIPS 140-2** | AES-256-GCM (FIPS 197), SHA-256/384/512 (FIPS 180-4), ECDSA P-256/P-384 (FIPS 186-4) | US government compliance |
+| **GOST R 34.10-2012** | Kuznechik/Magma, Streebog, GOST R 34.10-2012 | Russian compliance |
+| **SM2/SM3** | SM4, SM3, SM2 | Chinese national standards |
+
+FIPS mode supports HSM integration via PKCS#11, AWS CloudHSM, Azure Key Vault HSM, or GCP Cloud HSM.
+
+## SCM Integration
+
+The **scm** step connects Stella Ops to your source control system:
+
+| Provider | Authentication |
+|----------|----------------|
+| GitHub | Personal Access Token (ghp_...) |
+| GitLab | Personal Access Token (glpat-...) |
+| Gitea | Access Token |
+| Bitbucket | Username + App Password |
+| Azure DevOps | Personal Access Token |
+
+## Configuration File Format
+
+For non-interactive setup, provide a YAML configuration file:
+
+```yaml
+# setup.yaml
+database:
+  host: localhost
+  port: 5432
+  database: stellaops
+  user: postgres
+  password: ${DB_PASSWORD}  # Environment variable substitution
+  ssl: true
+
+cache:
+  host: localhost
+  port: 6379
+  password: ${CACHE_PASSWORD}
+  ssl: true
+
+authority:
+  provider: standard  # or 'ldap'
+
+users:
+  superuser:
+    username: admin
+    email: admin@example.com
+    password: ${ADMIN_PASSWORD}
+
+crypto:
+  provider: default  # or 'fips', 'gost', 'sm'
+
+vault:
+  provider: hashicorp
+  address: https://vault.example.com:8200
+  token: ${VAULT_TOKEN}
+
+scm:
+  provider: github
+  url: https://github.com
+  token: ${GITHUB_TOKEN}
+  organization: my-org
+
+sources:
+  enabled: nvd,ghsa,osv
+  nvd:
+    apiKey: ${NVD_API_KEY}
+
+telemetry:
+  otlpEndpoint: http://localhost:4317
+  enableTracing: true
+  enableMetrics: true
+
+notify:
+  provider: slack
+  slack:
+    webhookUrl: ${SLACK_WEBHOOK_URL}
+
+llm:
+  provider: openai
+  openai:
+    apiKey: ${OPENAI_API_KEY}
+    model: gpt-4o
+```
+
+## Validation Commands
+
+```bash
+# Validate current configuration
+stella setup validate
+
+# Validate specific step
+stella setup validate --step database
+
+# Show current setup status
+stella setup status
+```
+
+## Troubleshooting
+
+### Database Connection Failed
+
+```bash
+# Test PostgreSQL connectivity
+stella setup validate --step database --verbose
+```
+
+Verify:
+- PostgreSQL is running and accessible
+- Credentials are correct
+- SSL settings match server configuration
+
+### Cache Connection Failed
+
+```bash
+# Test Valkey/Redis connectivity
+stella setup validate --step cache --verbose
+```
+
+### SCM Authentication Failed
+
+```bash
+# Test SCM connectivity
+stella setup validate --step scm --verbose
+```
+
+Ensure your token has the required scopes:
+- GitHub: `repo`, `workflow`
+- GitLab: `api`, `read_repository`
+- Azure DevOps: `Code (Read)`, `Build (Read & Execute)`
+
+## Related Commands
+
+- `stella config get` - View current configuration
+- `stella config set` - Modify individual settings
+- `stella doctor run` - Run diagnostic checks
+- `stella admin db migrate` - Run database migrations