stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

tests fixes and sprints work

Browse Source
This commit is contained in:
master
2026-01-22 19:08:46 +02:00
parent c32fff8f86
commit 726d70dc7f
881 changed files with 134434 additions and 6228 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
docs/modules/cli/guides/commands/analytics.md Normal file
View File

@@ -0,0 +1,47 @@
# stella analytics - Command Guide
## Commands
- `stella analytics sbom-lake suppliers [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake licenses [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake vulnerabilities [--environment <env>] [--min-severity <level>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake backlog [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake attestation-coverage [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake trends [--environment <env>] [--days <n>] [--series vulnerabilities|components|all] [--limit <n>] [--format table|json|csv] [--output <path>]`
## Flags (common)
- `--format`: Output format for rendering (`table`, `json`, `csv`).
- `--output`: Write output to a file path instead of stdout.
- `--limit`: Cap the number of rows returned.
- `--environment`: Filter by environment name.
## SBOM lake notes
- Endpoints require the `analytics.read` scope.
- `--min-severity` accepts `critical`, `high`, `medium`, `low`.
- `--series` controls trend output (`vulnerabilities`, `components`, `all`).
- Tables use deterministic ordering (severity and counts first, then names).
## Examples
```bash
# Top suppliers
stella analytics sbom-lake suppliers --limit 20
# License distribution as CSV (prod)
stella analytics sbom-lake licenses --environment prod --format csv --output licenses.csv
# Vulnerability exposure in prod (high+)
stella analytics sbom-lake vulnerabilities --environment prod --min-severity high
# Fixable backlog with table output
stella analytics sbom-lake backlog --environment prod --limit 50
# Attestation coverage in staging, JSON output
stella analytics sbom-lake attestation-coverage --environment stage --format json
# 30-day trend snapshot (both series)
stella analytics sbom-lake trends --days 30 --series all --format csv --output trends.csv
```
## Offline/verification note
- If analytics exports arrive via offline bundles, verify the bundle first with
`stella bundle verify` before importing data into downstream reports.

596
docs/modules/cli/guides/commands/reference.md
View File

@@ -16,6 +16,7 @@ graph TD
CLI --> EXPLAIN[Explainability]
CLI --> VEX[VEX & Decisioning]
CLI --> SBOM[SBOM Operations]
CLI --> ANALYTICS[Analytics & Insights]
CLI --> REPORT[Reporting & Export]
CLI --> OFFLINE[Offline Operations]
CLI --> SYSTEM[System & Config]
@@ -742,6 +743,601 @@ stella sbom merge --sbom <path1> --sbom <path2> [--output <path>] [--verbose]
---
## Analytics Commands
### stella analytics sbom-lake
Query SBOM lake analytics views (suppliers, licenses, vulnerabilities, backlog,
attestation coverage, trends).
**Usage:**
```bash
stella analytics sbom-lake <subcommand> [options]
```
**Subcommands:**
- `suppliers` - Supplier concentration
- `licenses` - License distribution
- `vulnerabilities` - CVE exposure (VEX-adjusted)
- `backlog` - Fixable vulnerability backlog
- `attestation-coverage` - Provenance/SLSA coverage
- `trends` - Time-series trends (vulnerabilities/components)
**Common options:**
| Option | Description |
|--------|-------------|
| `--environment <env>` | Filter to a specific environment |
| `--min-severity <level>` | Minimum severity (`critical`, `high`, `medium`, `low`) |
| `--days <n>` | Lookback window in days (trends only) |
| `--series <name>` | Trend series (`vulnerabilities`, `components`, `all`) |
| `--limit <n>` | Maximum number of rows |
| `--format <fmt>` | Output format: `table`, `json`, `csv` |
| `--output <path>` | Output file path |
**Example:**
```bash
stella analytics sbom-lake vulnerabilities --environment prod --min-severity high --format csv --output vuln.csv
```
---
## Ground-Truth Corpus Commands
### stella groundtruth
Manage ground-truth corpus for patch-paired binary verification. The corpus supports
precision validation of security advisories by maintaining symbol and binary pairs
from upstream sources.
**Sprint:** SPRINT_20260121_035_BinaryIndex_golden_corpus_connectors_cli
**Usage:**
```bash
stella groundtruth <subcommand> [options]
```
**Subcommands:**
- `sources` - Manage symbol source connectors
- `symbols` - Query and search symbols in the corpus
- `pairs` - Manage security pairs (vuln/patch binary pairs)
- `validate` - Run validation and view metrics
---
### stella groundtruth sources
Manage upstream symbol source connectors.
**Usage:**
```bash
stella groundtruth sources <command> [options]
```
**Subcommands:**
#### stella groundtruth sources list
List available symbol source connectors.
```bash
stella groundtruth sources list [--output-format table|json] [--verbose]
```
**Output:**
```
ID Display Name Status Last Sync
------------------------------------------------------------------------------------------
debuginfod-fedora Fedora Debuginfod Enabled 2026-01-22T10:00:00Z
debuginfod-ubuntu Ubuntu Debuginfod Enabled 2026-01-22T10:00:00Z
ddeb-ubuntu Ubuntu ddebs Enabled 2026-01-22T09:30:00Z
buildinfo-debian Debian Buildinfo Enabled 2026-01-22T08:00:00Z
secdb-alpine Alpine SecDB Enabled 2026-01-22T06:00:00Z
```
#### stella groundtruth sources enable
Enable a symbol source connector.
```bash
stella groundtruth sources enable <source> [--verbose]
```
**Arguments:**
- `<source>` - Source connector ID (e.g., `debuginfod-fedora`)
**Example:**
```bash
stella groundtruth sources enable debuginfod-fedora
```
#### stella groundtruth sources disable
Disable a symbol source connector.
```bash
stella groundtruth sources disable <source> [--verbose]
```
#### stella groundtruth sources sync
Synchronize symbol sources from upstream.
```bash
stella groundtruth sources sync [--source <id>] [--full] [--verbose]
```
**Options:**
| Option | Description |
|--------|-------------|
| `--source <id>` | Source connector ID (all if not specified) |
| `--full` | Perform a full sync instead of incremental |
**Example:**
```bash
# Incremental sync of all sources
stella groundtruth sources sync
# Full sync of Debian buildinfo
stella groundtruth sources sync --source buildinfo-debian --full
```
---
### stella groundtruth symbols
Query and search symbols in the corpus.
**Usage:**
```bash
stella groundtruth symbols <command> [options]
```
#### stella groundtruth symbols lookup
Lookup symbols by debug ID (build-id).
```bash
stella groundtruth symbols lookup --debug-id <id> [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Required |
|--------|-------|-------------|----------|
| `--debug-id` | `-d` | Debug ID (build-id) to lookup | Yes |
| `--output-format` | `-O` | Output format: `table`, `json` | No |
**Example:**
```bash
stella groundtruth symbols lookup --debug-id 7f8a9b2c4d5e6f1a --output-format json
```
**Output (table):**
```
Binary: libcrypto.so.3
Architecture: x86_64
Distribution: debian-bookworm
Package: openssl@3.0.11-1
Symbol Count: 4523
Sources: debuginfod-fedora, buildinfo-debian
```
#### stella groundtruth symbols search
Search symbols by package or distribution.
```bash
stella groundtruth symbols search [--package <name>] [--distro <distro>] [--limit <n>] [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--package` | `-p` | Package name to search for | - |
| `--distro` | | Distribution filter (debian, ubuntu, alpine) | - |
| `--limit` | `-l` | Maximum results | 20 |
**Example:**
```bash
stella groundtruth symbols search --package openssl --distro debian --limit 50
```
---
### stella groundtruth pairs
Manage security pairs (vulnerable/patched binary pairs) in the corpus.
**Usage:**
```bash
stella groundtruth pairs <command> [options]
```
#### stella groundtruth pairs create
Create a new security pair.
```bash
stella groundtruth pairs create --cve <cve-id> --vuln-pkg <pkg=ver> --patch-pkg <pkg=ver> [--distro <distro>] [--verbose]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--cve` | CVE identifier | Yes |
| `--vuln-pkg` | Vulnerable package (name=version) | Yes |
| `--patch-pkg` | Patched package (name=version) | Yes |
| `--distro` | Distribution (e.g., `debian-bookworm`) | No |
**Example:**
```bash
stella groundtruth pairs create \
--cve CVE-2024-1234 \
--vuln-pkg openssl=3.0.10-1 \
--patch-pkg openssl=3.0.11-1 \
--distro debian-bookworm
```
#### stella groundtruth pairs list
List security pairs in the corpus.
```bash
stella groundtruth pairs list [--cve <pattern>] [--package <name>] [--limit <n>] [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--cve` | | Filter by CVE (supports wildcards: `CVE-2024-*`) | - |
| `--package` | `-p` | Filter by package name | - |
| `--limit` | `-l` | Maximum results | 50 |
**Example:**
```bash
stella groundtruth pairs list --cve CVE-2024-* --package openssl --limit 100
```
**Output:**
```
Pair ID CVE Package Vuln Version Patch Version
-------------------------------------------------------------------------------
pair-001 CVE-2024-1234 openssl 3.0.10-1 3.0.11-1
pair-002 CVE-2024-5678 curl 8.4.0-1 8.5.0-1
```
#### stella groundtruth pairs delete
Delete a security pair from the corpus.
```bash
stella groundtruth pairs delete <pair-id> [--force] [--verbose]
```
**Options:**
| Option | Alias | Description |
|--------|-------|-------------|
| `--force` | `-f` | Skip confirmation prompt |
---
### stella groundtruth validate
Run validation harness against security pairs.
**Usage:**
```bash
stella groundtruth validate <command> [options]
```
#### stella groundtruth validate run
Run validation on security pairs.
```bash
stella groundtruth validate run [--pairs <pattern>] [--matcher <type>] [--output <path>] [--parallel <n>] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--pairs` | `-p` | Pair filter pattern (e.g., `openssl:CVE-2024-*`) | all |
| `--matcher` | `-m` | Matcher type: `semantic-diffing`, `hash-based`, `hybrid` | `semantic-diffing` |
| `--output` | `-o` | Output file for validation report | - |
| `--parallel` | | Maximum parallel validations | 4 |
**Example:**
```bash
stella groundtruth validate run \
--pairs "openssl:CVE-2024-*" \
--matcher semantic-diffing \
--parallel 8 \
--output validation-report.md
```
**Output:**
```
Validating pairs: 10/10
Validation complete. Run ID: vr-20260122100532
Function Match Rate: 94.2%
False-Negative Rate: 2.1%
SBOM Hash Stability: 3/3
Report written to: validation-report.md
```
#### stella groundtruth validate metrics
View metrics for a validation run.
```bash
stella groundtruth validate metrics --run-id <id> [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Required |
|--------|-------|-------------|----------|
| `--run-id` | `-r` | Validation run ID | Yes |
**Example:**
```bash
stella groundtruth validate metrics --run-id vr-20260122100532 --output-format json
```
**Output (table):**
```
Run ID: vr-20260122100532
Duration: 2026-01-22T10:00:00Z - 2026-01-22T10:15:32Z
Pairs: 48/50 successful
Function Match Rate: 94.2%
False-Negative Rate: 2.1%
SBOM Hash Stability: 3/3
Verify Time (p50/p95): 423ms / 1.2s
```
#### stella groundtruth validate export
Export validation report.
```bash
stella groundtruth validate export --run-id <id> --output <path> [--format <fmt>] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--run-id` | `-r` | Validation run ID | (required) |
| `--output` | `-o` | Output file path | (required) |
| `--format` | `-f` | Export format: `markdown`, `html`, `json` | `markdown` |
**Example:**
```bash
stella groundtruth validate export \
--run-id vr-20260122100532 \
--format markdown \
--output validation-report.md
```
**See Also:** [Ground-Truth CLI Guide](../ground-truth-cli.md)
---
### stella groundtruth bundle
Manage evidence bundles for offline verification of patch provenance.
**Sprint:** SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
**Usage:**
```bash
stella groundtruth bundle <command> [options]
```
**Subcommands:**
- `export` - Create evidence bundles for air-gapped environments
- `import` - Import and verify evidence bundles
#### stella groundtruth bundle export
Export evidence bundles containing pre/post binaries, SBOMs, delta-sig predicates, and timestamps.
```bash
stella groundtruth bundle export [options]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--packages <list>` | Comma-separated package names (e.g., `openssl,curl`) | Yes |
| `--distros <list>` | Comma-separated distributions (e.g., `debian,ubuntu`) | Yes |
| `--output <path>` | Output bundle path (.tar.gz or .oci.tar) | Yes |
| `--sign-with <signer>` | Signing method: `cosign`, `sigstore`, `none` | No |
| `--include-debug` | Include debug symbols | No |
| `--include-kpis` | Include KPI validation results | No |
| `--include-timestamps` | Include RFC 3161 timestamps | No |
**Example:**
```bash
stella groundtruth bundle export \
--packages openssl,zlib,glibc \
--distros debian,fedora \
--output evidence/security-bundle.tar.gz \
--sign-with cosign \
--include-debug \
--include-kpis \
--include-timestamps
```
**Exit Codes:**
- `0` - Bundle created successfully
- `1` - Bundle creation failed
- `2` - Invalid input or configuration error
#### stella groundtruth bundle import
Import and verify evidence bundles in air-gapped environments.
```bash
stella groundtruth bundle import [options]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--input <path>` | Input bundle path | Yes |
| `--verify-signature` | Verify bundle signatures | No |
| `--trusted-keys <path>` | Path to trusted public keys | No |
| `--trust-profile <path>` | Trust profile for verification | No |
| `--output <path>` | Output verification report | No |
| `--format <fmt>` | Report format: `markdown`, `json`, `html` | No |
**Example:**
```bash
stella groundtruth bundle import \
--input symbol-bundle.tar.gz \
--verify-signature \
--trusted-keys /etc/stellaops/trusted-keys.pub \
--trust-profile /etc/stellaops/trust-profiles/global.json \
--output verification-report.md
```
**Verification Steps:**
1. Validate bundle manifest signature
2. Verify all blob digests match manifest
3. Validate DSSE envelope signatures against trusted keys
4. Verify RFC 3161 timestamps against trusted TSA certificates
5. Run IR matcher to confirm patched functions
6. Verify SBOM canonical hash matches signed predicate
7. Output verification report with KPI line items
**Exit Codes:**
- `0` - All verifications passed
- `1` - One or more verifications failed
- `2` - Invalid input or configuration error
---
### stella groundtruth validate check
Check KPI regression against baseline thresholds.
**Sprint:** SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
```bash
stella groundtruth validate check [options]
```
**Options:**
| Option | Description | Default |
|--------|-------------|---------|
| `--results <path>` | Path to validation results JSON | (required) |
| `--baseline <path>` | Path to baseline JSON | (required) |
| `--precision-threshold <pp>` | Max precision drop (percentage points) | 0.01 |
| `--recall-threshold <pp>` | Max recall drop (percentage points) | 0.01 |
| `--fn-rate-threshold <pp>` | Max FN rate increase (percentage points) | 0.01 |
| `--determinism-threshold <rate>` | Min determinism rate | 1.0 |
| `--ttfrp-threshold <pct>` | Max TTFRP p95 increase (percentage) | 0.20 |
| `--output <path>` | Output report path | stdout |
| `--format <fmt>` | Report format: `markdown`, `json` | `markdown` |
**Example:**
```bash
stella groundtruth validate check \
--results bench/results/20260122.json \
--baseline bench/baselines/current.json \
--precision-threshold 0.01 \
--recall-threshold 0.01 \
--fn-rate-threshold 0.01 \
--determinism-threshold 1.0 \
--output regression-report.md
```
**Regression Gates:**
| Metric | Threshold | Action |
|--------|-----------|--------|
| Precision | Drops > threshold | Fail |
| Recall | Drops > threshold | Fail |
| False-negative rate | Increases > threshold | Fail |
| Deterministic replay | Drops below threshold | Fail |
| TTFRP p95 | Increases > threshold | Warn |
**Exit Codes:**
- `0` - All gates passed
- `1` - One or more gates failed
- `2` - Invalid input or configuration error
---
### stella groundtruth baseline
Manage KPI baselines for regression detection.
**Sprint:** SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
**Usage:**
```bash
stella groundtruth baseline <command> [options]
```
**Subcommands:**
- `update` - Update baseline from validation results
- `show` - Display baseline contents
#### stella groundtruth baseline update
Update baseline from validation results.
```bash
stella groundtruth baseline update [options]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--from-results <path>` | Path to validation results JSON | Yes |
| `--output <path>` | Output baseline path | Yes |
| `--description <text>` | Description for the baseline update | No |
| `--source <commit>` | Source commit SHA for traceability | No |
**Example:**
```bash
stella groundtruth baseline update \
--from-results bench/results/20260122.json \
--output bench/baselines/current.json \
--description "Post algorithm-v2.3 update" \
--source "$(git rev-parse HEAD)"
```
#### stella groundtruth baseline show
Display baseline contents.
```bash
stella groundtruth baseline show --baseline <path> [--format table|json]
```
**Options:**
| Option | Description | Default |
|--------|-------------|---------|
| `--baseline <path>` | Path to baseline JSON | (required) |
| `--format` | Output format: `table`, `json` | `table` |
**Output (table):**
```
Baseline ID: baseline-20260122120000
Created: 2026-01-22T12:00:00Z
Source: abc123def456
Description: Post-semantic-diffing-v2 baseline
KPIs:
Precision: 0.9500
Recall: 0.9200
False Negative Rate: 0.0800
Determinism: 1.0000
TTFRP p95: 150ms
```
**See Also:** [Ground-Truth CLI Guide](../ground-truth-cli.md)
---
## Reporting & Export Commands
### stella report