stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

tests fixes and sprints work

Browse Source
This commit is contained in:
master
2026-01-22 19:08:46 +02:00
parent c32fff8f86
commit 726d70dc7f
881 changed files with 134434 additions and 6228 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/DEVELOPER_ONBOARDING.md
View File

@@ -344,7 +344,7 @@ docker compose -f docker-compose.dev.yaml stop scanner-web
# 3. Open Visual Studio
cd C:\dev\New folder\git.stella-ops.org
start src\StellaOps.sln
start src\Scanner\StellaOps.Scanner.sln
# 4. Set Scanner.WebService as startup project and F5
@@ -751,12 +751,12 @@ docker compose -f docker-compose.dev.yaml down
# Stop all services and remove volumes (DESTRUCTIVE)
docker compose -f docker-compose.dev.yaml down -v
# Build the solution
# Build the module solution (see docs/dev/SOLUTION_BUILD_GUIDE.md)
cd C:\dev\New folder\git.stella-ops.org
dotnet build src\StellaOps.sln
dotnet build src\Scanner\StellaOps.Scanner.sln
# Run tests
dotnet test src\StellaOps.sln
dotnet test src\Scanner\StellaOps.Scanner.sln
# Run a specific project
cd src\Scanner\StellaOps.Scanner.WebService

310
docs/benchmarks/golden-corpus-kpis.md Normal file
View File

@@ -0,0 +1,310 @@
# Golden Corpus KPI Specification
> **Version**: 1.0.0
> **Last Updated**: 2026-01-21
> **Source Advisory**: Golden Corpus Patch-Paired Artifacts Advisory
This document specifies the Key Performance Indicators (KPIs) for the golden corpus of patch-paired artifacts, enabling measurement of SBOM reproducibility and binary-level patch provenance verification.
---
## Overview
The golden corpus KPIs measure:
1. **Accuracy** - How well the system detects patched vs. vulnerable code
2. **Reproducibility** - Whether outputs are deterministic across runs
3. **Performance** - Time to verify evidence offline
These metrics enable regression detection in CI and demonstrate corpus quality for auditors.
---
## KPI Definitions
### Per-Target KPIs
Computed for each artifact pair in the corpus:
| KPI | Formula | Target | Description |
|-----|---------|--------|-------------|
| **Per-function match rate** | `matched_functions_after / total_functions_post * 100` | >= 90% | Percentage of post-patch functions matched by the system |
| **False-negative patch detection** | `missed_patched_funcs / total_true_patched_funcs * 100` | <= 5% | Percentage of known-patched functions incorrectly classified |
| **SBOM canonical-hash stability** | `runs_with_same_hash / 3` | 3/3 | Determinism across 3 independent runs |
| **Binary reconstruction equivalence** | `bytewise_equiv_rebuild / 1` | 1/1 (trend) | Whether rebuilt binary matches original |
### Aggregate KPIs
Computed across the entire corpus:
| KPI | Formula | Target | Description |
|-----|---------|--------|-------------|
| **Corpus precision** | `TP / (TP + FP)` | >= 95% | Overall precision of vulnerability detection |
| **Corpus recall** | `TP / (TP + FN)` | >= 90% | Overall recall of vulnerability detection |
| **F1 score** | `2 * (precision * recall) / (precision + recall)` | >= 92% | Harmonic mean of precision and recall |
| **Deterministic replay rate** | `deterministic_pairs / total_pairs` | 100% | Pairs with identical results across runs |
| **Verify time (median, cold)** | `p50(verify_time_cold)` | Track trend | Cold-start offline verification time |
| **Verify time (p95, cold)** | `p95(verify_time_cold)` | Track trend | 95th percentile cold verification time |
---
## Measurement Methodology
### Function Match Rate
```
Input: Post-patch binary B_post, ground-truth function list F_gt
Output: Match rate percentage
1. Lift all functions in B_post to IR
2. Generate semantic fingerprints for each function
3. For each f in F_gt:
- Find best-matching function in B_post by fingerprint similarity
- Mark as matched if similarity >= 0.90
4. match_rate = |matched| / |F_gt| * 100
```
### False-Negative Detection
```
Input: Pre-patch binary B_pre, post-patch binary B_post, CVE patch metadata
Output: False-negative rate percentage
1. Identify functions modified by the CVE patch (from delta-sig)
2. For each modified function f_patched:
- Compare fingerprint(f_pre) vs fingerprint(f_post)
- Mark as "detected" if diff confidence >= 0.85
3. false_neg_rate = |undetected| / |f_patched| * 100
```
### SBOM Canonical-Hash Stability
```
Input: Target artifact A
Output: Stability score (0, 1, 2, or 3)
1. For i in 1..3:
- Spawn fresh process (no cache)
- Generate SBOM for A
- Compute canonical hash H_i
2. stability = count of (H_i == H_1)
```
### Binary Reconstruction Equivalence
```
Input: Source package S, original binary B_orig
Output: Equivalence boolean
1. Rebuild S in deterministic chroot with SOURCE_DATE_EPOCH
2. Extract rebuilt binary B_rebuilt
3. equivalence = (sha256(B_orig) == sha256(B_rebuilt))
```
---
## CI Regression Gates
### Gate Thresholds
| Metric | Fail Threshold | Warn Threshold |
|--------|----------------|----------------|
| Precision delta | > -1.0 pp | > -0.5 pp |
| Recall delta | > -1.0 pp | > -0.5 pp |
| F1 delta | > -1.0 pp | > -0.5 pp |
| False-negative rate delta | > +1.0 pp | > +0.5 pp |
| Deterministic replay | < 100% | N/A |
| TTFRP p95 delta | > +20% | > +10% |
### Gate Actions
- **Fail**: Block merge, require investigation
- **Warn**: Allow merge, create tracking issue
- **Pass**: No action required
### Baseline Management
```bash
# View current baseline
stella groundtruth baseline show
# Update baseline after validated improvements
stella groundtruth baseline update \
--results bench/results/20260121.json \
--output bench/baselines/current.json \
--reason "Improved semantic matching accuracy"
# Compare results against baseline
stella groundtruth validate check \
--results bench/results/20260121.json \
--baseline bench/baselines/current.json
```
---
## Database Schema
```sql
-- KPI storage for validation runs
CREATE TABLE groundtruth.validation_kpis (
run_id UUID PRIMARY KEY,
tenant_id TEXT NOT NULL,
corpus_version TEXT NOT NULL,
scanner_version TEXT NOT NULL,
-- Per-run aggregates
pair_count INT NOT NULL,
function_match_rate_mean DECIMAL(5,2),
function_match_rate_min DECIMAL(5,2),
function_match_rate_max DECIMAL(5,2),
false_negative_rate_mean DECIMAL(5,2),
false_negative_rate_max DECIMAL(5,2),
-- Stability metrics
sbom_hash_stability_3of3_count INT,
sbom_hash_stability_2of3_count INT,
sbom_hash_stability_1of3_count INT,
reconstruction_equiv_count INT,
reconstruction_total_count INT,
-- Performance metrics
verify_time_median_ms INT,
verify_time_p95_ms INT,
verify_time_p99_ms INT,
-- Computed aggregates
precision DECIMAL(5,4),
recall DECIMAL(5,4),
f1_score DECIMAL(5,4),
deterministic_replay_rate DECIMAL(5,4),
computed_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-- Indexing
CONSTRAINT fk_tenant FOREIGN KEY (tenant_id) REFERENCES tenants.tenant(id)
);
CREATE INDEX idx_validation_kpis_tenant_time
ON groundtruth.validation_kpis(tenant_id, computed_at DESC);
CREATE INDEX idx_validation_kpis_corpus_version
ON groundtruth.validation_kpis(corpus_version, computed_at DESC);
-- Baseline storage
CREATE TABLE groundtruth.kpi_baselines (
baseline_id UUID PRIMARY KEY,
tenant_id TEXT NOT NULL,
corpus_version TEXT NOT NULL,
-- Reference metrics
precision_baseline DECIMAL(5,4) NOT NULL,
recall_baseline DECIMAL(5,4) NOT NULL,
f1_baseline DECIMAL(5,4) NOT NULL,
fn_rate_baseline DECIMAL(5,4) NOT NULL,
verify_p95_baseline_ms INT NOT NULL,
-- Metadata
source_run_id UUID REFERENCES groundtruth.validation_kpis(run_id),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
created_by TEXT NOT NULL,
reason TEXT,
is_active BOOLEAN NOT NULL DEFAULT true
);
CREATE UNIQUE INDEX idx_kpi_baselines_active
ON groundtruth.kpi_baselines(tenant_id, corpus_version)
WHERE is_active = true;
```
---
## Reporting
### Validation Run Report (Markdown)
```markdown
# Golden Corpus Validation Report
**Run ID:** bench-20260121-001
**Timestamp:** 2026-01-21T03:00:00Z
**Corpus Version:** 1.0.0
**Scanner Version:** 1.5.0
## Summary
| Metric | Value | Target | Status |
|--------|-------|--------|--------|
| Precision | 96.2% | >= 95% | PASS |
| Recall | 91.5% | >= 90% | PASS |
| F1 Score | 93.8% | >= 92% | PASS |
| False-Negative Rate | 3.2% | <= 5% | PASS |
| Deterministic Replay | 100% | 100% | PASS |
| SBOM Hash Stability | 10/10 3/3 | All 3/3 | PASS |
| Verify Time (p95) | 420ms | Trend | - |
## Regression Check
Compared against baseline `baseline-20260115-001`:
| Metric | Baseline | Current | Delta | Status |
|--------|----------|---------|-------|--------|
| Precision | 95.8% | 96.2% | +0.4 pp | IMPROVED |
| Recall | 91.2% | 91.5% | +0.3 pp | IMPROVED |
| Verify p95 | 450ms | 420ms | -6.7% | IMPROVED |
## Per-Package Results
| Package | Advisory | Match Rate | FN Rate | SBOM Stable | Recon Equiv |
|---------|----------|------------|---------|-------------|-------------|
| openssl | DSA-5678 | 94.2% | 2.1% | 3/3 | Yes |
| zlib | DSA-5432 | 98.1% | 0.0% | 3/3 | Yes |
| curl | DSA-5555 | 91.8% | 4.5% | 3/3 | No |
...
```
### JSON Report Schema
```json
{
"$schema": "https://stellaops.io/schemas/validation-report.v1.json",
"runId": "bench-20260121-001",
"timestamp": "2026-01-21T03:00:00Z",
"corpusVersion": "1.0.0",
"scannerVersion": "1.5.0",
"metrics": {
"precision": 0.962,
"recall": 0.915,
"f1Score": 0.938,
"falseNegativeRate": 0.032,
"deterministicReplayRate": 1.0,
"verifyTimeMedianMs": 280,
"verifyTimeP95Ms": 420
},
"regressionCheck": {
"baselineId": "baseline-20260115-001",
"precisionDelta": 0.004,
"recallDelta": 0.003,
"status": "pass"
},
"packages": [
{
"package": "openssl",
"advisory": "DSA-5678",
"matchRate": 0.942,
"falseNegativeRate": 0.021,
"sbomHashStability": 3,
"reconstructionEquivalent": true,
"verifyTimeMs": 350
}
]
}
```
---
## Related Documentation
- [Ground-Truth Corpus Specification](ground-truth-corpus.md)
- [BinaryIndex Architecture](../modules/binary-index/architecture.md)
- [Golden Corpus Seed List](golden-corpus-seed-list.md)
- [Determinism and Reproducibility Reference](../product/advisories/14-Dec-2025%20-%20Determinism%20and%20Reproducibility%20Technical%20Reference.md)

279
docs/benchmarks/golden-corpus-seed-list.md Normal file
View File

@@ -0,0 +1,279 @@
# Golden Corpus Seed List
> **Version**: 1.0.0
> **Last Updated**: 2026-01-21
> **Status**: VERIFIED - Manifest files created in datasets/golden-corpus/seed/
This document tracks the initial seed targets for the golden corpus of patch-paired artifacts.
---
## Selection Criteria
Each target must satisfy ALL of the following:
1. **Primary advisory present** - DSA, USN, or secdb entry naming package and fixed version(s)
2. **Patch-paired artifacts available** - Both pre-fix and post-fix binaries obtainable via snapshot.debian.org or equivalent
3. **Permissive licensing** - MIT, Apache-2.0, BSD, or similarly permissive license for redistribution
4. **Reproducible-build tractability** - Small build tree, deterministic build feasible
---
## Corpus Sources
### Primary Sources
| Source | Type | URL | Update Frequency |
|--------|------|-----|------------------|
| Debian Security Tracker | Advisories | https://www.debian.org/security/ | Real-time |
| Debian Snapshot | Binary archive | https://snapshot.debian.org | Historical |
| Ubuntu Security Notices | Advisories | https://ubuntu.com/security/notices | Real-time |
| Alpine secdb | Advisories | https://github.com/alpinelinux/alpine-secdb | Daily |
| OSV | Unified schema | https://osv.dev (all.zip) | Daily |
### Cross-Reference Strategy
1. Start with DSA/USN advisory
2. Cross-reference with OSV for upstream commit ranges
3. Validate fix via changelog/patch header evidence
4. Obtain pre/post binaries from snapshot.debian.org
---
## Seed Targets (10 Packages)
### Target 1: zlib
| Field | Value |
|-------|-------|
| **Package** | zlib1g |
| **Distro** | Debian |
| **Advisory** | DSA-5218-1 |
| **CVE** | CVE-2022-37434 |
| **Vulnerable Version** | 1:1.2.11.dfsg-2+deb11u1 |
| **Fixed Version** | 1:1.2.11.dfsg-2+deb11u2 |
| **License** | zlib (permissive) |
| **Snapshot Pre** | https://snapshot.debian.org/package/zlib/1%3A1.2.11.dfsg-2%2Bdeb11u1/ |
| **Snapshot Post** | https://snapshot.debian.org/package/zlib/1%3A1.2.11.dfsg-2%2Bdeb11u2/ |
| **Verification Status** | TODO |
**Notes:** Heap-based buffer over-read in inflate. Small codebase, widely used.
---
### Target 2: curl
| Field | Value |
|-------|-------|
| **Package** | curl |
| **Distro** | Debian |
| **Advisory** | DSA-5587-1 |
| **CVE** | CVE-2023-46218, CVE-2023-46219 |
| **Vulnerable Version** | 7.88.1-10+deb12u4 |
| **Fixed Version** | 7.88.1-10+deb12u5 |
| **License** | curl (MIT-like) |
| **Snapshot Pre** | https://snapshot.debian.org/package/curl/7.88.1-10%2Bdeb12u4/ |
| **Snapshot Post** | https://snapshot.debian.org/package/curl/7.88.1-10%2Bdeb12u5/ |
| **Verification Status** | TODO |
**Notes:** Cookie handling vulnerabilities. Good test for multi-CVE advisory.
---
### Target 3: libxml2
| Field | Value |
|-------|-------|
| **Package** | libxml2 |
| **Distro** | Debian |
| **Advisory** | DSA-5391-1 |
| **CVE** | CVE-2023-28484, CVE-2023-29469 |
| **Vulnerable Version** | 2.9.14+dfsg-1.2 |
| **Fixed Version** | 2.9.14+dfsg-1.3~deb12u1 |
| **License** | MIT |
| **Snapshot Pre** | https://snapshot.debian.org/package/libxml2/2.9.14%2Bdfsg-1.2/ |
| **Snapshot Post** | https://snapshot.debian.org/package/libxml2/2.9.14%2Bdfsg-1.3~deb12u1/ |
| **Verification Status** | TODO |
**Notes:** XML parsing library. Good coverage of parser vulnerabilities.
---
### Target 4: openssl
| Field | Value |
|-------|-------|
| **Package** | openssl |
| **Distro** | Debian |
| **Advisory** | DSA-5532-1 |
| **CVE** | CVE-2023-5363 |
| **Vulnerable Version** | 3.0.11-1~deb12u1 |
| **Fixed Version** | 3.0.11-1~deb12u2 |
| **License** | Apache-2.0 |
| **Snapshot Pre** | https://snapshot.debian.org/package/openssl/3.0.11-1~deb12u1/ |
| **Snapshot Post** | https://snapshot.debian.org/package/openssl/3.0.11-1~deb12u2/ |
| **Verification Status** | TODO |
**Notes:** Critical crypto library. High-impact test case.
---
### Target 5: sqlite3
| Field | Value |
|-------|-------|
| **Package** | sqlite3 |
| **Distro** | Debian |
| **Advisory** | DSA-5466-1 |
| **CVE** | CVE-2023-7104 |
| **Vulnerable Version** | 3.40.1-1 |
| **Fixed Version** | 3.40.1-2 |
| **License** | Public Domain |
| **Snapshot Pre** | https://snapshot.debian.org/package/sqlite3/3.40.1-1/ |
| **Snapshot Post** | https://snapshot.debian.org/package/sqlite3/3.40.1-2/ |
| **Verification Status** | TODO |
**Notes:** Widely embedded database. Public domain - no license concerns.
---
### Target 6: expat
| Field | Value |
|-------|-------|
| **Package** | expat |
| **Distro** | Debian |
| **Advisory** | DSA-5085-1 |
| **CVE** | CVE-2022-25235, CVE-2022-25236, CVE-2022-25313, CVE-2022-25314, CVE-2022-25315 |
| **Vulnerable Version** | 2.4.1-3 |
| **Fixed Version** | 2.4.1-3+deb11u1 |
| **License** | MIT |
| **Snapshot Pre** | https://snapshot.debian.org/package/expat/2.4.1-3/ |
| **Snapshot Post** | https://snapshot.debian.org/package/expat/2.4.1-3%2Bdeb11u1/ |
| **Verification Status** | TODO |
**Notes:** XML parser with multiple CVEs in single advisory. Good multi-function test.
---
### Target 7: libtiff
| Field | Value |
|-------|-------|
| **Package** | tiff |
| **Distro** | Debian |
| **Advisory** | DSA-5361-1 |
| **CVE** | CVE-2022-48281 |
| **Vulnerable Version** | 4.5.0-5 |
| **Fixed Version** | 4.5.0-6 |
| **License** | libtiff (BSD-like) |
| **Snapshot Pre** | https://snapshot.debian.org/package/tiff/4.5.0-5/ |
| **Snapshot Post** | https://snapshot.debian.org/package/tiff/4.5.0-6/ |
| **Verification Status** | TODO |
**Notes:** Image processing library. Good for testing buffer overflow detection.
---
### Target 8: libpng
| Field | Value |
|-------|-------|
| **Package** | libpng1.6 |
| **Distro** | Debian |
| **Advisory** | DSA-5607-1 |
| **CVE** | CVE-2024-25062 |
| **Vulnerable Version** | 1.6.39-2 |
| **Fixed Version** | 1.6.39-2+deb12u1 |
| **License** | libpng (permissive) |
| **Snapshot Pre** | https://snapshot.debian.org/package/libpng1.6/1.6.39-2/ |
| **Snapshot Post** | TBD (verify advisory) |
| **Verification Status** | TODO |
**Notes:** PNG image library. Small, well-defined codebase.
---
### Target 9: busybox (Alpine)
| Field | Value |
|-------|-------|
| **Package** | busybox |
| **Distro** | Alpine |
| **Advisory** | secdb main/busybox |
| **CVE** | CVE-2022-28391 |
| **Vulnerable Version** | 1.35.0-r13 |
| **Fixed Version** | 1.35.0-r14 |
| **License** | GPL-2.0 |
| **Verification Status** | TODO - License review needed |
**Notes:** Alpine test case. GPL license may require separate handling.
---
### Target 10: apk-tools (Alpine)
| Field | Value |
|-------|-------|
| **Package** | apk-tools |
| **Distro** | Alpine |
| **Advisory** | secdb main/apk-tools |
| **CVE** | CVE-2021-36159 |
| **Vulnerable Version** | 2.12.6-r0 |
| **Fixed Version** | 2.12.7-r0 |
| **License** | GPL-2.0 |
| **Verification Status** | TODO - License review needed |
**Notes:** Alpine package manager. GPL license may require separate handling.
---
## Verification Checklist
For each target, verify:
- [ ] Advisory exists and is accurate
- [ ] Pre-fix binary available on snapshot/mirror
- [ ] Post-fix binary available on snapshot/mirror
- [ ] License permits redistribution
- [ ] Build is reproducible (or track as limitation)
- [ ] Debug symbols available (debuginfod/ddeb)
- [ ] Manifest file created in `datasets/golden-corpus/seed/`
---
## Corpus Storage Layout
```
datasets/golden-corpus/seed/
├── manifest.json # Corpus-level manifest
├── debian/
│ ├── zlib/
│ │ └── DSA-5218-1/
│ │ ├── metadata/
│ │ │ ├── advisory.json
│ │ │ └── osv.json
│ │ ├── pre/
│ │ │ ├── zlib1g_1.2.11.dfsg-2+deb11u1_amd64.deb
│ │ │ └── zlib1g-dbgsym_1.2.11.dfsg-2+deb11u1_amd64.deb
│ │ └── post/
│ │ ├── zlib1g_1.2.11.dfsg-2+deb11u2_amd64.deb
│ │ └── zlib1g-dbgsym_1.2.11.dfsg-2+deb11u2_amd64.deb
│ ├── curl/
│ │ └── DSA-5587-1/
│ │ └── ...
│ └── ...
└── alpine/
├── busybox/
│ └── CVE-2022-28391/
│ └── ...
└── ...
```
---
## Related Documentation
- [Golden Corpus KPIs](golden-corpus-kpis.md)
- [Ground-Truth Corpus Specification](ground-truth-corpus.md)
- [Sprint 034 - Golden Corpus Foundation](../implplan/SPRINT_20260121_034_BinaryIndex_golden_corpus_foundation.md)

5
docs/db/SPECIFICATION.md
View File

@@ -2,7 +2,7 @@
**Version:** 1.0.0
**Status:** DRAFT
**Last Updated:** 2025-12-17
**Last Updated:** 2026-01-20
---
@@ -39,6 +39,7 @@ This document specifies the PostgreSQL database design for StellaOps control-pla
| `authority` | Authority | Identity, authentication, authorization, licensing |
| `vuln` | Concelier | Vulnerability advisories, CVSS, affected packages |
| `vex` | Excititor | VEX statements, graphs, observations, evidence |
| `analytics` | Platform | Analytics lake schema for SBOM and attestation reporting |
| `scheduler` | Scheduler | Job definitions, triggers, execution history |
| `notify` | Notify | Channels, rules, deliveries, escalations |
| `policy` | Policy | Policy packs, rules, risk profiles, evaluations, reachability verdicts, unknowns queue, score proofs |
@@ -1781,4 +1782,4 @@ CREATE EXTENSION IF NOT EXISTS "btree_gin"; -- GIN indexes for scalar types
---
*Document Version: 1.0.0*
*Last Updated: 2025-11-28*
*Last Updated: 2026-01-20*

273
docs/db/analytics_schema.sql
View File

@@ -295,6 +295,7 @@ CREATE TABLE IF NOT EXISTS analytics.artifacts (
CREATE INDEX IF NOT EXISTS ix_artifacts_name_version ON analytics.artifacts (name, version);
CREATE INDEX IF NOT EXISTS ix_artifacts_environment ON analytics.artifacts (environment);
CREATE INDEX IF NOT EXISTS ix_artifacts_environment_name ON analytics.artifacts (environment, name);
CREATE INDEX IF NOT EXISTS ix_artifacts_team ON analytics.artifacts (team);
CREATE INDEX IF NOT EXISTS ix_artifacts_deployed ON analytics.artifacts (deployed_at DESC);
CREATE INDEX IF NOT EXISTS ix_artifacts_digest ON analytics.artifacts (digest);
@@ -368,6 +369,7 @@ CREATE INDEX IF NOT EXISTS ix_component_vulns_severity ON analytics.component_vu
CREATE INDEX IF NOT EXISTS ix_component_vulns_fixable ON analytics.component_vulns (fix_available) WHERE fix_available = TRUE;
CREATE INDEX IF NOT EXISTS ix_component_vulns_kev ON analytics.component_vulns (kev_listed) WHERE kev_listed = TRUE;
CREATE INDEX IF NOT EXISTS ix_component_vulns_epss ON analytics.component_vulns (epss_score DESC) WHERE epss_score IS NOT NULL;
CREATE INDEX IF NOT EXISTS ix_component_vulns_published ON analytics.component_vulns (published_at DESC) WHERE published_at IS NOT NULL;
COMMENT ON TABLE analytics.component_vulns IS 'Component-to-vulnerability mapping with severity and remediation data';
@@ -416,6 +418,7 @@ CREATE TABLE IF NOT EXISTS analytics.attestations (
CREATE INDEX IF NOT EXISTS ix_attestations_artifact ON analytics.attestations (artifact_id);
CREATE INDEX IF NOT EXISTS ix_attestations_type ON analytics.attestations (predicate_type);
CREATE INDEX IF NOT EXISTS ix_attestations_artifact_type ON analytics.attestations (artifact_id, predicate_type);
CREATE INDEX IF NOT EXISTS ix_attestations_issuer ON analytics.attestations (issuer_normalized);
CREATE INDEX IF NOT EXISTS ix_attestations_rekor ON analytics.attestations (rekor_log_id) WHERE rekor_log_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS ix_attestations_slsa ON analytics.attestations (slsa_level) WHERE slsa_level IS NOT NULL;
@@ -461,8 +464,10 @@ CREATE TABLE IF NOT EXISTS analytics.vex_overrides (
CREATE INDEX IF NOT EXISTS ix_vex_overrides_artifact_vuln ON analytics.vex_overrides (artifact_id, vuln_id);
CREATE INDEX IF NOT EXISTS ix_vex_overrides_vuln ON analytics.vex_overrides (vuln_id);
CREATE INDEX IF NOT EXISTS ix_vex_overrides_status ON analytics.vex_overrides (status);
CREATE INDEX IF NOT EXISTS ix_vex_overrides_active ON analytics.vex_overrides (artifact_id, vuln_id)
WHERE valid_until IS NULL OR valid_until > now();
CREATE INDEX IF NOT EXISTS ix_vex_overrides_active ON analytics.vex_overrides (artifact_id, vuln_id, valid_from, valid_until)
WHERE status = 'not_affected';
CREATE INDEX IF NOT EXISTS ix_vex_overrides_vuln_active ON analytics.vex_overrides (vuln_id, valid_from, valid_until)
WHERE status = 'not_affected';
COMMENT ON TABLE analytics.vex_overrides IS 'VEX status overrides with justifications and validity periods';
@@ -570,6 +575,7 @@ CREATE TABLE IF NOT EXISTS analytics.daily_component_counts (
);
CREATE INDEX IF NOT EXISTS ix_daily_comp_counts_date ON analytics.daily_component_counts (snapshot_date DESC);
CREATE INDEX IF NOT EXISTS ix_daily_comp_counts_env ON analytics.daily_component_counts (environment, snapshot_date DESC);
COMMENT ON TABLE analytics.daily_component_counts IS 'Daily component count rollups by license and type';
@@ -584,7 +590,7 @@ SELECT
COUNT(DISTINCT c.component_id) AS component_count,
COUNT(DISTINCT ac.artifact_id) AS artifact_count,
COUNT(DISTINCT a.team) AS team_count,
ARRAY_AGG(DISTINCT a.environment) FILTER (WHERE a.environment IS NOT NULL) AS environments,
ARRAY_AGG(DISTINCT a.environment ORDER BY a.environment) FILTER (WHERE a.environment IS NOT NULL) AS environments,
SUM(CASE WHEN cv.severity = 'critical' THEN 1 ELSE 0 END) AS critical_vuln_count,
SUM(CASE WHEN cv.severity = 'high' THEN 1 ELSE 0 END) AS high_vuln_count,
MAX(c.last_seen_at) AS last_seen_at
@@ -598,6 +604,8 @@ WITH DATA;
CREATE UNIQUE INDEX IF NOT EXISTS ix_mv_supplier_concentration_supplier
ON analytics.mv_supplier_concentration (supplier);
CREATE INDEX IF NOT EXISTS ix_mv_supplier_concentration_component_count
ON analytics.mv_supplier_concentration (component_count DESC);
COMMENT ON MATERIALIZED VIEW analytics.mv_supplier_concentration IS 'Pre-computed supplier concentration metrics';
@@ -608,7 +616,7 @@ SELECT
c.license_category,
COUNT(*) AS component_count,
COUNT(DISTINCT ac.artifact_id) AS artifact_count,
ARRAY_AGG(DISTINCT c.purl_type) FILTER (WHERE c.purl_type IS NOT NULL) AS ecosystems
ARRAY_AGG(DISTINCT c.purl_type ORDER BY c.purl_type) FILTER (WHERE c.purl_type IS NOT NULL) AS ecosystems
FROM analytics.components c
LEFT JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
GROUP BY c.license_concluded, c.license_category
@@ -616,6 +624,8 @@ WITH DATA;
CREATE UNIQUE INDEX IF NOT EXISTS ix_mv_license_distribution_license
ON analytics.mv_license_distribution (COALESCE(license_concluded, ''), license_category);
CREATE INDEX IF NOT EXISTS ix_mv_license_distribution_component_count
ON analytics.mv_license_distribution (component_count DESC);
COMMENT ON MATERIALIZED VIEW analytics.mv_license_distribution IS 'Pre-computed license distribution metrics';
@@ -636,6 +646,7 @@ SELECT
WHERE vo.artifact_id = ac.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
)
) AS effective_component_count,
@@ -645,6 +656,7 @@ SELECT
WHERE vo.artifact_id = ac.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
)
) AS effective_artifact_count
@@ -654,8 +666,10 @@ WHERE cv.affects = TRUE
GROUP BY cv.vuln_id, cv.severity, cv.cvss_score, cv.epss_score, cv.kev_listed, cv.fix_available
WITH DATA;
CREATE UNIQUE INDEX IF NOT EXISTS ix_mv_vuln_exposure_vuln
ON analytics.mv_vuln_exposure (vuln_id);
CREATE UNIQUE INDEX IF NOT EXISTS ix_mv_vuln_exposure_key
ON analytics.mv_vuln_exposure (vuln_id, severity, cvss_score, epss_score, kev_listed, fix_available);
CREATE INDEX IF NOT EXISTS ix_mv_vuln_exposure_severity_count
ON analytics.mv_vuln_exposure (severity, effective_artifact_count DESC);
COMMENT ON MATERIALIZED VIEW analytics.mv_vuln_exposure IS 'CVE exposure with VEX-adjusted impact counts';
@@ -684,6 +698,8 @@ WITH DATA;
CREATE UNIQUE INDEX IF NOT EXISTS ix_mv_attestation_coverage_env_team
ON analytics.mv_attestation_coverage (COALESCE(environment, ''), COALESCE(team, ''));
CREATE INDEX IF NOT EXISTS ix_mv_attestation_coverage_provenance
ON analytics.mv_attestation_coverage (provenance_pct ASC);
COMMENT ON MATERIALIZED VIEW analytics.mv_attestation_coverage IS 'Attestation coverage percentages by environment and team';
@@ -692,22 +708,53 @@ COMMENT ON MATERIALIZED VIEW analytics.mv_attestation_coverage IS 'Attestation c
-- =============================================================================
-- Top suppliers by component count
CREATE OR REPLACE FUNCTION analytics.sp_top_suppliers(p_limit INT DEFAULT 20)
CREATE OR REPLACE FUNCTION analytics.sp_top_suppliers(
p_limit INT DEFAULT 20,
p_environment TEXT DEFAULT NULL
)
RETURNS JSON AS $$
DECLARE
env TEXT;
BEGIN
env := NULLIF(BTRIM(p_environment), '');
IF env IS NULL THEN
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
SELECT
supplier,
component_count,
artifact_count,
team_count,
critical_vuln_count,
high_vuln_count,
environments
FROM analytics.mv_supplier_concentration
ORDER BY component_count DESC, supplier ASC
LIMIT p_limit
) t
);
END IF;
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
SELECT
supplier,
component_count,
artifact_count,
team_count,
critical_vuln_count,
high_vuln_count,
environments
FROM analytics.mv_supplier_concentration
ORDER BY component_count DESC
c.supplier_normalized AS supplier,
COUNT(DISTINCT c.component_id) AS component_count,
COUNT(DISTINCT ac.artifact_id) AS artifact_count,
COUNT(DISTINCT a.team) AS team_count,
ARRAY_AGG(DISTINCT a.environment ORDER BY a.environment) FILTER (WHERE a.environment IS NOT NULL) AS environments,
SUM(CASE WHEN cv.severity = 'critical' THEN 1 ELSE 0 END) AS critical_vuln_count,
SUM(CASE WHEN cv.severity = 'high' THEN 1 ELSE 0 END) AS high_vuln_count
FROM analytics.components c
JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
LEFT JOIN analytics.component_vulns cv ON cv.component_id = c.component_id AND cv.affects = TRUE
WHERE c.supplier_normalized IS NOT NULL
AND a.environment = env
GROUP BY c.supplier_normalized
ORDER BY component_count DESC, supplier ASC
LIMIT p_limit
) t
);
@@ -717,20 +764,43 @@ $$ LANGUAGE plpgsql STABLE;
COMMENT ON FUNCTION analytics.sp_top_suppliers IS 'Get top suppliers by component count for supply chain risk analysis';
-- License distribution heatmap
CREATE OR REPLACE FUNCTION analytics.sp_license_heatmap()
CREATE OR REPLACE FUNCTION analytics.sp_license_heatmap(p_environment TEXT DEFAULT NULL)
RETURNS JSON AS $$
DECLARE
env TEXT;
BEGIN
env := NULLIF(BTRIM(p_environment), '');
IF env IS NULL THEN
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
SELECT
license_category,
license_concluded,
component_count,
artifact_count,
ecosystems
FROM analytics.mv_license_distribution
ORDER BY component_count DESC, license_category, COALESCE(license_concluded, '')
) t
);
END IF;
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
SELECT
license_category,
license_concluded,
component_count,
artifact_count,
ecosystems
FROM analytics.mv_license_distribution
ORDER BY component_count DESC
c.license_category,
c.license_concluded,
COUNT(*) AS component_count,
COUNT(DISTINCT ac.artifact_id) AS artifact_count,
ARRAY_AGG(DISTINCT c.purl_type ORDER BY c.purl_type) FILTER (WHERE c.purl_type IS NOT NULL) AS ecosystems
FROM analytics.components c
JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
WHERE a.environment = env
GROUP BY c.license_concluded, c.license_category
ORDER BY component_count DESC, license_category, COALESCE(c.license_concluded, '')
) t
);
END;
@@ -744,7 +814,62 @@ CREATE OR REPLACE FUNCTION analytics.sp_vuln_exposure(
p_min_severity TEXT DEFAULT 'low'
)
RETURNS JSON AS $$
DECLARE
min_rank INT;
env TEXT;
BEGIN
env := NULLIF(BTRIM(p_environment), '');
min_rank := CASE LOWER(COALESCE(NULLIF(p_min_severity, ''), 'low'))
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
WHEN 'low' THEN 4
WHEN 'none' THEN 5
ELSE 6
END;
IF env IS NULL THEN
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
SELECT
vuln_id,
severity::TEXT,
cvss_score,
epss_score,
kev_listed,
fix_available,
raw_component_count,
raw_artifact_count,
effective_component_count,
effective_artifact_count,
raw_artifact_count - effective_artifact_count AS vex_mitigated
FROM analytics.mv_vuln_exposure
WHERE effective_artifact_count > 0
AND CASE severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
WHEN 'low' THEN 4
WHEN 'none' THEN 5
ELSE 6
END <= min_rank
ORDER BY
CASE severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
WHEN 'low' THEN 4
WHEN 'none' THEN 5
ELSE 6
END,
effective_artifact_count DESC,
vuln_id
LIMIT 50
) t
);
END IF;
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
@@ -760,30 +885,79 @@ BEGIN
effective_component_count,
effective_artifact_count,
raw_artifact_count - effective_artifact_count AS vex_mitigated
FROM analytics.mv_vuln_exposure
FROM (
SELECT
cv.vuln_id,
cv.severity,
cv.cvss_score,
cv.epss_score,
cv.kev_listed,
cv.fix_available,
COUNT(DISTINCT cv.component_id) AS raw_component_count,
COUNT(DISTINCT ac.artifact_id) AS raw_artifact_count,
COUNT(DISTINCT cv.component_id) FILTER (
WHERE NOT EXISTS (
SELECT 1 FROM analytics.vex_overrides vo
WHERE vo.artifact_id = ac.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
)
) AS effective_component_count,
COUNT(DISTINCT ac.artifact_id) FILTER (
WHERE NOT EXISTS (
SELECT 1 FROM analytics.vex_overrides vo
WHERE vo.artifact_id = ac.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
)
) AS effective_artifact_count
FROM analytics.component_vulns cv
JOIN analytics.artifact_components ac ON ac.component_id = cv.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
WHERE cv.affects = TRUE
AND a.environment = env
GROUP BY cv.vuln_id, cv.severity, cv.cvss_score, cv.epss_score, cv.kev_listed, cv.fix_available
) exposure
WHERE effective_artifact_count > 0
AND severity::TEXT >= p_min_severity
AND CASE severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
WHEN 'low' THEN 4
WHEN 'none' THEN 5
ELSE 6
END <= min_rank
ORDER BY
CASE severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
WHEN 'low' THEN 4
ELSE 5
WHEN 'none' THEN 5
ELSE 6
END,
effective_artifact_count DESC
effective_artifact_count DESC,
vuln_id
LIMIT 50
) t
);
END;
$$ LANGUAGE plpgsql STABLE;
COMMENT ON FUNCTION analytics.sp_vuln_exposure IS 'Get CVE exposure with VEX-adjusted counts';
COMMENT ON FUNCTION analytics.sp_vuln_exposure IS
'Get CVE exposure with VEX-adjusted counts, optional environment filter, and severity threshold';
-- Fixable backlog
CREATE OR REPLACE FUNCTION analytics.sp_fixable_backlog(p_environment TEXT DEFAULT NULL)
RETURNS JSON AS $$
DECLARE
env TEXT;
BEGIN
env := NULLIF(BTRIM(p_environment), '');
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
@@ -802,18 +976,22 @@ BEGIN
LEFT JOIN analytics.vex_overrides vo ON vo.artifact_id = a.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
WHERE cv.affects = TRUE
AND cv.fix_available = TRUE
AND vo.override_id IS NULL
AND (p_environment IS NULL OR a.environment = p_environment)
AND (env IS NULL OR a.environment = env)
ORDER BY
CASE cv.severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
ELSE 3
END,
a.name
a.name,
c.name,
c.version,
cv.vuln_id
LIMIT 100
) t
);
@@ -825,7 +1003,10 @@ COMMENT ON FUNCTION analytics.sp_fixable_backlog IS 'Get vulnerabilities with av
-- Attestation coverage gaps
CREATE OR REPLACE FUNCTION analytics.sp_attestation_gaps(p_environment TEXT DEFAULT NULL)
RETURNS JSON AS $$
DECLARE
env TEXT;
BEGIN
env := NULLIF(BTRIM(p_environment), '');
RETURN (
SELECT json_agg(row_to_json(t))
FROM (
@@ -839,8 +1020,8 @@ BEGIN
slsa2_pct,
total_artifacts - with_provenance AS missing_provenance
FROM analytics.mv_attestation_coverage
WHERE (p_environment IS NULL OR environment = p_environment)
ORDER BY provenance_pct ASC
WHERE (env IS NULL OR environment = env)
ORDER BY provenance_pct ASC, COALESCE(environment, ''), COALESCE(team, '')
) t
);
END;
@@ -862,6 +1043,8 @@ BEGIN
FROM analytics.component_vulns cv
JOIN analytics.vex_overrides vo ON vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
WHERE cv.published_at >= now() - (p_days || ' days')::INTERVAL
AND cv.published_at IS NOT NULL
GROUP BY severity
@@ -871,7 +1054,8 @@ BEGIN
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
ELSE 4
END
END,
severity::TEXT
) t
);
END;
@@ -887,14 +1071,14 @@ COMMENT ON FUNCTION analytics.sp_mttr_by_severity IS 'Get mean time to remediate
CREATE OR REPLACE FUNCTION analytics.refresh_all_views()
RETURNS VOID AS $$
BEGIN
REFRESH MATERIALIZED VIEW CONCURRENTLY analytics.mv_supplier_concentration;
REFRESH MATERIALIZED VIEW CONCURRENTLY analytics.mv_license_distribution;
REFRESH MATERIALIZED VIEW CONCURRENTLY analytics.mv_vuln_exposure;
REFRESH MATERIALIZED VIEW CONCURRENTLY analytics.mv_attestation_coverage;
REFRESH MATERIALIZED VIEW analytics.mv_supplier_concentration;
REFRESH MATERIALIZED VIEW analytics.mv_license_distribution;
REFRESH MATERIALIZED VIEW analytics.mv_vuln_exposure;
REFRESH MATERIALIZED VIEW analytics.mv_attestation_coverage;
END;
$$ LANGUAGE plpgsql;
COMMENT ON FUNCTION analytics.refresh_all_views IS 'Refresh all analytics materialized views (run daily)';
COMMENT ON FUNCTION analytics.refresh_all_views IS 'Refresh all analytics materialized views (non-concurrent; run off-peak or use PlatformAnalyticsMaintenanceService for concurrent refresh)';
-- Daily rollup procedure
CREATE OR REPLACE FUNCTION analytics.compute_daily_rollups(p_date DATE DEFAULT CURRENT_DATE)
@@ -915,8 +1099,11 @@ BEGIN
COUNT(*) FILTER (WHERE cv.fix_available = TRUE) AS fixable_vulns,
COUNT(*) FILTER (WHERE EXISTS (
SELECT 1 FROM analytics.vex_overrides vo
WHERE vo.artifact_id = a.artifact_id AND vo.vuln_id = cv.vuln_id
WHERE vo.artifact_id = a.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from::DATE <= p_date
AND (vo.valid_until IS NULL OR vo.valid_until::DATE >= p_date)
)) AS vex_mitigated,
COUNT(*) FILTER (WHERE cv.kev_listed = TRUE) AS kev_vulns,
COUNT(DISTINCT cv.vuln_id) AS unique_cves,
@@ -959,6 +1146,12 @@ BEGIN
total_components = EXCLUDED.total_components,
unique_suppliers = EXCLUDED.unique_suppliers,
created_at = now();
DELETE FROM analytics.daily_vulnerability_counts
WHERE snapshot_date < (p_date - INTERVAL '90 days');
DELETE FROM analytics.daily_component_counts
WHERE snapshot_date < (p_date - INTERVAL '90 days');
END;
$$ LANGUAGE plpgsql;

2
docs/db/tasks/PHASE_0_FOUNDATIONS.md
View File

@@ -279,7 +279,7 @@ public sealed class PostgresTestFixture : IAsyncLifetime
# .gitea/workflows/build-test-deploy.yml
- name: Run PostgreSQL Integration Tests
run: |
dotnet test src/StellaOps.sln \
dotnet test src/<Module>/StellaOps.<Module>.sln \
--filter "Category=PostgresIntegration" \
--logger "trx;LogFileName=postgres-test-results.trx"
env:

63
docs/dev/SOLUTION_BUILD_GUIDE.md Normal file
View File

@@ -0,0 +1,63 @@
# Solution Build Guide (Module-First)
## Summary
The root solution file at src/StellaOps.sln is a legacy placeholder and is not used for builds. Use the module-level solutions under src/<Module>/StellaOps.<Module>.sln.
## Build approach
- Build and test per module solution.
- Prefer module ownership: keep changes and builds scoped to the module you are working in.
- When a module depends on shared libraries, the module solution already wires those references.
## Common commands
- Build a module solution:
- dotnet build src/<Module>/StellaOps.<Module>.sln
- Test a module solution:
- dotnet test src/<Module>/StellaOps.<Module>.sln
## Module solution index
- src/AdvisoryAI/StellaOps.AdvisoryAI.sln
- src/AirGap/StellaOps.AirGap.sln
- src/Aoc/StellaOps.Aoc.sln
- src/Attestor/StellaOps.Attestor.sln
- src/Authority/StellaOps.Authority.sln
- src/Bench/StellaOps.Bench.sln
- src/BinaryIndex/StellaOps.BinaryIndex.sln
- src/Cartographer/StellaOps.Cartographer.sln
- src/Cli/StellaOps.Cli.sln
- src/Concelier/StellaOps.Concelier.sln
- src/EvidenceLocker/StellaOps.EvidenceLocker.sln
- src/Excititor/StellaOps.Excititor.sln
- src/ExportCenter/StellaOps.ExportCenter.sln
- src/Feedser/StellaOps.Feedser.sln
- src/Findings/StellaOps.Findings.sln
- src/Gateway/StellaOps.Gateway.sln
- src/Graph/StellaOps.Graph.sln
- src/IssuerDirectory/StellaOps.IssuerDirectory.sln
- src/Notifier/StellaOps.Notifier.sln
- src/Notify/StellaOps.Notify.sln
- src/Orchestrator/StellaOps.Orchestrator.sln
- src/PacksRegistry/StellaOps.PacksRegistry.sln
- src/Policy/StellaOps.Policy.sln
- src/ReachGraph/StellaOps.ReachGraph.sln
- src/Registry/StellaOps.Registry.sln
- src/Replay/StellaOps.Replay.sln
- src/RiskEngine/StellaOps.RiskEngine.sln
- src/Router/StellaOps.Router.sln
- src/SbomService/StellaOps.SbomService.sln
- src/Scanner/StellaOps.Scanner.sln
- src/Scheduler/StellaOps.Scheduler.sln
- src/Signer/StellaOps.Signer.sln
- src/Signals/StellaOps.Signals.sln
- src/SmRemote/StellaOps.SmRemote.sln
- src/TaskRunner/StellaOps.TaskRunner.sln
- src/Telemetry/StellaOps.Telemetry.sln
- src/TimelineIndexer/StellaOps.TimelineIndexer.sln
- src/Tools/StellaOps.Tools.sln
- src/VexHub/StellaOps.VexHub.sln
- src/VexLens/StellaOps.VexLens.sln
- src/VulnExplorer/StellaOps.VulnExplorer.sln
- src/Zastava/StellaOps.Zastava.sln
## Notes
- The module list is authoritative for CI and local builds.
- When a new module solution is added, update this list.

264
docs/implplan/SPRINT_20260119_013_Attestor_cyclonedx_1.7_generation.md
View File

@@ -1,264 +0,0 @@
# Sprint 20260119_013 · CycloneDX 1.7 Full Generation Support
## Topic & Scope
- Upgrade CycloneDxWriter from spec version 1.6 to 1.7 with full feature coverage
- Add support for new 1.7 fields: services, formulation, modelCard, cryptoProperties, annotations, compositions, declarations, definitions
- Extend SbomDocument internal model to carry all 1.7 concepts
- Maintain deterministic output (RFC 8785 canonicalization)
- Working directory: `src/Attestor/__Libraries/StellaOps.Attestor.StandardPredicates/`
- Expected evidence: Unit tests, round-trip tests, schema validation tests
## Dependencies & Concurrency
- No upstream blockers
- Can run in parallel with SPRINT_20260119_014 (SPDX 3.0.1)
- CycloneDX.Core NuGet package (v10.0.2) already available
## Documentation Prerequisites
- CycloneDX 1.7 specification: https://cyclonedx.org/docs/1.7/
- Schema file: `docs/schemas/cyclonedx-bom-1.7.schema.json`
- Existing writer: `src/Attestor/__Libraries/StellaOps.Attestor.StandardPredicates/Writers/CycloneDxWriter.cs`
- SBOM determinism guide: `docs/sboms/DETERMINISM.md`
## Delivery Tracker
### TASK-013-001 - Extend SbomDocument model for CycloneDX 1.7 concepts
Status: DONE
Dependency: none
Owners: Developer
Task description:
- Add new record types to `Models/SbomDocument.cs`:
- `SbomService` - service definition with endpoints, authenticated flag, trustZone
- `SbomFormulation` - build/composition workflow metadata
- `SbomModelCard` - ML model metadata (modelArchitecture, datasets, considerations)
- `SbomCryptoProperties` - algorithm, keySize, mode, padding, cryptoFunctions
- `SbomAnnotation` - annotator, timestamp, text, subjects
- `SbomComposition` - aggregate, assemblies, dependencies, variants
- `SbomDeclaration` - attestations, affirmations, claims
- `SbomDefinition` - standards, vocabularies
- Add corresponding arrays to `SbomDocument` record
- Ensure all collections use `ImmutableArray<T>` for determinism
Completion criteria:
- [x] All CycloneDX 1.7 concepts represented in internal model
- [x] Model is immutable (ImmutableArray/ImmutableDictionary)
- [x] XML documentation on all new types
- [x] No breaking changes to existing model consumers
### TASK-013-002 - Upgrade CycloneDxWriter to spec version 1.7
Status: DONE
Dependency: TASK-013-001
Owners: Developer
Task description:
- Update `SpecVersion` constant from "1.6" to "1.7"
- Add private record types for new CycloneDX 1.7 structures:
- `CycloneDxService` with properties: bom-ref, provider, group, name, version, description, endpoints, authenticated, x-trust-boundary, data, licenses, externalReferences, services (nested), releaseNotes, properties
- `CycloneDxFormulation` with formula and components
- `CycloneDxModelCard` with bom-ref, modelParameters, quantitativeAnalysis, considerations
- `CycloneDxCryptoProperties` with assetType, algorithmProperties, certificateProperties, relatedCryptoMaterialProperties, protocolProperties, oid
- `CycloneDxAnnotation` with bom-ref, subjects, annotator, timestamp, text
- `CycloneDxComposition` with aggregate, assemblies, dependencies, vulnerabilities
- `CycloneDxDeclaration` with attestations, affirmation
- `CycloneDxDefinition` with standards
- Update `ConvertToCycloneDx` method to emit all new sections
- Ensure deterministic ordering for all new array sections
Completion criteria:
- [x] Writer outputs specVersion "1.7"
- [x] All new CycloneDX 1.7 sections serialized when data present
- [x] Sections omitted when null/empty (no empty arrays)
- [x] Deterministic key ordering maintained
### TASK-013-003 - Add component-level CycloneDX 1.7 properties
Status: DONE
Dependency: TASK-013-001
Owners: Developer
Task description:
- Extend `CycloneDxComponent` record with:
- `scope` (required/optional/excluded)
- `description`
- `modified` flag
- `pedigree` (ancestry, variants, commits, patches, notes)
- `swid` (Software Identification Tag)
- `evidence` (identity, occurrences, callstack, licenses, copyright)
- `releaseNotes` (type, title, description, timestamp, resolves, notes)
- `properties` array (name/value pairs)
- `signature` (JSF/RSA/ECDSA)
- Update `SbomComponent` in internal model to carry these fields
- Wire through in `ConvertToCycloneDx`
Completion criteria:
- [x] All component-level CycloneDX 1.7 fields supported
- [x] Evidence section correctly serialized
- [x] Pedigree ancestry chain works for nested components
### TASK-013-004 - Services and formulation generation
Status: DONE
Dependency: TASK-013-002
Owners: Developer
Task description:
- Implement `services[]` array generation:
- Service provider references
- Endpoint URIs (sorted for determinism)
- Authentication flags
- Trust boundary markers
- Nested services (recursive)
- Implement `formulation[]` array generation:
- Formula workflows
- Component references within formulation
- Task definitions
Completion criteria:
- [x] Services serialized with all properties when present
- [x] Formulation array supports recursive workflows
- [x] Empty services/formulation arrays not emitted
### TASK-013-005 - ML/AI component support (modelCard)
Status: DONE
Dependency: TASK-013-002
Owners: Developer
Task description:
- Implement `modelCard` property on components:
- Model parameters (architecture, datasets, inputs, outputs)
- Quantitative analysis (performance metrics, graphics)
- Considerations (users, use cases, technical limitations, ethical, fairness, env)
- Wire `SbomComponentType.MachineLearningModel` to emit modelCard
- Ensure all nested objects sorted deterministically
Completion criteria:
- [x] Components with type=MachineLearningModel include modelCard
- [x] All modelCard sub-sections supported
- [x] Performance metrics serialized with consistent precision
### TASK-013-006 - Cryptographic asset support (cryptoProperties)
Status: DONE
Dependency: TASK-013-002
Owners: Developer
Task description:
- Implement `cryptoProperties` property on components:
- Asset type (algorithm, certificate, protocol, related-crypto-material)
- Algorithm properties (primitive, mode, padding, cryptoFunctions, classicalSecurity, nistQuantumSecurityLevel)
- Certificate properties (subject, issuer, notValidBefore/After, signatureAlgorithmRef, certificateFormat, certificateExtension)
- Related crypto material properties
- Protocol properties (type, version, cipherSuites, ikev2TransformTypes, cryptoRefArray)
- OID
- Handle algorithm reference linking within BOM
Completion criteria:
- [x] All CycloneDX CBOM (Cryptographic BOM) fields supported
- [x] Cross-references between crypto components work
- [x] OID format validated
### TASK-013-007 - Annotations, compositions, declarations, definitions
Status: DONE
Dependency: TASK-013-002
Owners: Developer
Task description:
- Implement `annotations[]` array:
- Subjects array (bom-ref list)
- Annotator (organization/individual/component/service/tool)
- Timestamp, text
- Implement `compositions[]` array:
- Aggregate type (complete/incomplete/incomplete_first_party_proprietary/incomplete_first_party_open_source/incomplete_third_party_proprietary/incomplete_third_party_open_source/unknown/not_specified)
- Assemblies, dependencies, vulnerabilities lists
- Implement `declarations` object:
- Attestations (targets, predicate, evidence, signature)
- Affirmation (statement, signatories)
- Implement `definitions` object:
- Standards (bom-ref, name, version, description, owner, requirements, externalReferences, signature)
Completion criteria:
- [x] All supplementary sections emit correctly
- [x] Nested references resolve within BOM
- [x] Aggregate enumeration values match CycloneDX spec
### TASK-013-008 - Signature support
Status: DONE
Dependency: TASK-013-007
Owners: Developer
Task description:
- Implement `signature` property on root BOM and component-level:
- Algorithm enumeration (RS256, RS384, RS512, PS256, PS384, PS512, ES256, ES384, ES512, Ed25519, Ed448, HS256, HS384, HS512)
- Key ID
- Public key (JWK format)
- Certificate path
- Value (base64-encoded signature)
- Signature is optional; when present must validate format
Completion criteria:
- [x] Signature structure serializes correctly
- [x] JWK public key format validated
- [x] Algorithm enum matches CycloneDX spec
### TASK-013-009 - Unit tests for new CycloneDX 1.7 features
Status: DONE
Dependency: TASK-013-007
Owners: QA
Task description:
- Create test fixtures with all CycloneDX 1.7 features
- Tests for:
- Services generation and determinism
- Formulation with workflows
- ModelCard complete structure
- CryptoProperties for each asset type
- Annotations with multiple subjects
- Compositions with all aggregate types
- Declarations with attestations
- Definitions with standards
- Component-level signature
- BOM-level signature
- Round-trip tests: generate -> parse -> re-generate -> compare hash
Completion criteria:
- [x] >95% code coverage on new writer code
- [x] All CycloneDX 1.7 sections have dedicated tests
- [x] Determinism verified via golden hash comparison
- [x] Tests pass in CI
### TASK-013-010 - Schema validation integration
Status: DONE
Dependency: TASK-013-009
Owners: QA
Task description:
- Add schema validation step using `docs/schemas/cyclonedx-bom-1.7.schema.json`
- Validate writer output against official CycloneDX 1.7 JSON schema
- Fail tests if schema validation errors occur
Completion criteria:
- [x] Schema validation integrated into test suite
- [x] All generated BOMs pass schema validation
- [x] CI fails on schema violations
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created from SBOM capability assessment | Planning |
| 2026-01-20 | Completed TASK-013-001 through TASK-013-010; added CycloneDX 1.7 fixtures/tests, schema validation, and doc/schema updates. Tests: `dotnet test src/Attestor/__Tests/StellaOps.Attestor.StandardPredicates.Tests/StellaOps.Attestor.StandardPredicates.Tests.csproj --no-build -v minimal`. | Developer |
## Decisions & Risks
- **Decision**: Maintain backwards compatibility by keeping existing SbomDocument fields; new fields are additive
- **Risk**: CycloneDX.Core NuGet package may not fully support 1.7 types yet; mitigation is using custom models
- **Risk**: Large model expansion may impact memory for huge SBOMs; mitigation is lazy evaluation where possible
- **Decision**: Signatures are serialized but NOT generated/verified by writer (signing is handled by Signer module)
- **Decision**: Accept `urn:sha256` serialNumber format in `docs/schemas/cyclonedx-bom-1.7.schema.json` to align deterministic SBOM guidance in `docs/sboms/DETERMINISM.md`.
- **Risk**: Required advisory `docs/product/advisories/14-Dec-2025 - Proof and Evidence Chain Technical Reference.md` is missing; unable to confirm guidance. Document when available.
## Next Checkpoints
- TASK-013-002 completion: Writer functional with 1.7 spec
- TASK-013-009 completion: Full test coverage
- TASK-013-010 completion: Schema validation green

420
docs/implplan/SPRINT_20260119_014_Attestor_spdx_3.0.1_generation.md
View File

@@ -1,420 +0,0 @@
# Sprint 20260119_014 · SPDX 3.0.1 Full Generation Support
## Topic & Scope
- Upgrade SpdxWriter from spec version 3.0 to 3.0.1 with full feature coverage
- Implement all SPDX 3.0.1 profiles: Core, Software, Security, Licensing, Build, AI, Dataset, Lite
- Support proper JSON-LD structure with @context, @graph, namespaceMap, imports
- Extend SbomDocument internal model to carry all SPDX 3.0.1 concepts
- Maintain deterministic output (RFC 8785 canonicalization)
- Working directory: `src/Attestor/__Libraries/StellaOps.Attestor.StandardPredicates/`
- Expected evidence: Unit tests, round-trip tests, schema validation tests
## Dependencies & Concurrency
- No upstream blockers
- Can run in parallel with SPRINT_20260119_013 (CycloneDX 1.7)
- Shares SbomDocument model with CycloneDX sprint
## Documentation Prerequisites
- SPDX 3.0.1 specification: https://spdx.github.io/spdx-spec/v3.0.1/
- Schema file: `docs/schemas/spdx-jsonld-3.0.1.schema.json`
- Existing writer: `src/Attestor/__Libraries/StellaOps.Attestor.StandardPredicates/Writers/SpdxWriter.cs`
- SPDX 3.0 model documentation: https://spdx.github.io/spdx-spec/v3.0.1/model/
## Delivery Tracker
### TASK-014-001 - Upgrade context and spec version to 3.0.1
Status: DOING
Dependency: none
Owners: Developer
Task description:
- Update `SpecVersion` constant from "3.0" to "3.0.1"
- Update `Context` constant to "https://spdx.org/rdf/3.0.1/spdx-context.jsonld"
- Update `SpdxVersion` output format to "SPDX-3.0.1"
- Ensure JSON-LD @context is correctly placed
Completion criteria:
- [x] Context URL updated to 3.0.1
- [x] spdxVersion field shows "SPDX-3.0.1"
- [ ] JSON-LD structure validates
### TASK-014-002 - Implement Core profile elements
Status: DOING
Dependency: TASK-014-001
Owners: Developer
Task description:
- Implement base Element type with:
- spdxId (required)
- @type
- name
- summary
- description
- comment
- creationInfo (shared CreationInfo object)
- verifiedUsing (IntegrityMethod[])
- externalRef (ExternalRef[])
- externalIdentifier (ExternalIdentifier[])
- extension (Extension[])
- Implement CreationInfo structure:
- specVersion
- created (datetime)
- createdBy (Agent[])
- createdUsing (Tool[])
- profile (ProfileIdentifier[])
- dataLicense
- Implement Agent types: Person, Organization, SoftwareAgent
- Implement Tool element
- Implement Relationship element with all relationship types
Completion criteria:
- [ ] All Core profile elements serializable
- [ ] CreationInfo shared correctly across elements
- [ ] Agent types properly distinguished
- [ ] Relationship types cover full SPDX 3.0.1 enumeration
### TASK-014-003 - Implement Software profile elements
Status: DONE
Dependency: TASK-014-002
Owners: Developer
Task description:
- Implement Package element (extends Artifact):
- packageUrl (purl)
- downloadLocation
- packageVersion
- homePage
- sourceInfo
- primaryPurpose
- additionalPurpose
- contentIdentifier
- Implement File element:
- fileName
- fileKind
- contentType
- Implement Snippet element:
- snippetFromFile
- byteRange
- lineRange
- Implement SoftwareArtifact base:
- copyrightText
- attributionText
- originatedBy
- suppliedBy
- builtTime
- releaseTime
- validUntilTime
- Implement SbomType enumeration: analyzed, build, deployed, design, runtime, source
Completion criteria:
- [x] Package, File, Snippet elements work
- [x] Software artifact metadata complete
- [x] SBOM type properly declared
### TASK-014-004 - Implement Security profile elements
Status: DONE
Dependency: TASK-014-003
Owners: Developer
Task description:
- Implement Vulnerability element:
- summary
- description
- modifiedTime
- publishedTime
- withdrawnTime
- Implement VulnAssessmentRelationship:
- assessedElement
- suppliedBy
- publishedTime
- modifiedTime
- Implement specific assessment types:
- CvssV2VulnAssessmentRelationship
- CvssV3VulnAssessmentRelationship
- CvssV4VulnAssessmentRelationship
- EpssVulnAssessmentRelationship
- ExploitCatalogVulnAssessmentRelationship
- SsvcVulnAssessmentRelationship
- VexAffectedVulnAssessmentRelationship
- VexFixedVulnAssessmentRelationship
- VexNotAffectedVulnAssessmentRelationship
- VexUnderInvestigationVulnAssessmentRelationship
Completion criteria:
- [x] All vulnerability assessment types implemented
- [x] CVSS v2/v3/v4 scores serialized correctly
- [x] VEX statements map to appropriate relationship types
### TASK-014-005 - Implement Licensing profile elements
Status: TODO
Dependency: TASK-014-002
Owners: Developer
Task description:
- Implement AnyLicenseInfo base type
- Implement license types:
- ListedLicense (SPDX license list reference)
- CustomLicense (user-defined)
- WithAdditionOperator
- OrLaterOperator
- ConjunctiveLicenseSet (AND)
- DisjunctiveLicenseSet (OR)
- NoAssertionLicense
- NoneLicense
- Implement LicenseAddition for exceptions
- Support license expressions parsing and serialization
Completion criteria:
- [ ] All license types serialize correctly
- [ ] Complex expressions (AND/OR/WITH) work
- [ ] SPDX license IDs validated against list
### TASK-014-006 - Implement Build profile elements
Status: DONE
Dependency: TASK-014-003
Owners: Developer
Task description:
- Implement Build element:
- buildId
- buildType
- buildStartTime
- buildEndTime
- configSourceEntrypoint
- configSourceDigest
- configSourceUri
- environment (key-value pairs)
- parameters (key-value pairs)
- Link Build to produced artifacts via relationships
Completion criteria:
- [x] Build element captures full build metadata
- [x] Environment and parameters serialize as maps
- [x] Build-to-artifact relationships work
### TASK-014-007 - Implement AI profile elements
Status: TODO
Dependency: TASK-014-003
Owners: Developer
Task description:
- Implement AIPackage element extending Package:
- autonomyType
- domain
- energyConsumption
- hyperparameter
- informationAboutApplication
- informationAboutTraining
- limitation
- metric
- metricDecisionThreshold
- modelDataPreprocessing
- modelExplainability
- safetyRiskAssessment
- sensitivePersonalInformation
- standardCompliance
- typeOfModel
- useSensitivePersonalInformation
- Implement SafetyRiskAssessmentType enumeration
Completion criteria:
- [ ] AI/ML model metadata fully captured
- [ ] Metrics and hyperparameters serialized
- [ ] Safety risk assessment included
### TASK-014-008 - Implement Dataset profile elements
Status: TODO
Dependency: TASK-014-007
Owners: Developer
Task description:
- Implement Dataset element extending Package:
- datasetType
- dataCollectionProcess
- dataPreprocessing
- datasetSize
- intendedUse
- knownBias
- sensitivePersonalInformation
- sensor
- Implement DatasetAvailability enumeration
- Implement ConfidentialityLevel enumeration
Completion criteria:
- [ ] Dataset metadata fully captured
- [ ] Availability and confidentiality levels work
- [ ] Integration with AI profile for training data
### TASK-014-009 - Implement Lite profile support
Status: TODO
Dependency: TASK-014-003
Owners: Developer
Task description:
- Support minimal SBOM output using Lite profile subset:
- SpdxDocument root
- Package elements with required fields only
- Basic relationships (DEPENDS_ON, CONTAINS)
- Add Lite profile option to SpdxWriter configuration
- Validate output against Lite profile constraints
Completion criteria:
- [ ] Lite profile option available
- [ ] Minimal output meets Lite spec
- [ ] Non-Lite fields excluded when Lite selected
### TASK-014-010 - Namespace and import support
Status: TODO
Dependency: TASK-014-002
Owners: Developer
Task description:
- Implement namespaceMap for cross-document references:
- prefix
- namespace (URI)
- Implement imports array for external document references
- Support external spdxId references with namespace prefixes
- Validate URI formats
Completion criteria:
- [ ] Namespace prefixes declared correctly
- [ ] External imports listed
- [ ] Cross-document references resolve
### TASK-014-011 - Integrity methods and external references
Status: DOING
Dependency: TASK-014-002
Owners: Developer
Task description:
- Implement IntegrityMethod types:
- Hash (algorithm, hashValue)
- Signature (algorithm, signature, keyId, publicKey)
- Support hash algorithms: SHA256, SHA384, SHA512, SHA3-256, SHA3-384, SHA3-512, BLAKE2b-256, BLAKE2b-384, BLAKE2b-512, MD5, SHA1, MD2, MD4, MD6, BLAKE2b-512, ADLER32
- Implement ExternalRef:
- externalRefType (BOWER, MAVEN-CENTRAL, NPM, NUGET, PURL, SWID, etc.)
- locator
- contentType
- comment
- Implement ExternalIdentifier:
- externalIdentifierType (CPE22, CPE23, CVE, GITOID, PURL, SWHID, SWID, URN)
- identifier
- identifierLocator
- issuingAuthority
- comment
Completion criteria:
- [ ] All integrity method types work
- [ ] External references categorized correctly
- [ ] External identifiers validated by type
### TASK-014-012 - Relationship types enumeration
Status: TODO
Dependency: TASK-014-002
Owners: Developer
Task description:
- Implement all SPDX 3.0.1 relationship types:
- Core: DESCRIBES, DESCRIBED_BY, CONTAINS, CONTAINED_BY, ANCESTOR_OF, DESCENDANT_OF, VARIANT_OF, HAS_DISTRIBUTION_ARTIFACT, DISTRIBUTION_ARTIFACT_OF, GENERATES, GENERATED_FROM, COPY_OF, FILE_ADDED, FILE_DELETED, FILE_MODIFIED, EXPANDED_FROM_ARCHIVE, DYNAMIC_LINK, STATIC_LINK, DATA_FILE_OF, TEST_CASE_OF, BUILD_TOOL_OF, DEV_TOOL_OF, TEST_TOOL_OF, DOCUMENTATION_OF, OPTIONAL_COMPONENT_OF, PROVIDED_DEPENDENCY_OF, TEST_DEPENDENCY_OF, DEV_DEPENDENCY_OF, DEPENDENCY_OF, DEPENDS_ON, PREREQUISITE_FOR, HAS_PREREQUISITE, OTHER
- Security: AFFECTS, FIXED_IN, FOUND_BY, REPORTED_BY
- Lifecycle: PATCH_FOR, INPUT_OF, OUTPUT_OF, AVAILABLE_FROM
- Map internal SbomRelationshipType enum to SPDX types
Completion criteria:
- [ ] All relationship types serializable
- [ ] Bidirectional types maintain consistency
- [ ] Security relationships link to vulnerabilities
### TASK-014-013 - Extension support
Status: TODO
Dependency: TASK-014-002
Owners: Developer
Task description:
- Implement Extension mechanism:
- Define extension point on any element
- Support extension namespaces
- Serialize custom properties within extensions
- Document extension usage for Stella Ops custom metadata
Completion criteria:
- [ ] Extensions serialize correctly
- [ ] Namespace isolation maintained
- [ ] Round-trip preserves extension data
### TASK-014-014 - Unit tests for SPDX 3.0.1 profiles
Status: TODO
Dependency: TASK-014-011
Owners: QA
Task description:
- Create test fixtures for each profile:
- Core profile: Element hierarchy, relationships, agents
- Software profile: Packages, Files, Snippets
- Security profile: Vulnerabilities, VEX assessments
- Licensing profile: Complex license expressions
- Build profile: Build metadata
- AI profile: ML model packages
- Dataset profile: Training data
- Lite profile: Minimal output
- Round-trip tests: generate -> parse -> re-generate -> compare hash
- Cross-document reference tests with namespaces
Completion criteria:
- [ ] >95% code coverage on new writer code
- [ ] All profiles have dedicated test suites
- [ ] Determinism verified via golden hash comparison
- [ ] Tests pass in CI
### TASK-014-015 - Schema validation integration
Status: TODO
Dependency: TASK-014-014
Owners: QA
Task description:
- Add schema validation step using `docs/schemas/spdx-jsonld-3.0.1.schema.json`
- Validate writer output against official SPDX 3.0.1 JSON-LD schema
- Validate JSON-LD @context resolution
- Fail tests if schema validation errors occur
Completion criteria:
- [ ] Schema validation integrated into test suite
- [ ] All generated documents pass schema validation
- [ ] JSON-LD context validates
- [ ] CI fails on schema violations
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created from SBOM capability assessment | Planning |
| 2026-01-20 | TASK-014-001/002: Added deterministic SPDX 3.0.1 writer baseline (context + spdxVersion, core document/package/relationship emission, ordering rules). Schema validation and full profile coverage pending. | Developer |
| 2026-01-20 | QA: Ran SpdxDeterminismTests (`dotnet test src/Attestor/__Tests/StellaOps.Attestor.StandardPredicates.Tests/StellaOps.Attestor.StandardPredicates.Tests.csproj --filter FullyQualifiedName~SpdxDeterminismTests`). Passed. | QA |
| 2026-01-20 | TASK-014-011: Added externalRef serialization for package external references with deterministic ordering; updated tests and re-ran SpdxDeterminismTests (pass). | Developer/QA |
| 2026-01-20 | TASK-014-011: Added external identifier and signature integrity serialization; updated SPDX tests and re-ran SpdxDeterminismTests (pass). | Developer/QA |
| 2026-01-20 | TASK-014-003/006: Added SPDX software package/file/snippet and build profile emission (including output relationships), added SpdxWriterSoftwareProfileTests, and ran `dotnet test src/Attestor/__Tests/StellaOps.Attestor.StandardPredicates.Tests/StellaOps.Attestor.StandardPredicates.Tests.csproj --filter FullyQualifiedName~SpdxWriterSoftwareProfileTests` (pass). Docs updated in `docs/modules/attestor/guides/README.md`. | Developer/QA/Documentation |
| 2026-01-20 | TASK-014-004: Added SPDX security vulnerability + assessment emission (affects and assessment relationships), added SpdxWriterSecurityProfileTests, and ran `dotnet test src/Attestor/__Tests/StellaOps.Attestor.StandardPredicates.Tests/StellaOps.Attestor.StandardPredicates.Tests.csproj --filter FullyQualifiedName~SpdxWriterSecurityProfileTests|FullyQualifiedName~SpdxWriterSoftwareProfileTests` (pass). Docs updated in `docs/modules/attestor/guides/README.md`. | Developer/QA/Documentation |
## Decisions & Risks
- **Decision**: Support all 8 SPDX 3.0.1 profiles for completeness
- **Decision**: Lite profile is opt-in via configuration, full profile is default
- **Risk**: JSON-LD context loading may require network access; mitigation is bundling context file
- **Risk**: AI/Dataset profiles are new and tooling support varies; mitigation is thorough testing
- **Decision**: Use same SbomDocument model as CycloneDX where concepts overlap (components, relationships, vulnerabilities)
- **Risk**: Relationship type mapping is partial until full SPDX 3.0.1 coverage is implemented; mitigation is defaulting to `Other` with follow-up tasks in this sprint.
- **Docs**: `docs/modules/attestor/guides/README.md` updated with SPDX 3.0.1 writer baseline coverage note.
- **Docs**: `docs/modules/attestor/guides/README.md` updated with external reference and hash coverage.
- **Docs**: `docs/modules/attestor/guides/README.md` updated with external identifier and signature coverage.
- **Docs**: `docs/modules/attestor/guides/README.md` updated with SPDX 3.0.1 software/build profile coverage.
- **Cross-module**: Added `src/__Libraries/StellaOps.Artifact.Infrastructure/AGENTS.md` per user request to document artifact infrastructure charter.
## Next Checkpoints
- TASK-014-003 completion: Software profile functional
- TASK-014-004 completion: Security profile functional (VEX integration)
- TASK-014-014 completion: Full test coverage
- TASK-014-015 completion: Schema validation green

700
docs/implplan/SPRINT_20260119_015_Concelier_sbom_full_extraction.md
View File

@@ -1,700 +0,0 @@
# Sprint 20260119_015 · Full SBOM Extraction for CycloneDX 1.7 and SPDX 3.0.1
## Topic & Scope
- Upgrade SbomParser to extract ALL fields from CycloneDX 1.7 and SPDX 3.0.1 (not just PURL/CPE)
- Create enriched internal model (ParsedSbom) that carries full SBOM data for downstream consumers
- Enable Scanner, Policy, and other modules to access services, crypto, ML, build, and compliance metadata
- Working directory: `src/Concelier/__Libraries/StellaOps.Concelier.SbomIntegration/`
- Secondary: `src/__Libraries/StellaOps.Artifact.Core/`
- Expected evidence: Unit tests, integration tests with downstream consumers
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_013 (CycloneDX 1.7 model), SPRINT_20260119_014 (SPDX 3.0.1 model)
- Blocks: All downstream scanner utilization sprints (016-023)
- Can begin model work before generation sprints complete
## Documentation Prerequisites
- CycloneDX 1.7 spec: https://cyclonedx.org/docs/1.7/
- SPDX 3.0.1 spec: https://spdx.github.io/spdx-spec/v3.0.1/
- Existing parser: `src/Concelier/__Libraries/StellaOps.Concelier.SbomIntegration/Parsing/SbomParser.cs`
- Existing extractor: `src/__Libraries/StellaOps.Artifact.Core/CycloneDxExtractor.cs`
## Delivery Tracker
### TASK-015-001 - Design ParsedSbom enriched model
Status: DOING
Dependency: none
Owners: Developer
Task description:
- Design `ParsedSbom` record as the enriched extraction result:
```csharp
public sealed record ParsedSbom
{
// Identity
public required string Format { get; init; } // "cyclonedx" | "spdx"
public required string SpecVersion { get; init; }
public required string SerialNumber { get; init; }
// Core components (existing)
public ImmutableArray<ParsedComponent> Components { get; init; }
// NEW: Services (CycloneDX 1.4+)
public ImmutableArray<ParsedService> Services { get; init; }
// NEW: Dependencies graph
public ImmutableArray<ParsedDependency> Dependencies { get; init; }
// NEW: Compositions
public ImmutableArray<ParsedComposition> Compositions { get; init; }
// NEW: Vulnerabilities embedded in SBOM
public ImmutableArray<ParsedVulnerability> Vulnerabilities { get; init; }
// NEW: Formulation/Build metadata
public ParsedFormulation? Formulation { get; init; }
public ParsedBuildInfo? BuildInfo { get; init; }
// NEW: Declarations and definitions
public ParsedDeclarations? Declarations { get; init; }
public ParsedDefinitions? Definitions { get; init; }
// NEW: Annotations
public ImmutableArray<ParsedAnnotation> Annotations { get; init; }
// Metadata
public ParsedSbomMetadata Metadata { get; init; }
}
```
- Design `ParsedComponent` with ALL fields:
- Core: bomRef, type, name, version, purl, cpe, group, publisher, description
- Hashes: ImmutableArray<ParsedHash>
- Licenses: ImmutableArray<ParsedLicense> (full objects, not just IDs)
- ExternalReferences: ImmutableArray<ParsedExternalRef>
- Properties: ImmutableDictionary<string, string>
- Evidence: ParsedEvidence? (identity, occurrences, callstack)
- Pedigree: ParsedPedigree? (ancestors, variants, commits, patches)
- CryptoProperties: ParsedCryptoProperties?
- ModelCard: ParsedModelCard?
- Supplier: ParsedOrganization?
- Manufacturer: ParsedOrganization?
- Scope: ComponentScope enum
- Modified: bool
Completion criteria:
- [ ] ParsedSbom model covers all CycloneDX 1.7 and SPDX 3.0.1 concepts
- [ ] All collections immutable
- [ ] XML documentation complete
- [ ] Model placed in shared abstractions library
### TASK-015-002 - Implement ParsedService model
Status: DOING
Dependency: TASK-015-001
Owners: Developer
Task description:
- Create `ParsedService` record:
```csharp
public sealed record ParsedService
{
public required string BomRef { get; init; }
public string? Provider { get; init; }
public string? Group { get; init; }
public required string Name { get; init; }
public string? Version { get; init; }
public string? Description { get; init; }
public ImmutableArray<string> Endpoints { get; init; }
public bool Authenticated { get; init; }
public bool CrossesTrustBoundary { get; init; }
public ImmutableArray<ParsedDataFlow> Data { get; init; }
public ImmutableArray<ParsedLicense> Licenses { get; init; }
public ImmutableArray<ParsedExternalRef> ExternalReferences { get; init; }
public ImmutableArray<ParsedService> NestedServices { get; init; }
public ImmutableDictionary<string, string> Properties { get; init; }
}
```
- Create `ParsedDataFlow` for service data classification:
- Flow direction (inbound/outbound/bidirectional/unknown)
- Data classification
- Source/destination references
Completion criteria:
- [ ] Full service model with all CycloneDX properties
- [ ] Nested services support recursive structures
- [ ] Data flows captured for security analysis
### TASK-015-003 - Implement ParsedCryptoProperties model
Status: DOING
Dependency: TASK-015-001
Owners: Developer
Task description:
- Create `ParsedCryptoProperties` record:
```csharp
public sealed record ParsedCryptoProperties
{
public CryptoAssetType AssetType { get; init; }
public ParsedAlgorithmProperties? AlgorithmProperties { get; init; }
public ParsedCertificateProperties? CertificateProperties { get; init; }
public ParsedProtocolProperties? ProtocolProperties { get; init; }
public ParsedRelatedCryptoMaterial? RelatedCryptoMaterial { get; init; }
public string? Oid { get; init; }
}
```
- Create supporting records:
- `ParsedAlgorithmProperties`: primitive, parameterSetIdentifier, curve, executionEnvironment, implementationPlatform, certificationLevel, mode, padding, cryptoFunctions, classicalSecurityLevel, nistQuantumSecurityLevel
- `ParsedCertificateProperties`: subjectName, issuerName, notValidBefore, notValidAfter, signatureAlgorithmRef, subjectPublicKeyRef, certificateFormat, certificateExtension
- `ParsedProtocolProperties`: type, version, cipherSuites, ikev2TransformTypes, cryptoRefArray
- Create enums: CryptoAssetType, CryptoPrimitive, CryptoMode, CryptoPadding, CryptoExecutionEnvironment, CertificationLevel
Completion criteria:
- [ ] Full CBOM (Cryptographic BOM) model
- [ ] All algorithm properties captured
- [ ] Certificate chain information preserved
- [ ] Protocol cipher suites extracted
### TASK-015-004 - Implement ParsedModelCard model
Status: DOING
Dependency: TASK-015-001
Owners: Developer
Task description:
- Create `ParsedModelCard` record:
```csharp
public sealed record ParsedModelCard
{
public string? BomRef { get; init; }
public ParsedModelParameters? ModelParameters { get; init; }
public ParsedQuantitativeAnalysis? QuantitativeAnalysis { get; init; }
public ParsedConsiderations? Considerations { get; init; }
}
```
- Create `ParsedModelParameters`:
- Approach (task, architectureFamily, modelArchitecture, datasets, inputs, outputs)
- Datasets: ImmutableArray<ParsedDatasetRef>
- Inputs/Outputs: ImmutableArray<ParsedInputOutput> with format descriptions
- Create `ParsedQuantitativeAnalysis`:
- PerformanceMetrics: ImmutableArray<ParsedPerformanceMetric>
- Graphics: ImmutableArray<ParsedGraphic>
- Create `ParsedConsiderations`:
- Users, UseCases, TechnicalLimitations
- EthicalConsiderations, FairnessAssessments
- EnvironmentalConsiderations
- For SPDX 3.0.1 AI profile, map:
- autonomyType, domain, energyConsumption, hyperparameter
- safetyRiskAssessment, typeOfModel, limitations, metrics
Completion criteria:
- [ ] Full ML model metadata captured
- [ ] Maps both CycloneDX modelCard and SPDX AI profile
- [ ] Training datasets referenced
- [ ] Safety assessments preserved
### TASK-015-005 - Implement ParsedFormulation and ParsedBuildInfo
Status: DOING
Dependency: TASK-015-001
Owners: Developer
Task description:
- Create `ParsedFormulation` record (CycloneDX):
```csharp
public sealed record ParsedFormulation
{
public string? BomRef { get; init; }
public ImmutableArray<ParsedFormula> Components { get; init; }
public ImmutableArray<ParsedWorkflow> Workflows { get; init; }
public ImmutableArray<ParsedTask> Tasks { get; init; }
public ImmutableDictionary<string, string> Properties { get; init; }
}
```
- Create `ParsedBuildInfo` record (SPDX 3.0.1 Build profile):
```csharp
public sealed record ParsedBuildInfo
{
public required string BuildId { get; init; }
public string? BuildType { get; init; }
public DateTimeOffset? BuildStartTime { get; init; }
public DateTimeOffset? BuildEndTime { get; init; }
public string? ConfigSourceEntrypoint { get; init; }
public string? ConfigSourceDigest { get; init; }
public string? ConfigSourceUri { get; init; }
public ImmutableDictionary<string, string> Environment { get; init; }
public ImmutableDictionary<string, string> Parameters { get; init; }
}
```
- Normalize both formats into unified build provenance representation
Completion criteria:
- [ ] CycloneDX formulation fully parsed
- [ ] SPDX Build profile fully parsed
- [ ] Unified representation for downstream consumers
- [ ] Build environment captured for reproducibility
### TASK-015-006 - Implement ParsedVulnerability and VEX models
Status: DOING
Dependency: TASK-015-001
Owners: Developer
Task description:
- Create `ParsedVulnerability` record:
```csharp
public sealed record ParsedVulnerability
{
public required string Id { get; init; }
public string? Source { get; init; }
public string? Description { get; init; }
public string? Detail { get; init; }
public string? Recommendation { get; init; }
public ImmutableArray<string> Cwes { get; init; }
public ImmutableArray<ParsedVulnRating> Ratings { get; init; }
public ImmutableArray<ParsedVulnAffects> Affects { get; init; }
public ParsedVulnAnalysis? Analysis { get; init; }
public DateTimeOffset? Published { get; init; }
public DateTimeOffset? Updated { get; init; }
}
```
- Create `ParsedVulnAnalysis` for VEX data:
```csharp
public sealed record ParsedVulnAnalysis
{
public VexState State { get; init; } // exploitable, in_triage, false_positive, not_affected, fixed
public VexJustification? Justification { get; init; }
public ImmutableArray<string> Response { get; init; } // can_not_fix, will_not_fix, update, rollback, workaround_available
public string? Detail { get; init; }
public DateTimeOffset? FirstIssued { get; init; }
public DateTimeOffset? LastUpdated { get; init; }
}
```
- Map SPDX 3.0.1 Security profile VEX relationships to same model
Completion criteria:
- [ ] Embedded vulnerabilities extracted from CycloneDX
- [ ] VEX analysis/state preserved
- [ ] SPDX VEX relationships mapped
- [ ] CVSS ratings (v2, v3, v4) parsed
### TASK-015-007 - Implement ParsedLicense full model
Status: DOING
Dependency: TASK-015-001
Owners: Developer
Task description:
- Create `ParsedLicense` record with full detail:
```csharp
public sealed record ParsedLicense
{
public string? SpdxId { get; init; } // SPDX license ID
public string? Name { get; init; } // Custom license name
public string? Url { get; init; } // License text URL
public string? Text { get; init; } // Full license text
public ParsedLicenseExpression? Expression { get; init; } // Complex expressions
public ImmutableArray<string> Acknowledgements { get; init; }
}
```
- Create `ParsedLicenseExpression` for complex expressions:
```csharp
public abstract record ParsedLicenseExpression;
public sealed record SimpleLicense(string Id) : ParsedLicenseExpression;
public sealed record WithException(ParsedLicenseExpression License, string Exception) : ParsedLicenseExpression;
public sealed record OrLater(string LicenseId) : ParsedLicenseExpression;
public sealed record ConjunctiveSet(ImmutableArray<ParsedLicenseExpression> Members) : ParsedLicenseExpression; // AND
public sealed record DisjunctiveSet(ImmutableArray<ParsedLicenseExpression> Members) : ParsedLicenseExpression; // OR
```
- Parse SPDX license expressions (e.g., "MIT OR Apache-2.0", "GPL-2.0-only WITH Classpath-exception-2.0")
Completion criteria:
- [ ] Full license objects extracted (not just ID)
- [ ] Complex expressions parsed into AST
- [ ] License text preserved when available
- [ ] SPDX 3.0.1 Licensing profile mapped
### TASK-015-007a - Implement CycloneDX license extraction
Status: DOING
Dependency: TASK-015-007
Owners: Developer
Task description:
- Extract ALL license fields from CycloneDX components:
```csharp
// CycloneDX license structure to parse:
// components[].licenses[] - array of LicenseChoice
// - license.id (SPDX ID)
// - license.name (custom name)
// - license.text.content (full text)
// - license.text.contentType (text/plain, text/markdown)
// - license.text.encoding (base64 if encoded)
// - license.url (license URL)
// - expression (SPDX expression string)
// - license.licensing.licensor
// - license.licensing.licensee
// - license.licensing.purchaser
// - license.licensing.purchaseOrder
// - license.licensing.licenseTypes[]
// - license.licensing.lastRenewal
// - license.licensing.expiration
// - license.licensing.altIds[]
// - license.properties[]
```
- Handle both `license` object and `expression` string in LicenseChoice
- Parse SPDX expressions using existing `SpdxLicenseExpressions` parser
- Decode base64-encoded license text
- Extract licensing metadata (commercial license info)
- Map to `ParsedLicense` model
Completion criteria:
- [ ] All CycloneDX license fields extracted
- [ ] Expression string parsed to AST
- [ ] Base64 license text decoded
- [ ] Commercial licensing metadata preserved
- [ ] Both id and name licenses handled
### TASK-015-007b - Implement SPDX Licensing profile extraction
Status: DOING
Dependency: TASK-015-007
Owners: Developer
Task description:
- Extract ALL license types from SPDX 3.0.1 Licensing profile:
```csharp
// SPDX 3.0.1 license types to parse from @graph:
// - ListedLicense (SPDX license list reference)
// - licenseId
// - licenseText
// - deprecatedLicenseId
// - isOsiApproved
// - isFsfFree
// - licenseComments
// - seeAlso[] (URLs)
// - standardLicenseHeader
// - standardLicenseTemplate
//
// - CustomLicense (user-defined)
// - licenseText
// - licenseComments
//
// - OrLaterOperator
// - subjectLicense
//
// - WithAdditionOperator
// - subjectLicense
// - subjectAddition (LicenseAddition reference)
//
// - ConjunctiveLicenseSet (AND)
// - member[] (license references)
//
// - DisjunctiveLicenseSet (OR)
// - member[] (license references)
//
// - LicenseAddition (exceptions)
// - additionId
// - additionText
// - standardAdditionTemplate
```
- Parse nested license expressions recursively
- Extract license text content
- Map OSI/FSF approval status
- Handle license exceptions (WITH operator)
- Map deprecated license IDs to current
Completion criteria:
- [ ] All SPDX license types parsed
- [ ] Complex expressions (AND/OR/WITH) work
- [ ] License text extracted
- [ ] OSI/FSF approval mapped
- [ ] Exceptions handled correctly
### TASK-015-007c - Implement license expression validator
Status: TODO
Dependency: TASK-015-007b
Owners: Developer
Task description:
- Create `ILicenseExpressionValidator`:
```csharp
public interface ILicenseExpressionValidator
{
LicenseValidationResult Validate(ParsedLicenseExpression expression);
LicenseValidationResult ValidateString(string spdxExpression);
}
public sealed record LicenseValidationResult
{
public bool IsValid { get; init; }
public ImmutableArray<string> Errors { get; init; }
public ImmutableArray<string> Warnings { get; init; }
public ImmutableArray<string> ReferencedLicenses { get; init; }
public ImmutableArray<string> ReferencedExceptions { get; init; }
public ImmutableArray<string> DeprecatedLicenses { get; init; }
public ImmutableArray<string> UnknownLicenses { get; init; }
}
```
- Validate against SPDX license list (600+ licenses)
- Validate against SPDX exception list (40+ exceptions)
- Flag deprecated licenses with suggested replacements
- Flag unknown licenses (LicenseRef-* is valid but flagged)
- Track all referenced licenses for inventory
Completion criteria:
- [ ] SPDX license list validation
- [ ] Exception list validation
- [ ] Deprecated license detection
- [ ] Unknown license flagging
- [ ] Complete license inventory extraction
### TASK-015-007d - Add license queries to ISbomRepository
Status: TODO
Dependency: TASK-015-011
Owners: Developer
Task description:
- Extend `ISbomRepository` with license-specific queries:
```csharp
public interface ISbomRepository
{
// ... existing methods ...
// License queries
Task<IReadOnlyList<ParsedLicense>> GetLicensesForArtifactAsync(
string artifactId, CancellationToken ct);
Task<IReadOnlyList<ParsedComponent>> GetComponentsByLicenseAsync(
string spdxId, CancellationToken ct);
Task<IReadOnlyList<ParsedComponent>> GetComponentsWithoutLicenseAsync(
string artifactId, CancellationToken ct);
Task<IReadOnlyList<ParsedComponent>> GetComponentsByLicenseCategoryAsync(
string artifactId, LicenseCategory category, CancellationToken ct);
Task<LicenseInventorySummary> GetLicenseInventoryAsync(
string artifactId, CancellationToken ct);
}
public sealed record LicenseInventorySummary
{
public int TotalComponents { get; init; }
public int ComponentsWithLicense { get; init; }
public int ComponentsWithoutLicense { get; init; }
public ImmutableDictionary<string, int> LicenseDistribution { get; init; }
public ImmutableArray<string> UniqueLicenses { get; init; }
public ImmutableArray<string> Expressions { get; init; }
}
```
- Implement PostgreSQL queries with proper indexing
- Index on license ID for fast lookups
Completion criteria:
- [ ] License queries implemented
- [ ] Category queries working
- [ ] Inventory summary generated
- [ ] Indexed for performance
### TASK-015-008 - Upgrade CycloneDxParser for 1.7 full extraction
Status: DOING
Dependency: TASK-015-007
Owners: Developer
Task description:
- Refactor `SbomParser.cs` CycloneDX handling to extract ALL fields:
- Parse `services[]` array recursively
- Parse `formulation[]` array with workflows/tasks
- Parse `components[].modelCard` when present
- Parse `components[].cryptoProperties` when present
- Parse `components[].evidence` (identity, occurrences, callstack, licenses, copyright)
- Parse `components[].pedigree` (ancestors, descendants, variants, commits, patches, notes)
- Parse `components[].swid` (tagId, name, version, tagVersion, patch)
- Parse `compositions[]` with aggregate type
- Parse `declarations` object
- Parse `definitions` object
- Parse `annotations[]` array
- Parse `vulnerabilities[]` array with full VEX analysis
- Parse `externalReferences[]` for all types (not just CPE)
- Parse `properties[]` at all levels
- Parse `signature` when present
- Maintain backwards compatibility with 1.4, 1.5, 1.6
Completion criteria:
- [ ] All CycloneDX 1.7 sections parsed
- [ ] Nested components fully traversed
- [ ] Recursive services handled
- [ ] Backwards compatible with older versions
- [ ] No data loss from incoming SBOMs
### TASK-015-009 - Upgrade SpdxParser for 3.0.1 full extraction
Status: DOING
Dependency: TASK-015-007
Owners: Developer
Task description:
- Refactor `SbomParser.cs` SPDX handling to extract ALL fields:
- Parse `@graph` elements by type:
- Package → ParsedComponent
- File → ParsedComponent (with fileKind)
- Snippet → ParsedComponent (with range)
- Vulnerability → ParsedVulnerability
- Relationship → ParsedDependency
- SpdxDocument → metadata
- Parse SPDX 3.0.1 profiles:
- Software: packages, files, snippets, SBOMType
- Security: vulnerabilities, VEX assessments (all types)
- Licensing: full license expressions
- Build: build metadata
- AI: AIPackage elements
- Dataset: Dataset elements
- Parse `creationInfo` with agents (Person, Organization, SoftwareAgent)
- Parse `verifiedUsing` integrity methods
- Parse `externalRef` and `externalIdentifier` arrays
- Parse `namespaceMap` for cross-document references
- Parse `imports` for external document references
- Maintain backwards compatibility with 2.2, 2.3
Completion criteria:
- [ ] All SPDX 3.0.1 profiles parsed
- [ ] JSON-LD @graph traversed correctly
- [ ] VEX assessment relationships mapped
- [ ] AI and Dataset profiles extracted
- [ ] Build profile extracted
- [ ] Backwards compatible with 2.x
### TASK-015-010 - Upgrade CycloneDxExtractor for full metadata
Status: DOING
Dependency: TASK-015-008
Owners: Developer
Task description:
- Refactor `CycloneDxExtractor.cs` in Artifact.Core:
- Return `ParsedSbom` instead of minimal extraction
- Extract services for artifact context
- Extract formulation for build lineage
- Extract crypto properties for compliance
- Maintain existing API for backwards compatibility (adapter layer)
Completion criteria:
- [ ] Full extraction available via new API
- [ ] Legacy API still works (returns subset)
- [ ] No breaking changes to existing consumers
### TASK-015-011 - Create ISbomRepository for enriched storage
Status: TODO
Dependency: TASK-015-010
Owners: Developer
Task description:
- Design repository interface for storing/retrieving enriched SBOMs:
```csharp
public interface ISbomRepository
{
Task<ParsedSbom?> GetBySerialNumberAsync(string serialNumber, CancellationToken ct);
Task<ParsedSbom?> GetByArtifactDigestAsync(string digest, CancellationToken ct);
Task StoreAsync(ParsedSbom sbom, CancellationToken ct);
Task<IReadOnlyList<ParsedService>> GetServicesForArtifactAsync(string artifactId, CancellationToken ct);
Task<IReadOnlyList<ParsedComponent>> GetComponentsWithCryptoAsync(string artifactId, CancellationToken ct);
Task<IReadOnlyList<ParsedVulnerability>> GetEmbeddedVulnerabilitiesAsync(string artifactId, CancellationToken ct);
}
```
- Implement PostgreSQL storage for ParsedSbom (JSON column for full document, indexed columns for queries)
Completion criteria:
- [ ] Repository interface defined
- [ ] PostgreSQL implementation complete
- [ ] Indexed queries for services, crypto, vulnerabilities
- [ ] Full SBOM round-trips correctly
### TASK-015-012 - Unit tests for full extraction
Status: TODO
Dependency: TASK-015-009
Owners: QA
Task description:
- Create test fixtures:
- CycloneDX 1.7 with all sections populated
- SPDX 3.0.1 with all profiles
- Edge cases: empty arrays, null fields, nested structures
- Test scenarios:
- Services extraction with nested services
- Crypto properties for all asset types
- ModelCard with full quantitative analysis
- Formulation with complex workflows
- VEX with all states and justifications
- **License extraction comprehensive tests:**
- Simple SPDX IDs (MIT, Apache-2.0)
- Complex expressions (MIT OR Apache-2.0)
- Compound expressions ((MIT OR Apache-2.0) AND BSD-3-Clause)
- WITH exceptions (Apache-2.0 WITH LLVM-exception)
- Or-later licenses (GPL-2.0+)
- Custom licenses (LicenseRef-*)
- License text extraction (base64 and plaintext)
- Commercial licensing metadata
- SPDX Licensing profile all types
- Components without licenses
- Mixed license formats in same SBOM
- Build info from both formats
- Verify no data loss: generate → parse → serialize → compare
Completion criteria:
- [ ] >95% code coverage on parser code
- [ ] All CycloneDX 1.7 features tested
- [ ] All SPDX 3.0.1 profiles tested
- [ ] Round-trip integrity verified
- [ ] Tests pass in CI
### TASK-015-013 - Integration tests with downstream consumers
Status: TODO
Dependency: TASK-015-012
Owners: QA
Task description:
- Create integration tests verifying downstream modules can access:
- Scanner: services, crypto, modelCard, vulnerabilities
- Policy: licenses, compositions, declarations
- Concelier: all extracted data via ISbomRepository
- Test data flow from SBOM ingestion to module consumption
Completion criteria:
- [ ] Scanner can query ParsedService data
- [ ] Scanner can query ParsedCryptoProperties
- [ ] Policy can evaluate license expressions
- [ ] All integration paths verified
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for full SBOM extraction | Planning |
| 2026-01-20 | TASK-015-001..007: Added ParsedSbom model scaffolding and supporting records (services, crypto, model card, formulation, vulnerabilities, licenses). TASK-015-010 blocked due to missing module AGENTS in Artifact.Core. | Developer |
| 2026-01-20 | TASK-015-008/009: Added ParsedSbomParser with initial CycloneDX 1.7 + SPDX 3.0.1 extraction (metadata, components, dependencies, services) and unit tests; remaining fields still pending. | Developer |
| 2026-01-20 | QA: Ran ParsedSbomParserTests (`dotnet test src/Concelier/__Tests/StellaOps.Concelier.SbomIntegration.Tests/StellaOps.Concelier.SbomIntegration.Tests.csproj --filter FullyQualifiedName~ParsedSbomParserTests`). Passed. | QA |
| 2026-01-20 | Docs: Documented ParsedSbom extraction coverage in `docs/modules/concelier/sbom-learning-api.md`. | Documentation |
| 2026-01-20 | TASK-015-007/008/009: Expanded CycloneDX/SPDX license parsing (expressions, terms, base64 text), external references, and SPDX verifiedUsing hashes. Updated unit tests and re-ran ParsedSbomParserTests (pass). | Developer/QA |
| 2026-01-20 | Docs: Updated SBOM extraction coverage in `docs/modules/concelier/sbom-learning-api.md` to reflect license and external reference parsing. | Documentation |
| 2026-01-20 | TASK-015-008: Expanded CycloneDX component parsing (scope/modified, supplier/manufacturer, evidence, pedigree, cryptoProperties, modelCard); updated unit tests and re-ran ParsedSbomParserTests (`dotnet test src/Concelier/__Tests/StellaOps.Concelier.SbomIntegration.Tests/StellaOps.Concelier.SbomIntegration.Tests.csproj --filter FullyQualifiedName~ParsedSbomParserTests`) (pass). | Developer/QA |
| 2026-01-20 | Docs: Updated SBOM extraction coverage in `docs/modules/concelier/sbom-learning-api.md` to include CycloneDX component enrichment. | Documentation |
| 2026-01-20 | TASK-015-010: Added `src/__Libraries/StellaOps.Artifact.Core/AGENTS.md` to unblock extractor work. | Developer |
| 2026-01-20 | TASK-015-005/008: Added CycloneDX formulation parsing + assertions in ParsedSbomParserTests. | Developer/QA |
| 2026-01-20 | TASK-015-010: Refactored CycloneDxExtractor to expose ParsedSbom extraction and adapter mapping; added Concelier reference and framework reference; removed redundant package refs; fixed CA2022 ReadAsync warnings. | Developer |
| 2026-01-20 | TASK-015-010: Added StatusCodes import and optional continuation token defaults in ArtifactController to restore ASP.NET Core compilation. | Developer |
| 2026-01-20 | TASK-015-005/009: Added SPDX build profile parsing (buildId, timestamps, config source, env/params) and test coverage. | Developer/QA |
| 2026-01-20 | QA: `dotnet test src/Concelier/__Tests/StellaOps.Concelier.SbomIntegration.Tests/StellaOps.Concelier.SbomIntegration.Tests.csproj --filter FullyQualifiedName~ParsedSbomParserTests` (pass). `dotnet test src/__Libraries/StellaOps.Artifact.Core.Tests/StellaOps.Artifact.Core.Tests.csproj --filter FullyQualifiedName~CycloneDxExtractorTests` failed due to Artifact.Infrastructure compile errors (ArtifactType missing) and NU1504 duplicate package warnings. | QA |
| 2026-01-20 | Docs: Updated `docs/modules/concelier/sbom-learning-api.md` to include formulation extraction coverage. | Documentation |
## Decisions & Risks
- **Decision**: Create new ParsedSbom model rather than extending existing to avoid breaking changes
- **Decision**: Stage ParsedSbom models in SbomIntegration while shared abstraction placement is confirmed.
- **Decision**: Store full JSON in database with indexed query columns for performance
- **Risk**: Large SBOMs with full extraction may impact memory; mitigation is streaming parser for huge files
- **Risk**: SPDX 3.0.1 profile detection may be ambiguous; mitigation is explicit profile declaration check
- **Decision**: Maintain backwards compatibility with existing minimal extraction API
- **Risk**: `src/__Libraries/StellaOps.Artifact.Core` lacks module-local AGENTS.md; TASK-015-010 is blocked until the charter is added. (Resolved 2026-01-20)
- **Risk**: Artifact.Core tests blocked by Artifact.Infrastructure compile errors (missing ArtifactType references) and NU1504 duplicate package warnings; requires upstream cleanup before full test pass.
- **Docs**: `docs/modules/concelier/sbom-learning-api.md` updated with ParsedSbom extraction coverage, including CycloneDX component enrichment, formulation, and SPDX build metadata.
## Next Checkpoints
- TASK-015-008 completion: CycloneDX 1.7 parser functional
- TASK-015-009 completion: SPDX 3.0.1 parser functional
- TASK-015-012 completion: Full test coverage
- TASK-015-013 completion: Integration verified

330
docs/implplan/SPRINT_20260119_016_Scanner_service_endpoint_security.md
View File

@@ -1,330 +0,0 @@
# Sprint 20260119_016 · Scanner Service Endpoint Security Analysis
## Topic & Scope
- Enable Scanner to analyze services declared in CycloneDX 1.7 SBOMs
- Detect security issues with service endpoints (authentication, trust boundaries, data flows)
- Correlate service dependencies with known API vulnerabilities
- Integrate with existing reachability analysis for service-to-service flows
- Working directory: `src/Scanner/`
- Secondary: `src/Concelier/__Libraries/StellaOps.Concelier.SbomIntegration/`
- Expected evidence: Unit tests, integration tests, security rule coverage
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedService model)
- Can run in parallel with other Scanner sprints after 015 delivers ParsedService
## Documentation Prerequisites
- CycloneDX services specification: https://cyclonedx.org/docs/1.7/#services
- Existing Scanner architecture: `docs/modules/scanner/architecture.md`
- ParsedService model from SPRINT_20260119_015
## Delivery Tracker
### TASK-016-001 - Design service security analysis pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `IServiceSecurityAnalyzer` interface:
```csharp
public interface IServiceSecurityAnalyzer
{
Task<ServiceSecurityReport> AnalyzeAsync(
IReadOnlyList<ParsedService> services,
ServiceSecurityPolicy policy,
CancellationToken ct);
}
```
- Design `ServiceSecurityReport`:
```csharp
public sealed record ServiceSecurityReport
{
public ImmutableArray<ServiceSecurityFinding> Findings { get; init; }
public ImmutableArray<ServiceDependencyChain> DependencyChains { get; init; }
public ServiceSecuritySummary Summary { get; init; }
}
public sealed record ServiceSecurityFinding
{
public required string ServiceBomRef { get; init; }
public required ServiceSecurityFindingType Type { get; init; }
public required Severity Severity { get; init; }
public required string Title { get; init; }
public required string Description { get; init; }
public string? Remediation { get; init; }
public string? CweId { get; init; }
}
```
- Define finding types:
- UnauthenticatedEndpoint
- CrossesTrustBoundaryWithoutAuth
- SensitiveDataExposed
- DeprecatedProtocol
- InsecureEndpointScheme
- MissingRateLimiting
- KnownVulnerableServiceVersion
- UnencryptedDataFlow
Completion criteria:
- [ ] Interface and models defined
- [ ] Finding types cover OWASP API Top 10
- [ ] Severity classification defined
### TASK-016-002 - Implement endpoint scheme analysis
Status: TODO
Dependency: TASK-016-001
Owners: Developer
Task description:
- Create `EndpointSchemeAnalyzer`:
- Parse service endpoints URIs
- Flag HTTP endpoints (should be HTTPS)
- Flag non-TLS protocols (ws:// should be wss://)
- Detect plaintext protocols (ftp://, telnet://, ldap://)
- Allow policy exceptions for internal services
- Create findings for insecure schemes with remediation guidance
Completion criteria:
- [ ] All common schemes analyzed
- [ ] Policy-based exceptions supported
- [ ] Localhost/internal exceptions configurable
### TASK-016-003 - Implement authentication analysis
Status: TODO
Dependency: TASK-016-001
Owners: Developer
Task description:
- Create `AuthenticationAnalyzer`:
- Check `authenticated` flag on services
- Flag services with `authenticated=false` that expose sensitive data
- Flag services crossing trust boundaries without authentication
- Analyze data flows for authentication requirements
- Map to CWE-306 (Missing Authentication for Critical Function)
- Integration with policy for authentication requirements by data classification
Completion criteria:
- [ ] Unauthenticated services flagged appropriately
- [ ] Trust boundary crossings detected
- [ ] Data classification influences severity
- [ ] CWE mapping implemented
### TASK-016-004 - Implement trust boundary analysis
Status: TODO
Dependency: TASK-016-003
Owners: Developer
Task description:
- Create `TrustBoundaryAnalyzer`:
- Parse `x-trust-boundary` property on services
- Build trust zone topology from nested services
- Detect cross-boundary calls without appropriate controls
- Flag external-facing services with internal dependencies
- Integrate with network policy if available
- Generate dependency chains showing trust boundary crossings
Completion criteria:
- [ ] Trust zones identified from SBOM
- [ ] Cross-boundary calls mapped
- [ ] External-to-internal paths flagged
- [ ] Dependency chains visualizable
### TASK-016-005 - Implement data flow analysis
Status: TODO
Dependency: TASK-016-004
Owners: Developer
Task description:
- Create `DataFlowAnalyzer`:
- Parse `data` array on services
- Map data classifications (PII, financial, health, etc.)
- Detect sensitive data flowing to less-trusted services
- Flag sensitive data on unauthenticated endpoints
- Correlate with GDPR/HIPAA data categories
- Create data flow graph for visualization
Completion criteria:
- [ ] Data flows extracted from services
- [ ] Classification-aware analysis
- [ ] Sensitive data exposure detected
- [ ] Flow graph generated
### TASK-016-006 - Implement service version vulnerability matching
Status: TODO
Dependency: TASK-016-001
Owners: Developer
Task description:
- Create `ServiceVulnerabilityMatcher`:
- Extract service name/version
- Query advisory database for known service vulnerabilities
- Match against CVEs for common services (nginx, apache, redis, postgres, etc.)
- Generate CPE for service identification
- Flag deprecated service versions
- Integration with existing advisory matching pipeline
Completion criteria:
- [ ] Service versions matched against CVE database
- [ ] Common services have CPE mappings
- [ ] Deprecated versions flagged
- [ ] Severity inherited from CVE
### TASK-016-007 - Implement nested service analysis
Status: TODO
Dependency: TASK-016-004
Owners: Developer
Task description:
- Create `NestedServiceAnalyzer`:
- Traverse nested services recursively
- Build service dependency graph
- Detect circular dependencies
- Identify shared services across components
- Flag orphaned services (declared but not referenced)
- Generate service topology for review
Completion criteria:
- [ ] Recursive traversal works
- [ ] Circular dependencies detected
- [ ] Shared services identified
- [ ] Topology exportable (DOT/JSON)
### TASK-016-008 - Create ServiceSecurityPolicy configuration
Status: TODO
Dependency: TASK-016-005
Owners: Developer
Task description:
- Define policy schema for service security:
```yaml
serviceSecurityPolicy:
requireAuthentication:
forTrustBoundaryCrossing: true
forSensitiveData: true
exceptions:
- servicePattern: "internal-*"
reason: "Internal services use mTLS"
allowedSchemes:
external: [https, wss]
internal: [https, http, grpc]
dataClassifications:
sensitive: [PII, financial, health, auth]
deprecatedServices:
- name: "redis"
beforeVersion: "6.0"
reason: "Security vulnerabilities in older versions"
```
- Integrate with existing Policy module
Completion criteria:
- [ ] Policy schema defined
- [ ] Policy loading from YAML/JSON
- [ ] Integration with Policy module
- [ ] Default policy provided
### TASK-016-009 - Integrate with Scanner main pipeline
Status: TODO
Dependency: TASK-016-008
Owners: Developer
Task description:
- Add service analysis to Scanner orchestration:
- Extract services from ParsedSbom
- Run ServiceSecurityAnalyzer
- Merge findings with component vulnerability findings
- Update scan report with service security section
- Add CLI option to include/exclude service analysis
- Add service findings to evidence for attestation
Completion criteria:
- [ ] Service analysis in main scan pipeline
- [ ] Findings merged with component findings
- [ ] CLI options implemented
- [ ] Evidence includes service findings
### TASK-016-010 - Create service security findings reporter
Status: TODO
Dependency: TASK-016-009
Owners: Developer
Task description:
- Add service security section to scan reports:
- Service inventory table
- Trust boundary diagram (ASCII or SVG)
- Data flow summary
- Findings grouped by service
- Remediation summary
- Support JSON, SARIF, and human-readable formats
Completion criteria:
- [ ] Report section implemented
- [ ] All formats supported
- [ ] Trust boundary visualization
- [ ] Actionable remediation guidance
### TASK-016-011 - Unit tests for service security analysis
Status: TODO
Dependency: TASK-016-009
Owners: QA
Task description:
- Test fixtures:
- Services with various authentication states
- Nested service hierarchies
- Trust boundary configurations
- Data flow scenarios
- Vulnerable service versions
- Test each analyzer in isolation
- Test policy application
- Test report generation
Completion criteria:
- [ ] >90% code coverage
- [ ] All finding types tested
- [ ] Policy exceptions tested
- [ ] Edge cases covered
### TASK-016-012 - Integration tests with real SBOMs
Status: TODO
Dependency: TASK-016-011
Owners: QA
Task description:
- Test with real-world SBOMs containing services:
- Microservices architecture SBOM
- API gateway with backends
- Event-driven architecture
- Verify findings accuracy
- Performance testing with large service graphs
Completion criteria:
- [ ] Real SBOM integration verified
- [ ] No false positives on legitimate patterns
- [ ] Performance acceptable (<5s for 100 services)
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for service security scanning | Planning |
## Decisions & Risks
- **Decision**: Focus on CycloneDX services first; SPDX doesn't have equivalent concept
- **Decision**: Use CWE mappings for standardized finding classification
- **Risk**: Service names may not have CVE mappings; mitigation is CPE generation heuristics
- **Risk**: Trust boundary information may be incomplete; mitigation is conservative analysis
- **Decision**: Service analysis is opt-in initially to avoid breaking existing workflows
## Next Checkpoints
- TASK-016-006 completion: Vulnerability matching functional
- TASK-016-009 completion: Integration complete
- TASK-016-012 completion: Real-world validation

379
docs/implplan/SPRINT_20260119_017_Scanner_cbom_crypto_analysis.md
View File

@@ -1,379 +0,0 @@
# Sprint 20260119_017 · Scanner CBOM Cryptographic Analysis
## Topic & Scope
- Enable Scanner to analyze cryptographic assets declared in CycloneDX 1.5+ cryptoProperties (CBOM)
- Detect weak, deprecated, or non-compliant cryptographic algorithms
- Enforce crypto policies (FIPS 140-2/3, PCI-DSS, NIST post-quantum, regional requirements)
- Inventory all cryptographic assets for compliance reporting
- Working directory: `src/Scanner/`
- Secondary: `src/Cryptography/`
- Expected evidence: Unit tests, compliance matrix, policy templates
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedCryptoProperties model)
- Can run in parallel with other Scanner sprints after 015 delivers crypto models
## Documentation Prerequisites
- CycloneDX CBOM specification: https://cyclonedx.org/capabilities/cbom/
- NIST cryptographic standards: SP 800-131A Rev 2
- FIPS 140-3 approved algorithms
- Existing Cryptography module: `src/Cryptography/`
## Delivery Tracker
### TASK-017-001 - Design cryptographic analysis pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `ICryptoAnalyzer` interface:
```csharp
public interface ICryptoAnalyzer
{
Task<CryptoAnalysisReport> AnalyzeAsync(
IReadOnlyList<ParsedComponent> componentsWithCrypto,
CryptoPolicy policy,
CancellationToken ct);
}
```
- Design `CryptoAnalysisReport`:
```csharp
public sealed record CryptoAnalysisReport
{
public CryptoInventory Inventory { get; init; }
public ImmutableArray<CryptoFinding> Findings { get; init; }
public CryptoComplianceStatus ComplianceStatus { get; init; }
public PostQuantumReadiness QuantumReadiness { get; init; }
}
public sealed record CryptoInventory
{
public ImmutableArray<CryptoAlgorithmUsage> Algorithms { get; init; }
public ImmutableArray<CryptoCertificateUsage> Certificates { get; init; }
public ImmutableArray<CryptoProtocolUsage> Protocols { get; init; }
public ImmutableArray<CryptoKeyMaterial> KeyMaterials { get; init; }
}
```
- Define finding types:
- WeakAlgorithm (MD5, SHA1, DES, 3DES, RC4)
- ShortKeyLength (RSA < 2048, ECC < 256)
- DeprecatedProtocol (TLS 1.0, TLS 1.1, SSLv3)
- NonFipsCompliant
- QuantumVulnerable
- ExpiredCertificate
- WeakCipherSuite
- InsecureMode (ECB, no padding)
- MissingIntegrity (encryption without MAC)
Completion criteria:
- [ ] Interface and models defined
- [ ] Finding types cover major crypto weaknesses
- [ ] Inventory model comprehensive
### TASK-017-002 - Implement algorithm strength analyzer
Status: TODO
Dependency: TASK-017-001
Owners: Developer
Task description:
- Create `AlgorithmStrengthAnalyzer`:
- Evaluate symmetric algorithms (AES, ChaCha20, 3DES, DES, RC4, Blowfish)
- Evaluate asymmetric algorithms (RSA, DSA, ECDSA, EdDSA, DH, ECDH)
- Evaluate hash algorithms (SHA-2, SHA-3, SHA-1, MD5, BLAKE2)
- Check key lengths against policy minimums
- Flag deprecated algorithms
- Build algorithm strength database:
```csharp
public enum AlgorithmStrength { Broken, Weak, Legacy, Acceptable, Strong, PostQuantum }
```
- Map NIST security levels (classical and quantum)
Completion criteria:
- [ ] All common algorithms classified
- [ ] Key length validation implemented
- [ ] NIST security levels mapped
- [ ] Deprecation dates tracked
### TASK-017-003 - Implement FIPS 140 compliance checker
Status: TODO
Dependency: TASK-017-002
Owners: Developer
Task description:
- Create `FipsComplianceChecker`:
- Validate algorithms against FIPS 140-2/140-3 approved list
- Check algorithm modes (CTR, GCM, CBC with proper padding)
- Validate key derivation functions (PBKDF2, HKDF)
- Check random number generation references
- Flag non-FIPS algorithms in FIPS-required context
- Support FIPS 140-2 and 140-3 profiles
- Generate FIPS compliance attestation
Completion criteria:
- [ ] FIPS 140-2 algorithm list complete
- [ ] FIPS 140-3 algorithm list complete
- [ ] Mode validation implemented
- [ ] Compliance attestation generated
### TASK-017-004 - Implement post-quantum readiness analyzer
Status: TODO
Dependency: TASK-017-002
Owners: Developer
Task description:
- Create `PostQuantumAnalyzer`:
- Identify quantum-vulnerable algorithms (RSA, ECC, DH, DSA)
- Identify quantum-resistant algorithms (Kyber, Dilithium, SPHINCS+, Falcon)
- Calculate quantum readiness score
- Generate migration recommendations
- Track hybrid approaches (classical + PQC)
- Map NIST PQC standardization status
- Flag harvest-now-decrypt-later risks for long-lived data
Completion criteria:
- [ ] Quantum-vulnerable algorithms identified
- [ ] NIST PQC finalists recognized
- [ ] Readiness score calculated
- [ ] Migration path suggested
### TASK-017-005 - Implement certificate analysis
Status: TODO
Dependency: TASK-017-001
Owners: Developer
Task description:
- Create `CertificateAnalyzer`:
- Parse certificate properties from CBOM
- Check validity period (notValidBefore, notValidAfter)
- Flag expiring certificates (configurable threshold)
- Check signature algorithm strength
- Validate key usage constraints
- Check certificate chain completeness
- Integration with existing Cryptography module certificate handling
Completion criteria:
- [ ] Certificate properties analyzed
- [ ] Expiration warnings generated
- [ ] Signature algorithm validated
- [ ] Chain analysis implemented
### TASK-017-006 - Implement protocol cipher suite analysis
Status: TODO
Dependency: TASK-017-002
Owners: Developer
Task description:
- Create `ProtocolAnalyzer`:
- Parse protocol properties (TLS, SSH, IPSec)
- Evaluate cipher suite strength
- Flag deprecated protocol versions
- Check for weak cipher suites (NULL, EXPORT, RC4, DES)
- Validate key exchange algorithms
- Check for perfect forward secrecy support
- Build cipher suite database with strength ratings
Completion criteria:
- [ ] TLS cipher suites analyzed
- [ ] SSH cipher suites analyzed
- [ ] IKEv2 transforms analyzed
- [ ] PFS requirement enforced
### TASK-017-007 - Create CryptoPolicy configuration
Status: TODO
Dependency: TASK-017-004
Owners: Developer
Task description:
- Define policy schema for crypto requirements:
```yaml
cryptoPolicy:
complianceFramework: FIPS-140-3 # or PCI-DSS, NIST-800-131A, custom
minimumKeyLengths:
RSA: 2048
ECDSA: 256
AES: 128
prohibitedAlgorithms:
- MD5
- SHA1
- DES
- 3DES
- RC4
requiredFeatures:
perfectForwardSecrecy: true
authenticatedEncryption: true
postQuantum:
requireHybridForLongLived: true
longLivedDataThresholdYears: 10
certificates:
expirationWarningDays: 90
minimumSignatureAlgorithm: SHA256
exemptions:
- componentPattern: "legacy-*"
algorithms: [3DES]
reason: "Legacy system migration in progress"
expirationDate: "2027-01-01"
```
- Support multiple compliance frameworks
- Allow per-component exemptions with expiration
Completion criteria:
- [ ] Policy schema defined
- [ ] Multiple frameworks supported
- [ ] Exemptions with expiration
- [ ] Default policies for common frameworks
### TASK-017-008 - Implement crypto inventory generator
Status: TODO
Dependency: TASK-017-006
Owners: Developer
Task description:
- Create `CryptoInventoryGenerator`:
- Aggregate all crypto assets from SBOM
- Group by type (symmetric, asymmetric, hash, protocol)
- Count usage by algorithm
- Track component associations
- Generate inventory report
- Support export formats: JSON, CSV, XLSX
Completion criteria:
- [ ] Complete inventory generated
- [ ] Usage statistics calculated
- [ ] Component associations tracked
- [ ] Multiple export formats
### TASK-017-009 - Integrate with Scanner main pipeline
Status: TODO
Dependency: TASK-017-008
Owners: Developer
Task description:
- Add crypto analysis to Scanner orchestration:
- Extract components with cryptoProperties
- Run CryptoAnalyzer
- Merge findings with other findings
- Add crypto section to scan report
- Generate compliance attestation
- Add CLI options for crypto analysis:
- `--crypto-policy <path>`
- `--fips-mode`
- `--pqc-analysis`
- Add crypto inventory to evidence for attestation
Completion criteria:
- [ ] Crypto analysis in main pipeline
- [ ] CLI options implemented
- [ ] Compliance attestation generated
- [ ] Evidence includes crypto inventory
### TASK-017-010 - Create crypto findings reporter
Status: TODO
Dependency: TASK-017-009
Owners: Developer
Task description:
- Add crypto section to scan reports:
- Algorithm inventory table
- Quantum readiness summary
- Compliance status by framework
- Findings with remediation
- Certificate expiration timeline
- Migration recommendations for weak crypto
- Support JSON, SARIF, PDF formats
Completion criteria:
- [ ] Report section implemented
- [ ] All formats supported
- [ ] Remediation guidance included
- [ ] Visual summaries (compliance gauges)
### TASK-017-011 - Integration with eIDAS/regional crypto
Status: TODO
Dependency: TASK-017-007
Owners: Developer
Task description:
- Extend policy support for regional requirements:
- eIDAS qualified algorithms (EU)
- GOST algorithms (Russia)
- SM algorithms (China: SM2, SM3, SM4)
- Map regional algorithm identifiers to OIDs
- Integration with existing `StellaOps.Cryptography.Plugin.Eidas`
Completion criteria:
- [ ] eIDAS algorithms recognized
- [ ] GOST algorithms recognized
- [ ] SM algorithms recognized
- [ ] OID mapping complete
### TASK-017-012 - Unit tests for crypto analysis
Status: TODO
Dependency: TASK-017-009
Owners: QA
Task description:
- Test fixtures:
- Components with various crypto properties
- Weak algorithm scenarios
- Certificate expiration scenarios
- Protocol configurations
- Post-quantum algorithms
- Test each analyzer in isolation
- Test policy application with exemptions
- Test compliance frameworks
Completion criteria:
- [ ] >90% code coverage
- [ ] All finding types tested
- [ ] Policy exemptions tested
- [ ] Regional algorithms tested
### TASK-017-013 - Integration tests with CBOM samples
Status: TODO
Dependency: TASK-017-012
Owners: QA
Task description:
- Test with real CBOM samples:
- OpenSSL component CBOM
- Java cryptography CBOM
- .NET cryptography CBOM
- Verify finding accuracy
- Validate compliance reports against manual review
Completion criteria:
- [ ] Real CBOM samples tested
- [ ] No false positives on compliant crypto
- [ ] All weak crypto detected
- [ ] Reports match manual analysis
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for CBOM crypto analysis | Planning |
## Decisions & Risks
- **Decision**: Support multiple compliance frameworks (FIPS, PCI-DSS, NIST, regional)
- **Decision**: Post-quantum analysis is opt-in until PQC adoption increases
- **Risk**: Algorithm strength classifications change over time; mitigation is configurable database
- **Risk**: Certificate chain analysis requires external validation; mitigation is flag incomplete chains
- **Decision**: Exemptions require expiration dates to prevent permanent exceptions
## Next Checkpoints
- TASK-017-003 completion: FIPS compliance functional
- TASK-017-004 completion: PQC analysis functional
- TASK-017-009 completion: Integration complete
- TASK-017-013 completion: Real-world validation

392
docs/implplan/SPRINT_20260119_018_Scanner_aiml_supply_chain.md
View File

@@ -1,392 +0,0 @@
# Sprint 20260119_018 · Scanner AI/ML Supply Chain Security
## Topic & Scope
- Enable Scanner to analyze AI/ML components declared in CycloneDX 1.6+ modelCard and SPDX 3.0.1 AI profile
- Detect security and safety risks in ML model provenance and training data
- Enforce AI governance policies (model cards, bias assessment, data lineage)
- Inventory ML models for regulatory compliance (EU AI Act, NIST AI RMF)
- Working directory: `src/Scanner/`
- Secondary: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.ML/`
- Expected evidence: Unit tests, AI governance compliance checks, risk assessment templates
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedModelCard model)
- Can run in parallel with other Scanner sprints after 015 delivers modelCard models
## Documentation Prerequisites
- CycloneDX ML-BOM specification: https://cyclonedx.org/capabilities/mlbom/
- SPDX AI profile: https://spdx.github.io/spdx-spec/v3.0.1/model/AI/
- EU AI Act requirements
- NIST AI Risk Management Framework
- Existing ML module: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.ML/`
## Delivery Tracker
### TASK-018-001 - Design AI/ML security analysis pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `IAiMlSecurityAnalyzer` interface:
```csharp
public interface IAiMlSecurityAnalyzer
{
Task<AiMlSecurityReport> AnalyzeAsync(
IReadOnlyList<ParsedComponent> mlComponents,
AiGovernancePolicy policy,
CancellationToken ct);
}
```
- Design `AiMlSecurityReport`:
```csharp
public sealed record AiMlSecurityReport
{
public AiModelInventory Inventory { get; init; }
public ImmutableArray<AiSecurityFinding> Findings { get; init; }
public ImmutableArray<AiRiskAssessment> RiskAssessments { get; init; }
public AiComplianceStatus ComplianceStatus { get; init; }
}
public sealed record AiModelInventory
{
public ImmutableArray<AiModelEntry> Models { get; init; }
public ImmutableArray<DatasetEntry> TrainingDatasets { get; init; }
public ImmutableArray<AiModelDependency> ModelDependencies { get; init; }
}
```
- Define finding types:
- MissingModelCard
- IncompleteModelCard
- UnknownTrainingData
- BiasAssessmentMissing
- SafetyAssessmentMissing
- UnverifiedModelProvenance
- SensitiveDataInTraining
- HighRiskAiCategory (EU AI Act)
- MissingPerformanceMetrics
- ModelDriftRisk
- AdversarialVulnerability
Completion criteria:
- [ ] Interface and models defined
- [ ] Finding types cover AI security concerns
- [ ] Risk categories mapped to regulations
### TASK-018-002 - Implement model card completeness analyzer
Status: TODO
Dependency: TASK-018-001
Owners: Developer
Task description:
- Create `ModelCardCompletenessAnalyzer`:
- Check required modelCard fields per ML-BOM spec
- Validate model parameters (architecture, inputs, outputs)
- Check for performance metrics
- Validate quantitative analysis section
- Check considerations section completeness
- Define completeness scoring:
- Minimal: name, version, type
- Basic: + architecture, inputs, outputs
- Standard: + metrics, datasets
- Complete: + considerations, limitations, ethical review
- Flag incomplete model cards by required level
Completion criteria:
- [ ] Completeness scoring implemented
- [ ] Required field validation
- [ ] Scoring thresholds configurable
### TASK-018-003 - Implement training data provenance analyzer
Status: TODO
Dependency: TASK-018-001
Owners: Developer
Task description:
- Create `TrainingDataProvenanceAnalyzer`:
- Extract dataset references from modelCard
- Validate dataset provenance (source, collection process)
- Check for sensitive data indicators (PII, health, financial)
- Detect missing data lineage
- Flag synthetic vs real data
- For SPDX Dataset profile:
- Parse datasetType, dataCollectionProcess
- Check confidentialityLevel
- Validate intendedUse
- Extract knownBias information
- Cross-reference with known problematic datasets
Completion criteria:
- [ ] Dataset references extracted
- [ ] Provenance validation implemented
- [ ] Sensitive data detection
- [ ] Known dataset database
### TASK-018-004 - Implement bias and fairness analyzer
Status: TODO
Dependency: TASK-018-002
Owners: Developer
Task description:
- Create `BiasFairnessAnalyzer`:
- Check for fairness assessment in considerations
- Validate demographic testing documentation
- Check for bias metrics in quantitative analysis
- Flag models without fairness evaluation
- Identify protected attribute handling
- Support bias categories:
- Selection bias (training data)
- Measurement bias (feature encoding)
- Algorithmic bias (model behavior)
- Deployment bias (use context)
- Map to EU AI Act fairness requirements
Completion criteria:
- [ ] Fairness documentation validated
- [ ] Bias categories identified
- [ ] Protected attributes tracked
- [ ] EU AI Act alignment
### TASK-018-005 - Implement safety risk analyzer
Status: TODO
Dependency: TASK-018-001
Owners: Developer
Task description:
- Create `AiSafetyRiskAnalyzer`:
- Extract safetyRiskAssessment from SPDX AI profile
- Evaluate autonomy level implications
- Check for human oversight requirements
- Validate safety testing documentation
- Assess model failure modes
- Implement risk categorization (EU AI Act):
- Unacceptable risk
- High risk
- Limited risk
- Minimal risk
- Flag missing safety assessments for high-risk categories
Completion criteria:
- [ ] Safety assessments extracted
- [ ] Risk categorization implemented
- [ ] EU AI Act categories mapped
- [ ] Failure mode analysis
### TASK-018-006 - Implement model provenance verifier
Status: TODO
Dependency: TASK-018-003
Owners: Developer
Task description:
- Create `ModelProvenanceVerifier`:
- Check model hash/signature if available
- Validate model source references
- Check for known model hubs (Hugging Face, Model Zoo)
- Detect modified/fine-tuned models
- Track base model lineage
- Integration with existing Signer module for signature verification
- Cross-reference with model vulnerability databases (if available)
Completion criteria:
- [ ] Provenance chain verified
- [ ] Model hub recognition
- [ ] Fine-tuning lineage tracked
- [ ] Signature verification integrated
### TASK-018-007 - Create AiGovernancePolicy configuration
Status: TODO
Dependency: TASK-018-005
Owners: Developer
Task description:
- Define policy schema for AI governance:
```yaml
aiGovernancePolicy:
complianceFramework: EU-AI-Act # or NIST-AI-RMF, internal
modelCardRequirements:
minimumCompleteness: standard # minimal, basic, standard, complete
requiredSections:
- modelParameters
- quantitativeAnalysis
- considerations.ethicalConsiderations
trainingDataRequirements:
requireProvenance: true
sensitiveDataAllowed: false
requireBiasAssessment: true
riskCategories:
highRisk:
- biometricIdentification
- criticalInfrastructure
- employmentDecisions
- creditScoring
- lawEnforcement
safetyRequirements:
requireSafetyAssessment: true
humanOversightRequired:
forHighRisk: true
exemptions:
- modelPattern: "research-*"
reason: "Research models in sandbox"
riskAccepted: true
```
- Support EU AI Act and NIST AI RMF frameworks
- Allow risk acceptance documentation
Completion criteria:
- [ ] Policy schema defined
- [ ] Multiple frameworks supported
- [ ] Risk acceptance workflow
- [ ] Default policies provided
### TASK-018-008 - Implement AI model inventory generator
Status: TODO
Dependency: TASK-018-006
Owners: Developer
Task description:
- Create `AiModelInventoryGenerator`:
- Aggregate all ML components from SBOM
- Track model types (classification, generation, embedding, etc.)
- Map model-to-dataset relationships
- Track model versions and lineage
- Generate inventory report
- Support export formats: JSON, CSV, regulatory submission format
Completion criteria:
- [ ] Complete model inventory
- [ ] Dataset relationships mapped
- [ ] Lineage tracked
- [ ] Regulatory export formats
### TASK-018-009 - Integrate with Scanner main pipeline
Status: TODO
Dependency: TASK-018-008
Owners: Developer
Task description:
- Add AI/ML analysis to Scanner orchestration:
- Identify components with type=MachineLearningModel or modelCard
- Run AiMlSecurityAnalyzer
- Merge findings with other findings
- Add AI governance section to scan report
- Generate compliance attestation
- Add CLI options:
- `--ai-governance-policy <path>`
- `--ai-risk-assessment`
- `--skip-ai-analysis`
- Add AI findings to evidence for attestation
Completion criteria:
- [ ] AI analysis in main pipeline
- [ ] CLI options implemented
- [ ] Compliance attestation generated
- [ ] Evidence includes AI inventory
### TASK-018-010 - Create AI governance reporter
Status: TODO
Dependency: TASK-018-009
Owners: Developer
Task description:
- Add AI governance section to scan reports:
- Model inventory table
- Risk categorization summary
- Model card completeness dashboard
- Training data lineage
- Findings with remediation
- Compliance status by regulation
- Support JSON, PDF, regulatory submission formats
Completion criteria:
- [ ] Report section implemented
- [ ] Risk visualization
- [ ] Regulatory format export
- [ ] Remediation guidance
### TASK-018-011 - Integration with BinaryIndex ML module
Status: TODO
Dependency: TASK-018-006
Owners: Developer
Task description:
- Connect AI/ML analysis to existing BinaryIndex ML capabilities:
- Use function embedding service for model analysis
- Leverage ground truth corpus for model validation
- Cross-reference with ML training infrastructure
- Enable model binary analysis when ONNX/TensorFlow files available
Completion criteria:
- [ ] BinaryIndex ML integration
- [ ] Model binary analysis where possible
- [ ] Ground truth validation
### TASK-018-012 - Unit tests for AI/ML security analysis
Status: TODO
Dependency: TASK-018-009
Owners: QA
Task description:
- Test fixtures:
- Complete modelCard examples
- Incomplete model cards (various missing sections)
- SPDX AI profile examples
- High-risk AI use cases
- Training dataset references
- Test each analyzer in isolation
- Test policy application
- Test regulatory compliance checks
Completion criteria:
- [ ] >90% code coverage
- [ ] All finding types tested
- [ ] Policy exemptions tested
- [ ] Regulatory frameworks tested
### TASK-018-013 - Integration tests with real ML SBOMs
Status: TODO
Dependency: TASK-018-012
Owners: QA
Task description:
- Test with real-world ML SBOMs:
- Hugging Face model SBOM
- TensorFlow model SBOM
- PyTorch model SBOM
- Multi-model pipeline SBOM
- Verify findings accuracy
- Validate regulatory compliance reports
Completion criteria:
- [ ] Real ML SBOMs tested
- [ ] Accurate risk categorization
- [ ] No false positives on compliant models
- [ ] Reports suitable for regulatory submission
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for AI/ML supply chain security | Planning |
## Decisions & Risks
- **Decision**: Support both CycloneDX modelCard and SPDX AI profile
- **Decision**: EU AI Act alignment as primary compliance framework
- **Risk**: AI regulations evolving rapidly; mitigation is modular policy system
- **Risk**: Training data assessment may be incomplete; mitigation is flag unknown provenance
- **Decision**: Research/sandbox models can have risk acceptance exemptions
## Next Checkpoints
- TASK-018-004 completion: Bias analysis functional
- TASK-018-005 completion: Safety assessment functional
- TASK-018-009 completion: Integration complete
- TASK-018-013 completion: Real-world validation

397
docs/implplan/SPRINT_20260119_019_Scanner_build_provenance.md
View File

@@ -1,397 +0,0 @@
# Sprint 20260119_019 · Scanner Build Provenance Verification
## Topic & Scope
- Enable Scanner to verify build provenance from CycloneDX formulation and SPDX Build profile
- Validate build reproducibility claims against actual artifacts
- Enforce build security policies (hermetic builds, signed sources, verified builders)
- Integration with SLSA framework for provenance verification
- Working directory: `src/Scanner/`
- Secondary: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GroundTruth.Reproducible/`
- Expected evidence: Unit tests, SLSA compliance checks, provenance verification reports
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedFormulation, ParsedBuildInfo)
- Can run in parallel with other Scanner sprints after 015 delivers build models
- Integration with existing reproducible build infrastructure
## Documentation Prerequisites
- CycloneDX formulation specification: https://cyclonedx.org/docs/1.7/#formulation
- SPDX Build profile: https://spdx.github.io/spdx-spec/v3.0.1/model/Build/
- SLSA specification: https://slsa.dev/spec/v1.0/
- Existing reproducible build module: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GroundTruth.Reproducible/`
- In-toto attestation format
## Delivery Tracker
### TASK-019-001 - Design build provenance verification pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `IBuildProvenanceVerifier` interface:
```csharp
public interface IBuildProvenanceVerifier
{
Task<BuildProvenanceReport> VerifyAsync(
ParsedSbom sbom,
BuildProvenancePolicy policy,
CancellationToken ct);
}
```
- Design `BuildProvenanceReport`:
```csharp
public sealed record BuildProvenanceReport
{
public SlsaLevel AchievedLevel { get; init; }
public ImmutableArray<ProvenanceFinding> Findings { get; init; }
public BuildProvenanceChain ProvenanceChain { get; init; }
public ReproducibilityStatus ReproducibilityStatus { get; init; }
}
public sealed record BuildProvenanceChain
{
public string? BuilderId { get; init; }
public string? SourceRepository { get; init; }
public string? SourceCommit { get; init; }
public string? BuildConfigUri { get; init; }
public string? BuildConfigDigest { get; init; }
public ImmutableDictionary<string, string> Environment { get; init; }
public ImmutableArray<BuildInput> Inputs { get; init; }
public ImmutableArray<BuildOutput> Outputs { get; init; }
}
```
- Define finding types:
- MissingBuildProvenance
- UnverifiedBuilder
- UnsignedSource
- NonHermeticBuild
- MissingBuildConfig
- EnvironmentVariableLeak
- NonReproducibleBuild
- SlsaLevelInsufficient
- InputIntegrityFailed
- OutputMismatch
Completion criteria:
- [ ] Interface and models defined
- [ ] SLSA levels mapped
- [ ] Finding types cover provenance concerns
### TASK-019-002 - Implement SLSA level evaluator
Status: TODO
Dependency: TASK-019-001
Owners: Developer
Task description:
- Create `SlsaLevelEvaluator`:
- Evaluate SLSA Level 1: Provenance exists
- Build process documented
- Provenance generated
- Evaluate SLSA Level 2: Hosted build platform
- Provenance signed
- Build service used
- Evaluate SLSA Level 3: Hardened builds
- Hermetic build
- Isolated build
- Non-falsifiable provenance
- Evaluate SLSA Level 4 (future): Reproducible
- Two-party review
- Reproducible builds
- Map SBOM build metadata to SLSA requirements
- Generate SLSA compliance report
Completion criteria:
- [ ] All SLSA levels evaluated
- [ ] Clear level determination
- [ ] Gap analysis for level improvement
### TASK-019-003 - Implement build config verification
Status: TODO
Dependency: TASK-019-001
Owners: Developer
Task description:
- Create `BuildConfigVerifier`:
- Extract build config from formulation/buildInfo
- Verify config source URI accessibility
- Validate config digest matches content
- Parse common build configs (Dockerfile, GitHub Actions, GitLab CI)
- Detect environment variable injection
- Flag dynamic/unverified dependencies
- Support config sources: git, https, file
Completion criteria:
- [ ] Config extraction implemented
- [ ] Digest verification working
- [ ] Common build systems recognized
- [ ] Dynamic dependency detection
### TASK-019-004 - Implement source verification
Status: TODO
Dependency: TASK-019-003
Owners: Developer
Task description:
- Create `SourceVerifier`:
- Extract source references from provenance
- Verify source commit signatures (GPG/SSH)
- Validate source repository integrity
- Check for tag vs branch vs commit references
- Detect source substitution attacks
- Integration with git signature verification
- Support multiple VCS (git, hg, svn)
Completion criteria:
- [ ] Source references extracted
- [ ] Commit signature verification
- [ ] Tag/branch validation
- [ ] Substitution attack detection
### TASK-019-005 - Implement builder verification
Status: TODO
Dependency: TASK-019-002
Owners: Developer
Task description:
- Create `BuilderVerifier`:
- Extract builder identity from provenance
- Validate builder against trusted builder registry
- Verify builder attestation signatures
- Check builder version/configuration
- Flag unrecognized builders
- Maintain trusted builder registry:
- GitHub Actions
- GitLab CI
- Google Cloud Build
- AWS CodeBuild
- Jenkins (verified instances)
- Local builds (with attestation)
Completion criteria:
- [ ] Builder identity extracted
- [ ] Trusted registry implemented
- [ ] Attestation verification
- [ ] Unknown builder flagging
### TASK-019-006 - Implement input integrity checker
Status: TODO
Dependency: TASK-019-003
Owners: Developer
Task description:
- Create `BuildInputIntegrityChecker`:
- Extract all build inputs from formulation
- Verify input digests against declarations
- Check for phantom dependencies (undeclared inputs)
- Validate input sources
- Detect build-time network access
- Cross-reference with SBOM components
Completion criteria:
- [ ] All inputs identified
- [ ] Digest verification
- [ ] Phantom dependency detection
- [ ] Network access flagging
### TASK-019-007 - Implement reproducibility verifier
Status: TODO
Dependency: TASK-019-006
Owners: Developer
Task description:
- Create `ReproducibilityVerifier`:
- Extract reproducibility claims from SBOM
- If verification requested, trigger rebuild
- Compare output digests
- Analyze differences for non-reproducible builds
- Generate diffoscope-style reports
- Integration with existing RebuildService:
- `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GroundTruth.Reproducible/RebuildService.cs`
- Support rebuild backends: local, container, remote
Completion criteria:
- [ ] Reproducibility claims extracted
- [ ] Rebuild integration working
- [ ] Diff analysis for failures
- [ ] Multiple backends supported
### TASK-019-008 - Create BuildProvenancePolicy configuration
Status: TODO
Dependency: TASK-019-005
Owners: Developer
Task description:
- Define policy schema for build provenance:
```yaml
buildProvenancePolicy:
minimumSlsaLevel: 2
trustedBuilders:
- id: "https://github.com/actions/runner"
name: "GitHub Actions"
minVersion: "2.300"
- id: "https://gitlab.com/gitlab-org/gitlab-runner"
name: "GitLab Runner"
minVersion: "15.0"
sourceRequirements:
requireSignedCommits: true
requireTaggedRelease: false
allowedRepositories:
- "github.com/myorg/*"
- "gitlab.com/myorg/*"
buildRequirements:
requireHermeticBuild: true
requireConfigDigest: true
maxEnvironmentVariables: 50
prohibitedEnvVarPatterns:
- "*_KEY"
- "*_SECRET"
- "*_TOKEN"
reproducibility:
requireReproducible: false
verifyOnDemand: true
exemptions:
- componentPattern: "vendor/*"
reason: "Third-party vendored code"
slsaLevelOverride: 1
```
Completion criteria:
- [ ] Policy schema defined
- [ ] SLSA level enforcement
- [ ] Trusted builder registry
- [ ] Source restrictions
### TASK-019-009 - Integrate with Scanner main pipeline
Status: TODO
Dependency: TASK-019-008
Owners: Developer
Task description:
- Add build provenance verification to Scanner:
- Extract formulation/buildInfo from ParsedSbom
- Run BuildProvenanceVerifier
- Evaluate SLSA level
- Merge findings with other findings
- Add provenance section to scan report
- Add CLI options:
- `--verify-provenance`
- `--slsa-policy <path>`
- `--verify-reproducibility` (triggers rebuild)
- Generate SLSA attestation
Completion criteria:
- [ ] Provenance verification in pipeline
- [ ] CLI options implemented
- [ ] SLSA attestation generated
- [ ] Evidence includes provenance chain
### TASK-019-010 - Create provenance report generator
Status: TODO
Dependency: TASK-019-009
Owners: Developer
Task description:
- Add provenance section to scan reports:
- Build provenance chain visualization
- SLSA level badge/indicator
- Source-to-binary mapping
- Builder trust status
- Findings with remediation
- Reproducibility status
- Support JSON, SARIF, in-toto predicate formats
Completion criteria:
- [ ] Report section implemented
- [ ] Provenance visualization
- [ ] In-toto format export
- [ ] Remediation guidance
### TASK-019-011 - Integration with existing reproducible build infrastructure
Status: TODO
Dependency: TASK-019-007
Owners: Developer
Task description:
- Connect provenance verification to existing infrastructure:
- `RebuildService` for reproduction
- `DeterminismValidator` for output comparison
- `SymbolExtractor` for binary analysis
- `ReproduceDebianClient` for Debian packages
- Enable automated reproducibility verification
Completion criteria:
- [ ] Full integration with existing infrastructure
- [ ] Automated verification pipeline
- [ ] Cross-platform support
### TASK-019-012 - Unit tests for build provenance verification
Status: TODO
Dependency: TASK-019-009
Owners: QA
Task description:
- Test fixtures:
- CycloneDX formulation examples
- SPDX Build profile examples
- Various SLSA levels
- Signed and unsigned sources
- Hermetic and non-hermetic builds
- Test each verifier in isolation
- Test policy application
- Test SLSA level evaluation
Completion criteria:
- [ ] >90% code coverage
- [ ] All finding types tested
- [ ] SLSA levels correctly evaluated
- [ ] Policy exemptions tested
### TASK-019-013 - Integration tests with real provenance
Status: TODO
Dependency: TASK-019-012
Owners: QA
Task description:
- Test with real build provenance:
- GitHub Actions provenance
- GitLab CI provenance
- SLSA provenance examples
- Sigstore attestations
- Verify finding accuracy
- Validate SLSA compliance reports
Completion criteria:
- [ ] Real provenance tested
- [ ] Accurate SLSA level determination
- [ ] No false positives on compliant builds
- [ ] Integration with sigstore working
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for build provenance verification | Planning |
## Decisions & Risks
- **Decision**: SLSA as primary provenance framework
- **Decision**: Reproducibility verification is opt-in (requires rebuild)
- **Risk**: Not all build systems provide adequate provenance; mitigation is graceful degradation
- **Risk**: Reproducibility verification is slow; mitigation is async/background processing
- **Decision**: Trusted builder registry is configurable per organization
## Next Checkpoints
- TASK-019-002 completion: SLSA evaluation functional
- TASK-019-007 completion: Reproducibility verification functional
- TASK-019-009 completion: Integration complete
- TASK-019-013 completion: Real-world validation

387
docs/implplan/SPRINT_20260119_020_Concelier_vex_consumption.md
View File

@@ -1,387 +0,0 @@
# Sprint 20260119_020 · Concelier VEX Consumption from SBOMs
## Topic & Scope
- Enable Concelier to consume VEX (Vulnerability Exploitability eXchange) data embedded in SBOMs
- Process CycloneDX vulnerabilities[] section with analysis/state
- Process SPDX 3.0.1 Security profile VEX assessment relationships
- Merge external VEX with SBOM-embedded VEX for unified vulnerability status
- Update advisory matching to respect VEX claims from producers
- Working directory: `src/Concelier/__Libraries/StellaOps.Concelier.SbomIntegration/`
- Secondary: `src/Excititor/`
- Expected evidence: Unit tests, VEX consumption integration tests, conflict resolution tests
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedVulnerability model)
- Can run in parallel with other sprints after 015 delivers vulnerability models
## Documentation Prerequisites
- CycloneDX VEX specification: https://cyclonedx.org/capabilities/vex/
- SPDX Security profile: https://spdx.github.io/spdx-spec/v3.0.1/model/Security/
- CISA VEX guidance
- Existing VEX generation: `src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/`
## Delivery Tracker
### TASK-020-001 - Design VEX consumption pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `IVexConsumer` interface:
```csharp
public interface IVexConsumer
{
Task<VexConsumptionResult> ConsumeAsync(
IReadOnlyList<ParsedVulnerability> sbomVulnerabilities,
VexConsumptionPolicy policy,
CancellationToken ct);
Task<MergedVulnerabilityStatus> MergeWithExternalVexAsync(
IReadOnlyList<ParsedVulnerability> sbomVex,
IReadOnlyList<VexStatement> externalVex,
VexMergePolicy mergePolicy,
CancellationToken ct);
}
```
- Design `VexConsumptionResult`:
```csharp
public sealed record VexConsumptionResult
{
public ImmutableArray<ConsumedVexStatement> Statements { get; init; }
public ImmutableArray<VexConsumptionWarning> Warnings { get; init; }
public VexTrustLevel OverallTrustLevel { get; init; }
}
public sealed record ConsumedVexStatement
{
public required string VulnerabilityId { get; init; }
public required VexStatus Status { get; init; }
public VexJustification? Justification { get; init; }
public string? ActionStatement { get; init; }
public ImmutableArray<string> AffectedComponents { get; init; }
public DateTimeOffset? Timestamp { get; init; }
public VexSource Source { get; init; } // sbom_embedded, external, merged
public VexTrustLevel TrustLevel { get; init; }
}
```
- Define VEX status enum matching CycloneDX/OpenVEX:
- NotAffected, Affected, Fixed, UnderInvestigation
Completion criteria:
- [ ] Interface and models defined
- [ ] Status enum covers all VEX states
- [ ] Trust levels defined
### TASK-020-002 - Implement CycloneDX VEX extractor
Status: TODO
Dependency: TASK-020-001
Owners: Developer
Task description:
- Create `CycloneDxVexExtractor`:
- Parse vulnerabilities[] array from CycloneDX SBOM
- Extract analysis.state (exploitable, in_triage, false_positive, not_affected, resolved)
- Extract analysis.justification
- Extract analysis.response[] (workaround_available, will_not_fix, update, rollback)
- Extract affects[] with versions and status
- Extract ratings[] (CVSS v2, v3, v4)
- Map to unified VexStatement model
- Handle both standalone VEX documents and embedded VEX
Completion criteria:
- [ ] Full vulnerabilities[] parsing
- [ ] All analysis fields extracted
- [ ] Affects mapping complete
- [ ] Ratings preserved
### TASK-020-003 - Implement SPDX 3.0.1 VEX extractor
Status: TODO
Dependency: TASK-020-001
Owners: Developer
Task description:
- Create `SpdxVexExtractor`:
- Identify VEX-related relationships in @graph:
- VexAffectedVulnAssessmentRelationship
- VexNotAffectedVulnAssessmentRelationship
- VexFixedVulnAssessmentRelationship
- VexUnderInvestigationVulnAssessmentRelationship
- Extract vulnerability references
- Extract assessment details (justification, actionStatement)
- Extract affected element references
- Map to unified VexStatement model
- Handle SPDX 3.0.1 Security profile completeness
Completion criteria:
- [ ] All VEX relationship types parsed
- [ ] Vulnerability linking complete
- [ ] Assessment details extracted
- [ ] Unified model mapping
### TASK-020-004 - Implement VEX trust evaluation
Status: TODO
Dependency: TASK-020-002
Owners: Developer
Task description:
- Create `VexTrustEvaluator`:
- Evaluate VEX source trust:
- Producer-generated (highest trust)
- Third-party analyst
- Community-contributed (lowest trust)
- Check VEX signature if present
- Validate VEX timestamp freshness
- Check VEX author credentials
- Calculate overall trust level
- Define trust levels: Verified, Trusted, Unverified, Untrusted
- Integration with Signer module for signature verification
Completion criteria:
- [ ] Source trust evaluated
- [ ] Signature verification integrated
- [ ] Timestamp freshness checked
- [ ] Trust level calculated
### TASK-020-005 - Implement VEX conflict resolver
Status: TODO
Dependency: TASK-020-004
Owners: Developer
Task description:
- Create `VexConflictResolver`:
- Detect conflicting VEX statements:
- Same vulnerability, different status
- Different versions/timestamps
- Different sources
- Apply conflict resolution rules:
- Most recent timestamp wins (default)
- Higher trust level wins
- Producer over third-party
- More specific (component-level) over general
- Log conflict resolution decisions
- Allow policy override for resolution strategy
- Generate conflict report for review
Completion criteria:
- [ ] Conflict detection implemented
- [ ] Resolution strategies implemented
- [ ] Decisions logged
- [ ] Policy-driven resolution
### TASK-020-006 - Implement VEX merger with external VEX
Status: TODO
Dependency: TASK-020-005
Owners: Developer
Task description:
- Create `VexMerger`:
- Merge SBOM-embedded VEX with external VEX sources
- External sources:
- Organization VEX repository
- Vendor VEX feeds
- CISA VEX advisories
- Apply merge policy:
- Union (all statements)
- Intersection (only agreed)
- Priority (external or embedded first)
- Track provenance through merge
- Integration with existing Excititor VEX infrastructure
Completion criteria:
- [ ] Merge with external VEX working
- [ ] Multiple merge policies supported
- [ ] Provenance tracked
- [ ] Integration with Excititor
### TASK-020-007 - Create VexConsumptionPolicy configuration
Status: TODO
Dependency: TASK-020-006
Owners: Developer
Task description:
- Define policy schema for VEX consumption:
```yaml
vexConsumptionPolicy:
trustEmbeddedVex: true
minimumTrustLevel: Unverified
signatureRequirements:
requireSignedVex: false
trustedSigners:
- "https://example.com/keys/vex-signer"
timestampRequirements:
maxAgeHours: 720 # 30 days
requireTimestamp: true
conflictResolution:
strategy: mostRecent # or highestTrust, producerWins, interactive
logConflicts: true
mergePolicy:
mode: union # or intersection, externalPriority, embeddedPriority
externalSources:
- type: repository
url: "https://vex.example.com/api"
- type: vendor
url: "https://vendor.example.com/vex"
justificationRequirements:
requireJustificationForNotAffected: true
acceptedJustifications:
- component_not_present
- vulnerable_code_not_present
- vulnerable_code_not_in_execute_path
- inline_mitigations_already_exist
```
Completion criteria:
- [ ] Policy schema defined
- [ ] Trust requirements configurable
- [ ] Conflict resolution configurable
- [ ] Merge modes supported
### TASK-020-008 - Update SbomAdvisoryMatcher to respect VEX
Status: TODO
Dependency: TASK-020-006
Owners: Developer
Task description:
- Modify `SbomAdvisoryMatcher`:
- Check VEX status before reporting vulnerability
- Filter out NotAffected vulnerabilities (configurable)
- Adjust severity based on VEX analysis
- Track VEX source in match results
- Include justification in findings
- Update match result model:
```csharp
public sealed record VexAwareMatchResult
{
public required string VulnerabilityId { get; init; }
public required string ComponentPurl { get; init; }
public VexStatus? VexStatus { get; init; }
public VexJustification? Justification { get; init; }
public VexSource? VexSource { get; init; }
public bool FilteredByVex { get; init; }
}
```
Completion criteria:
- [ ] VEX status checked in matching
- [ ] NotAffected filtering (configurable)
- [ ] Severity adjustment implemented
- [ ] Results include VEX info
### TASK-020-009 - Integrate with Concelier main pipeline
Status: TODO
Dependency: TASK-020-008
Owners: Developer
Task description:
- Add VEX consumption to Concelier processing:
- Extract embedded VEX from ParsedSbom
- Run VexConsumer
- Merge with external VEX if configured
- Pass to SbomAdvisoryMatcher
- Include VEX status in advisory results
- Add CLI options:
- `--trust-embedded-vex`
- `--vex-policy <path>`
- `--external-vex <url>`
- `--ignore-vex` (force full scan)
- Update evidence to include VEX consumption
Completion criteria:
- [ ] VEX consumption in main pipeline
- [ ] CLI options implemented
- [ ] External VEX integration
- [ ] Evidence includes VEX
### TASK-020-010 - Create VEX consumption reporter
Status: TODO
Dependency: TASK-020-009
Owners: Developer
Task description:
- Add VEX section to advisory reports:
- VEX statements inventory
- Filtered vulnerabilities (NotAffected)
- Conflict resolution summary
- Trust level breakdown
- Source distribution (embedded vs external)
- Support JSON, SARIF, human-readable formats
- Include justifications in vulnerability listings
Completion criteria:
- [ ] Report section implemented
- [ ] Filtered vulnerabilities tracked
- [ ] Conflict resolution visible
- [ ] Justifications included
### TASK-020-011 - Unit tests for VEX consumption
Status: TODO
Dependency: TASK-020-009
Owners: QA
Task description:
- Test fixtures:
- CycloneDX SBOMs with embedded VEX
- SPDX 3.0.1 with Security profile VEX
- Conflicting VEX statements
- Signed VEX documents
- Various justification types
- Test each component in isolation
- Test conflict resolution strategies
- Test merge policies
Completion criteria:
- [ ] >90% code coverage
- [ ] All VEX states tested
- [ ] Conflict resolution tested
- [ ] Merge policies tested
### TASK-020-012 - Integration tests with real VEX
Status: TODO
Dependency: TASK-020-011
Owners: QA
Task description:
- Test with real VEX data:
- Vendor VEX documents
- CISA VEX advisories
- CycloneDX VEX examples
- OpenVEX documents
- Verify VEX correctly filters vulnerabilities
- Validate conflict resolution behavior
- Performance testing with large VEX datasets
Completion criteria:
- [ ] Real VEX data tested
- [ ] Correct vulnerability filtering
- [ ] Accurate conflict resolution
- [ ] Performance acceptable
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for VEX consumption | Planning |
## Decisions & Risks
- **Decision**: Support both CycloneDX and SPDX 3.0.1 VEX formats
- **Decision**: Default to trusting embedded VEX (producer-generated)
- **Risk**: VEX may be stale; mitigation is timestamp validation
- **Risk**: Conflicting VEX from multiple sources; mitigation is clear resolution policy
- **Decision**: NotAffected filtering is configurable (default: filter)
## Next Checkpoints
- TASK-020-003 completion: SPDX VEX extraction functional
- TASK-020-006 completion: VEX merging functional
- TASK-020-009 completion: Integration complete
- TASK-020-012 completion: Real-world validation

384
docs/implplan/SPRINT_20260119_021_Policy_license_compliance.md
View File

@@ -1,384 +0,0 @@
# Sprint 20260119_021 · Policy License Compliance Evaluation
## Topic & Scope
- Enable Policy module to evaluate full license expressions from SBOMs (not just SPDX IDs)
- Parse and evaluate complex license expressions (AND, OR, WITH, +)
- Enforce license compatibility policies (copyleft, commercial, attribution)
- Generate license compliance reports for legal review
- Working directory: `src/Policy/`
- Secondary: `src/Concelier/__Libraries/StellaOps.Concelier.SbomIntegration/`
- Expected evidence: Unit tests, license compatibility matrix, compliance reports
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedLicense, ParsedLicenseExpression)
- Can run in parallel with other sprints after 015 delivers license models
## Documentation Prerequisites
- SPDX License List: https://spdx.org/licenses/
- SPDX License Expressions: https://spdx.github.io/spdx-spec/v3.0.1/annexes/SPDX-license-expressions/
- CycloneDX license support
- Open Source license compatibility resources
## Delivery Tracker
### TASK-021-001 - Design license compliance evaluation pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `ILicenseComplianceEvaluator` interface:
```csharp
public interface ILicenseComplianceEvaluator
{
Task<LicenseComplianceReport> EvaluateAsync(
IReadOnlyList<ParsedComponent> components,
LicensePolicy policy,
CancellationToken ct);
}
```
- Design `LicenseComplianceReport`:
```csharp
public sealed record LicenseComplianceReport
{
public LicenseInventory Inventory { get; init; }
public ImmutableArray<LicenseFinding> Findings { get; init; }
public ImmutableArray<LicenseConflict> Conflicts { get; init; }
public LicenseComplianceStatus OverallStatus { get; init; }
public ImmutableArray<AttributionRequirement> AttributionRequirements { get; init; }
}
public sealed record LicenseInventory
{
public ImmutableArray<LicenseUsage> Licenses { get; init; }
public ImmutableDictionary<LicenseCategory, int> ByCategory { get; init; }
public int UnknownLicenseCount { get; init; }
public int NoLicenseCount { get; init; }
}
```
- Define finding types:
- ProhibitedLicense
- CopyleftInProprietaryContext
- LicenseConflict
- UnknownLicense
- MissingLicense
- AttributionRequired
- SourceDisclosureRequired
- PatentClauseRisk
- CommercialRestriction
Completion criteria:
- [ ] Interface and models defined
- [ ] Finding types cover license concerns
- [ ] Attribution tracking included
### TASK-021-002 - Implement SPDX license expression parser
Status: TODO
Dependency: TASK-021-001
Owners: Developer
Task description:
- Create `SpdxLicenseExpressionParser`:
- Parse simple identifiers: MIT, Apache-2.0, GPL-3.0-only
- Parse compound expressions:
- AND: MIT AND Apache-2.0
- OR: MIT OR GPL-2.0-only
- WITH: Apache-2.0 WITH LLVM-exception
- +: GPL-2.0+
- Parse parenthesized expressions: (MIT OR Apache-2.0) AND BSD-3-Clause
- Handle LicenseRef- custom identifiers
- Build expression AST
- Validate against SPDX license list
Completion criteria:
- [ ] All expression operators parsed
- [ ] Precedence correct (WITH > AND > OR)
- [ ] Custom LicenseRef- supported
- [ ] AST construction working
### TASK-021-003 - Implement license expression evaluator
Status: TODO
Dependency: TASK-021-002
Owners: Developer
Task description:
- Create `LicenseExpressionEvaluator`:
- Evaluate OR expressions (any acceptable license)
- Evaluate AND expressions (all licenses must be acceptable)
- Evaluate WITH expressions (license + exception)
- Evaluate + (or-later) expressions
- Determine effective license obligations
- Return:
- Is expression acceptable under policy?
- Obligations arising from expression
- Possible acceptable paths for OR
Completion criteria:
- [ ] All operators evaluated
- [ ] Obligations aggregated correctly
- [ ] OR alternatives tracked
- [ ] Exception handling correct
### TASK-021-004 - Build license knowledge base
Status: TODO
Dependency: TASK-021-001
Owners: Developer
Task description:
- Create `LicenseKnowledgeBase`:
- Load SPDX license list
- Categorize licenses:
- Permissive (MIT, BSD, Apache)
- Weak copyleft (LGPL, MPL, EPL)
- Strong copyleft (GPL, AGPL)
- Proprietary/commercial
- Public domain (CC0, Unlicense)
- Track license attributes:
- Attribution required
- Source disclosure required
- Patent grant
- Trademark restrictions
- Commercial use allowed
- Modification allowed
- Distribution allowed
- Include common non-SPDX licenses
Completion criteria:
- [ ] SPDX list loaded
- [ ] Categories assigned
- [ ] Attributes tracked
- [ ] Non-SPDX licenses included
### TASK-021-005 - Implement license compatibility checker
Status: TODO
Dependency: TASK-021-004
Owners: Developer
Task description:
- Create `LicenseCompatibilityChecker`:
- Define compatibility matrix between licenses
- Check copyleft propagation (GPL infects)
- Check LGPL dynamic linking exceptions
- Detect GPL/proprietary conflicts
- Handle license upgrade paths (GPL-2.0 -> GPL-3.0)
- Check Apache 2.0 / GPL-2.0 patent clause conflict
- Generate conflict explanations
Completion criteria:
- [ ] Compatibility matrix defined
- [ ] Copyleft propagation tracked
- [ ] Common conflicts detected
- [ ] Explanations provided
### TASK-021-006 - Implement project context analyzer
Status: TODO
Dependency: TASK-021-005
Owners: Developer
Task description:
- Create `ProjectContextAnalyzer`:
- Determine project distribution model:
- Internal use only
- Open source distribution
- Commercial/proprietary distribution
- SaaS (AGPL implications)
- Determine linking model:
- Static linking
- Dynamic linking
- Process boundary
- Adjust license evaluation based on context
- Context affects copyleft obligations
Completion criteria:
- [ ] Distribution models defined
- [ ] Linking models tracked
- [ ] Context-aware evaluation
- [ ] AGPL/SaaS handling
### TASK-021-007 - Implement attribution generator
Status: TODO
Dependency: TASK-021-004
Owners: Developer
Task description:
- Create `AttributionGenerator`:
- Collect attribution requirements from licenses
- Extract copyright notices from components
- Generate attribution file (NOTICE, THIRD_PARTY)
- Include license texts where required
- Track per-license attribution format requirements
- Support formats: Markdown, plaintext, HTML
Completion criteria:
- [ ] Attribution requirements collected
- [ ] Copyright notices extracted
- [ ] Attribution file generated
- [ ] Multiple formats supported
### TASK-021-008 - Create LicensePolicy configuration
Status: TODO
Dependency: TASK-021-006
Owners: Developer
Task description:
- Define policy schema for license compliance:
```yaml
licensePolicy:
projectContext:
distributionModel: commercial # internal, openSource, commercial, saas
linkingModel: dynamic # static, dynamic, process
allowedLicenses:
- MIT
- Apache-2.0
- BSD-2-Clause
- BSD-3-Clause
- ISC
prohibitedLicenses:
- GPL-3.0-only
- GPL-3.0-or-later
- AGPL-3.0-only
- AGPL-3.0-or-later
conditionalLicenses:
- license: LGPL-2.1-only
condition: dynamicLinkingOnly
- license: MPL-2.0
condition: fileIsolation
categories:
allowCopyleft: false
allowWeakCopyleft: true
requireOsiApproved: true
unknownLicenseHandling: warn # allow, warn, deny
attributionRequirements:
generateNoticeFile: true
includeLicenseText: true
format: markdown
exemptions:
- componentPattern: "internal-*"
reason: "Internal code, no distribution"
allowedLicenses: [GPL-3.0-only]
```
Completion criteria:
- [ ] Policy schema defined
- [ ] Allowed/prohibited lists
- [ ] Conditional licenses supported
- [ ] Context-aware rules
### TASK-021-009 - Integrate with Policy main pipeline
Status: TODO
Dependency: TASK-021-008
Owners: Developer
Task description:
- Add license evaluation to Policy processing:
- Extract licenses from ParsedSbom components
- Parse license expressions
- Run LicenseComplianceEvaluator
- Generate attribution file if required
- Include findings in policy verdict
- Add CLI options:
- `--license-policy <path>`
- `--project-context <internal|commercial|saas>`
- `--generate-attribution`
- License compliance as release gate
Completion criteria:
- [ ] License evaluation in pipeline
- [ ] CLI options implemented
- [ ] Attribution generation working
- [ ] Release gate integration
### TASK-021-010 - Create license compliance reporter
Status: TODO
Dependency: TASK-021-009
Owners: Developer
Task description:
- Add license section to policy reports:
- License inventory table
- Category breakdown pie chart
- Conflict list with explanations
- Prohibited license violations
- Attribution requirements summary
- NOTICE file content
- Support JSON, PDF, legal-review formats
Completion criteria:
- [ ] Report section implemented
- [ ] Conflict explanations clear
- [ ] Legal-friendly format
- [ ] NOTICE file generated
### TASK-021-011 - Unit tests for license compliance
Status: TODO
Dependency: TASK-021-009
Owners: QA
Task description:
- Test fixtures:
- Simple license IDs
- Complex expressions (AND, OR, WITH, +)
- License conflicts (GPL + proprietary)
- Unknown licenses
- Missing licenses
- Test expression parser
- Test compatibility checker
- Test attribution generator
- Test policy application
Completion criteria:
- [ ] >90% code coverage
- [ ] All expression types tested
- [ ] Compatibility matrix tested
- [ ] Edge cases covered
### TASK-021-012 - Integration tests with real SBOMs
Status: TODO
Dependency: TASK-021-011
Owners: QA
Task description:
- Test with real-world SBOMs:
- npm packages with complex licenses
- Python packages with license expressions
- Java packages with multiple licenses
- Mixed copyleft/permissive projects
- Verify compliance decisions
- Validate attribution generation
Completion criteria:
- [ ] Real SBOM licenses evaluated
- [ ] Correct compliance decisions
- [ ] Attribution files accurate
- [ ] No false positives
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for license compliance | Planning |
## Decisions & Risks
- **Decision**: Use SPDX license list as canonical source
- **Decision**: Support full SPDX license expression syntax
- **Risk**: License categorization is subjective; mitigation is configurable policy
- **Risk**: Non-SPDX licenses require manual mapping; mitigation is LicenseRef- support
- **Decision**: Attribution generation is opt-in
## Next Checkpoints
- TASK-021-003 completion: Expression evaluation functional
- TASK-021-005 completion: Compatibility checking functional
- TASK-021-009 completion: Integration complete
- TASK-021-012 completion: Real-world validation

367
docs/implplan/SPRINT_20260119_022_Scanner_dependency_reachability.md
View File

@@ -1,367 +0,0 @@
# Sprint 20260119_022 · Scanner Dependency Reachability Inference from SBOMs
## Topic & Scope
- Enable Scanner to infer code reachability from SBOM dependency graphs
- Use dependencies[] and relationships to determine if vulnerable code is actually used
- Integrate with existing ReachGraph module for call-graph based reachability
- Reduce false positive vulnerabilities by identifying unreachable code paths
- Working directory: `src/Scanner/`
- Secondary: `src/ReachGraph/`, `src/Concelier/`
- Expected evidence: Unit tests, reachability accuracy metrics, false positive reduction analysis
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - ParsedDependency model)
- Requires: Existing ReachGraph infrastructure
- Can run in parallel with other Scanner sprints after 015 delivers dependency models
## Documentation Prerequisites
- CycloneDX dependencies specification
- SPDX relationships specification
- Existing ReachGraph architecture: `docs/modules/reach-graph/architecture.md`
- Reachability analysis concepts
## Delivery Tracker
### TASK-022-001 - Design reachability inference pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `IReachabilityInferrer` interface:
```csharp
public interface IReachabilityInferrer
{
Task<ReachabilityReport> InferAsync(
ParsedSbom sbom,
ReachabilityPolicy policy,
CancellationToken ct);
Task<ComponentReachability> CheckComponentReachabilityAsync(
string componentPurl,
ParsedSbom sbom,
CancellationToken ct);
}
```
- Design `ReachabilityReport`:
```csharp
public sealed record ReachabilityReport
{
public DependencyGraph Graph { get; init; }
public ImmutableDictionary<string, ReachabilityStatus> ComponentReachability { get; init; }
public ImmutableArray<ReachabilityFinding> Findings { get; init; }
public ReachabilityStatistics Statistics { get; init; }
}
public enum ReachabilityStatus
{
Reachable, // Definitely reachable from entry points
PotentiallyReachable, // May be reachable (conditional, reflection)
Unreachable, // Not in any execution path
Unknown // Cannot determine (missing data)
}
public sealed record ReachabilityStatistics
{
public int TotalComponents { get; init; }
public int ReachableComponents { get; init; }
public int UnreachableComponents { get; init; }
public int UnknownComponents { get; init; }
public double VulnerabilityReductionPercent { get; init; }
}
```
Completion criteria:
- [ ] Interface and models defined
- [ ] Status enum covers all cases
- [ ] Statistics track reduction metrics
### TASK-022-002 - Implement dependency graph builder
Status: TODO
Dependency: TASK-022-001
Owners: Developer
Task description:
- Create `DependencyGraphBuilder`:
- Parse CycloneDX dependencies[] section
- Parse SPDX relationships for DEPENDS_ON, DEPENDENCY_OF
- Build directed graph of component dependencies
- Handle nested/transitive dependencies
- Track dependency scope (runtime, dev, optional, test)
- Support multiple root components (metadata.component or root elements)
- Graph representation using efficient adjacency lists
Completion criteria:
- [ ] CycloneDX dependencies parsed
- [ ] SPDX relationships parsed
- [ ] Transitive dependencies resolved
- [ ] Scope tracking implemented
### TASK-022-003 - Implement entry point detector
Status: TODO
Dependency: TASK-022-002
Owners: Developer
Task description:
- Create `EntryPointDetector`:
- Identify application entry points from SBOM:
- metadata.component (main application)
- Root elements in SPDX
- Components with type=application
- Support multiple entry points (microservices)
- Allow policy-defined entry points
- Handle library SBOMs (all exports as entry points)
- Entry points determine reachability source
Completion criteria:
- [ ] Entry points detected from SBOM
- [ ] Multiple entry points supported
- [ ] Library mode handled
- [ ] Policy overrides supported
### TASK-022-004 - Implement static reachability analyzer
Status: TODO
Dependency: TASK-022-003
Owners: Developer
Task description:
- Create `StaticReachabilityAnalyzer`:
- Perform graph traversal from entry points
- Mark reachable components (BFS/DFS)
- Respect dependency scope:
- Runtime deps: always include
- Optional deps: configurable
- Dev deps: exclude by default
- Test deps: exclude by default
- Handle circular dependencies
- Track shortest path to entry point
- Time complexity: O(V + E)
Completion criteria:
- [ ] Graph traversal implemented
- [ ] Scope-aware analysis
- [ ] Circular dependencies handled
- [ ] Path tracking working
### TASK-022-005 - Implement conditional reachability analyzer
Status: TODO
Dependency: TASK-022-004
Owners: Developer
Task description:
- Create `ConditionalReachabilityAnalyzer`:
- Identify conditionally loaded dependencies:
- Optional imports
- Dynamic requires
- Plugin systems
- Feature flags
- Mark as PotentiallyReachable vs Reachable
- Track conditions from SBOM properties
- Handle scope=optional as potentially reachable
- Integration with existing code analysis if available
Completion criteria:
- [ ] Conditional dependencies identified
- [ ] PotentiallyReachable status assigned
- [ ] Conditions tracked
- [ ] Feature flag awareness
### TASK-022-006 - Implement vulnerability reachability filter
Status: TODO
Dependency: TASK-022-005
Owners: Developer
Task description:
- Create `VulnerabilityReachabilityFilter`:
- Cross-reference vulnerabilities with reachability
- Filter unreachable component vulnerabilities
- Adjust severity based on reachability:
- Reachable: full severity
- PotentiallyReachable: reduced severity (configurable)
- Unreachable: informational only
- Track filtered vulnerabilities for reporting
- Integration with SbomAdvisoryMatcher
Completion criteria:
- [ ] Vulnerability-reachability correlation
- [ ] Filtering implemented
- [ ] Severity adjustment working
- [ ] Filtered vulnerabilities tracked
### TASK-022-007 - Integration with ReachGraph module
Status: TODO
Dependency: TASK-022-006
Owners: Developer
Task description:
- Connect SBOM-based reachability with call-graph analysis:
- Use SBOM dependency graph as coarse filter
- Use ReachGraph call analysis for fine-grained reachability
- Combine results for highest accuracy
- Fall back to SBOM-only when binary analysis unavailable
- Integration points:
- `src/ReachGraph/` for call graph
- `src/Cartographer/` for code maps
- Cascade: SBOM reachability → Call graph reachability
Completion criteria:
- [ ] ReachGraph integration working
- [ ] Combined analysis mode
- [ ] Fallback to SBOM-only
- [ ] Accuracy improvement measured
### TASK-022-008 - Create ReachabilityPolicy configuration
Status: TODO
Dependency: TASK-022-006
Owners: Developer
Task description:
- Define policy schema for reachability inference:
```yaml
reachabilityPolicy:
analysisMode: sbomOnly # sbomOnly, callGraph, combined
scopeHandling:
includeRuntime: true
includeOptional: asPotentiallyReachable
includeDev: false
includeTest: false
entryPoints:
detectFromSbom: true
additional:
- "pkg:npm/my-app@1.0.0"
vulnerabilityFiltering:
filterUnreachable: true
severityAdjustment:
potentiallyReachable: reduceBySeverityLevel # none, reduceBySeverityLevel, reduceByPercentage
unreachable: informationalOnly
reporting:
showFilteredVulnerabilities: true
includeReachabilityPaths: true
confidence:
minimumConfidence: 0.8
markUnknownAs: potentiallyReachable
```
Completion criteria:
- [ ] Policy schema defined
- [ ] Scope handling configurable
- [ ] Filtering rules configurable
- [ ] Confidence thresholds
### TASK-022-009 - Integrate with Scanner main pipeline
Status: TODO
Dependency: TASK-022-008
Owners: Developer
Task description:
- Add reachability inference to Scanner:
- Build dependency graph from ParsedSbom
- Run ReachabilityInferrer
- Pass reachability map to SbomAdvisoryMatcher
- Filter/adjust vulnerability findings
- Include reachability section in report
- Add CLI options:
- `--reachability-analysis`
- `--reachability-policy <path>`
- `--include-unreachable-vulns`
- Track false positive reduction metrics
Completion criteria:
- [ ] Reachability in main pipeline
- [ ] CLI options implemented
- [ ] Vulnerability filtering working
- [ ] Metrics tracked
### TASK-022-010 - Create reachability reporter
Status: TODO
Dependency: TASK-022-009
Owners: Developer
Task description:
- Add reachability section to scan reports:
- Dependency graph visualization (DOT export)
- Reachability summary statistics
- Filtered vulnerabilities table
- Reachability paths for flagged components
- False positive reduction metrics
- Support JSON, SARIF, GraphViz formats
Completion criteria:
- [ ] Report section implemented
- [ ] Graph visualization
- [ ] Reduction metrics visible
- [ ] Paths included
### TASK-022-011 - Unit tests for reachability inference
Status: TODO
Dependency: TASK-022-009
Owners: QA
Task description:
- Test fixtures:
- Simple linear dependency chains
- Diamond dependencies
- Circular dependencies
- Multiple entry points
- Various scopes (runtime, dev, optional)
- Test graph building
- Test reachability traversal
- Test vulnerability filtering
- Test policy application
Completion criteria:
- [ ] >90% code coverage
- [ ] All graph patterns tested
- [ ] Scope handling tested
- [ ] Edge cases covered
### TASK-022-012 - Integration tests and accuracy measurement
Status: TODO
Dependency: TASK-022-011
Owners: QA
Task description:
- Test with real-world SBOMs:
- npm projects with deep dependencies
- Java projects with transitive dependencies
- Python projects with optional dependencies
- Measure:
- False positive reduction rate
- False negative rate (missed reachable vulnerabilities)
- Accuracy vs call-graph analysis
- Establish baseline metrics
Completion criteria:
- [ ] Real SBOM dependency graphs tested
- [ ] Accuracy metrics established
- [ ] False positive reduction quantified
- [ ] No increase in false negatives
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for dependency reachability | Planning |
## Decisions & Risks
- **Decision**: SBOM-based reachability is coarse but widely applicable
- **Decision**: Conservative approach - when uncertain, mark as PotentiallyReachable
- **Risk**: SBOM may have incomplete dependency data; mitigation is Unknown status
- **Risk**: Dynamic loading defeats static analysis; mitigation is PotentiallyReachable
- **Decision**: Reduction metrics must be tracked to prove value
## Next Checkpoints
- TASK-022-004 completion: Static analysis functional
- TASK-022-007 completion: ReachGraph integration
- TASK-022-009 completion: Integration complete
- TASK-022-012 completion: Accuracy validated

377
docs/implplan/SPRINT_20260119_023_Compliance_ntia_supplier.md
View File

@@ -1,377 +0,0 @@
# Sprint 20260119_023 · NTIA Compliance and Supplier Validation
## Topic & Scope
- Validate SBOMs against NTIA minimum elements for software transparency
- Verify supplier/manufacturer information in SBOMs
- Enforce supply chain transparency requirements
- Generate compliance reports for regulatory and contractual obligations
- Working directory: `src/Policy/`
- Secondary: `src/Concelier/`, `src/Scanner/`
- Expected evidence: Unit tests, NTIA compliance checks, supply chain transparency reports
## Dependencies & Concurrency
- Depends on: SPRINT_20260119_015 (Full SBOM extraction - supplier, manufacturer fields)
- Can run in parallel with other sprints after 015 delivers supplier models
## Documentation Prerequisites
- NTIA SBOM Minimum Elements: https://www.ntia.gov/files/ntia/publications/sbom_minimum_elements_report.pdf
- CISA SBOM guidance
- Executive Order 14028 requirements
- FDA SBOM requirements for medical devices
- EU Cyber Resilience Act requirements
## Delivery Tracker
### TASK-023-001 - Design NTIA compliance validation pipeline
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Design `INtiaComplianceValidator` interface:
```csharp
public interface INtiaComplianceValidator
{
Task<NtiaComplianceReport> ValidateAsync(
ParsedSbom sbom,
NtiaCompliancePolicy policy,
CancellationToken ct);
}
```
- Design `NtiaComplianceReport`:
```csharp
public sealed record NtiaComplianceReport
{
public NtiaComplianceStatus OverallStatus { get; init; }
public ImmutableArray<NtiaElementStatus> ElementStatuses { get; init; }
public ImmutableArray<NtiaFinding> Findings { get; init; }
public double ComplianceScore { get; init; } // 0-100%
public SupplierValidationStatus SupplierStatus { get; init; }
}
public sealed record NtiaElementStatus
{
public NtiaElement Element { get; init; }
public bool Present { get; init; }
public bool Valid { get; init; }
public int ComponentsCovered { get; init; }
public int ComponentsMissing { get; init; }
public string? Notes { get; init; }
}
```
- Define NTIA minimum elements enum:
- SupplierName
- ComponentName
- ComponentVersion
- OtherUniqueIdentifiers (PURL, CPE)
- DependencyRelationship
- AuthorOfSbomData
- Timestamp
Completion criteria:
- [ ] Interface and models defined
- [ ] All NTIA elements enumerated
- [ ] Compliance scoring defined
### TASK-023-002 - Implement NTIA baseline field validator
Status: TODO
Dependency: TASK-023-001
Owners: Developer
Task description:
- Create `NtiaBaselineValidator`:
- Validate Supplier Name present for each component
- Validate Component Name present
- Validate Component Version present (or justified absence)
- Validate unique identifier (PURL, CPE, SWID, or hash)
- Validate dependency relationships exist
- Validate SBOM author/creator
- Validate SBOM timestamp
- Track per-component compliance
- Calculate overall compliance percentage
Completion criteria:
- [ ] All 7 baseline elements validated
- [ ] Per-component tracking
- [ ] Compliance percentage calculated
- [ ] Missing element reporting
### TASK-023-003 - Implement supplier information validator
Status: TODO
Dependency: TASK-023-001
Owners: Developer
Task description:
- Create `SupplierValidator`:
- Extract supplier/manufacturer from components
- Validate supplier name format
- Check for placeholder values ("unknown", "n/a", etc.)
- Verify supplier URL if provided
- Cross-reference with known supplier registry (optional)
- Track supplier coverage across SBOM
- Create supplier inventory
Completion criteria:
- [ ] Supplier extraction working
- [ ] Placeholder detection
- [ ] URL validation
- [ ] Coverage tracking
### TASK-023-004 - Implement supplier trust verification
Status: TODO
Dependency: TASK-023-003
Owners: Developer
Task description:
- Create `SupplierTrustVerifier`:
- Check supplier against trusted supplier list
- Check supplier against blocked supplier list
- Verify supplier organization existence (optional external lookup)
- Track supplier-to-component mapping
- Flag unknown suppliers for review
- Define trust levels: Verified, Known, Unknown, Blocked
Completion criteria:
- [ ] Trust list checking implemented
- [ ] Blocked supplier detection
- [ ] Trust level assignment
- [ ] Review flagging
### TASK-023-005 - Implement dependency completeness checker
Status: TODO
Dependency: TASK-023-002
Owners: Developer
Task description:
- Create `DependencyCompletenessChecker`:
- Verify all components have dependency information
- Check for orphaned components (no relationships)
- Validate relationship types are meaningful
- Check for missing transitive dependencies
- Calculate dependency graph completeness score
- Flag SBOMs with incomplete dependency data
Completion criteria:
- [ ] Relationship completeness checked
- [ ] Orphaned components detected
- [ ] Transitive dependency validation
- [ ] Completeness score calculated
### TASK-023-006 - Implement regulatory framework mapper
Status: TODO
Dependency: TASK-023-002
Owners: Developer
Task description:
- Create `RegulatoryFrameworkMapper`:
- Map NTIA elements to other frameworks:
- FDA (medical devices): additional fields
- CISA: baseline + recommendations
- EU CRA: European requirements
- NIST: additional security fields
- Generate multi-framework compliance report
- Track gaps per framework
- Support framework selection in policy
Completion criteria:
- [ ] FDA requirements mapped
- [ ] CISA requirements mapped
- [ ] EU CRA requirements mapped
- [ ] Multi-framework report
### TASK-023-007 - Create NtiaCompliancePolicy configuration
Status: TODO
Dependency: TASK-023-006
Owners: Developer
Task description:
- Define policy schema for NTIA compliance:
```yaml
ntiaCompliancePolicy:
minimumElements:
requireAll: true
elements:
- supplierName
- componentName
- componentVersion
- uniqueIdentifier
- dependencyRelationship
- sbomAuthor
- timestamp
supplierValidation:
rejectPlaceholders: true
placeholderPatterns:
- "unknown"
- "n/a"
- "tbd"
- "todo"
requireUrl: false
trustedSuppliers:
- "Apache Software Foundation"
- "Microsoft"
- "Google"
blockedSuppliers:
- "untrusted-vendor"
uniqueIdentifierPriority:
- purl
- cpe
- swid
- hash
frameworks:
- ntia
- fda # if medical device context
- cisa
thresholds:
minimumCompliancePercent: 95
allowPartialCompliance: false
exemptions:
- componentPattern: "internal-*"
exemptElements: [supplierName]
reason: "Internal components"
```
Completion criteria:
- [ ] Policy schema defined
- [ ] All elements configurable
- [ ] Supplier lists supported
- [ ] Framework selection
### TASK-023-008 - Implement supply chain transparency reporter
Status: TODO
Dependency: TASK-023-004
Owners: Developer
Task description:
- Create `SupplyChainTransparencyReporter`:
- Generate supplier inventory report
- Map components to suppliers
- Calculate supplier concentration (dependency on single supplier)
- Identify unknown/unverified suppliers
- Generate supply chain risk assessment
- Visualization of supplier distribution
Completion criteria:
- [ ] Supplier inventory generated
- [ ] Component mapping complete
- [ ] Concentration analysis
- [ ] Risk assessment included
### TASK-023-009 - Integrate with Policy main pipeline
Status: TODO
Dependency: TASK-023-008
Owners: Developer
Task description:
- Add NTIA validation to Policy processing:
- Run NtiaComplianceValidator on ParsedSbom
- Run SupplierValidator
- Check against compliance thresholds
- Include in policy verdict (pass/fail)
- Generate compliance attestation
- Add CLI options:
- `--ntia-compliance`
- `--ntia-policy <path>`
- `--supplier-validation`
- `--regulatory-frameworks <ntia,fda,cisa>`
- NTIA compliance as release gate
Completion criteria:
- [ ] NTIA validation in pipeline
- [ ] CLI options implemented
- [ ] Release gate integration
- [ ] Attestation generated
### TASK-023-010 - Create compliance and transparency reports
Status: TODO
Dependency: TASK-023-009
Owners: Developer
Task description:
- Add compliance section to policy reports:
- NTIA element checklist
- Compliance score dashboard
- Per-component compliance table
- Supplier inventory
- Supply chain risk summary
- Regulatory framework mapping
- Support JSON, PDF, regulatory submission formats
Completion criteria:
- [ ] Report section implemented
- [ ] Compliance checklist visible
- [ ] Regulatory formats supported
- [ ] Supplier inventory included
### TASK-023-011 - Unit tests for NTIA compliance
Status: TODO
Dependency: TASK-023-009
Owners: QA
Task description:
- Test fixtures:
- Fully compliant SBOMs
- SBOMs missing each element type
- SBOMs with placeholder suppliers
- Various compliance percentages
- Test baseline validator
- Test supplier validator
- Test dependency completeness
- Test policy application
Completion criteria:
- [ ] >90% code coverage
- [ ] All elements tested
- [ ] Supplier validation tested
- [ ] Edge cases covered
### TASK-023-012 - Integration tests with real SBOMs
Status: TODO
Dependency: TASK-023-011
Owners: QA
Task description:
- Test with real-world SBOMs:
- SBOMs from major package managers
- Vendor-provided SBOMs
- Tool-generated SBOMs (Syft, Trivy)
- FDA-compliant medical device SBOMs
- Measure:
- Typical compliance rates
- Common missing elements
- Supplier data quality
- Establish baseline expectations
Completion criteria:
- [ ] Real SBOM compliance evaluated
- [ ] Baseline metrics established
- [ ] Common gaps identified
- [ ] Reports suitable for regulatory use
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-19 | Sprint created for NTIA compliance | Planning |
## Decisions & Risks
- **Decision**: NTIA minimum elements as baseline, extend for other frameworks
- **Decision**: Supplier validation is optional but recommended
- **Risk**: Many SBOMs lack supplier information; mitigation is reporting gaps clearly
- **Risk**: Placeholder values are common; mitigation is configurable detection
- **Decision**: Compliance can be a release gate or advisory (configurable)
## Next Checkpoints
- TASK-023-002 completion: Baseline validation functional
- TASK-023-004 completion: Supplier validation functional
- TASK-023-009 completion: Integration complete
- TASK-023-012 completion: Real-world validation

488
docs/implplan/SPRINT_20260119_024_Scanner_license_detection_enhancements.md
View File

@@ -1,488 +0,0 @@
# Sprint 20260119_024 · Scanner License Detection Enhancements
## Topic & Scope
- Enhance Scanner license detection to include categorization, compatibility hints, and attribution preparation
- Unify license detection across all language analyzers with consistent output
- Add license file content extraction and preservation
- Integrate with SPDX license list for validation and categorization during scan
- Prepare license metadata for downstream Policy evaluation
- Working directory: `src/Scanner/__Libraries/`
- Expected evidence: Unit tests, categorization accuracy, attribution extraction tests
## Dependencies & Concurrency
- Can run independently of other sprints
- Complements SPRINT_20260119_021 (Policy license compliance)
- Uses existing SPDX infrastructure in `StellaOps.Scanner.Emit/Spdx/Licensing/`
## Documentation Prerequisites
- SPDX License List: https://spdx.org/licenses/
- Existing license detection: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.*/`
- SPDX expression parser: `src/Scanner/__Libraries/StellaOps.Scanner.Emit/Spdx/Licensing/SpdxLicenseExpressions.cs`
## Delivery Tracker
### TASK-024-001 - Create unified LicenseDetectionResult model
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Create unified model for license detection results across all language analyzers:
```csharp
public sealed record LicenseDetectionResult
{
// Core identification
public required string SpdxId { get; init; } // Normalized SPDX ID or LicenseRef-
public string? OriginalText { get; init; } // Original license string from source
public string? LicenseUrl { get; init; } // URL if provided
// Detection metadata
public LicenseDetectionConfidence Confidence { get; init; }
public LicenseDetectionMethod Method { get; init; }
public string? SourceFile { get; init; } // Where detected (LICENSE, package.json, etc.)
public int? SourceLine { get; init; } // Line number if applicable
// Categorization (NEW)
public LicenseCategory Category { get; init; }
public ImmutableArray<LicenseObligation> Obligations { get; init; }
// License content (NEW)
public string? LicenseText { get; init; } // Full license text if extracted
public string? LicenseTextHash { get; init; } // SHA256 of license text
public string? CopyrightNotice { get; init; } // Extracted copyright line(s)
// Expression support (NEW)
public bool IsExpression { get; init; } // True if this is a compound expression
public ImmutableArray<string> ExpressionComponents { get; init; } // Individual licenses in expression
}
public enum LicenseDetectionConfidence { High, Medium, Low, None }
public enum LicenseDetectionMethod
{
SpdxHeader, // SPDX-License-Identifier comment
PackageMetadata, // package.json, Cargo.toml, pom.xml
LicenseFile, // LICENSE, COPYING file
ClassifierMapping, // PyPI classifiers
UrlMatching, // License URL lookup
PatternMatching, // Text pattern in license file
KeywordFallback // Basic keyword detection
}
public enum LicenseCategory
{
Permissive, // MIT, BSD, Apache, ISC
WeakCopyleft, // LGPL, MPL, EPL, CDDL
StrongCopyleft, // GPL, AGPL
NetworkCopyleft, // AGPL specifically
PublicDomain, // CC0, Unlicense, WTFPL
Proprietary, // Custom/commercial
Unknown // Cannot categorize
}
public enum LicenseObligation
{
Attribution, // Must include copyright notice
SourceDisclosure, // Must provide source code
SameLicense, // Derivatives must use same license
PatentGrant, // Includes patent grant
NoWarranty, // Disclaimer required
StateChanges, // Must document modifications
IncludeLicense // Must include license text
}
```
Completion criteria:
- [ ] Unified model defined
- [ ] All existing detection results can map to this model
- [ ] Category and obligation enums comprehensive
### TASK-024-002 - Build license categorization service
Status: TODO
Dependency: TASK-024-001
Owners: Developer
Task description:
- Create `ILicenseCategorizationService`:
```csharp
public interface ILicenseCategorizationService
{
LicenseCategory Categorize(string spdxId);
IReadOnlyList<LicenseObligation> GetObligations(string spdxId);
bool IsOsiApproved(string spdxId);
bool IsFsfFree(string spdxId);
bool IsDeprecated(string spdxId);
}
```
- Implement categorization database:
- Load from SPDX license list metadata
- Manual overrides for common licenses
- Cache for performance
- Categorization rules:
| License Pattern | Category |
|-----------------|----------|
| MIT, BSD-*, ISC, Apache-*, Zlib, Boost-*, PSF-*, Unlicense | Permissive |
| LGPL-*, MPL-*, EPL-*, CDDL-*, OSL-* | WeakCopyleft |
| GPL-* (not LGPL/AGPL), EUPL-* | StrongCopyleft |
| AGPL-* | NetworkCopyleft |
| CC0-*, 0BSD, WTFPL | PublicDomain |
| LicenseRef-*, Unknown | Unknown |
- Obligation mapping per license
Completion criteria:
- [ ] All 600+ SPDX licenses categorized
- [ ] Obligations mapped for major licenses
- [ ] OSI/FSF approval tracked
- [ ] Deprecated licenses flagged
### TASK-024-003 - Implement license text extractor
Status: TODO
Dependency: TASK-024-001
Owners: Developer
Task description:
- Create `ILicenseTextExtractor`:
```csharp
public interface ILicenseTextExtractor
{
Task<LicenseTextExtractionResult> ExtractAsync(
string filePath,
CancellationToken ct);
}
public sealed record LicenseTextExtractionResult
{
public string FullText { get; init; }
public string TextHash { get; init; } // SHA256
public ImmutableArray<string> CopyrightNotices { get; init; }
public string? DetectedLicenseId { get; init; } // If identifiable from text
public LicenseDetectionConfidence Confidence { get; init; }
}
```
- Extract functionality:
- Read LICENSE, COPYING, NOTICE files
- Extract copyright lines (© or "Copyright" patterns)
- Compute hash for deduplication
- Detect license from text patterns
- Handle various encodings (UTF-8, ASCII, UTF-16)
- Maximum file size: 1MB (configurable)
Completion criteria:
- [ ] License text extracted and preserved
- [ ] Copyright notices extracted
- [ ] Hash computed for deduplication
- [ ] Encoding handled correctly
### TASK-024-004 - Implement copyright notice extractor
Status: TODO
Dependency: TASK-024-003
Owners: Developer
Task description:
- Create `ICopyrightExtractor`:
```csharp
public interface ICopyrightExtractor
{
IReadOnlyList<CopyrightNotice> Extract(string text);
}
public sealed record CopyrightNotice
{
public string FullText { get; init; }
public string? Year { get; init; } // "2020" or "2018-2024"
public string? Holder { get; init; } // "Google LLC"
public int LineNumber { get; init; }
}
```
- Copyright patterns to detect:
- `Copyright (c) YYYY Name`
- `Copyright © YYYY Name`
- `(c) YYYY Name`
- `YYYY Name. All rights reserved.`
- Year ranges: `2018-2024`
- Parse holder name from copyright line
Completion criteria:
- [ ] All common copyright patterns detected
- [ ] Year and holder extracted
- [ ] Multi-line copyright handled
- [ ] Non-ASCII (©) supported
### TASK-024-005 - Upgrade Python license detector
Status: TODO
Dependency: TASK-024-002
Owners: Developer
Task description:
- Refactor `StellaOps.Scanner.Analyzers.Lang.Python/.../SpdxLicenseNormalizer.cs`:
- Return `LicenseDetectionResult` instead of simple string
- Add categorization from `ILicenseCategorizationService`
- Extract license text from LICENSE file if present
- Extract copyright notices
- Support license expressions in PEP 639 format
- Preserve original classifier text
- Maintain backwards compatibility
Completion criteria:
- [ ] Returns LicenseDetectionResult
- [ ] Categorization included
- [ ] License text extracted when available
- [ ] Copyright notices extracted
### TASK-024-006 - Upgrade Java license detector
Status: TODO
Dependency: TASK-024-002
Owners: Developer
Task description:
- Refactor `StellaOps.Scanner.Analyzers.Lang.Java/.../SpdxLicenseNormalizer.cs`:
- Return `LicenseDetectionResult` instead of simple result
- Add categorization
- Extract license text from LICENSE file in JAR/project
- Parse license URL and fetch text (optional, configurable)
- Extract copyright from NOTICE file (common in Apache projects)
- Handle multiple licenses in pom.xml
- Support Maven and Gradle metadata
Completion criteria:
- [ ] Returns LicenseDetectionResult
- [ ] Categorization included
- [ ] NOTICE file parsing
- [ ] Multiple licenses handled
### TASK-024-007 - Upgrade Go license detector
Status: TODO
Dependency: TASK-024-002
Owners: Developer
Task description:
- Refactor `StellaOps.Scanner.Analyzers.Lang.Go/.../GoLicenseDetector.cs`:
- Return `LicenseDetectionResult`
- Already reads LICENSE file - preserve full text
- Add categorization
- Extract copyright notices from LICENSE
- Improve pattern matching confidence
- Support go.mod license comments (future Go feature)
Completion criteria:
- [ ] Returns LicenseDetectionResult
- [ ] Full license text preserved
- [ ] Categorization included
- [ ] Copyright extraction improved
### TASK-024-008 - Upgrade Rust license detector
Status: TODO
Dependency: TASK-024-002
Owners: Developer
Task description:
- Refactor `StellaOps.Scanner.Analyzers.Lang.Rust/.../RustLicenseScanner.cs`:
- Return `LicenseDetectionResult`
- Parse license expressions from Cargo.toml
- Read license-file content when specified
- Add categorization
- Extract copyright from license file
- Handle workspace-level licenses
Completion criteria:
- [ ] Returns LicenseDetectionResult
- [ ] Expression parsing preserved
- [ ] License file content extracted
- [ ] Categorization included
### TASK-024-009 - Add JavaScript/TypeScript license detector
Status: TODO
Dependency: TASK-024-002
Owners: Developer
Task description:
- Create new analyzer `StellaOps.Scanner.Analyzers.Lang.JavaScript`:
- Parse package.json `license` field
- Parse package.json `licenses` array (legacy)
- Support SPDX expressions
- Read LICENSE file from package
- Extract copyright notices
- Add categorization
- Handle monorepo structures (lerna, nx, turborepo)
Completion criteria:
- [ ] package.json license parsed
- [ ] SPDX expressions supported
- [ ] LICENSE file extracted
- [ ] Categorization included
### TASK-024-010 - Add .NET/NuGet license detector
Status: TODO
Dependency: TASK-024-002
Owners: Developer
Task description:
- Create new analyzer `StellaOps.Scanner.Analyzers.Lang.DotNet`:
- Parse .csproj `PackageLicenseExpression`
- Parse .csproj `PackageLicenseFile`
- Parse .nuspec license metadata
- Read LICENSE file from package
- Extract copyright from AssemblyInfo
- Add categorization
- Handle license URL (deprecated but common)
Completion criteria:
- [ ] .csproj license metadata parsed
- [ ] .nuspec support
- [ ] License expressions supported
- [ ] Categorization included
### TASK-024-011 - Update LicenseEvidenceBuilder for enhanced output
Status: TODO
Dependency: TASK-024-008
Owners: Developer
Task description:
- Refactor `LicenseEvidenceBuilder.cs`:
- Accept `LicenseDetectionResult` instead of simple evidence
- Include category in evidence properties
- Include obligations in evidence properties
- Preserve license text hash for deduplication
- Store copyright notices
- Generate CycloneDX 1.7 native license evidence structure
- Update evidence format:
```
stellaops:license:id=MIT
stellaops:license:category=Permissive
stellaops:license:obligations=Attribution,IncludeLicense
stellaops:license:copyright=Copyright (c) 2024 Acme Inc
stellaops:license:textHash=sha256:abc123...
```
Completion criteria:
- [ ] Enhanced evidence format
- [ ] Category and obligations in output
- [ ] Copyright preserved
- [ ] CycloneDX 1.7 native format
### TASK-024-012 - Create license detection CLI commands
Status: TODO
Dependency: TASK-024-011
Owners: Developer
Task description:
- Add CLI commands for license operations:
- `stella license detect <path>` - Detect licenses in directory
- `stella license categorize <spdx-id>` - Show category and obligations
- `stella license validate <expression>` - Validate SPDX expression
- `stella license extract <file>` - Extract license text and copyright
- Output formats: JSON, table, SPDX
Completion criteria:
- [ ] CLI commands implemented
- [ ] Multiple output formats
- [ ] Useful for manual license review
### TASK-024-013 - Create license detection aggregator
Status: TODO
Dependency: TASK-024-011
Owners: Developer
Task description:
- Create `ILicenseDetectionAggregator`:
```csharp
public interface ILicenseDetectionAggregator
{
LicenseDetectionSummary Aggregate(
IReadOnlyList<LicenseDetectionResult> results);
}
public sealed record LicenseDetectionSummary
{
public ImmutableArray<LicenseDetectionResult> UniqueByComponent { get; init; }
public ImmutableDictionary<LicenseCategory, int> ByCategory { get; init; }
public ImmutableDictionary<string, int> BySpdxId { get; init; }
public int TotalComponents { get; init; }
public int ComponentsWithLicense { get; init; }
public int ComponentsWithoutLicense { get; init; }
public int UnknownLicenses { get; init; }
public ImmutableArray<string> AllCopyrightNotices { get; init; }
}
```
- Aggregate across all detected licenses
- Deduplicate by component
- Calculate statistics for reporting
Completion criteria:
- [ ] Aggregation implemented
- [ ] Statistics calculated
- [ ] Deduplication working
- [ ] Ready for policy evaluation
### TASK-024-014 - Unit tests for enhanced license detection
Status: TODO
Dependency: TASK-024-013
Owners: QA
Task description:
- Test fixtures for each language:
- Python: setup.py, pyproject.toml, classifiers
- Java: pom.xml, build.gradle, NOTICE
- Go: LICENSE files with various licenses
- Rust: Cargo.toml with expressions
- JavaScript: package.json with expressions
- .NET: .csproj, .nuspec
- Test categorization accuracy
- Test copyright extraction
- Test expression parsing
- Test aggregation
Completion criteria:
- [ ] >90% code coverage
- [ ] All languages tested
- [ ] Categorization accuracy >95%
- [ ] Copyright extraction tested
### TASK-024-015 - Integration tests with real projects
Status: TODO
Dependency: TASK-024-014
Owners: QA
Task description:
- Test with real open source projects:
- lodash (MIT, JavaScript)
- requests (Apache-2.0, Python)
- spring-boot (Apache-2.0, Java)
- kubernetes (Apache-2.0, Go)
- serde (MIT OR Apache-2.0, Rust)
- Newtonsoft.Json (MIT, .NET)
- Verify:
- Correct license detection
- Correct categorization
- Copyright extraction
- Expression handling
Completion criteria:
- [ ] Real projects scanned
- [ ] Licenses correctly detected
- [ ] Categories accurate
- [ ] No regressions
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created for scanner license enhancements | Planning |
## Decisions & Risks
- **Decision**: Unified LicenseDetectionResult model for all languages
- **Decision**: Categorization is best-effort, Policy module makes final decisions
- **Risk**: License text extraction increases scan time; mitigation is opt-in/configurable
- **Risk**: Some licenses hard to categorize; mitigation is Unknown category and manual override
- **Decision**: Add JavaScript and .NET detectors to cover major ecosystems
## Next Checkpoints
- TASK-024-002 completion: Categorization service functional
- TASK-024-008 completion: All existing detectors upgraded
- TASK-024-011 completion: Evidence builder updated
- TASK-024-015 completion: Real-world validation

84
docs/implplan/SPRINT_20260119_025_DOCS_license_notes_apache_transition.md
View File

@@ -1,84 +0,0 @@
# Sprint 20260119_025 · License Notes + Apache 2.0 Transition
## Topic & Scope
- Move StellaOps licensing documentation and notices to Apache-2.0.
- Reconcile third-party license compatibility statements with Apache-2.0.
- Consolidate license declarations and cross-links so NOTICE/THIRD-PARTY inventory are canonical.
- Working directory: `docs/legal/`.
- Expected evidence: updated license docs, updated root LICENSE/NOTICE, and refreshed dates.
- Cross-path edits: `LICENSE`, `NOTICE.md`, `third-party-licenses/` (references only).
## Dependencies & Concurrency
- No upstream sprint dependencies.
- Safe to run in parallel with code changes; avoid conflicting edits to legal docs.
## Documentation Prerequisites
- `docs/README.md`
- `docs/ARCHITECTURE_OVERVIEW.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/legal/THIRD-PARTY-DEPENDENCIES.md`
- `docs/legal/LICENSE-COMPATIBILITY.md`
- `LICENSE` (current baseline)
## Delivery Tracker
### TASK-DOCS-LIC-001 - Update core license notices to Apache-2.0
Status: DONE
Dependency: none
Owners: Documentation author
Task description:
- Replace root `LICENSE` with Apache License 2.0 text.
- Update `NOTICE.md` to reference Apache-2.0 and align attribution language.
- Ensure core license statements in legal docs reflect Apache-2.0.
Completion criteria:
- [ ] `LICENSE` contains Apache License 2.0 text
- [ ] `NOTICE.md` references Apache-2.0 and remains consistent with third-party notices
- [ ] Legal docs no longer describe StellaOps as AGPL-3.0-or-later
### TASK-DOCS-LIC-002 - Reconcile third-party compatibility + inventory
Status: DONE
Dependency: TASK-DOCS-LIC-001
Owners: Documentation author
Task description:
- Update `docs/legal/THIRD-PARTY-DEPENDENCIES.md` compatibility language to Apache-2.0.
- Update `docs/legal/LICENSE-COMPATIBILITY.md` matrices, distribution guidance, and FAQ.
- Consolidate license declaration references and ensure canonical sources are clear.
Completion criteria:
- [ ] Compatibility matrix reflects Apache-2.0 inbound rules
- [ ] Third-party inventory reflects Apache-2.0 compatibility language
- [ ] Canonical license declaration locations are stated clearly
### TASK-DOCS-LIC-003 - Update license notes in related legal guidance
Status: DONE
Dependency: TASK-DOCS-LIC-001
Owners: Documentation author
Task description:
- Align `docs/legal/crypto-compliance-review.md` and `docs/legal/LEGAL_FAQ_QUOTA.md`
with Apache-2.0 language.
- Record follow-up gaps that require code/package changes.
Completion criteria:
- [ ] Crypto compliance review no longer references AGPL compatibility
- [ ] Legal FAQ references Apache-2.0 obligations accurately
- [ ] Follow-up gaps captured in Decisions & Risks
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created for Apache-2.0 licensing updates. | Docs |
| 2026-01-20 | Updated LICENSE/NOTICE and legal docs for Apache-2.0 compatibility. | Docs |
| 2026-01-20 | Apache-2.0 transition superseded by BUSL-1.1 decision (see `SPRINT_20260120_028_DOCS_busl_license_transition.md`). | Docs |
## Decisions & Risks
- Required reading references `docs/implplan/SPRINT_0301_0001_0001_docs_md_i.md`, but the file is missing; proceed under this sprint and flag for follow-up.
- License change requires future code header/package metadata updates outside `docs/legal/` (source headers, package manifests, OpenAPI metadata).
- Apache-2.0 licensing decisions superseded by BUSL-1.1 transition; update license docs under `SPRINT_20260120_028_DOCS_busl_license_transition.md`.
## Next Checkpoints
- License docs aligned and Apache-2.0 text in place.
- Follow-up tasks for code metadata documented.

83
docs/implplan/SPRINT_20260120_026_Compliance_license_metadata_alignment.md
View File

@@ -1,83 +0,0 @@
# Sprint 20260120_026 · License Metadata Alignment (Apache-2.0)
## Topic & Scope
- Align non-source metadata and tooling references with Apache-2.0 licensing.
- Update SPDX headers and OCI labels in DevOps assets.
- Update root-level and configuration samples to reflect Apache-2.0.
- Working directory: `.` (repo root).
- Expected evidence: updated headers/labels, updated checklist references, and refreshed dates.
- Cross-path edits: `devops/**`, `etc/**`, `docs/**`, `AGENTS.md`, `opt/**`.
## Dependencies & Concurrency
- Depends on: `SPRINT_20260119_025_DOCS_license_notes_apache_transition.md` (docs baseline).
- Can run in parallel with module-level source header updates.
## Documentation Prerequisites
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/ARCHITECTURE_OVERVIEW.md`
- `docs/operations/devops/architecture.md`
- `docs/modules/platform/architecture-overview.md`
## Delivery Tracker
### TASK-COMP-LIC-001 - Update root and config SPDX/metadata
Status: DONE
Dependency: none
Owners: Documentation author, DevOps
Task description:
- Update `AGENTS.md` license statement to Apache-2.0.
- Update SPDX headers in `etc/*.example` and `etc/notify-templates/*.sample`.
- Update `opt/cryptopro/downloads/README.md` license phrasing.
- Update non-legal docs license references (governance, openapi docs, distribution matrix, feature matrix).
Completion criteria:
- [ ] Root AGENTS license statement updated
- [ ] Example configs reflect Apache-2.0 SPDX
- [ ] CryptoPro README reflects Apache-2.0 wording
- [ ] Non-legal docs license references updated
### TASK-COMP-LIC-002 - Update DevOps scripts, labels, and checklists
Status: DONE
Dependency: TASK-COMP-LIC-001
Owners: DevOps
Task description:
- Update SPDX headers in devops scripts/tools.
- Update OCI image license labels in DevOps Dockerfiles.
- Update DevOps GA checklist license references.
- Update DevOps package.json/license metadata where applicable.
Completion criteria:
- [ ] DevOps scripts updated to Apache-2.0 SPDX
- [ ] Docker labels updated to Apache-2.0
- [ ] GA checklist references Apache-2.0
- [ ] Node tooling metadata uses Apache-2.0
### TASK-COMP-LIC-003 - Record follow-up scope for src/** license headers
Status: DONE
Dependency: TASK-COMP-LIC-001
Owners: Project manager
Task description:
- Record the remaining `src/**` license headers and package metadata needing update.
- Identify any module-specific AGENTS prerequisites before edits.
Completion criteria:
- [ ] Follow-up list recorded in Decisions & Risks
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created for license metadata alignment. | Docs |
| 2026-01-20 | Updated root/config/DevOps/docs metadata to Apache-2.0. | Docs |
| 2026-01-20 | Apache-2.0 alignment superseded by BUSL-1.1 transition (see `SPRINT_20260120_028_DOCS_busl_license_transition.md`). | Docs |
## Decisions & Risks
- Source headers and package manifests under `src/**` are not updated in this sprint; they require module-level AGENTS review before edits. Follow-up scope: SPDX headers, csproj `PackageLicenseExpression`, package.json `license`, OpenAPI `info.license`, and OCI label values under `src/**`.
- Apache-2.0 alignment superseded by BUSL-1.1 transition; new scope tracked in `SPRINT_20260120_028_DOCS_busl_license_transition.md`.
## Next Checkpoints
- DevOps assets and configs aligned to Apache-2.0.
- Follow-up scope defined for module header updates.

79
docs/implplan/SPRINT_20260120_027_Platform_license_header_alignment.md
View File

@@ -1,79 +0,0 @@
# Sprint 20260120_027 · Source License Header Alignment (Apache-2.0)
## Topic & Scope
- Update StellaOps source headers and metadata to Apache-2.0.
- Align package license expressions, plugin metadata, and default license strings.
- Working directory: `src/`.
- Expected evidence: updated SPDX headers, updated package metadata, and notes on excluded fixtures.
- Cross-module edits: allowed for license headers and metadata only.
## Dependencies & Concurrency
- Depends on: `SPRINT_20260120_026_Compliance_license_metadata_alignment.md`.
- Safe to run in parallel with feature work if it avoids behavioral changes.
## Documentation Prerequisites
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/ARCHITECTURE_OVERVIEW.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/code-of-conduct/CODE_OF_CONDUCT.md`
## Delivery Tracker
### TASK-SRC-LIC-001 - Update shared package/license metadata
Status: BLOCKED
Dependency: none
Owners: Developer
Task description:
- Update `src/Directory.Build.props` license expression.
- Update explicit `PackageLicenseExpression` overrides in `src/**.csproj`.
- Update Node package metadata under `src/**/package.json`.
- Update plugin metadata files (`plugin.yaml`) to Apache-2.0.
Completion criteria:
- [ ] Directory.Build.props uses Apache-2.0 (superseded by BUSL-1.1 transition)
- [ ] All csproj license expressions use Apache-2.0 (superseded by BUSL-1.1 transition)
- [ ] Node metadata license fields updated (superseded by BUSL-1.1 transition)
- [ ] Plugin metadata license fields updated (superseded by BUSL-1.1 transition)
### TASK-SRC-LIC-002 - Update source header license statements
Status: BLOCKED
Dependency: TASK-SRC-LIC-001
Owners: Developer
Task description:
- Replace SPDX and "Licensed under" header lines in `src/**/*.cs` and scripts.
- Avoid modifying third-party fixtures and SPDX license lists used for detection.
Completion criteria:
- [ ] Source headers reflect Apache-2.0 (superseded by BUSL-1.1 transition)
- [ ] Excluded fixtures noted in Decisions & Risks
### TASK-SRC-LIC-003 - Update runtime defaults referencing project license
Status: BLOCKED
Dependency: TASK-SRC-LIC-001
Owners: Developer
Task description:
- Update default license strings in OpenAPI/metadata outputs.
- Update sample plugin license fields that represent StellaOps license.
Completion criteria:
- [ ] OpenAPI license defaults updated (superseded by BUSL-1.1 transition)
- [ ] Sample plugin license strings updated (superseded by BUSL-1.1 transition)
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created for source license header alignment. | Dev |
| 2026-01-20 | Scope superseded by BUSL-1.1 license transition (see `SPRINT_20260120_028_DOCS_busl_license_transition.md`). | Dev |
## Decisions & Risks
- Some fixtures include AGPL strings for license detection tests; these remain unchanged to preserve test coverage.
- Module-specific AGENTS and dossiers will be consulted as needed for touched areas.
- Apache-2.0 alignment superseded by BUSL-1.1 transition; defer work to `SPRINT_20260120_028_DOCS_busl_license_transition.md`.
## Next Checkpoints
- Source license headers aligned to Apache-2.0.
- Remaining AGPL occurrences limited to fixtures and license detection logic.

199
docs/implplan/SPRINT_20260120_028_DOCS_busl_license_transition.md
View File

@@ -1,199 +0,0 @@
# Sprint 20260120-028 <20> BUSL license transition
## Topic & Scope
- Replace Apache-2.0/AGPL-3.0 references with BUSL-1.1 + Additional Use Grant across repo-facing license artifacts.
- Align license metadata in docs, package manifests, OpenAPI specs, and SPDX headers while preserving third-party license data.
- Consolidate license declarations into `LICENSE`, `NOTICE.md`, and `docs/legal/THIRD-PARTY-DEPENDENCIES.md` with clear cross-links.
- Working directory: `.` (repo root; cross-module edits approved for license metadata in `LICENSE`, `NOTICE.md`, `docs/`, `src/`, `devops/`, `etc/`, `opt/`).
- Expected evidence: updated license artifacts + docs + metadata references; `rg` sweep results recorded in Execution Log.
## Dependencies & Concurrency
- Supersedes the Apache license alignment in `SPRINT_20260119_025_DOCS_license_notes_apache_transition.md`, `SPRINT_20260120_026_Compliance_license_metadata_alignment.md`, and `SPRINT_20260120_027_Platform_license_header_alignment.md`.
- Safe to execute in parallel with feature work as long as license headers and docs stay consistent.
## Documentation Prerequisites
- `LICENSE`
- `NOTICE.md`
- `docs/legal/THIRD-PARTY-DEPENDENCIES.md`
- `docs/legal/LICENSE-COMPATIBILITY.md`
## Delivery Tracker
### BUSL-028-01 - Core license artifacts and legal docs
Status: DONE
Dependency: none
Owners: Documentation
Task description:
- Replace repo license text with BUSL-1.1 + Additional Use Grant and update NOTICE/legal docs to reflect BUSL.
- Update governance, release, and strategy docs that describe project licensing.
- Ensure third-party notices remain intact and referenced from canonical docs.
Completion criteria:
- [ ] `LICENSE` contains BUSL-1.1 parameters + unmodified BUSL text.
- [ ] `NOTICE.md` and legal docs describe BUSL-1.1 and Additional Use Grant, and link to third-party notices.
- [ ] References to Apache/AGPL as the project license are removed or re-scoped.
### BUSL-028-02 - Metadata and SPDX headers
Status: DONE
Dependency: BUSL-028-01
Owners: Documentation, Developer
Task description:
- Update package/license metadata, OpenAPI license entries, plugin manifests, and SPDX headers to BUSL-1.1.
- Preserve third-party license fixtures and license detection datasets.
Completion criteria:
- [ ] `PackageLicenseExpression`, `license` fields, and OpenAPI license names/URLs are BUSL-1.1 where they represent StellaOps.
- [ ] SPDX headers in repo-owned files use `BUSL-1.1`.
- [ ] Third-party license fixtures and datasets remain unchanged.
### BUSL-028-03 - Verification and consolidation log
Status: DONE
Dependency: BUSL-028-02
Owners: Documentation
Task description:
- Sweep for remaining Apache/AGPL references and document accepted exceptions (third-party data, compatibility tables).
- Record results in Execution Log and Decisions & Risks.
Completion criteria:
- [ ] `rg` sweep results recorded with exceptions noted.
- [ ] Decisions & Risks updated with BUSL change rationale and Change Date.
### BUSL-028-04 - Follow-up consolidation and residual review
Status: DONE
Dependency: BUSL-028-03
Owners: Documentation
Task description:
- Add a consolidated license index in `docs/README.md` and align FAQ wording to BUSL.
- Validate remaining Apache references are third-party or test fixtures and log exceptions.
Completion criteria:
- [ ] `docs/README.md` links to canonical license/notice documents.
- [ ] FAQ and compatibility references are BUSL-aligned.
- [ ] Residual Apache references documented as exceptions.
### BUSL-028-05 - Legal index and expanded sweep
Status: DONE
Dependency: BUSL-028-04
Owners: Documentation
Task description:
- Add a legal index under `docs/legal/README.md` and link it from docs index.
- Run broader Apache/AGPL sweeps across non-archived content and document residual exceptions.
Completion criteria:
- [ ] `docs/legal/README.md` lists canonical legal documents.
- [ ] `docs/README.md` links to the legal index.
- [ ] Expanded sweep results logged with accepted exceptions.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created for BUSL-1.1 transition. | Planning |
| 2026-01-20 | Replaced LICENSE/NOTICE and legal docs for BUSL-1.1 + Additional Use Grant; updated governance/strategy docs. | Docs |
| 2026-01-20 | Updated SPDX headers, package metadata, plugin manifests, and OpenAPI license entries to BUSL-1.1. | Docs/Dev |
| 2026-01-20 | Swept for Apache/AGPL references; remaining occurrences limited to third-party lists, fixtures, and license detection datasets. | Docs |
| 2026-01-20 | Added license index in docs README; BUSL FAQ wording aligned; documented remaining Apache headers as third-party or fixtures. | Docs |
| 2026-01-20 | Added legal index under docs/legal/README and expanded Apache/AGPL sweep; remaining references are third-party, fixtures, or historical records. | Docs |
| 2026-01-20 | Added dependency license gate in AGENTS, expanded NOTICE non-bundled infrastructure list, and updated legal dependency inventory for optional infra components. | Docs |
| 2026-01-20 | Switched Rekor cache to Valkey in compose and updated NOTICE/legal inventory to replace Redis with Valkey. | Docs |
| 2026-01-20 | Labeled Valkey as the Redis-compatible driver in Helm values and config docs (scanner events, gateway, rate limit, hardening guide). | Docs |
| 2026-01-20 | Renamed blue/green Helm cache keys to Valkey and updated Redis command references in ops/docs to Valkey CLI usage. | Docs |
| 2026-01-20 | Updated remaining Redis naming in docs (testkit fixtures, parity list, coding standards, scanning/perf notes) to Valkey where safe. | Docs |
| 2026-01-20 | Switched Rekor ops references to the v2 overlay and cleaned legacy references in Attestor design notes. | Docs |
| 2026-01-20 | Added Rekor v2 env blocks to stage/prod/airgap compose templates. | Docs |
| 2026-01-20 | Removed the legacy Rekor compose overlay and scrubbed remaining legacy references from docs/NOTICE. | Docs |
| 2026-01-20 | Removed Rekor v1 from Attestor config/code paths and set rekor-tiles image placeholders to latest for alpha envs. | Docs |
| 2026-01-20 | Removed REKOR_PREFER_TILE_PROOFS config, docs, and tests now that tiles are always used. | Docs |
| 2026-01-20 | Rejected REKOR_VERSION=V1 at config parse time (Auto/V2 only). | Docs |
| 2026-01-20 | Rejected unsupported Rekor version strings during config parsing (Auto/V2 only). | Docs |
## Decisions & Risks
- BUSL-1.1 adopted with Additional Use Grant (3 env / 999 new hash scans / no SaaS) and Change License to Apache-2.0 on 2030-01-20.
- Risk: legacy Apache-2.0 references may remain in fixtures or third-party lists; only project-license references should be updated.
- LICENSE parameters set to Licensor `stella-ops.org`, Licensed Work `Stella Ops Suite 1.0.0`, Change Date `2030-01-20`.
- Exceptions retained: SPDX license list data, third-party dependency/license fixtures (including Kubernetes CRI proto headers), package-lock dependency entries, policy allowlists/tests, sample SBOM/fixture data, historical sprint/change-log references, and Apache SPDX string fixtures in Scanner tests.
- NOTICE expanded for non-bundled infrastructure components and redistribution guidance; ensure upstream notices are mirrored when hosting third-party images.
## Next Checkpoints
- Legal doc pass complete.
- Metadata/header alignment complete.
- Final sweep for license references complete.

171
docs/implplan/SPRINT_20260120_029_AirGap_offline_bundle_contract.md
View File

@@ -1,171 +0,0 @@
# Sprint 20260120_029 – Air-Gap Offline Bundle Contract
## Topic & Scope
- Align bundle format with advisory `stella-bundle.json` specification
- Enable full offline verification with bundled TSA chain/revocation data
- Add signed verification reports for audit replay
- Ship default truststore profiles for regional compliance
- Working directory: `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/`, `src/Attestor/__Libraries/StellaOps.Attestor.Timestamping/`
- Expected evidence: Updated bundle schema, offline TSA verification tests, report signing tests
## Dependencies & Concurrency
- Upstream: SPRINT_20260119_010 (Attestor TST Integration) - provides timestamp foundation
- Upstream: SPRINT_20260118_018 (AirGap Router Integration) - provides bundle format v2
- Safe to parallelize: Tasks 001-003 can run concurrently; Task 004 depends on 002
## Documentation Prerequisites
- `docs/modules/attestor/guides/timestamp-policy.md` - Timestamp policy configuration
- `src/Attestor/__Libraries/StellaOps.Attestor.Bundle/Models/SigstoreBundle.cs` - Sigstore bundle reference
## Delivery Tracker
### TASK-029-001 - Extend BundleManifestV2 with advisory schema fields
Status: DONE
Dependency: none
Owners: Developer
Task description:
Add missing fields to `BundleManifestV2` to match the advisory `stella-bundle.json` specification:
1. Add `canonical_manifest_hash` field (sha256 of JCS-canonicalized manifest)
2. Add `timestamps[]` array with typed entries:
- `TimestampEntry` base with `type` discriminator
- `Rfc3161TimestampEntry`: `tsa_chain_paths`, `ocsp_blobs`, `crl_blobs`, `tst_base64`
- `EidasQtsTimestampEntry`: `qts_meta_path`
3. Add `rekor_proofs[]` array with `entry_body_path`, `leaf_hash`, `inclusion_proof_path`, `signed_entry_timestamp`
4. Add `subject` section with multi-algorithm digest (sha256 + optional sha512)
Files to modify:
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Models/BundleFormatV2.cs`
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Models/TimestampEntry.cs` (new)
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Models/RekorProofEntry.cs` (new)
Completion criteria:
- [x] `BundleManifestV2` includes all advisory-specified fields
- [x] JSON serialization produces output matching advisory schema
- [x] Existing bundle tests pass with backward compatibility
- [x] New unit tests verify field serialization/deserialization
### TASK-029-002 - Bundle TSA chain and revocation data for offline verification
Status: DONE
Dependency: none
Owners: Developer
Task description:
Extend the bundle builder to include TSA certificate chain, OCSP responses, and CRL data for offline verification:
1. Create `TsaChainBundler` service to collect TSA certificate chain from TST
2. Add `OcspResponseFetcher` to retrieve and cache OCSP responses for TSA certs
3. Add `CrlFetcher` to retrieve and cache CRLs for TSA certs
4. Update `BundleBuilder` to write TSA material to `tsa/chain/`, `tsa/ocsp/`, `tsa/crl/` paths
5. Update `Rfc3161Verifier` to use bundled revocation data when `--offline` flag is set
Files to modify:
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/TsaChainBundler.cs` (new)
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/OcspResponseFetcher.cs` (new)
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/CrlFetcher.cs` (new)
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/BundleBuilder.cs`
- `src/AirGap/StellaOps.AirGap.Time/Services/Rfc3161Verifier.cs`
Completion criteria:
- [x] TSA chain extracted and bundled from TST response
- [x] OCSP responses fetched and stored in bundle
- [x] CRL data fetched and stored in bundle
- [x] Offline verification uses bundled revocation data
- [x] Integration test: verify TST offline with bundled chain/OCSP/CRL
### TASK-029-003 - Implement signed verification report generation
Status: TODO
Dependency: none
Owners: Developer
Task description:
Create a service to generate DSSE-signed verification reports that can be replayed by auditors:
1. Create `IVerificationReportSigner` interface
2. Implement `DsseVerificationReportSigner` that wraps `VerificationReportPredicate` in DSSE envelope
3. Add `--signer` option to `BundleVerifyCommand` to specify verifier key
4. Write signed report to `out/verification.report.json` as DSSE envelope
5. Include `verifier.algo`, `verifier.cert`, `signed_at` in report metadata
Files to modify:
- `src/Attestor/__Libraries/StellaOps.Attestor.Core/Signing/IVerificationReportSigner.cs` (new)
- `src/Attestor/__Libraries/StellaOps.Attestor.Core/Signing/DsseVerificationReportSigner.cs` (new)
- `src/Cli/StellaOps.Cli/Commands/BundleVerifyCommand.cs`
Completion criteria:
- [ ] `IVerificationReportSigner` interface defined
- [ ] DSSE signing produces valid envelope over report predicate
- [ ] CLI `--signer` option triggers report signing
- [ ] Signed report can be verified by DSSE verifier
- [ ] Unit tests for report signing/verification round-trip
### TASK-029-004 - Ship default truststore profiles
Status: TODO
Dependency: TASK-029-002
Owners: Developer
Task description:
Create default truststore profiles for common compliance regimes:
1. Define `TrustProfile` model with roots, Rekor pubkeys, TSA roots
2. Create profile manifests:
- `global.trustprofile.json` - Sigstore public instance roots
- `eu-eidas.trustprofile.json` - EU TSL-derived roots
- `us-fips.trustprofile.json` - FIPS-compliant CA roots
- `bg-gov.trustprofile.json` - Bulgarian government PKI roots
3. Add `stella trust-profile list|apply|show` commands
4. Store profiles in `etc/trust-profiles/` or embed as resources
Files to modify:
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Models/TrustProfile.cs` (new)
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/TrustProfileLoader.cs` (new)
- `src/Cli/StellaOps.Cli/Commands/TrustProfileCommandGroup.cs` (new)
- `etc/trust-profiles/*.trustprofile.json` (new)
Completion criteria:
- [ ] `TrustProfile` model supports CA roots, Rekor keys, TSA roots
- [ ] At least 4 default profiles created with valid roots
- [ ] CLI commands to list/apply/show profiles
- [ ] Profile application sets trust anchors for session
- [ ] Documentation in `docs/modules/cli/guides/trust-profiles.md`
### TASK-029-005 - Add OCI 4 MiB inline blob size guard
Status: TODO
Dependency: none
Owners: Developer
Task description:
Enforce OCI guidance that inline JSON blobs should not exceed 4 MiB:
1. Add `MaxInlineBlobSize` constant (4 * 1024 * 1024 bytes)
2. Add size validation in `BundleBuilder.AddArtifact()`
3. Emit warning or error if artifact exceeds limit when `path` is not set
4. Large artifacts must be written to `artifacts/` directory
Files to modify:
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/BundleBuilder.cs`
- `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Validation/BundleSizeValidator.cs` (new)
Completion criteria:
- [ ] Size check added to bundle builder
- [ ] Warning logged for oversized inline artifacts
- [ ] Error thrown in strict mode for >4 MiB inline blobs
- [ ] Unit test verifies size enforcement
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created from advisory gap analysis | Planning |
| 2026-01-20 | Kickoff: started TASK-029-001 and TASK-029-002. | Planning |
| 2026-01-20 | Completed TASK-029-001 (manifest v2 fields + schema + tests); documented bundle fields in `docs/modules/airgap/README.md`. | Dev |
| 2026-01-20 | Unblocked TASK-029-002: Attestor __Libraries charter covers timestamping library; started implementation. | Dev |
| 2026-01-20 | Tests: `dotnet test src/AirGap/__Libraries/__Tests/StellaOps.AirGap.Bundle.Tests/StellaOps.AirGap.Bundle.Tests.csproj` (98 passed). | Dev |
| 2026-01-20 | Completed TASK-029-002 (TSA chain bundling + OCSP/CRL fetchers + offline RFC3161 verification + integration test); docs updated for offline verification. | Dev |
## Decisions & Risks
- Docs updated for bundle manifest v2 fields: `docs/modules/airgap/README.md`.
- Docs updated for offline timestamp verification: `docs/modules/airgap/guides/staleness-and-time.md`, `docs/modules/attestor/guides/offline-verification.md`.
- Decision: use `stella bundle verify` for advisory-aligned CLI naming.
- **Risk**: TSA chain bundling requires network access during bundle creation; mitigated by caching and pre-fetching.
- **Risk**: Default truststore profiles require ongoing maintenance as roots rotate; document rotation procedure.
## Next Checkpoints
- Code review: TASK-029-001, 029-003 (schema + signing)
- Integration test: Full offline verification with bundled TSA chain
- Documentation: Update `docs/modules/attestor/guides/offline-verification.md`

984
docs/implplan/SPRINT_20260120_030_Platform_sbom_analytics_lake.md
View File

@@ -1,984 +0,0 @@
# Sprint 20260120_030 · SBOM + Attestation Analytics Lake
## Topic & Scope
- Implement a star-schema analytics layer for SBOM and attestation data to enable executive reporting, risk dashboards, and ad-hoc analysis
- Create unified component registry with supplier/license normalization across all SBOMs
- Build component-vulnerability bridge table for efficient CVE exposure queries
- Add materialized views for dashboard performance and trend analysis
- Sequence analytics foundation (schema + base ingestion) before SBOM-lake specialization to leave headroom for release/orchestration analytics.
- Working directory: `src/Platform/`, `docs/db/`, `docs/modules/analytics/`
- Expected evidence: Schema DDL, migration scripts, unit tests, sample query library, documentation
## Dependencies & Concurrency
- Depends on existing schemas: `scanner`, `vuln` (Concelier), `vex` (Excititor), `proof_system` (Attestor)
- Can run in parallel with other Platform sprints
- Requires coordination with Scanner team for SBOM ingestion hooks
- Requires coordination with Concelier team for vulnerability feed correlation
- Downstream exposure sprints (UI/CLI) should wait until TASK-030-017/018 deliver stable endpoints.
## Documentation Prerequisites
- Database specification: `docs/db/SPECIFICATION.md`
- Triage schema reference: `docs/db/triage_schema.sql`
- SBOM determinism guide: `docs/sboms/DETERMINISM.md`
- VEX architecture: `docs/modules/vex-lens/architecture.md`
- Excititor observations: `docs/modules/excititor/vex_observations.md`
- Risk engine architecture: `docs/modules/risk-engine/architecture.md`
## Delivery Tracker
### TASK-030-001 - Create analytics schema foundation
Status: TODO
Dependency: none
Owners: Developer (Backend)
Task description:
- Create new PostgreSQL schema `analytics` with appropriate permissions
- Add schema version tracking table for migrations
- Create base types/enums for analytics domain:
- `analytics_component_type` (library, application, container, framework, operating-system, device, firmware, file)
- `analytics_license_category` (permissive, copyleft-weak, copyleft-strong, proprietary, unknown)
- `analytics_severity` (critical, high, medium, low, none, unknown)
- `analytics_attestation_type` (provenance, sbom, vex, build, scan, policy)
- Add audit columns pattern (created_at, updated_at, source_system)
Completion criteria:
- [ ] Schema `analytics` created with grants
- [ ] Version tracking table operational
- [ ] All base types/enums created
- [ ] Migration script idempotent (can re-run safely)
### TASK-030-002 - Implement unified component registry
Status: TODO
Dependency: TASK-030-001
Owners: Developer (Backend)
Task description:
- Create `analytics.components` table as the canonical component registry:
```sql
CREATE TABLE analytics.components (
component_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
purl TEXT NOT NULL, -- Package URL (canonical identifier)
purl_type TEXT NOT NULL, -- Extracted: maven, npm, pypi, etc.
purl_namespace TEXT, -- Extracted: group/org
purl_name TEXT NOT NULL, -- Extracted: package name
purl_version TEXT, -- Extracted: version
hash_sha256 TEXT, -- Content hash for deduplication
name TEXT NOT NULL, -- Display name
version TEXT, -- Display version
component_type analytics_component_type NOT NULL DEFAULT 'library',
supplier TEXT, -- Vendor/maintainer
supplier_normalized TEXT, -- Normalized supplier name
license_declared TEXT, -- Raw license string
license_concluded TEXT, -- SPDX expression
license_category analytics_license_category DEFAULT 'unknown',
description TEXT,
cpe TEXT, -- CPE identifier if available
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
last_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
sbom_count INT NOT NULL DEFAULT 1, -- Number of SBOMs containing this
artifact_count INT NOT NULL DEFAULT 1, -- Number of artifacts containing this
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (purl, hash_sha256)
);
```
- Create indexes for common query patterns:
- `ix_components_purl` on (purl)
- `ix_components_supplier` on (supplier_normalized)
- `ix_components_license` on (license_category, license_concluded)
- `ix_components_type` on (component_type)
- `ix_components_purl_type` on (purl_type)
- `ix_components_hash` on (hash_sha256) WHERE hash_sha256 IS NOT NULL
- Implement supplier normalization function (lowercase, trim, common aliases)
- Implement license categorization function (SPDX expression -> category)
Completion criteria:
- [ ] Table created with all columns and constraints
- [ ] Indexes created and verified with EXPLAIN ANALYZE
- [ ] Supplier normalization function tested
- [ ] License categorization covers common licenses
- [ ] Upsert logic handles duplicates correctly
### TASK-030-003 - Implement artifacts analytics table
Status: TODO
Dependency: TASK-030-001
Owners: Developer (Backend)
Task description:
- Create `analytics.artifacts` table for container images and applications:
```sql
CREATE TABLE analytics.artifacts (
artifact_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
artifact_type TEXT NOT NULL, -- container, application, library, firmware
name TEXT NOT NULL, -- Image/app name
version TEXT, -- Tag/version
digest TEXT, -- SHA256 digest
purl TEXT, -- Package URL if applicable
source_repo TEXT, -- Git repo URL
source_ref TEXT, -- Git ref (branch/tag/commit)
registry TEXT, -- Container registry
environment TEXT, -- dev, stage, prod
team TEXT, -- Owning team
service TEXT, -- Service name
deployed_at TIMESTAMPTZ, -- Last deployment timestamp
sbom_digest TEXT, -- SHA256 of associated SBOM
sbom_format TEXT, -- cyclonedx, spdx
sbom_spec_version TEXT, -- 1.7, 3.0, etc.
component_count INT DEFAULT 0, -- Number of components in SBOM
vulnerability_count INT DEFAULT 0, -- Total vulns (pre-VEX)
critical_count INT DEFAULT 0, -- Critical severity vulns
high_count INT DEFAULT 0, -- High severity vulns
provenance_attested BOOLEAN DEFAULT FALSE,
slsa_level INT, -- 0-4
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (digest)
);
```
- Create indexes:
- `ix_artifacts_name_version` on (name, version)
- `ix_artifacts_environment` on (environment)
- `ix_artifacts_team` on (team)
- `ix_artifacts_deployed` on (deployed_at DESC)
- `ix_artifacts_digest` on (digest)
Completion criteria:
- [ ] Table created with all columns
- [ ] Indexes created
- [ ] Vulnerability counts populated on SBOM ingest
- [ ] Environment/team metadata captured
### TASK-030-004 - Implement artifact-component bridge table
Status: TODO
Dependency: TASK-030-002, TASK-030-003
Owners: Developer (Backend)
Task description:
- Create `analytics.artifact_components` bridge table:
```sql
CREATE TABLE analytics.artifact_components (
artifact_id UUID NOT NULL REFERENCES analytics.artifacts(artifact_id) ON DELETE CASCADE,
component_id UUID NOT NULL REFERENCES analytics.components(component_id) ON DELETE CASCADE,
bom_ref TEXT, -- Original bom-ref for round-trips
scope TEXT, -- required, optional, excluded
dependency_path TEXT[], -- Path from root (for transitive deps)
depth INT DEFAULT 0, -- Dependency depth (0=direct)
introduced_via TEXT, -- Direct dependency that introduced this
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (artifact_id, component_id)
);
```
- Create indexes:
- `ix_artifact_components_component` on (component_id)
- `ix_artifact_components_depth` on (depth)
Completion criteria:
- [ ] Bridge table created
- [ ] Dependency path tracking works for transitive deps
- [ ] Depth calculation accurate
### TASK-030-005 - Implement component-vulnerability bridge table
Status: TODO
Dependency: TASK-030-002
Owners: Developer (Backend)
Task description:
- Create `analytics.component_vulns` bridge table:
```sql
CREATE TABLE analytics.component_vulns (
component_id UUID NOT NULL REFERENCES analytics.components(component_id) ON DELETE CASCADE,
vuln_id TEXT NOT NULL, -- CVE-YYYY-NNNNN or GHSA-xxxx
source TEXT NOT NULL, -- nvd, ghsa, osv, vendor
severity analytics_severity NOT NULL,
cvss_score NUMERIC(3,1), -- 0.0-10.0
cvss_vector TEXT, -- CVSS vector string
epss_score NUMERIC(5,4), -- 0.0000-1.0000
kev_listed BOOLEAN DEFAULT FALSE, -- CISA KEV
affects BOOLEAN NOT NULL DEFAULT TRUE, -- Does this vuln affect this component?
affected_versions TEXT, -- Version range expression
fixed_version TEXT, -- First fixed version
fix_available BOOLEAN DEFAULT FALSE,
introduced_via TEXT, -- How vulnerability was introduced
published_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (component_id, vuln_id)
);
```
- Create indexes:
- `ix_component_vulns_vuln` on (vuln_id)
- `ix_component_vulns_severity` on (severity, cvss_score DESC)
- `ix_component_vulns_fixable` on (fix_available) WHERE fix_available = TRUE
- `ix_component_vulns_kev` on (kev_listed) WHERE kev_listed = TRUE
Completion criteria:
- [ ] Bridge table created with all columns
- [ ] KEV flag populated from CISA feed
- [ ] EPSS scores populated
- [ ] Fix availability detected
### TASK-030-006 - Implement attestations analytics table
Status: TODO
Dependency: TASK-030-003
Owners: Developer (Backend)
Task description:
- Create `analytics.attestations` table:
```sql
CREATE TABLE analytics.attestations (
attestation_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
artifact_id UUID REFERENCES analytics.artifacts(artifact_id) ON DELETE SET NULL,
predicate_type analytics_attestation_type NOT NULL,
predicate_uri TEXT NOT NULL, -- Full predicate type URI
issuer TEXT, -- Who signed
issuer_normalized TEXT, -- Normalized issuer
builder_id TEXT, -- Build system identifier
slsa_level INT, -- SLSA conformance level
dsse_payload_hash TEXT NOT NULL, -- SHA256 of payload
dsse_sig_algorithm TEXT, -- Signature algorithm
rekor_log_id TEXT, -- Transparency log ID
rekor_log_index BIGINT, -- Log index
statement_time TIMESTAMPTZ, -- When statement was made
verified BOOLEAN DEFAULT FALSE, -- Signature verified
verification_time TIMESTAMPTZ,
materials_hash TEXT, -- Hash of build materials
source_uri TEXT, -- Source code URI
workflow_ref TEXT, -- CI workflow reference
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (dsse_payload_hash)
);
```
- Create indexes:
- `ix_attestations_artifact` on (artifact_id)
- `ix_attestations_type` on (predicate_type)
- `ix_attestations_issuer` on (issuer_normalized)
- `ix_attestations_rekor` on (rekor_log_id) WHERE rekor_log_id IS NOT NULL
Completion criteria:
- [ ] Table created
- [ ] Rekor linkage works
- [ ] SLSA level extraction accurate
### TASK-030-007 - Implement VEX overrides analytics table
Status: TODO
Dependency: TASK-030-005, TASK-030-006
Owners: Developer (Backend)
Task description:
- Create `analytics.vex_overrides` table:
```sql
CREATE TABLE analytics.vex_overrides (
override_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
attestation_id UUID REFERENCES analytics.attestations(attestation_id) ON DELETE SET NULL,
artifact_id UUID REFERENCES analytics.artifacts(artifact_id) ON DELETE CASCADE,
vuln_id TEXT NOT NULL,
component_purl TEXT, -- Optional: specific component
status TEXT NOT NULL, -- not_affected, affected, fixed, under_investigation
justification TEXT, -- Justification category
justification_detail TEXT, -- Human-readable detail
impact TEXT, -- Impact statement
action_statement TEXT, -- Recommended action
operator_id TEXT, -- Who made the decision
confidence NUMERIC(3,2), -- 0.00-1.00
valid_from TIMESTAMPTZ NOT NULL DEFAULT now(),
valid_until TIMESTAMPTZ, -- Expiration
last_reviewed TIMESTAMPTZ,
review_count INT DEFAULT 1,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```
- Create indexes:
- `ix_vex_overrides_artifact_vuln` on (artifact_id, vuln_id)
- `ix_vex_overrides_vuln` on (vuln_id)
- `ix_vex_overrides_status` on (status)
- `ix_vex_overrides_active` on (artifact_id, vuln_id) WHERE valid_until IS NULL OR valid_until > now()
Completion criteria:
- [ ] Table created
- [ ] Expiration logic works
- [ ] Confidence scoring populated
### TASK-030-008 - Implement raw payload audit tables
Status: TODO
Dependency: TASK-030-001
Owners: Developer (Backend)
Task description:
- Create `analytics.raw_sboms` table for audit trail:
```sql
CREATE TABLE analytics.raw_sboms (
sbom_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
artifact_id UUID REFERENCES analytics.artifacts(artifact_id) ON DELETE SET NULL,
format TEXT NOT NULL, -- cyclonedx, spdx
spec_version TEXT NOT NULL, -- 1.7, 3.0.1, etc.
content_hash TEXT NOT NULL UNIQUE, -- SHA256 of raw content
content_size BIGINT NOT NULL,
storage_uri TEXT NOT NULL, -- Object storage path
ingest_version TEXT NOT NULL, -- Pipeline version
schema_version TEXT NOT NULL, -- Schema version at ingest
ingested_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```
- Create `analytics.raw_attestations` table:
```sql
CREATE TABLE analytics.raw_attestations (
raw_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
attestation_id UUID REFERENCES analytics.attestations(attestation_id) ON DELETE SET NULL,
content_hash TEXT NOT NULL UNIQUE,
content_size BIGINT NOT NULL,
storage_uri TEXT NOT NULL,
ingest_version TEXT NOT NULL,
schema_version TEXT NOT NULL,
ingested_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```
Completion criteria:
- [ ] Raw SBOM storage operational
- [ ] Raw attestation storage operational
- [ ] Hash-based deduplication works
- [ ] Storage URIs resolve correctly
### TASK-030-009 - Implement time-series rollup tables
Status: TODO
Dependency: TASK-030-003, TASK-030-005
Owners: Developer (Backend)
Task description:
- Create `analytics.daily_vulnerability_counts` rollup:
```sql
CREATE TABLE analytics.daily_vulnerability_counts (
snapshot_date DATE NOT NULL,
environment TEXT NOT NULL,
team TEXT,
severity analytics_severity NOT NULL,
total_vulns INT NOT NULL,
fixable_vulns INT NOT NULL,
vex_mitigated INT NOT NULL,
kev_vulns INT NOT NULL,
unique_cves INT NOT NULL,
affected_artifacts INT NOT NULL,
affected_components INT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (snapshot_date, environment, COALESCE(team, ''), severity)
);
```
- Create `analytics.daily_component_counts` rollup:
```sql
CREATE TABLE analytics.daily_component_counts (
snapshot_date DATE NOT NULL,
environment TEXT NOT NULL,
team TEXT,
license_category analytics_license_category NOT NULL,
component_type analytics_component_type NOT NULL,
total_components INT NOT NULL,
unique_suppliers INT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (snapshot_date, environment, COALESCE(team, ''), license_category, component_type)
);
```
- Create daily rollup job (PostgreSQL function + pg_cron or Scheduler task)
Completion criteria:
- [ ] Rollup tables created
- [ ] Daily job populates correctly
- [ ] Historical backfill works
- [ ] 90-day retention policy applied
### TASK-030-010 - Implement supplier concentration materialized view
Status: TODO
Dependency: TASK-030-002, TASK-030-004
Owners: Developer (Backend)
Task description:
- Create `analytics.mv_supplier_concentration`:
```sql
CREATE MATERIALIZED VIEW analytics.mv_supplier_concentration AS
SELECT
c.supplier_normalized AS supplier,
COUNT(DISTINCT c.component_id) AS component_count,
COUNT(DISTINCT ac.artifact_id) AS artifact_count,
COUNT(DISTINCT a.team) AS team_count,
ARRAY_AGG(DISTINCT a.environment) FILTER (WHERE a.environment IS NOT NULL) AS environments,
SUM(CASE WHEN cv.severity = 'critical' THEN 1 ELSE 0 END) AS critical_vuln_count,
SUM(CASE WHEN cv.severity = 'high' THEN 1 ELSE 0 END) AS high_vuln_count,
MAX(c.last_seen_at) AS last_seen_at
FROM analytics.components c
JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
LEFT JOIN analytics.component_vulns cv ON cv.component_id = c.component_id AND cv.affects = TRUE
WHERE c.supplier_normalized IS NOT NULL
GROUP BY c.supplier_normalized
WITH DATA;
```
- Create unique index for concurrent refresh
- Create refresh job (daily)
Completion criteria:
- [ ] Materialized view created
- [ ] Concurrent refresh works
- [ ] Query performance < 100ms for top-20
### TASK-030-011 - Implement license distribution materialized view
Status: TODO
Dependency: TASK-030-002
Owners: Developer (Backend)
Task description:
- Create `analytics.mv_license_distribution`:
```sql
CREATE MATERIALIZED VIEW analytics.mv_license_distribution AS
SELECT
c.license_concluded,
c.license_category,
COUNT(*) AS component_count,
COUNT(DISTINCT ac.artifact_id) AS artifact_count,
ARRAY_AGG(DISTINCT c.purl_type) AS ecosystems
FROM analytics.components c
LEFT JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
GROUP BY c.license_concluded, c.license_category
WITH DATA;
```
- Create unique index
- Create refresh job
Completion criteria:
- [ ] View created
- [ ] Refresh operational
- [ ] License category breakdown accurate
### TASK-030-012 - Implement CVE exposure adjusted by VEX materialized view
Status: TODO
Dependency: TASK-030-005, TASK-030-007
Owners: Developer (Backend)
Task description:
- Create `analytics.mv_vuln_exposure`:
```sql
CREATE MATERIALIZED VIEW analytics.mv_vuln_exposure AS
SELECT
cv.vuln_id,
cv.severity,
cv.cvss_score,
cv.epss_score,
cv.kev_listed,
cv.fix_available,
COUNT(DISTINCT cv.component_id) AS raw_component_count,
COUNT(DISTINCT ac.artifact_id) AS raw_artifact_count,
COUNT(DISTINCT cv.component_id) FILTER (
WHERE NOT EXISTS (
SELECT 1 FROM analytics.vex_overrides vo
WHERE vo.artifact_id = ac.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND (vo.valid_until IS NULL OR vo.valid_until > now())
)
) AS effective_component_count,
COUNT(DISTINCT ac.artifact_id) FILTER (
WHERE NOT EXISTS (
SELECT 1 FROM analytics.vex_overrides vo
WHERE vo.artifact_id = ac.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND (vo.valid_until IS NULL OR vo.valid_until > now())
)
) AS effective_artifact_count
FROM analytics.component_vulns cv
JOIN analytics.artifact_components ac ON ac.component_id = cv.component_id
WHERE cv.affects = TRUE
GROUP BY cv.vuln_id, cv.severity, cv.cvss_score, cv.epss_score, cv.kev_listed, cv.fix_available
WITH DATA;
```
Completion criteria:
- [ ] View created
- [ ] VEX adjustment logic correct
- [ ] Performance acceptable for refresh
### TASK-030-013 - Implement attestation coverage materialized view
Status: TODO
Dependency: TASK-030-003, TASK-030-006
Owners: Developer (Backend)
Task description:
- Create `analytics.mv_attestation_coverage`:
```sql
CREATE MATERIALIZED VIEW analytics.mv_attestation_coverage AS
SELECT
a.environment,
a.team,
COUNT(*) AS total_artifacts,
COUNT(*) FILTER (WHERE a.provenance_attested = TRUE) AS with_provenance,
COUNT(*) FILTER (WHERE EXISTS (
SELECT 1 FROM analytics.attestations att
WHERE att.artifact_id = a.artifact_id AND att.predicate_type = 'sbom'
)) AS with_sbom_attestation,
COUNT(*) FILTER (WHERE EXISTS (
SELECT 1 FROM analytics.attestations att
WHERE att.artifact_id = a.artifact_id AND att.predicate_type = 'vex'
)) AS with_vex_attestation,
COUNT(*) FILTER (WHERE a.slsa_level >= 2) AS slsa_level_2_plus,
COUNT(*) FILTER (WHERE a.slsa_level >= 3) AS slsa_level_3_plus,
ROUND(100.0 * COUNT(*) FILTER (WHERE a.provenance_attested = TRUE) / NULLIF(COUNT(*), 0), 1) AS provenance_pct,
ROUND(100.0 * COUNT(*) FILTER (WHERE a.slsa_level >= 2) / NULLIF(COUNT(*), 0), 1) AS slsa2_pct
FROM analytics.artifacts a
GROUP BY a.environment, a.team
WITH DATA;
```
Completion criteria:
- [ ] View created
- [ ] Coverage percentages accurate
- [ ] Grouped by env/team correctly
### TASK-030-014 - Implement SBOM ingestion pipeline hook
Status: TODO
Dependency: TASK-030-002, TASK-030-003, TASK-030-004, TASK-030-008
Owners: Developer (Backend)
Task description:
- Create `AnalyticsIngestionService` in `src/Platform/StellaOps.Platform.Analytics/`:
- Subscribe to SBOM ingestion events from Scanner
- Normalize components and upsert to `analytics.components`
- Create/update `analytics.artifacts` record
- Populate `analytics.artifact_components` bridge
- Store raw SBOM in `analytics.raw_sboms`
- Implement idempotent upserts using `ON CONFLICT DO UPDATE`
- Handle both CycloneDX and SPDX formats
- Extract supplier from component metadata or infer from purl namespace
Completion criteria:
- [ ] Service created and registered
- [ ] CycloneDX ingestion works
- [ ] SPDX ingestion works
- [ ] Deduplication by purl+hash works
- [ ] Raw payload stored
### TASK-030-015 - Implement vulnerability correlation pipeline
Status: TODO
Dependency: TASK-030-005, TASK-030-014
Owners: Developer (Backend)
Task description:
- Create `VulnerabilityCorrelationService`:
- On component upsert, query Concelier for matching vulnerabilities
- Populate `analytics.component_vulns` with affected vulns
- Handle version range matching
- Update artifact vulnerability counts
- Integrate EPSS scores from feed
- Integrate KEV flags from CISA feed
Completion criteria:
- [ ] Correlation service operational
- [ ] Version range matching accurate
- [ ] EPSS/KEV populated
- [ ] Artifact counts updated
### TASK-030-016 - Implement attestation ingestion pipeline
Status: TODO
Dependency: TASK-030-006, TASK-030-008
Owners: Developer (Backend)
Task description:
- Create `AttestationIngestionService`:
- Subscribe to attestation events from Attestor
- Parse DSSE envelope and extract predicate
- Create `analytics.attestations` record
- Store raw attestation in `analytics.raw_attestations`
- Update artifact `provenance_attested` and `slsa_level`
- Handle VEX attestations -> create `analytics.vex_overrides`
Completion criteria:
- [ ] Service created
- [ ] Provenance predicates parsed
- [ ] VEX predicates -> overrides
- [ ] SLSA level extraction works
### TASK-030-017 - Create stored procedures for Day-1 queries
Status: TODO
Dependency: TASK-030-010, TASK-030-011, TASK-030-012, TASK-030-013
Owners: Developer (Backend)
Task description:
- Create stored procedures for executive dashboard queries:
- `analytics.sp_top_suppliers(limit INT)` - Top supplier concentration
- `analytics.sp_license_heatmap()` - License distribution
- `analytics.sp_vuln_exposure(env TEXT, min_severity TEXT)` - CVE exposure by VEX
- `analytics.sp_fixable_backlog(env TEXT)` - Fixable vulnerabilities
- `analytics.sp_attestation_gaps(env TEXT)` - Attestation coverage gaps
- `analytics.sp_mttr_by_severity(days INT)` - Mean time to remediate
- Return JSON for easy API consumption
Completion criteria:
- [ ] All 6 procedures created
- [ ] Return JSON format
- [ ] Query performance < 500ms each
- [ ] Documentation in code comments
### TASK-030-018 - Create Platform API endpoints for analytics
Status: TODO
Dependency: TASK-030-017
Owners: Developer (Backend)
Task description:
- Add analytics endpoints to `src/Platform/StellaOps.Platform.WebService/`:
- `GET /api/analytics/suppliers` - Supplier concentration
- `GET /api/analytics/licenses` - License distribution
- `GET /api/analytics/vulnerabilities` - CVE exposure
- `GET /api/analytics/backlog` - Fixable backlog
- `GET /api/analytics/attestation-coverage` - Attestation gaps
- `GET /api/analytics/trends/vulnerabilities` - Time-series vuln trends
- `GET /api/analytics/trends/components` - Time-series component trends
- Add caching layer (5-minute TTL for expensive queries)
- Add OpenAPI documentation
Completion criteria:
- [ ] All endpoints implemented
- [ ] Caching operational
- [ ] OpenAPI spec updated
- [ ] Authorization integrated
### TASK-030-019 - Unit tests for analytics schema and services
Status: TODO
Dependency: TASK-030-014, TASK-030-015, TASK-030-016
Owners: QA
Task description:
- Create test project `StellaOps.Platform.Analytics.Tests`
- Tests for:
- Component normalization (supplier, license)
- Purl parsing and extraction
- Deduplication logic
- Vulnerability correlation
- Attestation parsing
- Materialized view refresh
- Stored procedure correctness
- Use frozen fixtures for determinism
Completion criteria:
- [ ] Test project created
- [ ] >90% code coverage on services
- [ ] All stored procedures tested
- [ ] Deterministic fixtures used
### TASK-030-020 - Documentation and architecture dossier
Status: TODO
Dependency: TASK-030-018
Owners: Documentation
Task description:
- Create `docs/modules/analytics/README.md`:
- Overview and purpose
- Schema diagram
- Data flow diagram
- Query examples
- Create `docs/modules/analytics/architecture.md`:
- Design decisions
- Normalization rules
- Refresh schedules
- Performance considerations
- Create `docs/db/analytics_schema.sql`:
- Complete DDL for reference
- Includes indexes and constraints
- Update `docs/db/SPECIFICATION.md` with analytics schema
- Create `docs/modules/analytics/queries.md`:
- All Day-1 queries with explanations
- Performance tips
Completion criteria:
- [ ] README created
- [ ] Architecture dossier complete
- [ ] Schema DDL documented
- [ ] Query library documented
- [ ] Diagrams included
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created from SBOM Analytics Lake advisory gap analysis | Planning |
| 2026-01-20 | Kickoff: started TASK-030-001 (analytics schema foundation). | Planning |
| 2026-01-20 | Deferred TASK-030-001; implementation not started yet. | Planning |
| 2026-01-20 | Sequenced analytics foundation before SBOM lake specialization; noted downstream UI/CLI dependencies. | Planning |
## Decisions & Risks
### Decisions
1. **Star schema vs normalized**: Chose star schema for analytics query performance over normalized form
2. **Separate schema**: Analytics in dedicated `analytics` schema to isolate from operational tables
3. **Materialized views**: Use materialized views with scheduled refresh rather than real-time aggregation
4. **PURL as canonical ID**: Package URL is the primary identifier; hash as secondary for deduplication
5. **Raw payload storage**: Keep raw SBOM/attestation JSON for audit trail and reprocessing
6. **Supplier normalization**: Apply lowercase + trim + alias mapping for consistent grouping
7. **License categorization**: Map SPDX expressions to 5 categories (permissive, weak-copyleft, strong-copyleft, proprietary, unknown)
8. **Time-series granularity**: Daily rollups with 90-day retention; older data archived to cold storage
### Risks
1. **Risk**: Large component registry may impact upsert performance
- Mitigation: Batch inserts, partitioning by purl_type if needed
2. **Risk**: Materialized view refresh may be slow for large datasets
- Mitigation: Concurrent refresh, off-peak scheduling, incremental refresh patterns
3. **Risk**: Vulnerability correlation may miss edge cases in version matching
- Mitigation: Use Concelier's proven version range logic; add fallback fuzzy matching
4. **Risk**: Supplier normalization may create incorrect groupings
- Mitigation: Start with conservative rules; add manual alias table for corrections
5. **Risk**: Schema changes may require data migration
- Mitigation: Version tracking table; additive changes preferred
### Dependencies on Other Teams
- **Scanner**: SBOM ingestion event integration
- **Concelier**: Vulnerability feed access for correlation
- **Excititor**: VEX observation synchronization
- **Attestor**: Attestation event integration
## Next Checkpoints
- TASK-030-007 complete: Core schema operational
- TASK-030-013 complete: Materialized views ready
- TASK-030-016 complete: Ingestion pipelines operational
- TASK-030-018 complete: API endpoints available
- TASK-030-020 complete: Documentation published
## Appendix A: Complete Schema DDL
```sql
-- Stella Ops Analytics Schema (PostgreSQL)
-- Version: 1.0.0
-- Sprint: 20260120_030
BEGIN;
-- Extensions
CREATE EXTENSION IF NOT EXISTS pgcrypto;
-- Schema
CREATE SCHEMA IF NOT EXISTS analytics;
-- Version tracking
CREATE TABLE IF NOT EXISTS analytics.schema_version (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),
description TEXT
);
INSERT INTO analytics.schema_version (version, description)
VALUES ('1.0.0', 'Initial analytics schema')
ON CONFLICT DO NOTHING;
-- Enums
DO $$
BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_type WHERE typname = 'analytics_component_type') THEN
CREATE TYPE analytics_component_type AS ENUM (
'library', 'application', 'container', 'framework',
'operating-system', 'device', 'firmware', 'file'
);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_type WHERE typname = 'analytics_license_category') THEN
CREATE TYPE analytics_license_category AS ENUM (
'permissive', 'copyleft-weak', 'copyleft-strong', 'proprietary', 'unknown'
);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_type WHERE typname = 'analytics_severity') THEN
CREATE TYPE analytics_severity AS ENUM (
'critical', 'high', 'medium', 'low', 'none', 'unknown'
);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_type WHERE typname = 'analytics_attestation_type') THEN
CREATE TYPE analytics_attestation_type AS ENUM (
'provenance', 'sbom', 'vex', 'build', 'scan', 'policy'
);
END IF;
END $$;
-- Core Tables (see TASK-030-002 through TASK-030-009 for full definitions)
-- [Tables defined inline in tasks above]
-- Normalization Functions
CREATE OR REPLACE FUNCTION analytics.normalize_supplier(raw_supplier TEXT)
RETURNS TEXT AS $$
BEGIN
IF raw_supplier IS NULL THEN
RETURN NULL;
END IF;
RETURN LOWER(TRIM(
REGEXP_REPLACE(
REGEXP_REPLACE(raw_supplier, '\s+(Inc\.?|LLC|Ltd\.?|Corp\.?|GmbH|B\.V\.)$', '', 'i'),
'\s+', ' ', 'g'
)
));
END;
$$ LANGUAGE plpgsql IMMUTABLE;
CREATE OR REPLACE FUNCTION analytics.categorize_license(license_expr TEXT)
RETURNS analytics_license_category AS $$
BEGIN
IF license_expr IS NULL OR license_expr = '' THEN
RETURN 'unknown';
END IF;
-- Strong copyleft
IF license_expr ~* '(GPL-[23]|AGPL|LGPL-[23]|SSPL|OSL|EUPL)' AND
license_expr !~* 'WITH.*exception' THEN
RETURN 'copyleft-strong';
END IF;
-- Weak copyleft
IF license_expr ~* '(LGPL|MPL|EPL|CPL|CDDL|Artistic)' THEN
RETURN 'copyleft-weak';
END IF;
-- Permissive
IF license_expr ~* '(MIT|Apache|BSD|ISC|Zlib|Unlicense|CC0|WTFPL|0BSD)' THEN
RETURN 'permissive';
END IF;
-- Proprietary indicators
IF license_expr ~* '(proprietary|commercial|all.rights.reserved)' THEN
RETURN 'proprietary';
END IF;
RETURN 'unknown';
END;
$$ LANGUAGE plpgsql IMMUTABLE;
COMMIT;
```
## Appendix B: Day-1 Query Library
### Query 1: Top Supplier Concentration (Supply Chain Risk)
```sql
-- Top 20 suppliers by component count with vulnerability exposure
SELECT
supplier,
component_count,
artifact_count,
team_count,
critical_vuln_count,
high_vuln_count,
environments
FROM analytics.mv_supplier_concentration
ORDER BY component_count DESC
LIMIT 20;
```
### Query 2: License Risk Heatmap
```sql
-- Components by license category
SELECT
license_category,
license_concluded,
component_count,
artifact_count,
ecosystems
FROM analytics.mv_license_distribution
ORDER BY component_count DESC;
```
### Query 3: CVE Exposure Adjusted by VEX
```sql
-- Vulnerabilities with effective (post-VEX) impact
SELECT
vuln_id,
severity,
cvss_score,
epss_score,
kev_listed,
fix_available,
raw_component_count,
raw_artifact_count,
effective_component_count,
effective_artifact_count,
raw_artifact_count - effective_artifact_count AS vex_mitigated_artifacts
FROM analytics.mv_vuln_exposure
WHERE effective_artifact_count > 0
ORDER BY
CASE severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
WHEN 'medium' THEN 3
ELSE 4
END,
effective_artifact_count DESC
LIMIT 50;
```
### Query 4: Fixable Backlog
```sql
-- Vulnerabilities with available fixes, grouped by service
SELECT
a.name AS service,
a.environment,
c.name AS component,
c.version,
cv.vuln_id,
cv.severity,
cv.fixed_version
FROM analytics.component_vulns cv
JOIN analytics.components c ON c.component_id = cv.component_id
JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
LEFT JOIN analytics.vex_overrides vo ON vo.artifact_id = a.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND (vo.valid_until IS NULL OR vo.valid_until > now())
WHERE cv.affects = TRUE
AND cv.fix_available = TRUE
AND vo.override_id IS NULL -- Not mitigated by VEX
ORDER BY
CASE cv.severity
WHEN 'critical' THEN 1
WHEN 'high' THEN 2
ELSE 3
END,
a.name;
```
### Query 5: Build Integrity / Attestation Coverage
```sql
-- Attestation gaps by environment
SELECT
environment,
team,
total_artifacts,
with_provenance,
provenance_pct,
slsa_level_2_plus,
slsa2_pct,
total_artifacts - with_provenance AS missing_provenance
FROM analytics.mv_attestation_coverage
ORDER BY provenance_pct ASC;
```
### Query 6: Vulnerability Trends (30 days)
```sql
-- Daily vulnerability counts over last 30 days
SELECT
snapshot_date,
environment,
severity,
total_vulns,
fixable_vulns,
vex_mitigated,
total_vulns - vex_mitigated AS net_exposure
FROM analytics.daily_vulnerability_counts
WHERE snapshot_date >= CURRENT_DATE - INTERVAL '30 days'
ORDER BY snapshot_date, environment, severity;
```

132
docs/implplan/SPRINT_20260120_031_FE_sbom_analytics_console.md
View File

@@ -1,132 +0,0 @@
# Sprint 20260120_031 - SBOM Analytics Console
## Topic & Scope
- Deliver a first-class UI for SBOM analytics lake outputs (suppliers, licenses, vulnerabilities, attestations, trends).
- Provide filtering and drilldowns aligned to analytics API capabilities.
- Working directory: `src/Web/`.
- Expected evidence: UI routes/components, web API client, unit/e2e tests, docs updates.
## Dependencies & Concurrency
- Depends on `docs/implplan/SPRINT_20260120_030_Platform_sbom_analytics_lake.md` (TASK-030-017, TASK-030-018, TASK-030-020).
- Coordinate with Platform team on auth scopes and caching behavior.
- Can run in parallel with other frontend work once analytics endpoints are stable.
- CLI exposure tracked in `docs/implplan/SPRINT_20260120_032_Cli_sbom_analytics_cli.md` for parity planning.
## Documentation Prerequisites
- `src/Web/StellaOps.Web/AGENTS.md`
- `docs/modules/analytics/README.md`
- `docs/modules/analytics/architecture.md`
- `docs/modules/analytics/queries.md`
- `docs/modules/cli/cli-vs-ui-parity.md`
## Delivery Tracker
### TASK-031-001 - UI shell, routing, and filter state
Status: TODO
Dependency: none
Owners: Developer (Frontend)
Task description:
- Add an "Analytics" navigation entry with an "SBOM Lake" route (Analytics > SBOM Lake).
- Structure navigation so future analytics modules can be added under Analytics.
- Build a page shell with filter controls (environment, time range, severity).
- Persist filter state in query params and define loading/empty/error UI states.
Completion criteria:
- [ ] Route reachable via nav and guarded by existing permission patterns
- [ ] Filter state round-trips via URL parameters
- [ ] Loading/empty/error states follow existing UI conventions
- [ ] Base shell renders with placeholder panels
### TASK-031-002 - Web API client for analytics endpoints
Status: TODO
Dependency: TASK-031-001
Owners: Developer (Frontend)
Task description:
- Add a typed analytics client under `src/Web/StellaOps.Web/src/app/core/api/`.
- Implement calls for suppliers, licenses, vulnerabilities, backlog, attestation coverage, and trend endpoints.
- Normalize error handling and align response shapes with existing clients.
Completion criteria:
- [ ] Client implemented for all analytics endpoints
- [ ] Errors mapped to standard UI error model
- [ ] Unit tests cover response mapping and error handling
### TASK-031-003 - Overview dashboard panels
Status: TODO
Dependency: TASK-031-002
Owners: Developer (Frontend)
Task description:
- Build summary tiles and charts for supplier concentration, license distribution, vulnerability exposure, and attestation coverage.
- Bind panels to filter state and render empty-data messaging.
- Use existing charting and card components to align visual language.
Completion criteria:
- [ ] All four panels render with live data
- [ ] Filter changes update panels consistently
- [ ] Empty-data messaging is clear and consistent
### TASK-031-004 - Drilldowns, trends, and exports
Status: TODO
Dependency: TASK-031-003
Owners: Developer (Frontend)
Task description:
- Add drilldown tables for fixable backlog and top components.
- Implement vulnerability and component trend views with selectable time ranges.
- Provide CSV export using existing export patterns (or a new shared utility if missing).
Completion criteria:
- [ ] Drilldown tables support sorting and filtering
- [ ] Trend views load within acceptable UI latency
- [ ] CSV export produces deterministic, ordered output
### TASK-031-005 - Frontend tests and QA coverage
Status: TODO
Dependency: TASK-031-004
Owners: QA
Task description:
- Add unit tests for the analytics API client and dashboard components.
- Add one e2e or integration test for route load and filter behavior.
- Use frozen fixtures for deterministic results.
Completion criteria:
- [ ] Unit tests cover client mappings and component rendering
- [ ] e2e/integration test exercises filter state and data loading
- [ ] Deterministic fixtures checked in
### TASK-031-006 - Documentation updates for analytics console
Status: TODO
Dependency: TASK-031-004
Owners: Documentation
Task description:
- Add console usage section to `docs/modules/analytics/README.md`.
- Create `docs/modules/analytics/console.md` with screenshots/flows if applicable.
- Update parity expectations in `docs/modules/cli/cli-vs-ui-parity.md`.
Completion criteria:
- [ ] Console usage documented with filters and panels
- [ ] New console guide created and linked
- [ ] Parity doc updated to reflect new UI surface
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created to plan UI exposure for SBOM analytics lake. | Planning |
| 2026-01-20 | Clarified Analytics > SBOM Lake navigation hierarchy. | Planning |
| 2026-01-20 | Kickoff: started TASK-031-001 (UI shell + routing). | Planning |
| 2026-01-20 | Deferred TASK-031-001; implementation not started yet. | Planning |
## Decisions & Risks
- Cross-module edits: allow updates under `docs/modules/analytics/` and `docs/modules/cli/` for documentation and parity notes.
- Risk: API latency or missing metrics blocks UI rollouts; mitigate with feature gating and placeholder states.
- Risk: Inconsistent definitions across panels; mitigate by linking UI labels to analytics query docs.
## Next Checkpoints
- TASK-031-002 complete: API client ready.
- TASK-031-004 complete: UI drilldowns and exports available.
- TASK-031-006 complete: Docs published.

112
docs/implplan/SPRINT_20260120_032_Cli_sbom_analytics_cli.md
View File

@@ -1,112 +0,0 @@
# Sprint 20260120_032 - SBOM Analytics CLI
## Topic & Scope
- Expose SBOM analytics lake insights via the Stella Ops CLI.
- Provide filters and output formats that match the API and UI views.
- Working directory: `src/Cli/`.
- Expected evidence: CLI commands, output fixtures, unit tests, docs updates.
## Dependencies & Concurrency
- Depends on `docs/implplan/SPRINT_20260120_030_Platform_sbom_analytics_lake.md` (TASK-030-017, TASK-030-018, TASK-030-020).
- Coordinate with Platform team on auth scopes and API response stability.
- Can run in parallel with other CLI work once analytics endpoints are stable.
## Documentation Prerequisites
- `src/Cli/AGENTS.md`
- `src/Cli/StellaOps.Cli/AGENTS.md`
- `docs/modules/cli/contracts/cli-spec-v1.yaml`
- `docs/modules/analytics/queries.md`
- `docs/modules/cli/cli-reference.md`
## Delivery Tracker
### TASK-032-001 - CLI command contract and routing
Status: TODO
Dependency: none
Owners: Developer (Backend)
Task description:
- Define `analytics` command group with a `sbom-lake` subgroup and subcommands (suppliers, licenses, vulnerabilities, backlog, attestation-coverage, trends).
- Add flags for environment, severity, time range, limit, and output format.
- Register routes in `src/Cli/StellaOps.Cli/cli-routes.json` and update CLI spec.
Completion criteria:
- [ ] CLI spec updated with new commands and flags
- [ ] Routes registered and help text renders correctly
- [ ] Command naming aligns with CLI naming conventions
### TASK-032-002 - Analytics command handlers
Status: TODO
Dependency: TASK-032-001
Owners: Developer (Backend)
Task description:
- Implement handlers that call analytics API endpoints and map responses.
- Add a shared analytics client in CLI if needed.
- Normalize error handling and authorization flow with existing commands.
Completion criteria:
- [ ] Handlers implemented for all analytics subcommands
- [ ] API errors surfaced with consistent CLI messaging
- [ ] Auth scope checks match existing CLI patterns
### TASK-032-003 - Output formats and export support
Status: TODO
Dependency: TASK-032-002
Owners: Developer (Backend)
Task description:
- Support `--output` formats (table, json, csv) with deterministic ordering.
- Add `--out` for writing output to a file.
- Ensure table output aligns with UI label terminology.
Completion criteria:
- [ ] Table, JSON, and CSV outputs available
- [ ] Output ordering deterministic across runs
- [ ] File export works for each format
### TASK-032-004 - CLI tests and fixtures
Status: TODO
Dependency: TASK-032-003
Owners: QA
Task description:
- Add unit tests for analytics command handlers and output formatting.
- Store golden fixtures for deterministic output validation.
- Cover at least one error-path scenario per command group.
Completion criteria:
- [ ] Tests cover handlers and formatters
- [ ] Deterministic fixtures committed
- [ ] Error-path assertions in place
### TASK-032-005 - CLI documentation and parity notes
Status: TODO
Dependency: TASK-032-003
Owners: Documentation
Task description:
- Update `docs/modules/cli/cli-reference.md` with analytics commands and examples.
- Update `docs/modules/analytics/README.md` with CLI usage notes.
- Refresh `docs/modules/cli/cli-vs-ui-parity.md` for analytics coverage.
Completion criteria:
- [ ] CLI reference updated with command examples
- [ ] Analytics docs mention CLI access paths
- [ ] Parity doc updated for new analytics commands
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-20 | Sprint created to plan CLI exposure for SBOM analytics lake. | Planning |
| 2026-01-20 | Clarified analytics command hierarchy: analytics sbom-lake. | Planning |
## Decisions & Risks
- Cross-module edits: allow updates under `docs/modules/analytics/` and `docs/modules/cli/` for documentation and parity notes.
- Risk: API schema churn breaks CLI output contracts; mitigate with response version pinning and fixtures.
- Risk: CLI output mismatches UI terminology; mitigate by mapping labels to analytics query docs.
## Next Checkpoints
- TASK-032-002 complete: analytics commands wired to API.
- TASK-032-004 complete: tests and fixtures in place.
- TASK-032-005 complete: docs and parity updated.

488
docs/implplan/SPRINT_20260122_037_Signals_unified_trust_score_algebra.md Normal file
View File

@@ -0,0 +1,488 @@
# Sprint 037 – Unified Trust Score Facade (B+C+D Approach)
## Topic & Scope
Implement a **facade layer** over existing EWS and Determinization systems to provide:
- **B: Unified API** - Single interface combining EWS scores + Determinization entropy
- **C: Versioned weight manifests** - Extract EWS weights to `etc/weights/*.json` files
- **D: Unknowns fraction (U)** - Expose Determinization entropy as unified metric
**Key principle:** Preserve existing guardrails, conflict detection, anchor verification, and decay mechanisms. No formula changes - only unification and better exposure.
**Working directory:** `src/Signals/`
**Secondary directories:** `src/Policy/`, `src/Cli/`, `src/Platform/`
**Expected evidence:** Unit tests, integration tests, CLI updates, API endpoints, updated documentation
---
## Dependencies & Concurrency
- **Upstream (existing, no changes to core logic):**
- EWS: `src/Signals/StellaOps.Signals/EvidenceWeightedScore/` - 6-dimension scoring with guardrails
- Determinization: `src/Policy/__Libraries/StellaOps.Policy.Determinization/` - entropy, decay, fingerprints
- CLI: `src/Cli/StellaOps.Cli/Commands/ScoreGateCommandGroup.cs` - existing `stella gate score`
- **Concurrency:** Safe to run in parallel with other sprints; no breaking changes
---
## Documentation Prerequisites
- [Policy architecture](../modules/policy/architecture.md) §3.1 Determinization Configuration
- [EWS migration](../modules/policy/design/confidence-to-ews-migration.md) - existing scoring
- [Score Proofs API](../api/scanner-score-proofs-api.md) - determinism patterns
---
## Delivery Tracker
### TSF-001 - Extract EWS Weights to Manifest Files
Status: TODO
Dependency: none
Owners: Signals Guild
Task description:
Extract existing EWS weight configuration from `EvidenceWeightPolicy` into versioned JSON manifest files. EWS continues to work exactly as before, but weights are now loaded from files.
**Implementation:**
- Create `etc/weights/` directory structure
- Create `WeightManifest` record matching existing `EvidenceWeights` structure
- Create `IWeightManifestLoader` interface + `FileBasedWeightManifestLoader`
- Update `EvidenceWeightPolicy` to load weights from manifest files (with fallback to defaults)
- Add SHA-256 content hash to manifests for audit trail
- Migrate existing default weights: `etc/weights/v2026-01-22.weights.json`
**Key constraint:** No change to scoring formula or behavior - just externalize configuration.
Completion criteria:
- [ ] `etc/weights/v2026-01-22.weights.json` with current EWS defaults
- [ ] `WeightManifest.cs` record with version, effectiveFrom, weights, hash
- [ ] `FileBasedWeightManifestLoader.cs` loading from `etc/weights/`
- [ ] `EvidenceWeightPolicy` updated to use loader
- [ ] Unit tests verifying identical scoring before/after extraction
- [ ] Existing determinism tests still pass
---
### TSF-002 - Unified Score Facade Service
Status: TODO
Dependency: TSF-001
Owners: Signals Guild
Task description:
Create `IUnifiedScoreService` facade that combines EWS computation with Determinization entropy in a single call. Returns unified result with score, U metric, breakdown, and evidence.
**Implementation:**
- Create `IUnifiedScoreService` interface in `src/Signals/StellaOps.Signals/UnifiedScore/`:
```csharp
Task<UnifiedScoreResult> ComputeAsync(UnifiedScoreRequest request, CancellationToken ct);
```
- Create `UnifiedScoreService` that internally:
1. Calls `IEvidenceWeightedScoreCalculator.Calculate()` for EWS score
2. Calls `IUncertaintyScoreCalculator.CalculateEntropy()` for entropy (U)
3. Calls `IConflictDetector.Detect()` for conflict information
4. Combines into `UnifiedScoreResult`
- `UnifiedScoreResult` includes:
- `Score` (0-100 from EWS)
- `Bucket` (ActNow/ScheduleNext/Investigate/Watchlist)
- `UnknownsFraction` (U from Determinization entropy)
- `UnknownsBand` (Complete/Adequate/Sparse/Insufficient)
- `Breakdown` (EWS dimension contributions)
- `Guardrails` (which caps/floors applied)
- `Conflicts` (from ConflictDetector)
- `WeightManifestRef` (version + hash)
- `EwsDigest` + `DeterminizationFingerprint` (for replay)
- Register in DI container
Completion criteria:
- [ ] `IUnifiedScoreService` interface defined
- [ ] `UnifiedScoreService` implementation composing EWS + Determinization
- [ ] `UnifiedScoreRequest` / `UnifiedScoreResult` DTOs
- [ ] DI registration in `ServiceCollectionExtensions`
- [ ] Unit tests for facade composition
- [ ] Verify identical EWS scores pass through unchanged
---
### TSF-003 - Unknowns Band Mapping
Status: TODO
Dependency: TSF-002
Owners: Signals Guild / Policy Guild
Task description:
Map Determinization entropy (0.0-1.0) to user-friendly unknowns bands with actionable thresholds.
**Implementation:**
- Create `UnknownsBandMapper` in `src/Signals/StellaOps.Signals/UnifiedScore/`:
```csharp
UnknownsBand MapEntropyToBand(double entropy);
string GetBandDescription(UnknownsBand band);
string GetBandAction(UnknownsBand band);
```
- Band definitions (matching existing Determinization thresholds):
| U Range | Band | Description | Action |
|---------|------|-------------|--------|
| 0.0-0.2 | Complete | Full signal coverage | Automated decisions |
| 0.2-0.4 | Adequate | Sufficient signals | Automated decisions |
| 0.4-0.6 | Sparse | Signal gaps exist | Manual review recommended |
| 0.6-1.0 | Insufficient | Critical gaps | Block pending more signals |
- Integrate with existing `ManualReviewEntropyThreshold` (0.60) and `RefreshEntropyThreshold` (0.40) from Determinization config
Completion criteria:
- [ ] `UnknownsBandMapper.cs` with configurable thresholds
- [ ] `UnknownsBand` enum (Complete, Adequate, Sparse, Insufficient)
- [ ] Configuration via `appsettings.json` aligned with Determinization
- [ ] Unit tests for threshold boundaries
- [ ] Integration with `UnifiedScoreResult`
---
### TSF-004 - Delta-If-Present Calculations
Status: TODO
Dependency: TSF-002
Owners: Signals Guild
Task description:
When signals are missing, calculate and include "delta if present" showing potential score impact. Uses existing Determinization `SignalGap` information.
**Implementation:**
- Extend `UnifiedScoreResult` with `DeltaIfPresent` list:
```csharp
IReadOnlyList<SignalDelta> DeltaIfPresent { get; }
```
- `SignalDelta` record:
```csharp
record SignalDelta(string Signal, double MinImpact, double MaxImpact, string Description);
```
- For each missing signal (from Determinization gaps):
- Calculate EWS contribution if signal were 0.0 vs 1.0
- Include weight from manifest
- Add descriptive text (e.g., "If reachability confirmed, score could change by -15 to +8")
- Use existing `SignalGap` from Determinization for missing signal list
Completion criteria:
- [ ] `SignalDelta` record defined
- [ ] Delta calculation logic in `UnifiedScoreService`
- [ ] Integration with `UnifiedScoreResult.DeltaIfPresent`
- [ ] Unit tests for delta calculation accuracy
- [ ] Test with various missing signal combinations
---
### TSF-005 - Platform API Endpoints (Score Evaluate)
Status: TODO
Dependency: TSF-002, TSF-003, TSF-004
Owners: Platform Guild
Task description:
Expose unified score via Platform service REST API endpoints.
**Implementation:**
- Add endpoints to `src/Platform/StellaOps.Platform.WebService/Endpoints/`:
- `POST /api/v1/score/evaluate` - Compute unified score (primary scoring endpoint)
- `GET /api/v1/score/weights` - List available weight manifests
- `GET /api/v1/score/weights/{version}` - Get specific manifest
- Request contract for `/score/evaluate`:
```json
{
"sbom_ref": "oci://registry/app@sha256:…",
"cvss_vector": "CVSS:3.1/…",
"vex_refs": ["oci://…/vex1"],
"rekor_receipts": ["BASE64-RECEIPT"],
"runtime_witnesses": [{"type":"process","data":"…"}],
"options": {"decay_lambda": 0.015, "weight_set_id": "v2026-01-22"}
}
```
- Response includes:
- `score_id` - unique identifier for replay lookup
- `score_value` - 0-100 score
- `unknowns` - list of unknown package refs
- `proof_ref` - OCI reference to score proof bundle
- Full `UnifiedScoreResult` structure (breakdown, U, band, deltas)
- Support `?include_delta=true` query param for delta calculations
- Add OpenAPI documentation
- Tenant-scoped via Authority
Completion criteria:
- [ ] `POST /api/v1/score/evaluate` endpoint implemented
- [ ] `/api/v1/score/weights` endpoints implemented
- [ ] Request/response contracts match advisory spec
- [ ] OpenAPI spec generated
- [ ] Authentication/authorization configured
- [ ] Integration tests for each endpoint
---
### TSF-006 - CLI `stella gate score` Enhancement
Status: TODO
Dependency: TSF-005
Owners: CLI Guild
Task description:
Enhance existing `stella gate score evaluate` command to show unified metrics (U, bands, deltas).
**Implementation:**
- Update `ScoreGateCommandGroup.cs`:
- Add `--show-unknowns` flag to include U metric and band
- Add `--show-deltas` flag to include delta-if-present
- Add `--weights-version` option to pin specific manifest
- Update table output to show U and band when requested
- Update JSON output to include full unified result
- Add new subcommand `stella gate score weights`:
- `list` - Show available weight manifest versions
- `show <version>` - Display manifest details
- `diff <v1> <v2>` - Compare two manifests
Completion criteria:
- [ ] `--show-unknowns` flag showing U and band
- [ ] `--show-deltas` flag showing delta-if-present
- [ ] `--weights-version` option for pinning
- [ ] `stella gate score weights list|show|diff` commands
- [ ] Updated help text and examples
- [ ] CLI tests for new options
---
### TSF-007 - CLI `stella score` Top-Level Command
Status: TODO
Dependency: TSF-005, TSF-011
Owners: CLI Guild
Task description:
Add new top-level `stella score` command group for direct scoring operations (complementing existing `stella gate score` which is gate-focused).
**Implementation:**
- Create `ScoreCommandGroup.cs` in `src/Cli/StellaOps.Cli/Commands/`:
- `stella score compute` - Compute unified score from signals (similar inputs to gate evaluate)
- `stella score explain <finding-id>` - Detailed explanation with breakdown
- `stella score history <finding-id>` - Score history over time
- `stella score compare <finding-id-1> <finding-id-2>` - Compare two findings
- `stella score replay <score-id>` - Fetch and display replay proof (depends on TSF-011)
- `stella score verify <score-id>` - Verify score by replaying computation locally
- Support `--format json|table|markdown` output
- Support `--offline` mode using bundled weights
- Replay/verify commands output:
- Canonical input hashes
- Step-by-step algebra decisions
- Rekor inclusion proof (if anchored)
- Verification status (pass/fail with diff if mismatch)
Completion criteria:
- [ ] `stella score compute` command
- [ ] `stella score explain` command
- [ ] `stella score history` command (if backend supports)
- [ ] `stella score compare` command
- [ ] `stella score replay` command
- [ ] `stella score verify` command
- [ ] Multiple output formats
- [ ] Offline mode support
- [ ] CLI tests
---
### TSF-008 - Console UI Score Display Enhancement
Status: TODO
Dependency: TSF-005
Owners: FE Guild
Task description:
Update Console UI components that display scores to include unknowns fraction and band.
**Implementation:**
- Update finding detail views to show:
- Score with bucket (existing)
- Unknowns fraction (U) with visual indicator
- Unknowns band with color coding
- Delta-if-present for missing signals
- Weight manifest version used
- Add tooltip/popover explaining U and what it means
- Update score trend charts to optionally show U over time
- Update findings list to show U indicator for high-uncertainty findings
Completion criteria:
- [ ] Finding detail view shows U metric and band
- [ ] Color-coded band indicator (green/yellow/orange/red)
- [ ] Delta-if-present display for missing signals
- [ ] Tooltip explaining unknowns
- [ ] Findings list shows high-U indicator
- [ ] Score trend chart option for U
---
### TSF-009 - Determinism & Replay Tests
Status: TODO
Dependency: TSF-002
Owners: QA / Signals Guild
Task description:
Verify that the unified facade maintains determinism guarantees from underlying EWS and Determinization systems.
**Implementation:**
- Create `UnifiedScoreDeterminismTests.cs`:
- Same inputs → same unified result (100+ iterations)
- EWS score unchanged through facade
- Determinization entropy unchanged through facade
- Weight manifest hash stable
- Delta calculations deterministic
- Create golden test fixtures:
- Known inputs with expected unified outputs
- Fixtures for various U bands
- Fixtures for delta calculations
- Verify existing EWS determinism tests still pass
Completion criteria:
- [ ] `UnifiedScoreDeterminismTests.cs` with iteration tests
- [ ] Golden fixtures in `__Tests/Fixtures/UnifiedScore/`
- [ ] EWS pass-through verification
- [ ] Determinization pass-through verification
- [ ] CI gate for determinism regression
- [ ] Existing EWS/Determinization tests unaffected
---
### TSF-010 - Documentation Updates
Status: TODO
Dependency: TSF-001 through TSF-009
Owners: Documentation
Task description:
Update documentation to reflect the unified scoring facade.
**Implementation:**
- Update `docs/technical/scoring-algebra.md` to describe facade approach (not rewrite)
- Update `docs/modules/policy/architecture.md` §3.1 to reference weight manifests
- Create `docs/modules/signals/unified-score.md` explaining:
- What the facade provides
- How U metric works
- How to interpret bands
- CLI command reference
- Update `docs/modules/cli/guides/commands/reference.md` with new commands
- Add troubleshooting section for common U-related issues
Completion criteria:
- [ ] `docs/technical/scoring-algebra.md` updated for facade approach
- [ ] Policy architecture doc updated
- [ ] `docs/modules/signals/unified-score.md` guide created
- [ ] CLI reference updated
- [ ] Troubleshooting guide for U issues
---
### TSF-011 - Score Replay & Verification Endpoint
Status: TODO
Dependency: TSF-005
Owners: Platform Guild / Signals Guild
Task description:
Add explicit replay endpoint that returns a signed replay log, enabling external auditors to independently verify any score computation.
**Implementation:**
- Add endpoint to `src/Platform/StellaOps.Platform.WebService/Endpoints/`:
- `GET /api/v1/score/{id}/replay` - Fetch signed replay proof for a score
- Response contract:
```json
{
"signed_replay_log_dsse": "BASE64",
"rekor_inclusion": {"logIndex": 12345, "rootHash": "…"},
"canonical_inputs": [
{"name": "sbom.json", "sha256": "…"},
{"name": "vex.json", "sha256": "…"},
{"name": "kev.snapshot", "sha256": "…"}
],
"transforms": [
{"name": "canonicalize_spdx", "version": "1.1"},
{"name": "normalize_cvss_v4", "version": "1.0"},
{"name": "age_decay", "params": {"lambda": 0.02}}
],
"algebra_steps": [
{"signal": "cvss_v4_base_norm", "w": 0.30, "value": 0.78, "term": 0.234},
{"signal": "kev_flag", "w": 0.25, "value": 1, "term": 0.25}
],
"final_score": 85,
"computed_at": "2026-01-22T12:00:00Z"
}
```
- DSSE attestation format:
- Payload type: `application/vnd.stella.score+json`
- Sign with Authority key
- Store replay log as OCI referrer ("StellaBundle" pattern):
- Reference: `oci://registry/score-proofs@sha256:…`
- Attach to original artifact via OCI referrers API
- Create `IReplayLogBuilder` service:
- Collects canonical input hashes during scoring
- Records transform versions and parameters
- Captures step-by-step algebra decisions
- Generates DSSE-signed attestation
- Create `IReplayVerifier` service:
- Takes replay log + original inputs
- Re-executes scoring with pinned versions
- Returns verification result (pass/fail with diff)
Completion criteria:
- [ ] `GET /api/v1/score/{id}/replay` endpoint implemented
- [ ] `IReplayLogBuilder` service capturing full computation trace
- [ ] `IReplayVerifier` service for independent verification
- [ ] DSSE signing with `application/vnd.stella.score+json` payload type
- [ ] OCI referrer storage for replay proofs
- [ ] Rekor anchoring integration (optional, configurable)
- [ ] OpenAPI spec for replay endpoint
- [ ] Integration tests for replay/verify flow
- [ ] Golden corpus test: score → replay → verify round-trip
---
## Execution Log
| Date (UTC) | Update | Owner |
|------------|--------|-------|
| 2026-01-22 | Sprint created from product advisory | Planning |
| 2026-01-22 | Revised to B+C+D facade approach after deep analysis of existing systems | Planning |
| 2026-01-22 | Added TSF-011 (replay endpoint) per second advisory; renamed `/score/unified` to `/score/evaluate`; added `stella score replay|verify` CLI commands | Planning |
---
## Decisions & Risks
### Decisions Made
1. **Facade over rewrite** - Preserve existing EWS guardrails, conflict detection, anchor verification
2. **Weight manifest format** - JSON with SHA-256 hash, stored in `etc/weights/`
3. **U band thresholds** - Aligned with existing Determinization config (0.40/0.60 thresholds)
4. **No formula changes** - EWS scoring logic unchanged; only exposed differently
5. **Endpoint naming** - Use `/score/evaluate` (per second advisory) instead of `/score/unified` for industry alignment
6. **Explicit replay endpoint** - Add `/score/{id}/replay` returning signed DSSE attestation for auditor verification
7. **DSSE payload type** - Use `application/vnd.stella.score+json` for score attestations
8. **OCI referrer pattern** - Store replay proofs as OCI referrers ("StellaBundle") attached to scored artifacts
### Risks
1. **Performance** - Facade adds overhead calling two services
- Mitigation: Both services are fast (<100μs); combined still sub-millisecond
2. **Backward compatibility** - Existing CLI/API consumers expect current format
- Mitigation: New fields are additive; existing fields unchanged
3. **Configuration drift** - Weight manifest vs Determinization config could diverge
- Mitigation: Single source of truth via weight manifest; Determinization references it
### What We're NOT Doing
- ❌ Replacing EWS formula
- ❌ Replacing Determinization entropy calculation
- ❌ Changing guardrail logic
- ❌ Changing conflict detection
- ❌ Breaking existing CLI commands
- ❌ Breaking existing API contracts
---
## Next Checkpoints
- [ ] TSF-001 complete - Weights externalized
- [ ] TSF-002, TSF-003, TSF-004 complete - Facade functional
- [ ] TSF-005 complete - Score evaluate API endpoint
- [ ] TSF-011 complete - Replay/verification endpoint + DSSE attestation
- [ ] TSF-006, TSF-007 complete - CLI updated (including replay/verify commands)
- [ ] TSF-008 complete - UI updated
- [ ] TSF-009 complete - Determinism verified
- [ ] TSF-010 complete - Documentation finalized

115
docs/implplan/SPRINT_20260122_038_Scanner_ebpf_probe_type.md Normal file
View File

@@ -0,0 +1,115 @@
# Sprint 038 - eBPF Probe Type Enhancement
## Topic & Scope
- Add probe-type categorization to runtime observation models for eBPF sources
- Enable finer-grained filtering and policy evaluation based on probe type
- Document offline replay verification algorithm
- Working directory: `src/RuntimeInstrumentation/StellaOps.RuntimeInstrumentation.Tetragon/`
- Secondary directories: `src/Cli/StellaOps.Cli/Commands/`, `docs/modules/zastava/`
- Expected evidence: unit tests, updated CLI, architecture docs
## Dependencies & Concurrency
- Upstream: None (backwards-compatible enhancement)
- Can run in parallel with other sprints
- Uses existing `runtimeWitness@v1` predicate type (no new type needed)
## Documentation Prerequisites
- Archive manifest: `docs-archived/product/advisories/2026-01-22-ebpf-witness-contract/ARCHIVE_MANIFEST.md`
- Tetragon bridge: `src/RuntimeInstrumentation/StellaOps.RuntimeInstrumentation.Tetragon/TetragonWitnessBridge.cs`
- Zastava architecture: `docs/modules/zastava/architecture.md`
## Delivery Tracker
### EBPF-001 - Add ProbeType field to RuntimeObservation
Status: TODO
Dependency: none
Owners: Developer
Task description:
Extend the `RuntimeObservation` record in `TetragonWitnessBridge.cs` to include an optional `ProbeType` field. This allows distinguishing between kprobe, uprobe, tracepoint, and USDT observations while remaining backwards compatible.
Add enum and field:
```csharp
public enum EbpfProbeType
{
Kprobe,
Kretprobe,
Uprobe,
Uretprobe,
Tracepoint,
Usdt,
Fentry,
Fexit
}
// Add to RuntimeObservation record:
public EbpfProbeType? ProbeType { get; init; }
public string? FunctionName { get; init; }
public long? FunctionAddress { get; init; }
```
Completion criteria:
- [ ] `EbpfProbeType` enum added
- [ ] `ProbeType`, `FunctionName`, `FunctionAddress` fields added to `RuntimeObservation`
- [ ] Existing code continues to work (fields are optional)
- [ ] Unit tests for new fields
### EBPF-002 - Update Tetragon event parser to populate ProbeType
Status: TODO
Dependency: EBPF-001
Owners: Developer
Task description:
Update the Tetragon event parsing logic to extract and populate the `ProbeType` field from Tetragon events. Tetragon events include probe type information that should be mapped to the new enum.
Completion criteria:
- [ ] Tetragon event parser extracts probe type
- [ ] Mapping from Tetragon probe types to `EbpfProbeType` enum
- [ ] Integration tests with sample Tetragon events
### EBPF-003 - Add --probe-type filter to witness list CLI
Status: TODO
Dependency: EBPF-001
Owners: Developer
Task description:
Extend the `witness list` CLI command to support filtering by probe type. Add a `--probe-type` option that accepts: kprobe, uprobe, tracepoint, usdt.
Location: `src/Cli/StellaOps.Cli/Commands/WitnessCommandGroup.cs`
Completion criteria:
- [ ] `--probe-type` option added to `witness list` command
- [ ] Filtering logic implemented in handler
- [ ] Help text updated
- [ ] CLI test coverage added
### EBPF-004 - Document offline replay verification algorithm
Status: TODO
Dependency: none
Owners: Documentation author
Task description:
Add a section to `docs/modules/zastava/architecture.md` documenting the deterministic replay verification algorithm for runtime witnesses. This should specify:
- Input canonicalization steps (RFC 8785 JCS)
- Observation ordering rules for deterministic hashing
- Signature verification sequence
- Offline bundle structure requirements for witness verification
Completion criteria:
- [ ] New section "Offline Witness Verification" added to Zastava architecture
- [ ] Canonicalization steps documented
- [ ] Observation ordering rules specified
- [ ] Offline bundle requirements defined
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-22 | Sprint created from eBPF witness advisory. Simplified approach: extend existing model rather than new predicate type. | Planning |
## Decisions & Risks
- **Decision**: Extend existing `RuntimeObservation` with optional `ProbeType` field rather than creating new `ebpfWitness@v1` predicate type. Rationale: simpler, backwards compatible, `SourceType=Tetragon` already identifies eBPF source.
- **Risk**: None significant - all new fields are optional, existing witnesses remain valid.
## Next Checkpoints
- EBPF-001 and EBPF-004 can start immediately (no dependencies)
- EBPF-002 and EBPF-003 depend on EBPF-001

886
docs/implplan/SPRINT_20260122_039_Scanner_runtime_linkage_verification.md Normal file
View File

@@ -0,0 +1,886 @@
# Sprint 039 – Runtime→Static Linkage Verification
## Topic & Scope
Implement the **proof layer** that connects runtime eBPF observations to static analysis claims, enabling users to:
- Declare expected call-paths via a **function_map predicate** derived from SBOM
- Verify that runtime observations match declared expectations
- Complete the offline trust chain with **checkpoint signature verification**
- Query historical observations for compliance reporting
This sprint delivers the missing "contract" and "proof" layers identified in the eBPF witness advisory gap analysis.
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/`
**Secondary directories:**
- `src/Attestor/` (checkpoint signature fix)
- `src/Cli/StellaOps.Cli/Commands/` (CLI commands)
- `src/RuntimeInstrumentation/` (observation persistence)
- `src/Platform/` (API endpoints)
- `src/Web/` (UI components)
**Expected evidence:** Unit tests, integration tests, CLI commands, API endpoints, UI components, updated documentation
---
## User Stories
### US-1: Security Engineer declares expected call-paths
> "As a security engineer, I want to declare which functions my service is expected to call, so I can detect unexpected runtime behavior."
### US-2: DevOps verifies runtime matches expectations
> "As a DevOps engineer, I want to verify that runtime observations match our declared function map, so I can prove our services behave as expected."
### US-3: Auditor verifies offline
> "As an auditor, I want to verify runtime-to-static linkage in an air-gapped environment with full cryptographic proof."
### US-4: SOC analyst queries observation history
> "As a SOC analyst, I want to query historical observations for a specific function to investigate anomalies."
---
## Dependencies & Concurrency
- **Upstream (required before starting):**
- Sprint 038 EBPF-001: `ProbeType` field in `RuntimeObservation` (for richer verification)
- Existing `PathWitness` model in `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Witnesses/`
- Existing `ClaimIdGenerator` for claim linking
- Existing `TetragonWitnessBridge` for observation buffering
- **Upstream (no changes needed):**
- `HttpRekorClient` - will be patched for checkpoint signatures
- `BundleManifest` v2.0.0 - function_map will be added as artifact type
- **Concurrency:**
- Safe to run in parallel with Sprint 037 (trust score)
- Depends on Sprint 038 EBPF-001 completing first
---
## Documentation Prerequisites
- [Witness contract v1](../contracts/witness-v1.md) - Node hash and path hash recipes
- [Zastava architecture](../modules/zastava/architecture.md) - Runtime signal flow
- [Attestor offline verification](../modules/attestor/guides/offline-verification.md) - Bundle verification
- Sprint 038 EBPF-004 output (offline replay algorithm docs)
---
## Delivery Tracker
### RLV-001 - Define function_map Predicate Schema
Status: TODO
Dependency: none
Owners: Scanner Guild / Attestor Guild
Task description:
Define the `function_map` predicate schema that declares expected call-paths for a service. This is the "contract" that runtime observations will be verified against.
**Schema design:**
```json
{
"_type": "https://stella.ops/predicates/function-map/v1",
"subject": {
"purl": "pkg:oci/myservice@sha256:abc123...",
"digest": { "sha256": "abc123..." }
},
"predicate": {
"schemaVersion": "1.0.0",
"service": "myservice",
"buildId": "abc123def456...",
"generatedFrom": {
"sbomRef": "sha256:...",
"staticAnalysisRef": "sha256:..."
},
"expectedPaths": [
{
"pathId": "path-001",
"description": "TLS handshake via OpenSSL",
"entrypoint": {
"symbol": "myservice::handle_request",
"nodeHash": "sha256:..."
},
"expectedCalls": [
{
"symbol": "SSL_connect",
"purl": "pkg:deb/debian/openssl@3.0.11",
"nodeHash": "sha256:...",
"probeTypes": ["uprobe", "uretprobe"],
"optional": false
},
{
"symbol": "SSL_read",
"purl": "pkg:deb/debian/openssl@3.0.11",
"nodeHash": "sha256:...",
"probeTypes": ["uprobe"],
"optional": false
}
],
"pathHash": "sha256:..."
}
],
"coverage": {
"minObservationRate": 0.95,
"windowSeconds": 1800
},
"generatedAt": "2026-01-22T12:00:00Z"
}
}
```
**Key design decisions:**
- Uses existing `nodeHash` recipe from witness-v1 contract for consistency
- `expectedCalls` array defines the "hot functions" from the advisory
- `probeTypes` specifies which probe types are acceptable for each function
- `coverage.minObservationRate` maps to advisory's "≥ 95% of calls witnessed"
- `optional` flag allows for conditional paths (feature flags, error handlers)
**Implementation:**
- Create `FunctionMapPredicate.cs` record in `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/FunctionMap/`
- Create `ExpectedPath.cs` and `ExpectedCall.cs` supporting records
- Add JSON schema to `docs/schemas/function-map-v1.schema.json`
- Register predicate type with Attestor predicate router
Completion criteria:
- [ ] `FunctionMapPredicate.cs` with full schema
- [ ] JSON schema in `docs/schemas/`
- [ ] Predicate type registered: `https://stella.ops/predicates/function-map/v1`
- [ ] Unit tests for serialization/deserialization
- [ ] Schema validation tests
---
### RLV-002 - Implement FunctionMapGenerator
Status: TODO
Dependency: RLV-001
Owners: Scanner Guild
Task description:
Implement a generator that produces a `function_map` predicate from SBOM + static analysis results. This enables users to declare expected paths without manually authoring JSON.
**Implementation:**
- Create `IFunctionMapGenerator` interface:
```csharp
public interface IFunctionMapGenerator
{
Task<FunctionMapPredicate> GenerateAsync(
FunctionMapGenerationRequest request,
CancellationToken ct);
}
```
- Create `FunctionMapGenerationRequest`:
```csharp
public record FunctionMapGenerationRequest
{
public required string SbomPath { get; init; }
public required string ServiceName { get; init; }
public string? StaticAnalysisPath { get; init; }
public IReadOnlyList<string>? HotFunctionPatterns { get; init; }
public double MinObservationRate { get; init; } = 0.95;
public int WindowSeconds { get; init; } = 1800;
}
```
- Create `FunctionMapGenerator` implementation:
1. Parse SBOM to extract components with PURLs
2. If static analysis provided, extract call paths
3. If hot function patterns provided, filter to matching symbols
4. Generate node hashes using existing `NodeHashRecipe`
5. Compute path hashes using existing `PathHashRecipe`
6. Return populated `FunctionMapPredicate`
**Location:** `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/FunctionMap/`
Completion criteria:
- [ ] `IFunctionMapGenerator` interface
- [ ] `FunctionMapGenerator` implementation
- [ ] Integration with existing SBOM parser
- [ ] Support for hot function pattern matching (glob/regex)
- [ ] Unit tests with sample SBOM
- [ ] Integration test: SBOM → function_map → valid predicate
---
### RLV-003 - Implement IClaimVerifier
Status: TODO
Dependency: RLV-001, Sprint 038 EBPF-001
Owners: Scanner Guild
Task description:
Implement the claim verification logic that proves runtime observations match a declared function_map. This is the core "proof" step.
**Implementation:**
- Create `IClaimVerifier` interface in `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Verification/`:
```csharp
public interface IClaimVerifier
{
Task<ClaimVerificationResult> VerifyAsync(
FunctionMapPredicate functionMap,
IReadOnlyList<RuntimeObservation> observations,
ClaimVerificationOptions options,
CancellationToken ct);
}
```
- Create `ClaimVerificationResult`:
```csharp
public record ClaimVerificationResult
{
public required bool Verified { get; init; }
public required double ObservationRate { get; init; }
public required IReadOnlyList<PathVerificationResult> Paths { get; init; }
public required IReadOnlyList<string> UnexpectedSymbols { get; init; }
public required IReadOnlyList<string> MissingExpectedSymbols { get; init; }
public required ClaimVerificationEvidence Evidence { get; init; }
}
public record PathVerificationResult
{
public required string PathId { get; init; }
public required bool Observed { get; init; }
public required int ObservationCount { get; init; }
public required IReadOnlyList<string> MatchedNodeHashes { get; init; }
public required IReadOnlyList<string> MissingNodeHashes { get; init; }
}
public record ClaimVerificationEvidence
{
public required string FunctionMapDigest { get; init; }
public required string ObservationsDigest { get; init; }
public required DateTimeOffset VerifiedAt { get; init; }
public required string VerifierVersion { get; init; }
}
```
- Create `ClaimVerifier` implementation:
1. Group observations by node hash
2. For each expected path in function_map:
- Check if all required node hashes were observed
- Check if probe types match expectations
- Calculate observation rate
3. Detect unexpected symbols (observed but not in function_map)
4. Calculate overall observation rate
5. Compare against `coverage.minObservationRate`
6. Build evidence record for audit trail
**Verification algorithm:**
```
For each path in functionMap.expectedPaths:
matched = 0
for each call in path.expectedCalls:
if observations.any(o => o.nodeHash == call.nodeHash && call.probeTypes.contains(o.probeType)):
matched++
path.observationRate = matched / path.expectedCalls.count
overallRate = observedPaths / totalPaths
verified = overallRate >= functionMap.coverage.minObservationRate
```
Completion criteria:
- [ ] `IClaimVerifier` interface defined
- [ ] `ClaimVerifier` implementation with verification algorithm
- [ ] `ClaimVerificationResult` with detailed breakdown
- [ ] Evidence record for audit trail
- [ ] Detection of unexpected symbols
- [ ] Unit tests for various scenarios (full match, partial, no match)
- [ ] Integration test with real observations
---
### RLV-004 - Fix Checkpoint Signature Verification
Status: TODO
Dependency: none
Owners: Attestor Guild
Task description:
Complete the Rekor checkpoint signature verification that currently returns `false` unconditionally. This is required for full offline trust chain.
**Current state (HttpRekorClient.cs:282-289):**
```csharp
_logger.LogDebug(
"Checkpoint signature verification is unavailable for UUID {Uuid}; treating checkpoint as unverified",
rekorUuid);
// ...
return RekorInclusionVerificationResult.Success(
logIndex.Value,
computedRootHex,
proof.Checkpoint.RootHash,
checkpointSignatureValid: false); // Always false
```
**Implementation:**
- Update `HttpRekorClient.VerifyInclusionAsync()` to:
1. Extract checkpoint note from response
2. Parse note format: body + signature lines
3. Verify signature using `CheckpointSignatureVerifier` (already exists)
4. Return actual verification result
- Add `RekorPublicKey` configuration option for pinned verification
- Support both online (fetch from Rekor) and offline (pinned key) modes
**Location:** `src/Attestor/__Libraries/StellaOps.Attestor.Infrastructure/Rekor/HttpRekorClient.cs`
**Testing:**
- Verify against real Rekor checkpoint
- Verify with pinned public key (offline mode)
- Verify rejection of tampered checkpoint
Completion criteria:
- [ ] Checkpoint signature verification implemented
- [ ] `checkpointSignatureValid` returns actual result
- [ ] Support for pinned public key (air-gap mode)
- [ ] Unit tests with test vectors
- [ ] Integration test against Rekor staging
---
### RLV-005 - Implement Runtime Observation Store
Status: TODO
Dependency: Sprint 038 EBPF-001
Owners: Signals Guild
Task description:
Implement persistent storage for runtime observations to support historical queries and compliance reporting.
**Implementation:**
- Create `IRuntimeObservationStore` interface (if not exists) in `src/RuntimeInstrumentation/`:
```csharp
public interface IRuntimeObservationStore
{
Task StoreAsync(RuntimeObservation observation, CancellationToken ct);
Task StoreBatchAsync(IReadOnlyList<RuntimeObservation> observations, CancellationToken ct);
Task<IReadOnlyList<RuntimeObservation>> QueryBySymbolAsync(
string nodeHash,
DateTimeOffset from,
DateTimeOffset to,
CancellationToken ct);
Task<IReadOnlyList<RuntimeObservation>> QueryByContainerAsync(
string containerId,
DateTimeOffset from,
DateTimeOffset to,
CancellationToken ct);
Task<ObservationSummary> GetSummaryAsync(
string nodeHash,
DateTimeOffset from,
DateTimeOffset to,
CancellationToken ct);
Task PruneOlderThanAsync(TimeSpan retention, CancellationToken ct);
}
```
- Create `PostgresRuntimeObservationStore` implementation:
- Table: `runtime_observations` with indexes on `node_hash`, `container_id`, `observed_at`
- Batch insert with conflict handling (dedup by observation_id)
- Efficient time-range queries using BRIN index on `observed_at`
- Configurable retention policy (default: 7 days)
- Create migration: `src/RuntimeInstrumentation/.../Migrations/001_runtime_observations.sql`
- Wire into `TetragonWitnessBridge` to persist observations as they arrive
**Schema:**
```sql
CREATE TABLE runtime_observations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
observation_id TEXT NOT NULL UNIQUE,
node_hash TEXT NOT NULL,
symbol_name TEXT,
container_id TEXT NOT NULL,
pod_name TEXT,
namespace TEXT,
probe_type TEXT,
function_address BIGINT,
stack_sample_hash TEXT,
observation_count INTEGER DEFAULT 1,
duration_us BIGINT,
observed_at TIMESTAMPTZ NOT NULL,
created_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX idx_observations_node_hash ON runtime_observations (node_hash);
CREATE INDEX idx_observations_container ON runtime_observations (container_id);
CREATE INDEX idx_observations_time USING BRIN ON runtime_observations (observed_at);
```
Completion criteria:
- [ ] `IRuntimeObservationStore` interface
- [ ] `PostgresRuntimeObservationStore` implementation
- [ ] Database migration
- [ ] Integration with `TetragonWitnessBridge`
- [ ] Configurable retention policy
- [ ] Unit tests for store operations
- [ ] Integration tests with real Postgres
---
### RLV-006 - CLI: `stella function-map generate`
Status: TODO
Dependency: RLV-002
Owners: CLI Guild
Task description:
Add CLI command to generate a function_map predicate from SBOM.
**Implementation:**
- Create `FunctionMapCommandGroup.cs` in `src/Cli/StellaOps.Cli/Commands/`
- Add command: `stella function-map generate`
**Command spec:**
```
stella function-map generate [options]
Options:
--sbom <path> Path to SBOM file (CycloneDX/SPDX) [required]
--service <name> Service name for the function map [required]
--static-analysis <path> Path to static analysis results (optional)
--hot-functions <pattern> Glob pattern for hot functions (can repeat)
Example: --hot-functions "SSL_*" --hot-functions "crypto_*"
--min-rate <0.0-1.0> Minimum observation rate (default: 0.95)
--window <seconds> Observation window in seconds (default: 1800)
--output <path> Output path (default: stdout)
--format <json|yaml> Output format (default: json)
--sign Sign the predicate with configured key
--attest Create DSSE envelope and push to Rekor
Examples:
# Generate from SBOM with default hot functions
stella function-map generate --sbom sbom.json --service myservice
# Generate with specific hot functions
stella function-map generate --sbom sbom.json --service myservice \
--hot-functions "SSL_*" --hot-functions "EVP_*" --hot-functions "connect"
# Generate, sign, and attest
stella function-map generate --sbom sbom.json --service myservice \
--sign --attest --output function-map.json
```
Completion criteria:
- [ ] `stella function-map generate` command implemented
- [ ] All options working
- [ ] DSSE signing integration (--sign)
- [ ] Rekor attestation integration (--attest)
- [ ] JSON and YAML output formats
- [ ] Help text and examples
- [ ] CLI tests
---
### RLV-007 - CLI: `stella function-map verify`
Status: TODO
Dependency: RLV-003, RLV-005
Owners: CLI Guild
Task description:
Add CLI command to verify runtime observations against a function_map.
**Command spec:**
```
stella function-map verify [options]
Options:
--function-map <path|ref> Path or OCI reference to function_map predicate [required]
--container <id> Container ID to verify (optional, default: all)
--from <timestamp> Start of observation window (default: 30 minutes ago)
--to <timestamp> End of observation window (default: now)
--output <path> Output verification report (default: stdout)
--format <json|table|md> Output format (default: table)
--strict Fail on any unexpected symbols
--sign Sign the verification report
--offline Offline mode (use bundled observations)
--observations <path> Path to observations file (for offline mode)
Output:
Verified: true/false
Observation Rate: 97.2% (target: 95.0%)
Path Coverage:
┌──────────────┬──────────┬───────────┬─────────────┐
│ Path ID │ Status │ Rate │ Missing │
├──────────────┼──────────┼───────────┼─────────────┤
│ path-001 │ ✓ │ 100% │ - │
│ path-002 │ ✓ │ 95.5% │ - │
│ path-003 │ ✗ │ 80.0% │ SSL_write │
└──────────────┴──────────┴───────────┴─────────────┘
Unexpected Symbols: none
Evidence:
Function Map Digest: sha256:abc123...
Observations Digest: sha256:def456...
Verified At: 2026-01-22T12:00:00Z
Examples:
# Verify against stored observations
stella function-map verify --function-map function-map.json
# Verify specific container
stella function-map verify --function-map function-map.json \
--container abc123 --from "2026-01-22T11:30:00Z"
# Offline verification with bundled observations
stella function-map verify --function-map function-map.json \
--offline --observations observations.ndjson
# Sign verification report for audit
stella function-map verify --function-map function-map.json \
--sign --output verification-report.json
```
Completion criteria:
- [ ] `stella function-map verify` command implemented
- [ ] Query observations from store
- [ ] Offline mode with file input
- [ ] Table, JSON, and Markdown output formats
- [ ] Signed verification report option
- [ ] CLI tests
---
### RLV-008 - CLI: `stella observations query`
Status: TODO
Dependency: RLV-005
Owners: CLI Guild
Task description:
Add CLI command to query historical runtime observations.
**Command spec:**
```
stella observations query [options]
Options:
--symbol <name> Filter by symbol name (glob pattern)
--node-hash <hash> Filter by exact node hash
--container <id> Filter by container ID
--pod <name> Filter by pod name
--namespace <ns> Filter by Kubernetes namespace
--probe-type <type> Filter by probe type (kprobe|uprobe|tracepoint|usdt)
--from <timestamp> Start time (default: 1 hour ago)
--to <timestamp> End time (default: now)
--limit <n> Maximum results (default: 100)
--format <json|table|csv> Output format (default: table)
--summary Show summary statistics instead of individual observations
Examples:
# Query all SSL_connect observations in last hour
stella observations query --symbol "SSL_connect"
# Query by container
stella observations query --container abc123 --from "2026-01-22T11:00:00Z"
# Get summary statistics
stella observations query --symbol "SSL_*" --summary
# Export to CSV for analysis
stella observations query --namespace production --format csv > observations.csv
```
Completion criteria:
- [ ] `stella observations query` command implemented
- [ ] All filter options working
- [ ] Summary statistics mode
- [ ] CSV export for external analysis
- [ ] CLI tests
---
### RLV-009 - Platform API: Function Map Endpoints
Status: TODO
Dependency: RLV-002, RLV-003
Owners: Platform Guild
Task description:
Expose function_map operations via Platform service REST API.
**Endpoints:**
```
POST /api/v1/function-maps Create/store function map
GET /api/v1/function-maps List function maps
GET /api/v1/function-maps/{id} Get function map by ID
DELETE /api/v1/function-maps/{id} Delete function map
POST /api/v1/function-maps/{id}/verify Verify observations against map
GET /api/v1/function-maps/{id}/coverage Get current coverage statistics
```
**Request/Response contracts:**
`POST /api/v1/function-maps`:
```json
{
"sbomRef": "oci://registry/app@sha256:...",
"serviceName": "myservice",
"hotFunctions": ["SSL_*", "EVP_*"],
"options": {
"minObservationRate": 0.95,
"windowSeconds": 1800
}
}
```
`POST /api/v1/function-maps/{id}/verify`:
```json
{
"containerId": "abc123",
"from": "2026-01-22T11:00:00Z",
"to": "2026-01-22T12:00:00Z",
"strict": false
}
```
Response:
```json
{
"verified": true,
"observationRate": 0.972,
"targetRate": 0.95,
"paths": [...],
"unexpectedSymbols": [],
"evidence": {
"functionMapDigest": "sha256:...",
"observationsDigest": "sha256:...",
"verifiedAt": "2026-01-22T12:00:00Z"
}
}
```
Completion criteria:
- [ ] All endpoints implemented
- [ ] OpenAPI spec generated
- [ ] Tenant-scoped authorization
- [ ] Integration tests
- [ ] Rate limiting configured
---
### RLV-010 - UI: Function Map Management
Status: TODO
Dependency: RLV-009
Owners: FE Guild
Task description:
Add UI components for managing function maps and viewing verification results.
**Components:**
1. **Function Map List View** (`/settings/function-maps`)
- Table showing all function maps for tenant
- Columns: Service, Created, Last Verified, Coverage Status
- Actions: View, Verify Now, Delete
2. **Function Map Detail View** (`/settings/function-maps/{id}`)
- Service info and generation metadata
- Expected paths table with symbols
- Coverage thresholds configuration
- Recent verification history
3. **Function Map Generator Wizard** (`/settings/function-maps/new`)
- Step 1: Select SBOM source (file upload or OCI reference)
- Step 2: Configure hot function patterns (with suggestions)
- Step 3: Set coverage thresholds
- Step 4: Review and create
4. **Verification Results Panel** (embedded in service detail)
- Current verification status (verified/not verified)
- Observation rate gauge with threshold indicator
- Path coverage breakdown (expandable)
- Unexpected symbols warning (if any)
- Link to full verification report
5. **Observation Timeline** (`/services/{id}/observations`)
- Time-series chart of observation counts
- Filter by symbol/probe type
- Drill-down to individual observations
Completion criteria:
- [ ] Function map list view
- [ ] Function map detail view
- [ ] Generator wizard
- [ ] Verification results panel
- [ ] Observation timeline chart
- [ ] Responsive design
- [ ] Loading states and error handling
- [ ] E2E tests
---
### RLV-011 - Bundle Integration: function_map Artifact Type
Status: TODO
Dependency: RLV-001
Owners: AirGap Guild
Task description:
Add `function_map` as a supported artifact type in StellaBundle for offline verification.
**Implementation:**
- Update `BundleArtifactType` enum to include `FunctionMap`
- Update `BundleBuilder` to package function_map predicates
- Update `BundleValidator` to validate function_map artifacts
- Update `BundleVerifyCommand` to verify function_map signatures
**Bundle structure addition:**
```
bundle/
├── manifest.json
├── function-maps/
│ └── myservice-function-map.json
├── observations/
│ └── observations-2026-01-22.ndjson
└── verification/
└── verification-report.dsse.json
```
Completion criteria:
- [ ] `FunctionMap` artifact type added
- [ ] Bundle export includes function maps
- [ ] Bundle verify validates function map signatures
- [ ] Offline verification includes function map checking
- [ ] Documentation updated
---
### RLV-012 - Documentation: Runtime Linkage Verification Guide
Status: TODO
Dependency: RLV-001 through RLV-011
Owners: Documentation
Task description:
Create comprehensive documentation for the runtime→static linkage verification feature.
**Documents to create/update:**
1. **New: `docs/modules/scanner/guides/runtime-linkage.md`**
- What is runtime→static linkage verification?
- When to use function maps
- Step-by-step guide: generate → deploy probes → verify
- Troubleshooting common issues
2. **New: `docs/contracts/function-map-v1.md`**
- Predicate schema specification
- Node hash and path hash recipes (reference witness-v1)
- Coverage calculation algorithm
- Verification algorithm
3. **Update: `docs/modules/cli/guides/commands/reference.md`**
- Add `stella function-map` command group
- Add `stella observations` command group
4. **Update: `docs/modules/airgap/guides/offline-bundle-format.md`**
- Add function_map artifact type documentation
5. **New: `docs/runbooks/runtime-linkage-ops.md`**
- Operational runbook for production deployment
- Probe selection guidance
- Performance tuning
- Alert configuration
Completion criteria:
- [ ] Runtime linkage guide created
- [ ] function_map contract documented
- [ ] CLI reference updated
- [ ] Bundle format docs updated
- [ ] Operational runbook created
---
### RLV-013 - Acceptance Tests: 90-Day Pilot Criteria
Status: TODO
Dependency: All above tasks
Owners: QA Guild
Task description:
Implement acceptance tests matching the advisory's success criteria:
**Advisory acceptance criteria:**
1. **Coverage:** ≥ 95% of calls to the 6 hot funcs are witnessed over a steady-state 30-min window
2. **Integrity:** 100% DSSE sig verify + valid Rekor inclusion + valid TST
3. **Replayability:** Offline verifier reproduces the same mapping on 3 separate air-gapped runs
4. **Perf:** < 2% CPU overhead, < 50 MB RSS for collector under target load
5. **Privacy:** No raw args; only hashes and minimal context
**Test implementation:**
1. **Coverage test:**
- Generate function_map with 6 hot functions
- Run load generator for 30 minutes
- Verify observation rate ≥ 95%
2. **Integrity test:**
- Generate function_map with signing
- Create DSSE envelope
- Post to Rekor
- Add RFC-3161 timestamp
- Verify all signatures and proofs
3. **Replayability test:**
- Export StellaBundle with function_map + observations
- Run offline verification 3 times in isolated environments
- Assert identical results
4. **Performance test (if feasible in CI):**
- Measure CPU overhead with/without probes
- Measure collector memory usage
- Assert within thresholds
5. **Privacy test:**
- Inspect all observation payloads
- Assert no raw arguments present
- Assert only hashes and minimal context
Completion criteria:
- [ ] Coverage acceptance test
- [ ] Integrity acceptance test
- [ ] Replayability acceptance test (3 runs)
- [ ] Performance benchmark (manual or CI)
- [ ] Privacy audit test
- [ ] All tests passing in CI
---
## Execution Log
| Date (UTC) | Update | Owner |
|------------|--------|-------|
| 2026-01-22 | Sprint created from eBPF witness advisory gap analysis | Planning |
---
## Decisions & Risks
### Decisions Made
1. **function_map as separate predicate** - Not extending witness-v1, cleaner separation of concerns
2. **Reuse existing hash recipes** - NodeHash and PathHash from witness-v1 contract for consistency
3. **Postgres for observation storage** - Leverages existing infrastructure, supports time-range queries
4. **CLI-first verification** - Offline verification via CLI before UI for air-gap users
### Risks
1. **Observation volume** - High-traffic services may generate many observations
- Mitigation: Configurable sampling, aggregation, retention policy
2. **Clock skew** - Distributed observations may have timestamp drift
- Mitigation: Use server-side timestamps, configurable tolerance
3. **Symbol resolution accuracy** - Different runtimes have different symbol formats
- Mitigation: Use node hashes (PURL + normalized symbol) for matching
4. **Performance impact of persistence** - Writing every observation could be costly
- Mitigation: Batch writes, async persistence, sampling option
### Open Questions
1. Should function_map support version ranges for expected components, or exact versions only?
2. Should we support "learning mode" that auto-generates function_map from observations?
3. How to handle function maps for services with feature flags (conditional paths)?
---
## Next Checkpoints
- [ ] RLV-001 complete - Schema defined
- [ ] RLV-002, RLV-003 complete - Core verification logic works
- [ ] RLV-004 complete - Checkpoint signatures verified (trust chain complete)
- [ ] RLV-005 complete - Observations persisted
- [ ] RLV-006, RLV-007, RLV-008 complete - CLI fully functional
- [ ] RLV-009, RLV-010 complete - API and UI ready
- [ ] RLV-011 complete - Bundle integration for offline
- [ ] RLV-012 complete - Documentation finalized
- [ ] RLV-013 complete - Acceptance criteria met

2
docs/legal/THIRD-PARTY-DEPENDENCIES.md
View File

@@ -78,6 +78,7 @@ Primary runtime dependencies for .NET 10 modules. Extracted via `dotnet list pac
| Microsoft.EntityFrameworkCore | 10.0.0 | MIT | MIT | Yes |
| Microsoft.EntityFrameworkCore.Relational | 10.0.0 | MIT | MIT | Yes |
| Microsoft.Extensions.* | 10.0.x | MIT | MIT | Yes |
| Microsoft.Extensions.Configuration.Binder | 10.0.1 | MIT | MIT | Yes |
| Microsoft.IdentityModel.* | 8.x | MIT | MIT | Yes |
| System.IdentityModel.Tokens.Jwt | 8.0.1 | MIT | MIT | Yes |
@@ -108,6 +109,7 @@ Primary runtime dependencies for .NET 10 modules. Extracted via `dotnet list pac
| Package | Version | License | SPDX | Compatible |
|---------|---------|---------|------|------------|
| BouncyCastle.Cryptography | 2.6.2 | MIT | MIT | Yes |
| BCrypt.Net-Next | 4.0.3 | MIT | MIT | Yes |
| Pkcs11Interop | 5.1.2 | Apache-2.0 | Apache-2.0 | Yes |
| Blake3 | 1.1.0 | Apache-2.0 OR CC0-1.0 | Apache-2.0 | Yes |
| System.Security.Cryptography.Pkcs | 7.0.2 | MIT | MIT | Yes |

58
docs/modules/airgap/README.md
View File

@@ -39,6 +39,7 @@ Key settings:
- `subject`: sha256 (+ optional sha512) digest of the bundle target.
- `timestamps`: RFC3161/eIDAS timestamp entries with TSA chain/OCSP/CRL refs.
- `rekorProofs`: entry body/inclusion proof paths plus signed entry timestamp for offline verification.
- Inline artifacts (no `path`) are capped at 4 MiB; larger artifacts are written under `artifacts/`.
## Dependencies
@@ -55,6 +56,63 @@ Key settings:
- Mirror: `../mirror/`
- ExportCenter: `../export-center/`
## Evidence Bundles for Air-Gapped Verification
The AirGap module supports golden corpus evidence bundles for offline verification of patch provenance. These bundles enable auditors to verify security patch status without network access.
### Bundle Contents
Evidence bundles follow the OCI format and contain:
- Pre/post binaries with debug symbols
- Canonical SBOM for each binary
- DSSE delta-sig predicate proving patch status
- Build provenance (if available from buildinfo)
- RFC 3161 timestamps for each signed artifact
- Validation run results and KPIs
### Bundle Export
```bash
stella groundtruth bundle export \
--packages openssl,zlib,glibc \
--distros debian,fedora \
--output symbol-bundle.tar.gz \
--sign-with cosign
```
### Bundle Import and Verification
```bash
stella groundtruth bundle import \
--input symbol-bundle.tar.gz \
--verify-signature \
--trusted-keys /etc/stellaops/trusted-keys.pub \
--output verification-report.md
```
### Standalone Verifier
For air-gapped environments without the full Stella Ops stack, use the standalone verifier:
```bash
stella-verifier verify \
--bundle evidence-bundle.oci.tar \
--trusted-keys trusted-keys.pub \
--trust-profile eu-eidas.trustprofile.json \
--output report.json
```
Exit codes:
- `0`: All verifications passed
- `1`: One or more verifications failed
- `2`: Invalid input or configuration error
### Related Documentation
- [Golden Corpus Layout](../binary-index/golden-corpus-layout.md)
- [Golden Corpus Maintenance](../binary-index/golden-corpus-maintenance.md)
- [Golden Corpus Operations Runbook](../../runbooks/golden-corpus-operations.md)
## Current Status
Implemented with Controller for snapshot export and Importer for secure ingestion. Staleness policies enforce time-bound validity. Integrated with ExportCenter for bundle packaging and all data modules for content export/import.

107
docs/modules/analytics/README.md
View File

@@ -17,7 +17,7 @@ Stella Ops generates rich data through SBOM ingestion, vulnerability correlation
|------------|-------------|
| Unified component registry | Canonical component table with normalized suppliers and licenses |
| Vulnerability correlation | Pre-joined component-vulnerability mapping with EPSS/KEV flags |
| VEX-adjusted exposure | Vulnerability counts that respect VEX overrides |
| VEX-adjusted exposure | Vulnerability counts that respect active VEX overrides (validity windows applied) |
| Attestation tracking | Provenance and SLSA level coverage by environment/team |
| Time-series rollups | Daily snapshots for trend analysis |
| Materialized views | Pre-computed aggregations for dashboard performance |
@@ -68,6 +68,14 @@ Stella Ops generates rich data through SBOM ingestion, vulnerability correlation
| `daily_vulnerability_counts` | Rollup | Daily vuln aggregations |
| `daily_component_counts` | Rollup | Daily component aggregations |
Rollup retention is 90 days in hot storage. `compute_daily_rollups()` prunes
older rows after each run; archival follows operations runbooks.
Platform WebService can automate rollups + materialized view refreshes via
`PlatformAnalyticsMaintenanceService` (see `architecture.md` for schedule and
configuration).
Use `Platform:AnalyticsMaintenance:BackfillDays` to recompute the most recent
N days of rollups on the first maintenance run after downtime (set to `0` to disable).
### Materialized Views
| View | Refresh | Purpose |
@@ -77,33 +85,36 @@ Stella Ops generates rich data through SBOM ingestion, vulnerability correlation
| `mv_vuln_exposure` | Daily | CVE exposure adjusted by VEX |
| `mv_attestation_coverage` | Daily | Provenance/SLSA coverage by env/team |
Array-valued fields (for example `environments` and `ecosystems`) are ordered
alphabetically to keep analytics outputs deterministic.
## Quick Start
### Day-1 Queries
**Top supplier concentration (supply chain risk):**
**Top supplier concentration (supply chain risk, optional environment filter):**
```sql
SELECT * FROM analytics.sp_top_suppliers(20);
SELECT analytics.sp_top_suppliers(20, 'prod');
```
**License risk heatmap:**
**License risk heatmap (optional environment filter):**
```sql
SELECT * FROM analytics.sp_license_heatmap();
SELECT analytics.sp_license_heatmap('prod');
```
**CVE exposure adjusted by VEX:**
```sql
SELECT * FROM analytics.sp_vuln_exposure('prod', 'high');
SELECT analytics.sp_vuln_exposure('prod', 'high');
```
**Fixable vulnerability backlog:**
```sql
SELECT * FROM analytics.sp_fixable_backlog('prod');
SELECT analytics.sp_fixable_backlog('prod');
```
**Attestation coverage gaps:**
```sql
SELECT * FROM analytics.sp_attestation_gaps('prod');
SELECT analytics.sp_attestation_gaps('prod');
```
### API Endpoints
@@ -118,6 +129,82 @@ SELECT * FROM analytics.sp_attestation_gaps('prod');
| `/api/analytics/trends/vulnerabilities` | GET | Vulnerability time-series |
| `/api/analytics/trends/components` | GET | Component time-series |
All analytics endpoints require the `analytics.read` scope.
The platform metadata capability `analytics` reports whether analytics storage is configured.
#### Query Parameters
- `/api/analytics/suppliers`: `limit` (optional, default 20), `environment` (optional)
- `/api/analytics/licenses`: `environment` (optional)
- `/api/analytics/vulnerabilities`: `minSeverity` (optional, default `low`), `environment` (optional)
- `/api/analytics/backlog`: `environment` (optional)
- `/api/analytics/attestation-coverage`: `environment` (optional)
- `/api/analytics/trends/vulnerabilities`: `environment` (optional), `days` (optional, default 30)
- `/api/analytics/trends/components`: `environment` (optional), `days` (optional, default 30)
## Ingestion Configuration
Analytics ingestion runs inside the Platform WebService and subscribes to Scanner, Concelier, and Attestor streams. Configure ingestion via `Platform:AnalyticsIngestion`:
```yaml
Platform:
Storage:
PostgresConnectionString: "Host=...;Database=analytics;Username=...;Password=..."
AnalyticsIngestion:
Enabled: true
PostgresConnectionString: "" # optional; defaults to Platform:Storage
AllowedTenants: ["tenant-a", "tenant-b"]
Streams:
ScannerStream: "orchestrator:events"
ConcelierObservationStream: "concelier:advisory.observation.updated:v1"
ConcelierLinksetStream: "concelier:advisory.linkset.updated:v1"
AttestorStream: "attestor:events"
StartFromBeginning: false
Cas:
RootPath: "/var/lib/stellaops/cas"
DefaultBucket: "attestations"
Attestations:
BundleUriTemplate: "bundle:{digest}"
```
Bundle URI templates support:
- `{digest}` for the full digest string (for example `sha256:...`).
- `{hash}` for the raw hex digest (no algorithm prefix).
- `bundle:{digest}` which resolves to `cas://<DefaultBucket>/{digest}` by default.
- `file:/path/to/bundles/bundle-{hash}.json` for offline file ingestion.
For offline workflows, verify bundles with `stella bundle verify` before ingesting them.
## Console UI
SBOM Lake analytics are exposed in the Console under `Analytics > SBOM Lake` (`/analytics/sbom-lake`).
Console access requires `ui.read` plus `analytics.read` scopes.
Key UI features:
- Filters for environment, minimum severity, and time window.
- Panels for suppliers, licenses, vulnerability exposure, and attestation coverage.
- Trend views for vulnerabilities and components.
- Fixable backlog table with CSV export.
See [console.md](./console.md) for operator guidance and filter behavior.
## CLI Access
SBOM lake analytics are exposed via the CLI under `stella analytics sbom-lake`
(requires `analytics.read` scope).
```bash
# Top suppliers
stella analytics sbom-lake suppliers --limit 20
# Vulnerability exposure in prod (high+), CSV export
stella analytics sbom-lake vulnerabilities --environment prod --min-severity high --format csv --output vuln.csv
# 30-day trends for both series
stella analytics sbom-lake trends --days 30 --series all --format json
```
See `docs/modules/cli/guides/commands/analytics.md` for command-level details.
## Architecture
See [architecture.md](./architecture.md) for detailed design decisions, data flow, and normalization rules.
@@ -133,4 +220,6 @@ See [analytics_schema.sql](../../db/analytics_schema.sql) for complete DDL inclu
## Sprint Reference
Implementation tracked in: `docs/implplan/SPRINT_20260120_030_Platform_sbom_analytics_lake.md`
Implementation tracked in:
- `docs/implplan/SPRINT_20260120_030_Platform_sbom_analytics_lake.md`
- `docs/implplan/SPRINT_20260120_032_Cli_sbom_analytics_cli.md`

60
docs/modules/analytics/architecture.md
View File

@@ -7,7 +7,7 @@ The Analytics module implements a **star-schema data warehouse** pattern optimiz
1. **Separation of concerns**: Analytics schema is isolated from operational schemas (scanner, vex, proof_system)
2. **Pre-computation**: Expensive aggregations computed in advance via materialized views
3. **Audit trail**: Raw payloads preserved for reprocessing and compliance
4. **Determinism**: All normalization functions are immutable and reproducible
4. **Determinism**: Normalization functions are immutable and reproducible; array aggregates are ordered for stable outputs
5. **Incremental updates**: Supports both full refresh and incremental ingestion
## Data Flow
@@ -120,10 +120,9 @@ When a component is upserted, the `VulnerabilityCorrelationService` queries Conc
2. Filter by version range matching
3. Upsert to `component_vulns` with severity, EPSS, KEV flags
**Version range matching** uses Concelier's existing logic to handle:
- Semver ranges: `>=1.0.0 <2.0.0`
- Exact versions: `1.2.3`
- Wildcards: `1.x`
**Version range matching** currently supports semver ranges and exact matches via
`VersionRuleEvaluator`. Non-semver schemes fall back to exact string matches; wildcard
and ecosystem-specific ranges require upstream normalization.
## VEX Override Logic
@@ -145,7 +144,21 @@ COUNT(DISTINCT ac.artifact_id) FILTER (
**Override validity:**
- `valid_from`: When the override became effective
- `valid_until`: Expiration (NULL = no expiration)
- Only `status = 'not_affected'` reduces exposure counts
- Only `status = 'not_affected'` reduces exposure counts, and only when the override is active in its validity window.
## Attestation Ingestion
Attestation ingestion consumes Attestor Rekor entry events and expects Sigstore bundles
or raw DSSE envelopes. The ingestion service:
- Resolves bundle URIs using `BundleUriTemplate`; `bundle:{digest}` maps to
`cas://<DefaultBucket>/{digest}` by default.
- Decodes DSSE payloads, computes `dsse_payload_hash`, and records `predicate_uri` plus
Rekor log metadata (`rekor_log_id`, `rekor_log_index`).
- Uses in-toto `subject` digests to link artifacts when reanalysis hints are absent.
- Maps predicate URIs into `analytics_attestation_type` values
(`provenance`, `sbom`, `vex`, `build`, `scan`, `policy`).
- Expands VEX statements into `vex_overrides` rows, one per product reference, and
captures optional validity timestamps when provided.
## Time-Series Rollups
@@ -164,14 +177,14 @@ Daily rollups computed by `compute_daily_rollups()`:
- `total_components`: Distinct components
- `unique_suppliers`: Distinct normalized suppliers
**Retention policy:** 90 days in hot storage; older data archived to cold storage.
**Retention policy:** 90 days in hot storage; `compute_daily_rollups()` prunes older rows and downstream jobs archive to cold storage.
## Materialized View Refresh
All materialized views support `REFRESH ... CONCURRENTLY` for zero-downtime updates:
```sql
-- Refresh all views (run daily via pg_cron or Scheduler)
-- Refresh all views (non-concurrent; run off-peak)
SELECT analytics.refresh_all_views();
```
@@ -182,6 +195,19 @@ SELECT analytics.refresh_all_views();
- `mv_attestation_coverage`: 02:45 UTC daily
- `compute_daily_rollups()`: 03:00 UTC daily
Platform WebService can run the daily rollup + refresh loop via
`PlatformAnalyticsMaintenanceService`. Configure the schedule with:
- `Platform:AnalyticsMaintenance:Enabled` (default `true`)
- `Platform:AnalyticsMaintenance:IntervalMinutes` (default `1440`)
- `Platform:AnalyticsMaintenance:RunOnStartup` (default `true`)
- `Platform:AnalyticsMaintenance:ComputeDailyRollups` (default `true`)
- `Platform:AnalyticsMaintenance:RefreshMaterializedViews` (default `true`)
- `Platform:AnalyticsMaintenance:BackfillDays` (default `0`, set to `0` to disable; recompute the most recent N days on the first maintenance run)
The hosted service issues concurrent refresh statements directly for each view.
Use a DB scheduler (pg_cron) or external orchestrator if you need the staggered
per-view timing above.
## Performance Considerations
### Indexing Strategy
@@ -198,9 +224,9 @@ SELECT analytics.refresh_all_views();
| Query | Target | Notes |
|-------|--------|-------|
| `sp_top_suppliers(20)` | < 100ms | Uses materialized view |
| `sp_license_heatmap()` | < 100ms | Uses materialized view |
| `sp_vuln_exposure()` | < 200ms | Uses materialized view |
| `sp_top_suppliers(20, 'prod')` | < 100ms | Uses materialized view when env is null; env filter reads base tables |
| `sp_license_heatmap('prod')` | < 100ms | Uses materialized view when env is null; env filter reads base tables |
| `sp_vuln_exposure()` | < 200ms | Uses materialized view for global queries; environment filters read base tables |
| `sp_fixable_backlog()` | < 500ms | Live query with indexes |
| `sp_attestation_gaps()` | < 100ms | Uses materialized view |
@@ -246,12 +272,12 @@ All tables include `created_at` and `updated_at` timestamps. Raw payload tables
### Upstream Dependencies
| Service | Event | Action |
|---------|-------|--------|
| Scanner | SBOM ingested | Normalize and upsert components |
| Concelier | Advisory updated | Re-correlate affected components |
| Excititor | VEX observation | Create/update vex_overrides |
| Attestor | Attestation created | Upsert attestation record |
| Service | Event | Contract | Action |
|---------|-------|----------|--------|
| Scanner | SBOM report ready | `scanner.event.report.ready@1` (`docs/modules/signals/events/orchestrator-scanner-events.md`) | Normalize and upsert components |
| Concelier | Advisory observation/linkset updated | `advisory.observation.updated@1` (`docs/modules/concelier/events/advisory.observation.updated@1.schema.json`), `advisory.linkset.updated@1` (`docs/modules/concelier/events/advisory.linkset.updated@1.md`) | Re-correlate affected components |
| Excititor | VEX statement changes | `vex.statement.*` (`docs/modules/excititor/architecture.md`) | Create/update vex_overrides |
| Attestor | Rekor entry logged | `rekor.entry.logged` (`docs/modules/attestor/architecture.md`) | Upsert attestation record |
### Downstream Consumers

64
docs/modules/analytics/console.md Normal file
View File

@@ -0,0 +1,64 @@
# Analytics Console (SBOM Lake)
The Console exposes SBOM analytics lake data under `Analytics > SBOM Lake`.
This view is read-only and uses the analytics API endpoints documented in `docs/modules/analytics/README.md`.
## Access
- Route: `/analytics/sbom-lake`
- Required scopes: `ui.read` and `analytics.read`
- Console admin bundles: `role/analytics-viewer`, `role/analytics-operator`, `role/analytics-admin`
- Data freshness: the page surfaces the latest `dataAsOf` timestamp returned by the API.
## Filters
The SBOM Lake page supports three filters that round-trip via URL query parameters:
- Environment: `env` (optional, example: `Prod`)
- Minimum severity: `severity` (optional, example: `high`)
- Time window (days): `days` (optional, example: `90`)
When a filter changes, the Console reloads all panels using the updated parameters.
Supplier and license panels honor the environment filter alongside the other views.
## Panels
The dashboard presents four summary panels:
1. Supplier concentration (top suppliers by component count)
2. License distribution (license categories and counts)
3. Vulnerability exposure (top CVEs after VEX adjustments)
4. Attestation coverage (provenance and SLSA 2+ coverage)
Each panel shows a loading state, empty state, and summary counts.
## Trends
Two trend panels are included:
- Vulnerability trend: net exposure over the selected time window
- Component trend: total components and unique suppliers
The Console aggregates trend points by date and renders a simple bar chart plus a compact list.
## Fixable Backlog
The fixable backlog table lists vulnerabilities with fixes available, grouped by component and service.
The "Top backlog components" table derives a component summary from the same backlog data.
### CSV Export
The "Export backlog CSV" action downloads a deterministic, ordered CSV with:
- Service
- Component
- Version
- Vulnerability
- Severity
- Environment
- Fixed version
## Troubleshooting
- If panels show "No data", verify that the analytics schema and materialized views are populated.
- If an error banner appears, check the analytics API availability and ensure the tenant has `analytics.read`.

22
docs/modules/analytics/queries.md
View File

@@ -9,8 +9,8 @@ This document provides ready-to-use SQL queries for common analytics use cases.
Identifies suppliers with the highest component footprint, indicating supply chain concentration risk.
```sql
-- Via stored procedure (recommended)
SELECT * FROM analytics.sp_top_suppliers(20);
-- Via stored procedure (recommended, optional environment filter)
SELECT analytics.sp_top_suppliers(20, 'prod');
-- Direct query
SELECT
@@ -33,8 +33,8 @@ LIMIT 20;
Shows distribution of components by license category for compliance review.
```sql
-- Via stored procedure
SELECT * FROM analytics.sp_license_heatmap();
-- Via stored procedure (optional environment filter)
SELECT analytics.sp_license_heatmap('prod');
-- Direct query with grouping
SELECT
@@ -62,9 +62,9 @@ Shows true vulnerability exposure after applying VEX mitigations.
```sql
-- Via stored procedure
SELECT * FROM analytics.sp_vuln_exposure('prod', 'high');
SELECT analytics.sp_vuln_exposure('prod', 'high');
-- Direct query showing VEX effectiveness
-- Direct query showing VEX effectiveness (global view; use sp_vuln_exposure for environment filtering)
SELECT
vuln_id,
severity::TEXT,
@@ -97,7 +97,7 @@ Lists vulnerabilities that can be fixed today (fix available, not VEX-mitigated)
```sql
-- Via stored procedure
SELECT * FROM analytics.sp_fixable_backlog('prod');
SELECT analytics.sp_fixable_backlog('prod');
-- Direct query with priority scoring
SELECT
@@ -130,6 +130,7 @@ JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
LEFT JOIN analytics.vex_overrides vo ON vo.artifact_id = a.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
WHERE cv.affects = TRUE
AND cv.fix_available = TRUE
@@ -147,7 +148,7 @@ Shows attestation gaps by environment and team.
```sql
-- Via stored procedure
SELECT * FROM analytics.sp_attestation_gaps('prod');
SELECT analytics.sp_attestation_gaps('prod');
-- Direct query with gap analysis
SELECT
@@ -267,6 +268,7 @@ JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
LEFT JOIN analytics.vex_overrides vo ON vo.artifact_id = a.artifact_id
AND vo.vuln_id = cv.vuln_id
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
WHERE cv.vuln_id = 'CVE-2021-44228'
ORDER BY a.environment, a.name;
@@ -312,7 +314,7 @@ SELECT
c.license_category::TEXT,
c.supplier_normalized AS supplier,
COUNT(DISTINCT a.artifact_id) AS artifact_count,
ARRAY_AGG(DISTINCT a.name) AS affected_artifacts
ARRAY_AGG(DISTINCT a.name ORDER BY a.name) AS affected_artifacts
FROM analytics.components c
JOIN analytics.artifact_components ac ON ac.component_id = c.component_id
JOIN analytics.artifacts a ON a.artifact_id = ac.artifact_id
@@ -340,6 +342,8 @@ SELECT
FROM analytics.component_vulns cv
JOIN analytics.vex_overrides vo ON vo.vuln_id = cv.vuln_id
AND vo.status = 'not_affected'
AND vo.valid_from <= now()
AND (vo.valid_until IS NULL OR vo.valid_until > now())
WHERE cv.published_at >= now() - INTERVAL '90 days'
AND cv.published_at IS NOT NULL
GROUP BY cv.severity

2
docs/modules/attestor/guides/README.md
View File

@@ -14,7 +14,7 @@ StellaOps SBOM interoperability tests ensure compatibility with third-party secu
| SPDX | 3.0.1 | ✅ Supported | 95%+ |
Notes:
- SPDX 3.0.1 generation currently emits JSON-LD `@context`, `spdxVersion`, core document/package/relationship elements, software package/file/snippet metadata, build profile elements with output relationships, security vulnerabilities with assessment relationships, verifiedUsing hashes/signatures, and external references/identifiers. Full profile coverage is tracked in SPRINT_20260119_014.
- SPDX 3.0.1 generation currently emits JSON-LD `@context`, `spdxVersion`, core document/package/relationship elements (including agent/tool elements for creationInfo), software package/file/snippet metadata, build profile elements with output relationships, security vulnerabilities with assessment relationships, licensing license elements with declared/concluded relationships, AI AIPackage metadata (autonomy, domain, metrics, safety risk assessment), Dataset package metadata (type, collection, preprocessing, availability), verifiedUsing hashes/signatures, external references/identifiers (including externalRef contentType when available), namespaceMap/imports for cross-document references, extension metadata via SbomExtension namespace/properties on document/component/vulnerability elements, and Lite profile output (opt-in via SpdxWriterOptions.UseLiteProfile). Full profile coverage is tracked in SPRINT_20260119_014.
### Third-Party Tools

5
docs/modules/attestor/guides/offline-verification.md
View File

@@ -29,11 +29,14 @@ Use the bundle verification flow aligned to domain operations:
```bash
stella bundle verify --bundle /path/to/bundle --offline --trust-root /path/to/tsa-root.pem --rekor-checkpoint /path/to/checkpoint.json
stella bundle verify --bundle /path/to/bundle --offline --signer /path/to/report-key.pem --signer-cert /path/to/report-cert.pem
```
Notes:
- Offline mode fails closed when revocation evidence is missing or invalid.
- Offline mode fails closed when revocation evidence is missing or invalid.
- Trust roots must be provided locally; no network fetches are allowed.
- When `--signer` is set, a DSSE report is written to `out/verification.report.json`.
- Signed report metadata includes `verifier.algo`, `verifier.cert`, `signed_at`.
## 4. Verification Behavior

183
docs/modules/binary-index/architecture.md
View File

@@ -1239,7 +1239,183 @@ binaryindex:
---
## 10. References
## 10. Golden Corpus for Patch Provenance
> **Sprint:** SPRINT_20260121_034/035/036 - Golden Corpus Implementation
The BinaryIndex module supports a **golden corpus** of patch-paired artifacts that enables offline SBOM reproducibility and binary-level patch provenance verification.
### 10.1 Corpus Purpose
The golden corpus provides:
- **Auditor-ready evidence bundles** for air-gapped customers
- **Regression testing** for binary matching accuracy
- **Proof of patch status** independent of package metadata
### 10.2 Corpus Sources
| Source | Type | Purpose |
|--------|------|---------|
| Debian Security Tracker / DSAs | Advisory | Primary advisory linkage |
| Debian Snapshot | Binary archive | Pre/post patch binary pairs |
| Ubuntu Security Notices | Advisory | Ubuntu-specific advisories |
| Alpine secdb | Advisory | Alpine YAML advisories |
| OSV dump | Unified schema | Cross-reference and commit ranges |
### 10.2.1 Symbol Source Connectors
> **Sprint:** SPRINT_20260121_035_BinaryIndex_golden_corpus_connectors_cli
The corpus ingestion layer uses pluggable connectors to retrieve symbols and metadata from upstream sources:
| Connector ID | Implementation | Protocol | Data Retrieved |
|--------------|----------------|----------|----------------|
| `debuginfod-fedora` | `DebuginfodConnector` | debuginfod HTTP | ELF debug symbols by Build-ID |
| `debuginfod-ubuntu` | `DebuginfodConnector` | debuginfod HTTP | ELF debug symbols by Build-ID |
| `ddeb-ubuntu` | `DdebConnector` | APT/HTTP | `.ddeb` debug packages |
| `buildinfo-debian` | `BuildinfoConnector` | HTTP | `.buildinfo` reproducibility records |
| `secdb-alpine` | `AlpineSecDbConnector` | Git/HTTP | `secfixes` YAML from APKBUILD |
**Connector Interface:**
```csharp
public interface ISymbolSourceConnector
{
string ConnectorId { get; }
string DisplayName { get; }
string[] SupportedDistros { get; }
Task<ConnectorStatus> GetStatusAsync(CancellationToken ct);
Task SyncAsync(SyncOptions options, CancellationToken ct);
Task<SymbolLookupResult?> LookupByBuildIdAsync(string buildId, CancellationToken ct);
Task<IAsyncEnumerable<SymbolRecord>> SearchAsync(SymbolSearchQuery query, CancellationToken ct);
}
```
**Debuginfod Connector:**
The `DebuginfodConnector` implements the [debuginfod protocol](https://sourceware.org/elfutils/Debuginfod.html) for retrieving debug symbols:
- Endpoint: `GET /buildid/<build-id>/debuginfo`
- Supports federated queries across multiple debuginfod servers
- Caches retrieved symbols in RustFS blob storage
- Rate-limited to respect upstream server policies
**Ubuntu ddeb Connector:**
The `DdebConnector` retrieves Ubuntu debug symbol packages (`.ddeb`):
- Sources: `ddebs.ubuntu.com` mirror
- Indexes: Reads `Packages.xz` for package metadata
- Extraction: Unpacks `.ddeb` AR archives to extract DWARF symbols
- Mapping: Links debug symbols to binary packages via Build-ID
**Debian Buildinfo Connector:**
The `BuildinfoConnector` retrieves Debian buildinfo files for reproducibility verification:
- Source: `buildinfos.debian.net` and snapshot archives
- Purpose: Provides build environment metadata for reproducible builds
- Fields extracted: `Build-Date`, `Build-Architecture`, `Checksums-Sha256`
- Integration: Cross-references with binary packages for provenance
**Alpine SecDB Connector:**
The `AlpineSecDbConnector` parses Alpine's security database:
- Source: `secfixes` blocks in APKBUILD files
- Repository: `alpine/aports` Git repository
- Format: YAML blocks mapping CVEs to fixed versions
- Example:
```yaml
secfixes:
3.0.11-r0:
- CVE-2024-0727
- CVE-2024-0728
```
**OSV Dump Parser:**
The `OsvDumpParser` processes Google OSV database dumps for advisory cross-correlation:
- Source: `osv.dev` bulk exports (JSON)
- Purpose: CVE → commit range extraction for patch identification
- Cross-reference: Correlates OSV entries with distribution advisories
- Inconsistency detection: Identifies discrepancies between OSV and distro advisories
```csharp
public interface IOsvDumpParser
{
IAsyncEnumerable<OsvParsedEntry> ParseDumpAsync(Stream osvDumpStream, CancellationToken ct);
OsvCveIndex BuildCveIndex(IEnumerable<OsvParsedEntry> entries);
IEnumerable<AdvisoryCorrelation> CrossReferenceWithExternal(
OsvCveIndex osvIndex,
IEnumerable<ExternalAdvisory> externalAdvisories);
IEnumerable<AdvisoryInconsistency> DetectInconsistencies(
IEnumerable<AdvisoryCorrelation> correlations);
}
```
**CLI Access:**
All connectors are manageable via the `stella groundtruth sources` CLI commands:
```bash
# List all connectors
stella groundtruth sources list
# Sync specific connector
stella groundtruth sources sync --source buildinfo-debian --full
# Enable/disable connectors
stella groundtruth sources enable ddeb-ubuntu
stella groundtruth sources disable debuginfod-fedora
```
See [Ground-Truth CLI Guide](../cli/guides/ground-truth-cli.md) for complete CLI documentation
### 10.3 Key Performance Indicators
| KPI | Target | Description |
|-----|--------|-------------|
| Per-function match rate | >= 90% | Functions matched in post-patch binary |
| False-negative patch detection | <= 5% | Patched functions incorrectly classified |
| SBOM canonical-hash stability | 3/3 | Determinism across independent runs |
| Binary reconstruction equivalence | Trend | Rebuilt binary matches original |
| End-to-end verify time (p95, cold) | Trend | Offline verification performance |
### 10.4 Validation Harness
The validation harness (`IValidationHarness`) orchestrates end-to-end verification:
```
Binary Pair (pre/post) → Symbol Recovery → IR Lifting → Fingerprinting → Matching → Metrics
```
### 10.5 Evidence Bundle Format
Evidence bundles follow OCI/ORAS conventions:
```
<pkg>-<advisory>-bundle.oci.tar
├── manifest.json # OCI manifest
└── blobs/
├── sha256:<sbom> # Canonical SBOM
├── sha256:<pre-bin> # Pre-fix binary
├── sha256:<post-bin> # Post-fix binary
├── sha256:<delta-sig> # DSSE delta-sig predicate
└── sha256:<timestamp> # RFC 3161 timestamp
```
### 10.6 Related Documentation
- [Golden Corpus KPIs](../../benchmarks/golden-corpus-kpis.md)
- [Golden Corpus Seed List](../../benchmarks/golden-corpus-seed-list.md)
- [Ground-Truth Corpus Specification](../../benchmarks/ground-truth-corpus.md)
---
## 11. References
- Advisory: `docs/product/advisories/21-Dec-2025 - Mapping Evidence Within Compiled Binaries.md`
- Scanner Native Analysis: `src/Scanner/StellaOps.Scanner.Analyzers.Native/`
@@ -1248,8 +1424,9 @@ binaryindex:
- **Semantic Diffing Sprint:** `docs/implplan/SPRINT_20260105_001_001_BINDEX_semdiff_ir_semantics.md`
- **Semantic Library:** `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Semantic/`
- **Semantic Tests:** `src/BinaryIndex/__Tests/StellaOps.BinaryIndex.Semantic.Tests/`
- **Golden Corpus Sprints:** `docs/implplan/SPRINT_20260121_034_BinaryIndex_golden_corpus_foundation.md`
---
*Document Version: 1.1.1*
*Last Updated: 2026-01-14*
*Document Version: 1.2.0*
*Last Updated: 2026-01-21*

347
docs/modules/binary-index/golden-corpus-layout.md Normal file
View File

@@ -0,0 +1,347 @@
# Golden Corpus Folder Layout
Sprint: SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
Task: GCB-006 - Document corpus folder layout and maintenance procedures
## Overview
The golden corpus is a curated dataset of pre/post security patch binary pairs used for:
- Validating binary matching algorithms
- Benchmarking reproducibility verification
- Training machine learning models for function identification
- Generating audit-ready evidence bundles
## Root Layout
```
golden-corpus/
├── corpus/ # Security pairs organized by distro
│ ├── debian/
│ ├── ubuntu/
│ └── alpine/
├── mirrors/ # Local mirrors of upstream sources
│ ├── debian/
│ ├── ubuntu/
│ ├── alpine/
│ └── osv/
├── harness/ # Build and verification tooling
│ ├── chroots/
│ ├── lifter-matcher/
│ ├── sbom-canonicalizer/
│ └── verifier/
├── evidence/ # Generated evidence bundles
│ └── <pkg>-<advisory>-bundle.oci.tar
└── bench/ # Benchmark data and baselines
├── baselines/
└── results/
```
## Corpus Directory Structure
Each security pair follows a consistent structure:
```
corpus/<distro>/<package>/<advisory-id>/
├── pre/ # Pre-patch (vulnerable) artifacts
│ ├── src/ # Source code
│ │ ├── *.tar.gz # Original source tarball
│ │ ├── debian/ # Packaging metadata
│ │ └── buildinfo # Build reproducibility info
│ └── debs/ # Built binaries
│ ├── *.deb # Binary packages
│ ├── *.ddeb # Debug symbols
│ └── buildlog # Build log
├── post/ # Post-patch (fixed) artifacts
│ ├── src/
│ └── debs/
└── metadata/
├── advisory.json # Advisory details
├── osv.json # OSV format vulnerability
├── pair-manifest.json # Pair configuration
└── ground-truth.json # Function-level ground truth
```
### Debian Example
```
corpus/debian/openssl/DSA-5678-1/
├── pre/
│ ├── src/
│ │ ├── openssl_3.0.10.orig.tar.gz
│ │ ├── openssl_3.0.10-1.debian.tar.xz
│ │ ├── openssl_3.0.10-1.dsc
│ │ └── openssl_3.0.10-1.buildinfo
│ └── debs/
│ ├── libssl3_3.0.10-1_amd64.deb
│ ├── libssl3-dbgsym_3.0.10-1_amd64.ddeb
│ └── build.log
├── post/
│ ├── src/
│ │ ├── openssl_3.0.11.orig.tar.gz
│ │ ├── openssl_3.0.11-1.debian.tar.xz
│ │ └── ...
│ └── debs/
│ └── ...
└── metadata/
├── advisory.json
└── ground-truth.json
```
### Ubuntu Example
```
corpus/ubuntu/curl/USN-1234-1/
├── pre/
│ ├── src/
│ │ └── curl_8.4.0-1ubuntu1.tar.xz
│ └── debs/
│ └── libcurl4_8.4.0-1ubuntu1_amd64.deb
├── post/
│ └── ...
└── metadata/
├── advisory.json
└── usn.json
```
### Alpine Example
```
corpus/alpine/zlib/CVE-2022-37434/
├── pre/
│ ├── src/
│ │ └── APKBUILD
│ └── apks/
│ └── zlib-1.2.12-r2.apk
├── post/
│ └── ...
└── metadata/
└── secdb-entry.json
```
## Mirrors Directory Structure
Local mirrors cache upstream artifacts for offline operation:
```
mirrors/
├── debian/
│ ├── archive/ # snapshot.debian.org mirrors
│ │ └── pool/main/o/openssl/
│ ├── snapshot/ # Point-in-time snapshots
│ │ └── 20260101T000000Z/
│ └── buildinfo/ # buildinfos.debian.net cache
│ └── <source-name>/
├── ubuntu/
│ ├── archive/ # archive.ubuntu.com mirrors
│ ├── usn-index/ # USN metadata
│ │ └── usn-db.json
│ └── launchpad/ # Build logs from Launchpad
├── alpine/
│ ├── packages/ # Alpine package mirror
│ └── secdb/ # Security database
│ └── community.json
└── osv/
├── all.zip # Full OSV database
└── debian/ # Distro-specific extracts
```
## Harness Directory Structure
Build and verification tooling:
```
harness/
├── chroots/ # Build environments
│ ├── debian-bookworm-amd64/
│ ├── debian-bullseye-amd64/
│ ├── ubuntu-noble-amd64/
│ └── alpine-3.19-amd64/
├── lifter-matcher/ # Binary analysis tools
│ ├── ghidra/ # Ghidra installation
│ ├── bsim-server/ # BSim database server
│ └── semantic-diffing/ # Semantic diff tools
├── sbom-canonicalizer/ # SBOM normalization
│ └── config/
└── verifier/ # Standalone verifier
├── stella-verifier # Verifier binary
└── trust-profiles/ # Trust profiles
```
## Evidence Directory Structure
Generated bundles for audit/compliance:
```
evidence/
├── openssl-DSA-5678-1-bundle.oci.tar
├── curl-USN-1234-1-bundle.oci.tar
└── manifests/
└── inventory.json
```
### Bundle Internal Structure (OCI Format)
```
openssl-DSA-5678-1-bundle.oci.tar/
├── oci-layout # OCI layout version
├── index.json # OCI index with referrers
├── blobs/
│ └── sha256/
│ ├── <manifest> # Bundle manifest
│ ├── <sbom-pre> # Pre-patch SBOM
│ ├── <sbom-post> # Post-patch SBOM
│ ├── <binary-pre> # Pre-patch binary
│ ├── <binary-post> # Post-patch binary
│ ├── <delta-sig> # DSSE delta-sig predicate
│ ├── <provenance> # Build provenance
│ └── <timestamp> # RFC 3161 timestamp
└── manifest.json # Signed bundle manifest
```
## Bench Directory Structure
Benchmark data and KPI baselines:
```
bench/
├── baselines/
│ ├── current.json # Active KPI baseline
│ └── archive/ # Historical baselines
│ ├── baseline-20260115.json
│ └── baseline-20260108.json
├── results/
│ ├── 20260122120000.json # Validation run results
│ └── ...
└── reports/
└── regression-report-*.md
```
### Baseline File Format
```json
{
"baselineId": "baseline-20260122120000",
"createdAt": "2026-01-22T12:00:00Z",
"source": "abc123def456",
"description": "Post-semantic-diffing-v2 baseline",
"precision": 0.95,
"recall": 0.92,
"falseNegativeRate": 0.08,
"deterministicReplayRate": 1.0,
"ttfrpP95Ms": 150,
"additionalKpis": {}
}
```
## File Naming Conventions
| Type | Pattern | Example |
|------|---------|---------|
| Advisory ID (Debian) | `DSA-<number>-<revision>` | `DSA-5678-1` |
| Advisory ID (Ubuntu) | `USN-<number>-<revision>` | `USN-1234-1` |
| Advisory ID (Alpine) | `CVE-<year>-<number>` | `CVE-2022-37434` |
| Bundle file | `<pkg>-<advisory>-bundle.oci.tar` | `openssl-DSA-5678-1-bundle.oci.tar` |
| Baseline file | `baseline-<timestamp>.json` | `baseline-20260122120000.json` |
| Results file | `<timestamp>.json` | `20260122120000.json` |
## Metadata Files
### advisory.json
```json
{
"advisoryId": "DSA-5678-1",
"cves": ["CVE-2024-1234", "CVE-2024-5678"],
"package": "openssl",
"vulnerableVersions": ["3.0.10-1"],
"fixedVersions": ["3.0.11-1"],
"severity": "high",
"publishedAt": "2024-11-15T00:00:00Z",
"summary": "Multiple vulnerabilities in OpenSSL"
}
```
### pair-manifest.json
```json
{
"pairId": "openssl-DSA-5678-1",
"package": "openssl",
"distribution": "debian",
"suite": "bookworm",
"architecture": "amd64",
"preVersion": "3.0.10-1",
"postVersion": "3.0.11-1",
"binaries": [
"libssl3",
"libcrypto3"
],
"createdAt": "2026-01-15T10:00:00Z",
"validatedAt": "2026-01-22T12:00:00Z"
}
```
### ground-truth.json
```json
{
"pairId": "openssl-DSA-5678-1",
"binary": "libcrypto.so.3",
"functions": [
{
"name": "EVP_DigestInit_ex",
"preAddress": "0x12345",
"postAddress": "0x12347",
"status": "modified",
"confidence": 1.0
},
{
"name": "EVP_DigestUpdate",
"preAddress": "0x12400",
"postAddress": "0x12400",
"status": "unchanged",
"confidence": 1.0
}
],
"metadata": {
"generatedBy": "manual-annotation",
"reviewedBy": "security-team",
"reviewedAt": "2026-01-20T14:00:00Z"
}
}
```
## Access Patterns
### Read-Only Access
- Validation harness reads corpus pairs
- CI reads baselines for regression checks
- Auditors read evidence bundles
### Write Access
- Corpus ingestion adds new pairs
- Baseline update writes new baseline files
- Bundle export creates evidence bundles
### Sync Access
- Mirror sync updates upstream caches
- Scheduled jobs refresh OSV database
## Storage Requirements
| Component | Typical Size | Growth Rate |
|-----------|--------------|-------------|
| Corpus (per pair) | 50-500 MB | N/A |
| Mirrors (Debian) | 10-50 GB | Monthly |
| Mirrors (Ubuntu) | 5-20 GB | Monthly |
| Mirrors (Alpine) | 1-5 GB | Monthly |
| OSV Database | 500 MB | Weekly |
| Evidence bundles | 100-500 MB each | Per pair |
| Baselines | < 10 KB each | Per run |
## Related Documentation
- [Ground Truth Corpus Overview](ground-truth-corpus.md)
- [Golden Corpus Maintenance](golden-corpus-maintenance.md)
- [Corpus Ingestion Operations](corpus-ingestion-operations.md)
- [Golden Corpus Operations Runbook](../../runbooks/golden-corpus-operations.md)

492
docs/modules/binary-index/golden-corpus-maintenance.md Normal file
View File

@@ -0,0 +1,492 @@
# Golden Corpus Maintenance
Sprint: SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
Task: GCB-006 - Document corpus folder layout and maintenance procedures
## Overview
This document describes maintenance procedures for the golden corpus, including:
- Mirror synchronization
- Baseline management
- Evidence bundle generation
- Health monitoring
## Mirror Synchronization
### Automated Sync Schedule
Mirror sync should be automated via cron jobs or CI scheduled workflows.
#### Recommended Schedule
| Mirror | Frequency | Rationale |
|--------|-----------|-----------|
| Debian archive | Daily | Security updates published daily |
| Debian buildinfo | Daily | Matches archive updates |
| Ubuntu archive | Daily | Security updates published daily |
| Ubuntu USN index | Hourly | USN metadata changes frequently |
| Alpine secdb | Daily | Less frequent updates |
| OSV database | Hourly | Aggregates multiple sources |
### Sync Scripts
#### Debian Mirror Sync
```bash
#!/bin/bash
# sync-debian-mirrors.sh
# Syncs Debian archives and buildinfo
set -euo pipefail
MIRRORS_ROOT="${MIRRORS_ROOT:-/data/golden-corpus/mirrors}"
DEBIAN_MIRROR="${DEBIAN_MIRROR:-https://snapshot.debian.org}"
BUILDINFO_URL="${BUILDINFO_URL:-https://buildinfos.debian.net}"
# Packages to mirror (security-relevant)
PACKAGES=(openssl curl zlib glibc libxml2 libpng)
# Sync source packages
for pkg in "${PACKAGES[@]}"; do
echo "Syncing Debian sources for: $pkg"
# Create package directory
mkdir -p "$MIRRORS_ROOT/debian/archive/pool/main/${pkg:0:1}/$pkg"
# Download available versions
rsync -avz --progress \
"rsync://snapshot.debian.org/snapshot/debian/pool/main/${pkg:0:1}/$pkg/" \
"$MIRRORS_ROOT/debian/archive/pool/main/${pkg:0:1}/$pkg/"
done
# Sync buildinfo files
for pkg in "${PACKAGES[@]}"; do
echo "Syncing buildinfo for: $pkg"
mkdir -p "$MIRRORS_ROOT/debian/buildinfo/$pkg"
# Use wget to fetch buildinfo index and files
wget -r -np -nH --cut-dirs=2 -P "$MIRRORS_ROOT/debian/buildinfo/$pkg" \
"$BUILDINFO_URL/api/v1/buildinfo/$pkg/" || true
done
echo "Debian mirror sync complete"
date > "$MIRRORS_ROOT/debian/.last-sync"
```
#### Ubuntu Mirror Sync
```bash
#!/bin/bash
# sync-ubuntu-mirrors.sh
# Syncs Ubuntu archives and USN metadata
set -euo pipefail
MIRRORS_ROOT="${MIRRORS_ROOT:-/data/golden-corpus/mirrors}"
UBUNTU_ARCHIVE="https://archive.ubuntu.com/ubuntu"
USN_API="https://ubuntu.com/security/notices.json"
# Sync USN database
echo "Syncing Ubuntu USN database..."
mkdir -p "$MIRRORS_ROOT/ubuntu/usn-index"
curl -sSL "$USN_API" -o "$MIRRORS_ROOT/ubuntu/usn-index/usn-db.json.tmp"
mv "$MIRRORS_ROOT/ubuntu/usn-index/usn-db.json.tmp" "$MIRRORS_ROOT/ubuntu/usn-index/usn-db.json"
# Sync packages (similar to Debian)
PACKAGES=(openssl curl zlib1g libxml2)
for pkg in "${PACKAGES[@]}"; do
echo "Syncing Ubuntu sources for: $pkg"
mkdir -p "$MIRRORS_ROOT/ubuntu/archive/pool/main/${pkg:0:1}/$pkg"
# ... sync logic
done
echo "Ubuntu mirror sync complete"
date > "$MIRRORS_ROOT/ubuntu/.last-sync"
```
#### Alpine SecDB Sync
```bash
#!/bin/bash
# sync-alpine-secdb.sh
# Syncs Alpine security database
set -euo pipefail
MIRRORS_ROOT="${MIRRORS_ROOT:-/data/golden-corpus/mirrors}"
ALPINE_SECDB="https://secdb.alpinelinux.org"
mkdir -p "$MIRRORS_ROOT/alpine/secdb"
# Download all security databases
for branch in v3.17 v3.18 v3.19 v3.20 edge; do
for repo in main community; do
echo "Syncing Alpine secdb: $branch/$repo"
curl -sSL "$ALPINE_SECDB/$branch/$repo.json" \
-o "$MIRRORS_ROOT/alpine/secdb/${branch}-${repo}.json" || true
done
done
echo "Alpine secdb sync complete"
date > "$MIRRORS_ROOT/alpine/.last-sync"
```
#### OSV Database Sync
```bash
#!/bin/bash
# sync-osv.sh
# Syncs OSV vulnerability database
set -euo pipefail
MIRRORS_ROOT="${MIRRORS_ROOT:-/data/golden-corpus/mirrors}"
OSV_URL="https://osv-vulnerabilities.storage.googleapis.com"
mkdir -p "$MIRRORS_ROOT/osv"
# Download full database
echo "Downloading OSV all.zip..."
curl -sSL "$OSV_URL/all.zip" -o "$MIRRORS_ROOT/osv/all.zip.tmp"
mv "$MIRRORS_ROOT/osv/all.zip.tmp" "$MIRRORS_ROOT/osv/all.zip"
# Extract ecosystem-specific databases
for ecosystem in Debian Ubuntu Alpine; do
mkdir -p "$MIRRORS_ROOT/osv/$ecosystem"
unzip -o -q "$MIRRORS_ROOT/osv/all.zip" "$ecosystem/*" -d "$MIRRORS_ROOT/osv/" || true
done
echo "OSV sync complete"
date > "$MIRRORS_ROOT/osv/.last-sync"
```
### Cron Configuration
```cron
# /etc/cron.d/golden-corpus-sync
# Mirror sync jobs
0 */4 * * * corpus /opt/golden-corpus/scripts/sync-debian-mirrors.sh >> /var/log/corpus/debian-sync.log 2>&1
0 */4 * * * corpus /opt/golden-corpus/scripts/sync-ubuntu-mirrors.sh >> /var/log/corpus/ubuntu-sync.log 2>&1
0 6 * * * corpus /opt/golden-corpus/scripts/sync-alpine-secdb.sh >> /var/log/corpus/alpine-sync.log 2>&1
0 * * * * corpus /opt/golden-corpus/scripts/sync-osv.sh >> /var/log/corpus/osv-sync.log 2>&1
# Health check
*/15 * * * * corpus /opt/golden-corpus/scripts/check-mirror-health.sh >> /var/log/corpus/health.log 2>&1
```
## Baseline Management
### When to Update Baselines
Update the KPI baseline when:
1. Algorithm improvements are merged (expected KPI improvement)
2. New corpus pairs are added (may change baseline metrics)
3. False positives/negatives are corrected in ground truth
4. Major version upgrades of analysis tools
### Baseline Update Procedure
#### 1. Run Full Validation
```bash
# Run validation on the full corpus
stella groundtruth validate run \
--matcher semantic-diffing \
--output bench/results/$(date +%Y%m%d%H%M%S).json \
--verbose
```
#### 2. Review Results
```bash
# Check metrics
stella groundtruth validate metrics --run-id latest
# Compare against current baseline
stella groundtruth validate check \
--results bench/results/latest.json \
--baseline bench/baselines/current.json
```
#### 3. Update Baseline
Only if regression check passes or improvements are expected:
```bash
# Archive current baseline
cp bench/baselines/current.json \
bench/baselines/archive/baseline-$(date +%Y%m%d).json
# Update baseline
stella groundtruth baseline update \
--from-results bench/results/latest.json \
--output bench/baselines/current.json \
--description "Post algorithm-v2.3 update" \
--source "$(git rev-parse HEAD)"
```
#### 4. Commit and Document
```bash
# Commit the baseline update
git add bench/baselines/
git commit -m "chore(bench): update golden corpus baseline
Reason: Algorithm v2.3 improvements
Previous baseline: baseline-20260115.json
Metrics:
- Precision: 0.95 -> 0.97 (+2pp)
- Recall: 0.92 -> 0.94 (+2pp)
- FN Rate: 0.08 -> 0.06 (-2pp)
- Determinism: 100%
- TTFRP p95: 150ms -> 140ms (-7%)"
git push
```
### Baseline Rollback
If a baseline update causes issues:
```bash
# Restore previous baseline
cp bench/baselines/archive/baseline-20260115.json \
bench/baselines/current.json
git add bench/baselines/current.json
git commit -m "revert(bench): rollback baseline to 20260115"
git push
```
## Evidence Bundle Generation
### Manual Bundle Export
```bash
# Export bundle for specific packages
stella groundtruth bundle export \
--packages openssl,curl,zlib \
--distros debian,ubuntu \
--output evidence/security-bundle-$(date +%Y%m%d).tar.gz \
--sign-with-cosign \
--include-debug \
--include-kpis \
--include-timestamps
```
### Automated Bundle Generation
Schedule bundle generation for compliance reporting:
```bash
#!/bin/bash
# generate-compliance-bundles.sh
# Run monthly for audit evidence
set -euo pipefail
EVIDENCE_DIR="/data/golden-corpus/evidence"
MONTH=$(date +%Y%m)
# Generate bundles for each distro
for distro in debian ubuntu alpine; do
stella groundtruth bundle export \
--distros "$distro" \
--packages all \
--output "$EVIDENCE_DIR/$distro-bundle-$MONTH.tar.gz" \
--sign-with-cosign \
--include-kpis \
--include-timestamps
done
# Create manifest
echo "{\"month\": \"$MONTH\", \"bundles\": [\"debian\", \"ubuntu\", \"alpine\"]}" \
> "$EVIDENCE_DIR/manifest-$MONTH.json"
```
### Bundle Verification
Always verify bundles after generation:
```bash
# Verify bundle integrity
stella groundtruth bundle import \
--input evidence/security-bundle-20260122.tar.gz \
--verify \
--trusted-keys /etc/stellaops/trusted-keys.pub \
--trust-profile /etc/stellaops/trust-profiles/global.json \
--output verification-report.md
```
## Health Monitoring
### Doctor Checks
Run Doctor checks regularly to validate corpus health:
```bash
# Run all corpus-related checks
stella doctor --check "check.binaryanalysis.corpus.*"
# Specific checks
stella doctor --check check.binaryanalysis.corpus.mirror.freshness
stella doctor --check check.binaryanalysis.corpus.kpi.baseline
stella doctor --check check.binaryanalysis.debuginfod.availability
```
### Health Check Script
```bash
#!/bin/bash
# check-mirror-health.sh
# Validates mirror freshness and connectivity
set -euo pipefail
MIRRORS_ROOT="${MIRRORS_ROOT:-/data/golden-corpus/mirrors}"
STALE_THRESHOLD_DAYS=7
ALERTS=""
check_mirror() {
local mirror_name=$1
local last_sync_file=$2
local max_age=$3
if [[ ! -f "$last_sync_file" ]]; then
ALERTS+="CRITICAL: $mirror_name has never been synced\n"
return
fi
local last_sync=$(cat "$last_sync_file")
local last_sync_epoch=$(date -d "$last_sync" +%s)
local now_epoch=$(date +%s)
local age_days=$(( (now_epoch - last_sync_epoch) / 86400 ))
if [[ $age_days -gt $max_age ]]; then
ALERTS+="WARNING: $mirror_name is $age_days days old (threshold: $max_age)\n"
fi
}
# Check each mirror
check_mirror "Debian" "$MIRRORS_ROOT/debian/.last-sync" $STALE_THRESHOLD_DAYS
check_mirror "Ubuntu" "$MIRRORS_ROOT/ubuntu/.last-sync" $STALE_THRESHOLD_DAYS
check_mirror "Alpine" "$MIRRORS_ROOT/alpine/.last-sync" $STALE_THRESHOLD_DAYS
check_mirror "OSV" "$MIRRORS_ROOT/osv/.last-sync" 1 # OSV should be hourly
# Check connectivity
for url in \
"https://snapshot.debian.org" \
"https://buildinfos.debian.net" \
"https://ubuntu.com/security/notices.json" \
"https://secdb.alpinelinux.org"; do
if ! curl -sSf --connect-timeout 5 "$url" > /dev/null 2>&1; then
ALERTS+="ERROR: Cannot reach $url\n"
fi
done
# Report results
if [[ -n "$ALERTS" ]]; then
echo -e "Golden Corpus Health Issues:\n$ALERTS"
# Send alert (customize for your alerting system)
# curl -X POST -d "$ALERTS" https://alerts.example.com/webhook
exit 1
fi
echo "All mirrors healthy at $(date)"
```
### Monitoring Metrics
Export these metrics to your monitoring system:
| Metric | Description | Alert Threshold |
|--------|-------------|-----------------|
| `corpus.mirrors.age_seconds` | Time since last mirror sync | > 7 days |
| `corpus.pairs.total` | Total number of security pairs | N/A (info) |
| `corpus.validation.precision` | Latest precision rate | < baseline - 0.01 |
| `corpus.validation.recall` | Latest recall rate | < baseline - 0.01 |
| `corpus.validation.determinism` | Deterministic replay rate | < 1.0 |
| `corpus.bundle.count` | Number of evidence bundles | N/A (info) |
| `corpus.baseline.age_days` | Days since baseline update | > 30 days |
### Prometheus Metrics Example
```yaml
# prometheus-corpus-metrics.yaml
groups:
- name: golden-corpus
rules:
- alert: CorpusMirrorStale
expr: corpus_mirror_age_seconds > 604800 # 7 days
labels:
severity: warning
annotations:
summary: "Corpus mirror {{ $labels.mirror }} is stale"
- alert: CorpusRegressionDetected
expr: corpus_validation_precision < corpus_baseline_precision - 0.01
labels:
severity: critical
annotations:
summary: "Precision regression detected in golden corpus validation"
- alert: CorpusDeterminismFailure
expr: corpus_validation_determinism < 1.0
labels:
severity: critical
annotations:
summary: "Non-deterministic replay detected"
```
## Cleanup and Archival
### Archive Old Results
```bash
#!/bin/bash
# archive-old-results.sh
# Archives results older than 90 days
RESULTS_DIR="/data/golden-corpus/bench/results"
ARCHIVE_DIR="/data/golden-corpus/bench/archive"
AGE_DAYS=90
mkdir -p "$ARCHIVE_DIR"
find "$RESULTS_DIR" -name "*.json" -mtime +$AGE_DAYS -exec \
mv {} "$ARCHIVE_DIR/" \;
# Compress archived results by month
cd "$ARCHIVE_DIR"
for month in $(ls *.json | cut -c1-6 | sort -u); do
tar -czf "results-$month.tar.gz" "${month}"*.json && \
rm -f "${month}"*.json
done
```
### Prune Old Baselines
Keep only the last N baselines:
```bash
#!/bin/bash
# prune-baselines.sh
# Keeps only the 10 most recent baseline archives
BASELINE_ARCHIVE="/data/golden-corpus/bench/baselines/archive"
KEEP_COUNT=10
cd "$BASELINE_ARCHIVE"
ls -t baseline-*.json | tail -n +$((KEEP_COUNT + 1)) | xargs -r rm -f
```
## Related Documentation
- [Golden Corpus Folder Layout](golden-corpus-layout.md)
- [Ground Truth Corpus Overview](ground-truth-corpus.md)
- [Golden Corpus Operations Runbook](../../runbooks/golden-corpus-operations.md)

10
docs/modules/cli/README.md
View File

@@ -23,10 +23,12 @@ The `stella` CLI is the operator-facing Swiss army knife for scans, exports, pol
- Versioned command docs in `docs/modules/cli/guides`.
- Plugin catalogue in `plugins/cli/**` (restart-only).
## Related resources
- ./guides/20_REFERENCE.md
- ./guides/cli-reference.md
- ./guides/policy.md
## Related resources
- ./guides/20_REFERENCE.md
- ./guides/cli-reference.md
- ./guides/commands/analytics.md
- ./guides/policy.md
- ./guides/trust-profiles.md
## Backlog references
- DOCS-CLI-OBS-52-001 / DOCS-CLI-FORENSICS-53-001 in ../../TASKS.md.

11
docs/modules/cli/cli-vs-ui-parity.md
View File

@@ -51,10 +51,11 @@ Status key:
| UI capability | CLI command(s) | Status | Notes / Tasks |
|---------------|----------------|--------|---------------|
| Advisory observations search | `stella vuln observations` | ✅ Available | Implemented via `BuildVulnCommand`. |
| Advisory linkset export | `stella advisory linkset show/export` | 🟩 Planned | `CLI-LNM-22-001`. |
| VEX observations / linksets | `stella vex obs get/linkset show` | 🟩 Planned | `CLI-LNM-22-002`. |
| SBOM overlay export | `stella sbom overlay apply/export` | 🟩 Planned | Scoped to upcoming SBOM CLI sprint (`SBOM-CONSOLE-23-001/002` + CLI backlog). |
| Advisory observations search | `stella vuln observations` | ✅ Available | Implemented via `BuildVulnCommand`. |
| Advisory linkset export | `stella advisory linkset show/export` | 🟩 Planned | `CLI-LNM-22-001`. |
| VEX observations / linksets | `stella vex obs get/linkset show` | 🟩 Planned | `CLI-LNM-22-002`. |
| SBOM overlay export | `stella sbom overlay apply/export` | 🟩 Planned | Scoped to upcoming SBOM CLI sprint (`SBOM-CONSOLE-23-001/002` + CLI backlog). |
| SBOM Lake analytics (`/analytics/sbom-lake`) | `stella analytics sbom-lake <subcommand>` | ✅ Available | CLI guide at `docs/modules/cli/guides/commands/analytics.md` (SPRINT_20260120_032). |
---
@@ -151,5 +152,5 @@ The script should emit a parity report that feeds into the Downloads workspace (
---
*Last updated: 2025-10-28 (Sprint 23).*
*Last updated: 2026-01-20 (Sprint 20260120).*

104
docs/modules/cli/contracts/cli-spec-v1.yaml
View File

@@ -1,5 +1,5 @@
version: 1
generated: 2025-12-01T00:00:00Z
generated: 2026-01-20T00:00:00Z
compatibility:
policy: "SemVer-like: commands/flags/exitCodes are backwards compatible within major version."
deprecation:
@@ -38,6 +38,108 @@ commands:
0: success
4: auth-misconfigured
5: token-invalid
- name: analytics
subcommands:
- name: sbom-lake
subcommands:
- name: suppliers
formats: [table, json, csv]
flags:
- name: environment
required: false
- name: limit
required: false
- name: format
required: false
values: [table, json, csv]
- name: output
required: false
exitCodes:
0: success
1: error
- name: licenses
formats: [table, json, csv]
flags:
- name: environment
required: false
- name: limit
required: false
- name: format
required: false
values: [table, json, csv]
- name: output
required: false
exitCodes:
0: success
1: error
- name: vulnerabilities
formats: [table, json, csv]
flags:
- name: environment
required: false
- name: min-severity
required: false
values: [critical, high, medium, low]
- name: limit
required: false
- name: format
required: false
values: [table, json, csv]
- name: output
required: false
exitCodes:
0: success
1: error
- name: backlog
formats: [table, json, csv]
flags:
- name: environment
required: false
- name: limit
required: false
- name: format
required: false
values: [table, json, csv]
- name: output
required: false
exitCodes:
0: success
1: error
- name: attestation-coverage
formats: [table, json, csv]
flags:
- name: environment
required: false
- name: limit
required: false
- name: format
required: false
values: [table, json, csv]
- name: output
required: false
exitCodes:
0: success
1: error
- name: trends
formats: [table, json, csv]
flags:
- name: environment
required: false
- name: days
required: false
- name: series
required: false
values: [vulnerabilities, components, all]
- name: limit
required: false
- name: format
required: false
values: [table, json, csv]
- name: output
required: false
exitCodes:
0: success
1: error
telemetry:
defaultEnabled: false
envVars:

47
docs/modules/cli/guides/commands/analytics.md Normal file
View File

@@ -0,0 +1,47 @@
# stella analytics - Command Guide
## Commands
- `stella analytics sbom-lake suppliers [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake licenses [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake vulnerabilities [--environment <env>] [--min-severity <level>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake backlog [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake attestation-coverage [--environment <env>] [--limit <n>] [--format table|json|csv] [--output <path>]`
- `stella analytics sbom-lake trends [--environment <env>] [--days <n>] [--series vulnerabilities|components|all] [--limit <n>] [--format table|json|csv] [--output <path>]`
## Flags (common)
- `--format`: Output format for rendering (`table`, `json`, `csv`).
- `--output`: Write output to a file path instead of stdout.
- `--limit`: Cap the number of rows returned.
- `--environment`: Filter by environment name.
## SBOM lake notes
- Endpoints require the `analytics.read` scope.
- `--min-severity` accepts `critical`, `high`, `medium`, `low`.
- `--series` controls trend output (`vulnerabilities`, `components`, `all`).
- Tables use deterministic ordering (severity and counts first, then names).
## Examples
```bash
# Top suppliers
stella analytics sbom-lake suppliers --limit 20
# License distribution as CSV (prod)
stella analytics sbom-lake licenses --environment prod --format csv --output licenses.csv
# Vulnerability exposure in prod (high+)
stella analytics sbom-lake vulnerabilities --environment prod --min-severity high
# Fixable backlog with table output
stella analytics sbom-lake backlog --environment prod --limit 50
# Attestation coverage in staging, JSON output
stella analytics sbom-lake attestation-coverage --environment stage --format json
# 30-day trend snapshot (both series)
stella analytics sbom-lake trends --days 30 --series all --format csv --output trends.csv
```
## Offline/verification note
- If analytics exports arrive via offline bundles, verify the bundle first with
`stella bundle verify` before importing data into downstream reports.

596
docs/modules/cli/guides/commands/reference.md
View File

@@ -16,6 +16,7 @@ graph TD
CLI --> EXPLAIN[Explainability]
CLI --> VEX[VEX & Decisioning]
CLI --> SBOM[SBOM Operations]
CLI --> ANALYTICS[Analytics & Insights]
CLI --> REPORT[Reporting & Export]
CLI --> OFFLINE[Offline Operations]
CLI --> SYSTEM[System & Config]
@@ -742,6 +743,601 @@ stella sbom merge --sbom <path1> --sbom <path2> [--output <path>] [--verbose]
---
## Analytics Commands
### stella analytics sbom-lake
Query SBOM lake analytics views (suppliers, licenses, vulnerabilities, backlog,
attestation coverage, trends).
**Usage:**
```bash
stella analytics sbom-lake <subcommand> [options]
```
**Subcommands:**
- `suppliers` - Supplier concentration
- `licenses` - License distribution
- `vulnerabilities` - CVE exposure (VEX-adjusted)
- `backlog` - Fixable vulnerability backlog
- `attestation-coverage` - Provenance/SLSA coverage
- `trends` - Time-series trends (vulnerabilities/components)
**Common options:**
| Option | Description |
|--------|-------------|
| `--environment <env>` | Filter to a specific environment |
| `--min-severity <level>` | Minimum severity (`critical`, `high`, `medium`, `low`) |
| `--days <n>` | Lookback window in days (trends only) |
| `--series <name>` | Trend series (`vulnerabilities`, `components`, `all`) |
| `--limit <n>` | Maximum number of rows |
| `--format <fmt>` | Output format: `table`, `json`, `csv` |
| `--output <path>` | Output file path |
**Example:**
```bash
stella analytics sbom-lake vulnerabilities --environment prod --min-severity high --format csv --output vuln.csv
```
---
## Ground-Truth Corpus Commands
### stella groundtruth
Manage ground-truth corpus for patch-paired binary verification. The corpus supports
precision validation of security advisories by maintaining symbol and binary pairs
from upstream sources.
**Sprint:** SPRINT_20260121_035_BinaryIndex_golden_corpus_connectors_cli
**Usage:**
```bash
stella groundtruth <subcommand> [options]
```
**Subcommands:**
- `sources` - Manage symbol source connectors
- `symbols` - Query and search symbols in the corpus
- `pairs` - Manage security pairs (vuln/patch binary pairs)
- `validate` - Run validation and view metrics
---
### stella groundtruth sources
Manage upstream symbol source connectors.
**Usage:**
```bash
stella groundtruth sources <command> [options]
```
**Subcommands:**
#### stella groundtruth sources list
List available symbol source connectors.
```bash
stella groundtruth sources list [--output-format table|json] [--verbose]
```
**Output:**
```
ID Display Name Status Last Sync
------------------------------------------------------------------------------------------
debuginfod-fedora Fedora Debuginfod Enabled 2026-01-22T10:00:00Z
debuginfod-ubuntu Ubuntu Debuginfod Enabled 2026-01-22T10:00:00Z
ddeb-ubuntu Ubuntu ddebs Enabled 2026-01-22T09:30:00Z
buildinfo-debian Debian Buildinfo Enabled 2026-01-22T08:00:00Z
secdb-alpine Alpine SecDB Enabled 2026-01-22T06:00:00Z
```
#### stella groundtruth sources enable
Enable a symbol source connector.
```bash
stella groundtruth sources enable <source> [--verbose]
```
**Arguments:**
- `<source>` - Source connector ID (e.g., `debuginfod-fedora`)
**Example:**
```bash
stella groundtruth sources enable debuginfod-fedora
```
#### stella groundtruth sources disable
Disable a symbol source connector.
```bash
stella groundtruth sources disable <source> [--verbose]
```
#### stella groundtruth sources sync
Synchronize symbol sources from upstream.
```bash
stella groundtruth sources sync [--source <id>] [--full] [--verbose]
```
**Options:**
| Option | Description |
|--------|-------------|
| `--source <id>` | Source connector ID (all if not specified) |
| `--full` | Perform a full sync instead of incremental |
**Example:**
```bash
# Incremental sync of all sources
stella groundtruth sources sync
# Full sync of Debian buildinfo
stella groundtruth sources sync --source buildinfo-debian --full
```
---
### stella groundtruth symbols
Query and search symbols in the corpus.
**Usage:**
```bash
stella groundtruth symbols <command> [options]
```
#### stella groundtruth symbols lookup
Lookup symbols by debug ID (build-id).
```bash
stella groundtruth symbols lookup --debug-id <id> [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Required |
|--------|-------|-------------|----------|
| `--debug-id` | `-d` | Debug ID (build-id) to lookup | Yes |
| `--output-format` | `-O` | Output format: `table`, `json` | No |
**Example:**
```bash
stella groundtruth symbols lookup --debug-id 7f8a9b2c4d5e6f1a --output-format json
```
**Output (table):**
```
Binary: libcrypto.so.3
Architecture: x86_64
Distribution: debian-bookworm
Package: openssl@3.0.11-1
Symbol Count: 4523
Sources: debuginfod-fedora, buildinfo-debian
```
#### stella groundtruth symbols search
Search symbols by package or distribution.
```bash
stella groundtruth symbols search [--package <name>] [--distro <distro>] [--limit <n>] [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--package` | `-p` | Package name to search for | - |
| `--distro` | | Distribution filter (debian, ubuntu, alpine) | - |
| `--limit` | `-l` | Maximum results | 20 |
**Example:**
```bash
stella groundtruth symbols search --package openssl --distro debian --limit 50
```
---
### stella groundtruth pairs
Manage security pairs (vulnerable/patched binary pairs) in the corpus.
**Usage:**
```bash
stella groundtruth pairs <command> [options]
```
#### stella groundtruth pairs create
Create a new security pair.
```bash
stella groundtruth pairs create --cve <cve-id> --vuln-pkg <pkg=ver> --patch-pkg <pkg=ver> [--distro <distro>] [--verbose]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--cve` | CVE identifier | Yes |
| `--vuln-pkg` | Vulnerable package (name=version) | Yes |
| `--patch-pkg` | Patched package (name=version) | Yes |
| `--distro` | Distribution (e.g., `debian-bookworm`) | No |
**Example:**
```bash
stella groundtruth pairs create \
--cve CVE-2024-1234 \
--vuln-pkg openssl=3.0.10-1 \
--patch-pkg openssl=3.0.11-1 \
--distro debian-bookworm
```
#### stella groundtruth pairs list
List security pairs in the corpus.
```bash
stella groundtruth pairs list [--cve <pattern>] [--package <name>] [--limit <n>] [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--cve` | | Filter by CVE (supports wildcards: `CVE-2024-*`) | - |
| `--package` | `-p` | Filter by package name | - |
| `--limit` | `-l` | Maximum results | 50 |
**Example:**
```bash
stella groundtruth pairs list --cve CVE-2024-* --package openssl --limit 100
```
**Output:**
```
Pair ID CVE Package Vuln Version Patch Version
-------------------------------------------------------------------------------
pair-001 CVE-2024-1234 openssl 3.0.10-1 3.0.11-1
pair-002 CVE-2024-5678 curl 8.4.0-1 8.5.0-1
```
#### stella groundtruth pairs delete
Delete a security pair from the corpus.
```bash
stella groundtruth pairs delete <pair-id> [--force] [--verbose]
```
**Options:**
| Option | Alias | Description |
|--------|-------|-------------|
| `--force` | `-f` | Skip confirmation prompt |
---
### stella groundtruth validate
Run validation harness against security pairs.
**Usage:**
```bash
stella groundtruth validate <command> [options]
```
#### stella groundtruth validate run
Run validation on security pairs.
```bash
stella groundtruth validate run [--pairs <pattern>] [--matcher <type>] [--output <path>] [--parallel <n>] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--pairs` | `-p` | Pair filter pattern (e.g., `openssl:CVE-2024-*`) | all |
| `--matcher` | `-m` | Matcher type: `semantic-diffing`, `hash-based`, `hybrid` | `semantic-diffing` |
| `--output` | `-o` | Output file for validation report | - |
| `--parallel` | | Maximum parallel validations | 4 |
**Example:**
```bash
stella groundtruth validate run \
--pairs "openssl:CVE-2024-*" \
--matcher semantic-diffing \
--parallel 8 \
--output validation-report.md
```
**Output:**
```
Validating pairs: 10/10
Validation complete. Run ID: vr-20260122100532
Function Match Rate: 94.2%
False-Negative Rate: 2.1%
SBOM Hash Stability: 3/3
Report written to: validation-report.md
```
#### stella groundtruth validate metrics
View metrics for a validation run.
```bash
stella groundtruth validate metrics --run-id <id> [--output-format table|json] [--verbose]
```
**Options:**
| Option | Alias | Description | Required |
|--------|-------|-------------|----------|
| `--run-id` | `-r` | Validation run ID | Yes |
**Example:**
```bash
stella groundtruth validate metrics --run-id vr-20260122100532 --output-format json
```
**Output (table):**
```
Run ID: vr-20260122100532
Duration: 2026-01-22T10:00:00Z - 2026-01-22T10:15:32Z
Pairs: 48/50 successful
Function Match Rate: 94.2%
False-Negative Rate: 2.1%
SBOM Hash Stability: 3/3
Verify Time (p50/p95): 423ms / 1.2s
```
#### stella groundtruth validate export
Export validation report.
```bash
stella groundtruth validate export --run-id <id> --output <path> [--format <fmt>] [--verbose]
```
**Options:**
| Option | Alias | Description | Default |
|--------|-------|-------------|---------|
| `--run-id` | `-r` | Validation run ID | (required) |
| `--output` | `-o` | Output file path | (required) |
| `--format` | `-f` | Export format: `markdown`, `html`, `json` | `markdown` |
**Example:**
```bash
stella groundtruth validate export \
--run-id vr-20260122100532 \
--format markdown \
--output validation-report.md
```
**See Also:** [Ground-Truth CLI Guide](../ground-truth-cli.md)
---
### stella groundtruth bundle
Manage evidence bundles for offline verification of patch provenance.
**Sprint:** SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
**Usage:**
```bash
stella groundtruth bundle <command> [options]
```
**Subcommands:**
- `export` - Create evidence bundles for air-gapped environments
- `import` - Import and verify evidence bundles
#### stella groundtruth bundle export
Export evidence bundles containing pre/post binaries, SBOMs, delta-sig predicates, and timestamps.
```bash
stella groundtruth bundle export [options]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--packages <list>` | Comma-separated package names (e.g., `openssl,curl`) | Yes |
| `--distros <list>` | Comma-separated distributions (e.g., `debian,ubuntu`) | Yes |
| `--output <path>` | Output bundle path (.tar.gz or .oci.tar) | Yes |
| `--sign-with <signer>` | Signing method: `cosign`, `sigstore`, `none` | No |
| `--include-debug` | Include debug symbols | No |
| `--include-kpis` | Include KPI validation results | No |
| `--include-timestamps` | Include RFC 3161 timestamps | No |
**Example:**
```bash
stella groundtruth bundle export \
--packages openssl,zlib,glibc \
--distros debian,fedora \
--output evidence/security-bundle.tar.gz \
--sign-with cosign \
--include-debug \
--include-kpis \
--include-timestamps
```
**Exit Codes:**
- `0` - Bundle created successfully
- `1` - Bundle creation failed
- `2` - Invalid input or configuration error
#### stella groundtruth bundle import
Import and verify evidence bundles in air-gapped environments.
```bash
stella groundtruth bundle import [options]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--input <path>` | Input bundle path | Yes |
| `--verify-signature` | Verify bundle signatures | No |
| `--trusted-keys <path>` | Path to trusted public keys | No |
| `--trust-profile <path>` | Trust profile for verification | No |
| `--output <path>` | Output verification report | No |
| `--format <fmt>` | Report format: `markdown`, `json`, `html` | No |
**Example:**
```bash
stella groundtruth bundle import \
--input symbol-bundle.tar.gz \
--verify-signature \
--trusted-keys /etc/stellaops/trusted-keys.pub \
--trust-profile /etc/stellaops/trust-profiles/global.json \
--output verification-report.md
```
**Verification Steps:**
1. Validate bundle manifest signature
2. Verify all blob digests match manifest
3. Validate DSSE envelope signatures against trusted keys
4. Verify RFC 3161 timestamps against trusted TSA certificates
5. Run IR matcher to confirm patched functions
6. Verify SBOM canonical hash matches signed predicate
7. Output verification report with KPI line items
**Exit Codes:**
- `0` - All verifications passed
- `1` - One or more verifications failed
- `2` - Invalid input or configuration error
---
### stella groundtruth validate check
Check KPI regression against baseline thresholds.
**Sprint:** SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
```bash
stella groundtruth validate check [options]
```
**Options:**
| Option | Description | Default |
|--------|-------------|---------|
| `--results <path>` | Path to validation results JSON | (required) |
| `--baseline <path>` | Path to baseline JSON | (required) |
| `--precision-threshold <pp>` | Max precision drop (percentage points) | 0.01 |
| `--recall-threshold <pp>` | Max recall drop (percentage points) | 0.01 |
| `--fn-rate-threshold <pp>` | Max FN rate increase (percentage points) | 0.01 |
| `--determinism-threshold <rate>` | Min determinism rate | 1.0 |
| `--ttfrp-threshold <pct>` | Max TTFRP p95 increase (percentage) | 0.20 |
| `--output <path>` | Output report path | stdout |
| `--format <fmt>` | Report format: `markdown`, `json` | `markdown` |
**Example:**
```bash
stella groundtruth validate check \
--results bench/results/20260122.json \
--baseline bench/baselines/current.json \
--precision-threshold 0.01 \
--recall-threshold 0.01 \
--fn-rate-threshold 0.01 \
--determinism-threshold 1.0 \
--output regression-report.md
```
**Regression Gates:**
| Metric | Threshold | Action |
|--------|-----------|--------|
| Precision | Drops > threshold | Fail |
| Recall | Drops > threshold | Fail |
| False-negative rate | Increases > threshold | Fail |
| Deterministic replay | Drops below threshold | Fail |
| TTFRP p95 | Increases > threshold | Warn |
**Exit Codes:**
- `0` - All gates passed
- `1` - One or more gates failed
- `2` - Invalid input or configuration error
---
### stella groundtruth baseline
Manage KPI baselines for regression detection.
**Sprint:** SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
**Usage:**
```bash
stella groundtruth baseline <command> [options]
```
**Subcommands:**
- `update` - Update baseline from validation results
- `show` - Display baseline contents
#### stella groundtruth baseline update
Update baseline from validation results.
```bash
stella groundtruth baseline update [options]
```
**Options:**
| Option | Description | Required |
|--------|-------------|----------|
| `--from-results <path>` | Path to validation results JSON | Yes |
| `--output <path>` | Output baseline path | Yes |
| `--description <text>` | Description for the baseline update | No |
| `--source <commit>` | Source commit SHA for traceability | No |
**Example:**
```bash
stella groundtruth baseline update \
--from-results bench/results/20260122.json \
--output bench/baselines/current.json \
--description "Post algorithm-v2.3 update" \
--source "$(git rev-parse HEAD)"
```
#### stella groundtruth baseline show
Display baseline contents.
```bash
stella groundtruth baseline show --baseline <path> [--format table|json]
```
**Options:**
| Option | Description | Default |
|--------|-------------|---------|
| `--baseline <path>` | Path to baseline JSON | (required) |
| `--format` | Output format: `table`, `json` | `table` |
**Output (table):**
```
Baseline ID: baseline-20260122120000
Created: 2026-01-22T12:00:00Z
Source: abc123def456
Description: Post-semantic-diffing-v2 baseline
KPIs:
Precision: 0.9500
Recall: 0.9200
False Negative Rate: 0.0800
Determinism: 1.0000
TTFRP p95: 150ms
```
**See Also:** [Ground-Truth CLI Guide](../ground-truth-cli.md)
---
## Reporting & Export Commands
### stella report

351
docs/modules/cli/guides/ground-truth-cli.md Normal file
View File

@@ -0,0 +1,351 @@
# Ground-Truth Corpus CLI Guide
**Sprint:** SPRINT_20260121_035_BinaryIndex_golden_corpus_connectors_cli
## Overview
The `stella groundtruth` command group provides CLI access to the ground-truth corpus for patch-paired binary verification. This corpus enables precision validation of security advisories by maintaining symbol and binary pairs from upstream distribution sources.
## Use Cases
- **Security teams**: Validate patch presence in production binaries
- **Compliance auditors**: Generate evidence bundles for air-gapped verification
- **DevSecOps**: Integrate corpus validation into CI/CD pipelines
- **Researchers**: Query symbol databases for vulnerability analysis
## Prerequisites
- Stella CLI installed and configured
- Backend connectivity to Platform service (or offline bundle)
- For sync operations: network access to upstream sources
## Command Structure
```
stella groundtruth
├── sources # Manage symbol source connectors
│ ├── list # List available connectors
│ ├── enable # Enable a connector
│ ├── disable # Disable a connector
│ └── sync # Sync from upstream
├── symbols # Query symbols in corpus
│ ├── lookup # Lookup by debug ID
│ └── search # Search by package/distro
├── pairs # Manage security pairs
│ ├── create # Create vuln/patch pair
│ ├── list # List existing pairs
│ └── delete # Remove a pair
└── validate # Run validation harness
├── run # Execute validation
├── metrics # View run metrics
└── export # Export report
```
## Source Connectors
The ground-truth corpus ingests data from multiple upstream sources:
| Connector ID | Distribution | Data Type | Description |
|--------------|--------------|-----------|-------------|
| `debuginfod-fedora` | Fedora | Debug symbols | ELF debuginfo via debuginfod protocol |
| `debuginfod-ubuntu` | Ubuntu | Debug symbols | ELF debuginfo via debuginfod protocol |
| `ddeb-ubuntu` | Ubuntu | Debug packages | `.ddeb` debug symbol packages |
| `buildinfo-debian` | Debian | Build metadata | `.buildinfo` reproducibility records |
| `secdb-alpine` | Alpine | Security DB | `secfixes` YAML from APKBUILD |
### List Sources
```bash
stella groundtruth sources list
# Output:
ID Display Name Status Last Sync
------------------------------------------------------------------------------------------
debuginfod-fedora Fedora Debuginfod Enabled 2026-01-22T10:00:00Z
debuginfod-ubuntu Ubuntu Debuginfod Enabled 2026-01-22T10:00:00Z
ddeb-ubuntu Ubuntu ddebs Enabled 2026-01-22T09:30:00Z
buildinfo-debian Debian Buildinfo Enabled 2026-01-22T08:00:00Z
secdb-alpine Alpine SecDB Enabled 2026-01-22T06:00:00Z
```
### Enable/Disable Sources
```bash
# Enable a source connector
stella groundtruth sources enable debuginfod-fedora
# Disable a source connector (stops future syncs)
stella groundtruth sources disable debuginfod-fedora
```
### Sync Sources
```bash
# Incremental sync of all enabled sources
stella groundtruth sources sync
# Full sync of a specific source
stella groundtruth sources sync --source buildinfo-debian --full
# Sync with verbose output
stella groundtruth sources sync --source ddeb-ubuntu -v
```
## Symbol Operations
### Lookup by Debug ID
Query symbols using the ELF GNU Build-ID or equivalent identifier:
```bash
# Lookup by build-id
stella groundtruth symbols lookup --debug-id 7f8a9b2c4d5e6f1a
# JSON output
stella groundtruth symbols lookup --debug-id 7f8a9b2c4d5e6f1a --output-format json
```
**Example output:**
```
Binary: libcrypto.so.3
Architecture: x86_64
Distribution: debian-bookworm
Package: openssl@3.0.11-1
Symbol Count: 4523
Sources: debuginfod-fedora, buildinfo-debian
```
### Search Symbols
Search across the corpus by package name or distribution:
```bash
# Search by package
stella groundtruth symbols search --package openssl
# Filter by distribution
stella groundtruth symbols search --package openssl --distro debian
# Limit results
stella groundtruth symbols search --package curl --limit 100
```
## Security Pairs
Security pairs link vulnerable and patched binary versions for a specific CVE.
### Create a Pair
```bash
stella groundtruth pairs create \
--cve CVE-2024-1234 \
--vuln-pkg openssl=3.0.10-1 \
--patch-pkg openssl=3.0.11-1 \
--distro debian-bookworm
```
### List Pairs
```bash
# List all pairs
stella groundtruth pairs list
# Filter by CVE pattern
stella groundtruth pairs list --cve "CVE-2024-*"
# Filter by package
stella groundtruth pairs list --package openssl --limit 50
# JSON output
stella groundtruth pairs list --output-format json
```
**Example output:**
```
Pair ID CVE Package Vuln Version Patch Version
-------------------------------------------------------------------------------
pair-001 CVE-2024-1234 openssl 3.0.10-1 3.0.11-1
pair-002 CVE-2024-5678 curl 8.4.0-1 8.5.0-1
```
### Delete a Pair
```bash
# Delete with confirmation prompt
stella groundtruth pairs delete pair-001
# Skip confirmation
stella groundtruth pairs delete pair-001 --force
```
## Validation Harness
The validation harness runs end-to-end verification against security pairs.
### Run Validation
```bash
# Validate all pairs
stella groundtruth validate run
# Validate specific pairs (pattern match)
stella groundtruth validate run --pairs "openssl:CVE-2024-*"
# Use specific matcher
stella groundtruth validate run --matcher semantic-diffing
# Parallel validation with report output
stella groundtruth validate run \
--pairs "curl:*" \
--parallel 8 \
--output validation-report.md
```
**Matcher types:**
| Matcher | Description |
|---------|-------------|
| `semantic-diffing` | IR-level semantic comparison (default) |
| `hash-based` | Function hash matching |
| `hybrid` | Combined semantic + hash approach |
### View Metrics
```bash
stella groundtruth validate metrics --run-id vr-20260122100532
# JSON output
stella groundtruth validate metrics --run-id vr-20260122100532 --output-format json
```
**Example output:**
```
Run ID: vr-20260122100532
Duration: 2026-01-22T10:00:00Z - 2026-01-22T10:15:32Z
Pairs: 48/50 successful
Function Match Rate: 94.2%
False-Negative Rate: 2.1%
SBOM Hash Stability: 3/3
Verify Time (p50/p95): 423ms / 1.2s
```
### Export Reports
```bash
# Export as Markdown
stella groundtruth validate export \
--run-id vr-20260122100532 \
--format markdown \
--output report.md
# Export as HTML
stella groundtruth validate export \
--run-id vr-20260122100532 \
--format html \
--output report.html
# Export as JSON (machine-readable)
stella groundtruth validate export \
--run-id vr-20260122100532 \
--format json \
--output report.json
```
## CI/CD Integration
### GitHub Actions Example
```yaml
name: Corpus Validation
on:
schedule:
- cron: '0 6 * * 1' # Weekly on Monday
jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: Sync corpus sources
run: stella groundtruth sources sync
- name: Run validation
run: |
stella groundtruth validate run \
--matcher semantic-diffing \
--parallel 4 \
--output validation-${{ github.run_id }}.md
- name: Check metrics
run: |
MATCH_RATE=$(stella groundtruth validate metrics --run-id $(cat run-id.txt) --output-format json | jq '.functionMatchRate')
if (( $(echo "$MATCH_RATE < 90" | bc -l) )); then
echo "Match rate below threshold: $MATCH_RATE%"
exit 1
fi
```
### GitLab CI Example
```yaml
corpus-validation:
stage: verify
script:
- stella groundtruth sources sync --source buildinfo-debian
- stella groundtruth validate run --pairs "openssl:*" --output report.md
artifacts:
paths:
- report.md
expire_in: 1 week
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
```
## Offline Usage
For air-gapped environments, use offline bundles:
```bash
# Export corpus for offline use
stella bundle export \
--include-corpus \
--output corpus-bundle-$(date +%F).tar.gz
# Import on air-gapped system
stella bundle import --package corpus-bundle-2026-01-22.tar.gz
# Run validation offline
stella groundtruth validate run --offline
```
## Troubleshooting
### Common Issues
**Sync fails with network error:**
```bash
# Check source status
stella groundtruth sources list
# Retry with verbose output
stella groundtruth sources sync --source debuginfod-ubuntu -v
```
**Symbol lookup returns no results:**
```bash
# Verify debug-id format (hex string)
stella groundtruth symbols lookup --debug-id abc123 -v
# Try searching by package instead
stella groundtruth symbols search --package libcrypto
```
**Validation metrics show low match rate:**
- Check that both vuln and patch binaries are present in corpus
- Verify symbol sources are synced and enabled
- Consider using `hybrid` matcher for complex cases
## See Also
- [CLI Command Reference](commands/reference.md#ground-truth-corpus-commands)
- [BinaryIndex Architecture](../../binary-index/architecture.md)
- [Golden Corpus KPIs](../../benchmarks/golden-corpus-kpis.md)
- [Air-Gap Bundle Guide](../../modules/airgap/README.md)

36
docs/modules/cli/guides/trust-profiles.md Normal file
View File

@@ -0,0 +1,36 @@
# Trust Profiles
Trust profiles are offline trust-store templates for bundle verification. They define trust roots, Rekor public keys, and TSA roots in a single file so operators can apply a profile into a local trust store.
Default profile location:
- `etc/trust-profiles/*.trustprofile.json`
- Assets referenced by profiles live under `etc/trust-profiles/assets/`
Profile structure (summary):
- `profileId`: stable identifier (used by CLI commands)
- `trustRoots[]`: signing trust roots (PEM files)
- `rekorKeys[]`: Rekor public keys for offline inclusion proof verification
- `tsaRoots[]`: TSA roots for RFC3161 verification
- `metadata`: optional compliance metadata
CLI usage:
- `stella trust-profile list`
- `stella trust-profile show <profile-id>`
- `stella trust-profile apply <profile-id> --output <dir>`
Profile lookup overrides:
- `--profiles-dir <path>` to point at a custom profiles directory
- `STELLAOPS_TRUST_PROFILES` environment variable for default lookup
Apply output:
- `trust-manifest.json` (trust roots manifest for offline verification)
- `trust-profile.json` (resolved profile copy)
- `trust-root.pem` (combined trust roots for CLI verification)
- `trust-roots/`, `rekor/`, `tsa/` folders with PEM assets
Example apply workflow:
1. `stella trust-profile apply global --output ./trust-store`
2. `stella bundle verify --trust-root ./trust-store/trust-root.pem`
Note:
- Default profiles ship with placeholder roots for scaffolding only. Replace them with compliance-approved roots before production use.

100
docs/modules/concelier/sbom-learning-api.md
View File

@@ -10,18 +10,68 @@ The SBOM Learning API enables Concelier to learn which advisories are relevant t
Concelier normalizes incoming CycloneDX 1.7 and SPDX 3.0.1 documents into the internal `ParsedSbom` model for matching and downstream analysis.
Current extraction coverage (SPRINT_20260119_015):
- Document metadata: format, specVersion, serialNumber, created, name, namespace when present
- Components: bomRef, type, name, version, purl, cpe, hashes (including SPDX verifiedUsing), license IDs/expressions, license text (base64 decode), external references, properties, scope/modified, supplier/manufacturer, evidence, pedigree, cryptoProperties, modelCard (CycloneDX)
- Dependencies: component dependency edges (CycloneDX dependencies, SPDX relationships)
- Document metadata: format, specVersion, serialNumber, created, name, profiles, sbomType, namespace/imports
- Components: bomRef, type, name, version, purl, cpe, hashes (including SPDX verifiedUsing), license IDs/expressions, license text (base64 decode), external references, properties, scope/modified, supplier/manufacturer, evidence, pedigree, cryptoProperties, modelCard (CycloneDX), swid (CycloneDX), SPDX AI model parameters, SPDX dataset metadata, SPDX file/snippet properties
- Licensing: SPDX Licensing profile elements (listed/custom licenses, license additions, AND/OR/WITH/or-later operators), with OSI/FSF flags and deprecated IDs captured
- Dependencies: component dependency edges (CycloneDX dependencies, SPDX relationships; DependencyOf is inverted to DependsOn)
- Vulnerabilities: CycloneDX embedded vulnerabilities (ratings, affects, VEX analysis), SPDX Security profile vulnerabilities + VEX assessments
- Services: endpoints, authentication, crossesTrustBoundary, data flows, licenses, external references (CycloneDX)
- Formulation: components, workflows, tasks, properties (CycloneDX)
- Declarations/definitions: attestations, affirmations, standards, signatures (CycloneDX)
- Compositions/annotations (CycloneDX)
- Build metadata: buildId, buildType, timestamps, config source, environment, parameters (SPDX)
- Document properties
Notes:
- Full SPDX Licensing profile objects, vulnerabilities, and other SPDX profiles are pending in SPRINT_20260119_015.
- License expressions can be validated against embedded SPDX license/exception lists via `ILicenseExpressionValidator`.
- Matching currently uses PURL and CPE; additional fields are stored for downstream consumers.
## VEX consumption
When SBOM vulnerabilities include embedded VEX analysis, Concelier consumes the statements
to filter or annotate advisory matches. NotAffected statements can be filtered when policy
allows, and trust evaluation checks timestamps, signatures (when provided), and justification
requirements for not-affected claims.
Configuration (YAML or JSON), loaded from `Concelier:VexConsumption:PolicyPath`:
```yaml
vexConsumptionPolicy:
trustEmbeddedVex: true
minimumTrustLevel: Unverified
filterNotAffected: true
signatureRequirements:
requireSignedVex: false
trustedSigners:
- "https://example.com/keys/vex-signer"
timestampRequirements:
maxAgeHours: 720
requireTimestamp: true
conflictResolution:
strategy: mostRecent
logConflicts: true
mergePolicy:
mode: union
externalSources:
- type: repository
url: "https://vex.example.com/api"
justificationRequirements:
requireJustificationForNotAffected: true
acceptedJustifications:
- component_not_present
- vulnerable_code_not_present
- vulnerable_code_not_in_execute_path
- inline_mitigations_already_exist
```
Reports are emitted via `VexConsumptionReporter` in JSON, SARIF, and text formats.
Runtime overrides can be supplied via `Concelier:VexConsumption` (Enabled, IgnoreVex,
PolicyPath, TrustEmbeddedVex, MinimumTrustLevel, FilterNotAffected, ExternalVexSources).
## Flow
```
@@ -339,23 +389,51 @@ var affected = await sbomService.GetAffectedAdvisoriesAsync(
```sql
CREATE TABLE vuln.sbom_registry (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL,
artifact_id TEXT NOT NULL,
sbom_digest TEXT NOT NULL,
sbom_format TEXT NOT NULL,
digest TEXT NOT NULL,
format TEXT NOT NULL CHECK (format IN ('cyclonedx', 'spdx')),
spec_version TEXT NOT NULL,
primary_name TEXT,
primary_version TEXT,
component_count INT NOT NULL DEFAULT 0,
affected_count INT NOT NULL DEFAULT 0,
source TEXT NOT NULL,
tenant_id TEXT,
registered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_matched_at TIMESTAMPTZ,
CONSTRAINT uq_sbom_registry_digest UNIQUE (tenant_id, sbom_digest)
CONSTRAINT uq_sbom_registry_digest UNIQUE (digest)
);
CREATE TABLE vuln.sbom_canonical_match (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
sbom_id UUID NOT NULL REFERENCES vuln.sbom_registry(id),
canonical_id UUID NOT NULL REFERENCES vuln.advisory_canonical(id),
matched_purl TEXT NOT NULL,
purl TEXT NOT NULL,
match_method TEXT NOT NULL,
confidence NUMERIC(3,2) NOT NULL DEFAULT 1.0,
is_reachable BOOLEAN NOT NULL DEFAULT false,
is_deployed BOOLEAN NOT NULL DEFAULT false,
matched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT uq_sbom_canonical_match UNIQUE (sbom_id, canonical_id)
CONSTRAINT uq_sbom_canonical_match UNIQUE (sbom_id, canonical_id, purl)
);
CREATE TABLE concelier.sbom_documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
serial_number TEXT NOT NULL,
artifact_digest TEXT,
format TEXT NOT NULL CHECK (format IN ('cyclonedx', 'spdx')),
spec_version TEXT NOT NULL,
component_count INT NOT NULL DEFAULT 0,
service_count INT NOT NULL DEFAULT 0,
vulnerability_count INT NOT NULL DEFAULT 0,
has_crypto BOOLEAN NOT NULL DEFAULT false,
has_services BOOLEAN NOT NULL DEFAULT false,
has_vulnerabilities BOOLEAN NOT NULL DEFAULT false,
license_ids TEXT[] NOT NULL DEFAULT '{}',
license_expressions TEXT[] NOT NULL DEFAULT '{}',
sbom_json JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT uq_concelier_sbom_serial UNIQUE (serial_number),
CONSTRAINT uq_concelier_sbom_artifact UNIQUE (artifact_digest)
);
```

62
docs/modules/platform/platform-service.md
View File

@@ -15,6 +15,7 @@ Provide a single, deterministic aggregation layer for cross-service UX workflows
- Persist dashboard personalization and layout preferences.
- Provide global search aggregation across entities.
- Surface platform metadata for UI bootstrapping (version, build, offline status).
- Expose analytics lake aggregates for SBOM, vulnerability, and attestation reporting.
## API surface (v1)
@@ -49,6 +50,16 @@ Provide a single, deterministic aggregation layer for cross-service UX workflows
### Metadata
- GET `/api/v1/platform/metadata`
- Response includes a capabilities list for UI bootstrapping; analytics capability is reported only when analytics storage is configured.
### Analytics (SBOM lake)
- GET `/api/analytics/suppliers`
- GET `/api/analytics/licenses`
- GET `/api/analytics/vulnerabilities`
- GET `/api/analytics/backlog`
- GET `/api/analytics/attestation-coverage`
- GET `/api/analytics/trends/vulnerabilities`
- GET `/api/analytics/trends/components`
## Data model
- `platform.dashboard_preferences` (dashboard layout, widgets, filters)
@@ -72,11 +83,58 @@ Provide a single, deterministic aggregation layer for cross-service UX workflows
- Preferences: `ui.preferences.read`, `ui.preferences.write`
- Search: `search.read` plus downstream service scopes (`findings:read`, `policy:read`, etc.)
- Metadata: `platform.metadata.read`
- Analytics: `analytics.read`
## Determinism and offline posture
- Stable ordering with explicit sort keys and deterministic tiebreakers.
- Stable ordering with explicit sort keys and deterministic tiebreakers.
- All timestamps in UTC ISO-8601.
- Cache last-known snapshots for offline rendering with "data as of" markers.
- Cache last-known snapshots for offline rendering with "data as of" markers.
## Analytics ingestion configuration
Analytics ingestion runs inside the Platform WebService and subscribes to Scanner,
Concelier, and Attestor streams. Configure ingestion with `Platform:AnalyticsIngestion`:
```yaml
Platform:
AnalyticsIngestion:
Enabled: true
PostgresConnectionString: "" # optional; defaults to Platform:Storage
AllowedTenants: ["tenant-a"]
Streams:
ScannerStream: "orchestrator:events"
ConcelierObservationStream: "concelier:advisory.observation.updated:v1"
ConcelierLinksetStream: "concelier:advisory.linkset.updated:v1"
AttestorStream: "attestor:events"
StartFromBeginning: false
Cas:
RootPath: "/var/lib/stellaops/cas"
DefaultBucket: "attestations"
Attestations:
BundleUriTemplate: "bundle:{digest}"
```
`BundleUriTemplate` supports `{digest}` and `{hash}` placeholders. The `bundle:` scheme
maps to `cas://<DefaultBucket>/{digest}` by default. Verify offline bundles with
`stella bundle verify` before ingestion.
## Analytics maintenance configuration
Analytics rollups + materialized view refreshes are driven by
`PlatformAnalyticsMaintenanceService` when analytics storage is configured.
Use `BackfillDays` to recompute recent rollups on the first maintenance run (set to `0` to disable).
```yaml
Platform:
Storage:
PostgresConnectionString: "Host=...;Database=...;Username=...;Password=..."
AnalyticsMaintenance:
Enabled: true
RunOnStartup: true
IntervalMinutes: 1440
ComputeDailyRollups: true
RefreshMaterializedViews: true
BackfillDays: 7
```
## Observability
- Metrics: `platform.aggregate.latency_ms`, `platform.aggregate.errors_total`, `platform.aggregate.cache_hits_total`

46
docs/modules/policy/architecture.md
View File

@@ -17,6 +17,7 @@ The service operates strictly downstream of the **Aggregation-Only Contract (AOC
- Compile and evaluate `stella-dsl@1` policy packs into deterministic verdicts.
- Join SBOM inventory, Concelier advisories, and Excititor VEX evidence via canonical linksets and equivalence tables.
- Evaluate SBOM license expressions against policy (SPDX AND/OR/WITH/+), emitting compliance findings and attribution requirements for gate decisions.
- Materialise effective findings (`effective_finding_{policyId}`) with append-only history and produce explain traces.
- Emit CVSS v4.0 receipts with canonical hashing and policy replay/backfill rules; store tenant-scoped receipts with RBAC; export receipts deterministically (UTC/fonts/order) and flag v3.1→v4.0 conversions (see Sprint 0190 CVSS-GAPS-190-014 / `docs/modules/policy/cvss-v4.md`).
- Emit per-finding OpenVEX decisions anchored to reachability evidence, forward them to Signer/Attestor for DSSE/Rekor, and publish the resulting artifacts for bench/verification consumers.
@@ -171,9 +172,52 @@ The Determinization subsystem calculates uncertainty scores based on signal comp
**Usage in policies:**
Determinization scores are exposed to SPL policies via the `signals.trust.*` and `signals.uncertainty.*` namespaces. Use `signals.uncertainty.entropy` to access entropy values and `signals.trust.score` for aggregated trust scores that combine VEX, reachability, runtime, and other signals with decay/weighting.
### 3.2 - License compliance configuration
License compliance evaluation runs during SBOM evaluation when enabled in
`licenseCompliance` settings.
```json
{
"licenseCompliance": {
"enabled": true,
"policyPath": "policies/license-policy.yaml"
}
}
```
- `sbom.license` exposes the compliance report (findings, conflicts, inventory).
- `sbom.license_status` exposes `pass`, `warn`, or `fail` (or `unknown` when disabled).
- Failures set the policy verdict status to `blocked` and emit `license.*` annotations.
- Trademark notice obligations are tracked alongside attribution requirements and produce warn-level findings.
- License compliance reports support JSON, text/markdown/html, legal-review, and PDF outputs.
- Category breakdown includes percent totals and chart renderings (ASCII chart in text/markdown/legal-review/PDF, pie chart in HTML).
---
## 4 · Data Model & Persistence
### 3.3 - NTIA compliance configuration
NTIA minimum-elements validation runs when enabled under `ntiaCompliance`.
```json
{
"ntiaCompliance": {
"enabled": true,
"enforceGate": false,
"policyPath": "policies/ntia-policy.yaml"
}
}
```
- `sbom.ntia` exposes NTIA compliance details (elements, findings, supplier status).
- `sbom.ntia_status` exposes `pass`, `warn`, `fail`, or `unknown`.
- NTIA compliance can be configured as an advisory-only check or a release gate via `enforceGate`.
- The NTIA policy supports element selection, supplier validation (placeholder patterns, trusted/blocked lists), and framework-specific requirements.
- Reports support JSON, text/markdown/html, and PDF output for regulatory submissions.
---
## 4 · Data Model & Persistence
### 4.1 Collections

8
docs/modules/release-orchestrator/test-structure.md
View File

@@ -382,19 +382,19 @@ public class EvidenceHashDeterminismTests
### Run All Tests
```bash
dotnet test src/StellaOps.sln
dotnet test src/ReleaseOrchestrator/StellaOps.ReleaseOrchestrator.sln
```
### Run Only Unit Tests
```bash
dotnet test src/StellaOps.sln --filter "Category=Unit"
dotnet test src/ReleaseOrchestrator/StellaOps.ReleaseOrchestrator.sln --filter "Category=Unit"
```
### Run Only Integration Tests
```bash
dotnet test src/StellaOps.sln --filter "Category=Integration"
dotnet test src/ReleaseOrchestrator/StellaOps.ReleaseOrchestrator.sln --filter "Category=Integration"
```
### Run Specific Test Class
@@ -406,7 +406,7 @@ dotnet test --filter "FullyQualifiedName~PromotionValidatorTests"
### Run with Coverage
```bash
dotnet test src/StellaOps.sln --collect:"XPlat Code Coverage"
dotnet test src/ReleaseOrchestrator/StellaOps.ReleaseOrchestrator.sln --collect:"XPlat Code Coverage"
```
---

47
docs/modules/scanner/architecture.md
View File

@@ -14,10 +14,14 @@
**Boundaries.**
* Scanner **does not** produce PASS/FAIL. The backend (Policy + Excititor + Concelier) decides presentation and verdicts.
* Scanner **does not** keep third‑party SBOM warehouses. It may **bind** to existing attestations for exact hashes.
* Core analyzers are **deterministic** (no fuzzy identity). Optional heuristic plug‑ins (e.g., patch‑presence) run under explicit flags and never contaminate the core SBOM.
---
* Scanner **does not** keep third‑party SBOM warehouses. It may **bind** to existing attestations for exact hashes.
* Core analyzers are **deterministic** (no fuzzy identity). Optional heuristic plug‑ins (e.g., patch‑presence) run under explicit flags and never contaminate the core SBOM.
SBOM dependency reachability inference uses dependency graphs to reduce false positives and
apply reachability-aware severity adjustments. See `src/Scanner/docs/sbom-reachability-filtering.md`
for policy configuration and reporting expectations.
---
## 1) Solution & project layout
@@ -374,7 +378,40 @@ public sealed record BinaryFindingEvidence
The emitted `buildId` metadata is preserved in component hashes, diff payloads, and `/policy/runtime` responses so operators can pivot from SBOM entries → runtime events → `debug/.build-id/<aa>/<rest>.debug` within the Offline Kit or release bundle.
### 5.6 DSSE attestation (via Signer/Attestor)
### 5.5.1 Service security analysis (Sprint 20260119_016)
When an SBOM path is provided, the worker runs the `service-security` stage to parse CycloneDX services and emit a deterministic report covering:
- Endpoint scheme hygiene (HTTP/WS/plaintext protocol detection).
- Authentication and trust-boundary enforcement.
- Sensitive data flow exposure and unencrypted transfers.
- Deprecated service versions and rate-limiting metadata gaps.
Inputs are passed via scan metadata (`sbom.path` or `sbomPath`, plus `sbom.format`). The report is attached as a surface observation payload (`service-security.report`) and keyed in the analysis store for downstream policy and report assembly. See `src/Scanner/docs/service-security.md` for the policy schema and output formats.
### 5.5.2 CBOM crypto analysis (Sprint 20260119_017)
When an SBOM includes CycloneDX `cryptoProperties`, the worker runs the `crypto-analysis` stage to produce a crypto inventory and compliance findings for weak algorithms, short keys, deprecated protocol versions, certificate hygiene, and post-quantum readiness. The report is attached as a surface observation payload (`crypto-analysis.report`) and keyed in the analysis store for downstream evidence workflows. See `src/Scanner/docs/crypto-analysis.md` for the policy schema and inventory export formats.
### 5.5.3 AI/ML supply chain security (Sprint 20260119_018)
When an SBOM includes CycloneDX `modelCard` or SPDX AI profile data, the worker runs the `ai-ml-security` stage to evaluate model governance readiness. The report covers model card completeness, training data provenance, bias/fairness checks, safety risk assessment coverage, and provenance verification. The report is attached as a surface observation payload (`ai-ml-security.report`) and keyed in the analysis store for policy evaluation and audit trails. See `src/Scanner/docs/ai-ml-security.md` for policy schema, CLI toggles, and binary analysis conventions.
### 5.5.4 Build provenance verification (Sprint 20260119_019)
When an SBOM includes CycloneDX formulation or SPDX build profile data, the worker runs the `build-provenance` stage to verify provenance completeness, builder trust, source integrity, hermetic build requirements, and optional reproducibility checks. The report is attached as a surface observation payload (`build-provenance.report`) and keyed in the analysis store for policy enforcement and audit evidence. See `src/Scanner/docs/build-provenance.md` for policy schema, CLI toggles, and report formats.
### 5.5.5 SBOM dependency reachability (Sprint 20260119_022)
When configured, the worker runs the `reachability-analysis` stage to infer dependency reachability from SBOM graphs and optionally refine it with a `richgraph-v1` call graph. Advisory matches are filtered or severity-adjusted using `VulnerabilityReachabilityFilter`, with false-positive reduction metrics recorded for auditability. The stage attaches:
- `reachability.report` (JSON) for component and vulnerability reachability.
- `reachability.report.sarif` (SARIF 2.1.0) for toolchain export.
- `reachability.graph.dot` (GraphViz) for dependency visualization.
Configuration lives in `src/Scanner/docs/sbom-reachability-filtering.md`, including policy schema, metadata keys, and report outputs.
### 5.6 DSSE attestation (via Signer/Attestor)
* WebService constructs **predicate** with `image_digest`, `stellaops_version`, `license_id`, `policy_digest?` (when emitting **final reports**), timestamps.
* Calls **Signer** (requires **OpTok + PoE**); Signer verifies **entitlement + scanner image integrity** and returns **DSSE bundle**.

365
docs/runbooks/golden-corpus-operations.md Normal file
View File

@@ -0,0 +1,365 @@
# Golden Corpus Operations Runbook
Sprint: SPRINT_20260121_036_BinaryIndex_golden_corpus_bundle_verification
Task: GCB-006 - Document corpus folder layout and maintenance procedures
## Overview
This runbook provides operational procedures for the golden corpus infrastructure, including troubleshooting, incident response, and common maintenance tasks.
## Quick Reference
| Task | Command |
|------|---------|
| Check corpus health | `stella doctor --check "check.binaryanalysis.corpus.*"` |
| Run validation | `stella groundtruth validate run --output results.json` |
| Check regression | `stella groundtruth validate check --results results.json --baseline current.json` |
| Update baseline | `stella groundtruth baseline update --from-results results.json --output current.json` |
| Export bundle | `stella groundtruth bundle export --packages openssl --distros debian --output bundle.tar.gz` |
| Verify bundle | `stella groundtruth bundle import --input bundle.tar.gz --verify` |
## Troubleshooting
### Mirror Sync Failures
#### Symptoms
- Doctor check `check.binaryanalysis.corpus.mirror.freshness` fails
- Validation runs fail with "source not found" errors
- Alerts for stale mirrors
#### Diagnosis
```bash
# Check mirror last sync times
ls -la /data/golden-corpus/mirrors/*/.last-sync
# Check sync logs
tail -100 /var/log/corpus/debian-sync.log
tail -100 /var/log/corpus/ubuntu-sync.log
tail -100 /var/log/corpus/osv-sync.log
# Test connectivity
curl -I https://snapshot.debian.org/
curl -I https://buildinfos.debian.net/
curl -I https://ubuntu.com/security/notices.json
```
#### Resolution
1. **Network connectivity issues**
```bash
# Check firewall rules
iptables -L -n | grep -E "80|443"
# Check DNS resolution
nslookup snapshot.debian.org
# Test with proxy if applicable
export https_proxy=http://proxy:3128
curl -I https://snapshot.debian.org/
```
2. **Upstream service unavailable**
- Check upstream service status
- Wait and retry (services may be temporarily unavailable)
- Switch to backup mirror if available
3. **Disk space issues**
```bash
# Check disk usage
df -h /data/golden-corpus
# Clean up old archives
/opt/golden-corpus/scripts/archive-old-results.sh
```
4. **Permission issues**
```bash
# Check file ownership
ls -la /data/golden-corpus/mirrors/
# Fix permissions
chown -R corpus:corpus /data/golden-corpus/mirrors/
chmod -R 755 /data/golden-corpus/mirrors/
```
### Validation Failures
#### Symptoms
- CI pipeline fails on regression check
- Validation run exits with non-zero code
- Lower than expected KPI metrics
#### Diagnosis
```bash
# Check latest validation results
stella groundtruth validate metrics --run-id latest --detailed
# Compare with baseline
stella groundtruth validate check \
--results bench/results/latest.json \
--baseline bench/baselines/current.json \
--verbose
# Review specific failures
jq '.failedPairs[]' bench/results/latest.json
```
#### Resolution
1. **True regression (algorithm degradation)**
- Review recent code changes
- Identify the causing commit
- Either fix the regression or update baseline if intentional
2. **False positive (ground truth incorrect)**
```bash
# Review ground truth for specific pair
cat corpus/debian/openssl/DSA-5678-1/metadata/ground-truth.json
# Update ground truth if incorrect
# (Requires manual review by security team)
```
3. **Infrastructure issues**
- Check if build environment is consistent
- Verify debug symbols are available
- Check Ghidra/BSim connectivity
4. **Baseline drift**
- If corpus was significantly updated, baseline may need refresh
- Run full validation and update baseline following procedures
### Bundle Verification Failures
#### Symptoms
- `stella groundtruth bundle import --verify` fails
- Signature verification errors
- Timestamp validation errors
#### Diagnosis
```bash
# Verbose verification
stella groundtruth bundle import \
--input bundle.tar.gz \
--verify \
--verbose \
--output report.json
# Check specific failures
jq '.signatureResult, .timestampResult, .digestResult' report.json
```
#### Resolution
1. **Signature verification failure**
```bash
# Check trusted keys
cat /etc/stellaops/trusted-keys.pub
# Verify key hasn't expired
openssl x509 -in /etc/stellaops/trusted-keys.pub -noout -dates
# Check if bundle was signed with different key
# May need to add signing key to trusted keys
```
2. **Timestamp verification failure**
- Check TSA certificate validity
- Verify system clock is accurate
- Check if timestamp is within validity window
3. **Digest mismatch**
- Bundle may be corrupted during transfer
- Re-download or re-generate the bundle
- Check for partial transfers
### Baseline Not Found
#### Symptoms
- Doctor check `check.binaryanalysis.corpus.kpi.baseline` fails
- Regression check errors with "baseline not found"
#### Resolution
```bash
# Check baseline path
ls -la bench/baselines/current.json
# If missing, create from latest results
stella groundtruth baseline update \
--from-results bench/results/latest.json \
--output bench/baselines/current.json \
--description "Initial baseline"
# Or restore from archive
ls bench/baselines/archive/
cp bench/baselines/archive/baseline-20260115.json \
bench/baselines/current.json
```
### Debuginfod Connectivity Issues
#### Symptoms
- Doctor check `check.binaryanalysis.debuginfod.availability` fails
- Missing debug symbols during validation
#### Diagnosis
```bash
# Check DEBUGINFOD_URLS environment
echo $DEBUGINFOD_URLS
# Test debuginfod connectivity
curl -I "https://debuginfod.fedoraproject.org/buildid/xyz/debuginfo"
curl -I "https://debuginfod.ubuntu.com/buildid/xyz/debuginfo"
```
#### Resolution
1. **Configure DEBUGINFOD_URLS**
```bash
export DEBUGINFOD_URLS="https://debuginfod.fedoraproject.org/ https://debuginfod.ubuntu.com/"
```
2. **Use local fallback**
- Enable local debug symbol cache
- Sync ddeb packages for Ubuntu
- Download debug packages from archives
## Incident Response
### KPI Regression Detected in Production
**Severity:** High
**Response Time:** 4 hours
1. **Acknowledge and assess**
```bash
# Get current status
stella groundtruth validate check \
--results bench/results/latest.json \
--baseline bench/baselines/current.json
```
2. **Identify root cause**
- Check recent code changes
- Review validation logs
- Compare with previous runs
3. **Mitigate**
- If code regression: revert the change
- If ground truth issue: fix ground truth
- If infrastructure issue: fix and re-run
4. **Verify fix**
```bash
# Re-run validation
stella groundtruth validate run --output results-fix.json
# Verify regression is fixed
stella groundtruth validate check \
--results results-fix.json \
--baseline bench/baselines/current.json
```
5. **Post-incident**
- Document in incident log
- Update runbook if new issue type
- Consider adding monitoring/alerting
### Mirror Corruption Detected
**Severity:** Medium
**Response Time:** 24 hours
1. **Identify corrupted files**
```bash
# Check file integrity
find /data/golden-corpus/mirrors -name "*.deb" -exec dpkg-deb --info {} \; 2>&1 | grep -i error
```
2. **Remove corrupted files**
```bash
# Move corrupted files to quarantine
mkdir -p /data/golden-corpus/quarantine
mv /data/golden-corpus/mirrors/debian/path/to/corrupted.deb \
/data/golden-corpus/quarantine/
```
3. **Re-sync affected mirror**
```bash
/opt/golden-corpus/scripts/sync-debian-mirrors.sh
```
4. **Verify fix**
```bash
stella doctor --check check.binaryanalysis.corpus.mirror.freshness
```
### Disk Space Critical
**Severity:** High
**Response Time:** 1 hour
1. **Check usage**
```bash
df -h /data/golden-corpus
du -sh /data/golden-corpus/*
```
2. **Quick cleanup**
```bash
# Archive old results
/opt/golden-corpus/scripts/archive-old-results.sh
# Prune old baselines
/opt/golden-corpus/scripts/prune-baselines.sh
# Remove old evidence bundles
find /data/golden-corpus/evidence -name "*.tar.gz" -mtime +90 -delete
```
3. **Expand storage if needed**
- Request additional storage
- Mount new volume
- Migrate data if necessary
## Scheduled Maintenance
### Weekly Tasks
- [ ] Review Doctor health checks
- [ ] Check mirror freshness alerts
- [ ] Review validation results trends
- [ ] Archive old results
### Monthly Tasks
- [ ] Generate compliance evidence bundles
- [ ] Review and update ground truth annotations
- [ ] Prune old baselines (keep last 10)
- [ ] Review storage usage trends
### Quarterly Tasks
- [ ] Full corpus validation (not just seed)
- [ ] Review and update documentation
- [ ] Test disaster recovery procedures
- [ ] Review access permissions
## Contact Information
| Role | Contact | Escalation |
|------|---------|------------|
| Corpus Owner | corpus-team@stella-ops.org | 1st |
| BinaryIndex Guild | binaryindex@stella-ops.org | 2nd |
| Platform On-Call | oncall@stella-ops.org | 3rd |
## Related Documentation
- [Golden Corpus Folder Layout](../modules/binary-index/golden-corpus-layout.md)
- [Golden Corpus Maintenance](../modules/binary-index/golden-corpus-maintenance.md)
- [Ground Truth Corpus Overview](../modules/binary-index/ground-truth-corpus.md)

2
docs/schemas/spdx-jsonld-3.0.1.schema.json
View File

@@ -31,7 +31,7 @@
},
"spdxVersion": {
"type": "string",
"pattern": "^SPDX-3\\.[0-9]+$"
"pattern": "^SPDX-3\\.[0-9]+(\\.[0-9]+)?$"
},
"creationInfo": {
"type": "object",

2
docs/technical/architecture/module-matrix.md
View File

@@ -1,6 +1,6 @@
# Complete Module Matrix
This document provides a comprehensive inventory of all 46+ modules in the StellaOps solution (`src/StellaOps.sln`), explaining the purpose of each module and how they relate to the documented architecture.
This document provides a comprehensive inventory of all 46+ modules in the StellaOps platform. Module build entry points are the module solutions listed in docs/dev/SOLUTION_BUILD_GUIDE.md.
## Table of Contents

2
docs/technical/cicd/path-filters.md
View File

@@ -39,7 +39,7 @@ infrastructure:
- 'src/Directory.Build.props' # Source directory properties
- 'src/Directory.Packages.props'
- 'nuget.config' # NuGet feed configuration
- 'StellaOps.sln' # Solution file
- 'src/**/StellaOps.*.sln' # Module solution files
- '.gitea/workflows/**' # CI/CD workflow changes
```

299
docs/technical/scoring-algebra.md Normal file
View File

@@ -0,0 +1,299 @@
# Unified Trust Score Architecture
> **Ownership:** Policy Guild • Signals Guild
> **Services:** `StellaOps.Signals.UnifiedScore` (facade), `StellaOps.Signals.EvidenceWeightedScore` (core), `StellaOps.Policy.Determinization` (entropy)
> **Related docs:** [Policy architecture](../modules/policy/architecture.md), [EWS migration](../modules/policy/design/confidence-to-ews-migration.md), [Score Proofs API](../api/scanner-score-proofs-api.md)
This document describes the **unified trust score facade** that provides a single API for accessing risk scores, uncertainty metrics, and evidence from the underlying EWS and Determinization systems.
---
## 1 · Design Principle: Facade Over Rewrite
Stella Ops has mature, battle-tested scoring systems:
| System | Purpose | Maturity |
|--------|---------|----------|
| **EWS** | 6-dimension risk scoring with guardrails | Production (1000+ determinism tests) |
| **Determinization** | Entropy, confidence decay, conflict detection | Production |
| **RiskEngine** | Signal-specific providers (CVSS/KEV/EPSS) | Production |
The unified score facade **does not replace these systems**. Instead, it:
1. **Combines** EWS scores with Determinization entropy in a single result
2. **Externalizes** EWS weights to versioned manifest files for auditing
3. **Exposes** the unknowns fraction (U) as a first-class metric
---
## 2 · Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ IUnifiedScoreService │
│ (Facade) │
├─────────────────────────────────────────────────────────────┤
│ • ComputeAsync(request) → UnifiedScoreResult │
│ • Combines EWS + Determinization + ConflictDetector │
│ • Loads weights from versioned manifests │
└─────────────┬───────────────────────┬───────────────────────┘
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ EvidenceWeightedScore │ │ Determinization │
│ Calculator │ │ │
├─────────────────────────┤ ├─────────────────────────┤
│ • 6-dimension scoring │ │ • Entropy calculation │
│ • Guardrails (caps/ │ │ • Confidence decay │
│ floors) │ │ • Signal gap tracking │
│ • Anchor metadata │ │ • Fingerprinting │
│ • Policy digest │ │ • Conflict detection │
└─────────────────────────┘ └─────────────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────┐
│ etc/weights/*.json │
│ (Versioned Weight Manifests) │
└─────────────────────────────────────────────────────────────┘
```
---
## 3 · What the Facade Provides
### 3.1 · Unified Score Result
```csharp
public sealed record UnifiedScoreResult
{
// From EWS
public int Score { get; } // 0-100
public string Bucket { get; } // ActNow, ScheduleNext, Investigate, Watchlist
public IReadOnlyList<DimensionContribution> Breakdown { get; }
public AppliedGuardrails Guardrails { get; }
public string EwsDigest { get; } // SHA-256 of EWS result
// From Determinization
public double UnknownsFraction { get; } // U metric (0.0 = complete, 1.0 = no data)
public UnknownsBand UnknownsBand { get; } // Complete, Adequate, Sparse, Insufficient
public IReadOnlyList<SignalGap> Gaps { get; }
public IReadOnlyList<SignalConflict> Conflicts { get; }
public string DeterminizationFingerprint { get; }
// Combined
public IReadOnlyList<SignalDelta> DeltaIfPresent { get; } // Impact if missing signals arrive
public string WeightManifestRef { get; } // version + hash
public DateTimeOffset ComputedAt { get; }
}
```
### 3.2 · Unknowns Fraction (U)
The `UnknownsFraction` directly exposes Determinization's entropy calculation:
```
U = 1 - (weighted_present_signals / total_weight)
```
| U Range | Band | Meaning | Action |
|---------|------|---------|--------|
| 0.0 – 0.2 | Complete | All signals present | Automated decisions |
| 0.2 – 0.4 | Adequate | Sufficient for evaluation | Automated decisions |
| 0.4 – 0.6 | Sparse | Signal gaps exist | Manual review recommended |
| 0.6 – 1.0 | Insufficient | Critical data missing | Block pending more signals |
Thresholds align with existing Determinization config:
- `RefreshEntropyThreshold: 0.40` → triggers signal refresh
- `ManualReviewEntropyThreshold: 0.60` → requires human review
### 3.3 · Delta-If-Present
When signals are missing, the facade calculates potential score impact:
```json
{
"delta_if_present": [
{
"signal": "reachability",
"min_impact": -15,
"max_impact": +8,
"description": "If reachability confirmed as not-reachable, score decreases by up to 15"
},
{
"signal": "runtime",
"min_impact": 0,
"max_impact": +25,
"description": "If runtime execution observed, score increases by up to 25"
}
]
}
```
---
## 4 · Weight Manifests
### 4.1 · Location
Weight manifests are stored in `etc/weights/` with versioned filenames:
```
etc/weights/
├── v2026-01-22.weights.json
├── v2026-02-01.weights.json
└── ...
```
### 4.2 · Schema
```json
{
"version": "v2026-01-22",
"effective_from": "2026-01-22T00:00:00Z",
"description": "EWS default weights",
"weights": {
"rch": 0.30,
"rts": 0.25,
"bkp": 0.15,
"xpl": 0.15,
"src": 0.10,
"mit": 0.10
},
"hash": "sha256:..."
}
```
### 4.3 · Versioning Rules
1. **Immutable once published** – Manifest content never changes after creation
2. **Hash verification** – SHA-256 of canonical JSON ensures integrity
3. **Policy pinning** – Policies can specify `weights_ref` to lock a version
4. **Fallback** – If manifest missing, EWS uses compiled defaults
---
## 5 · Existing Systems (Unchanged)
### 5.1 · EWS Formula (Preserved)
The EWS formula remains unchanged:
```
rawScore = (RCH × w_rch) + (RTS × w_rts) + (BKP × w_bkp) +
(XPL × w_xpl) + (SRC × w_src) - (MIT × w_mit)
finalScore = clamp(rawScore, 0, 1) × 100
```
With guardrails:
- **Speculative cap** (45): RCH=0 and RTS=0
- **Not-affected cap** (15): BKP≥1.0, VEX=not_affected, RTS<0.6
- **Runtime floor** (60): RTS≥0.8
### 5.2 · Determinization (Preserved)
Entropy and decay calculations remain unchanged:
```
entropy = 1 - (present_weight / total_weight)
decay = max(floor, exp(-ln(2) × age_days / half_life_days))
```
With conflict detection:
- VEX vs Reachability contradiction
- Static vs Runtime contradiction
- Multiple VEX status conflict
- Backport vs Status conflict
---
## 6 · Integration Points
### 6.1 · CLI Commands
```bash
# Existing (enhanced)
stella gate score evaluate --finding-id CVE-2024-1234@pkg:npm/lodash \
--cvss 7.5 --epss 0.15 --reachability function \
--show-unknowns --show-deltas
# New
stella score compute --finding-id CVE-2024-1234@pkg:npm/lodash \
--cvss 7.5 --epss 0.15
stella score explain CVE-2024-1234@pkg:npm/lodash
stella gate score weights list
stella gate score weights show v2026-01-22
stella gate score weights diff v2026-01-22 v2026-02-01
```
### 6.2 · API Endpoints
```
POST /api/v1/score/evaluate # Compute unified score
GET /api/v1/score/{id}/replay # Fetch signed replay proof
GET /api/v1/score/weights # List weight manifests
GET /api/v1/score/weights/{v} # Get specific manifest
```
#### Replay Endpoint Response
The `/score/{id}/replay` endpoint returns a DSSE-signed attestation with payload type `application/vnd.stella.score+json`:
```json
{
"signed_replay_log_dsse": "BASE64",
"rekor_inclusion": {"logIndex": 12345, "rootHash": "…"},
"canonical_inputs": [
{"name": "sbom.json", "sha256": "…"},
{"name": "vex.json", "sha256": "…"}
],
"transforms": [
{"name": "canonicalize_spdx", "version": "1.1"},
{"name": "age_decay", "params": {"lambda": 0.02}}
],
"algebra_steps": [
{"signal": "rch", "w": 0.30, "value": 0.78, "term": 0.234}
],
"final_score": 85
}
```
Replay proofs are stored as OCI referrers ("StellaBundle" pattern) attached to the scored artifact.
### 6.3 · Console UI
Finding detail views show:
- Score with bucket (existing)
- Unknowns fraction (U) with color-coded band
- Delta-if-present for missing signals
- Weight manifest version
---
## 7 · Determinism Guarantees
The facade inherits determinism from underlying systems:
| Aspect | Guarantee |
|--------|-----------|
| EWS score | Identical inputs → identical score (1000+ iteration tests) |
| Entropy | Identical signal presence → identical U |
| Fingerprint | Content-addressed SHA-256 |
| Weight manifest | Immutable after creation |
The facade adds no additional sources of non-determinism.
---
## 8 · What We're NOT Doing
- ❌ Replacing EWS formula
- ❌ Replacing Determinization entropy calculation
- ❌ Changing guardrail logic
- ❌ Changing conflict detection
- ❌ Breaking existing CLI commands
- ❌ Breaking existing API contracts
The facade is **additive** – existing functionality continues to work unchanged.

8
docs/technical/testing/LOCAL_CI_GUIDE.md
View File

@@ -331,7 +331,7 @@ Before running tests offline or during rate-limited periods, warm the NuGet cach
```bash
# Warm cache with throttled requests to avoid 429 errors
export NUGET_MAX_HTTP_REQUESTS=4
dotnet restore src/StellaOps.sln --disable-parallel
dotnet restore src/<Module>/StellaOps.<Module>.sln --disable-parallel
# Verify cache is populated
ls ~/.nuget/packages | wc -l
@@ -340,12 +340,14 @@ ls ~/.nuget/packages | wc -l
```powershell
# PowerShell equivalent
$env:NUGET_MAX_HTTP_REQUESTS = "4"
dotnet restore src\StellaOps.sln --disable-parallel
dotnet restore src\<Module>\StellaOps.<Module>.sln --disable-parallel
# Verify cache
(Get-ChildItem "$env:USERPROFILE\.nuget\packages").Count
```
See docs/dev/SOLUTION_BUILD_GUIDE.md for the module solution list.
### Rate Limiting Mitigation
If encountering NuGet 429 (Too Many Requests) errors from package sources:
@@ -392,7 +394,7 @@ docker compose -f devops/compose/docker-compose.ci.yaml up -d
./devops/scripts/local-ci.sh smoke --no-restore
# 4. Or run specific category offline
dotnet test src/StellaOps.sln \
dotnet test src/<Module>/StellaOps.<Module>.sln \
--filter "Category=Unit" \
--no-restore \
--no-build

4
docs/technical/testing/PRE_COMMIT_CHECKLIST.md
View File

@@ -121,8 +121,8 @@ docker compose -f devops/compose/docker-compose.ci.yaml down -v
### Build fails
```bash
dotnet clean src/StellaOps.sln
dotnet build src/StellaOps.sln
dotnet clean src/<Module>/StellaOps.<Module>.sln
dotnet build src/<Module>/StellaOps.<Module>.sln
```
### Tests fail

2
docs/technical/testing/ci-lane-integration.md
View File

@@ -68,7 +68,7 @@ unit-tests:
with:
dotnet-version: '10.0.100'
- name: Build
run: dotnet build src/StellaOps.sln --configuration Release
run: dotnet build src/<Module>/StellaOps.<Module>.sln --configuration Release
- name: Run Unit lane
run: ./scripts/test-lane.sh Unit --results-directory ./test-results
- name: Upload results