wip - advisories and ui extensions

2025-12-29 08:39:52 +02:00
parent c2b9cd8d1f
commit 1b61c72c90
56 changed files with 15187 additions and 24 deletions
--- a/docs/product-advisories/archived/2025-12-29_deterministic_verdicts_and_sbom_lineage.md
+++ b/docs/product-advisories/archived/2025-12-29_deterministic_verdicts_and_sbom_lineage.md
@@ -0,0 +1,133 @@
+# Advisory Analysis: Deterministic Verdicts (CGS) & SBOM Lineage Graph
+
+**Advisory Date:** 2025-12-29
+**Status:** ANALYZED - Superseded by Existing Consolidations
+**Strategic Value:** HIGH
+**Implementation Effort:** MEDIUM (gaps only)
+
+---
+
+## Executive Summary
+
+This advisory proposes:
+1. **SBOM Lineage Graph** - Git-like visualization with hover-to-proof micro-interactions
+2. **Canonical Graph Signature (CGS)** - Deterministic, replayable verdicts
+3. **Proof Studio UX** - Explainable confidence scoring
+
+**Verdict:** The advisory validates StellaOps' existing architecture direction. **~90% is already implemented.** The remaining work is minor integration, not invention.
+
+**Revision Note (2025-12-29):** Deeper exploration revealed the frontend is more complete than initially assessed:
+- 41 TypeScript files in lineage feature
+- 31 visualization components already exist
+- Proof tree, hover cards, compare mode, diff views all implemented
+- Frontend sprints revised to minor integration tasks
+
+---
+
+## Prior Art (Already Consolidated)
+
+| Advisory Concept | Existing Document | Status |
+|-----------------|-------------------|--------|
+| SBOM Lineage Graph | `ADVISORY_SBOM_LINEAGE_GRAPH.md` | 70% backend |
+| Deterministic Verdicts | `CONSOLIDATED - Deterministic Evidence and Verdict Architecture.md` | 85% complete |
+| Diff-Aware Gates | `CONSOLIDATED - Diff-Aware Release Gates and Risk Budgets.md` | 75% complete |
+
+---
+
+## What's Already Implemented
+
+### ✅ Complete
+
+| Component | Location |
+|-----------|----------|
+| Canonical JSON (RFC 8785 JCS) | `StellaOps.Canonical.Json` |
+| NFC String Normalization | `StellaOps.Resolver.NfcStringNormalizer` |
+| Content-Addressed IDs | `Attestor.ProofChain/Identifiers/` |
+| DSSE Signing | `Signer/`, `Attestor/` |
+| Merkle Trees | `ProofChain/Merkle/DeterministicMerkleTreeBuilder` |
+| Determinism Guards | `Policy.Engine/DeterminismGuard/` |
+| Replay Manifest | `StellaOps.Replay.Core` |
+| Evidence Sealing | `EvidenceLocker.Core` |
+| VEX Trust Lattice | `VexLens/OpenVexStatementMerger` |
+| Delta Verdicts | `Policy/Deltas/DeltaVerdict.cs` |
+| Rekor Verification | `Attestor.Core/Verification/` |
+| SBOM Ledger with Lineage | `SbomService/SbomLedgerService` |
+
+### 🔄 Gaps Identified
+
+| Gap | Sprint |
+|-----|--------|
+| Unified VerdictBuilder service | SPRINT_20251229_001_001_BE |
+| `POST /verdicts/build` API | SPRINT_20251229_001_001_BE |
+| Fulcio keyless signing wiring | SPRINT_20251229_001_001_BE |
+| `policy.lock.json` generator | SPRINT_20251229_001_001_BE |
+| VEX delta table migration | SPRINT_20251229_001_002_BE |
+| SBOM-verdict link table | SPRINT_20251229_001_002_BE |
+| VexLens PostgreSQL backend | SPRINT_20251229_001_002_BE |
+| Lineage Graph UI component | SPRINT_20251229_001_003_FE |
+| Hover card micro-interactions | SPRINT_20251229_001_003_FE |
+| Proof Studio UI | SPRINT_20251229_001_004_FE |
+| What-if confidence slider | SPRINT_20251229_001_004_FE |
+
+---
+
+## Created Sprints
+
+1. `SPRINT_20251229_001_001_BE_cgs_infrastructure.md` - VerdictBuilder, APIs, Fulcio
+2. `SPRINT_20251229_001_002_BE_vex_delta.md` - Database migrations
+3. `SPRINT_20251229_001_003_FE_lineage_graph.md` - Graph visualization
+4. `SPRINT_20251229_001_004_FE_proof_studio.md` - Explainability UX
+
+---
+
+## Recommendation
+
+**Archive this advisory** as a validation of architecture direction. Reference existing consolidated documents for implementation. Execute the gap-focused sprints above.
+
+---
+
+## Original Advisory Content
+
+The original advisory proposed:
+
+### Canonical Graph Signature (CGS)
+> Turn all inputs into a graph (nodes: packages, files, build steps, attestations; edges: depends-on, produced-by), serialize canonically, then hash. **Rule:** `same inputs (bytes + rule set + policy versions) → same CGS → same verdict`.
+
+**StellaOps Status:** Implemented via `ProofChain/Merkle/DeterministicMerkleTreeBuilder` + content-addressed IDs.
+
+### Canonicalization Rules
+> - Sort all collections (lexicographic, locale-independent)
+> - Normalize IDs (PURL casing, semver normalization)
+> - Stable timestamps: truncated ISO8601Z or logical time
+> - No environmental entropy
+
+**StellaOps Status:** Implemented via `Rfc8785JsonCanonicalizer`, `NfcStringNormalizer`, Policy determinism guards.
+
+### API Surface
+> - `POST /verdicts/build`
+> - `GET /verdicts/{cgs_hash}`
+> - `POST /verdicts/diff`
+
+**StellaOps Status:** Gap - needs VerdictBuilder service composition.
+
+### Rollout Phases
+> 1. Canonicalize & Hash ✅
+> 2. CGS & Deterministic Engine ✅
+> 3. Signed Verdicts (OCI-attach) 🔄
+> 4. Diff & Time-travel 🔄
+> 5. Confidence & Proof Studio ❌
+
+**StellaOps Status:** Phases 1-2 complete, 3-4 partial, 5 needs frontend.
+
+---
+
+## References
+
+- `docs/product-advisories/archived/CONSOLIDATED - Deterministic Evidence and Verdict Architecture.md`
+- `docs/product-advisories/archived/CONSOLIDATED - Diff-Aware Release Gates and Risk Budgets.md`
+- `docs/product-advisories/archived/ADVISORY_SBOM_LINEAGE_GRAPH.md`
+- `docs/modules/attestor/architecture.md` (ProofChain section)
+- `docs/modules/policy/architecture.md` (Determinism section)
+- `docs/modules/sbomservice/lineage/architecture.md`
+- `docs/modules/replay/architecture.md`
+
--- a/docs/product-advisories/archived/ANALYSIS_20251229_lineage_crossdistro_gap.md
+++ b/docs/product-advisories/archived/ANALYSIS_20251229_lineage_crossdistro_gap.md
@@ -0,0 +1,247 @@
+# Gap Analysis: SBOM Lineage Graph & Cross-Distro Vulnerability Intelligence
+
+> **Analysis Date:** 2025-12-29
+> **Advisory Source:** Product advisory proposing SBOM Lineage visualization and cross-distro CSAF/VEX unification
+> **Conclusion:** Advisory significantly underestimates existing implementation. ~85% already complete. Proceed with targeted sprints.
+
+---
+
+## Executive Summary
+
+The product advisory proposed two major features:
+
+1. **SBOM Lineage Graph** - Git-like visualization of container image ancestry with hover-to-proof micro-interactions
+2. **Cross-Distro Vulnerability Intelligence** - Unified CSAF/VEX ingestion across Linux distributions
+
+**Key Finding:** Both features are substantially implemented. The advisory dramatically underestimates existing capability.
+
+| Feature Area | Advisory Implied | Actual Status | Gap |
+|--------------|------------------|---------------|-----|
+| Lineage Architecture | New design needed | 100% documented | None |
+| Lineage UI Components | Build from scratch | ~80% complete (41 files) | API wiring |
+| Version Comparators | Need all new | 100% complete | None |
+| Distro Connectors | Need 5+ connectors | 5/6 complete | Astra only |
+| Patch Fingerprinting | New capability | 100% complete | None |
+| Trust Lattice | New framework | 100% complete | None |
+| Proposed UAS Schema | Adopt schema | **SKIP** | Existing model superior |
+
+**Recommendation:** Execute 3 targeted sprints (~34 tasks) instead of ~50+ implied by advisory.
+
+---
+
+## Detailed Gap Analysis
+
+### 1. SBOM Lineage Graph
+
+#### Architecture (docs/modules/sbomservice/lineage/architecture.md)
+
+| Component | Status | Evidence |
+|-----------|--------|----------|
+| DAG data model | ✅ Complete | `LineageNode`, `LineageEdge` records defined |
+| Edge types (parent/build/base) | ✅ Complete | `LineageRelationship` enum with 3 types |
+| Node badges (vulns/signature) | ✅ Complete | Badge structure in architecture |
+| Replay hash integration | ✅ Complete | `replayHash` field on nodes |
+| API contracts | ✅ Documented | 3 endpoints fully specified |
+| Database schema | ✅ Designed | 3 tables with indexes |
+| Caching strategy | ✅ Designed | Valkey keys with TTLs |
+| Determinism rules | ✅ Specified | Ordering rules documented |
+
+**Gap:** API endpoints not implemented. Database tables not migrated.
+
+#### UI Components (src/Web/StellaOps.Web/src/app/features/lineage/)
+
+| Component | Files | Status |
+|-----------|-------|--------|
+| Main graph visualization | `lineage-graph.component.ts` | ✅ 1000+ LOC |
+| Hover cards | `lineage-hover-card.component.ts` | ✅ Complete |
+| SBOM diff display | `lineage-sbom-diff.component.ts` | ✅ Complete |
+| VEX diff display | `lineage-vex-diff.component.ts` | ✅ Complete |
+| Compare panel | `lineage-compare-panel.component.ts` | ✅ Complete |
+| Services | `lineage.service.ts` | ⚠️ Stubs only |
+
+**Gap:** Services use mock data. Need API wiring.
+
+### 2. Cross-Distro Vulnerability Intelligence
+
+#### Advisory Connectors (src/Concelier/__Connectors/)
+
+| Distro | Connector | Version Comparator | Status |
+|--------|-----------|-------------------|--------|
+| Red Hat | `StellaOps.Concelier.Connector.RedHat` | rpm NEVRA | ✅ Complete |
+| SUSE | `StellaOps.Concelier.Connector.Suse` | rpm NEVRA | ✅ Complete |
+| Ubuntu | `StellaOps.Concelier.Connector.Ubuntu` | dpkg EVR | ✅ Complete |
+| Debian | `StellaOps.Concelier.Connector.Debian` | dpkg EVR | ✅ Complete |
+| Alpine | `StellaOps.Concelier.Connector.Alpine` | apk -r<pkgrel> | ✅ Complete |
+| **Astra Linux** | None | dpkg EVR (inherit) | ❌ **Gap** |
+
+#### Version Comparators (src/__Libraries/StellaOps.VersionComparison/)
+
+| Comparator | Location | Status |
+|------------|----------|--------|
+| `RpmVersionComparer` | `Comparers/RpmVersionComparer.cs` | ✅ Complete |
+| `DebianVersionComparer` | `Comparers/DebianVersionComparer.cs` | ✅ Complete |
+| `ApkVersionComparer` | `src/Concelier/__Libraries/.../ApkVersionComparer.cs` | ✅ Complete |
+| `SemVerComparer` | `Comparers/SemVerComparer.cs` | ✅ Complete |
+
+**Gap:** None. All version comparators implemented.
+
+#### Patch Fingerprinting (Feedser)
+
+| Component | Location | Status |
+|-----------|----------|--------|
+| HunkSig extractor | `src/Feedser/StellaOps.Feedser.Core/HunkSigExtractor.cs` | ✅ Complete |
+| Binary fingerprinting | `src/Feedser/StellaOps.Feedser.BinaryAnalysis/` | ✅ Complete |
+| TLSH fuzzy hashing | `Fingerprinters/SimplifiedTlshFingerprinter.cs` | ✅ Complete |
+| Instruction hash | `Fingerprinters/InstructionHashFingerprinter.cs` | ✅ Complete |
+
+**Gap:** None. Four-tier evidence system fully implemented.
+
+#### Trust Lattice (VexLens)
+
+| Component | Status | Evidence |
+|-----------|--------|----------|
+| 3-component trust vector | ✅ Complete | Provenance/Coverage/Replayability |
+| Lattice join semantics | ✅ Complete | `unknown < under_investigation < ...` |
+| Weighted scoring | ✅ Complete | Configurable weights in consensus |
+| Issuer trust profiles | ✅ Complete | IssuerDirectory integration |
+
+**Gap:** None. Trust framework fully implemented.
+
+### 3. Proposed UAS Schema - **RECOMMENDATION: SKIP**
+
+The advisory proposed a "Unified Advisory Schema" (UAS). Analysis shows this should be **skipped**:
+
+| Aspect | Proposed UAS | Existing Model | Decision |
+|--------|--------------|----------------|----------|
+| Conflict handling | Silent merge | Link-Not-Merge (preserves conflicts) | **Existing superior** |
+| Trust modeling | Single score | 3-component vector | **Existing superior** |
+| Evidence provenance | Lost in merge | AdvisoryLinkset preserves | **Existing superior** |
+| AOC compliance | Unknown | Append-Only Contract enforced | **Existing superior** |
+
+The existing `AdvisoryObservation` + `AdvisoryLinkset` model with Link-Not-Merge semantics is architecturally superior. UAS would require significant regression.
+
+---
+
+## Sprint Execution Plan
+
+### Sprint Dependency Graph
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                                                             │
+│  SPRINT_20251229_005_001_BE_sbom_lineage_api               │
+│  (13 tasks)                                                 │
+│  - Database migrations                                      │
+│  - Repository implementations                               │
+│  - API endpoints                                            │
+│  - Caching layer                                            │
+│                                                             │
+└──────────────────────────┬──────────────────────────────────┘
+                           │
+                           ▼
+┌─────────────────────────────────────────────────────────────┐
+│                                                             │
+│  SPRINT_20251229_005_003_FE_lineage_ui_wiring              │
+│  (9 tasks)                                                  │
+│  - Service API calls                                        │
+│  - Component data binding                                   │
+│  - Error/loading states                                     │
+│  - E2E tests                                                │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+
+┌─────────────────────────────────────────────────────────────┐
+│                                                             │
+│  SPRINT_20251229_005_002_CONCEL_astra_connector            │
+│  (12 tasks) - INDEPENDENT                                   │
+│  - Research advisory format                                 │
+│  - Connector implementation                                 │
+│  - Version matcher (dpkg EVR)                               │
+│  - Integration tests                                        │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Task Summary
+
+| Sprint | Module | Tasks | Effort Estimate |
+|--------|--------|-------|-----------------|
+| 005_001 | BE (SbomService) | 13 | Medium |
+| 005_002 | CONCEL (Concelier) | 12 | Medium-High (research required) |
+| 005_003 | FE (Web) | 9 | Low-Medium |
+| **Total** | | **34** | |
+
+### Critical Path
+
+1. **BE API (005_001)** must complete before **FE Wiring (005_003)**
+2. **Astra Connector (005_002)** is independent - can run in parallel
+3. No blocking dependencies on existing CGS infrastructure sprint (005_001_001)
+
+---
+
+## Architecture Decisions
+
+### Confirmed Decisions (No Change Needed)
+
+| ID | Decision | Rationale |
+|----|----------|-----------|
+| AD-001 | Link-Not-Merge for advisories | Preserves conflict evidence |
+| AD-002 | 3-component trust vector | Superior to single score |
+| AD-003 | Deterministic JSON serialization | Enables replay verification |
+| AD-004 | Valkey for hover cache | Matches existing infrastructure |
+| AD-005 | dpkg EVR for Astra | Astra is Debian-based |
+
+### Pending Decisions
+
+| ID | Decision | Owner | Deadline |
+|----|----------|-------|----------|
+| PD-001 | Astra advisory feed format | Research in Sprint 005_002 | Before ASTRA-002 |
+| PD-002 | Evidence pack size limit | Product | Before LIN-010 |
+| PD-003 | Astra air-gap bundle strategy | Operations | Before ASTRA-007 |
+
+---
+
+## Risk Assessment
+
+| Risk | Probability | Impact | Mitigation |
+|------|------------|--------|------------|
+| Astra feed unavailable or undocumented | Medium | High | Contact Astra directly; fall back to manual advisory import |
+| UI components need significant refactoring | Low | Medium | Components are well-structured; only service layer changes |
+| Backend API performance under load | Low | Medium | Caching strategy designed; load test before production |
+| Database migration conflicts | Low | Low | Migrations are additive only |
+
+---
+
+## Appendix: Evidence Locations
+
+### Documentation
+- `docs/modules/sbomservice/lineage/architecture.md` - Lineage architecture
+- `docs/modules/concelier/architecture.md` - Advisory ingestion
+- `docs/modules/feedser/architecture.md` - Patch fingerprinting
+- `docs/modules/vex-lens/architecture.md` - Trust lattice
+
+### Code
+- `src/Web/StellaOps.Web/src/app/features/lineage/` - UI components (41 files)
+- `src/Concelier/__Connectors/` - Advisory connectors (5 implemented)
+- `src/__Libraries/StellaOps.VersionComparison/` - Version comparators
+- `src/Feedser/` - Patch signature extraction
+
+### Sprints Created
+- `docs/implplan/SPRINT_20251229_005_001_BE_sbom_lineage_api.md`
+- `docs/implplan/SPRINT_20251229_005_002_CONCEL_astra_connector.md`
+- `docs/implplan/SPRINT_20251229_005_003_FE_lineage_ui_wiring.md`
+
+---
+
+## Conclusion
+
+The product advisory is **valuable for prioritization** but significantly underestimates existing implementation maturity. The StellaOps codebase already contains:
+
+- Complete architecture documentation for SBOM Lineage
+- ~80% complete UI implementation
+- 5 of 6 distro connectors fully implemented
+- All version comparators implemented
+- Complete patch fingerprinting and trust frameworks
+
+**Recommended Action:** Execute the 3 targeted sprints totaling 34 tasks. Skip the proposed UAS schema in favor of the existing superior model. The only significant new development is the Astra Linux connector.
+