Fix build and code structure improvements. New but essential UI functionality. CI improvements. Documentation improvements. AI module improvements.
This commit is contained in:
StellaOps Bot
469
docs/modules/sbomservice/lineage/architecture.md
Normal file
469
docs/modules/sbomservice/lineage/architecture.md
Normal file
@@ -0,0 +1,469 @@
|
||||
# SBOM Lineage Graph Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The SBOM Lineage Graph provides a Git-like visualization of container image ancestry with hover-to-proof micro-interactions. It enables auditors and developers to explore SBOM/VEX deltas across artifact versions, turning evidence into an explorable UX.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Lineage Graph
|
||||
|
||||
A directed acyclic graph (DAG) where:
|
||||
- **Nodes** represent artifact versions (SBOM snapshots)
|
||||
- **Edges** represent relationships between versions
|
||||
|
||||
### Edge Types
|
||||
|
||||
| Type | Description | Example |
|
||||
|------|-------------|---------|
|
||||
| `parent` | Direct version succession | v1.0 → v1.1 of same image |
|
||||
| `build` | Same CI build produced multiple artifacts | Multi-arch build |
|
||||
| `base` | Derived from base image | `FROM alpine:3.19` |
|
||||
|
||||
### Node Attributes
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Node: sha256:abc123... │
|
||||
├─────────────────────────────────────┤
|
||||
│ Artifact: registry/app:v1.2 │
|
||||
│ Sequence: 42 │
|
||||
│ Created: 2025-12-28T10:30:00Z │
|
||||
│ Source: scanner │
|
||||
├─────────────────────────────────────┤
|
||||
│ Badges: │
|
||||
│ • 3 new vulns (🔴) │
|
||||
│ • 2 resolved (🟢) │
|
||||
│ • signature ✓ │
|
||||
├─────────────────────────────────────┤
|
||||
│ Replay Hash: sha256:def456... │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌─────────────────┐ ┌──────────────────┐
|
||||
│ Scanner │────▶│ SbomService │────▶│ VexLens │
|
||||
│ │ │ │ │ │
|
||||
│ • OCI Parse │ │ • Ledger Store │ │ • Consensus │
|
||||
│ • Ancestry │ │ • Edge Persist │ │ • Delta Compute │
|
||||
│ • SBOM Gen │ │ • Diff Engine │ │ • Status Track │
|
||||
└──────────────┘ └─────────────────┘ └──────────────────┘
|
||||
│ │ │
|
||||
└────────────────────┼───────────────────────┘
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Lineage API │
|
||||
│ │
|
||||
│ • Graph Query │
|
||||
│ • Diff Compute │
|
||||
│ • Export Pack │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Frontend UI │
|
||||
│ │
|
||||
│ • Lane View │
|
||||
│ • Hover Cards │
|
||||
│ • Compare Mode │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## Component Architecture
|
||||
|
||||
### 1. OCI Ancestry Extractor (Scanner)
|
||||
|
||||
Extracts parent/base image information from OCI manifests.
|
||||
|
||||
```csharp
|
||||
public interface IOciAncestryExtractor
|
||||
{
|
||||
ValueTask<OciAncestry> ExtractAncestryAsync(
|
||||
string imageReference,
|
||||
CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public sealed record OciAncestry(
|
||||
string ImageDigest,
|
||||
string? BaseImageDigest,
|
||||
string? BaseImageRef,
|
||||
IReadOnlyList<string> LayerDigests,
|
||||
IReadOnlyList<OciHistoryEntry> History);
|
||||
|
||||
public sealed record OciHistoryEntry(
|
||||
string CreatedBy,
|
||||
DateTimeOffset Created,
|
||||
bool EmptyLayer);
|
||||
```
|
||||
|
||||
**Implementation Notes:**
|
||||
- Parse OCI image config `history` field
|
||||
- Extract `FROM` instruction from first non-empty layer
|
||||
- Handle multi-stage builds by tracking layer boundaries
|
||||
- Fall back to layer digest heuristics when history unavailable
|
||||
|
||||
### 2. Lineage Edge Repository (SbomService)
|
||||
|
||||
Persists relationships between artifact versions.
|
||||
|
||||
```csharp
|
||||
public interface ISbomLineageEdgeRepository
|
||||
{
|
||||
ValueTask<LineageEdge> AddAsync(
|
||||
LineageEdge edge,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<IReadOnlyList<LineageEdge>> GetChildrenAsync(
|
||||
string parentDigest,
|
||||
Guid tenantId,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<IReadOnlyList<LineageEdge>> GetParentsAsync(
|
||||
string childDigest,
|
||||
Guid tenantId,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<LineageGraph> GetGraphAsync(
|
||||
string artifactDigest,
|
||||
Guid tenantId,
|
||||
int maxDepth,
|
||||
CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public sealed record LineageEdge(
|
||||
Guid Id,
|
||||
string ParentDigest,
|
||||
string ChildDigest,
|
||||
LineageRelationship Relationship,
|
||||
Guid TenantId,
|
||||
DateTimeOffset CreatedAt);
|
||||
|
||||
public enum LineageRelationship
|
||||
{
|
||||
Parent,
|
||||
Build,
|
||||
Base
|
||||
}
|
||||
```
|
||||
|
||||
### 3. VEX Delta Repository (Excititor)
|
||||
|
||||
Tracks VEX status changes between artifact versions.
|
||||
|
||||
```csharp
|
||||
public interface IVexDeltaRepository
|
||||
{
|
||||
ValueTask<VexDelta> AddAsync(
|
||||
VexDelta delta,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<IReadOnlyList<VexDelta>> GetDeltasAsync(
|
||||
string fromDigest,
|
||||
string toDigest,
|
||||
Guid tenantId,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<IReadOnlyList<VexDelta>> GetDeltasByCveAsync(
|
||||
string cve,
|
||||
Guid tenantId,
|
||||
int limit,
|
||||
CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public sealed record VexDelta(
|
||||
Guid Id,
|
||||
string FromArtifactDigest,
|
||||
string ToArtifactDigest,
|
||||
string Cve,
|
||||
VexStatus FromStatus,
|
||||
VexStatus ToStatus,
|
||||
VexDeltaRationale Rationale,
|
||||
string ReplayHash,
|
||||
string? AttestationDigest,
|
||||
Guid TenantId,
|
||||
DateTimeOffset CreatedAt);
|
||||
|
||||
public sealed record VexDeltaRationale(
|
||||
string Reason,
|
||||
string? EvidenceLink,
|
||||
IReadOnlyDictionary<string, string> Metadata);
|
||||
```
|
||||
|
||||
### 4. SBOM-Verdict Link Repository (SbomService)
|
||||
|
||||
Links SBOM versions to VEX consensus decisions.
|
||||
|
||||
```csharp
|
||||
public interface ISbomVerdictLinkRepository
|
||||
{
|
||||
ValueTask LinkAsync(
|
||||
SbomVerdictLink link,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<IReadOnlyList<SbomVerdictLink>> GetVerdictsBySbomAsync(
|
||||
Guid sbomVersionId,
|
||||
Guid tenantId,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<IReadOnlyList<SbomVerdictLink>> GetSbomsByCveAsync(
|
||||
string cve,
|
||||
Guid tenantId,
|
||||
int limit,
|
||||
CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public sealed record SbomVerdictLink(
|
||||
Guid SbomVersionId,
|
||||
string Cve,
|
||||
Guid ConsensusProjectionId,
|
||||
VexStatus VerdictStatus,
|
||||
decimal ConfidenceScore,
|
||||
Guid TenantId,
|
||||
DateTimeOffset LinkedAt);
|
||||
```
|
||||
|
||||
### 5. Lineage Graph Service (SbomService)
|
||||
|
||||
Orchestrates lineage queries and diff computation.
|
||||
|
||||
```csharp
|
||||
public interface ILineageGraphService
|
||||
{
|
||||
ValueTask<LineageGraphResponse> GetLineageAsync(
|
||||
string artifactDigest,
|
||||
Guid tenantId,
|
||||
LineageQueryOptions options,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<LineageDiffResponse> GetDiffAsync(
|
||||
string fromDigest,
|
||||
string toDigest,
|
||||
Guid tenantId,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
ValueTask<LineageCompareResponse> CompareAsync(
|
||||
string digestA,
|
||||
string digestB,
|
||||
Guid tenantId,
|
||||
CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public sealed record LineageQueryOptions(
|
||||
int MaxDepth = 10,
|
||||
bool IncludeVerdicts = true,
|
||||
bool IncludeBadges = true);
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
### sbom_lineage_edges
|
||||
|
||||
```sql
|
||||
CREATE TABLE sbom_lineage_edges (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
parent_digest TEXT NOT NULL,
|
||||
child_digest TEXT NOT NULL,
|
||||
relationship TEXT NOT NULL CHECK (relationship IN ('parent', 'build', 'base')),
|
||||
tenant_id UUID NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
UNIQUE (parent_digest, child_digest, tenant_id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_lineage_edges_parent ON sbom_lineage_edges(parent_digest, tenant_id);
|
||||
CREATE INDEX idx_lineage_edges_child ON sbom_lineage_edges(child_digest, tenant_id);
|
||||
CREATE INDEX idx_lineage_edges_created ON sbom_lineage_edges(tenant_id, created_at DESC);
|
||||
```
|
||||
|
||||
### vex_deltas
|
||||
|
||||
```sql
|
||||
CREATE TABLE vex_deltas (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
from_artifact_digest TEXT NOT NULL,
|
||||
to_artifact_digest TEXT NOT NULL,
|
||||
cve TEXT NOT NULL,
|
||||
from_status TEXT NOT NULL,
|
||||
to_status TEXT NOT NULL,
|
||||
rationale JSONB NOT NULL DEFAULT '{}',
|
||||
replay_hash TEXT NOT NULL,
|
||||
attestation_digest TEXT,
|
||||
tenant_id UUID NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
UNIQUE (from_artifact_digest, to_artifact_digest, cve, tenant_id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_vex_deltas_to ON vex_deltas(to_artifact_digest, tenant_id);
|
||||
CREATE INDEX idx_vex_deltas_cve ON vex_deltas(cve, tenant_id);
|
||||
CREATE INDEX idx_vex_deltas_created ON vex_deltas(tenant_id, created_at DESC);
|
||||
```
|
||||
|
||||
### sbom_verdict_links
|
||||
|
||||
```sql
|
||||
CREATE TABLE sbom_verdict_links (
|
||||
sbom_version_id UUID NOT NULL,
|
||||
cve TEXT NOT NULL,
|
||||
consensus_projection_id UUID NOT NULL,
|
||||
verdict_status TEXT NOT NULL,
|
||||
confidence_score DECIMAL(5,4) NOT NULL,
|
||||
tenant_id UUID NOT NULL,
|
||||
linked_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
PRIMARY KEY (sbom_version_id, cve, tenant_id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_verdict_links_cve ON sbom_verdict_links(cve, tenant_id);
|
||||
CREATE INDEX idx_verdict_links_projection ON sbom_verdict_links(consensus_projection_id);
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### GET /api/v1/lineage/{artifactDigest}
|
||||
|
||||
Returns the lineage graph for an artifact.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"artifact": "sha256:abc123...",
|
||||
"nodes": [
|
||||
{
|
||||
"id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"digest": "sha256:abc123...",
|
||||
"artifactRef": "registry/app:v1.2",
|
||||
"sequenceNumber": 42,
|
||||
"createdAt": "2025-12-28T10:30:00Z",
|
||||
"source": "scanner",
|
||||
"badges": {
|
||||
"newVulns": 3,
|
||||
"resolvedVulns": 2,
|
||||
"signatureStatus": "valid"
|
||||
},
|
||||
"replayHash": "sha256:def456..."
|
||||
}
|
||||
],
|
||||
"edges": [
|
||||
{
|
||||
"from": "sha256:parent...",
|
||||
"to": "sha256:abc123...",
|
||||
"relationship": "parent"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/v1/lineage/diff
|
||||
|
||||
Returns component and VEX diffs between two versions.
|
||||
|
||||
**Query Parameters:**
|
||||
- `from` - Source artifact digest
|
||||
- `to` - Target artifact digest
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"sbomDiff": {
|
||||
"added": [
|
||||
{"purl": "pkg:npm/lodash@4.17.21", "version": "4.17.21", "license": "MIT"}
|
||||
],
|
||||
"removed": [
|
||||
{"purl": "pkg:npm/lodash@4.17.20", "version": "4.17.20", "license": "MIT"}
|
||||
],
|
||||
"versionChanged": [
|
||||
{"purl": "pkg:npm/axios@1.6.0", "fromVersion": "1.5.0", "toVersion": "1.6.0"}
|
||||
]
|
||||
},
|
||||
"vexDiff": [
|
||||
{
|
||||
"cve": "CVE-2024-1234",
|
||||
"fromStatus": "affected",
|
||||
"toStatus": "not_affected",
|
||||
"reason": "Component removed",
|
||||
"evidenceLink": "/evidence/abc123"
|
||||
}
|
||||
],
|
||||
"reachabilityDiff": [
|
||||
{
|
||||
"cve": "CVE-2024-5678",
|
||||
"fromStatus": "reachable",
|
||||
"toStatus": "unreachable",
|
||||
"pathsRemoved": 3,
|
||||
"gatesAdded": ["auth_required"]
|
||||
}
|
||||
],
|
||||
"replayHash": "sha256:ghi789..."
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/v1/lineage/export
|
||||
|
||||
Exports evidence pack for artifact(s).
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"artifactDigests": ["sha256:abc123..."],
|
||||
"includeAttestations": true,
|
||||
"sign": true
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"downloadUrl": "/exports/pack-xyz.zip",
|
||||
"bundleDigest": "sha256:bundle...",
|
||||
"expiresAt": "2025-12-28T11:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Caching Strategy
|
||||
|
||||
### Hover Card Cache (Valkey)
|
||||
|
||||
- **Key:** `lineage:hover:{tenantId}:{artifactDigest}`
|
||||
- **TTL:** 5 minutes
|
||||
- **Invalidation:** On new SBOM version or VEX update
|
||||
- **Target:** <150ms response time
|
||||
|
||||
### Compare Cache (Valkey)
|
||||
|
||||
- **Key:** `lineage:compare:{tenantId}:{digestA}:{digestB}`
|
||||
- **TTL:** 10 minutes
|
||||
- **Invalidation:** On new VEX data for either artifact
|
||||
|
||||
## Determinism Guarantees
|
||||
|
||||
1. **Node Ordering:** Sorted by `sequenceNumber DESC`, then `createdAt DESC`
|
||||
2. **Edge Ordering:** Sorted by `(from, to, relationship)` lexicographically
|
||||
3. **Component Diff:** Components sorted by `purl` (ordinal)
|
||||
4. **VEX Diff:** Sorted by `cve` (ordinal)
|
||||
5. **Replay Hash:** SHA256 of deterministically serialized inputs
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Tenant Isolation:** All queries scoped by `tenant_id`
|
||||
2. **Digest Validation:** Verify artifact digest format before queries
|
||||
3. **Rate Limiting:** Apply per-tenant rate limits on graph queries
|
||||
4. **Export Authorization:** Verify `lineage:export` scope for pack generation
|
||||
|
||||
## Metrics
|
||||
|
||||
| Metric | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `sbom_lineage_graph_queries_total` | Counter | Graph queries by tenant |
|
||||
| `sbom_lineage_diff_latency_seconds` | Histogram | Diff computation latency |
|
||||
| `sbom_lineage_hover_cache_hits_total` | Counter | Hover card cache hits |
|
||||
| `sbom_lineage_export_size_bytes` | Histogram | Evidence pack sizes |
|
||||
| `vex_deltas_created_total` | Counter | VEX deltas stored |
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error Code | Description | HTTP Status |
|
||||
|------------|-------------|-------------|
|
||||
| `LINEAGE_NOT_FOUND` | Artifact not in lineage graph | 404 |
|
||||
| `LINEAGE_DEPTH_EXCEEDED` | Max depth limit reached | 400 |
|
||||
| `LINEAGE_DIFF_INVALID` | Same digest for from/to | 400 |
|
||||
| `LINEAGE_EXPORT_TOO_LARGE` | Pack exceeds size limit | 413 |
|
||||
Reference in New Issue
Block a user