Add reference architecture and testing strategy documentation
- Created a new document for the Stella Ops Reference Architecture outlining the system's topology, trust boundaries, artifact association, and interfaces. - Developed a comprehensive Testing Strategy document detailing the importance of offline readiness, interoperability, determinism, and operational guardrails. - Introduced a README for the Testing Strategy, summarizing processing details and key concepts implemented. - Added guidance for AI agents and developers in the tests directory, including directory structure, test categories, key patterns, and rules for test development.
This commit is contained in:
558
docs/modules/binaryindex/architecture.md
Normal file
558
docs/modules/binaryindex/architecture.md
Normal file
@@ -0,0 +1,558 @@
|
||||
# BinaryIndex Module Architecture
|
||||
|
||||
> **Ownership:** Scanner Guild + Concelier Guild
|
||||
> **Status:** DRAFT
|
||||
> **Version:** 1.0.0
|
||||
> **Related:** [High-Level Architecture](../../07_HIGH_LEVEL_ARCHITECTURE.md), [Scanner Architecture](../scanner/architecture.md), [Concelier Architecture](../concelier/architecture.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The **BinaryIndex** module provides a vulnerable binaries database that enables detection of vulnerable code at the binary level, independent of package metadata. This addresses a critical gap in vulnerability scanning: package version strings can lie (backports, custom builds, stripped metadata), but **binary identity doesn't lie**.
|
||||
|
||||
### 1.1 Problem Statement
|
||||
|
||||
Traditional vulnerability scanners rely on package version matching, which fails in several scenarios:
|
||||
|
||||
1. **Backported patches** - Distros backport security fixes without changing upstream version
|
||||
2. **Custom/vendored builds** - Binaries compiled from source without package metadata
|
||||
3. **Stripped binaries** - Debug info and version strings removed
|
||||
4. **Static linking** - Vulnerable library code embedded in final binary
|
||||
5. **Container base images** - Distroless or scratch images with no package DB
|
||||
|
||||
### 1.2 Solution: Binary-First Vulnerability Detection
|
||||
|
||||
BinaryIndex provides three tiers of binary identification:
|
||||
|
||||
| Tier | Method | Precision | Coverage |
|
||||
|------|--------|-----------|----------|
|
||||
| A | Package/version range matching | Medium | High |
|
||||
| B | Build-ID/hash catalog (exact binary identity) | High | Medium |
|
||||
| C | Function fingerprints (CFG/basic-block hashes) | Very High | Targeted |
|
||||
|
||||
### 1.3 Module Scope
|
||||
|
||||
**In Scope:**
|
||||
- Binary identity extraction (Build-ID, PE CodeView GUID, Mach-O UUID)
|
||||
- Binary-to-advisory mapping database
|
||||
- Fingerprint storage and matching engine
|
||||
- Fix index for patch-aware backport handling
|
||||
- Integration with Scanner.Worker for binary lookup
|
||||
|
||||
**Out of Scope:**
|
||||
- Binary disassembly/analysis (provided by Scanner.Analyzers.Native)
|
||||
- Runtime binary tracing (provided by Zastava)
|
||||
- SBOM generation (provided by Scanner)
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
### 2.1 System Context
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ External Systems │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Distro Repos │ │ Debug Symbol │ │ Upstream Source │ │
|
||||
│ │ (Debian, RPM, │ │ Servers │ │ (GitHub, etc.) │ │
|
||||
│ │ Alpine) │ │ (debuginfod) │ │ │ │
|
||||
│ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │
|
||||
└───────────│─────────────────────│─────────────────────│──────────────────┘
|
||||
│ │ │
|
||||
v v v
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ BinaryIndex Module │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Corpus Ingestion Layer │ │
|
||||
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
||||
│ │ │ DebianCorpus │ │ RpmCorpus │ │ AlpineCorpus │ │ │
|
||||
│ │ │ Connector │ │ Connector │ │ Connector │ │ │
|
||||
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ v │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Processing Layer │ │
|
||||
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
||||
│ │ │ BinaryFeature│ │ FixIndex │ │ Fingerprint │ │ │
|
||||
│ │ │ Extractor │ │ Builder │ │ Generator │ │ │
|
||||
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ v │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Storage Layer │ │
|
||||
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
||||
│ │ │ PostgreSQL │ │ RustFS │ │ Valkey │ │ │
|
||||
│ │ │ (binaries │ │ (fingerprint │ │ (lookup │ │ │
|
||||
│ │ │ schema) │ │ blobs) │ │ cache) │ │ │
|
||||
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ v │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Query Layer │ │
|
||||
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ IBinaryVulnerabilityService │ │ │
|
||||
│ │ │ - LookupByBuildIdAsync(buildId) │ │ │
|
||||
│ │ │ - LookupByFingerprintAsync(fingerprint) │ │ │
|
||||
│ │ │ - LookupBatchAsync(identities) │ │ │
|
||||
│ │ │ - GetFixStatusAsync(distro, release, sourcePkg, cve) │ │ │
|
||||
│ │ └──────────────────────────────────────────────────────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
v
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ Consuming Modules │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Scanner.Worker │ │ Policy Engine │ │ Findings Ledger │ │
|
||||
│ │ (binary lookup │ │ (evidence in │ │ (match records) │ │
|
||||
│ │ during scan) │ │ proof chain) │ │ │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 2.2 Component Breakdown
|
||||
|
||||
#### 2.2.1 Corpus Connectors
|
||||
|
||||
Plugin-based connectors that ingest binaries from distribution repositories.
|
||||
|
||||
```csharp
|
||||
public interface IBinaryCorpusConnector
|
||||
{
|
||||
string ConnectorId { get; }
|
||||
string[] SupportedDistros { get; }
|
||||
|
||||
Task<CorpusSnapshot> FetchSnapshotAsync(CorpusQuery query, CancellationToken ct);
|
||||
Task<IAsyncEnumerable<ExtractedBinary>> ExtractBinariesAsync(PackageReference pkg, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
**Implementations:**
|
||||
- `DebianBinaryCorpusConnector` - Debian/Ubuntu packages + debuginfo
|
||||
- `RpmBinaryCorpusConnector` - RHEL/Fedora/CentOS + SRPM
|
||||
- `AlpineBinaryCorpusConnector` - Alpine APK + APKBUILD
|
||||
|
||||
#### 2.2.2 Binary Feature Extractor
|
||||
|
||||
Extracts identity and features from binaries. Reuses existing Scanner.Analyzers.Native capabilities.
|
||||
|
||||
```csharp
|
||||
public interface IBinaryFeatureExtractor
|
||||
{
|
||||
Task<BinaryIdentity> ExtractIdentityAsync(Stream binaryStream, CancellationToken ct);
|
||||
Task<BinaryFeatures> ExtractFeaturesAsync(Stream binaryStream, ExtractorOptions opts, CancellationToken ct);
|
||||
}
|
||||
|
||||
public sealed record BinaryIdentity(
|
||||
string Format, // elf, pe, macho
|
||||
string? BuildId, // ELF GNU Build-ID
|
||||
string? PeCodeViewGuid, // PE CodeView GUID + Age
|
||||
string? MachoUuid, // Mach-O LC_UUID
|
||||
string FileSha256,
|
||||
string TextSectionSha256);
|
||||
|
||||
public sealed record BinaryFeatures(
|
||||
BinaryIdentity Identity,
|
||||
string[] DynamicDeps, // DT_NEEDED
|
||||
string[] ExportedSymbols,
|
||||
string[] ImportedSymbols,
|
||||
BinaryHardening Hardening);
|
||||
```
|
||||
|
||||
#### 2.2.3 Fix Index Builder
|
||||
|
||||
Builds the patch-aware CVE fix index from distro sources.
|
||||
|
||||
```csharp
|
||||
public interface IFixIndexBuilder
|
||||
{
|
||||
Task BuildIndexAsync(DistroRelease distro, CancellationToken ct);
|
||||
Task<FixRecord?> GetFixRecordAsync(string distro, string release, string sourcePkg, string cveId, CancellationToken ct);
|
||||
}
|
||||
|
||||
public sealed record FixRecord(
|
||||
string Distro,
|
||||
string Release,
|
||||
string SourcePkg,
|
||||
string CveId,
|
||||
FixState State, // fixed, vulnerable, not_affected, wontfix, unknown
|
||||
string? FixedVersion, // Distro version string
|
||||
FixMethod Method, // security_feed, changelog, patch_header
|
||||
decimal Confidence, // 0.00-1.00
|
||||
FixEvidence Evidence);
|
||||
|
||||
public enum FixState { Fixed, Vulnerable, NotAffected, Wontfix, Unknown }
|
||||
public enum FixMethod { SecurityFeed, Changelog, PatchHeader, UpstreamPatchMatch }
|
||||
```
|
||||
|
||||
#### 2.2.4 Fingerprint Generator
|
||||
|
||||
Generates function-level fingerprints for vulnerable code detection.
|
||||
|
||||
```csharp
|
||||
public interface IVulnFingerprintGenerator
|
||||
{
|
||||
Task<ImmutableArray<VulnFingerprint>> GenerateAsync(
|
||||
string cveId,
|
||||
BinaryPair vulnAndFixed, // Reference builds
|
||||
FingerprintOptions opts,
|
||||
CancellationToken ct);
|
||||
}
|
||||
|
||||
public sealed record VulnFingerprint(
|
||||
string CveId,
|
||||
string Component, // e.g., openssl
|
||||
string Architecture, // x86-64, aarch64
|
||||
FingerprintType Type, // basic_block, cfg, combined
|
||||
string FingerprintId, // e.g., "bb-abc123..."
|
||||
byte[] FingerprintHash, // 16-32 bytes
|
||||
string? FunctionHint, // Function name if known
|
||||
decimal Confidence,
|
||||
FingerprintEvidence Evidence);
|
||||
|
||||
public enum FingerprintType { BasicBlock, ControlFlowGraph, StringReferences, Combined }
|
||||
```
|
||||
|
||||
#### 2.2.5 Binary Vulnerability Service
|
||||
|
||||
Main query interface for consumers.
|
||||
|
||||
```csharp
|
||||
public interface IBinaryVulnerabilityService
|
||||
{
|
||||
/// <summary>
|
||||
/// Look up vulnerabilities by Build-ID or equivalent binary identity.
|
||||
/// </summary>
|
||||
Task<ImmutableArray<BinaryVulnMatch>> LookupByIdentityAsync(
|
||||
BinaryIdentity identity,
|
||||
LookupOptions? opts = null,
|
||||
CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Look up vulnerabilities by function fingerprint.
|
||||
/// </summary>
|
||||
Task<ImmutableArray<BinaryVulnMatch>> LookupByFingerprintAsync(
|
||||
CodeFingerprint fingerprint,
|
||||
decimal minSimilarity = 0.95m,
|
||||
CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Batch lookup for scan performance.
|
||||
/// </summary>
|
||||
Task<ImmutableDictionary<string, ImmutableArray<BinaryVulnMatch>>> LookupBatchAsync(
|
||||
IEnumerable<BinaryIdentity> identities,
|
||||
LookupOptions? opts = null,
|
||||
CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Get distro-specific fix status (patch-aware).
|
||||
/// </summary>
|
||||
Task<FixRecord?> GetFixStatusAsync(
|
||||
string distro,
|
||||
string release,
|
||||
string sourcePkg,
|
||||
string cveId,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record BinaryVulnMatch(
|
||||
string CveId,
|
||||
string VulnerablePurl,
|
||||
MatchMethod Method, // buildid_catalog, fingerprint_match, range_match
|
||||
decimal Confidence,
|
||||
MatchEvidence Evidence);
|
||||
|
||||
public enum MatchMethod { BuildIdCatalog, FingerprintMatch, RangeMatch }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Data Model
|
||||
|
||||
### 3.1 PostgreSQL Schema (`binaries`)
|
||||
|
||||
The `binaries` schema stores binary identity, fingerprint, and match data.
|
||||
|
||||
```sql
|
||||
CREATE SCHEMA IF NOT EXISTS binaries;
|
||||
CREATE SCHEMA IF NOT EXISTS binaries_app;
|
||||
|
||||
-- RLS helper
|
||||
CREATE OR REPLACE FUNCTION binaries_app.require_current_tenant()
|
||||
RETURNS TEXT LANGUAGE plpgsql STABLE SECURITY DEFINER AS $$
|
||||
DECLARE v_tenant TEXT;
|
||||
BEGIN
|
||||
v_tenant := current_setting('app.tenant_id', true);
|
||||
IF v_tenant IS NULL OR v_tenant = '' THEN
|
||||
RAISE EXCEPTION 'app.tenant_id session variable not set';
|
||||
END IF;
|
||||
RETURN v_tenant;
|
||||
END;
|
||||
$$;
|
||||
```
|
||||
|
||||
#### 3.1.1 Core Tables
|
||||
|
||||
See `docs/db/schemas/binaries_schema_specification.md` for complete DDL.
|
||||
|
||||
**Key Tables:**
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `binaries.binary_identity` | Known binary identities (Build-ID, hashes) |
|
||||
| `binaries.binary_package_map` | Binary → package mapping per snapshot |
|
||||
| `binaries.vulnerable_buildids` | Build-IDs known to be vulnerable |
|
||||
| `binaries.vulnerable_fingerprints` | Function fingerprints for CVEs |
|
||||
| `binaries.cve_fix_index` | Patch-aware fix status per distro |
|
||||
| `binaries.fingerprint_matches` | Match results (findings evidence) |
|
||||
| `binaries.corpus_snapshots` | Corpus ingestion tracking |
|
||||
|
||||
### 3.2 RustFS Layout
|
||||
|
||||
```
|
||||
rustfs://stellaops/binaryindex/
|
||||
fingerprints/<algorithm>/<prefix>/<fingerprint_id>.bin
|
||||
corpus/<distro>/<release>/<snapshot_id>/manifest.json
|
||||
corpus/<distro>/<release>/<snapshot_id>/packages/<pkg>.metadata.json
|
||||
evidence/<match_id>.dsse.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Integration Points
|
||||
|
||||
### 4.1 Scanner.Worker Integration
|
||||
|
||||
During container scanning, Scanner.Worker queries BinaryIndex for each extracted binary:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant SW as Scanner.Worker
|
||||
participant BI as BinaryIndex
|
||||
participant PG as PostgreSQL
|
||||
participant FL as Findings Ledger
|
||||
|
||||
SW->>SW: Extract binary from layer
|
||||
SW->>SW: Compute BinaryIdentity
|
||||
SW->>BI: LookupByIdentityAsync(identity)
|
||||
BI->>PG: Query binaries.vulnerable_buildids
|
||||
PG-->>BI: Matches
|
||||
BI->>PG: Query binaries.cve_fix_index (if distro known)
|
||||
PG-->>BI: Fix status
|
||||
BI-->>SW: BinaryVulnMatch[]
|
||||
SW->>FL: RecordFinding(match, evidence)
|
||||
```
|
||||
|
||||
### 4.2 Concelier Integration
|
||||
|
||||
BinaryIndex subscribes to Concelier's advisory updates:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant CO as Concelier
|
||||
participant BI as BinaryIndex
|
||||
participant PG as PostgreSQL
|
||||
|
||||
CO->>CO: Ingest new advisory
|
||||
CO->>BI: advisory.created event
|
||||
BI->>BI: Check if affected packages in corpus
|
||||
BI->>PG: Update binaries.binary_vuln_assertion
|
||||
BI->>BI: Queue fingerprint generation (if high-impact)
|
||||
```
|
||||
|
||||
### 4.3 Policy Integration
|
||||
|
||||
Binary matches are recorded as proof segments:
|
||||
|
||||
```json
|
||||
{
|
||||
"segment_type": "binary_fingerprint_evidence",
|
||||
"payload": {
|
||||
"binary_identity": {
|
||||
"format": "elf",
|
||||
"build_id": "abc123...",
|
||||
"file_sha256": "def456..."
|
||||
},
|
||||
"matches": [
|
||||
{
|
||||
"cve_id": "CVE-2024-1234",
|
||||
"method": "buildid_catalog",
|
||||
"confidence": 0.98,
|
||||
"vulnerable_purl": "pkg:deb/debian/libssl3@1.1.1n-0+deb11u3"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. MVP Roadmap
|
||||
|
||||
### MVP 1: Known-Build Binary Catalog (Sprint 6000.0001)
|
||||
|
||||
**Goal:** Query "is this Build-ID vulnerable?" with distro-level precision.
|
||||
|
||||
**Deliverables:**
|
||||
- `binaries` PostgreSQL schema
|
||||
- Build-ID to package mapping tables
|
||||
- Basic CVE lookup by binary identity
|
||||
- Debian/Ubuntu corpus connector
|
||||
|
||||
### MVP 2: Patch-Aware Backport Handling (Sprint 6000.0002)
|
||||
|
||||
**Goal:** Handle "version says vulnerable but distro backported the fix."
|
||||
|
||||
**Deliverables:**
|
||||
- Fix index builder (changelog + patch header parsing)
|
||||
- Distro-specific version comparison
|
||||
- RPM corpus connector
|
||||
- Scanner.Worker integration
|
||||
|
||||
### MVP 3: Binary Fingerprint Factory (Sprint 6000.0003)
|
||||
|
||||
**Goal:** Detect vulnerable code independent of package metadata.
|
||||
|
||||
**Deliverables:**
|
||||
- Fingerprint storage and matching
|
||||
- Reference build generation pipeline
|
||||
- Fingerprint validation corpus
|
||||
- High-impact CVE coverage (OpenSSL, glibc, zlib, curl)
|
||||
|
||||
### MVP 4: Full Scanner Integration (Sprint 6000.0004)
|
||||
|
||||
**Goal:** Binary evidence in production scans.
|
||||
|
||||
**Deliverables:**
|
||||
- Scanner.Worker binary lookup integration
|
||||
- Findings Ledger binary match records
|
||||
- Proof segment attestations
|
||||
- CLI binary match inspection
|
||||
|
||||
---
|
||||
|
||||
## 6. Security Considerations
|
||||
|
||||
### 6.1 Trust Boundaries
|
||||
|
||||
1. **Corpus Ingestion** - Packages are untrusted; extraction runs in sandboxed workers
|
||||
2. **Fingerprint Generation** - Reference builds compiled in isolated environments
|
||||
3. **Query API** - Tenant-isolated via RLS; no cross-tenant data leakage
|
||||
|
||||
### 6.2 Signing & Provenance
|
||||
|
||||
- All corpus snapshots are signed (DSSE)
|
||||
- Fingerprint sets are versioned and signed
|
||||
- Every match result references evidence digests
|
||||
|
||||
### 6.3 Sandbox Requirements
|
||||
|
||||
Binary extraction and fingerprint generation MUST run with:
|
||||
- Seccomp profile restricting syscalls
|
||||
- Read-only root filesystem
|
||||
- No network access during analysis
|
||||
- Memory/CPU limits
|
||||
|
||||
---
|
||||
|
||||
## 7. Observability
|
||||
|
||||
### 7.1 Metrics
|
||||
|
||||
| Metric | Type | Labels |
|
||||
|--------|------|--------|
|
||||
| `binaryindex_lookup_total` | Counter | method, result |
|
||||
| `binaryindex_lookup_latency_ms` | Histogram | method |
|
||||
| `binaryindex_corpus_packages_total` | Gauge | distro, release |
|
||||
| `binaryindex_fingerprints_indexed` | Gauge | algorithm, component |
|
||||
| `binaryindex_match_confidence` | Histogram | method |
|
||||
|
||||
### 7.2 Traces
|
||||
|
||||
- `binaryindex.lookup` - Full lookup span
|
||||
- `binaryindex.corpus.ingest` - Corpus ingestion
|
||||
- `binaryindex.fingerprint.generate` - Fingerprint generation
|
||||
|
||||
---
|
||||
|
||||
## 8. Configuration
|
||||
|
||||
```yaml
|
||||
# binaryindex.yaml
|
||||
binaryindex:
|
||||
enabled: true
|
||||
|
||||
corpus:
|
||||
connectors:
|
||||
- type: debian
|
||||
enabled: true
|
||||
mirror: http://deb.debian.org/debian
|
||||
releases: [bookworm, bullseye]
|
||||
architectures: [amd64, arm64]
|
||||
- type: ubuntu
|
||||
enabled: true
|
||||
mirror: http://archive.ubuntu.com/ubuntu
|
||||
releases: [jammy, noble]
|
||||
|
||||
fingerprinting:
|
||||
enabled: true
|
||||
algorithms: [basic_block, cfg]
|
||||
target_components:
|
||||
- openssl
|
||||
- glibc
|
||||
- zlib
|
||||
- curl
|
||||
- sqlite
|
||||
min_function_size: 16 # bytes
|
||||
max_functions_per_binary: 10000
|
||||
|
||||
lookup:
|
||||
cache_ttl: 3600
|
||||
batch_size: 100
|
||||
timeout_ms: 5000
|
||||
|
||||
storage:
|
||||
postgres_schema: binaries
|
||||
rustfs_bucket: stellaops/binaryindex
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Testing Strategy
|
||||
|
||||
### 9.1 Unit Tests
|
||||
|
||||
- Identity extraction (Build-ID, hashes)
|
||||
- Fingerprint generation determinism
|
||||
- Fix index parsing (changelog, patch headers)
|
||||
|
||||
### 9.2 Integration Tests
|
||||
|
||||
- PostgreSQL schema validation
|
||||
- Full corpus ingestion flow
|
||||
- Scanner.Worker lookup integration
|
||||
|
||||
### 9.3 Regression Tests
|
||||
|
||||
- Known CVE detection (golden corpus)
|
||||
- Backport handling (Debian libssl example)
|
||||
- False positive rate validation
|
||||
|
||||
---
|
||||
|
||||
## 10. References
|
||||
|
||||
- Advisory: `docs/product-advisories/21-Dec-2025 - Mapping Evidence Within Compiled Binaries.md`
|
||||
- Scanner Native Analysis: `src/Scanner/StellaOps.Scanner.Analyzers.Native/`
|
||||
- Existing Fingerprinting: `src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/Binary/`
|
||||
- Build-ID Index: `src/Scanner/StellaOps.Scanner.Analyzers.Native/Index/`
|
||||
|
||||
---
|
||||
|
||||
*Document Version: 1.0.0*
|
||||
*Last Updated: 2025-12-21*
|
||||
461
docs/modules/gateway/architecture.md
Normal file
461
docs/modules/gateway/architecture.md
Normal file
@@ -0,0 +1,461 @@
|
||||
# component_architecture_gateway.md — **Stella Ops Gateway** (Sprint 3600)
|
||||
|
||||
> Derived from Reference Architecture Advisory and Router Architecture Specification
|
||||
|
||||
> **Scope.** The Gateway WebService is the single HTTP ingress point for all external traffic. It authenticates requests via Authority (DPoP/mTLS), routes to microservices via the Router binary protocol, aggregates OpenAPI specifications, and enforces tenant isolation.
|
||||
> **Ownership:** Platform Guild
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & Boundaries
|
||||
|
||||
### What Gateway Does
|
||||
|
||||
- **HTTP Ingress**: Single entry point for all external HTTP/HTTPS traffic
|
||||
- **Authentication**: DPoP and mTLS token validation via Authority integration
|
||||
- **Routing**: Routes HTTP requests to microservices via binary protocol (TCP/TLS)
|
||||
- **OpenAPI Aggregation**: Combines endpoint specs from all registered microservices
|
||||
- **Health Aggregation**: Provides unified health status from downstream services
|
||||
- **Rate Limiting**: Per-tenant and per-identity request throttling
|
||||
- **Tenant Propagation**: Extracts tenant context and propagates to microservices
|
||||
|
||||
### What Gateway Does NOT Do
|
||||
|
||||
- **Business Logic**: No domain logic; pure routing and auth
|
||||
- **Data Storage**: Stateless; no persistent state beyond connection cache
|
||||
- **Direct Database Access**: Never connects to PostgreSQL directly
|
||||
- **SBOM/VEX Processing**: Delegates to Scanner, Excititor, etc.
|
||||
|
||||
---
|
||||
|
||||
## 1) Solution & Project Layout
|
||||
|
||||
```
|
||||
src/Gateway/
|
||||
├── StellaOps.Gateway.WebService/
|
||||
│ ├── StellaOps.Gateway.WebService.csproj
|
||||
│ ├── Program.cs # DI bootstrap, transport init
|
||||
│ ├── Dockerfile
|
||||
│ ├── appsettings.json
|
||||
│ ├── appsettings.Development.json
|
||||
│ ├── Configuration/
|
||||
│ │ ├── GatewayOptions.cs # All configuration options
|
||||
│ │ └── TransportOptions.cs # TCP/TLS transport config
|
||||
│ ├── Middleware/
|
||||
│ │ ├── TenantMiddleware.cs # Tenant context extraction
|
||||
│ │ ├── RequestRoutingMiddleware.cs # HTTP → binary routing
|
||||
│ │ ├── AuthenticationMiddleware.cs # DPoP/mTLS validation
|
||||
│ │ └── RateLimitingMiddleware.cs # Per-tenant throttling
|
||||
│ ├── Services/
|
||||
│ │ ├── GatewayHostedService.cs # Transport lifecycle
|
||||
│ │ ├── OpenApiAggregationService.cs # Spec aggregation
|
||||
│ │ └── HealthAggregationService.cs # Downstream health
|
||||
│ └── Endpoints/
|
||||
│ ├── HealthEndpoints.cs # /health/*, /metrics
|
||||
│ └── OpenApiEndpoints.cs # /openapi.json, /openapi.yaml
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
```xml
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\__Libraries\StellaOps.Router.Gateway\..." />
|
||||
<ProjectReference Include="..\..\__Libraries\StellaOps.Router.Transport.Tcp\..." />
|
||||
<ProjectReference Include="..\..\__Libraries\StellaOps.Router.Transport.Tls\..." />
|
||||
<ProjectReference Include="..\..\Auth\StellaOps.Auth.ServerIntegration\..." />
|
||||
</ItemGroup>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2) External Dependencies
|
||||
|
||||
| Dependency | Purpose | Required |
|
||||
|------------|---------|----------|
|
||||
| **Authority** | OpTok validation, DPoP/mTLS | Yes |
|
||||
| **Router.Gateway** | Routing state, endpoint discovery | Yes |
|
||||
| **Router.Transport.Tcp** | Binary transport (dev) | Yes |
|
||||
| **Router.Transport.Tls** | Binary transport (prod) | Yes |
|
||||
| **Valkey/Redis** | Rate limiting state | Optional |
|
||||
|
||||
---
|
||||
|
||||
## 3) Contracts & Data Model
|
||||
|
||||
### Request Flow
|
||||
|
||||
```
|
||||
┌──────────────┐ HTTPS ┌─────────────────┐ Binary ┌─────────────────┐
|
||||
│ Client │ ─────────────► │ Gateway │ ────────────► │ Microservice │
|
||||
│ (CLI/UI) │ │ WebService │ Frame │ (Scanner, │
|
||||
│ │ ◄───────────── │ │ ◄──────────── │ Policy, etc) │
|
||||
└──────────────┘ HTTPS └─────────────────┘ Binary └─────────────────┘
|
||||
```
|
||||
|
||||
### Binary Frame Protocol
|
||||
|
||||
Gateway uses the Router binary protocol for internal communication:
|
||||
|
||||
| Frame Type | Purpose |
|
||||
|------------|---------|
|
||||
| HELLO | Microservice registration with endpoints |
|
||||
| HEARTBEAT | Health check and latency measurement |
|
||||
| REQUEST | HTTP request serialized to binary |
|
||||
| RESPONSE | HTTP response serialized from binary |
|
||||
| STREAM_DATA | Streaming response chunks |
|
||||
| CANCEL | Request cancellation propagation |
|
||||
|
||||
### Endpoint Descriptor
|
||||
|
||||
```csharp
|
||||
public sealed class EndpointDescriptor
|
||||
{
|
||||
public required string Method { get; init; } // GET, POST, etc.
|
||||
public required string Path { get; init; } // /api/v1/scans/{id}
|
||||
public required string ServiceName { get; init; } // scanner
|
||||
public required string Version { get; init; } // 1.0.0
|
||||
public TimeSpan DefaultTimeout { get; init; } // 30s
|
||||
public bool SupportsStreaming { get; init; } // true for large responses
|
||||
public IReadOnlyList<ClaimRequirement> RequiringClaims { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### Routing State
|
||||
|
||||
```csharp
|
||||
public interface IRoutingStateManager
|
||||
{
|
||||
ValueTask RegisterEndpointsAsync(ConnectionState conn, HelloPayload hello);
|
||||
ValueTask<InstanceSelection?> SelectInstanceAsync(string method, string path);
|
||||
ValueTask UpdateHealthAsync(ConnectionState conn, HeartbeatPayload heartbeat);
|
||||
ValueTask DrainConnectionAsync(string connectionId);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4) REST API
|
||||
|
||||
Gateway exposes minimal management endpoints; all business APIs are routed to microservices.
|
||||
|
||||
### Health Endpoints
|
||||
|
||||
| Endpoint | Auth | Description |
|
||||
|----------|------|-------------|
|
||||
| `GET /health/live` | None | Liveness probe |
|
||||
| `GET /health/ready` | None | Readiness probe |
|
||||
| `GET /health/startup` | None | Startup probe |
|
||||
| `GET /metrics` | None | Prometheus metrics |
|
||||
|
||||
### OpenAPI Endpoints
|
||||
|
||||
| Endpoint | Auth | Description |
|
||||
|----------|------|-------------|
|
||||
| `GET /openapi.json` | None | Aggregated OpenAPI 3.1.0 spec |
|
||||
| `GET /openapi.yaml` | None | YAML format spec |
|
||||
|
||||
---
|
||||
|
||||
## 5) Execution Flow
|
||||
|
||||
### Request Routing
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as Client
|
||||
participant G as Gateway
|
||||
participant A as Authority
|
||||
participant M as Microservice
|
||||
|
||||
C->>G: HTTPS Request + DPoP Token
|
||||
G->>A: Validate Token
|
||||
A-->>G: Claims (sub, tid, scope)
|
||||
G->>G: Select Instance (Method, Path)
|
||||
G->>M: Binary REQUEST Frame
|
||||
M-->>G: Binary RESPONSE Frame
|
||||
G-->>C: HTTPS Response
|
||||
```
|
||||
|
||||
### Microservice Registration
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant M as Microservice
|
||||
participant G as Gateway
|
||||
|
||||
M->>G: TCP/TLS Connect
|
||||
M->>G: HELLO (ServiceName, Version, Endpoints)
|
||||
G->>G: Register Endpoints
|
||||
G-->>M: HELLO ACK
|
||||
|
||||
loop Every 10s
|
||||
G->>M: HEARTBEAT
|
||||
M-->>G: HEARTBEAT (latency, health)
|
||||
G->>G: Update Health State
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) Instance Selection Algorithm
|
||||
|
||||
```csharp
|
||||
public ValueTask<InstanceSelection?> SelectInstanceAsync(string method, string path)
|
||||
{
|
||||
// 1. Find all endpoints matching (method, path)
|
||||
var candidates = _endpoints
|
||||
.Where(e => e.Method == method && MatchPath(e.Path, path))
|
||||
.ToList();
|
||||
|
||||
// 2. Filter by health
|
||||
candidates = candidates
|
||||
.Where(c => c.Health is InstanceHealthStatus.Healthy or InstanceHealthStatus.Degraded)
|
||||
.ToList();
|
||||
|
||||
// 3. Region preference
|
||||
var localRegion = candidates.Where(c => c.Region == _config.Region).ToList();
|
||||
var neighborRegions = candidates.Where(c => _config.NeighborRegions.Contains(c.Region)).ToList();
|
||||
var otherRegions = candidates.Except(localRegion).Except(neighborRegions).ToList();
|
||||
|
||||
var preferred = localRegion.Any() ? localRegion
|
||||
: neighborRegions.Any() ? neighborRegions
|
||||
: otherRegions;
|
||||
|
||||
// 4. Within tier: prefer lower latency, then most recent heartbeat
|
||||
return preferred
|
||||
.OrderBy(c => c.AveragePingMs)
|
||||
.ThenByDescending(c => c.LastHeartbeatUtc)
|
||||
.FirstOrDefault();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Configuration
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
node:
|
||||
region: "eu1"
|
||||
nodeId: "gw-eu1-01"
|
||||
environment: "prod"
|
||||
|
||||
transports:
|
||||
tcp:
|
||||
enabled: true
|
||||
port: 9100
|
||||
maxConnections: 1000
|
||||
receiveBufferSize: 65536
|
||||
sendBufferSize: 65536
|
||||
tls:
|
||||
enabled: true
|
||||
port: 9443
|
||||
certificatePath: "/certs/gateway.pfx"
|
||||
certificatePassword: "${GATEWAY_CERT_PASSWORD}"
|
||||
clientCertificateMode: "RequireCertificate"
|
||||
allowedClientCertificateThumbprints: []
|
||||
|
||||
routing:
|
||||
defaultTimeout: "30s"
|
||||
maxRequestBodySize: "100MB"
|
||||
streamingEnabled: true
|
||||
streamingBufferSize: 16384
|
||||
neighborRegions: ["eu2", "us1"]
|
||||
|
||||
auth:
|
||||
dpopEnabled: true
|
||||
dpopMaxClockSkew: "60s"
|
||||
mtlsEnabled: true
|
||||
rateLimiting:
|
||||
enabled: true
|
||||
requestsPerMinute: 1000
|
||||
burstSize: 100
|
||||
redisConnectionString: "${REDIS_URL}"
|
||||
|
||||
openapi:
|
||||
enabled: true
|
||||
cacheTtlSeconds: 300
|
||||
title: "Stella Ops API"
|
||||
version: "1.0.0"
|
||||
|
||||
health:
|
||||
heartbeatIntervalSeconds: 10
|
||||
heartbeatTimeoutSeconds: 30
|
||||
unhealthyThreshold: 3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8) Scale & Performance
|
||||
|
||||
| Metric | Target | Notes |
|
||||
|--------|--------|-------|
|
||||
| Routing latency (P50) | <2ms | Overhead only; excludes downstream |
|
||||
| Routing latency (P99) | <5ms | Under normal load |
|
||||
| Concurrent connections | 10,000 | Per gateway instance |
|
||||
| Requests/second | 50,000 | Per gateway instance |
|
||||
| Memory footprint | <512MB | Base; scales with connections |
|
||||
|
||||
### Scaling Strategy
|
||||
|
||||
- Horizontal scaling behind load balancer
|
||||
- Sticky sessions NOT required (stateless)
|
||||
- Regional deployment for latency optimization
|
||||
- Rate limiting via distributed Valkey/Redis
|
||||
|
||||
---
|
||||
|
||||
## 9) Security Posture
|
||||
|
||||
### Authentication
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| DPoP | Proof-of-possession tokens from Authority |
|
||||
| mTLS | Certificate-bound tokens for machine clients |
|
||||
|
||||
### Authorization
|
||||
|
||||
- Claims-based authorization per endpoint
|
||||
- Required claims defined in endpoint descriptors
|
||||
- Tenant isolation via `tid` claim
|
||||
|
||||
### Transport Security
|
||||
|
||||
| Component | Encryption |
|
||||
|-----------|------------|
|
||||
| Client → Gateway | TLS 1.3 (HTTPS) |
|
||||
| Gateway → Microservices | TLS (prod), TCP (dev only) |
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
- Per-tenant: Configurable requests/minute
|
||||
- Per-identity: Burst protection
|
||||
- Global: Circuit breaker for overload
|
||||
|
||||
---
|
||||
|
||||
## 10) Observability & Audit
|
||||
|
||||
### Metrics (Prometheus)
|
||||
|
||||
```
|
||||
gateway_requests_total{service,method,path,status}
|
||||
gateway_request_duration_seconds{service,method,path,quantile}
|
||||
gateway_active_connections{service}
|
||||
gateway_transport_frames_total{type}
|
||||
gateway_auth_failures_total{reason}
|
||||
gateway_rate_limit_exceeded_total{tenant}
|
||||
```
|
||||
|
||||
### Traces (OpenTelemetry)
|
||||
|
||||
- Span per request: `gateway.route`
|
||||
- Child span: `gateway.auth.validate`
|
||||
- Child span: `gateway.transport.send`
|
||||
|
||||
### Logs (Structured)
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2025-12-21T10:00:00Z",
|
||||
"level": "info",
|
||||
"message": "Request routed",
|
||||
"correlationId": "abc123",
|
||||
"tenantId": "tenant-1",
|
||||
"method": "GET",
|
||||
"path": "/api/v1/scans/xyz",
|
||||
"service": "scanner",
|
||||
"durationMs": 45,
|
||||
"status": 200
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11) Testing Matrix
|
||||
|
||||
| Test Type | Scope | Coverage Target |
|
||||
|-----------|-------|-----------------|
|
||||
| Unit | Routing algorithm, auth validation | 90% |
|
||||
| Integration | Transport + routing flow | 80% |
|
||||
| E2E | Full request path with mock services | Key flows |
|
||||
| Performance | Latency, throughput, connection limits | SLO targets |
|
||||
| Chaos | Connection failures, microservice crashes | Resilience |
|
||||
|
||||
### Test Fixtures
|
||||
|
||||
- `StellaOps.Router.Transport.InMemory` for transport mocking
|
||||
- Mock Authority for auth testing
|
||||
- `WebApplicationFactory` for integration tests
|
||||
|
||||
---
|
||||
|
||||
## 12) DevOps & Operations
|
||||
|
||||
### Deployment
|
||||
|
||||
```yaml
|
||||
# Kubernetes deployment excerpt
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: gateway
|
||||
spec:
|
||||
replicas: 3
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: gateway
|
||||
image: stellaops/gateway:1.0.0
|
||||
ports:
|
||||
- containerPort: 8080 # HTTPS
|
||||
- containerPort: 9443 # TLS (microservices)
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "250m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "1000m"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health/live
|
||||
port: 8080
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health/ready
|
||||
port: 8080
|
||||
```
|
||||
|
||||
### SLOs
|
||||
|
||||
| SLO | Target | Measurement |
|
||||
|-----|--------|-------------|
|
||||
| Availability | 99.9% | Uptime over 30 days |
|
||||
| Latency P99 | <50ms | Includes downstream |
|
||||
| Error rate | <0.1% | 5xx responses |
|
||||
|
||||
---
|
||||
|
||||
## 13) Roadmap
|
||||
|
||||
| Feature | Sprint | Status |
|
||||
|---------|--------|--------|
|
||||
| Core implementation | 3600.0001.0001 | TODO |
|
||||
| WebSocket support | Future | Planned |
|
||||
| gRPC passthrough | Future | Planned |
|
||||
| GraphQL aggregation | Future | Exploration |
|
||||
|
||||
---
|
||||
|
||||
## 14) References
|
||||
|
||||
- Router Architecture: `docs/modules/router/architecture.md`
|
||||
- OpenAPI Aggregation: `docs/modules/gateway/openapi.md`
|
||||
- Authority Integration: `docs/modules/authority/architecture.md`
|
||||
- Reference Architecture: `docs/product-advisories/archived/2025-12-21-reference-architecture/`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-21 (Sprint 3600)
|
||||
223
docs/modules/platform/reference-architecture-card.md
Normal file
223
docs/modules/platform/reference-architecture-card.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Stella Ops Reference Architecture Card (Dec 2025)
|
||||
|
||||
> **One-Pager** for product managers, architects, and auditors.
|
||||
> Full specification: `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
|
||||
---
|
||||
|
||||
## Topology & Trust Boundaries
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ TRUST BOUNDARY 1 │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ EDGE LAYER │ StellaRouter (Gateway) / UI │
|
||||
│ │ │ OAuth2/OIDC Authentication │
|
||||
│ └────────┬────────┘ │
|
||||
│ │ Signed credentials/attestations required │
|
||||
├───────────┼─────────────────────────────────────────────────────────────────┤
|
||||
│ ▼ TRUST BOUNDARY 2 │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ CONTROL PLANE │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
|
||||
│ │ │Scheduler │ │ Policy │ │Authority │ │ Attestor │ │ │
|
||||
│ │ │ │ │ Engine │ │ │ │ │ │ │
|
||||
│ │ │ Routes │ │ Signed │ │ Keys & │ │ DSSE + │ │ │
|
||||
│ │ │ work │ │ verdicts │ │ identity │ │ Rekor │ │ │
|
||||
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────────────┐ │ │
|
||||
│ │ │ Timeline / Notify │ │ │
|
||||
│ │ │ Immutable audit + notifications │ │ │
|
||||
│ │ └──────────────────────────────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ Only blessed evidence/identities influence decisions │
|
||||
├───────────┼─────────────────────────────────────────────────────────────────┤
|
||||
│ ▼ TRUST BOUNDARY 3 │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ EVIDENCE PLANE │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
|
||||
│ │ │ Sbomer │ │Excititor │ │Concelier │ │Reachabil-│ │ │
|
||||
│ │ │ │ │ │ │ │ │ity/Sigs │ │ │
|
||||
│ │ │CDX 1.7 / │ │ VEX │ │Advisory │ │ Is vuln │ │ │
|
||||
│ │ │SPDX 3.0.1│ │ claims │ │ feeds │ │reachable?│ │ │
|
||||
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ Tamper-evident, separately signed; opinions in Policy only │
|
||||
├───────────┼─────────────────────────────────────────────────────────────────┤
|
||||
│ ▼ TRUST BOUNDARY 4 │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ DATA PLANE │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Workers / Scanners │ │ │
|
||||
│ │ │ Pull tasks → compute → emit artifacts + attestations │ │ │
|
||||
│ │ │ Isolated per tenant; outputs tied to inputs cryptographically│ │ │
|
||||
│ │ └──────────────────────────────────────────────────────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Artifact Association (OCI Referrers)
|
||||
|
||||
```
|
||||
Image Digest (Subject)
|
||||
│
|
||||
├──► SBOM (CycloneDX 1.7 / SPDX 3.0.1)
|
||||
│ └──► DSSE Attestation
|
||||
│ └──► Rekor Log Entry
|
||||
│
|
||||
├──► VEX Claims
|
||||
│ └──► DSSE Attestation
|
||||
│
|
||||
├──► Reachability Subgraph
|
||||
│ └──► DSSE Attestation
|
||||
│
|
||||
└──► Policy Verdict
|
||||
└──► DSSE Attestation
|
||||
└──► Rekor Log Entry
|
||||
```
|
||||
|
||||
- Every artifact is a **subject** in the registry
|
||||
- SBOMs, VEX, verdicts attached as **OCI referrers**
|
||||
- Multiple versioned, signed facts per image without altering the image
|
||||
|
||||
---
|
||||
|
||||
## Data Flows
|
||||
|
||||
### Evidence Flow
|
||||
|
||||
```
|
||||
Workers ──► SBOM (CDX 1.7) ──► DSSE Sign ──► OCI Referrer ──► Registry
|
||||
├─► VEX Claims ──► DSSE Sign ──► OCI Referrer ──►
|
||||
├─► Reachability ──► DSSE Sign ──► OCI Referrer ──►
|
||||
└─► All wrapped as in-toto attestations
|
||||
```
|
||||
|
||||
### Verdict Flow
|
||||
|
||||
```
|
||||
Policy Engine ──► Ingests SBOM/VEX/Reachability/Signals
|
||||
──► Applies rules (deterministic IR)
|
||||
──► Emits signed verdict
|
||||
──► Verdict attached via OCI referrer
|
||||
──► Replayable: same inputs → same output
|
||||
```
|
||||
|
||||
### Audit Flow
|
||||
|
||||
```
|
||||
Timeline ──► Captures all events (immutable)
|
||||
──► Links to attestation digests
|
||||
──► Enables replay and forensics
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tenant Isolation
|
||||
|
||||
| Layer | Mechanism |
|
||||
|-------|-----------|
|
||||
| Database | PostgreSQL RLS (Row-Level Security) |
|
||||
| Application | AsyncLocal tenant context |
|
||||
| Storage | Tenant-scoped paths |
|
||||
| Crypto | Per-tenant keys & trust roots |
|
||||
| Network | Tenant header propagation |
|
||||
|
||||
---
|
||||
|
||||
## Minimal Day-1 Policy
|
||||
|
||||
```yaml
|
||||
rules:
|
||||
# Block reachable HIGH/CRITICAL unless VEX says not_affected
|
||||
- match: { severity: [CRITICAL, HIGH], reachability: reachable }
|
||||
unless: { vexStatus: not_affected }
|
||||
action: block
|
||||
|
||||
# Fail on >5% unknowns
|
||||
- match: { unknownsRatio: { gt: 0.05 } }
|
||||
action: block
|
||||
|
||||
# Require signed SBOM + verdict for production
|
||||
- match: { environment: production }
|
||||
require: { signedSbom: true, signedVerdict: true }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SBOM Format Support
|
||||
|
||||
| Format | Generation | Parsing | Notes |
|
||||
|--------|------------|---------|-------|
|
||||
| CycloneDX 1.7 | Yes | Yes | Primary format |
|
||||
| CycloneDX 1.6 | - | Yes | Backward compat |
|
||||
| SPDX 3.0.1 | Yes | Yes | Alternative format |
|
||||
| SPDX 2.x | - | Yes | Import only |
|
||||
|
||||
---
|
||||
|
||||
## Key Capabilities
|
||||
|
||||
| Capability | Status | Notes |
|
||||
|------------|--------|-------|
|
||||
| Deterministic SBOMs | Complete | Same input → same output |
|
||||
| Signed Verdicts | Complete | DSSE + in-toto |
|
||||
| Replayable Verdicts | Complete | Content-addressed proofs |
|
||||
| OCI Referrers | Complete | Subject digest model |
|
||||
| Rekor Transparency | Complete | v2 tile-backed |
|
||||
| Tenant Isolation | Complete | RLS + crypto separation |
|
||||
| Air-Gap Operation | Complete | Offline bundles |
|
||||
| CycloneDX 1.7 | Planned | Sprint 3600.0002 |
|
||||
| SPDX 3.0.1 Generation | Planned | Sprint 3600.0003 |
|
||||
| Gateway WebService | Planned | Sprint 3600.0001 |
|
||||
| Proof Chain UI | Planned | Sprint 4200.0001 |
|
||||
|
||||
---
|
||||
|
||||
## Quick Glossary
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **SBOM** | Software Bill of Materials (what's inside) |
|
||||
| **VEX** | Vulnerability Exploitability eXchange (is CVE relevant?) |
|
||||
| **Reachability** | Graph proof that vulnerable code is (not) callable |
|
||||
| **DSSE** | Dead Simple Signing Envelope |
|
||||
| **in-toto** | Supply chain attestation framework |
|
||||
| **OCI Referrers** | Registry mechanism to link artifacts to image digest |
|
||||
| **OpTok** | Short-lived operation token from Authority |
|
||||
| **DPoP** | Demonstrating Proof of Possession (RFC 9449) |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Sprints
|
||||
|
||||
| Sprint | Title | Priority |
|
||||
|--------|-------|----------|
|
||||
| 3600.0001.0001 | Gateway WebService | HIGH |
|
||||
| 3600.0002.0001 | CycloneDX 1.7 Upgrade | HIGH |
|
||||
| 3600.0003.0001 | SPDX 3.0.1 Generation | MEDIUM |
|
||||
| 4200.0001.0001 | Proof Chain Verification UI | HIGH |
|
||||
| 5200.0001.0001 | Starter Policy Template | HIGH |
|
||||
|
||||
---
|
||||
|
||||
## Audit Checklist
|
||||
|
||||
- [ ] All SBOMs have DSSE signatures
|
||||
- [ ] All verdicts have DSSE signatures
|
||||
- [ ] Rekor log entries exist for production artifacts
|
||||
- [ ] Tenant isolation verified (RLS + crypto)
|
||||
- [ ] Replay tokens verify (same inputs → same verdict)
|
||||
- [ ] Air-gap bundles include all evidence
|
||||
- [ ] OCI referrers discoverable for all images
|
||||
|
||||
---
|
||||
|
||||
**Source**: Reference Architecture Advisory (Dec 2025)
|
||||
**Last Updated**: 2025-12-21
|
||||
Reference in New Issue
Block a user