Files

StellaOps Bot efd6850c38 Add unit tests for VexLens normalizer, CPE parser, product mapper, and PURL parser

- Implemented comprehensive tests for VexLensNormalizer including format detection and normalization scenarios.
- Added tests for CpeParser covering CPE 2.3 and 2.2 formats, invalid inputs, and canonical key generation.
- Created tests for ProductMapper to validate parsing and matching logic across different strictness levels.
- Developed tests for PurlParser to ensure correct parsing of various PURL formats and validation of identifiers.
- Introduced stubs for Monaco editor and worker to facilitate testing in the web application.
- Updated project file for the test project to include necessary dependencies.

2025-12-06 16:28:12 +02:00

11 KiB

Raw Blame History

component_architecture_vexlens.md — Stella Ops VexLens (2025Q4)

Supports deliverables from Epic 30 – VEX Consensus Engine and Epic 31 – Advisory AI Integration.

Scope. Implementation-ready architecture for VexLens: the consensus engine for computing authoritative VEX (Vulnerability Exploitability eXchange) status from multiple overlapping statements. It supports trust-weighted voting, lattice-based conflict resolution, and provides policy integration for vulnerability decisioning.

0) Mission & Boundaries

Mission. Compute deterministic VEX consensus status from multiple sources with full audit trail, enabling automated vulnerability triage based on exploitability data.

Boundaries.

VexLens does not fetch VEX documents — it receives normalized statements from Excititor or direct API input.
VexLens does not store raw VEX documents — it stores computed projections and consensus results.
VexLens does not make policy decisions — it provides VEX status to Policy Engine for final determination.

1) Responsibilities (contract)

Normalize VEX documents from OpenVEX, CSAF VEX, CycloneDX VEX, and SPDX VEX formats.
Map products using PURL and CPE identifiers with configurable matching strictness.
Verify signatures on VEX documents (DSSE, JWS, PGP, PKCS#7).
Compute trust weights based on issuer authority, signature status, freshness, and other factors.
Compute consensus using configurable modes:
- HighestWeight: Single highest-weighted statement wins
- WeightedVote: Weighted voting among all statements
- Lattice: Most conservative status wins (affected > under_investigation > not_affected > fixed)
- AuthoritativeFirst: Authoritative sources override others
- MostRecent: Most recent statement wins
Store projections for historical tracking and audit.
Emit events on consensus computation, status changes, and conflict detection.
Integrate with Policy Engine for vulnerability suppression and severity adjustment.

2) External Dependencies

Excititor: Provides normalized VEX statements from connectors.
Policy Engine: Consumes VEX consensus for vulnerability decisioning.
Vuln Explorer: Enriches vulnerability data with VEX status.
Orchestrator: Schedules consensus compute jobs for batch processing.
Authority: Validates issuer trust and key fingerprints.
Config stores: MongoDB (projections, issuer directory), Redis (caches).

3) API Surface

Base path: /api/v1/vexlens. Full OpenAPI spec at docs/api/vexlens-openapi.yaml.

3.1 Consensus Operations

Endpoint	Method	Description
`/consensus`	POST	Compute consensus for a vulnerability-product pair
`/consensus/batch`	POST	Compute consensus for multiple pairs in batch

3.2 Projection Queries

Endpoint	Method	Description
`/projections`	GET	Query consensus projections with filtering
`/projections/{projectionId}`	GET	Get a projection by ID
`/projections/latest`	GET	Get latest projection for a vuln-product pair
`/projections/history`	GET	Get projection history

3.3 Issuer Directory

Endpoint	Method	Description
`/issuers`	GET	List registered issuers
`/issuers`	POST	Register a new issuer
`/issuers/{issuerId}`	GET	Get issuer details
`/issuers/{issuerId}`	DELETE	Revoke an issuer
`/issuers/{issuerId}/keys`	POST	Add a key to an issuer
`/issuers/{issuerId}/keys/{fingerprint}`	DELETE	Revoke a key

3.4 Statistics

Endpoint	Method	Description
`/statistics`	GET	Get consensus statistics

4) Data Flow

┌─────────────┐     ┌──────────────┐     ┌─────────────────┐
│  Excititor  │────▶│  Normalizer  │────▶│ Trust Weighting │
│ (VEX Docs)  │     │  (OpenVEX,   │     │  (9 factors)    │
└─────────────┘     │ CSAF, CDX)   │     └────────┬────────┘
                    └──────────────┘              │
                                                  ▼
┌─────────────┐     ┌──────────────┐     ┌─────────────────┐
│   Policy    │◀────│ Projection   │◀────│   Consensus     │
│   Engine    │     │   Store      │     │    Engine       │
└─────────────┘     └──────────────┘     └─────────────────┘
                           │
                           ▼
                    ┌──────────────┐
                    │    Events    │
                    │ (Computed,   │
                    │ StatusChange,│
                    │ Conflict)    │
                    └──────────────┘

5) VEX Status Lattice

VexLens uses a status lattice for conservative conflict resolution:

affected (most restrictive)
    │
    ▼
under_investigation
    │
    ▼
not_affected
    │
    ▼
fixed (least restrictive)

In lattice mode, the most restrictive status always wins. This ensures that when sources disagree, the system errs on the side of caution.

6) Trust Weight Factors

Factor	Weight	Description
IssuerBase	25%	Base trust from issuer directory
SignatureStatus	15%	Valid/invalid/unsigned signature
Freshness	15%	Document age with exponential decay
IssuerCategory	10%	Vendor > Distributor > Aggregator
IssuerTier	10%	Authoritative > Trusted > Untrusted
StatusQuality	10%	Has justification, specific status
TransparencyLog	5%	Sigstore Rekor entry
SourceMatch	5%	Source URI pattern match
ProductAuthority	5%	Issuer is authoritative for product

7) Configuration

vexlens:
  consensus:
    defaultMode: WeightedVote  # HighestWeight, WeightedVote, Lattice, AuthoritativeFirst, MostRecent
    minimumConfidence: 0.1
    conflictThreshold: 0.3
    requireJustificationForNotAffected: false
  trust:
    freshnessHalfLifeDays: 90
    minimumFreshness: 0.3
    allowUnsigned: true
    unsignedPenalty: 0.3
    allowUnknownIssuers: true
    unknownIssuerPenalty: 0.5
  storage:
    projectionRetentionDays: 365
    eventRetentionDays: 90
  issuerDirectory:
    source: mongodb  # mongodb, file, api
    refreshIntervalMinutes: 60

8) Storage Schema

8.1 Consensus Projection

{
  "projectionId": "proj-abc123",
  "vulnerabilityId": "CVE-2024-1234",
  "productKey": "pkg:npm/lodash@4.17.21",
  "tenantId": "tenant-001",
  "status": "not_affected",
  "justification": "vulnerable_code_not_present",
  "confidenceScore": 0.95,
  "outcome": "Unanimous",
  "statementCount": 3,
  "conflictCount": 0,
  "rationaleSummary": "Unanimous consensus from 3 authoritative sources",
  "computedAt": "2025-12-06T12:00:00Z",
  "storedAt": "2025-12-06T12:00:01Z",
  "previousProjectionId": null,
  "statusChanged": true
}

8.2 Issuer Record

{
  "issuerId": "npm-security",
  "name": "npm Security Team",
  "category": "Vendor",
  "trustTier": "Authoritative",
  "status": "Active",
  "keyFingerprints": [
    {
      "fingerprint": "ABCD1234EFGH5678",
      "keyType": "Pgp",
      "algorithm": "EdDSA",
      "status": "Active",
      "registeredAt": "2025-01-01T00:00:00Z",
      "expiresAt": null
    }
  ],
  "metadata": {
    "description": "Official npm security advisories",
    "uri": "https://www.npmjs.com/advisories",
    "email": "security@npmjs.com"
  },
  "registeredAt": "2025-01-01T00:00:00Z"
}

9) Events

9.1 ConsensusComputedEvent

Emitted after every consensus computation.

{
  "eventId": "evt-abc123",
  "projectionId": "proj-abc123",
  "vulnerabilityId": "CVE-2024-1234",
  "productKey": "pkg:npm/lodash@4.17.21",
  "status": "not_affected",
  "confidenceScore": 0.95,
  "outcome": "Unanimous",
  "statementCount": 3,
  "computedAt": "2025-12-06T12:00:00Z",
  "emittedAt": "2025-12-06T12:00:01Z"
}

9.2 ConsensusStatusChangedEvent

Emitted when consensus status changes from previous projection.

9.3 ConsensusConflictDetectedEvent

Emitted when conflicts are detected during consensus computation.

10) Observability

10.1 Metrics (OpenTelemetry)

Metric	Type	Description
`vexlens.consensus.computed_total`	Counter	Total consensus computations
`vexlens.consensus.conflicts_total`	Counter	Total conflicts detected
`vexlens.consensus.confidence`	Histogram	Confidence score distribution
`vexlens.consensus.duration_seconds`	Histogram	Computation duration
`vexlens.consensus.status_changes_total`	Counter	Status changes detected
`vexlens.normalization.documents_total`	Counter	Documents normalized
`vexlens.trust.weight_value`	Histogram	Trust weight distribution
`vexlens.issuer.registered_total`	Counter	Issuers registered

10.2 Traces

Activity source: StellaOps.VexLens

Activity	Description
`vexlens.normalize`	VEX document normalization
`vexlens.compute_trust_weight`	Trust weight computation
`vexlens.compute_consensus`	Consensus computation
`vexlens.store_projection`	Projection storage
`vexlens.query_projections`	Projection query

10.3 Logging

Structured logging with event IDs in VexLensLogEvents:

1xxx: Normalization events
2xxx: Product mapping events
3xxx: Signature verification events
4xxx: Trust weight events
5xxx: Consensus events
6xxx: Projection events
7xxx: Issuer directory events

11) Security Considerations

Issuer Trust: All issuers must be registered with verified key fingerprints.
Signature Verification: Documents should be cryptographically signed for production use.
Tenant Isolation: Projections are scoped to tenants; no cross-tenant data access.
Audit Trail: All consensus computations are logged with full rationale.
Determinism: All computations are deterministic for reproducibility.

12) Test Matrix

Test Category	Coverage	Notes
Unit tests	Normalizer, Parser, Trust, Consensus	89+ tests
Determinism harness	Normalization, Trust, Consensus	Verify reproducibility
Integration tests	API service, Storage, Events	End-to-end flows
Property-based tests	Lattice semantics, Weight computation	Invariant verification

11 KiB Raw Blame History Unescape Escape