tests fixes and sprints work

This commit is contained in:
master
2026-01-22 19:08:46 +02:00
parent c32fff8f86
commit 726d70dc7f
881 changed files with 134434 additions and 6228 deletions

View File

@@ -10,18 +10,68 @@ The SBOM Learning API enables Concelier to learn which advisories are relevant t
Concelier normalizes incoming CycloneDX 1.7 and SPDX 3.0.1 documents into the internal `ParsedSbom` model for matching and downstream analysis.
Current extraction coverage (SPRINT_20260119_015):
- Document metadata: format, specVersion, serialNumber, created, name, namespace when present
- Components: bomRef, type, name, version, purl, cpe, hashes (including SPDX verifiedUsing), license IDs/expressions, license text (base64 decode), external references, properties, scope/modified, supplier/manufacturer, evidence, pedigree, cryptoProperties, modelCard (CycloneDX)
- Dependencies: component dependency edges (CycloneDX dependencies, SPDX relationships)
- Document metadata: format, specVersion, serialNumber, created, name, profiles, sbomType, namespace/imports
- Components: bomRef, type, name, version, purl, cpe, hashes (including SPDX verifiedUsing), license IDs/expressions, license text (base64 decode), external references, properties, scope/modified, supplier/manufacturer, evidence, pedigree, cryptoProperties, modelCard (CycloneDX), swid (CycloneDX), SPDX AI model parameters, SPDX dataset metadata, SPDX file/snippet properties
- Licensing: SPDX Licensing profile elements (listed/custom licenses, license additions, AND/OR/WITH/or-later operators), with OSI/FSF flags and deprecated IDs captured
- Dependencies: component dependency edges (CycloneDX dependencies, SPDX relationships; DependencyOf is inverted to DependsOn)
- Vulnerabilities: CycloneDX embedded vulnerabilities (ratings, affects, VEX analysis), SPDX Security profile vulnerabilities + VEX assessments
- Services: endpoints, authentication, crossesTrustBoundary, data flows, licenses, external references (CycloneDX)
- Formulation: components, workflows, tasks, properties (CycloneDX)
- Declarations/definitions: attestations, affirmations, standards, signatures (CycloneDX)
- Compositions/annotations (CycloneDX)
- Build metadata: buildId, buildType, timestamps, config source, environment, parameters (SPDX)
- Document properties
Notes:
- Full SPDX Licensing profile objects, vulnerabilities, and other SPDX profiles are pending in SPRINT_20260119_015.
- License expressions can be validated against embedded SPDX license/exception lists via `ILicenseExpressionValidator`.
- Matching currently uses PURL and CPE; additional fields are stored for downstream consumers.
## VEX consumption
When SBOM vulnerabilities include embedded VEX analysis, Concelier consumes the statements
to filter or annotate advisory matches. NotAffected statements can be filtered when policy
allows, and trust evaluation checks timestamps, signatures (when provided), and justification
requirements for not-affected claims.
Configuration (YAML or JSON), loaded from `Concelier:VexConsumption:PolicyPath`:
```yaml
vexConsumptionPolicy:
trustEmbeddedVex: true
minimumTrustLevel: Unverified
filterNotAffected: true
signatureRequirements:
requireSignedVex: false
trustedSigners:
- "https://example.com/keys/vex-signer"
timestampRequirements:
maxAgeHours: 720
requireTimestamp: true
conflictResolution:
strategy: mostRecent
logConflicts: true
mergePolicy:
mode: union
externalSources:
- type: repository
url: "https://vex.example.com/api"
justificationRequirements:
requireJustificationForNotAffected: true
acceptedJustifications:
- component_not_present
- vulnerable_code_not_present
- vulnerable_code_not_in_execute_path
- inline_mitigations_already_exist
```
Reports are emitted via `VexConsumptionReporter` in JSON, SARIF, and text formats.
Runtime overrides can be supplied via `Concelier:VexConsumption` (Enabled, IgnoreVex,
PolicyPath, TrustEmbeddedVex, MinimumTrustLevel, FilterNotAffected, ExternalVexSources).
## Flow
```
@@ -339,23 +389,51 @@ var affected = await sbomService.GetAffectedAdvisoriesAsync(
```sql
CREATE TABLE vuln.sbom_registry (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL,
artifact_id TEXT NOT NULL,
sbom_digest TEXT NOT NULL,
sbom_format TEXT NOT NULL,
digest TEXT NOT NULL,
format TEXT NOT NULL CHECK (format IN ('cyclonedx', 'spdx')),
spec_version TEXT NOT NULL,
primary_name TEXT,
primary_version TEXT,
component_count INT NOT NULL DEFAULT 0,
affected_count INT NOT NULL DEFAULT 0,
source TEXT NOT NULL,
tenant_id TEXT,
registered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_matched_at TIMESTAMPTZ,
CONSTRAINT uq_sbom_registry_digest UNIQUE (tenant_id, sbom_digest)
CONSTRAINT uq_sbom_registry_digest UNIQUE (digest)
);
CREATE TABLE vuln.sbom_canonical_match (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
sbom_id UUID NOT NULL REFERENCES vuln.sbom_registry(id),
canonical_id UUID NOT NULL REFERENCES vuln.advisory_canonical(id),
matched_purl TEXT NOT NULL,
purl TEXT NOT NULL,
match_method TEXT NOT NULL,
confidence NUMERIC(3,2) NOT NULL DEFAULT 1.0,
is_reachable BOOLEAN NOT NULL DEFAULT false,
is_deployed BOOLEAN NOT NULL DEFAULT false,
matched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT uq_sbom_canonical_match UNIQUE (sbom_id, canonical_id)
CONSTRAINT uq_sbom_canonical_match UNIQUE (sbom_id, canonical_id, purl)
);
CREATE TABLE concelier.sbom_documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
serial_number TEXT NOT NULL,
artifact_digest TEXT,
format TEXT NOT NULL CHECK (format IN ('cyclonedx', 'spdx')),
spec_version TEXT NOT NULL,
component_count INT NOT NULL DEFAULT 0,
service_count INT NOT NULL DEFAULT 0,
vulnerability_count INT NOT NULL DEFAULT 0,
has_crypto BOOLEAN NOT NULL DEFAULT false,
has_services BOOLEAN NOT NULL DEFAULT false,
has_vulnerabilities BOOLEAN NOT NULL DEFAULT false,
license_ids TEXT[] NOT NULL DEFAULT '{}',
license_expressions TEXT[] NOT NULL DEFAULT '{}',
sbom_json JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT uq_concelier_sbom_serial UNIQUE (serial_number),
CONSTRAINT uq_concelier_sbom_artifact UNIQUE (artifact_digest)
);
```