stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
2a06f780cfc7d07e265cb856b3f52ffd56c10e58
git.stella-ops.org/docs/testing/schema-validation.md
StellaOps Bot 223843f1d1 docs consolidation
2025-12-25 12:16:13 +02:00

6.1 KiB
Raw Blame History

SBOM Schema Validation

This document describes the schema validation system for SBOM (Software Bill of Materials) fixtures in StellaOps.

Overview

StellaOps validates all SBOM fixtures against official JSON schemas to detect schema drift before runtime. This ensures:

  • CycloneDX 1.6 fixtures are compliant with the official schema
  • SPDX 3.0.1 fixtures meet specification requirements
  • OpenVEX fixtures follow the 0.2.0 specification
  • Invalid fixtures are detected early in the CI pipeline

Supported Formats

Format Version Schema Location Validator
CycloneDX 1.6 docs/schemas/cyclonedx-bom-1.6.schema.json sbom-utility
SPDX 3.0.1 docs/schemas/spdx-jsonld-3.0.1.schema.json pyspdxtools / check-jsonschema
OpenVEX 0.2.0 docs/schemas/openvex-0.2.0.schema.json ajv-cli

CI Workflows

Schema Validation Workflow

File: .gitea/workflows/schema-validation.yml

Runs on:

  • Pull requests touching bench/golden-corpus/**, src/Scanner/**, docs/schemas/**, or scripts/validate-*.sh
  • Push to main branch

Jobs:

  1. validate-cyclonedx - Validates all CycloneDX 1.6 fixtures
  2. validate-spdx - Validates all SPDX 3.0.1 fixtures
  3. validate-vex - Validates all OpenVEX 0.2.0 fixtures
  4. validate-negative - Verifies invalid fixtures are correctly rejected
  5. summary - Aggregates results

Determinism Gate Integration

File: .gitea/workflows/determinism-gate.yml

The determinism gate includes schema validation as a prerequisite step. If schema validation fails, determinism checks are blocked.

To skip schema validation (e.g., during debugging):

# Via workflow_dispatch
skip_schema_validation: true

Fixture Directories

Validation scans these directories for SBOM fixtures:

Directory Purpose
bench/golden-corpus/ Golden reference fixtures for reproducibility testing
tests/fixtures/ Test fixtures for unit and integration tests
seed-data/ Initial seed data for development environments
tests/fixtures/invalid/ Excluded - Contains intentionally invalid fixtures for negative testing

Local Validation

Using the Validation Scripts

# Validate a single CycloneDX file
./scripts/validate-sbom.sh path/to/sbom.json

# Validate all CycloneDX files in a directory
./scripts/validate-sbom.sh --all path/to/directory

# Validate SPDX file
./scripts/validate-spdx.sh path/to/sbom.spdx.json

# Validate OpenVEX file
./scripts/validate-vex.sh path/to/vex.openvex.json

Using sbom-utility Directly

# Install sbom-utility
curl -sSfL "https://github.com/CycloneDX/sbom-utility/releases/download/v0.16.0/sbom-utility-v0.16.0-linux-amd64.tar.gz" | tar xz
sudo mv sbom-utility /usr/local/bin/

# Validate
sbom-utility validate --input-file sbom.json --schema docs/schemas/cyclonedx-bom-1.6.schema.json

Troubleshooting

Common Validation Errors

1. Invalid specVersion

Error: enum: must be equal to one of the allowed values

Cause: The specVersion field contains an invalid or unsupported version.

Solution:

// Invalid
"specVersion": "2.0"

// Valid
"specVersion": "1.6"

2. Missing Required Fields

Error: required: must have required property 'name'

Cause: A component is missing required fields.

Solution: Ensure all components have required fields:

{
  "type": "library",
  "name": "example-package",
  "version": "1.0.0"
}

3. Invalid Component Type

Error: enum: type must be equal to one of the allowed values

Cause: The component type is not a valid CycloneDX type.

Solution: Use valid types: application, framework, library, container, operating-system, device, firmware, file, data

4. Invalid PURL Format

Error: format: must match format "purl"

Cause: The package URL (purl) is malformed.

Solution: Use correct purl format:

// Invalid
"purl": "npm:example@1.0.0"

// Valid
"purl": "pkg:npm/example@1.0.0"

CI Failure Recovery

  1. Identify the failing fixture: Check CI logs for the specific file
  2. Download the fixture: cat path/to/failing-fixture.json
  3. Run local validation: ./scripts/validate-sbom.sh path/to/failing-fixture.json
  4. Fix the schema issues: Use the error messages to guide corrections
  5. Verify the fix: Re-run local validation
  6. Push and verify CI passes

Negative Test Failures

If negative tests fail with "UNEXPECTED PASS":

  1. The invalid fixture in tests/fixtures/invalid/ somehow passed validation
  2. Review the fixture to ensure it contains actual schema violations
  3. Update the fixture to include more obvious violations
  4. Document the expected error in tests/fixtures/invalid/README.md

Adding New Fixtures

Valid Fixtures

  1. Create fixture in appropriate directory (bench/golden-corpus/, tests/fixtures/)
  2. Ensure it contains the format marker:
    • CycloneDX: "bomFormat": "CycloneDX"
    • SPDX: "spdxVersion" or "@context" with SPDX
    • OpenVEX: "@context" with openvex
  3. Run local validation before committing
  4. CI will automatically validate on PR

Invalid Fixtures (Negative Testing)

  1. Create fixture in tests/fixtures/invalid/
  2. Add $comment field explaining the defect
  3. Update tests/fixtures/invalid/README.md with expected error
  4. Ensure the fixture has the correct format marker
  5. CI will verify it fails validation

Schema Updates

When updating schema versions:

  1. Download new schema to docs/schemas/
  2. Update SBOM_UTILITY_VERSION in workflows if needed
  3. Run full validation to check for new violations
  4. Update documentation with new version
  5. Update docs/reproducibility.md with schema version changes

References