git.stella-ops.org/docs/testing/schema-validation.md

# SBOM Schema Validation

This document describes the schema validation system for SBOM (Software Bill of Materials) fixtures in StellaOps.

## Overview

StellaOps validates all SBOM fixtures against official JSON schemas to detect schema drift before runtime. This ensures:

- CycloneDX 1.6 fixtures are compliant with the official schema
- SPDX 3.0.1 fixtures meet specification requirements
- OpenVEX fixtures follow the 0.2.0 specification
- Invalid fixtures are detected early in the CI pipeline

## Supported Formats

| Format | Version | Schema Location | Validator |
|--------|---------|-----------------|-----------|
| CycloneDX | 1.6 | `docs/schemas/cyclonedx-bom-1.6.schema.json` | sbom-utility |
| SPDX | 3.0.1 | `docs/schemas/spdx-jsonld-3.0.1.schema.json` | pyspdxtools / check-jsonschema |
| OpenVEX | 0.2.0 | `docs/schemas/openvex-0.2.0.schema.json` | ajv-cli |

## CI Workflows

### Schema Validation Workflow

**File:** `.gitea/workflows/schema-validation.yml`

Runs on:
- Pull requests touching `bench/golden-corpus/**`, `src/Scanner/**`, `docs/schemas/**`, or `scripts/validate-*.sh`
- Push to `main` branch

Jobs:
1. **validate-cyclonedx** - Validates all CycloneDX 1.6 fixtures
2. **validate-spdx** - Validates all SPDX 3.0.1 fixtures
3. **validate-vex** - Validates all OpenVEX 0.2.0 fixtures
4. **validate-negative** - Verifies invalid fixtures are correctly rejected
5. **summary** - Aggregates results

### Determinism Gate Integration

**File:** `.gitea/workflows/determinism-gate.yml`

The determinism gate includes schema validation as a prerequisite step. If schema validation fails, determinism checks are blocked.

To skip schema validation (e.g., during debugging):
```bash
# Via workflow_dispatch
skip_schema_validation: true
```

## Fixture Directories

Validation scans these directories for SBOM fixtures:

| Directory | Purpose |
|-----------|---------|
| `bench/golden-corpus/` | Golden reference fixtures for reproducibility testing |
| `tests/fixtures/` | Test fixtures for unit and integration tests |
| `seed-data/` | Initial seed data for development environments |
| `tests/fixtures/invalid/` | **Excluded** - Contains intentionally invalid fixtures for negative testing |

## Local Validation

### Using the Validation Scripts

```bash
# Validate a single CycloneDX file
./scripts/validate-sbom.sh path/to/sbom.json

# Validate all CycloneDX files in a directory
./scripts/validate-sbom.sh --all path/to/directory

# Validate SPDX file
./scripts/validate-spdx.sh path/to/sbom.spdx.json

# Validate OpenVEX file
./scripts/validate-vex.sh path/to/vex.openvex.json
```

### Using sbom-utility Directly

```bash
# Install sbom-utility
curl -sSfL "https://github.com/CycloneDX/sbom-utility/releases/download/v0.16.0/sbom-utility-v0.16.0-linux-amd64.tar.gz" | tar xz
sudo mv sbom-utility /usr/local/bin/

# Validate
sbom-utility validate --input-file sbom.json --schema docs/schemas/cyclonedx-bom-1.6.schema.json
```

## Troubleshooting

### Common Validation Errors

#### 1. Invalid specVersion

**Error:** `enum: must be equal to one of the allowed values`

**Cause:** The `specVersion` field contains an invalid or unsupported version.

**Solution:**
```json
// Invalid
"specVersion": "2.0"

// Valid
"specVersion": "1.6"
```

#### 2. Missing Required Fields

**Error:** `required: must have required property 'name'`

**Cause:** A component is missing required fields.

**Solution:** Ensure all components have required fields:
```json
{
  "type": "library",
  "name": "example-package",
  "version": "1.0.0"
}
```

#### 3. Invalid Component Type

**Error:** `enum: type must be equal to one of the allowed values`

**Cause:** The component type is not a valid CycloneDX type.

**Solution:** Use valid types: `application`, `framework`, `library`, `container`, `operating-system`, `device`, `firmware`, `file`, `data`

#### 4. Invalid PURL Format

**Error:** `format: must match format "purl"`

**Cause:** The package URL (purl) is malformed.

**Solution:** Use correct purl format:
```json
// Invalid
"purl": "npm:example@1.0.0"

// Valid
"purl": "pkg:npm/example@1.0.0"
```

### CI Failure Recovery

1. **Identify the failing fixture:** Check CI logs for the specific file
2. **Download the fixture:** `cat path/to/failing-fixture.json`
3. **Run local validation:** `./scripts/validate-sbom.sh path/to/failing-fixture.json`
4. **Fix the schema issues:** Use the error messages to guide corrections
5. **Verify the fix:** Re-run local validation
6. **Push and verify CI passes**

### Negative Test Failures

If negative tests fail with "UNEXPECTED PASS":

1. The invalid fixture in `tests/fixtures/invalid/` somehow passed validation
2. Review the fixture to ensure it contains actual schema violations
3. Update the fixture to include more obvious violations
4. Document the expected error in `tests/fixtures/invalid/README.md`

## Adding New Fixtures

### Valid Fixtures

1. Create fixture in appropriate directory (`bench/golden-corpus/`, `tests/fixtures/`)
2. Ensure it contains the format marker:
   - CycloneDX: `"bomFormat": "CycloneDX"`
   - SPDX: `"spdxVersion"` or `"@context"` with SPDX
   - OpenVEX: `"@context"` with openvex
3. Run local validation before committing
4. CI will automatically validate on PR

### Invalid Fixtures (Negative Testing)

1. Create fixture in `tests/fixtures/invalid/`
2. Add `$comment` field explaining the defect
3. Update `tests/fixtures/invalid/README.md` with expected error
4. Ensure the fixture has the correct format marker
5. CI will verify it fails validation

## Schema Updates

When updating schema versions:

1. Download new schema to `docs/schemas/`
2. Update `SBOM_UTILITY_VERSION` in workflows if needed
3. Run full validation to check for new violations
4. Update documentation with new version
5. Update `docs/reproducibility.md` with schema version changes

## References

- [CycloneDX Specification](https://cyclonedx.org/specification/overview/)
- [CycloneDX sbom-utility](https://github.com/CycloneDX/sbom-utility)
- [SPDX Specification](https://spdx.github.io/spdx-spec/v3.0.1/)
- [SPDX Python Tools](https://github.com/spdx/tools-python)
- [OpenVEX Specification](https://github.com/openvex/spec)