Add tests for SBOM generation determinism across multiple formats

- Created `StellaOps.TestKit.Tests` project for unit tests related to determinism.
- Implemented `DeterminismManifestTests` to validate deterministic output for canonical bytes and strings, file read/write operations, and error handling for invalid schema versions.
- Added `SbomDeterminismTests` to ensure identical inputs produce consistent SBOMs across SPDX 3.0.1 and CycloneDX 1.6/1.7 formats, including parallel execution tests.
- Updated project references in `StellaOps.Integration.Determinism` to include the new determinism testing library.
This commit is contained in:
master
2025-12-23 18:56:12 +02:00
parent 7ac70ece71
commit bc4318ef97
88 changed files with 6974 additions and 1230 deletions

36
docs2/risk/api.md Normal file
View File

@@ -0,0 +1,36 @@
# Risk API
Purpose
- Expose risk jobs, profiles, simulations, explainability, and exports.
Endpoints (v1)
- POST /api/v1/risk/jobs: submit scoring job.
- GET /api/v1/risk/jobs/{job_id}: job status and results.
- GET /api/v1/risk/explain/{job_id}: explainability payload.
- GET /api/v1/risk/profiles: list profiles with hashes and versions.
- POST /api/v1/risk/profiles: create or update profiles with DSSE metadata.
- POST /api/v1/risk/simulations: dry-run scoring with fixtures.
- GET /api/v1/risk/export/{job_id}: export bundle for audit.
Auth and tenancy
- Headers: X-Stella-Tenant, Authorization Bearer token.
- Optional X-Stella-Scope for imposed rule reminders.
Error model
- Envelope: code, message, correlation_id, severity, remediation.
- Rate-limit headers: Retry-After, X-RateLimit-Remaining.
- ETag headers for profile and explain responses.
Feature flags
- risk.jobs, risk.explain, risk.simulations, risk.export.
Determinism and offline
- Samples in docs/risk/samples/api/ with SHA256SUMS.
- Stable field ordering and UTC timestamps.
Related references
- risk/overview.md
- risk/profiles.md
- risk/factors.md
- risk/formulas.md
- risk/explainability.md

View File

@@ -0,0 +1,28 @@
# Risk explainability
Purpose
- Provide per-factor contributions with provenance and gating rationale.
Explainability envelope
- job_id, tenant_id, context_id
- profile_id, profile_version, profile_hash
- finding_id, raw_score, normalized_score, severity
- signal_values and signal_contributions
- override_applied, override_reason, gates_triggered
- scored_at and provenance hashes
UI and CLI expectations
- Deterministic ordering by factor type, source, then timestamp.
- Highlight top contributors and gates.
- Export Center bundles include explain payload and manifest hashes.
Determinism and offline
- Fixtures under docs/risk/samples/explain/ with SHA256SUMS.
- No live calls in examples or captures.
Related references
- risk/overview.md
- risk/factors.md
- risk/formulas.md
- risk/profiles.md
- risk/api.md

29
docs2/risk/factors.md Normal file
View File

@@ -0,0 +1,29 @@
# Risk factors
Purpose
- Define factor catalog and normalization rules for risk scoring.
Factor catalog (examples)
- CVSS or exploit likelihood: numeric 0-10 normalized to 0-1.
- KEV flag: boolean boost with provenance.
- Reachability: numeric with entrypoint and path provenance.
- Runtime facts: categorical or numeric with trace references.
- Fix availability: vendor status and mitigation context.
- Asset criticality: tenant or service criticality signals.
- Provenance trust: categorical trust tier with attestation hash.
- Custom overrides: scoped, expiring, and auditable.
Normalization rules
- Validate against profile signal types and transforms.
- Clamp numeric inputs to 0-1 and record original values in provenance.
- Apply TTL or decay deterministically; drop expired signals.
- Precedence: signed over unsigned, runtime over static, newer over older.
Determinism and ordering
- Sort factors by factor type, source, then timestamp.
- Hash fixtures and record SHA256 in docs/risk/samples/factors/.
Related references
- risk/overview.md
- risk/formulas.md
- risk/profiles.md

28
docs2/risk/formulas.md Normal file
View File

@@ -0,0 +1,28 @@
# Risk formulas
Purpose
- Define how normalized factors combine into a risk score and severity.
Formula building blocks
- Weighted sum with per-factor caps and family caps.
- Normalize raw score to 0-1 and apply gates.
- VEX gate: not_affected can short-circuit to 0.0.
- CVSS + KEV boost: clamp01((cvss/10) + kev_bonus).
- Trust gates: fail or down-weight low-trust provenance.
- Decay: apply time-based decay to stale signals.
- Overrides: tenant or asset overrides with expiry and audit.
Severity mapping
- Map normalized_score to critical, high, medium, low, informational.
- Store band rationale in explainability output.
Determinism
- Stable factor ordering before aggregation.
- Fixed precision (example: 4 decimals) before severity mapping.
- Hash fixtures and record SHA256 in docs/risk/samples/formulas/.
Related references
- risk/overview.md
- risk/factors.md
- risk/profiles.md
- risk/explainability.md

36
docs2/risk/overview.md Normal file
View File

@@ -0,0 +1,36 @@
# Risk overview
Purpose
- Explain risk scoring concepts, lifecycle, and artifacts.
- Preserve deterministic, provenance-backed outputs.
Core concepts
- Signals become evidence after validation and normalization.
- Profiles define weights, thresholds, overrides, and severity mapping.
- Formulas aggregate normalized factors into a 0-1 score.
- Provenance carries source hashes and attestation references.
Lifecycle
1. Submit a risk job with tenant, context, profile, and findings.
2. Ingest evidence from scanners, reachability, VEX, runtime signals, and KEV.
3. Normalize and dedupe by provenance hash.
4. Evaluate profile rules, gates, and overrides.
5. Assign severity band and emit explainability output.
6. Export bundles with profile hash and evidence references.
Artifacts
- Profile schema: id, version, signals, weights, overrides, metadata, provenance.
- Job and result fields: job_id, profile_hash, normalized_score, severity.
- Explainability envelope: signal_values, signal_contributions, gates_triggered.
Determinism and offline posture
- Stable ordering for factors and contributions.
- Fixed precision math with UTC timestamps only.
- Fixtures and hashes live under docs/risk/samples/.
Related references
- risk/factors.md
- risk/formulas.md
- risk/profiles.md
- risk/explainability.md
- risk/api.md

37
docs2/risk/profiles.md Normal file
View File

@@ -0,0 +1,37 @@
# Risk profiles
Purpose
- Define profile schema, lifecycle, and governance for risk scoring.
Schema essentials
- id, version, description, signals[], weights, metadata.
- signals[] fields: name, source, type (numeric, boolean, categorical), path, transform, unit.
- overrides: severity rules and decision rules.
- Optional: extends, rollout flags, valid_from, valid_until.
Severity levels
- critical, high, medium, low, informational.
Lifecycle
1. Author profiles in Policy Studio.
2. Simulate against deterministic fixtures.
3. Review and approve with DSSE signatures.
4. Promote and activate in Policy Engine.
5. Roll back by activating a previous version.
Governance and determinism
- Profiles are immutable after promotion.
- Each version carries a profile_hash and signed manifest entry.
- Simulation and production share the same evaluation codepath.
- Offline bundles include profiles and fixtures with hashes.
Explainability and observability
- Emit per-factor contributions with stable ordering.
- Track evaluation latency, factor coverage, profile hit rate, and override usage.
Related references
- risk/overview.md
- risk/factors.md
- risk/formulas.md
- risk/explainability.md
- risk/api.md