Add tests for SBOM generation determinism across multiple formats
- Created `StellaOps.TestKit.Tests` project for unit tests related to determinism. - Implemented `DeterminismManifestTests` to validate deterministic output for canonical bytes and strings, file read/write operations, and error handling for invalid schema versions. - Added `SbomDeterminismTests` to ensure identical inputs produce consistent SBOMs across SPDX 3.0.1 and CycloneDX 1.6/1.7 formats, including parallel execution tests. - Updated project references in `StellaOps.Integration.Determinism` to include the new determinism testing library.
This commit is contained in:
36
docs2/risk/api.md
Normal file
36
docs2/risk/api.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Risk API
|
||||
|
||||
Purpose
|
||||
- Expose risk jobs, profiles, simulations, explainability, and exports.
|
||||
|
||||
Endpoints (v1)
|
||||
- POST /api/v1/risk/jobs: submit scoring job.
|
||||
- GET /api/v1/risk/jobs/{job_id}: job status and results.
|
||||
- GET /api/v1/risk/explain/{job_id}: explainability payload.
|
||||
- GET /api/v1/risk/profiles: list profiles with hashes and versions.
|
||||
- POST /api/v1/risk/profiles: create or update profiles with DSSE metadata.
|
||||
- POST /api/v1/risk/simulations: dry-run scoring with fixtures.
|
||||
- GET /api/v1/risk/export/{job_id}: export bundle for audit.
|
||||
|
||||
Auth and tenancy
|
||||
- Headers: X-Stella-Tenant, Authorization Bearer token.
|
||||
- Optional X-Stella-Scope for imposed rule reminders.
|
||||
|
||||
Error model
|
||||
- Envelope: code, message, correlation_id, severity, remediation.
|
||||
- Rate-limit headers: Retry-After, X-RateLimit-Remaining.
|
||||
- ETag headers for profile and explain responses.
|
||||
|
||||
Feature flags
|
||||
- risk.jobs, risk.explain, risk.simulations, risk.export.
|
||||
|
||||
Determinism and offline
|
||||
- Samples in docs/risk/samples/api/ with SHA256SUMS.
|
||||
- Stable field ordering and UTC timestamps.
|
||||
|
||||
Related references
|
||||
- risk/overview.md
|
||||
- risk/profiles.md
|
||||
- risk/factors.md
|
||||
- risk/formulas.md
|
||||
- risk/explainability.md
|
||||
28
docs2/risk/explainability.md
Normal file
28
docs2/risk/explainability.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# Risk explainability
|
||||
|
||||
Purpose
|
||||
- Provide per-factor contributions with provenance and gating rationale.
|
||||
|
||||
Explainability envelope
|
||||
- job_id, tenant_id, context_id
|
||||
- profile_id, profile_version, profile_hash
|
||||
- finding_id, raw_score, normalized_score, severity
|
||||
- signal_values and signal_contributions
|
||||
- override_applied, override_reason, gates_triggered
|
||||
- scored_at and provenance hashes
|
||||
|
||||
UI and CLI expectations
|
||||
- Deterministic ordering by factor type, source, then timestamp.
|
||||
- Highlight top contributors and gates.
|
||||
- Export Center bundles include explain payload and manifest hashes.
|
||||
|
||||
Determinism and offline
|
||||
- Fixtures under docs/risk/samples/explain/ with SHA256SUMS.
|
||||
- No live calls in examples or captures.
|
||||
|
||||
Related references
|
||||
- risk/overview.md
|
||||
- risk/factors.md
|
||||
- risk/formulas.md
|
||||
- risk/profiles.md
|
||||
- risk/api.md
|
||||
29
docs2/risk/factors.md
Normal file
29
docs2/risk/factors.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# Risk factors
|
||||
|
||||
Purpose
|
||||
- Define factor catalog and normalization rules for risk scoring.
|
||||
|
||||
Factor catalog (examples)
|
||||
- CVSS or exploit likelihood: numeric 0-10 normalized to 0-1.
|
||||
- KEV flag: boolean boost with provenance.
|
||||
- Reachability: numeric with entrypoint and path provenance.
|
||||
- Runtime facts: categorical or numeric with trace references.
|
||||
- Fix availability: vendor status and mitigation context.
|
||||
- Asset criticality: tenant or service criticality signals.
|
||||
- Provenance trust: categorical trust tier with attestation hash.
|
||||
- Custom overrides: scoped, expiring, and auditable.
|
||||
|
||||
Normalization rules
|
||||
- Validate against profile signal types and transforms.
|
||||
- Clamp numeric inputs to 0-1 and record original values in provenance.
|
||||
- Apply TTL or decay deterministically; drop expired signals.
|
||||
- Precedence: signed over unsigned, runtime over static, newer over older.
|
||||
|
||||
Determinism and ordering
|
||||
- Sort factors by factor type, source, then timestamp.
|
||||
- Hash fixtures and record SHA256 in docs/risk/samples/factors/.
|
||||
|
||||
Related references
|
||||
- risk/overview.md
|
||||
- risk/formulas.md
|
||||
- risk/profiles.md
|
||||
28
docs2/risk/formulas.md
Normal file
28
docs2/risk/formulas.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# Risk formulas
|
||||
|
||||
Purpose
|
||||
- Define how normalized factors combine into a risk score and severity.
|
||||
|
||||
Formula building blocks
|
||||
- Weighted sum with per-factor caps and family caps.
|
||||
- Normalize raw score to 0-1 and apply gates.
|
||||
- VEX gate: not_affected can short-circuit to 0.0.
|
||||
- CVSS + KEV boost: clamp01((cvss/10) + kev_bonus).
|
||||
- Trust gates: fail or down-weight low-trust provenance.
|
||||
- Decay: apply time-based decay to stale signals.
|
||||
- Overrides: tenant or asset overrides with expiry and audit.
|
||||
|
||||
Severity mapping
|
||||
- Map normalized_score to critical, high, medium, low, informational.
|
||||
- Store band rationale in explainability output.
|
||||
|
||||
Determinism
|
||||
- Stable factor ordering before aggregation.
|
||||
- Fixed precision (example: 4 decimals) before severity mapping.
|
||||
- Hash fixtures and record SHA256 in docs/risk/samples/formulas/.
|
||||
|
||||
Related references
|
||||
- risk/overview.md
|
||||
- risk/factors.md
|
||||
- risk/profiles.md
|
||||
- risk/explainability.md
|
||||
36
docs2/risk/overview.md
Normal file
36
docs2/risk/overview.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Risk overview
|
||||
|
||||
Purpose
|
||||
- Explain risk scoring concepts, lifecycle, and artifacts.
|
||||
- Preserve deterministic, provenance-backed outputs.
|
||||
|
||||
Core concepts
|
||||
- Signals become evidence after validation and normalization.
|
||||
- Profiles define weights, thresholds, overrides, and severity mapping.
|
||||
- Formulas aggregate normalized factors into a 0-1 score.
|
||||
- Provenance carries source hashes and attestation references.
|
||||
|
||||
Lifecycle
|
||||
1. Submit a risk job with tenant, context, profile, and findings.
|
||||
2. Ingest evidence from scanners, reachability, VEX, runtime signals, and KEV.
|
||||
3. Normalize and dedupe by provenance hash.
|
||||
4. Evaluate profile rules, gates, and overrides.
|
||||
5. Assign severity band and emit explainability output.
|
||||
6. Export bundles with profile hash and evidence references.
|
||||
|
||||
Artifacts
|
||||
- Profile schema: id, version, signals, weights, overrides, metadata, provenance.
|
||||
- Job and result fields: job_id, profile_hash, normalized_score, severity.
|
||||
- Explainability envelope: signal_values, signal_contributions, gates_triggered.
|
||||
|
||||
Determinism and offline posture
|
||||
- Stable ordering for factors and contributions.
|
||||
- Fixed precision math with UTC timestamps only.
|
||||
- Fixtures and hashes live under docs/risk/samples/.
|
||||
|
||||
Related references
|
||||
- risk/factors.md
|
||||
- risk/formulas.md
|
||||
- risk/profiles.md
|
||||
- risk/explainability.md
|
||||
- risk/api.md
|
||||
37
docs2/risk/profiles.md
Normal file
37
docs2/risk/profiles.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# Risk profiles
|
||||
|
||||
Purpose
|
||||
- Define profile schema, lifecycle, and governance for risk scoring.
|
||||
|
||||
Schema essentials
|
||||
- id, version, description, signals[], weights, metadata.
|
||||
- signals[] fields: name, source, type (numeric, boolean, categorical), path, transform, unit.
|
||||
- overrides: severity rules and decision rules.
|
||||
- Optional: extends, rollout flags, valid_from, valid_until.
|
||||
|
||||
Severity levels
|
||||
- critical, high, medium, low, informational.
|
||||
|
||||
Lifecycle
|
||||
1. Author profiles in Policy Studio.
|
||||
2. Simulate against deterministic fixtures.
|
||||
3. Review and approve with DSSE signatures.
|
||||
4. Promote and activate in Policy Engine.
|
||||
5. Roll back by activating a previous version.
|
||||
|
||||
Governance and determinism
|
||||
- Profiles are immutable after promotion.
|
||||
- Each version carries a profile_hash and signed manifest entry.
|
||||
- Simulation and production share the same evaluation codepath.
|
||||
- Offline bundles include profiles and fixtures with hashes.
|
||||
|
||||
Explainability and observability
|
||||
- Emit per-factor contributions with stable ordering.
|
||||
- Track evaluation latency, factor coverage, profile hit rate, and override usage.
|
||||
|
||||
Related references
|
||||
- risk/overview.md
|
||||
- risk/factors.md
|
||||
- risk/formulas.md
|
||||
- risk/explainability.md
|
||||
- risk/api.md
|
||||
Reference in New Issue
Block a user