# Determinism and Reproducibility Technical Reference **Source Advisories**: - 07-Dec-2025 - Designing Deterministic Vulnerability Scores - 12-Dec-2025 - Designing a Deterministic Vulnerability Scoring Matrix - 12-Dec-2025 - Replay Fidelity as a Proof Metric - 01-Dec-2025 - Benchmarks for a Testable Security Moat - 02-Dec-2025 - Benchmarking a Testable Security Moat **Last Updated**: 2025-12-14 --- ## 1. SCORE FORMULA (BASIS POINTS) **Total Score:** ``` riskScore = (wB*B + wR*R + wE*E + wP*P) / 10000 ``` **Default Weights (basis points, sum = 10000):** - `wB=1000` (10%) - Base Severity - `wR=4500` (45%) - Reachability - `wE=3000` (30%) - Evidence - `wP=1500` (15%) - Provenance ## 2. SUBSCORE DEFINITIONS (0-100 integers) ### 2.1 BaseSeverity (B) ``` B = round(CVSS * 10) // CVSS 0.0-10.0 → 0-100 ``` ### 2.2 Reachability (R) Hop Buckets: ``` 0-2 hops: 100 3 hops: 85 4 hops: 70 5 hops: 55 6 hops: 45 7 hops: 35 8+ hops: 20 unreachable: 0 ``` Gate Multipliers (in basis points): ``` behind feature flag: ×7000 auth required: ×8000 admin only: ×8500 non-default config: ×7500 ``` Final R: ``` R = bucketScore * gateMultiplier / 10000 ``` ### 2.3 Evidence (E) Points: ``` runtime trace: +60 DAST/integration test: +30 SAST precise sink: +20 SCA presence only: +10 ``` Freshness Multiplier (basis points): ``` ≤ 7 days: ×10000 ≤ 30 days: ×9000 ≤ 90 days: ×7500 ≤ 180 days: ×6000 ≤ 365 days: ×4000 > 365 days: ×2000 ``` Final E: ``` E = min(100, sum(points)) * freshness / 10000 ``` ### 2.4 Provenance (P) ``` unsigned/unknown: 0 signed image: 30 signed + SBOM hash-linked: 60 signed + SBOM + DSSE attestations: 80 above + reproducible build match: 100 ``` ## 3. SCORE POLICY YAML SCHEMA ```yaml policyVersion: score.v1 weightsBps: baseSeverity: 1000 reachability: 4500 evidence: 3000 provenance: 1500 reachability: hopBuckets: - { maxHops: 2, score: 100 } - { maxHops: 3, score: 85 } - { maxHops: 4, score: 70 } - { maxHops: 5, score: 55 } - { maxHops: 6, score: 45 } - { maxHops: 7, score: 35 } - { maxHops: 9999, score: 20 } unreachableScore: 0 gateMultipliersBps: featureFlag: 7000 authRequired: 8000 adminOnly: 8500 nonDefaultConfig: 7500 evidence: points: runtime: 60 dast: 30 sast: 20 sca: 10 freshnessBuckets: - { maxAgeDays: 7, multiplierBps: 10000 } - { maxAgeDays: 30, multiplierBps: 9000 } - { maxAgeDays: 90, multiplierBps: 7500 } - { maxAgeDays: 180, multiplierBps: 6000 } - { maxAgeDays: 365, multiplierBps: 4000 } - { maxAgeDays: 99999, multiplierBps: 2000 } provenance: levels: unsigned: 0 signed: 30 signedWithSbom: 60 signedWithSbomAndAttestations: 80 reproducible: 100 overrides: - name: knownExploitedAndReachable when: flags: knownExploited: true minReachability: 70 setScore: 95 - name: unreachableAndOnlySca when: maxReachability: 0 maxEvidence: 10 clampMaxScore: 25 ``` ## 4. SCORE DATA CONTRACTS ### 4.1 ScoreInput ```json { "asOf": "2025-12-14T10:20:30Z", "policyVersion": "score.v1", "reachabilityDigest": "sha256:...", "evidenceDigest": "sha256:...", "provenanceDigest": "sha256:...", "baseSeverityDigest": "sha256:..." } ``` ### 4.2 ScoreResult ```json { "scoreId": "score_...", "riskScore": 73, "subscores": { "baseSeverity": 75, "reachability": 85, "evidence": 60, "provenance": 60 }, "cvss": { "v": "3.1", "base": 7.5, "environmental": 5.3, "vector": "CVSS:3.1/AV:N/AC:L/..." }, "inputsRef": ["evidence_sha256:...", "env_sha256:..."], "policyVersion": "score.v1", "policyDigest": "sha256:...", "engineVersion": "stella-scorer@1.8.2", "computedAt": "2025-12-09T10:20:30Z", "resultDigest": "sha256:...", "explain": [ {"factor": "reachability", "value": 85, "reason": "3 hops from HTTP endpoint"}, {"factor": "evidence", "value": 60, "reason": "Runtime trace (60pts), 20 days old (×90%)"} ] } ``` ### 4.3 ReachabilityReport ```json { "artifactDigest": "sha256:...", "graphDigest": "sha256:...", "vulnId": "CVE-2024-1234", "vulnerableSymbol": "org.example.VulnClass.vulnMethod", "entrypoints": ["POST /api/upload"], "shortestPath": { "hops": 3, "nodes": [ {"symbol": "UploadController.handleUpload", "file": "Controller.cs", "line": 42}, {"symbol": "ProcessorService.process", "file": "Service.cs", "line": 18}, {"symbol": "org.example.VulnClass.vulnMethod", "file": null, "line": null} ] }, "gates": [ {"type": "authRequired", "detail": "Requires JWT token"}, {"type": "featureFlag", "detail": "FEATURE_UPLOAD_V2=true"} ], "computedAt": "2025-12-14T10:15:30Z", "toolVersion": "reachability-analyzer@2.1.0" } ``` ### 4.4 EvidenceBundle ```json { "evidenceId": "sha256:...", "artifactDigest": "sha256:...", "vulnId": "CVE-2024-1234", "type": "RUNTIME", "tool": "runtime-tracer@1.0.0", "timestamp": "2025-12-10T14:30:00Z", "confidence": 95, "subject": "org.example.VulnClass.vulnMethod", "payloadDigest": "sha256:..." } ``` ### 4.5 ProvenanceReport ```json { "artifactDigest": "sha256:...", "signatureChecks": [ {"signer": "CI-KEY-1", "algorithm": "ECDSA-P256", "result": "VALID"} ], "sbomDigest": "sha256:...", "sbomType": "cyclonedx-1.6", "attestations": ["sha256:...", "sha256:..."], "transparencyLogRefs": ["rekor://..."], "reproducibleMatch": true, "computedAt": "2025-12-14T10:15:30Z", "toolVersion": "provenance-verifier@1.0.0" } ``` ## 5. DETERMINISM CONSTRAINTS ### 5.1 Fixed-Point Math - Use integer basis points (100% = 10,000 bps) - No floating point in scoring math - Round only at final display ### 5.2 Canonical Serialization - RFC-style canonical JSON (JCS) - Sort keys and arrays deterministically - Stable ordering for explanation lists by `(factorId, contributingObjectDigest)` ### 5.3 Time Handling - No implicit time - `asOf` is explicit input - Freshness = `asOf - evidence.timestamp` - Use monotonic time internally ## 6. FIDELITY METRICS ### 6.1 Bitwise Fidelity (BF) ``` BF = identical_outputs / total_replays Target: ≥ 0.98 ``` ### 6.2 Semantic Fidelity (SF) - Normalized object comparison (same packages, versions, CVEs, severities, verdicts) - Allows formatting differences ### 6.3 Policy Fidelity (PF) - Final policy decision (pass/fail + reason codes) matches ## 7. SCAN MANIFEST SCHEMA ```json { "manifest_version": "1.0", "scan_id": "scan_123", "created_at": "2025-12-12T10:15:30Z", "input": { "type": "oci_image", "image_ref": "registry/app@sha256:...", "layers": ["sha256:...", "sha256:..."], "source_provenance": {"repo_sha": "abc123", "build_id": "ci-999"} }, "scanner": { "engine": "stella", "scanner_image_digest": "sha256:...", "scanner_version": "2025.12.0", "config_digest": "sha256:...", "flags": ["--deep", "--vex"] }, "feeds": { "vuln_feed_bundle_digest": "sha256:...", "license_db_digest": "sha256:..." }, "policy": { "policy_bundle_digest": "sha256:...", "policy_set": "prod-default" }, "environment": { "arch": "amd64", "os": "linux", "tz": "UTC", "locale": "C", "network": "disabled", "clock_mode": "frozen", "clock_value": "2025-12-12T10:15:30Z" }, "normalization": { "canonicalizer_version": "1.2.0", "sbom_schema": "cyclonedx-1.6", "vex_schema": "cyclonedx-vex-1.0" } } ``` ## 8. MISMATCH CLASSIFICATION TAXONOMY ``` - Feed drift - Policy drift - Runtime drift - Scanner drift - Nondeterminism (ordering, concurrency, RNG, time-based logic) - External IO ``` ## 9. POSTGRESQL SCHEMA ```sql CREATE TABLE scan_manifest ( manifest_id UUID PRIMARY KEY, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), artifact_digest TEXT NOT NULL, feeds_merkle_root TEXT NOT NULL, engine_build_hash TEXT NOT NULL, policy_lattice_hash TEXT NOT NULL, ruleset_hash TEXT NOT NULL, config_flags JSONB NOT NULL, environment_fingerprint JSONB NOT NULL, raw_manifest JSONB NOT NULL, raw_manifest_sha256 TEXT NOT NULL ); CREATE TABLE scan_execution ( execution_id UUID PRIMARY KEY, manifest_id UUID NOT NULL REFERENCES scan_manifest(manifest_id) ON DELETE CASCADE, started_at TIMESTAMPTZ NOT NULL, finished_at TIMESTAMPTZ NOT NULL, t_ingest_ms INT NOT NULL, t_analyze_ms INT NOT NULL, t_reachability_ms INT NOT NULL, t_vex_ms INT NOT NULL, t_sign_ms INT NOT NULL, t_publish_ms INT NOT NULL, proof_bundle_sha256 TEXT NOT NULL, findings_sha256 TEXT NOT NULL, vex_bundle_sha256 TEXT NOT NULL, replay_mode BOOLEAN NOT NULL DEFAULT FALSE ); CREATE TABLE classification_history ( id BIGSERIAL PRIMARY KEY, artifact_digest TEXT NOT NULL, manifest_id UUID NOT NULL REFERENCES scan_manifest(manifest_id) ON DELETE CASCADE, execution_id UUID NOT NULL REFERENCES scan_execution(execution_id) ON DELETE CASCADE, previous_status TEXT NOT NULL, new_status TEXT NOT NULL, cause TEXT NOT NULL, changed_at TIMESTAMPTZ NOT NULL DEFAULT NOW() ); CREATE VIEW scan_tte AS SELECT execution_id, manifest_id, (finished_at - started_at) AS tte_interval FROM scan_execution; CREATE MATERIALIZED VIEW fn_drift_stats AS SELECT date_trunc('day', changed_at) AS day_bucket, COUNT(*) FILTER (WHERE new_status = 'affected') AS affected_count, COUNT(*) AS total_reclassified, ROUND( (COUNT(*) FILTER (WHERE new_status = 'affected')::numeric / NULLIF(COUNT(*), 0)) * 100, 4 ) AS drift_percent FROM classification_history GROUP BY 1; ``` ## 10. C# CANONICAL DATA STRUCTURES ```csharp public sealed record CanonicalScanManifest { public required string ArtifactDigest { get; init; } public required string FeedsMerkleRoot { get; init; } public required string EngineBuildHash { get; init; } public required string PolicyLatticeHash { get; init; } public required string RulesetHash { get; init; } public required IReadOnlyDictionary ConfigFlags { get; init; } public required EnvironmentFingerprint Environment { get; init; } } public sealed record EnvironmentFingerprint { public required string CpuModel { get; init; } public required string RuntimeVersion { get; init; } public required string Os { get; init; } public required IReadOnlyDictionary Extra { get; init; } } public sealed record ScanExecutionMetrics { public required int IngestMs { get; init; } public required int AnalyzeMs { get; init; } public required int ReachabilityMs { get; init; } public required int VexMs { get; init; } public required int SignMs { get; init; } public required int PublishMs { get; init; } } ``` ## 11. CANONICALIZATION IMPLEMENTATION ```csharp internal static class CanonicalJson { private static readonly JsonSerializerOptions Options = new() { WriteIndented = false, PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; public static string Serialize(object obj) { using var stream = new MemoryStream(); using (var writer = new Utf8JsonWriter(stream, new JsonWriterOptions { Indented = false, SkipValidation = false })) { JsonSerializer.Serialize(writer, obj, obj.GetType(), Options); } var bytes = stream.ToArray(); var canonical = JsonCanonicalizer.Canonicalize(bytes); return canonical; } } ``` ## 12. REPLAY RUNNER ```csharp public static class ReplayRunner { public static ReplayResult Replay(Guid manifestId, IScannerEngine engine) { var manifest = ManifestRepository.Load(manifestId); var canonical = CanonicalJson.Serialize(manifest.RawObject); var canonicalHash = Sha256(canonical); if (canonicalHash != manifest.RawManifestSHA256) throw new InvalidOperationException("Manifest integrity violation."); using var feeds = FeedSnapshotResolver.Open(manifest.FeedsMerkleRoot); var exec = engine.Scan(new ScanRequest { ArtifactDigest = manifest.ArtifactDigest, Feeds = feeds, LatticeHash = manifest.PolicyLatticeHash, EngineBuildHash = manifest.EngineBuildHash, CanonicalManifest = canonical }); return new ReplayResult( exec.FindingsHash == manifest.FindingsSHA256, exec.VexBundleHash == manifest.VexBundleSHA256, exec.ProofBundleHash == manifest.ProofBundleSHA256, exec ); } } ``` ## 13. BENCHMARK METRICS ### 13.1 Time-to-Evidence (TTE) **Definition:** ``` TTE = t(proof_ready) – t(artifact_ingested) ``` **Targets:** - P50 < 2m for typical containers (≤ 500 MB) - P95 < 5m including cold-start/offline-bundle mode **Stage Breakdown:** - t_ingest_ms - t_analyze_ms - t_reachability_ms - t_vex_ms - t_sign_ms - t_publish_ms ### 13.2 False-Negative Drift Rate (FN-DRIFT) **Definition (rolling 30d window):** ``` FN-Drift = (# artifacts re-classified from {unaffected/unknown} → affected) / (total artifacts re-evaluated) ``` **Stratification:** - feed delta - rule delta - lattice/policy delta - reachability delta **Targets:** - Engine-caused FN-Drift ≈ 0 - Feed-caused FN-Drift: faster is better ### 13.3 Deterministic Reproducibility **Proof Object:** ```json { "artifact_digest": "sha256:...", "scan_manifest_hash": "sha256:...", "feeds_merkle_root": "sha256:...", "engine_build_hash": "sha256:...", "policy_lattice_hash": "sha256:...", "findings_sha256": "sha256:...", "vex_bundle_sha256": "sha256:...", "proof_bundle_sha256": "sha256:..." } ``` **Metric:** ``` Repro rate = identical_outputs / total_replays Target: 100% ``` ### 13.4 Detection Metrics ``` true_positive_count (TP) false_positive_count (FP) false_negative_count (FN) precision = TP / (TP + FP) recall = TP / (TP + FN) fp_reduction = (baseline_fp_rate - stella_fp_rate) / baseline_fp_rate ``` ### 13.5 Proof Coverage ``` proof_coverage_all = findings_with_valid_receipts / total_findings proof_coverage_vex = vex_items_with_valid_receipts / total_vex_items proof_coverage_reachable = reachable_findings_with_proofs / total_reachable_findings ``` ## 14. SLO THRESHOLDS **Fidelity:** - BF ≥ 0.98 (general) - BF ≥ 0.95 (regulated projects) - PF ≈ 1.0 (unless policy changed intentionally) **Alerts:** - BF drops ≥2% week-over-week → warn - BF < 0.90 overall → page/block release - Regulated BF < 0.95 → page/block release --- **Document Version**: 1.0 **Target Platform**: .NET 10, PostgreSQL ≥16, Angular v17