git.stella-ops.org/docs/ci/scoring-configuration.md

Smart-Diff Scoring Configuration Guide

Sprint: SPRINT_3500_0004_0001
Task: SDIFF-BIN-031 - Documentation for scoring configuration

Overview

Smart-Diff uses configurable scoring weights to prioritize material risk changes. This guide explains how to customize scoring for your organization's risk appetite.

Configuration Location

Smart-Diff scoring can be configured via:

1. PolicyScoringConfig - Integrated with policy engine
2. SmartDiffScoringConfig - Standalone configuration
3. Environment variables - Runtime overrides
4. API - Dynamic configuration

Default Configuration

{
  "name": "default",
  "version": "1.0",
  "reachabilityFlipUpWeight": 1.0,
  "reachabilityFlipDownWeight": 0.8,
  "vexFlipToAffectedWeight": 0.9,
  "vexFlipToNotAffectedWeight": 0.7,
  "vexFlipToFixedWeight": 0.6,
  "vexFlipToUnderInvestigationWeight": 0.3,
  "rangeEntryWeight": 0.8,
  "rangeExitWeight": 0.6,
  "kevAddedWeight": 1.0,
  "epssThreshold": 0.1,
  "epssThresholdCrossWeight": 0.5,
  "hardeningRegressionWeight": 0.7,
  "hardeningImprovementWeight": 0.3,
  "hardeningRegressionThreshold": 0.1
}

Weight Categories

Reachability Weights (R1)

Controls scoring for reachability status changes.

Parameter	Default	Description
reachabilityFlipUpWeight	1.0	Unreachable → Reachable (risk increase)
reachabilityFlipDownWeight	0.8	Reachable → Unreachable (risk decrease)
useLatticeConfidence	true	Factor in reachability confidence

Example scenarios:

• Vulnerability becomes reachable after code refactoring → weight = 1.0
• Dependency removed, vulnerability no longer reachable → weight = 0.8

VEX Status Weights (R2)

Controls scoring for VEX statement changes.

Parameter	Default	Description
vexFlipToAffectedWeight	0.9	Status changed to "affected"
vexFlipToNotAffectedWeight	0.7	Status changed to "not_affected"
vexFlipToFixedWeight	0.6	Status changed to "fixed"
vexFlipToUnderInvestigationWeight	0.3	Status changed to "under_investigation"

Rationale:

• "affected" is highest weight as it confirms exploitability
• "fixed" is lower as it indicates remediation
• "under_investigation" is lowest as status is uncertain

Version Range Weights (R3)

Controls scoring for affected version range changes.

Parameter	Default	Description
rangeEntryWeight	0.8	Version entered affected range
rangeExitWeight	0.6	Version exited affected range

Intelligence Signal Weights (R4)

Controls scoring for external intelligence changes.

Parameter	Default	Description
kevAddedWeight	1.0	Vulnerability added to CISA KEV
epssThreshold	0.1	EPSS score threshold for significance
epssThresholdCrossWeight	0.5	Weight when EPSS crosses threshold

Binary Hardening Weights (R5)

Controls scoring for binary hardening flag changes.

Parameter	Default	Description
hardeningRegressionWeight	0.7	Security flag disabled (e.g., NX removed)
hardeningImprovementWeight	0.3	Security flag enabled (e.g., PIE added)
hardeningRegressionThreshold	0.1	Minimum score drop to flag regression

Presets

Default Preset

Balanced configuration suitable for most organizations.

SmartDiffScoringConfig.Default

Strict Preset

Higher weights for regressions, recommended for security-critical applications.

SmartDiffScoringConfig.Strict

Configuration:

{
  "name": "strict",
  "reachabilityFlipUpWeight": 1.2,
  "vexFlipToAffectedWeight": 1.1,
  "kevAddedWeight": 1.5,
  "hardeningRegressionWeight": 1.0,
  "hardeningRegressionThreshold": 0.05
}

Lenient Preset

Lower weights for alerts, suitable for development/staging environments.

{
  "name": "lenient",
  "reachabilityFlipUpWeight": 0.7,
  "vexFlipToAffectedWeight": 0.6,
  "kevAddedWeight": 0.8,
  "hardeningRegressionWeight": 0.4,
  "epssThreshold": 0.2
}

Policy Integration

Smart-Diff scoring integrates with PolicyScoringConfig:

var config = new PolicyScoringConfig(
    Version: "1.0",
    SeverityWeights: severityWeights,
    QuietPenalty: 0.1,
    WarnPenalty: 0.5,
    IgnorePenalty: 0.0,
    TrustOverrides: trustOverrides,
    ReachabilityBuckets: reachabilityBuckets,
    UnknownConfidence: unknownConfig,
    SmartDiff: new SmartDiffPolicyScoringConfig(
        ReachabilityFlipUpWeight: 1.0,
        VexFlipToAffectedWeight: 0.9,
        KevAddedWeight: 1.2
    )
);

Environment Variable Overrides

# Override reachability weights
export STELLAOPS_SMARTDIFF_REACHABILITY_FLIP_UP_WEIGHT=1.2
export STELLAOPS_SMARTDIFF_REACHABILITY_FLIP_DOWN_WEIGHT=0.7

# Override KEV weight
export STELLAOPS_SMARTDIFF_KEV_ADDED_WEIGHT=1.5

# Override hardening threshold
export STELLAOPS_SMARTDIFF_HARDENING_REGRESSION_THRESHOLD=0.05

API Configuration

Get Current Configuration

GET /api/v1/config/smart-diff/scoring

Response:
{
  "name": "default",
  "version": "1.0",
  "weights": { ... }
}

Update Configuration

PUT /api/v1/config/smart-diff/scoring
Content-Type: application/json

{
  "reachabilityFlipUpWeight": 1.2,
  "kevAddedWeight": 1.5
}

Score Calculation Formula

The final priority score is calculated as:

priority_score = base_severity × Σ(change_weight × rule_match)

Where:

• base_severity is the CVSS/severity normalized to 0-1
• change_weight is the configured weight for the change type
• rule_match is 1 if the rule triggered, 0 otherwise

Example Calculation

Given:

• CVE-2024-1234 with CVSS 7.5 (base_severity = 0.75)
• Became reachable (reachabilityFlipUpWeight = 1.0)
• Added to KEV (kevAddedWeight = 1.0)

priority_score = 0.75 × (1.0 + 1.0) = 1.5 → capped at 1.0

Tuning Recommendations

For CI/CD Pipelines

{
  "kevAddedWeight": 1.5,
  "hardeningRegressionWeight": 1.2,
  "epssThreshold": 0.05
}

Focus on blocking builds for known exploited vulnerabilities and hardening regressions.

For Alert Fatigue Reduction

{
  "reachabilityFlipDownWeight": 0.3,
  "vexFlipToNotAffectedWeight": 0.2,
  "rangeExitWeight": 0.2
}

Lower weights for positive changes to reduce noise.

For Compliance Focus

{
  "kevAddedWeight": 2.0,
  "vexFlipToAffectedWeight": 1.2,
  "hardeningRegressionThreshold": 0.02
}

Higher weights for regulatory-relevant changes.

Monitoring and Metrics

Track scoring effectiveness with:

-- Average priority score by rule type
SELECT 
  change_type,
  AVG(priority_score) as avg_score,
  COUNT(*) as count
FROM smart_diff_changes
WHERE created_at > now() - interval '30 days'
GROUP BY change_type
ORDER BY avg_score DESC;

Related Documentation

• Smart-Diff Detection Rules
• Policy Engine Configuration
• SARIF Integration