audit work, fixed StellaOps.sln warnings/errors, fixed tests, sprints work, new advisories
This commit is contained in:
292
docs/technical/cicd/scoring-configuration.md
Normal file
292
docs/technical/cicd/scoring-configuration.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# Smart-Diff Scoring Configuration Guide
|
||||
|
||||
**Sprint:** SPRINT_3500_0004_0001
|
||||
**Task:** SDIFF-BIN-031 - Documentation for scoring configuration
|
||||
|
||||
## Overview
|
||||
|
||||
Smart-Diff uses configurable scoring weights to prioritize material risk changes. This guide explains how to customize scoring for your organization's risk appetite.
|
||||
|
||||
## Configuration Location
|
||||
|
||||
Smart-Diff scoring can be configured via:
|
||||
1. **PolicyScoringConfig** - Integrated with policy engine
|
||||
2. **SmartDiffScoringConfig** - Standalone configuration
|
||||
3. **Environment variables** - Runtime overrides
|
||||
4. **API** - Dynamic configuration
|
||||
|
||||
## Default Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "default",
|
||||
"version": "1.0",
|
||||
"reachabilityFlipUpWeight": 1.0,
|
||||
"reachabilityFlipDownWeight": 0.8,
|
||||
"vexFlipToAffectedWeight": 0.9,
|
||||
"vexFlipToNotAffectedWeight": 0.7,
|
||||
"vexFlipToFixedWeight": 0.6,
|
||||
"vexFlipToUnderInvestigationWeight": 0.3,
|
||||
"rangeEntryWeight": 0.8,
|
||||
"rangeExitWeight": 0.6,
|
||||
"kevAddedWeight": 1.0,
|
||||
"epssThreshold": 0.1,
|
||||
"epssThresholdCrossWeight": 0.5,
|
||||
"hardeningRegressionWeight": 0.7,
|
||||
"hardeningImprovementWeight": 0.3,
|
||||
"hardeningRegressionThreshold": 0.1
|
||||
}
|
||||
```
|
||||
|
||||
## Weight Categories
|
||||
|
||||
### Reachability Weights (R1)
|
||||
|
||||
Controls scoring for reachability status changes.
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `reachabilityFlipUpWeight` | 1.0 | Unreachable → Reachable (risk increase) |
|
||||
| `reachabilityFlipDownWeight` | 0.8 | Reachable → Unreachable (risk decrease) |
|
||||
| `useLatticeConfidence` | true | Factor in reachability confidence |
|
||||
|
||||
**Example scenarios:**
|
||||
- Vulnerability becomes reachable after code refactoring → weight = 1.0
|
||||
- Dependency removed, vulnerability no longer reachable → weight = 0.8
|
||||
|
||||
### VEX Status Weights (R2)
|
||||
|
||||
Controls scoring for VEX statement changes.
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `vexFlipToAffectedWeight` | 0.9 | Status changed to "affected" |
|
||||
| `vexFlipToNotAffectedWeight` | 0.7 | Status changed to "not_affected" |
|
||||
| `vexFlipToFixedWeight` | 0.6 | Status changed to "fixed" |
|
||||
| `vexFlipToUnderInvestigationWeight` | 0.3 | Status changed to "under_investigation" |
|
||||
|
||||
**Rationale:**
|
||||
- "affected" is highest weight as it confirms exploitability
|
||||
- "fixed" is lower as it indicates remediation
|
||||
- "under_investigation" is lowest as status is uncertain
|
||||
|
||||
### Version Range Weights (R3)
|
||||
|
||||
Controls scoring for affected version range changes.
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `rangeEntryWeight` | 0.8 | Version entered affected range |
|
||||
| `rangeExitWeight` | 0.6 | Version exited affected range |
|
||||
|
||||
### Intelligence Signal Weights (R4)
|
||||
|
||||
Controls scoring for external intelligence changes.
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `kevAddedWeight` | 1.0 | Vulnerability added to CISA KEV |
|
||||
| `epssThreshold` | 0.1 | EPSS score threshold for significance |
|
||||
| `epssThresholdCrossWeight` | 0.5 | Weight when EPSS crosses threshold |
|
||||
|
||||
### Binary Hardening Weights (R5)
|
||||
|
||||
Controls scoring for binary hardening flag changes.
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `hardeningRegressionWeight` | 0.7 | Security flag disabled (e.g., NX removed) |
|
||||
| `hardeningImprovementWeight` | 0.3 | Security flag enabled (e.g., PIE added) |
|
||||
| `hardeningRegressionThreshold` | 0.1 | Minimum score drop to flag regression |
|
||||
|
||||
## Presets
|
||||
|
||||
### Default Preset
|
||||
|
||||
Balanced configuration suitable for most organizations.
|
||||
|
||||
```csharp
|
||||
SmartDiffScoringConfig.Default
|
||||
```
|
||||
|
||||
### Strict Preset
|
||||
|
||||
Higher weights for regressions, recommended for security-critical applications.
|
||||
|
||||
```csharp
|
||||
SmartDiffScoringConfig.Strict
|
||||
```
|
||||
|
||||
Configuration:
|
||||
```json
|
||||
{
|
||||
"name": "strict",
|
||||
"reachabilityFlipUpWeight": 1.2,
|
||||
"vexFlipToAffectedWeight": 1.1,
|
||||
"kevAddedWeight": 1.5,
|
||||
"hardeningRegressionWeight": 1.0,
|
||||
"hardeningRegressionThreshold": 0.05
|
||||
}
|
||||
```
|
||||
|
||||
### Lenient Preset
|
||||
|
||||
Lower weights for alerts, suitable for development/staging environments.
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "lenient",
|
||||
"reachabilityFlipUpWeight": 0.7,
|
||||
"vexFlipToAffectedWeight": 0.6,
|
||||
"kevAddedWeight": 0.8,
|
||||
"hardeningRegressionWeight": 0.4,
|
||||
"epssThreshold": 0.2
|
||||
}
|
||||
```
|
||||
|
||||
## Policy Integration
|
||||
|
||||
Smart-Diff scoring integrates with `PolicyScoringConfig`:
|
||||
|
||||
```csharp
|
||||
var config = new PolicyScoringConfig(
|
||||
Version: "1.0",
|
||||
SeverityWeights: severityWeights,
|
||||
QuietPenalty: 0.1,
|
||||
WarnPenalty: 0.5,
|
||||
IgnorePenalty: 0.0,
|
||||
TrustOverrides: trustOverrides,
|
||||
ReachabilityBuckets: reachabilityBuckets,
|
||||
UnknownConfidence: unknownConfig,
|
||||
SmartDiff: new SmartDiffPolicyScoringConfig(
|
||||
ReachabilityFlipUpWeight: 1.0,
|
||||
VexFlipToAffectedWeight: 0.9,
|
||||
KevAddedWeight: 1.2
|
||||
)
|
||||
);
|
||||
```
|
||||
|
||||
## Environment Variable Overrides
|
||||
|
||||
```bash
|
||||
# Override reachability weights
|
||||
export STELLAOPS_SMARTDIFF_REACHABILITY_FLIP_UP_WEIGHT=1.2
|
||||
export STELLAOPS_SMARTDIFF_REACHABILITY_FLIP_DOWN_WEIGHT=0.7
|
||||
|
||||
# Override KEV weight
|
||||
export STELLAOPS_SMARTDIFF_KEV_ADDED_WEIGHT=1.5
|
||||
|
||||
# Override hardening threshold
|
||||
export STELLAOPS_SMARTDIFF_HARDENING_REGRESSION_THRESHOLD=0.05
|
||||
```
|
||||
|
||||
## API Configuration
|
||||
|
||||
### Get Current Configuration
|
||||
|
||||
```bash
|
||||
GET /api/v1/config/smart-diff/scoring
|
||||
|
||||
Response:
|
||||
{
|
||||
"name": "default",
|
||||
"version": "1.0",
|
||||
"weights": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### Update Configuration
|
||||
|
||||
```bash
|
||||
PUT /api/v1/config/smart-diff/scoring
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"reachabilityFlipUpWeight": 1.2,
|
||||
"kevAddedWeight": 1.5
|
||||
}
|
||||
```
|
||||
|
||||
## Score Calculation Formula
|
||||
|
||||
The final priority score is calculated as:
|
||||
|
||||
```
|
||||
priority_score = base_severity × Σ(change_weight × rule_match)
|
||||
```
|
||||
|
||||
Where:
|
||||
- `base_severity` is the CVSS/severity normalized to 0-1
|
||||
- `change_weight` is the configured weight for the change type
|
||||
- `rule_match` is 1 if the rule triggered, 0 otherwise
|
||||
|
||||
### Example Calculation
|
||||
|
||||
Given:
|
||||
- CVE-2024-1234 with CVSS 7.5 (base_severity = 0.75)
|
||||
- Became reachable (reachabilityFlipUpWeight = 1.0)
|
||||
- Added to KEV (kevAddedWeight = 1.0)
|
||||
|
||||
```
|
||||
priority_score = 0.75 × (1.0 + 1.0) = 1.5 → capped at 1.0
|
||||
```
|
||||
|
||||
## Tuning Recommendations
|
||||
|
||||
### For CI/CD Pipelines
|
||||
|
||||
```json
|
||||
{
|
||||
"kevAddedWeight": 1.5,
|
||||
"hardeningRegressionWeight": 1.2,
|
||||
"epssThreshold": 0.05
|
||||
}
|
||||
```
|
||||
|
||||
Focus on blocking builds for known exploited vulnerabilities and hardening regressions.
|
||||
|
||||
### For Alert Fatigue Reduction
|
||||
|
||||
```json
|
||||
{
|
||||
"reachabilityFlipDownWeight": 0.3,
|
||||
"vexFlipToNotAffectedWeight": 0.2,
|
||||
"rangeExitWeight": 0.2
|
||||
}
|
||||
```
|
||||
|
||||
Lower weights for positive changes to reduce noise.
|
||||
|
||||
### For Compliance Focus
|
||||
|
||||
```json
|
||||
{
|
||||
"kevAddedWeight": 2.0,
|
||||
"vexFlipToAffectedWeight": 1.2,
|
||||
"hardeningRegressionThreshold": 0.02
|
||||
}
|
||||
```
|
||||
|
||||
Higher weights for regulatory-relevant changes.
|
||||
|
||||
## Monitoring and Metrics
|
||||
|
||||
Track scoring effectiveness with:
|
||||
|
||||
```sql
|
||||
-- Average priority score by rule type
|
||||
SELECT
|
||||
change_type,
|
||||
AVG(priority_score) as avg_score,
|
||||
COUNT(*) as count
|
||||
FROM smart_diff_changes
|
||||
WHERE created_at > now() - interval '30 days'
|
||||
GROUP BY change_type
|
||||
ORDER BY avg_score DESC;
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Smart-Diff Detection Rules](../modules/scanner/smart-diff-rules.md)
|
||||
- [Policy Engine Configuration](../modules/policy/architecture.md)
|
||||
- [SARIF Integration](./sarif-integration.md)
|
||||
Reference in New Issue
Block a user