Add comprehensive security tests for OWASP A02, A05, A07, and A08 categories
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Findings Ledger CI / build-test (push) Has been cancelled
Findings Ledger CI / migration-validation (push) Has been cancelled
Findings Ledger CI / generate-manifest (push) Has been cancelled
Manifest Integrity / Validate Schema Integrity (push) Has been cancelled
Lighthouse CI / Lighthouse Audit (push) Has been cancelled
Lighthouse CI / Axe Accessibility Audit (push) Has been cancelled
Manifest Integrity / Validate Contract Documents (push) Has been cancelled
Manifest Integrity / Validate Pack Fixtures (push) Has been cancelled
Manifest Integrity / Audit SHA256SUMS Files (push) Has been cancelled
Manifest Integrity / Verify Merkle Roots (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Findings Ledger CI / build-test (push) Has been cancelled
Findings Ledger CI / migration-validation (push) Has been cancelled
Findings Ledger CI / generate-manifest (push) Has been cancelled
Manifest Integrity / Validate Schema Integrity (push) Has been cancelled
Lighthouse CI / Lighthouse Audit (push) Has been cancelled
Lighthouse CI / Axe Accessibility Audit (push) Has been cancelled
Manifest Integrity / Validate Contract Documents (push) Has been cancelled
Manifest Integrity / Validate Pack Fixtures (push) Has been cancelled
Manifest Integrity / Audit SHA256SUMS Files (push) Has been cancelled
Manifest Integrity / Verify Merkle Roots (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
- Implemented tests for Cryptographic Failures (A02) to ensure proper handling of sensitive data, secure algorithms, and key management. - Added tests for Security Misconfiguration (A05) to validate production configurations, security headers, CORS settings, and feature management. - Developed tests for Authentication Failures (A07) to enforce strong password policies, rate limiting, session management, and MFA support. - Created tests for Software and Data Integrity Failures (A08) to verify artifact signatures, SBOM integrity, attestation chains, and feed updates.
This commit is contained in:
66
src/Policy/AGENTS.md
Normal file
66
src/Policy/AGENTS.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# AGENTS · Policy Module
|
||||
|
||||
> Sprint: SPRINT_3500_0002_0001 (Smart-Diff Foundation)
|
||||
|
||||
## Roles
|
||||
- **Backend / Policy Engineer**: .NET 10 (preview) for policy engine, gateways, scoring; keep evaluation deterministic.
|
||||
- **QA Engineer**: Adds policy test fixtures, regression tests under `__Tests`.
|
||||
- **Docs Touches (light)**: Update module docs when contracts change; mirror in sprint notes.
|
||||
|
||||
## Required Reading
|
||||
- `docs/README.md`
|
||||
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
- `docs/modules/policy/architecture.md`
|
||||
- `docs/product-advisories/14-Dec-2025 - Smart-Diff Technical Reference.md` (for suppression contracts)
|
||||
- Current sprint file
|
||||
|
||||
## Working Directory & Boundaries
|
||||
- Primary scope: `src/Policy/**` (Engine, Gateway, Registry, RiskProfile, Scoring, __Libraries, __Tests).
|
||||
- Avoid cross-module edits unless sprint explicitly permits.
|
||||
|
||||
## Suppression Contracts (Sprint 3500)
|
||||
|
||||
The Policy module includes suppression primitives for Smart-Diff:
|
||||
|
||||
### Namespace
|
||||
- `StellaOps.Policy.Suppression` - Pre-filter suppression rules
|
||||
|
||||
### Key Types
|
||||
- `SuppressionRule` - Individual suppression rule definition
|
||||
- `SuppressionRuleEvaluator` - Evaluates rules against findings
|
||||
- `ISuppressionOverrideProvider` - Interface for runtime overrides
|
||||
- `PatchChurnSuppression` - Special handling for patch churn
|
||||
|
||||
### Suppression Rule Types
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `cve_pattern` | Suppress by CVE pattern (regex) |
|
||||
| `purl_pattern` | Suppress by PURL pattern |
|
||||
| `severity_below` | Suppress by severity threshold |
|
||||
| `patch_churn` | Suppress if patch churn detected |
|
||||
| `sink_category` | Suppress by sink category |
|
||||
| `reachability_class` | Suppress by reachability gate class |
|
||||
|
||||
### Integration Points
|
||||
- Scanner SmartDiff calls `SuppressionRuleEvaluator` before emitting findings
|
||||
- Suppressed count tracked in `SmartDiffPredicate.suppressedCount`
|
||||
- Override providers allow runtime/tenant-specific rules
|
||||
|
||||
## Engineering Rules
|
||||
- Target `net10.0`; prefer latest C# preview allowed in repo.
|
||||
- Determinism: stable ordering, UTC timestamps, no `DateTime.Now`/random without seed.
|
||||
- Policy evaluation must be pure (no side effects) and reproducible.
|
||||
- Logging: structured (`ILogger` message templates).
|
||||
- Security: policy files are treated as trusted; validate before loading.
|
||||
|
||||
## Testing & Verification
|
||||
- Default: `dotnet test src/Policy/StellaOps.Policy.sln`.
|
||||
- Add/extend tests in `src/Policy/__Tests/**`.
|
||||
- Golden outputs should be deterministic (sorted keys, stable ordering).
|
||||
- Suppression: Add test cases for each rule type in `SuppressionRuleEvaluatorTests`.
|
||||
|
||||
## Workflow Expectations
|
||||
- Mirror task state in sprint tracker (`TODO → DOING → DONE/BLOCKED`).
|
||||
- Note blockers with the specific decision needed.
|
||||
- When policy contracts change, update both module docs and consumer documentation.
|
||||
@@ -0,0 +1,460 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// AdvancedScoringEngine.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-004
|
||||
// Description: Advanced entropy-based + CVSS hybrid scoring engine
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
using Microsoft.Extensions.Logging;
|
||||
using StellaOps.Policy.Scoring;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Engines;
|
||||
|
||||
/// <summary>
|
||||
/// Advanced entropy-based + CVSS hybrid scoring engine.
|
||||
/// Uses uncertainty tiers, entropy penalties, and CVSS v4.0 receipts.
|
||||
/// This is the default scoring engine.
|
||||
/// </summary>
|
||||
public sealed class AdvancedScoringEngine : IScoringEngine
|
||||
{
|
||||
private readonly EvidenceFreshnessCalculator _freshnessCalculator;
|
||||
private readonly ILogger<AdvancedScoringEngine> _logger;
|
||||
|
||||
public ScoringProfile Profile => ScoringProfile.Advanced;
|
||||
|
||||
public AdvancedScoringEngine(
|
||||
EvidenceFreshnessCalculator freshnessCalculator,
|
||||
ILogger<AdvancedScoringEngine> logger)
|
||||
{
|
||||
_freshnessCalculator = freshnessCalculator ?? throw new ArgumentNullException(nameof(freshnessCalculator));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
public Task<ScoringEngineResult> ScoreAsync(
|
||||
ScoringInput input,
|
||||
ScorePolicy policy,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
ArgumentNullException.ThrowIfNull(policy);
|
||||
|
||||
var explain = new ScoreExplainBuilder();
|
||||
var weights = policy.WeightsBps;
|
||||
|
||||
// 1. Base Severity with CVSS entropy consideration
|
||||
var baseSeverity = CalculateAdvancedBaseSeverity(input, explain);
|
||||
|
||||
// 2. Reachability with semantic analysis
|
||||
var reachability = CalculateAdvancedReachability(input.Reachability, policy, explain);
|
||||
|
||||
// 3. Evidence with uncertainty tiers
|
||||
var evidence = CalculateAdvancedEvidence(input.Evidence, input.AsOf, policy, explain);
|
||||
|
||||
// 4. Provenance with attestation weighting
|
||||
var provenance = CalculateAdvancedProvenance(input.Provenance, policy, explain);
|
||||
|
||||
// Apply KEV boost if applicable
|
||||
var kevBoost = 0;
|
||||
if (input.IsKnownExploited)
|
||||
{
|
||||
kevBoost = 20; // Boost for known exploited vulnerabilities
|
||||
explain.Add("kev_boost", kevBoost, "Known exploited vulnerability (KEV) boost");
|
||||
}
|
||||
|
||||
// Final score calculation with entropy penalty
|
||||
var rawScoreLong =
|
||||
((long)weights.BaseSeverity * baseSeverity) +
|
||||
((long)weights.Reachability * reachability) +
|
||||
((long)weights.Evidence * evidence) +
|
||||
((long)weights.Provenance * provenance);
|
||||
|
||||
var rawScore = (int)(rawScoreLong / 10000) + kevBoost;
|
||||
rawScore = Math.Clamp(rawScore, 0, 100);
|
||||
|
||||
// Apply uncertainty penalty
|
||||
var uncertaintyPenalty = CalculateUncertaintyPenalty(input, explain);
|
||||
var penalizedScore = Math.Max(0, rawScore - uncertaintyPenalty);
|
||||
|
||||
// Apply overrides
|
||||
var (finalScore, appliedOverride) = ApplyOverrides(
|
||||
penalizedScore, reachability, evidence, input.IsKnownExploited, policy);
|
||||
|
||||
var signalValues = new Dictionary<string, int>
|
||||
{
|
||||
["baseSeverity"] = baseSeverity,
|
||||
["reachability"] = reachability,
|
||||
["evidence"] = evidence,
|
||||
["provenance"] = provenance,
|
||||
["kevBoost"] = kevBoost,
|
||||
["uncertaintyPenalty"] = uncertaintyPenalty
|
||||
};
|
||||
|
||||
var signalContributions = new Dictionary<string, double>
|
||||
{
|
||||
["baseSeverity"] = (weights.BaseSeverity * baseSeverity) / 10000.0,
|
||||
["reachability"] = (weights.Reachability * reachability) / 10000.0,
|
||||
["evidence"] = (weights.Evidence * evidence) / 10000.0,
|
||||
["provenance"] = (weights.Provenance * provenance) / 10000.0,
|
||||
["kevBoost"] = kevBoost,
|
||||
["uncertaintyPenalty"] = -uncertaintyPenalty
|
||||
};
|
||||
|
||||
var result = new ScoringEngineResult
|
||||
{
|
||||
FindingId = input.FindingId,
|
||||
ProfileId = input.ProfileId,
|
||||
ProfileVersion = "advanced-v1",
|
||||
RawScore = rawScore,
|
||||
FinalScore = finalScore,
|
||||
Severity = MapToSeverity(finalScore),
|
||||
SignalValues = signalValues,
|
||||
SignalContributions = signalContributions,
|
||||
OverrideApplied = appliedOverride,
|
||||
OverrideReason = appliedOverride is not null ? $"Override applied: {appliedOverride}" : null,
|
||||
ScoringProfile = ScoringProfile.Advanced,
|
||||
ScoredAt = input.AsOf,
|
||||
Explain = explain.Build()
|
||||
};
|
||||
|
||||
_logger.LogDebug(
|
||||
"Advanced score for {FindingId}: B={B}, R={R}, E={E}, P={P}, KEV={KEV}, Penalty={Penalty} -> Raw={RawScore}, Final={FinalScore}",
|
||||
input.FindingId, baseSeverity, reachability, evidence, provenance, kevBoost, uncertaintyPenalty, rawScore, finalScore);
|
||||
|
||||
return Task.FromResult(result);
|
||||
}
|
||||
|
||||
private int CalculateAdvancedBaseSeverity(
|
||||
ScoringInput input,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
// Base severity from CVSS
|
||||
var baseSeverity = (int)Math.Round(input.CvssBase * 10);
|
||||
|
||||
// Apply version-specific adjustments
|
||||
var versionMultiplier = input.CvssVersion switch
|
||||
{
|
||||
"4.0" => 10000, // No adjustment for CVSS 4.0
|
||||
"3.1" => 9500, // Slight reduction for older versions
|
||||
"3.0" => 9000,
|
||||
"2.0" => 8500,
|
||||
_ => 9000 // Default for unknown versions
|
||||
};
|
||||
|
||||
var adjustedSeverity = (baseSeverity * versionMultiplier) / 10000;
|
||||
adjustedSeverity = Math.Clamp(adjustedSeverity, 0, 100);
|
||||
|
||||
var versionInfo = input.CvssVersion ?? "unknown";
|
||||
explain.Add("baseSeverity", adjustedSeverity,
|
||||
$"CVSS {input.CvssBase:F1} (v{versionInfo}) with version adjustment");
|
||||
|
||||
return adjustedSeverity;
|
||||
}
|
||||
|
||||
private int CalculateAdvancedReachability(
|
||||
ReachabilityInput input,
|
||||
ScorePolicy policy,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
// Use advanced score if available
|
||||
if (input.AdvancedScore.HasValue)
|
||||
{
|
||||
var advScore = (int)Math.Round(input.AdvancedScore.Value * 100);
|
||||
advScore = Math.Clamp(advScore, 0, 100);
|
||||
|
||||
var category = input.Category ?? "computed";
|
||||
explain.Add("reachability", advScore, $"Advanced reachability: {category}");
|
||||
return advScore;
|
||||
}
|
||||
|
||||
var config = policy.Reachability ?? ReachabilityPolicyConfig.Default;
|
||||
|
||||
// Fall back to hop-based scoring
|
||||
int bucketScore;
|
||||
if (input.HopCount is null)
|
||||
{
|
||||
bucketScore = config.UnreachableScore;
|
||||
explain.AddReachability(-1, bucketScore, "unreachable");
|
||||
}
|
||||
else
|
||||
{
|
||||
var hops = input.HopCount.Value;
|
||||
|
||||
// Apply semantic category boost/penalty
|
||||
var categoryMultiplier = input.Category?.ToLowerInvariant() switch
|
||||
{
|
||||
"direct_entrypoint" => 12000, // 120% - Direct entry points are high risk
|
||||
"api_endpoint" => 11000, // 110% - API endpoints are high risk
|
||||
"internal_service" => 9000, // 90% - Internal services lower risk
|
||||
"dead_code" => 2000, // 20% - Dead code very low risk
|
||||
_ => 10000 // 100% - Default
|
||||
};
|
||||
|
||||
bucketScore = GetBucketScore(hops, config.HopBuckets);
|
||||
bucketScore = (bucketScore * categoryMultiplier) / 10000;
|
||||
bucketScore = Math.Clamp(bucketScore, 0, 100);
|
||||
|
||||
explain.AddReachability(hops, bucketScore, input.Category ?? "call graph");
|
||||
}
|
||||
|
||||
// Apply gate multiplier if gates present
|
||||
if (input.Gates is { Count: > 0 })
|
||||
{
|
||||
var gateMultiplier = CalculateGateMultiplierBps(input.Gates, config.GateMultipliersBps);
|
||||
bucketScore = (bucketScore * gateMultiplier) / 10000;
|
||||
|
||||
var primaryGate = input.Gates.OrderByDescending(g => g.Confidence).First();
|
||||
explain.Add("gate", gateMultiplier / 100,
|
||||
$"Gate: {primaryGate.Type}" + (primaryGate.Detail is not null ? $" ({primaryGate.Detail})" : ""));
|
||||
}
|
||||
|
||||
return bucketScore;
|
||||
}
|
||||
|
||||
private int CalculateAdvancedEvidence(
|
||||
EvidenceInput input,
|
||||
DateTimeOffset asOf,
|
||||
ScorePolicy policy,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
// Use advanced score if available
|
||||
if (input.AdvancedScore.HasValue)
|
||||
{
|
||||
var advScore = (int)Math.Round(input.AdvancedScore.Value * 100);
|
||||
advScore = Math.Clamp(advScore, 0, 100);
|
||||
explain.Add("evidence", advScore, "Advanced evidence score");
|
||||
return advScore;
|
||||
}
|
||||
|
||||
var config = policy.Evidence ?? EvidencePolicyConfig.Default;
|
||||
var points = config.Points ?? EvidencePoints.Default;
|
||||
|
||||
// Sum evidence points with overlap bonus
|
||||
var totalPoints = 0;
|
||||
var typeCount = 0;
|
||||
|
||||
foreach (var type in input.Types)
|
||||
{
|
||||
totalPoints += type switch
|
||||
{
|
||||
EvidenceType.Runtime => points.Runtime,
|
||||
EvidenceType.Dast => points.Dast,
|
||||
EvidenceType.Sast => points.Sast,
|
||||
EvidenceType.Sca => points.Sca,
|
||||
_ => 0
|
||||
};
|
||||
typeCount++;
|
||||
}
|
||||
|
||||
// Multi-evidence overlap bonus (10% per additional type beyond first)
|
||||
if (typeCount > 1)
|
||||
{
|
||||
var overlapBonus = (totalPoints * (typeCount - 1) * 1000) / 10000;
|
||||
totalPoints += overlapBonus;
|
||||
}
|
||||
|
||||
totalPoints = Math.Min(100, totalPoints);
|
||||
|
||||
// Apply freshness multiplier
|
||||
var freshnessMultiplier = 10000;
|
||||
var ageDays = 0;
|
||||
if (input.NewestEvidenceAt.HasValue)
|
||||
{
|
||||
ageDays = Math.Max(0, (int)(asOf - input.NewestEvidenceAt.Value).TotalDays);
|
||||
freshnessMultiplier = _freshnessCalculator.CalculateMultiplierBps(
|
||||
input.NewestEvidenceAt.Value, asOf);
|
||||
}
|
||||
|
||||
var finalEvidence = (totalPoints * freshnessMultiplier) / 10000;
|
||||
explain.AddEvidence(totalPoints, freshnessMultiplier, ageDays);
|
||||
|
||||
return finalEvidence;
|
||||
}
|
||||
|
||||
private int CalculateAdvancedProvenance(
|
||||
ProvenanceInput input,
|
||||
ScorePolicy policy,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
// Use advanced score if available
|
||||
if (input.AdvancedScore.HasValue)
|
||||
{
|
||||
var advScore = (int)Math.Round(input.AdvancedScore.Value * 100);
|
||||
advScore = Math.Clamp(advScore, 0, 100);
|
||||
explain.Add("provenance", advScore, "Advanced provenance score");
|
||||
return advScore;
|
||||
}
|
||||
|
||||
var config = policy.Provenance ?? ProvenancePolicyConfig.Default;
|
||||
var levels = config.Levels ?? ProvenanceLevels.Default;
|
||||
|
||||
var score = input.Level switch
|
||||
{
|
||||
ProvenanceLevel.Unsigned => levels.Unsigned,
|
||||
ProvenanceLevel.Signed => levels.Signed,
|
||||
ProvenanceLevel.SignedWithSbom => levels.SignedWithSbom,
|
||||
ProvenanceLevel.SignedWithSbomAndAttestations => levels.SignedWithSbomAndAttestations,
|
||||
ProvenanceLevel.Reproducible => levels.Reproducible,
|
||||
_ => levels.Unsigned
|
||||
};
|
||||
|
||||
explain.AddProvenance(input.Level.ToString(), score);
|
||||
return score;
|
||||
}
|
||||
|
||||
private int CalculateUncertaintyPenalty(
|
||||
ScoringInput input,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
var penalty = 0;
|
||||
|
||||
// Penalty for missing reachability data
|
||||
if (input.Reachability.HopCount is null &&
|
||||
input.Reachability.AdvancedScore is null)
|
||||
{
|
||||
penalty += 5;
|
||||
}
|
||||
|
||||
// Penalty for no evidence
|
||||
if (input.Evidence.Types.Count == 0 &&
|
||||
input.Evidence.AdvancedScore is null)
|
||||
{
|
||||
penalty += 10;
|
||||
}
|
||||
|
||||
// Penalty for unsigned provenance
|
||||
if (input.Provenance.Level == ProvenanceLevel.Unsigned &&
|
||||
input.Provenance.AdvancedScore is null)
|
||||
{
|
||||
penalty += 5;
|
||||
}
|
||||
|
||||
// Penalty for missing CVSS version
|
||||
if (string.IsNullOrEmpty(input.CvssVersion))
|
||||
{
|
||||
penalty += 3;
|
||||
}
|
||||
|
||||
if (penalty > 0)
|
||||
{
|
||||
explain.Add("uncertainty_penalty", -penalty, $"Uncertainty penalty for missing data");
|
||||
}
|
||||
|
||||
return penalty;
|
||||
}
|
||||
|
||||
private static int GetBucketScore(int hops, IReadOnlyList<HopBucket>? buckets)
|
||||
{
|
||||
if (buckets is null or { Count: 0 })
|
||||
{
|
||||
return hops switch
|
||||
{
|
||||
0 => 100,
|
||||
1 => 90,
|
||||
<= 3 => 70,
|
||||
<= 5 => 50,
|
||||
<= 10 => 30,
|
||||
_ => 10
|
||||
};
|
||||
}
|
||||
|
||||
foreach (var bucket in buckets)
|
||||
{
|
||||
if (hops <= bucket.MaxHops)
|
||||
{
|
||||
return bucket.Score;
|
||||
}
|
||||
}
|
||||
|
||||
return buckets[^1].Score;
|
||||
}
|
||||
|
||||
private static int CalculateGateMultiplierBps(
|
||||
IReadOnlyList<DetectedGate> gates,
|
||||
GateMultipliersBps? config)
|
||||
{
|
||||
config ??= GateMultipliersBps.Default;
|
||||
|
||||
var lowestMultiplier = 10000;
|
||||
|
||||
foreach (var gate in gates)
|
||||
{
|
||||
var multiplier = gate.Type.ToLowerInvariant() switch
|
||||
{
|
||||
"feature_flag" or "featureflag" => config.FeatureFlag,
|
||||
"auth_required" or "authrequired" => config.AuthRequired,
|
||||
"admin_only" or "adminonly" => config.AdminOnly,
|
||||
"non_default_config" or "nondefaultconfig" => config.NonDefaultConfig,
|
||||
_ => 10000
|
||||
};
|
||||
|
||||
var weightedMultiplier = (int)(multiplier + ((10000 - multiplier) * (1.0 - gate.Confidence)));
|
||||
lowestMultiplier = Math.Min(lowestMultiplier, weightedMultiplier);
|
||||
}
|
||||
|
||||
return lowestMultiplier;
|
||||
}
|
||||
|
||||
private static (int Score, string? Override) ApplyOverrides(
|
||||
int score,
|
||||
int reachability,
|
||||
int evidence,
|
||||
bool isKnownExploited,
|
||||
ScorePolicy policy)
|
||||
{
|
||||
if (policy.Overrides is null or { Count: 0 })
|
||||
return (score, null);
|
||||
|
||||
foreach (var rule in policy.Overrides)
|
||||
{
|
||||
if (!MatchesCondition(rule.When, reachability, evidence, isKnownExploited))
|
||||
continue;
|
||||
|
||||
if (rule.SetScore.HasValue)
|
||||
return (rule.SetScore.Value, rule.Name);
|
||||
|
||||
if (rule.ClampMaxScore.HasValue && score > rule.ClampMaxScore.Value)
|
||||
return (rule.ClampMaxScore.Value, $"{rule.Name} (clamped)");
|
||||
|
||||
if (rule.ClampMinScore.HasValue && score < rule.ClampMinScore.Value)
|
||||
return (rule.ClampMinScore.Value, $"{rule.Name} (clamped)");
|
||||
}
|
||||
|
||||
return (score, null);
|
||||
}
|
||||
|
||||
private static bool MatchesCondition(
|
||||
ScoreOverrideCondition condition,
|
||||
int reachability,
|
||||
int evidence,
|
||||
bool isKnownExploited)
|
||||
{
|
||||
if (condition.Flags?.TryGetValue("knownExploited", out var kevRequired) == true)
|
||||
{
|
||||
if (kevRequired != isKnownExploited)
|
||||
return false;
|
||||
}
|
||||
|
||||
if (condition.MinReachability.HasValue && reachability < condition.MinReachability.Value)
|
||||
return false;
|
||||
|
||||
if (condition.MaxReachability.HasValue && reachability > condition.MaxReachability.Value)
|
||||
return false;
|
||||
|
||||
if (condition.MinEvidence.HasValue && evidence < condition.MinEvidence.Value)
|
||||
return false;
|
||||
|
||||
if (condition.MaxEvidence.HasValue && evidence > condition.MaxEvidence.Value)
|
||||
return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private static string MapToSeverity(int score) => score switch
|
||||
{
|
||||
>= 90 => "critical",
|
||||
>= 70 => "high",
|
||||
>= 40 => "medium",
|
||||
>= 20 => "low",
|
||||
_ => "info"
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,326 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// SimpleScoringEngine.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-003
|
||||
// Description: Simple 4-factor basis-points scoring engine
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
using Microsoft.Extensions.Logging;
|
||||
using StellaOps.Policy.Scoring;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Engines;
|
||||
|
||||
/// <summary>
|
||||
/// Simple 4-factor basis-points scoring engine.
|
||||
/// Formula: riskScore = (wB*B + wR*R + wE*E + wP*P) / 10000
|
||||
/// </summary>
|
||||
public sealed class SimpleScoringEngine : IScoringEngine
|
||||
{
|
||||
private readonly EvidenceFreshnessCalculator _freshnessCalculator;
|
||||
private readonly ILogger<SimpleScoringEngine> _logger;
|
||||
|
||||
public ScoringProfile Profile => ScoringProfile.Simple;
|
||||
|
||||
public SimpleScoringEngine(
|
||||
EvidenceFreshnessCalculator freshnessCalculator,
|
||||
ILogger<SimpleScoringEngine> logger)
|
||||
{
|
||||
_freshnessCalculator = freshnessCalculator ?? throw new ArgumentNullException(nameof(freshnessCalculator));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
public Task<ScoringEngineResult> ScoreAsync(
|
||||
ScoringInput input,
|
||||
ScorePolicy policy,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
ArgumentNullException.ThrowIfNull(policy);
|
||||
|
||||
var explain = new ScoreExplainBuilder();
|
||||
var weights = policy.WeightsBps;
|
||||
|
||||
// 1. Base Severity: B = round(CVSS * 10)
|
||||
var baseSeverity = (int)Math.Round(input.CvssBase * 10);
|
||||
baseSeverity = Math.Clamp(baseSeverity, 0, 100);
|
||||
explain.AddBaseSeverity(input.CvssBase, baseSeverity);
|
||||
|
||||
// 2. Reachability: R = bucketScore * gateMultiplier / 10000
|
||||
var reachability = CalculateReachability(input.Reachability, policy, explain);
|
||||
|
||||
// 3. Evidence: E = min(100, sum(points)) * freshness / 10000
|
||||
var evidence = CalculateEvidence(input.Evidence, input.AsOf, policy, explain);
|
||||
|
||||
// 4. Provenance: P = level score
|
||||
var provenance = CalculateProvenance(input.Provenance, policy, explain);
|
||||
|
||||
// Final score: (wB*B + wR*R + wE*E + wP*P) / 10000
|
||||
var rawScoreLong =
|
||||
((long)weights.BaseSeverity * baseSeverity) +
|
||||
((long)weights.Reachability * reachability) +
|
||||
((long)weights.Evidence * evidence) +
|
||||
((long)weights.Provenance * provenance);
|
||||
|
||||
var rawScore = (int)(rawScoreLong / 10000);
|
||||
rawScore = Math.Clamp(rawScore, 0, 100);
|
||||
|
||||
// Apply overrides
|
||||
var (finalScore, appliedOverride) = ApplyOverrides(
|
||||
rawScore, reachability, evidence, input.IsKnownExploited, policy);
|
||||
|
||||
var signalValues = new Dictionary<string, int>
|
||||
{
|
||||
["baseSeverity"] = baseSeverity,
|
||||
["reachability"] = reachability,
|
||||
["evidence"] = evidence,
|
||||
["provenance"] = provenance
|
||||
};
|
||||
|
||||
var signalContributions = new Dictionary<string, double>
|
||||
{
|
||||
["baseSeverity"] = (weights.BaseSeverity * baseSeverity) / 10000.0,
|
||||
["reachability"] = (weights.Reachability * reachability) / 10000.0,
|
||||
["evidence"] = (weights.Evidence * evidence) / 10000.0,
|
||||
["provenance"] = (weights.Provenance * provenance) / 10000.0
|
||||
};
|
||||
|
||||
var result = new ScoringEngineResult
|
||||
{
|
||||
FindingId = input.FindingId,
|
||||
ProfileId = input.ProfileId,
|
||||
ProfileVersion = "simple-v1",
|
||||
RawScore = rawScore,
|
||||
FinalScore = finalScore,
|
||||
Severity = MapToSeverity(finalScore),
|
||||
SignalValues = signalValues,
|
||||
SignalContributions = signalContributions,
|
||||
OverrideApplied = appliedOverride,
|
||||
OverrideReason = appliedOverride is not null ? $"Override applied: {appliedOverride}" : null,
|
||||
ScoringProfile = ScoringProfile.Simple,
|
||||
ScoredAt = input.AsOf,
|
||||
Explain = explain.Build()
|
||||
};
|
||||
|
||||
_logger.LogDebug(
|
||||
"Simple score for {FindingId}: B={B}, R={R}, E={E}, P={P} -> Raw={RawScore}, Final={FinalScore} (override: {Override})",
|
||||
input.FindingId, baseSeverity, reachability, evidence, provenance, rawScore, finalScore, appliedOverride);
|
||||
|
||||
return Task.FromResult(result);
|
||||
}
|
||||
|
||||
private int CalculateReachability(
|
||||
ReachabilityInput input,
|
||||
ScorePolicy policy,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
var config = policy.Reachability ?? ReachabilityPolicyConfig.Default;
|
||||
|
||||
// Get bucket score
|
||||
int bucketScore;
|
||||
if (input.HopCount is null)
|
||||
{
|
||||
bucketScore = config.UnreachableScore;
|
||||
explain.AddReachability(-1, bucketScore, "unreachable");
|
||||
}
|
||||
else
|
||||
{
|
||||
var hops = input.HopCount.Value;
|
||||
bucketScore = GetBucketScore(hops, config.HopBuckets);
|
||||
explain.AddReachability(hops, bucketScore, hops == 0 ? "direct call" : "call graph");
|
||||
}
|
||||
|
||||
// Apply gate multiplier if gates are present
|
||||
if (input.Gates is { Count: > 0 })
|
||||
{
|
||||
var gateMultiplier = CalculateGateMultiplierBps(input.Gates, config.GateMultipliersBps);
|
||||
bucketScore = (bucketScore * gateMultiplier) / 10000;
|
||||
|
||||
var primaryGate = input.Gates.OrderByDescending(g => g.Confidence).First();
|
||||
explain.Add("gate", gateMultiplier / 100,
|
||||
$"Gate: {primaryGate.Type}" + (primaryGate.Detail is not null ? $" ({primaryGate.Detail})" : ""));
|
||||
}
|
||||
|
||||
return bucketScore;
|
||||
}
|
||||
|
||||
private static int GetBucketScore(int hops, IReadOnlyList<HopBucket>? buckets)
|
||||
{
|
||||
if (buckets is null or { Count: 0 })
|
||||
{
|
||||
// Default buckets
|
||||
return hops switch
|
||||
{
|
||||
0 => 100,
|
||||
1 => 90,
|
||||
<= 3 => 70,
|
||||
<= 5 => 50,
|
||||
<= 10 => 30,
|
||||
_ => 10
|
||||
};
|
||||
}
|
||||
|
||||
foreach (var bucket in buckets)
|
||||
{
|
||||
if (hops <= bucket.MaxHops)
|
||||
{
|
||||
return bucket.Score;
|
||||
}
|
||||
}
|
||||
|
||||
return buckets[^1].Score;
|
||||
}
|
||||
|
||||
private static int CalculateGateMultiplierBps(
|
||||
IReadOnlyList<DetectedGate> gates,
|
||||
GateMultipliersBps? config)
|
||||
{
|
||||
config ??= GateMultipliersBps.Default;
|
||||
|
||||
// Find the most restrictive gate (lowest multiplier = highest mitigation)
|
||||
var lowestMultiplier = 10000; // 100% = no mitigation
|
||||
|
||||
foreach (var gate in gates)
|
||||
{
|
||||
var multiplier = gate.Type.ToLowerInvariant() switch
|
||||
{
|
||||
"feature_flag" or "featureflag" => config.FeatureFlag,
|
||||
"auth_required" or "authrequired" => config.AuthRequired,
|
||||
"admin_only" or "adminonly" => config.AdminOnly,
|
||||
"non_default_config" or "nondefaultconfig" => config.NonDefaultConfig,
|
||||
_ => 10000
|
||||
};
|
||||
|
||||
// Weight by confidence
|
||||
var weightedMultiplier = (int)(multiplier + ((10000 - multiplier) * (1.0 - gate.Confidence)));
|
||||
lowestMultiplier = Math.Min(lowestMultiplier, weightedMultiplier);
|
||||
}
|
||||
|
||||
return lowestMultiplier;
|
||||
}
|
||||
|
||||
private int CalculateEvidence(
|
||||
EvidenceInput input,
|
||||
DateTimeOffset asOf,
|
||||
ScorePolicy policy,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
var config = policy.Evidence ?? EvidencePolicyConfig.Default;
|
||||
var points = config.Points ?? EvidencePoints.Default;
|
||||
|
||||
// Sum evidence points
|
||||
var totalPoints = 0;
|
||||
foreach (var type in input.Types)
|
||||
{
|
||||
totalPoints += type switch
|
||||
{
|
||||
EvidenceType.Runtime => points.Runtime,
|
||||
EvidenceType.Dast => points.Dast,
|
||||
EvidenceType.Sast => points.Sast,
|
||||
EvidenceType.Sca => points.Sca,
|
||||
_ => 0
|
||||
};
|
||||
}
|
||||
totalPoints = Math.Min(100, totalPoints);
|
||||
|
||||
// Apply freshness multiplier
|
||||
var freshnessMultiplier = 10000;
|
||||
var ageDays = 0;
|
||||
if (input.NewestEvidenceAt.HasValue)
|
||||
{
|
||||
ageDays = Math.Max(0, (int)(asOf - input.NewestEvidenceAt.Value).TotalDays);
|
||||
freshnessMultiplier = _freshnessCalculator.CalculateMultiplierBps(
|
||||
input.NewestEvidenceAt.Value, asOf);
|
||||
}
|
||||
|
||||
var finalEvidence = (totalPoints * freshnessMultiplier) / 10000;
|
||||
explain.AddEvidence(totalPoints, freshnessMultiplier, ageDays);
|
||||
|
||||
return finalEvidence;
|
||||
}
|
||||
|
||||
private static int CalculateProvenance(
|
||||
ProvenanceInput input,
|
||||
ScorePolicy policy,
|
||||
ScoreExplainBuilder explain)
|
||||
{
|
||||
var config = policy.Provenance ?? ProvenancePolicyConfig.Default;
|
||||
var levels = config.Levels ?? ProvenanceLevels.Default;
|
||||
|
||||
var score = input.Level switch
|
||||
{
|
||||
ProvenanceLevel.Unsigned => levels.Unsigned,
|
||||
ProvenanceLevel.Signed => levels.Signed,
|
||||
ProvenanceLevel.SignedWithSbom => levels.SignedWithSbom,
|
||||
ProvenanceLevel.SignedWithSbomAndAttestations => levels.SignedWithSbomAndAttestations,
|
||||
ProvenanceLevel.Reproducible => levels.Reproducible,
|
||||
_ => levels.Unsigned
|
||||
};
|
||||
|
||||
explain.AddProvenance(input.Level.ToString(), score);
|
||||
return score;
|
||||
}
|
||||
|
||||
private static (int Score, string? Override) ApplyOverrides(
|
||||
int score,
|
||||
int reachability,
|
||||
int evidence,
|
||||
bool isKnownExploited,
|
||||
ScorePolicy policy)
|
||||
{
|
||||
if (policy.Overrides is null or { Count: 0 })
|
||||
return (score, null);
|
||||
|
||||
foreach (var rule in policy.Overrides)
|
||||
{
|
||||
if (!MatchesCondition(rule.When, reachability, evidence, isKnownExploited))
|
||||
continue;
|
||||
|
||||
if (rule.SetScore.HasValue)
|
||||
return (rule.SetScore.Value, rule.Name);
|
||||
|
||||
if (rule.ClampMaxScore.HasValue && score > rule.ClampMaxScore.Value)
|
||||
return (rule.ClampMaxScore.Value, $"{rule.Name} (clamped)");
|
||||
|
||||
if (rule.ClampMinScore.HasValue && score < rule.ClampMinScore.Value)
|
||||
return (rule.ClampMinScore.Value, $"{rule.Name} (clamped)");
|
||||
}
|
||||
|
||||
return (score, null);
|
||||
}
|
||||
|
||||
private static bool MatchesCondition(
|
||||
ScoreOverrideCondition condition,
|
||||
int reachability,
|
||||
int evidence,
|
||||
bool isKnownExploited)
|
||||
{
|
||||
if (condition.Flags?.TryGetValue("knownExploited", out var kevRequired) == true)
|
||||
{
|
||||
if (kevRequired != isKnownExploited)
|
||||
return false;
|
||||
}
|
||||
|
||||
if (condition.MinReachability.HasValue && reachability < condition.MinReachability.Value)
|
||||
return false;
|
||||
|
||||
if (condition.MaxReachability.HasValue && reachability > condition.MaxReachability.Value)
|
||||
return false;
|
||||
|
||||
if (condition.MinEvidence.HasValue && evidence < condition.MinEvidence.Value)
|
||||
return false;
|
||||
|
||||
if (condition.MaxEvidence.HasValue && evidence > condition.MaxEvidence.Value)
|
||||
return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private static string MapToSeverity(int score) => score switch
|
||||
{
|
||||
>= 90 => "critical",
|
||||
>= 70 => "high",
|
||||
>= 40 => "medium",
|
||||
>= 20 => "low",
|
||||
_ => "info"
|
||||
};
|
||||
}
|
||||
291
src/Policy/StellaOps.Policy.Engine/Scoring/IScoringEngine.cs
Normal file
291
src/Policy/StellaOps.Policy.Engine/Scoring/IScoringEngine.cs
Normal file
@@ -0,0 +1,291 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// IScoringEngine.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-002
|
||||
// Description: Interface for pluggable scoring engines
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
using StellaOps.Policy.Scoring;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring;
|
||||
|
||||
/// <summary>
|
||||
/// Interface for pluggable scoring engines.
|
||||
/// </summary>
|
||||
public interface IScoringEngine
|
||||
{
|
||||
/// <summary>
|
||||
/// Scoring profile this engine implements.
|
||||
/// </summary>
|
||||
ScoringProfile Profile { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Computes risk score for a finding.
|
||||
/// </summary>
|
||||
/// <param name="input">Scoring input with all factors.</param>
|
||||
/// <param name="policy">Score policy configuration.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
/// <returns>Scoring result with explanation.</returns>
|
||||
Task<ScoringEngineResult> ScoreAsync(
|
||||
ScoringInput input,
|
||||
ScorePolicy policy,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Input for scoring calculation.
|
||||
/// </summary>
|
||||
public sealed record ScoringInput
|
||||
{
|
||||
/// <summary>
|
||||
/// Finding identifier.
|
||||
/// </summary>
|
||||
public required string FindingId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Tenant identifier.
|
||||
/// </summary>
|
||||
public required string TenantId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Profile identifier.
|
||||
/// </summary>
|
||||
public required string ProfileId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Explicit reference time for determinism.
|
||||
/// </summary>
|
||||
public required DateTimeOffset AsOf { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// CVSS base score (0.0-10.0).
|
||||
/// </summary>
|
||||
public required decimal CvssBase { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// CVSS version used.
|
||||
/// </summary>
|
||||
public string? CvssVersion { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Reachability analysis result.
|
||||
/// </summary>
|
||||
public required ReachabilityInput Reachability { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Evidence analysis result.
|
||||
/// </summary>
|
||||
public required EvidenceInput Evidence { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Provenance verification result.
|
||||
/// </summary>
|
||||
public required ProvenanceInput Provenance { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Known Exploited Vulnerability flag.
|
||||
/// </summary>
|
||||
public bool IsKnownExploited { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Input digests for determinism tracking.
|
||||
/// </summary>
|
||||
public IReadOnlyDictionary<string, string>? InputDigests { get; init; }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reachability analysis input.
|
||||
/// </summary>
|
||||
public sealed record ReachabilityInput
|
||||
{
|
||||
/// <summary>
|
||||
/// Hop count to vulnerable code (null = unreachable).
|
||||
/// </summary>
|
||||
public int? HopCount { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Detected gates on the path.
|
||||
/// </summary>
|
||||
public IReadOnlyList<DetectedGate>? Gates { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Semantic reachability category (current advanced model).
|
||||
/// </summary>
|
||||
public string? Category { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Raw reachability score from advanced engine.
|
||||
/// </summary>
|
||||
public double? AdvancedScore { get; init; }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A detected gate that may mitigate reachability.
|
||||
/// </summary>
|
||||
/// <param name="Type">Gate type (e.g., "feature_flag", "auth_required").</param>
|
||||
/// <param name="Detail">Additional detail about the gate.</param>
|
||||
/// <param name="Confidence">Confidence level (0.0-1.0).</param>
|
||||
public sealed record DetectedGate(string Type, string? Detail, double Confidence);
|
||||
|
||||
/// <summary>
|
||||
/// Evidence analysis input.
|
||||
/// </summary>
|
||||
public sealed record EvidenceInput
|
||||
{
|
||||
/// <summary>
|
||||
/// Evidence types present.
|
||||
/// </summary>
|
||||
public required IReadOnlySet<EvidenceType> Types { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Newest evidence timestamp.
|
||||
/// </summary>
|
||||
public DateTimeOffset? NewestEvidenceAt { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Raw evidence score from advanced engine.
|
||||
/// </summary>
|
||||
public double? AdvancedScore { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Creates an empty evidence input.
|
||||
/// </summary>
|
||||
public static EvidenceInput Empty => new()
|
||||
{
|
||||
Types = new HashSet<EvidenceType>()
|
||||
};
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Evidence types that can contribute to scoring.
|
||||
/// </summary>
|
||||
public enum EvidenceType
|
||||
{
|
||||
/// <summary>Runtime execution evidence (highest value).</summary>
|
||||
Runtime,
|
||||
|
||||
/// <summary>Dynamic analysis security testing.</summary>
|
||||
Dast,
|
||||
|
||||
/// <summary>Static analysis security testing.</summary>
|
||||
Sast,
|
||||
|
||||
/// <summary>Software composition analysis.</summary>
|
||||
Sca
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Provenance verification input.
|
||||
/// </summary>
|
||||
public sealed record ProvenanceInput
|
||||
{
|
||||
/// <summary>
|
||||
/// Provenance level.
|
||||
/// </summary>
|
||||
public required ProvenanceLevel Level { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Raw provenance score from advanced engine.
|
||||
/// </summary>
|
||||
public double? AdvancedScore { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Creates default provenance input (unsigned).
|
||||
/// </summary>
|
||||
public static ProvenanceInput Default => new()
|
||||
{
|
||||
Level = ProvenanceLevel.Unsigned
|
||||
};
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Provenance verification levels.
|
||||
/// </summary>
|
||||
public enum ProvenanceLevel
|
||||
{
|
||||
/// <summary>No signature or provenance.</summary>
|
||||
Unsigned,
|
||||
|
||||
/// <summary>Basic signature present.</summary>
|
||||
Signed,
|
||||
|
||||
/// <summary>Signed with SBOM.</summary>
|
||||
SignedWithSbom,
|
||||
|
||||
/// <summary>Signed with SBOM and attestations.</summary>
|
||||
SignedWithSbomAndAttestations,
|
||||
|
||||
/// <summary>Fully reproducible build.</summary>
|
||||
Reproducible
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Result from a scoring engine.
|
||||
/// </summary>
|
||||
public sealed record ScoringEngineResult
|
||||
{
|
||||
/// <summary>
|
||||
/// Finding identifier.
|
||||
/// </summary>
|
||||
public required string FindingId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Profile identifier.
|
||||
/// </summary>
|
||||
public required string ProfileId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Profile version/digest.
|
||||
/// </summary>
|
||||
public required string ProfileVersion { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Raw score before overrides (0-100).
|
||||
/// </summary>
|
||||
public required int RawScore { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Final score after overrides (0-100).
|
||||
/// </summary>
|
||||
public required int FinalScore { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Severity classification.
|
||||
/// </summary>
|
||||
public required string Severity { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Individual signal values used in scoring.
|
||||
/// </summary>
|
||||
public required IReadOnlyDictionary<string, int> SignalValues { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Contribution of each signal to the final score.
|
||||
/// </summary>
|
||||
public required IReadOnlyDictionary<string, double> SignalContributions { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Override rule that was applied, if any.
|
||||
/// </summary>
|
||||
public string? OverrideApplied { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Reason for override, if any.
|
||||
/// </summary>
|
||||
public string? OverrideReason { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Scoring profile used.
|
||||
/// </summary>
|
||||
public required ScoringProfile ScoringProfile { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Timestamp when scoring was performed.
|
||||
/// </summary>
|
||||
public required DateTimeOffset ScoredAt { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Structured explanation of score contributions.
|
||||
/// </summary>
|
||||
public required IReadOnlyList<ScoreExplanation> Explain { get; init; }
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// ProfileAwareScoringService.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-008
|
||||
// Description: Integrates profile switching into the scoring pipeline
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
using Microsoft.Extensions.Logging;
|
||||
using StellaOps.Policy.Scoring;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring;
|
||||
|
||||
/// <summary>
|
||||
/// Profile-aware scoring service that routes to the appropriate scoring engine.
|
||||
/// </summary>
|
||||
public interface IProfileAwareScoringService
|
||||
{
|
||||
/// <summary>
|
||||
/// Scores a finding using the tenant's configured profile.
|
||||
/// </summary>
|
||||
Task<ScoringEngineResult> ScoreAsync(
|
||||
ScoringInput input,
|
||||
CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Scores a finding using a specific profile (for comparison/testing).
|
||||
/// </summary>
|
||||
Task<ScoringEngineResult> ScoreWithProfileAsync(
|
||||
ScoringInput input,
|
||||
ScoringProfile profile,
|
||||
CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Compares scores across different profiles for the same input.
|
||||
/// </summary>
|
||||
Task<ProfileComparisonResult> CompareProfilesAsync(
|
||||
ScoringInput input,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Result of comparing scores across different profiles.
|
||||
/// </summary>
|
||||
public sealed record ProfileComparisonResult
|
||||
{
|
||||
/// <summary>
|
||||
/// Finding identifier.
|
||||
/// </summary>
|
||||
public required string FindingId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Results from each profile.
|
||||
/// </summary>
|
||||
public required IReadOnlyDictionary<ScoringProfile, ScoringEngineResult> Results { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Score variance across profiles.
|
||||
/// </summary>
|
||||
public required int ScoreVariance { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Whether severity differs across profiles.
|
||||
/// </summary>
|
||||
public required bool SeverityDiffers { get; init; }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Implementation of profile-aware scoring service.
|
||||
/// </summary>
|
||||
public sealed class ProfileAwareScoringService : IProfileAwareScoringService
|
||||
{
|
||||
private readonly IScoringEngineFactory _engineFactory;
|
||||
private readonly IScorePolicyService _policyService;
|
||||
private readonly ILogger<ProfileAwareScoringService> _logger;
|
||||
|
||||
public ProfileAwareScoringService(
|
||||
IScoringEngineFactory engineFactory,
|
||||
IScorePolicyService policyService,
|
||||
ILogger<ProfileAwareScoringService> logger)
|
||||
{
|
||||
_engineFactory = engineFactory ?? throw new ArgumentNullException(nameof(engineFactory));
|
||||
_policyService = policyService ?? throw new ArgumentNullException(nameof(policyService));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
public async Task<ScoringEngineResult> ScoreAsync(
|
||||
ScoringInput input,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
|
||||
var engine = _engineFactory.GetEngineForTenant(input.TenantId);
|
||||
var policy = _policyService.GetPolicy(input.TenantId);
|
||||
|
||||
_logger.LogDebug(
|
||||
"Scoring finding {FindingId} with {Profile} profile for tenant {TenantId}",
|
||||
input.FindingId, engine.Profile, input.TenantId);
|
||||
|
||||
return await engine.ScoreAsync(input, policy, ct).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
public async Task<ScoringEngineResult> ScoreWithProfileAsync(
|
||||
ScoringInput input,
|
||||
ScoringProfile profile,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
|
||||
var engine = _engineFactory.GetEngine(profile);
|
||||
var policy = _policyService.GetPolicy(input.TenantId);
|
||||
|
||||
_logger.LogDebug(
|
||||
"Scoring finding {FindingId} with explicit {Profile} profile",
|
||||
input.FindingId, profile);
|
||||
|
||||
return await engine.ScoreAsync(input, policy, ct).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
public async Task<ProfileComparisonResult> CompareProfilesAsync(
|
||||
ScoringInput input,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
|
||||
var profiles = _engineFactory.GetAvailableProfiles();
|
||||
var policy = _policyService.GetPolicy(input.TenantId);
|
||||
var results = new Dictionary<ScoringProfile, ScoringEngineResult>();
|
||||
|
||||
foreach (var profile in profiles)
|
||||
{
|
||||
var engine = _engineFactory.GetEngine(profile);
|
||||
var result = await engine.ScoreAsync(input, policy, ct).ConfigureAwait(false);
|
||||
results[profile] = result;
|
||||
}
|
||||
|
||||
var scores = results.Values.Select(r => r.FinalScore).ToList();
|
||||
var severities = results.Values.Select(r => r.Severity).Distinct().ToList();
|
||||
|
||||
var comparison = new ProfileComparisonResult
|
||||
{
|
||||
FindingId = input.FindingId,
|
||||
Results = results,
|
||||
ScoreVariance = scores.Count > 0 ? scores.Max() - scores.Min() : 0,
|
||||
SeverityDiffers = severities.Count > 1
|
||||
};
|
||||
|
||||
_logger.LogInformation(
|
||||
"Profile comparison for {FindingId}: variance={Variance}, severity_differs={SeverityDiffers}",
|
||||
input.FindingId, comparison.ScoreVariance, comparison.SeverityDiffers);
|
||||
|
||||
return comparison;
|
||||
}
|
||||
}
|
||||
@@ -132,6 +132,7 @@ public enum RiskScoringJobStatus
|
||||
/// <param name="OverrideApplied">Override rule that was applied, if any.</param>
|
||||
/// <param name="OverrideReason">Reason for the override, if any.</param>
|
||||
/// <param name="ScoredAt">Timestamp when scoring was performed.</param>
|
||||
/// <param name="ScoringProfile">Scoring profile used (Simple, Advanced, Custom).</param>
|
||||
public sealed record RiskScoringResult(
|
||||
[property: JsonPropertyName("finding_id")] string FindingId,
|
||||
[property: JsonPropertyName("profile_id")] string ProfileId,
|
||||
@@ -143,7 +144,8 @@ public sealed record RiskScoringResult(
|
||||
[property: JsonPropertyName("signal_contributions")] IReadOnlyDictionary<string, double> SignalContributions,
|
||||
[property: JsonPropertyName("override_applied")] string? OverrideApplied,
|
||||
[property: JsonPropertyName("override_reason")] string? OverrideReason,
|
||||
[property: JsonPropertyName("scored_at")] DateTimeOffset ScoredAt)
|
||||
[property: JsonPropertyName("scored_at")] DateTimeOffset ScoredAt,
|
||||
[property: JsonPropertyName("scoring_profile")] string? ScoringProfile = null)
|
||||
{
|
||||
private IReadOnlyList<ScoreExplanation> _explain = Array.Empty<ScoreExplanation>();
|
||||
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// ScoringEngineFactory.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-005
|
||||
// Description: Factory for creating scoring engines based on profile
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using StellaOps.Policy.Engine.Scoring.Engines;
|
||||
using StellaOps.Policy.Scoring;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring;
|
||||
|
||||
/// <summary>
|
||||
/// Factory for creating scoring engines based on profile.
|
||||
/// </summary>
|
||||
public interface IScoringEngineFactory
|
||||
{
|
||||
/// <summary>
|
||||
/// Gets a scoring engine for the specified profile.
|
||||
/// </summary>
|
||||
IScoringEngine GetEngine(ScoringProfile profile);
|
||||
|
||||
/// <summary>
|
||||
/// Gets a scoring engine for a tenant's configured profile.
|
||||
/// </summary>
|
||||
IScoringEngine GetEngineForTenant(string tenantId);
|
||||
|
||||
/// <summary>
|
||||
/// Gets all available profiles.
|
||||
/// </summary>
|
||||
IReadOnlyList<ScoringProfile> GetAvailableProfiles();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Default implementation of scoring engine factory.
|
||||
/// </summary>
|
||||
public sealed class ScoringEngineFactory : IScoringEngineFactory
|
||||
{
|
||||
private readonly IServiceProvider _services;
|
||||
private readonly IScoringProfileService _profileService;
|
||||
private readonly ILogger<ScoringEngineFactory> _logger;
|
||||
|
||||
public ScoringEngineFactory(
|
||||
IServiceProvider services,
|
||||
IScoringProfileService profileService,
|
||||
ILogger<ScoringEngineFactory> logger)
|
||||
{
|
||||
_services = services ?? throw new ArgumentNullException(nameof(services));
|
||||
_profileService = profileService ?? throw new ArgumentNullException(nameof(profileService));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets a scoring engine for the specified profile.
|
||||
/// </summary>
|
||||
public IScoringEngine GetEngine(ScoringProfile profile)
|
||||
{
|
||||
var engine = profile switch
|
||||
{
|
||||
ScoringProfile.Simple => _services.GetRequiredService<SimpleScoringEngine>(),
|
||||
ScoringProfile.Advanced => _services.GetRequiredService<AdvancedScoringEngine>(),
|
||||
ScoringProfile.Custom => throw new NotSupportedException(
|
||||
"Custom scoring profile requires Rego policy configuration. Use GetCustomEngine instead."),
|
||||
_ => throw new ArgumentOutOfRangeException(nameof(profile), profile, "Unknown scoring profile")
|
||||
};
|
||||
|
||||
_logger.LogDebug("Created scoring engine for profile {Profile}", profile);
|
||||
return engine;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets a scoring engine for a tenant's configured profile.
|
||||
/// </summary>
|
||||
public IScoringEngine GetEngineForTenant(string tenantId)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(tenantId);
|
||||
|
||||
var profileConfig = _profileService.GetProfileForTenant(tenantId);
|
||||
var profile = profileConfig?.Profile ?? ScoringProfile.Advanced;
|
||||
|
||||
_logger.LogDebug(
|
||||
"Resolved scoring profile {Profile} for tenant {TenantId}",
|
||||
profile, tenantId);
|
||||
|
||||
return GetEngine(profile);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets all available profiles.
|
||||
/// </summary>
|
||||
public IReadOnlyList<ScoringProfile> GetAvailableProfiles()
|
||||
{
|
||||
return
|
||||
[
|
||||
ScoringProfile.Simple,
|
||||
ScoringProfile.Advanced
|
||||
// Custom is not listed as generally available
|
||||
];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// ScoringProfileService.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-006
|
||||
// Description: Service for managing tenant scoring profile configurations
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
using System.Collections.Concurrent;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using StellaOps.Policy.Scoring;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring;
|
||||
|
||||
/// <summary>
|
||||
/// Service for managing tenant scoring profile configurations.
|
||||
/// </summary>
|
||||
public interface IScoringProfileService
|
||||
{
|
||||
/// <summary>
|
||||
/// Gets the scoring profile configuration for a tenant.
|
||||
/// </summary>
|
||||
/// <param name="tenantId">Tenant identifier.</param>
|
||||
/// <returns>Profile configuration, or null for default.</returns>
|
||||
ScoringProfileConfig? GetProfileForTenant(string tenantId);
|
||||
|
||||
/// <summary>
|
||||
/// Sets the scoring profile for a tenant.
|
||||
/// </summary>
|
||||
/// <param name="tenantId">Tenant identifier.</param>
|
||||
/// <param name="config">Profile configuration.</param>
|
||||
void SetProfileForTenant(string tenantId, ScoringProfileConfig config);
|
||||
|
||||
/// <summary>
|
||||
/// Removes custom profile for a tenant (reverts to default).
|
||||
/// </summary>
|
||||
/// <param name="tenantId">Tenant identifier.</param>
|
||||
/// <returns>True if a profile was removed.</returns>
|
||||
bool RemoveProfileForTenant(string tenantId);
|
||||
|
||||
/// <summary>
|
||||
/// Gets all tenants with custom profile configurations.
|
||||
/// </summary>
|
||||
IReadOnlyDictionary<string, ScoringProfileConfig> GetAllProfiles();
|
||||
|
||||
/// <summary>
|
||||
/// Gets the default profile for new tenants.
|
||||
/// </summary>
|
||||
ScoringProfileConfig DefaultProfile { get; }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// In-memory implementation of scoring profile service.
|
||||
/// For production, this should be backed by persistent storage.
|
||||
/// </summary>
|
||||
public sealed class ScoringProfileService : IScoringProfileService
|
||||
{
|
||||
private readonly ConcurrentDictionary<string, ScoringProfileConfig> _profiles = new();
|
||||
private readonly IScorePolicyService _policyService;
|
||||
private readonly ILogger<ScoringProfileService> _logger;
|
||||
|
||||
public ScoringProfileConfig DefaultProfile { get; } = ScoringProfileConfig.DefaultAdvanced;
|
||||
|
||||
public ScoringProfileService(
|
||||
IScorePolicyService policyService,
|
||||
ILogger<ScoringProfileService> logger)
|
||||
{
|
||||
_policyService = policyService ?? throw new ArgumentNullException(nameof(policyService));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
public ScoringProfileConfig? GetProfileForTenant(string tenantId)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(tenantId);
|
||||
|
||||
// First, check for explicit tenant profile
|
||||
if (_profiles.TryGetValue(tenantId, out var profile))
|
||||
{
|
||||
return profile;
|
||||
}
|
||||
|
||||
// Then, check the score policy for profile setting
|
||||
try
|
||||
{
|
||||
var policy = _policyService.GetPolicy(tenantId);
|
||||
var policyProfile = ParseProfileFromPolicy(policy);
|
||||
if (policyProfile.HasValue)
|
||||
{
|
||||
return new ScoringProfileConfig
|
||||
{
|
||||
Profile = policyProfile.Value
|
||||
};
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex,
|
||||
"Failed to get score policy for tenant {TenantId}, using default profile",
|
||||
tenantId);
|
||||
}
|
||||
|
||||
// Default: return null (caller uses default)
|
||||
return null;
|
||||
}
|
||||
|
||||
public void SetProfileForTenant(string tenantId, ScoringProfileConfig config)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(tenantId);
|
||||
ArgumentNullException.ThrowIfNull(config);
|
||||
|
||||
_profiles[tenantId] = config;
|
||||
|
||||
_logger.LogInformation(
|
||||
"Set scoring profile {Profile} for tenant {TenantId}",
|
||||
config.Profile, tenantId);
|
||||
}
|
||||
|
||||
public bool RemoveProfileForTenant(string tenantId)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(tenantId);
|
||||
|
||||
var removed = _profiles.TryRemove(tenantId, out _);
|
||||
|
||||
if (removed)
|
||||
{
|
||||
_logger.LogInformation(
|
||||
"Removed custom scoring profile for tenant {TenantId}, reverted to default",
|
||||
tenantId);
|
||||
}
|
||||
|
||||
return removed;
|
||||
}
|
||||
|
||||
public IReadOnlyDictionary<string, ScoringProfileConfig> GetAllProfiles()
|
||||
{
|
||||
return _profiles.ToDictionary(kvp => kvp.Key, kvp => kvp.Value);
|
||||
}
|
||||
|
||||
private static ScoringProfile? ParseProfileFromPolicy(ScorePolicy policy)
|
||||
{
|
||||
// Check if policy has a scoring profile setting
|
||||
// This would be read from the YAML scoringProfile field
|
||||
var profileStr = policy.ScoringProfile;
|
||||
if (string.IsNullOrWhiteSpace(profileStr))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return profileStr.ToLowerInvariant() switch
|
||||
{
|
||||
"simple" => ScoringProfile.Simple,
|
||||
"advanced" => ScoringProfile.Advanced,
|
||||
"custom" => ScoringProfile.Custom,
|
||||
_ => null
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,13 @@ namespace StellaOps.Policy.Scoring;
|
||||
public sealed record ScorePolicy
|
||||
{
|
||||
public required string PolicyVersion { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Scoring profile to use. Defaults to "advanced".
|
||||
/// Options: "simple", "advanced", "custom"
|
||||
/// </summary>
|
||||
public string ScoringProfile { get; init; } = "advanced";
|
||||
|
||||
public required WeightsBps WeightsBps { get; init; }
|
||||
public ReachabilityPolicyConfig? Reachability { get; init; }
|
||||
public EvidencePolicyConfig? Evidence { get; init; }
|
||||
@@ -28,6 +35,7 @@ public sealed record ScorePolicy
|
||||
public static ScorePolicy Default => new()
|
||||
{
|
||||
PolicyVersion = "score.v1",
|
||||
ScoringProfile = "advanced",
|
||||
WeightsBps = WeightsBps.Default,
|
||||
Reachability = ReachabilityPolicyConfig.Default,
|
||||
Evidence = EvidencePolicyConfig.Default,
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
// =============================================================================
|
||||
// ScorePolicyValidator.cs
|
||||
// Sprint: SPRINT_3402_0001_0001
|
||||
// Task: YAML-3402-003 - Implement ScorePolicyValidator with JSON Schema validation
|
||||
// =============================================================================
|
||||
|
||||
using System.Text.Json;
|
||||
using Json.Schema;
|
||||
|
||||
namespace StellaOps.Policy.Scoring;
|
||||
|
||||
/// <summary>
|
||||
/// Validates score policies against JSON Schema.
|
||||
/// </summary>
|
||||
public sealed class ScorePolicyValidator
|
||||
{
|
||||
private readonly JsonSchema _schema;
|
||||
|
||||
/// <summary>
|
||||
/// Creates a validator with the embedded score.v1 schema.
|
||||
/// </summary>
|
||||
public ScorePolicyValidator()
|
||||
{
|
||||
_schema = JsonSchema.FromText(ScorePolicySchemaJson);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates a validator with a custom schema.
|
||||
/// </summary>
|
||||
public ScorePolicyValidator(string schemaJson)
|
||||
{
|
||||
_schema = JsonSchema.FromText(schemaJson);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Validates a score policy.
|
||||
/// </summary>
|
||||
/// <param name="policy">The policy to validate</param>
|
||||
/// <returns>Validation result with errors if any</returns>
|
||||
public ScorePolicyValidationResult Validate(ScorePolicy policy)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(policy);
|
||||
|
||||
var json = JsonSerializer.Serialize(policy, JsonOptions);
|
||||
var jsonDoc = JsonDocument.Parse(json);
|
||||
|
||||
var result = _schema.Evaluate(jsonDoc.RootElement);
|
||||
|
||||
if (result.IsValid)
|
||||
{
|
||||
return new ScorePolicyValidationResult(true, []);
|
||||
}
|
||||
|
||||
var errors = CollectErrors(result);
|
||||
return new ScorePolicyValidationResult(false, errors);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Validates JSON content against the schema.
|
||||
/// </summary>
|
||||
public ScorePolicyValidationResult ValidateJson(string json)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(json))
|
||||
{
|
||||
return new ScorePolicyValidationResult(false, ["JSON content is empty"]);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
var jsonDoc = JsonDocument.Parse(json);
|
||||
var result = _schema.Evaluate(jsonDoc.RootElement);
|
||||
|
||||
if (result.IsValid)
|
||||
{
|
||||
return new ScorePolicyValidationResult(true, []);
|
||||
}
|
||||
|
||||
var errors = CollectErrors(result);
|
||||
return new ScorePolicyValidationResult(false, errors);
|
||||
}
|
||||
catch (JsonException ex)
|
||||
{
|
||||
return new ScorePolicyValidationResult(false, [$"Invalid JSON: {ex.Message}"]);
|
||||
}
|
||||
}
|
||||
|
||||
private static List<string> CollectErrors(EvaluationResults result)
|
||||
{
|
||||
var errors = new List<string>();
|
||||
CollectErrorsRecursive(result, errors);
|
||||
return errors;
|
||||
}
|
||||
|
||||
private static void CollectErrorsRecursive(EvaluationResults result, List<string> errors)
|
||||
{
|
||||
if (!result.IsValid && result.Errors is { Count: > 0 })
|
||||
{
|
||||
foreach (var error in result.Errors)
|
||||
{
|
||||
errors.Add($"{result.InstanceLocation}: {error.Key} - {error.Value}");
|
||||
}
|
||||
}
|
||||
|
||||
if (result.Details is null) return;
|
||||
|
||||
foreach (var detail in result.Details)
|
||||
{
|
||||
CollectErrorsRecursive(detail, errors);
|
||||
}
|
||||
}
|
||||
|
||||
private static readonly JsonSerializerOptions JsonOptions = new()
|
||||
{
|
||||
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
|
||||
WriteIndented = false
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// Embedded JSON Schema for score.v1 policies.
|
||||
/// </summary>
|
||||
private const string ScorePolicySchemaJson = """
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.dev/schemas/score-policy.v1.json",
|
||||
"title": "Score Policy",
|
||||
"type": "object",
|
||||
"required": ["policyVersion", "policyId", "weightsBps"],
|
||||
"properties": {
|
||||
"policyVersion": {
|
||||
"type": "string",
|
||||
"const": "score.v1"
|
||||
},
|
||||
"policyId": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"policyName": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": {
|
||||
"type": "string"
|
||||
},
|
||||
"weightsBps": {
|
||||
"$ref": "#/$defs/WeightsBps"
|
||||
},
|
||||
"reachabilityConfig": {
|
||||
"$ref": "#/$defs/ReachabilityConfig"
|
||||
},
|
||||
"evidenceConfig": {
|
||||
"$ref": "#/$defs/EvidenceConfig"
|
||||
},
|
||||
"provenanceConfig": {
|
||||
"$ref": "#/$defs/ProvenanceConfig"
|
||||
},
|
||||
"overrides": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/ScoreOverride"
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"WeightsBps": {
|
||||
"type": "object",
|
||||
"required": ["baseSeverity", "reachability", "evidence", "provenance"],
|
||||
"properties": {
|
||||
"baseSeverity": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 10000
|
||||
},
|
||||
"reachability": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 10000
|
||||
},
|
||||
"evidence": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 10000
|
||||
},
|
||||
"provenance": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 10000
|
||||
}
|
||||
}
|
||||
},
|
||||
"ReachabilityConfig": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"reachableMultiplier": {
|
||||
"type": "number",
|
||||
"minimum": 0,
|
||||
"maximum": 2
|
||||
},
|
||||
"unreachableMultiplier": {
|
||||
"type": "number",
|
||||
"minimum": 0,
|
||||
"maximum": 2
|
||||
},
|
||||
"unknownMultiplier": {
|
||||
"type": "number",
|
||||
"minimum": 0,
|
||||
"maximum": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
"EvidenceConfig": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kevWeight": {
|
||||
"type": "number",
|
||||
"minimum": 0
|
||||
},
|
||||
"epssThreshold": {
|
||||
"type": "number",
|
||||
"minimum": 0,
|
||||
"maximum": 1
|
||||
},
|
||||
"epssWeight": {
|
||||
"type": "number",
|
||||
"minimum": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
"ProvenanceConfig": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"signedBonus": {
|
||||
"type": "number"
|
||||
},
|
||||
"rekorVerifiedBonus": {
|
||||
"type": "number"
|
||||
},
|
||||
"unsignedPenalty": {
|
||||
"type": "number"
|
||||
}
|
||||
}
|
||||
},
|
||||
"ScoreOverride": {
|
||||
"type": "object",
|
||||
"required": ["id", "match"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"match": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"cvePattern": {
|
||||
"type": "string"
|
||||
},
|
||||
"purlPattern": {
|
||||
"type": "string"
|
||||
},
|
||||
"severityEquals": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"action": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"setScore": {
|
||||
"type": "number"
|
||||
},
|
||||
"addScore": {
|
||||
"type": "number"
|
||||
},
|
||||
"multiplyScore": {
|
||||
"type": "number"
|
||||
}
|
||||
}
|
||||
},
|
||||
"reason": {
|
||||
"type": "string"
|
||||
},
|
||||
"expires": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
""";
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Result of score policy validation.
|
||||
/// </summary>
|
||||
/// <param name="IsValid">Whether the policy is valid</param>
|
||||
/// <param name="Errors">List of validation errors (empty if valid)</param>
|
||||
public readonly record struct ScorePolicyValidationResult(bool IsValid, IReadOnlyList<string> Errors)
|
||||
{
|
||||
/// <summary>
|
||||
/// Throws if validation failed.
|
||||
/// </summary>
|
||||
public void ThrowIfInvalid(string context = "")
|
||||
{
|
||||
if (!IsValid)
|
||||
{
|
||||
var prefix = string.IsNullOrEmpty(context) ? "" : $"{context}: ";
|
||||
throw new ScorePolicyValidationException(
|
||||
$"{prefix}Score policy validation failed: {string.Join("; ", Errors)}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Exception thrown when score policy validation fails.
|
||||
/// </summary>
|
||||
public sealed class ScorePolicyValidationException : Exception
|
||||
{
|
||||
public ScorePolicyValidationException(string message) : base(message) { }
|
||||
public ScorePolicyValidationException(string message, Exception inner) : base(message, inner) { }
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
// -----------------------------------------------------------------------------
|
||||
// ScoringProfile.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-001
|
||||
// Description: Defines scoring profiles for pluggable scoring engines
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
namespace StellaOps.Policy.Scoring;
|
||||
|
||||
/// <summary>
|
||||
/// Available scoring profiles.
|
||||
/// </summary>
|
||||
public enum ScoringProfile
|
||||
{
|
||||
/// <summary>
|
||||
/// Simple 4-factor basis-points weighted scoring.
|
||||
/// Formula: riskScore = (wB*B + wR*R + wE*E + wP*P) / 10000
|
||||
/// Transparent, customer-configurable via YAML.
|
||||
/// </summary>
|
||||
Simple,
|
||||
|
||||
/// <summary>
|
||||
/// Advanced entropy-based + CVSS hybrid scoring.
|
||||
/// Uses uncertainty tiers, entropy penalties, and CVSS v4.0 receipts.
|
||||
/// Default for new deployments.
|
||||
/// </summary>
|
||||
Advanced,
|
||||
|
||||
/// <summary>
|
||||
/// Custom scoring using fully user-defined rules.
|
||||
/// Requires Rego policy configuration.
|
||||
/// </summary>
|
||||
Custom
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Scoring profile configuration.
|
||||
/// </summary>
|
||||
public sealed record ScoringProfileConfig
|
||||
{
|
||||
/// <summary>
|
||||
/// Active scoring profile.
|
||||
/// </summary>
|
||||
public required ScoringProfile Profile { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Profile-specific settings.
|
||||
/// </summary>
|
||||
public IReadOnlyDictionary<string, string>? Settings { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// For Custom profile: path to Rego policy.
|
||||
/// </summary>
|
||||
public string? CustomPolicyPath { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Creates default configuration for Advanced profile.
|
||||
/// </summary>
|
||||
public static ScoringProfileConfig DefaultAdvanced => new()
|
||||
{
|
||||
Profile = ScoringProfile.Advanced
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// Creates default configuration for Simple profile.
|
||||
/// </summary>
|
||||
public static ScoringProfileConfig DefaultSimple => new()
|
||||
{
|
||||
Profile = ScoringProfile.Simple
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,330 @@
|
||||
// =============================================================================
|
||||
// AdvancedScoringEngineTests.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-011 - Unit tests for AdvancedScoringEngine (regression)
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using StellaOps.Policy.Engine.Scoring.Engines;
|
||||
using StellaOps.Policy.Scoring;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Unit tests for AdvancedScoringEngine.
|
||||
/// Ensures regression testing for existing advanced scoring functionality.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3407")]
|
||||
public sealed class AdvancedScoringEngineTests
|
||||
{
|
||||
private readonly AdvancedScoringEngine _engine;
|
||||
private readonly EvidenceFreshnessCalculator _freshnessCalculator;
|
||||
private readonly ScorePolicy _defaultPolicy;
|
||||
|
||||
public AdvancedScoringEngineTests()
|
||||
{
|
||||
_freshnessCalculator = new EvidenceFreshnessCalculator();
|
||||
_engine = new AdvancedScoringEngine(
|
||||
_freshnessCalculator,
|
||||
NullLogger<AdvancedScoringEngine>.Instance);
|
||||
_defaultPolicy = ScorePolicy.Default;
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Profile returns Advanced")]
|
||||
public void Profile_ReturnsAdvanced()
|
||||
{
|
||||
_engine.Profile.Should().Be(ScoringProfile.Advanced);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync applies CVSS version adjustment")]
|
||||
public async Task ScoreAsync_AppliesCvssVersionAdjustment()
|
||||
{
|
||||
var v4Input = CreateInput(cvss: 8.0m, hopCount: 0, cvssVersion: "4.0");
|
||||
var v31Input = CreateInput(cvss: 8.0m, hopCount: 0, cvssVersion: "3.1");
|
||||
var v2Input = CreateInput(cvss: 8.0m, hopCount: 0, cvssVersion: "2.0");
|
||||
|
||||
var v4Result = await _engine.ScoreAsync(v4Input, _defaultPolicy);
|
||||
var v31Result = await _engine.ScoreAsync(v31Input, _defaultPolicy);
|
||||
var v2Result = await _engine.ScoreAsync(v2Input, _defaultPolicy);
|
||||
|
||||
// v4.0 should have highest base severity, v2.0 lowest
|
||||
v4Result.SignalValues["baseSeverity"].Should().BeGreaterThan(v31Result.SignalValues["baseSeverity"]);
|
||||
v31Result.SignalValues["baseSeverity"].Should().BeGreaterThan(v2Result.SignalValues["baseSeverity"]);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync applies KEV boost for known exploited")]
|
||||
public async Task ScoreAsync_AppliesKevBoost()
|
||||
{
|
||||
var normalInput = CreateInput(cvss: 5.0m, hopCount: 2);
|
||||
var kevInput = CreateInput(cvss: 5.0m, hopCount: 2) with
|
||||
{
|
||||
IsKnownExploited = true
|
||||
};
|
||||
|
||||
var normalResult = await _engine.ScoreAsync(normalInput, _defaultPolicy);
|
||||
var kevResult = await _engine.ScoreAsync(kevInput, _defaultPolicy);
|
||||
|
||||
kevResult.RawScore.Should().BeGreaterThan(normalResult.RawScore);
|
||||
kevResult.SignalValues["kevBoost"].Should().Be(20);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync applies uncertainty penalty for missing data")]
|
||||
public async Task ScoreAsync_AppliesUncertaintyPenalty()
|
||||
{
|
||||
var completeInput = CreateInput(cvss: 5.0m, hopCount: 2, cvssVersion: "4.0");
|
||||
completeInput = completeInput with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = DateTimeOffset.UtcNow
|
||||
},
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Signed }
|
||||
};
|
||||
|
||||
var incompleteInput = CreateInput(cvss: 5.0m, hopCount: null);
|
||||
|
||||
var completeResult = await _engine.ScoreAsync(completeInput, _defaultPolicy);
|
||||
var incompleteResult = await _engine.ScoreAsync(incompleteInput, _defaultPolicy);
|
||||
|
||||
incompleteResult.SignalValues["uncertaintyPenalty"].Should().BeGreaterThan(0);
|
||||
completeResult.SignalValues["uncertaintyPenalty"].Should().Be(0);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync uses advanced reachability score when provided")]
|
||||
public async Task ScoreAsync_UsesAdvancedReachabilityScore()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 5);
|
||||
input = input with
|
||||
{
|
||||
Reachability = input.Reachability with
|
||||
{
|
||||
AdvancedScore = 0.95,
|
||||
Category = "api_endpoint"
|
||||
}
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["reachability"].Should().Be(95);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync applies semantic category multiplier")]
|
||||
public async Task ScoreAsync_AppliesSemanticCategoryMultiplier()
|
||||
{
|
||||
var apiInput = CreateInput(cvss: 5.0m, hopCount: 2);
|
||||
apiInput = apiInput with
|
||||
{
|
||||
Reachability = apiInput.Reachability with
|
||||
{
|
||||
Category = "api_endpoint"
|
||||
}
|
||||
};
|
||||
|
||||
var internalInput = CreateInput(cvss: 5.0m, hopCount: 2);
|
||||
internalInput = internalInput with
|
||||
{
|
||||
Reachability = internalInput.Reachability with
|
||||
{
|
||||
Category = "internal_service"
|
||||
}
|
||||
};
|
||||
|
||||
var deadCodeInput = CreateInput(cvss: 5.0m, hopCount: 2);
|
||||
deadCodeInput = deadCodeInput with
|
||||
{
|
||||
Reachability = deadCodeInput.Reachability with
|
||||
{
|
||||
Category = "dead_code"
|
||||
}
|
||||
};
|
||||
|
||||
var apiResult = await _engine.ScoreAsync(apiInput, _defaultPolicy);
|
||||
var internalResult = await _engine.ScoreAsync(internalInput, _defaultPolicy);
|
||||
var deadCodeResult = await _engine.ScoreAsync(deadCodeInput, _defaultPolicy);
|
||||
|
||||
apiResult.SignalValues["reachability"].Should().BeGreaterThan(internalResult.SignalValues["reachability"]);
|
||||
internalResult.SignalValues["reachability"].Should().BeGreaterThan(deadCodeResult.SignalValues["reachability"]);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync applies multi-evidence overlap bonus")]
|
||||
public async Task ScoreAsync_AppliesMultiEvidenceOverlapBonus()
|
||||
{
|
||||
var asOf = DateTimeOffset.UtcNow;
|
||||
var singleInput = CreateInput(cvss: 5.0m, hopCount: 0, asOf: asOf);
|
||||
singleInput = singleInput with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Sca },
|
||||
NewestEvidenceAt = asOf
|
||||
}
|
||||
};
|
||||
|
||||
var multiInput = CreateInput(cvss: 5.0m, hopCount: 0, asOf: asOf);
|
||||
multiInput = multiInput with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Sca, EvidenceType.Sast, EvidenceType.Dast },
|
||||
NewestEvidenceAt = asOf
|
||||
}
|
||||
};
|
||||
|
||||
var singleResult = await _engine.ScoreAsync(singleInput, _defaultPolicy);
|
||||
var multiResult = await _engine.ScoreAsync(multiInput, _defaultPolicy);
|
||||
|
||||
// Multi-evidence should have higher score due to overlap bonus
|
||||
multiResult.SignalValues["evidence"].Should().BeGreaterThan(singleResult.SignalValues["evidence"]);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync uses advanced evidence score when provided")]
|
||||
public async Task ScoreAsync_UsesAdvancedEvidenceScore()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
input = input with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType>(),
|
||||
AdvancedScore = 0.75
|
||||
}
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["evidence"].Should().Be(75);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync uses advanced provenance score when provided")]
|
||||
public async Task ScoreAsync_UsesAdvancedProvenanceScore()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
input = input with
|
||||
{
|
||||
Provenance = new ProvenanceInput
|
||||
{
|
||||
Level = ProvenanceLevel.Unsigned,
|
||||
AdvancedScore = 0.80
|
||||
}
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["provenance"].Should().Be(80);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync is deterministic")]
|
||||
public async Task ScoreAsync_IsDeterministic()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 7.5m, hopCount: 2, asOf: asOf);
|
||||
|
||||
var result1 = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
var result2 = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result1.RawScore.Should().Be(result2.RawScore);
|
||||
result1.FinalScore.Should().Be(result2.FinalScore);
|
||||
result1.Severity.Should().Be(result2.Severity);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync generates explain entries with advanced factors")]
|
||||
public async Task ScoreAsync_GeneratesExplainEntriesWithAdvancedFactors()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 3) with
|
||||
{
|
||||
IsKnownExploited = true
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.Explain.Should().NotBeEmpty();
|
||||
result.Explain.Should().Contain(e => e.Factor == "baseSeverity");
|
||||
result.Explain.Should().Contain(e => e.Factor == "reachability");
|
||||
result.Explain.Should().Contain(e => e.Factor == "kev_boost");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with missing CVSS version applies uncertainty penalty")]
|
||||
public async Task ScoreAsync_MissingCvssVersion_AppliesUncertaintyPenalty()
|
||||
{
|
||||
var withVersionInput = CreateInput(cvss: 5.0m, hopCount: 0, cvssVersion: "4.0");
|
||||
var noVersionInput = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
noVersionInput = noVersionInput with { CvssVersion = null };
|
||||
|
||||
var withVersionResult = await _engine.ScoreAsync(withVersionInput, _defaultPolicy);
|
||||
var noVersionResult = await _engine.ScoreAsync(noVersionInput, _defaultPolicy);
|
||||
|
||||
noVersionResult.SignalValues["uncertaintyPenalty"].Should().BeGreaterThan(0);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with all factors maxed returns critical")]
|
||||
public async Task ScoreAsync_AllFactorsMaxed_ReturnsCritical()
|
||||
{
|
||||
var asOf = DateTimeOffset.UtcNow;
|
||||
var input = CreateInput(cvss: 10.0m, hopCount: 0, asOf: asOf, cvssVersion: "4.0");
|
||||
input = input with
|
||||
{
|
||||
IsKnownExploited = true,
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = asOf
|
||||
},
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Reproducible }
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.Severity.Should().Be("critical");
|
||||
result.FinalScore.Should().BeGreaterOrEqualTo(90);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with gate applies gate multiplier")]
|
||||
public async Task ScoreAsync_WithGate_AppliesGateMultiplier()
|
||||
{
|
||||
var noGateInput = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
var withGateInput = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
withGateInput = withGateInput with
|
||||
{
|
||||
Reachability = withGateInput.Reachability with
|
||||
{
|
||||
Gates =
|
||||
[
|
||||
new DetectedGate("admin_only", "requires admin role", 1.0)
|
||||
]
|
||||
}
|
||||
};
|
||||
|
||||
var noGateResult = await _engine.ScoreAsync(noGateInput, _defaultPolicy);
|
||||
var withGateResult = await _engine.ScoreAsync(withGateInput, _defaultPolicy);
|
||||
|
||||
withGateResult.SignalValues["reachability"].Should().BeLessThan(noGateResult.SignalValues["reachability"]);
|
||||
}
|
||||
|
||||
private static ScoringInput CreateInput(
|
||||
decimal cvss,
|
||||
int? hopCount,
|
||||
DateTimeOffset? asOf = null,
|
||||
string? cvssVersion = null)
|
||||
{
|
||||
return new ScoringInput
|
||||
{
|
||||
FindingId = "test-finding-1",
|
||||
TenantId = "test-tenant",
|
||||
ProfileId = "test-profile",
|
||||
AsOf = asOf ?? DateTimeOffset.UtcNow,
|
||||
CvssBase = cvss,
|
||||
CvssVersion = cvssVersion ?? "3.1",
|
||||
Reachability = new ReachabilityInput
|
||||
{
|
||||
HopCount = hopCount
|
||||
},
|
||||
Evidence = EvidenceInput.Empty,
|
||||
Provenance = ProvenanceInput.Default,
|
||||
IsKnownExploited = false
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,263 @@
|
||||
// =============================================================================
|
||||
// ProfileComparisonIntegrationTests.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-013 - Integration test: same input, different profiles
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using StellaOps.Policy.Engine.Scoring.Engines;
|
||||
using StellaOps.Policy.Scoring;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Integration tests comparing scores across different profiles for identical inputs.
|
||||
/// </summary>
|
||||
[Trait("Category", "Integration")]
|
||||
[Trait("Sprint", "3407")]
|
||||
public sealed class ProfileComparisonIntegrationTests
|
||||
{
|
||||
private readonly SimpleScoringEngine _simpleEngine;
|
||||
private readonly AdvancedScoringEngine _advancedEngine;
|
||||
private readonly ScorePolicy _defaultPolicy;
|
||||
|
||||
public ProfileComparisonIntegrationTests()
|
||||
{
|
||||
var freshnessCalculator = new EvidenceFreshnessCalculator();
|
||||
|
||||
_simpleEngine = new SimpleScoringEngine(
|
||||
freshnessCalculator,
|
||||
NullLogger<SimpleScoringEngine>.Instance);
|
||||
|
||||
_advancedEngine = new AdvancedScoringEngine(
|
||||
freshnessCalculator,
|
||||
NullLogger<AdvancedScoringEngine>.Instance);
|
||||
|
||||
_defaultPolicy = ScorePolicy.Default;
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Same input produces comparable scores across profiles")]
|
||||
public async Task SameInput_ProducesComparableScores()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 7.5m, hopCount: 2, asOf: asOf);
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
// Both should produce valid results
|
||||
simpleResult.Should().NotBeNull();
|
||||
advancedResult.Should().NotBeNull();
|
||||
|
||||
// Scores should be in valid range
|
||||
simpleResult.FinalScore.Should().BeInRange(0, 100);
|
||||
advancedResult.FinalScore.Should().BeInRange(0, 100);
|
||||
|
||||
// Both should use correct profiles
|
||||
simpleResult.ScoringProfile.Should().Be(ScoringProfile.Simple);
|
||||
advancedResult.ScoringProfile.Should().Be(ScoringProfile.Advanced);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Same high-risk input produces similar severity across profiles")]
|
||||
public async Task HighRiskInput_ProducesSimilarSeverity()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 9.8m, hopCount: 0, asOf: asOf);
|
||||
input = input with
|
||||
{
|
||||
IsKnownExploited = true,
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = asOf
|
||||
},
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Reproducible }
|
||||
};
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
// Both should identify this as high risk
|
||||
simpleResult.Severity.Should().BeOneOf("critical", "high");
|
||||
advancedResult.Severity.Should().BeOneOf("critical", "high");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Same low-risk input produces similar severity across profiles")]
|
||||
public async Task LowRiskInput_ProducesSimilarSeverity()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 2.0m, hopCount: null, asOf: asOf);
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
// Both should identify this as low risk
|
||||
simpleResult.Severity.Should().BeOneOf("info", "low");
|
||||
advancedResult.Severity.Should().BeOneOf("info", "low");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Both profiles are deterministic with same input")]
|
||||
public async Task BothProfiles_AreDeterministic()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 6, 15, 12, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 6.5m, hopCount: 3, asOf: asOf);
|
||||
input = input with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Sca, EvidenceType.Sast },
|
||||
NewestEvidenceAt = asOf.AddDays(-14)
|
||||
}
|
||||
};
|
||||
|
||||
var simpleResult1 = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var simpleResult2 = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult1 = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult2 = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
simpleResult1.FinalScore.Should().Be(simpleResult2.FinalScore);
|
||||
advancedResult1.FinalScore.Should().Be(advancedResult2.FinalScore);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Score variance across profiles is reasonable")]
|
||||
public async Task ScoreVariance_IsReasonable()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 2, asOf: asOf);
|
||||
input = input with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Sca },
|
||||
NewestEvidenceAt = asOf.AddDays(-30)
|
||||
},
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Signed }
|
||||
};
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
var variance = Math.Abs(simpleResult.FinalScore - advancedResult.FinalScore);
|
||||
|
||||
// Variance should be reasonable (< 30 points for typical input)
|
||||
variance.Should().BeLessThan(30,
|
||||
"score variance between profiles should be reasonable for typical inputs");
|
||||
}
|
||||
|
||||
[Theory(DisplayName = "Both profiles respect policy weights")]
|
||||
[InlineData(1000, 4500, 3000, 1500)] // Default weights
|
||||
[InlineData(5000, 2500, 1500, 1000)] // High base severity weight
|
||||
[InlineData(2000, 6000, 1000, 1000)] // High reachability weight
|
||||
public async Task BothProfiles_RespectPolicyWeights(
|
||||
int baseSeverityWeight,
|
||||
int reachabilityWeight,
|
||||
int evidenceWeight,
|
||||
int provenanceWeight)
|
||||
{
|
||||
var customPolicy = ScorePolicy.Default with
|
||||
{
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = baseSeverityWeight,
|
||||
Reachability = reachabilityWeight,
|
||||
Evidence = evidenceWeight,
|
||||
Provenance = provenanceWeight
|
||||
}
|
||||
};
|
||||
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 1, asOf: asOf);
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, customPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, customPolicy);
|
||||
|
||||
// Both should produce valid results with custom weights
|
||||
simpleResult.FinalScore.Should().BeInRange(0, 100);
|
||||
advancedResult.FinalScore.Should().BeInRange(0, 100);
|
||||
|
||||
// Signal contributions should reflect weights
|
||||
simpleResult.SignalContributions.Should().NotBeEmpty();
|
||||
advancedResult.SignalContributions.Should().NotBeEmpty();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Both profiles generate explanations")]
|
||||
public async Task BothProfiles_GenerateExplanations()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 7.0m, hopCount: 2, asOf: asOf);
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
simpleResult.Explain.Should().NotBeEmpty();
|
||||
advancedResult.Explain.Should().NotBeEmpty();
|
||||
|
||||
// Both should have base severity explanation
|
||||
simpleResult.Explain.Should().Contain(e => e.Factor == "baseSeverity");
|
||||
advancedResult.Explain.Should().Contain(e => e.Factor == "baseSeverity");
|
||||
|
||||
// Both should have reachability explanation
|
||||
simpleResult.Explain.Should().Contain(e => e.Factor == "reachability");
|
||||
advancedResult.Explain.Should().Contain(e => e.Factor == "reachability");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Advanced profile applies additional factors not in Simple")]
|
||||
public async Task AdvancedProfile_AppliesAdditionalFactors()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 2, asOf: asOf) with
|
||||
{
|
||||
IsKnownExploited = true
|
||||
};
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
// Advanced should have KEV boost
|
||||
advancedResult.SignalValues.Should().ContainKey("kevBoost");
|
||||
advancedResult.SignalValues["kevBoost"].Should().BeGreaterThan(0);
|
||||
|
||||
// Simple doesn't have KEV boost in signal values (handled via override)
|
||||
simpleResult.SignalValues.Should().NotContainKey("kevBoost");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Profile results include profile identification for audit")]
|
||||
public async Task ProfileResults_IncludeProfileIdentification()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 2);
|
||||
|
||||
var simpleResult = await _simpleEngine.ScoreAsync(input, _defaultPolicy);
|
||||
var advancedResult = await _advancedEngine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
simpleResult.ProfileVersion.Should().Contain("simple");
|
||||
advancedResult.ProfileVersion.Should().Contain("advanced");
|
||||
|
||||
simpleResult.ScoringProfile.Should().Be(ScoringProfile.Simple);
|
||||
advancedResult.ScoringProfile.Should().Be(ScoringProfile.Advanced);
|
||||
}
|
||||
|
||||
private static ScoringInput CreateInput(
|
||||
decimal cvss,
|
||||
int? hopCount,
|
||||
DateTimeOffset? asOf = null)
|
||||
{
|
||||
return new ScoringInput
|
||||
{
|
||||
FindingId = $"test-finding-{Guid.NewGuid():N}",
|
||||
TenantId = "test-tenant",
|
||||
ProfileId = "test-profile",
|
||||
AsOf = asOf ?? DateTimeOffset.UtcNow,
|
||||
CvssBase = cvss,
|
||||
CvssVersion = "3.1",
|
||||
Reachability = new ReachabilityInput
|
||||
{
|
||||
HopCount = hopCount
|
||||
},
|
||||
Evidence = EvidenceInput.Empty,
|
||||
Provenance = ProvenanceInput.Default,
|
||||
IsKnownExploited = false
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,277 @@
|
||||
// =============================================================================
|
||||
// ProfileSwitchingTests.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-012 - Unit tests for profile switching
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
using StellaOps.Policy.Engine.Scoring.Engines;
|
||||
using StellaOps.Policy.Scoring;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Unit tests for profile switching functionality.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3407")]
|
||||
public sealed class ProfileSwitchingTests
|
||||
{
|
||||
private readonly Mock<IScorePolicyService> _policyServiceMock;
|
||||
private readonly Mock<IScoringProfileService> _profileServiceMock;
|
||||
private readonly IServiceProvider _serviceProvider;
|
||||
private readonly ScoringEngineFactory _factory;
|
||||
|
||||
public ProfileSwitchingTests()
|
||||
{
|
||||
_policyServiceMock = new Mock<IScorePolicyService>();
|
||||
_profileServiceMock = new Mock<IScoringProfileService>();
|
||||
|
||||
var freshnessCalculator = new EvidenceFreshnessCalculator();
|
||||
var simpleEngine = new SimpleScoringEngine(
|
||||
freshnessCalculator,
|
||||
NullLogger<SimpleScoringEngine>.Instance);
|
||||
var advancedEngine = new AdvancedScoringEngine(
|
||||
freshnessCalculator,
|
||||
NullLogger<AdvancedScoringEngine>.Instance);
|
||||
|
||||
var services = new ServiceCollection();
|
||||
services.AddSingleton(simpleEngine);
|
||||
services.AddSingleton(advancedEngine);
|
||||
_serviceProvider = services.BuildServiceProvider();
|
||||
|
||||
_factory = new ScoringEngineFactory(
|
||||
_serviceProvider,
|
||||
_profileServiceMock.Object,
|
||||
NullLogger<ScoringEngineFactory>.Instance);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetEngine returns SimpleScoringEngine for Simple profile")]
|
||||
public void GetEngine_Simple_ReturnsSimpleScoringEngine()
|
||||
{
|
||||
var engine = _factory.GetEngine(ScoringProfile.Simple);
|
||||
|
||||
engine.Should().BeOfType<SimpleScoringEngine>();
|
||||
engine.Profile.Should().Be(ScoringProfile.Simple);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetEngine returns AdvancedScoringEngine for Advanced profile")]
|
||||
public void GetEngine_Advanced_ReturnsAdvancedScoringEngine()
|
||||
{
|
||||
var engine = _factory.GetEngine(ScoringProfile.Advanced);
|
||||
|
||||
engine.Should().BeOfType<AdvancedScoringEngine>();
|
||||
engine.Profile.Should().Be(ScoringProfile.Advanced);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetEngine throws for Custom profile")]
|
||||
public void GetEngine_Custom_Throws()
|
||||
{
|
||||
var action = () => _factory.GetEngine(ScoringProfile.Custom);
|
||||
|
||||
action.Should().Throw<NotSupportedException>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetEngineForTenant uses tenant profile configuration")]
|
||||
public void GetEngineForTenant_UsesTenantProfile()
|
||||
{
|
||||
_profileServiceMock
|
||||
.Setup(p => p.GetProfileForTenant("tenant-1"))
|
||||
.Returns(ScoringProfileConfig.DefaultSimple);
|
||||
|
||||
var engine = _factory.GetEngineForTenant("tenant-1");
|
||||
|
||||
engine.Should().BeOfType<SimpleScoringEngine>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetEngineForTenant defaults to Advanced when no profile configured")]
|
||||
public void GetEngineForTenant_DefaultsToAdvanced()
|
||||
{
|
||||
_profileServiceMock
|
||||
.Setup(p => p.GetProfileForTenant("tenant-no-config"))
|
||||
.Returns((ScoringProfileConfig?)null);
|
||||
|
||||
var engine = _factory.GetEngineForTenant("tenant-no-config");
|
||||
|
||||
engine.Should().BeOfType<AdvancedScoringEngine>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetAvailableProfiles returns Simple and Advanced")]
|
||||
public void GetAvailableProfiles_ReturnsSimpleAndAdvanced()
|
||||
{
|
||||
var profiles = _factory.GetAvailableProfiles();
|
||||
|
||||
profiles.Should().Contain(ScoringProfile.Simple);
|
||||
profiles.Should().Contain(ScoringProfile.Advanced);
|
||||
profiles.Should().NotContain(ScoringProfile.Custom);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Integration tests for profile-aware scoring service.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3407")]
|
||||
public sealed class ProfileAwareScoringServiceTests
|
||||
{
|
||||
private readonly Mock<IScoringEngineFactory> _factoryMock;
|
||||
private readonly Mock<IScorePolicyService> _policyServiceMock;
|
||||
private readonly ProfileAwareScoringService _service;
|
||||
|
||||
public ProfileAwareScoringServiceTests()
|
||||
{
|
||||
_factoryMock = new Mock<IScoringEngineFactory>();
|
||||
_policyServiceMock = new Mock<IScorePolicyService>();
|
||||
_service = new ProfileAwareScoringService(
|
||||
_factoryMock.Object,
|
||||
_policyServiceMock.Object,
|
||||
NullLogger<ProfileAwareScoringService>.Instance);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync uses tenant's configured engine")]
|
||||
public async Task ScoreAsync_UsesTenantEngine()
|
||||
{
|
||||
var input = CreateInput("tenant-1");
|
||||
var policy = ScorePolicy.Default;
|
||||
var expectedResult = CreateResult(ScoringProfile.Simple);
|
||||
|
||||
var mockEngine = new Mock<IScoringEngine>();
|
||||
mockEngine.Setup(e => e.Profile).Returns(ScoringProfile.Simple);
|
||||
mockEngine
|
||||
.Setup(e => e.ScoreAsync(input, policy, It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(expectedResult);
|
||||
|
||||
_factoryMock
|
||||
.Setup(f => f.GetEngineForTenant("tenant-1"))
|
||||
.Returns(mockEngine.Object);
|
||||
_policyServiceMock
|
||||
.Setup(p => p.GetPolicy("tenant-1"))
|
||||
.Returns(policy);
|
||||
|
||||
var result = await _service.ScoreAsync(input);
|
||||
|
||||
result.Should().BeSameAs(expectedResult);
|
||||
_factoryMock.Verify(f => f.GetEngineForTenant("tenant-1"), Times.Once);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreWithProfileAsync uses specified profile")]
|
||||
public async Task ScoreWithProfileAsync_UsesSpecifiedProfile()
|
||||
{
|
||||
var input = CreateInput("tenant-1");
|
||||
var policy = ScorePolicy.Default;
|
||||
var expectedResult = CreateResult(ScoringProfile.Advanced);
|
||||
|
||||
var mockEngine = new Mock<IScoringEngine>();
|
||||
mockEngine.Setup(e => e.Profile).Returns(ScoringProfile.Advanced);
|
||||
mockEngine
|
||||
.Setup(e => e.ScoreAsync(input, policy, It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(expectedResult);
|
||||
|
||||
_factoryMock
|
||||
.Setup(f => f.GetEngine(ScoringProfile.Advanced))
|
||||
.Returns(mockEngine.Object);
|
||||
_policyServiceMock
|
||||
.Setup(p => p.GetPolicy("tenant-1"))
|
||||
.Returns(policy);
|
||||
|
||||
var result = await _service.ScoreWithProfileAsync(input, ScoringProfile.Advanced);
|
||||
|
||||
result.Should().BeSameAs(expectedResult);
|
||||
_factoryMock.Verify(f => f.GetEngine(ScoringProfile.Advanced), Times.Once);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "CompareProfilesAsync returns results for all profiles")]
|
||||
public async Task CompareProfilesAsync_ReturnsAllProfiles()
|
||||
{
|
||||
var input = CreateInput("tenant-1");
|
||||
var policy = ScorePolicy.Default;
|
||||
|
||||
var simpleResult = CreateResult(ScoringProfile.Simple, finalScore: 50);
|
||||
var advancedResult = CreateResult(ScoringProfile.Advanced, finalScore: 60);
|
||||
|
||||
var simpleEngine = new Mock<IScoringEngine>();
|
||||
simpleEngine.Setup(e => e.Profile).Returns(ScoringProfile.Simple);
|
||||
simpleEngine
|
||||
.Setup(e => e.ScoreAsync(input, policy, It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(simpleResult);
|
||||
|
||||
var advancedEngine = new Mock<IScoringEngine>();
|
||||
advancedEngine.Setup(e => e.Profile).Returns(ScoringProfile.Advanced);
|
||||
advancedEngine
|
||||
.Setup(e => e.ScoreAsync(input, policy, It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(advancedResult);
|
||||
|
||||
_factoryMock
|
||||
.Setup(f => f.GetAvailableProfiles())
|
||||
.Returns([ScoringProfile.Simple, ScoringProfile.Advanced]);
|
||||
_factoryMock
|
||||
.Setup(f => f.GetEngine(ScoringProfile.Simple))
|
||||
.Returns(simpleEngine.Object);
|
||||
_factoryMock
|
||||
.Setup(f => f.GetEngine(ScoringProfile.Advanced))
|
||||
.Returns(advancedEngine.Object);
|
||||
_policyServiceMock
|
||||
.Setup(p => p.GetPolicy("tenant-1"))
|
||||
.Returns(policy);
|
||||
|
||||
var comparison = await _service.CompareProfilesAsync(input);
|
||||
|
||||
comparison.FindingId.Should().Be("test-finding-1");
|
||||
comparison.Results.Should().HaveCount(2);
|
||||
comparison.Results.Should().ContainKey(ScoringProfile.Simple);
|
||||
comparison.Results.Should().ContainKey(ScoringProfile.Advanced);
|
||||
comparison.ScoreVariance.Should().Be(10);
|
||||
comparison.SeverityDiffers.Should().BeFalse();
|
||||
}
|
||||
|
||||
private static ScoringInput CreateInput(string tenantId)
|
||||
{
|
||||
return new ScoringInput
|
||||
{
|
||||
FindingId = "test-finding-1",
|
||||
TenantId = tenantId,
|
||||
ProfileId = "test-profile",
|
||||
AsOf = DateTimeOffset.UtcNow,
|
||||
CvssBase = 5.0m,
|
||||
CvssVersion = "3.1",
|
||||
Reachability = new ReachabilityInput { HopCount = 2 },
|
||||
Evidence = EvidenceInput.Empty,
|
||||
Provenance = ProvenanceInput.Default,
|
||||
IsKnownExploited = false
|
||||
};
|
||||
}
|
||||
|
||||
private static ScoringEngineResult CreateResult(ScoringProfile profile, int finalScore = 50)
|
||||
{
|
||||
return new ScoringEngineResult
|
||||
{
|
||||
FindingId = "test-finding-1",
|
||||
ProfileId = "test-profile",
|
||||
ProfileVersion = "v1",
|
||||
RawScore = finalScore,
|
||||
FinalScore = finalScore,
|
||||
Severity = finalScore >= 70 ? "high" : "medium",
|
||||
SignalValues = new Dictionary<string, int>
|
||||
{
|
||||
["baseSeverity"] = 50,
|
||||
["reachability"] = 70,
|
||||
["evidence"] = 30,
|
||||
["provenance"] = 30
|
||||
},
|
||||
SignalContributions = new Dictionary<string, double>
|
||||
{
|
||||
["baseSeverity"] = 5.0,
|
||||
["reachability"] = 31.5,
|
||||
["evidence"] = 9.0,
|
||||
["provenance"] = 4.5
|
||||
},
|
||||
ScoringProfile = profile,
|
||||
ScoredAt = DateTimeOffset.UtcNow,
|
||||
Explain = []
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
// =============================================================================
|
||||
// ScorePolicyDigestReplayIntegrationTests.cs
|
||||
// Sprint: SPRINT_3402_0001_0001
|
||||
// Task: YAML-3402-012 - Integration test: policy digest in replay manifest
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using StellaOps.Replay.Core;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Integration tests verifying score policy digest flows into replay manifests.
|
||||
/// </summary>
|
||||
[Trait("Category", "Integration")]
|
||||
[Trait("Sprint", "3402")]
|
||||
public sealed class ScorePolicyDigestReplayIntegrationTests
|
||||
{
|
||||
[Fact(DisplayName = "ReplayManifest includes ScorePolicyDigest field")]
|
||||
public void ReplayManifest_HasScorePolicyDigest()
|
||||
{
|
||||
var manifest = new ReplayManifest
|
||||
{
|
||||
SchemaVersion = ReplayManifestVersions.V2,
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "scan-123",
|
||||
Time = DateTimeOffset.UtcNow,
|
||||
ScorePolicyDigest = "sha256:abc123def456"
|
||||
}
|
||||
};
|
||||
|
||||
manifest.Scan.ScorePolicyDigest.Should().Be("sha256:abc123def456");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScorePolicyDigest is null when not set")]
|
||||
public void ScorePolicyDigest_IsNull_WhenNotSet()
|
||||
{
|
||||
var manifest = new ReplayManifest
|
||||
{
|
||||
SchemaVersion = ReplayManifestVersions.V2,
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "scan-123",
|
||||
Time = DateTimeOffset.UtcNow
|
||||
}
|
||||
};
|
||||
|
||||
manifest.Scan.ScorePolicyDigest.Should().BeNull();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScorePolicyDigest serializes correctly to JSON")]
|
||||
public void ScorePolicyDigest_SerializesToJson()
|
||||
{
|
||||
var manifest = new ReplayManifest
|
||||
{
|
||||
SchemaVersion = ReplayManifestVersions.V2,
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "scan-123",
|
||||
Time = DateTimeOffset.UtcNow,
|
||||
ScorePolicyDigest = "sha256:abc123def456"
|
||||
}
|
||||
};
|
||||
|
||||
var json = System.Text.Json.JsonSerializer.Serialize(manifest);
|
||||
|
||||
json.Should().Contain("\"scorePolicyDigest\":\"sha256:abc123def456\"");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScorePolicyDigest is omitted from JSON when null")]
|
||||
public void ScorePolicyDigest_OmittedFromJson_WhenNull()
|
||||
{
|
||||
var manifest = new ReplayManifest
|
||||
{
|
||||
SchemaVersion = ReplayManifestVersions.V2,
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "scan-123",
|
||||
Time = DateTimeOffset.UtcNow,
|
||||
ScorePolicyDigest = null
|
||||
}
|
||||
};
|
||||
|
||||
var json = System.Text.Json.JsonSerializer.Serialize(manifest);
|
||||
|
||||
json.Should().NotContain("scorePolicyDigest");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScorePolicyDigest roundtrips through JSON serialization")]
|
||||
public void ScorePolicyDigest_Roundtrips()
|
||||
{
|
||||
var original = new ReplayManifest
|
||||
{
|
||||
SchemaVersion = ReplayManifestVersions.V2,
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "scan-456",
|
||||
Time = DateTimeOffset.UtcNow,
|
||||
PolicyDigest = "sha256:policy-digest",
|
||||
ScorePolicyDigest = "sha256:score-policy-digest"
|
||||
}
|
||||
};
|
||||
|
||||
var json = System.Text.Json.JsonSerializer.Serialize(original);
|
||||
var deserialized = System.Text.Json.JsonSerializer.Deserialize<ReplayManifest>(json);
|
||||
|
||||
deserialized.Should().NotBeNull();
|
||||
deserialized!.Scan.ScorePolicyDigest.Should().Be("sha256:score-policy-digest");
|
||||
deserialized.Scan.PolicyDigest.Should().Be("sha256:policy-digest");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScorePolicyDigest is separate from PolicyDigest")]
|
||||
public void ScorePolicyDigest_IsSeparateFromPolicyDigest()
|
||||
{
|
||||
var manifest = new ReplayManifest
|
||||
{
|
||||
SchemaVersion = ReplayManifestVersions.V2,
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "scan-789",
|
||||
PolicyDigest = "sha256:gate-policy",
|
||||
ScorePolicyDigest = "sha256:scoring-policy"
|
||||
}
|
||||
};
|
||||
|
||||
manifest.Scan.PolicyDigest.Should().NotBe(manifest.Scan.ScorePolicyDigest);
|
||||
manifest.Scan.PolicyDigest.Should().Be("sha256:gate-policy");
|
||||
manifest.Scan.ScorePolicyDigest.Should().Be("sha256:scoring-policy");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScorePolicyDigest format is content-addressed")]
|
||||
public void ScorePolicyDigest_HasContentAddressedFormat()
|
||||
{
|
||||
var validDigests = new[]
|
||||
{
|
||||
"sha256:a".PadRight(71, 'a'),
|
||||
"sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
|
||||
};
|
||||
|
||||
foreach (var digest in validDigests)
|
||||
{
|
||||
var manifest = new ReplayManifest
|
||||
{
|
||||
Scan = new ReplayScanMetadata
|
||||
{
|
||||
Id = "test",
|
||||
ScorePolicyDigest = digest
|
||||
}
|
||||
};
|
||||
|
||||
manifest.Scan.ScorePolicyDigest.Should().StartWith("sha256:");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
// =============================================================================
|
||||
// ScorePolicyServiceCachingTests.cs
|
||||
// Sprint: SPRINT_3402_0001_0001
|
||||
// Task: YAML-3402-011 - Unit tests for policy service caching
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
using StellaOps.Policy.Scoring;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Tests for ScorePolicyService caching behavior.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3402")]
|
||||
public sealed class ScorePolicyServiceCachingTests
|
||||
{
|
||||
private readonly Mock<IScorePolicyProvider> _providerMock;
|
||||
private readonly ScorePolicyService _service;
|
||||
|
||||
public ScorePolicyServiceCachingTests()
|
||||
{
|
||||
_providerMock = new Mock<IScorePolicyProvider>();
|
||||
_service = new ScorePolicyService(
|
||||
_providerMock.Object,
|
||||
NullLogger<ScorePolicyService>.Instance);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetPolicy returns cached policy on second call")]
|
||||
public void GetPolicy_ReturnsCached()
|
||||
{
|
||||
var policy = CreateTestPolicy("tenant-1");
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-1")).Returns(policy);
|
||||
|
||||
var first = _service.GetPolicy("tenant-1");
|
||||
var second = _service.GetPolicy("tenant-1");
|
||||
|
||||
first.Should().BeSameAs(second);
|
||||
_providerMock.Verify(p => p.GetPolicy("tenant-1"), Times.Once());
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetPolicy caches per tenant")]
|
||||
public void GetPolicy_CachesPerTenant()
|
||||
{
|
||||
var policy1 = CreateTestPolicy("tenant-1");
|
||||
var policy2 = CreateTestPolicy("tenant-2");
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-1")).Returns(policy1);
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-2")).Returns(policy2);
|
||||
|
||||
var result1 = _service.GetPolicy("tenant-1");
|
||||
var result2 = _service.GetPolicy("tenant-2");
|
||||
|
||||
result1.Should().NotBeSameAs(result2);
|
||||
result1.PolicyId.Should().Be("tenant-1");
|
||||
result2.PolicyId.Should().Be("tenant-2");
|
||||
_providerMock.Verify(p => p.GetPolicy("tenant-1"), Times.Once());
|
||||
_providerMock.Verify(p => p.GetPolicy("tenant-2"), Times.Once());
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetCachedDigest returns null before policy is loaded")]
|
||||
public void GetCachedDigest_BeforeLoad_ReturnsNull()
|
||||
{
|
||||
var digest = _service.GetCachedDigest("tenant-1");
|
||||
digest.Should().BeNull();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetCachedDigest returns digest after policy is loaded")]
|
||||
public void GetCachedDigest_AfterLoad_ReturnsDigest()
|
||||
{
|
||||
var policy = CreateTestPolicy("tenant-1");
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-1")).Returns(policy);
|
||||
|
||||
_ = _service.GetPolicy("tenant-1");
|
||||
var digest = _service.GetCachedDigest("tenant-1");
|
||||
|
||||
digest.Should().NotBeNullOrEmpty();
|
||||
digest.Should().StartWith("sha256:");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ComputePolicyDigest is deterministic")]
|
||||
public void ComputePolicyDigest_IsDeterministic()
|
||||
{
|
||||
var policy = CreateTestPolicy("test");
|
||||
|
||||
var digest1 = _service.ComputePolicyDigest(policy);
|
||||
var digest2 = _service.ComputePolicyDigest(policy);
|
||||
|
||||
digest1.Should().Be(digest2);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ComputePolicyDigest differs for different policies")]
|
||||
public void ComputePolicyDigest_DiffersForDifferentPolicies()
|
||||
{
|
||||
var policy1 = CreateTestPolicy("policy-1");
|
||||
var policy2 = CreateTestPolicy("policy-2");
|
||||
|
||||
var digest1 = _service.ComputePolicyDigest(policy1);
|
||||
var digest2 = _service.ComputePolicyDigest(policy2);
|
||||
|
||||
digest1.Should().NotBe(digest2);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ComputePolicyDigest has correct format")]
|
||||
public void ComputePolicyDigest_HasCorrectFormat()
|
||||
{
|
||||
var policy = CreateTestPolicy("test");
|
||||
|
||||
var digest = _service.ComputePolicyDigest(policy);
|
||||
|
||||
digest.Should().MatchRegex(@"^sha256:[a-f0-9]{64}$");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Reload clears cache")]
|
||||
public void Reload_ClearsCache()
|
||||
{
|
||||
var policy = CreateTestPolicy("tenant-1");
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-1")).Returns(policy);
|
||||
|
||||
_ = _service.GetPolicy("tenant-1");
|
||||
_service.GetCachedDigest("tenant-1").Should().NotBeNull();
|
||||
|
||||
_service.Reload();
|
||||
|
||||
_service.GetCachedDigest("tenant-1").Should().BeNull();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Reload causes provider to be called again")]
|
||||
public void Reload_CausesProviderToBeCalled()
|
||||
{
|
||||
var policy = CreateTestPolicy("tenant-1");
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-1")).Returns(policy);
|
||||
|
||||
_ = _service.GetPolicy("tenant-1");
|
||||
_service.Reload();
|
||||
_ = _service.GetPolicy("tenant-1");
|
||||
|
||||
_providerMock.Verify(p => p.GetPolicy("tenant-1"), Times.Exactly(2));
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetPolicy with null tenant throws")]
|
||||
public void GetPolicy_NullTenant_Throws()
|
||||
{
|
||||
var act = () => _service.GetPolicy(null!);
|
||||
act.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "GetPolicy with empty tenant throws")]
|
||||
public void GetPolicy_EmptyTenant_Throws()
|
||||
{
|
||||
var act = () => _service.GetPolicy("");
|
||||
act.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ComputePolicyDigest with null policy throws")]
|
||||
public void ComputePolicyDigest_NullPolicy_Throws()
|
||||
{
|
||||
var act = () => _service.ComputePolicyDigest(null!);
|
||||
act.Should().Throw<ArgumentNullException>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Concurrent access is thread-safe")]
|
||||
public void ConcurrentAccess_IsThreadSafe()
|
||||
{
|
||||
var policy = CreateTestPolicy("tenant-1");
|
||||
var callCount = 0;
|
||||
_providerMock.Setup(p => p.GetPolicy("tenant-1"))
|
||||
.Returns(() =>
|
||||
{
|
||||
Interlocked.Increment(ref callCount);
|
||||
Thread.Sleep(10); // Simulate slow load
|
||||
return policy;
|
||||
});
|
||||
|
||||
var tasks = Enumerable.Range(0, 100)
|
||||
.Select(_ => Task.Run(() => _service.GetPolicy("tenant-1")))
|
||||
.ToArray();
|
||||
|
||||
Task.WaitAll(tasks);
|
||||
|
||||
// ConcurrentDictionary's GetOrAdd may call factory multiple times
|
||||
// but should converge to same cached value
|
||||
var results = tasks.Select(t => t.Result).Distinct().ToList();
|
||||
results.Should().HaveCount(1);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Digest is stable across equal policies created separately")]
|
||||
public void Digest_IsStable_AcrossEqualPolicies()
|
||||
{
|
||||
var policy1 = new ScorePolicy
|
||||
{
|
||||
PolicyVersion = "score.v1",
|
||||
PolicyId = "stable-test",
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = 2500,
|
||||
Reachability = 2500,
|
||||
Evidence = 2500,
|
||||
Provenance = 2500
|
||||
}
|
||||
};
|
||||
|
||||
var policy2 = new ScorePolicy
|
||||
{
|
||||
PolicyVersion = "score.v1",
|
||||
PolicyId = "stable-test",
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = 2500,
|
||||
Reachability = 2500,
|
||||
Evidence = 2500,
|
||||
Provenance = 2500
|
||||
}
|
||||
};
|
||||
|
||||
var digest1 = _service.ComputePolicyDigest(policy1);
|
||||
var digest2 = _service.ComputePolicyDigest(policy2);
|
||||
|
||||
digest1.Should().Be(digest2);
|
||||
}
|
||||
|
||||
private static ScorePolicy CreateTestPolicy(string id) => new()
|
||||
{
|
||||
PolicyVersion = "score.v1",
|
||||
PolicyId = id,
|
||||
PolicyName = $"Test Policy {id}",
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = 2500,
|
||||
Reachability = 2500,
|
||||
Evidence = 2500,
|
||||
Provenance = 2500
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,344 @@
|
||||
// =============================================================================
|
||||
// SimpleScoringEngineTests.cs
|
||||
// Sprint: SPRINT_3407_0001_0001_configurable_scoring
|
||||
// Task: PROF-3407-010 - Unit tests for SimpleScoringEngine
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using StellaOps.Policy.Engine.Scoring.Engines;
|
||||
using StellaOps.Policy.Scoring;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Engine.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Unit tests for SimpleScoringEngine.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3407")]
|
||||
public sealed class SimpleScoringEngineTests
|
||||
{
|
||||
private readonly SimpleScoringEngine _engine;
|
||||
private readonly EvidenceFreshnessCalculator _freshnessCalculator;
|
||||
private readonly ScorePolicy _defaultPolicy;
|
||||
|
||||
public SimpleScoringEngineTests()
|
||||
{
|
||||
_freshnessCalculator = new EvidenceFreshnessCalculator();
|
||||
_engine = new SimpleScoringEngine(
|
||||
_freshnessCalculator,
|
||||
NullLogger<SimpleScoringEngine>.Instance);
|
||||
_defaultPolicy = ScorePolicy.Default;
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Profile returns Simple")]
|
||||
public void Profile_ReturnsSimple()
|
||||
{
|
||||
_engine.Profile.Should().Be(ScoringProfile.Simple);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with max CVSS returns high base severity")]
|
||||
public async Task ScoreAsync_MaxCvss_HighBaseSeverity()
|
||||
{
|
||||
var input = CreateInput(cvss: 10.0m, hopCount: 0);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.Should().NotBeNull();
|
||||
result.SignalValues["baseSeverity"].Should().Be(100);
|
||||
result.ScoringProfile.Should().Be(ScoringProfile.Simple);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with min CVSS returns low base severity")]
|
||||
public async Task ScoreAsync_MinCvss_LowBaseSeverity()
|
||||
{
|
||||
var input = CreateInput(cvss: 0.0m, hopCount: 0);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["baseSeverity"].Should().Be(0);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with direct call returns max reachability")]
|
||||
public async Task ScoreAsync_DirectCall_MaxReachability()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["reachability"].Should().Be(100);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with multiple hops reduces reachability")]
|
||||
public async Task ScoreAsync_MultipleHops_ReducedReachability()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 5);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["reachability"].Should().BeLessThan(100);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with unreachable returns zero reachability")]
|
||||
public async Task ScoreAsync_Unreachable_ZeroReachability()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: null);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["reachability"].Should().Be(0);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with gates applies gate multiplier")]
|
||||
public async Task ScoreAsync_WithGates_AppliesMultiplier()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
input = input with
|
||||
{
|
||||
Reachability = input.Reachability with
|
||||
{
|
||||
Gates =
|
||||
[
|
||||
new DetectedGate("auth_required", "JWT validation", 0.9)
|
||||
]
|
||||
}
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
// Gate should reduce reachability
|
||||
result.SignalValues["reachability"].Should().BeLessThan(100);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with runtime evidence gives high evidence score")]
|
||||
public async Task ScoreAsync_RuntimeEvidence_HighEvidenceScore()
|
||||
{
|
||||
var asOf = DateTimeOffset.UtcNow;
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 0, asOf: asOf);
|
||||
input = input with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = asOf.AddDays(-1)
|
||||
}
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["evidence"].Should().BeGreaterThan(0);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with stale evidence applies freshness decay")]
|
||||
public async Task ScoreAsync_StaleEvidence_FreshnessDecay()
|
||||
{
|
||||
var asOf = DateTimeOffset.UtcNow;
|
||||
var freshInput = CreateInput(cvss: 5.0m, hopCount: 0, asOf: asOf);
|
||||
freshInput = freshInput with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = asOf.AddDays(-1)
|
||||
}
|
||||
};
|
||||
|
||||
var staleInput = CreateInput(cvss: 5.0m, hopCount: 0, asOf: asOf);
|
||||
staleInput = staleInput with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = asOf.AddDays(-180)
|
||||
}
|
||||
};
|
||||
|
||||
var freshResult = await _engine.ScoreAsync(freshInput, _defaultPolicy);
|
||||
var staleResult = await _engine.ScoreAsync(staleInput, _defaultPolicy);
|
||||
|
||||
staleResult.SignalValues["evidence"].Should().BeLessThan(freshResult.SignalValues["evidence"]);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with signed provenance increases provenance score")]
|
||||
public async Task ScoreAsync_SignedProvenance_IncreasesScore()
|
||||
{
|
||||
var unsignedInput = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
var signedInput = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
signedInput = signedInput with
|
||||
{
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Signed }
|
||||
};
|
||||
|
||||
var unsignedResult = await _engine.ScoreAsync(unsignedInput, _defaultPolicy);
|
||||
var signedResult = await _engine.ScoreAsync(signedInput, _defaultPolicy);
|
||||
|
||||
signedResult.SignalValues["provenance"].Should().BeGreaterThan(unsignedResult.SignalValues["provenance"]);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with reproducible provenance gives max provenance score")]
|
||||
public async Task ScoreAsync_ReproducibleProvenance_MaxScore()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 0);
|
||||
input = input with
|
||||
{
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Reproducible }
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.SignalValues["provenance"].Should().Be(100);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync applies weights correctly")]
|
||||
public async Task ScoreAsync_AppliesWeightsCorrectly()
|
||||
{
|
||||
var asOf = DateTimeOffset.UtcNow;
|
||||
var input = CreateInput(cvss: 10.0m, hopCount: 0, asOf: asOf);
|
||||
input = input with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = asOf
|
||||
},
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Reproducible }
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
// All factors maxed: should be close to 100
|
||||
result.FinalScore.Should().BeGreaterThan(90);
|
||||
result.SignalContributions.Values.Sum().Should().BeApproximately(result.RawScore, 1.0);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync maps score to correct severity")]
|
||||
public async Task ScoreAsync_MapsToCorrectSeverity()
|
||||
{
|
||||
var criticalInput = CreateInput(cvss: 10.0m, hopCount: 0);
|
||||
criticalInput = criticalInput with
|
||||
{
|
||||
Evidence = new EvidenceInput
|
||||
{
|
||||
Types = new HashSet<EvidenceType> { EvidenceType.Runtime },
|
||||
NewestEvidenceAt = DateTimeOffset.UtcNow
|
||||
},
|
||||
Provenance = new ProvenanceInput { Level = ProvenanceLevel.Reproducible }
|
||||
};
|
||||
|
||||
var infoInput = CreateInput(cvss: 1.0m, hopCount: null);
|
||||
|
||||
var criticalResult = await _engine.ScoreAsync(criticalInput, _defaultPolicy);
|
||||
var infoResult = await _engine.ScoreAsync(infoInput, _defaultPolicy);
|
||||
|
||||
criticalResult.Severity.Should().Be("critical");
|
||||
infoResult.Severity.Should().Be("info");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync generates explain entries")]
|
||||
public async Task ScoreAsync_GeneratesExplainEntries()
|
||||
{
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 3);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result.Explain.Should().NotBeEmpty();
|
||||
result.Explain.Should().Contain(e => e.Factor == "baseSeverity");
|
||||
result.Explain.Should().Contain(e => e.Factor == "reachability");
|
||||
result.Explain.Should().Contain(e => e.Factor == "provenance");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync is deterministic")]
|
||||
public async Task ScoreAsync_IsDeterministic()
|
||||
{
|
||||
var asOf = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
var input = CreateInput(cvss: 7.5m, hopCount: 2, asOf: asOf);
|
||||
|
||||
var result1 = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
var result2 = await _engine.ScoreAsync(input, _defaultPolicy);
|
||||
|
||||
result1.RawScore.Should().Be(result2.RawScore);
|
||||
result1.FinalScore.Should().Be(result2.FinalScore);
|
||||
result1.Severity.Should().Be(result2.Severity);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with override applies set score")]
|
||||
public async Task ScoreAsync_WithOverride_AppliesSetScore()
|
||||
{
|
||||
var policy = _defaultPolicy with
|
||||
{
|
||||
Overrides =
|
||||
[
|
||||
new ScoreOverride
|
||||
{
|
||||
Name = "kev_boost",
|
||||
When = new ScoreOverrideCondition
|
||||
{
|
||||
Flags = new Dictionary<string, bool> { ["knownExploited"] = true }
|
||||
},
|
||||
SetScore = 95
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
var input = CreateInput(cvss: 5.0m, hopCount: 5) with
|
||||
{
|
||||
IsKnownExploited = true
|
||||
};
|
||||
|
||||
var result = await _engine.ScoreAsync(input, policy);
|
||||
|
||||
result.FinalScore.Should().Be(95);
|
||||
result.OverrideApplied.Should().Be("kev_boost");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ScoreAsync with override applies clamp")]
|
||||
public async Task ScoreAsync_WithOverride_AppliesClamp()
|
||||
{
|
||||
var policy = _defaultPolicy with
|
||||
{
|
||||
Overrides =
|
||||
[
|
||||
new ScoreOverride
|
||||
{
|
||||
Name = "max_unreachable",
|
||||
When = new ScoreOverrideCondition
|
||||
{
|
||||
MaxReachability = 0
|
||||
},
|
||||
ClampMaxScore = 30
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
var input = CreateInput(cvss: 10.0m, hopCount: null);
|
||||
|
||||
var result = await _engine.ScoreAsync(input, policy);
|
||||
|
||||
result.FinalScore.Should().BeLessOrEqualTo(30);
|
||||
result.OverrideApplied.Should().Contain("max_unreachable");
|
||||
}
|
||||
|
||||
private static ScoringInput CreateInput(
|
||||
decimal cvss,
|
||||
int? hopCount,
|
||||
DateTimeOffset? asOf = null)
|
||||
{
|
||||
return new ScoringInput
|
||||
{
|
||||
FindingId = "test-finding-1",
|
||||
TenantId = "test-tenant",
|
||||
ProfileId = "test-profile",
|
||||
AsOf = asOf ?? DateTimeOffset.UtcNow,
|
||||
CvssBase = cvss,
|
||||
CvssVersion = "3.1",
|
||||
Reachability = new ReachabilityInput
|
||||
{
|
||||
HopCount = hopCount
|
||||
},
|
||||
Evidence = EvidenceInput.Empty,
|
||||
Provenance = ProvenanceInput.Default,
|
||||
IsKnownExploited = false
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,277 @@
|
||||
// =============================================================================
|
||||
// ScorePolicyLoaderEdgeCaseTests.cs
|
||||
// Sprint: SPRINT_3402_0001_0001
|
||||
// Task: YAML-3402-009 - Unit tests for YAML parsing edge cases
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Tests for YAML parsing edge cases in ScorePolicyLoader.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3402")]
|
||||
public sealed class ScorePolicyLoaderEdgeCaseTests
|
||||
{
|
||||
private readonly ScorePolicyLoader _loader = new();
|
||||
|
||||
[Fact(DisplayName = "Empty YAML throws ScorePolicyLoadException")]
|
||||
public void EmptyYaml_Throws()
|
||||
{
|
||||
var act = () => _loader.LoadFromYaml("");
|
||||
act.Should().Throw<ScorePolicyLoadException>()
|
||||
.WithMessage("*Empty YAML content*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Whitespace-only YAML throws ScorePolicyLoadException")]
|
||||
public void WhitespaceOnlyYaml_Throws()
|
||||
{
|
||||
var act = () => _loader.LoadFromYaml(" \n \t ");
|
||||
act.Should().Throw<ScorePolicyLoadException>()
|
||||
.WithMessage("*Empty YAML content*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Null path throws ArgumentException")]
|
||||
public void NullPath_Throws()
|
||||
{
|
||||
var act = () => _loader.LoadFromFile(null!);
|
||||
act.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Empty path throws ArgumentException")]
|
||||
public void EmptyPath_Throws()
|
||||
{
|
||||
var act = () => _loader.LoadFromFile("");
|
||||
act.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Non-existent file throws ScorePolicyLoadException")]
|
||||
public void NonExistentFile_Throws()
|
||||
{
|
||||
var act = () => _loader.LoadFromFile("/nonexistent/path/score.yaml");
|
||||
act.Should().Throw<ScorePolicyLoadException>()
|
||||
.WithMessage("*not found*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Invalid YAML syntax throws ScorePolicyLoadException")]
|
||||
public void InvalidYamlSyntax_Throws()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: test
|
||||
weightsBps:
|
||||
baseSeverity: 2500
|
||||
- invalid nested list
|
||||
""";
|
||||
|
||||
var act = () => _loader.LoadFromYaml(yaml);
|
||||
act.Should().Throw<ScorePolicyLoadException>()
|
||||
.WithMessage("*YAML parse error*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Unsupported policy version throws ScorePolicyLoadException")]
|
||||
public void UnsupportedPolicyVersion_Throws()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v2
|
||||
policyId: test
|
||||
weightsBps:
|
||||
baseSeverity: 2500
|
||||
reachability: 2500
|
||||
evidence: 2500
|
||||
provenance: 2500
|
||||
""";
|
||||
|
||||
var act = () => _loader.LoadFromYaml(yaml);
|
||||
act.Should().Throw<ScorePolicyLoadException>()
|
||||
.WithMessage("*Unsupported policy version 'score.v2'*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Weights not summing to 10000 throws ScorePolicyLoadException")]
|
||||
public void WeightsSumNot10000_Throws()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: test
|
||||
weightsBps:
|
||||
baseSeverity: 5000
|
||||
reachability: 2500
|
||||
evidence: 2500
|
||||
provenance: 1000
|
||||
""";
|
||||
|
||||
var act = () => _loader.LoadFromYaml(yaml);
|
||||
act.Should().Throw<ScorePolicyLoadException>()
|
||||
.WithMessage("*Weight basis points must sum to 10000*Got: 11000*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Valid minimal policy parses successfully")]
|
||||
public void ValidMinimalPolicy_Parses()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: minimal-test
|
||||
weightsBps:
|
||||
baseSeverity: 2500
|
||||
reachability: 2500
|
||||
evidence: 2500
|
||||
provenance: 2500
|
||||
""";
|
||||
|
||||
var policy = _loader.LoadFromYaml(yaml);
|
||||
|
||||
policy.Should().NotBeNull();
|
||||
policy.PolicyVersion.Should().Be("score.v1");
|
||||
policy.PolicyId.Should().Be("minimal-test");
|
||||
policy.WeightsBps.BaseSeverity.Should().Be(2500);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with optional fields parses successfully")]
|
||||
public void PolicyWithOptionalFields_Parses()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: full-test
|
||||
policyName: Full Test Policy
|
||||
description: A comprehensive test policy
|
||||
weightsBps:
|
||||
baseSeverity: 3000
|
||||
reachability: 3000
|
||||
evidence: 2000
|
||||
provenance: 2000
|
||||
reachabilityConfig:
|
||||
reachableMultiplier: 1.5
|
||||
unreachableMultiplier: 0.5
|
||||
unknownMultiplier: 1.0
|
||||
evidenceConfig:
|
||||
kevWeight: 1.2
|
||||
epssThreshold: 0.5
|
||||
epssWeight: 0.8
|
||||
provenanceConfig:
|
||||
signedBonus: 0.1
|
||||
rekorVerifiedBonus: 0.2
|
||||
unsignedPenalty: -0.1
|
||||
""";
|
||||
|
||||
var policy = _loader.LoadFromYaml(yaml);
|
||||
|
||||
policy.Should().NotBeNull();
|
||||
policy.PolicyName.Should().Be("Full Test Policy");
|
||||
policy.Description.Should().Be("A comprehensive test policy");
|
||||
policy.ReachabilityConfig.Should().NotBeNull();
|
||||
policy.ReachabilityConfig!.ReachableMultiplier.Should().Be(1.5m);
|
||||
policy.EvidenceConfig.Should().NotBeNull();
|
||||
policy.EvidenceConfig!.KevWeight.Should().Be(1.2m);
|
||||
policy.ProvenanceConfig.Should().NotBeNull();
|
||||
policy.ProvenanceConfig!.SignedBonus.Should().Be(0.1m);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with overrides parses correctly")]
|
||||
public void PolicyWithOverrides_Parses()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: override-test
|
||||
weightsBps:
|
||||
baseSeverity: 2500
|
||||
reachability: 2500
|
||||
evidence: 2500
|
||||
provenance: 2500
|
||||
overrides:
|
||||
- id: cve-log4j
|
||||
match:
|
||||
cvePattern: "CVE-2021-44228"
|
||||
action:
|
||||
setScore: 10.0
|
||||
reason: Known critical vulnerability
|
||||
- id: low-severity-suppress
|
||||
match:
|
||||
severityEquals: LOW
|
||||
action:
|
||||
multiplyScore: 0.5
|
||||
""";
|
||||
|
||||
var policy = _loader.LoadFromYaml(yaml);
|
||||
|
||||
policy.Should().NotBeNull();
|
||||
policy.Overrides.Should().HaveCount(2);
|
||||
policy.Overrides![0].Id.Should().Be("cve-log4j");
|
||||
policy.Overrides[0].Match!.CvePattern.Should().Be("CVE-2021-44228");
|
||||
policy.Overrides[0].Action!.SetScore.Should().Be(10.0m);
|
||||
policy.Overrides[1].Id.Should().Be("low-severity-suppress");
|
||||
policy.Overrides[1].Action!.MultiplyScore.Should().Be(0.5m);
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "TryLoadFromFile returns null for non-existent file")]
|
||||
public void TryLoadFromFile_NonExistent_ReturnsNull()
|
||||
{
|
||||
var result = _loader.TryLoadFromFile("/nonexistent/path/score.yaml");
|
||||
result.Should().BeNull();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Extra YAML fields are ignored")]
|
||||
public void ExtraYamlFields_Ignored()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: extra-fields-test
|
||||
unknownField: should be ignored
|
||||
anotherUnknown:
|
||||
nested: value
|
||||
weightsBps:
|
||||
baseSeverity: 2500
|
||||
reachability: 2500
|
||||
evidence: 2500
|
||||
provenance: 2500
|
||||
extraWeight: 1000
|
||||
""";
|
||||
|
||||
// Should not throw despite extra fields
|
||||
var policy = _loader.LoadFromYaml(yaml);
|
||||
policy.Should().NotBeNull();
|
||||
policy.PolicyId.Should().Be("extra-fields-test");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Unicode in policy name and description is preserved")]
|
||||
public void UnicodePreserved()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: unicode-test
|
||||
policyName: "Política de Segurança 安全策略"
|
||||
description: "Deutsche Sicherheitsrichtlinie für контейнеры"
|
||||
weightsBps:
|
||||
baseSeverity: 2500
|
||||
reachability: 2500
|
||||
evidence: 2500
|
||||
provenance: 2500
|
||||
""";
|
||||
|
||||
var policy = _loader.LoadFromYaml(yaml);
|
||||
|
||||
policy.PolicyName.Should().Be("Política de Segurança 安全策略");
|
||||
policy.Description.Should().Contain("контейнеры");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Boundary weight values (0 and 10000) are valid")]
|
||||
public void BoundaryWeightValues_Valid()
|
||||
{
|
||||
var yaml = """
|
||||
policyVersion: score.v1
|
||||
policyId: boundary-test
|
||||
weightsBps:
|
||||
baseSeverity: 10000
|
||||
reachability: 0
|
||||
evidence: 0
|
||||
provenance: 0
|
||||
""";
|
||||
|
||||
var policy = _loader.LoadFromYaml(yaml);
|
||||
|
||||
policy.WeightsBps.BaseSeverity.Should().Be(10000);
|
||||
policy.WeightsBps.Reachability.Should().Be(0);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,298 @@
|
||||
// =============================================================================
|
||||
// ScorePolicyValidatorTests.cs
|
||||
// Sprint: SPRINT_3402_0001_0001
|
||||
// Task: YAML-3402-010 - Unit tests for schema validation
|
||||
// =============================================================================
|
||||
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Policy.Scoring.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Tests for JSON Schema validation in ScorePolicyValidator.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
[Trait("Sprint", "3402")]
|
||||
public sealed class ScorePolicyValidatorTests
|
||||
{
|
||||
private readonly ScorePolicyValidator _validator = new();
|
||||
|
||||
[Fact(DisplayName = "Valid policy passes validation")]
|
||||
public void ValidPolicy_Passes()
|
||||
{
|
||||
var policy = CreateValidPolicy();
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeTrue();
|
||||
result.Errors.Should().BeEmpty();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with wrong version fails validation")]
|
||||
public void WrongVersion_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with { PolicyVersion = "score.v2" };
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
result.Errors.Should().NotBeEmpty();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with missing policyId fails validation")]
|
||||
public void MissingPolicyId_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with { PolicyId = "" };
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with negative weight fails validation")]
|
||||
public void NegativeWeight_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = -100,
|
||||
Reachability = 2500,
|
||||
Evidence = 2500,
|
||||
Provenance = 5100
|
||||
}
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
result.Errors.Should().Contain(e => e.Contains("baseSeverity") || e.Contains("minimum"));
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with weight over 10000 fails validation")]
|
||||
public void WeightOver10000_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = 15000,
|
||||
Reachability = 0,
|
||||
Evidence = 0,
|
||||
Provenance = 0
|
||||
}
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with valid reachability config passes")]
|
||||
public void ValidReachabilityConfig_Passes()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
ReachabilityConfig = new ReachabilityConfig
|
||||
{
|
||||
ReachableMultiplier = 1.5m,
|
||||
UnreachableMultiplier = 0.5m,
|
||||
UnknownMultiplier = 1.0m
|
||||
}
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with reachable multiplier over 2 fails")]
|
||||
public void ReachableMultiplierOver2_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
ReachabilityConfig = new ReachabilityConfig
|
||||
{
|
||||
ReachableMultiplier = 3.0m,
|
||||
UnreachableMultiplier = 0.5m,
|
||||
UnknownMultiplier = 1.0m
|
||||
}
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with valid evidence config passes")]
|
||||
public void ValidEvidenceConfig_Passes()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
EvidenceConfig = new EvidenceConfig
|
||||
{
|
||||
KevWeight = 1.5m,
|
||||
EpssThreshold = 0.5m,
|
||||
EpssWeight = 1.0m
|
||||
}
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with EPSS threshold over 1 fails")]
|
||||
public void EpssThresholdOver1_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
EvidenceConfig = new EvidenceConfig
|
||||
{
|
||||
KevWeight = 1.0m,
|
||||
EpssThreshold = 1.5m,
|
||||
EpssWeight = 1.0m
|
||||
}
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Policy with valid override passes")]
|
||||
public void ValidOverride_Passes()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
Overrides =
|
||||
[
|
||||
new ScoreOverride
|
||||
{
|
||||
Id = "test-override",
|
||||
Match = new OverrideMatch { CvePattern = "CVE-2021-.*" },
|
||||
Action = new OverrideAction { SetScore = 10.0m },
|
||||
Reason = "Test override"
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
result.IsValid.Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "Override without id fails")]
|
||||
public void OverrideWithoutId_Fails()
|
||||
{
|
||||
var policy = CreateValidPolicy() with
|
||||
{
|
||||
Overrides =
|
||||
[
|
||||
new ScoreOverride
|
||||
{
|
||||
Id = "",
|
||||
Match = new OverrideMatch { CvePattern = "CVE-2021-.*" }
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
// id is required but empty string is invalid
|
||||
result.IsValid.Should().BeFalse();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ThrowIfInvalid throws for invalid policy")]
|
||||
public void ThrowIfInvalid_Throws()
|
||||
{
|
||||
var policy = CreateValidPolicy() with { PolicyVersion = "invalid" };
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
var act = () => result.ThrowIfInvalid("test context");
|
||||
|
||||
act.Should().Throw<ScorePolicyValidationException>()
|
||||
.WithMessage("test context*");
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ThrowIfInvalid does not throw for valid policy")]
|
||||
public void ThrowIfInvalid_DoesNotThrow()
|
||||
{
|
||||
var policy = CreateValidPolicy();
|
||||
var result = _validator.Validate(policy);
|
||||
|
||||
var act = () => result.ThrowIfInvalid();
|
||||
|
||||
act.Should().NotThrow();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ValidateJson with valid JSON passes")]
|
||||
public void ValidateJson_Valid_Passes()
|
||||
{
|
||||
var json = """
|
||||
{
|
||||
"policyVersion": "score.v1",
|
||||
"policyId": "json-test",
|
||||
"weightsBps": {
|
||||
"baseSeverity": 2500,
|
||||
"reachability": 2500,
|
||||
"evidence": 2500,
|
||||
"provenance": 2500
|
||||
}
|
||||
}
|
||||
""";
|
||||
|
||||
var result = _validator.ValidateJson(json);
|
||||
|
||||
result.IsValid.Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ValidateJson with invalid JSON fails")]
|
||||
public void ValidateJson_InvalidJson_Fails()
|
||||
{
|
||||
var json = "{ invalid json }";
|
||||
|
||||
var result = _validator.ValidateJson(json);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
result.Errors.Should().Contain(e => e.Contains("Invalid JSON"));
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ValidateJson with empty string fails")]
|
||||
public void ValidateJson_Empty_Fails()
|
||||
{
|
||||
var result = _validator.ValidateJson("");
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
result.Errors.Should().Contain(e => e.Contains("empty"));
|
||||
}
|
||||
|
||||
[Fact(DisplayName = "ValidateJson with missing required fields fails")]
|
||||
public void ValidateJson_MissingRequired_Fails()
|
||||
{
|
||||
var json = """
|
||||
{
|
||||
"policyVersion": "score.v1"
|
||||
}
|
||||
""";
|
||||
|
||||
var result = _validator.ValidateJson(json);
|
||||
|
||||
result.IsValid.Should().BeFalse();
|
||||
}
|
||||
|
||||
private static ScorePolicy CreateValidPolicy() => new()
|
||||
{
|
||||
PolicyVersion = "score.v1",
|
||||
PolicyId = "test-policy",
|
||||
PolicyName = "Test Policy",
|
||||
WeightsBps = new WeightsBps
|
||||
{
|
||||
BaseSeverity = 2500,
|
||||
Reachability = 2500,
|
||||
Evidence = 2500,
|
||||
Provenance = 2500
|
||||
}
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user