stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

consolidate the tests locations

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-26 01:48:24 +02:00
parent 17613acf57
commit 39359da171
2031 changed files with 2607 additions and 476 deletions
Download Patch File Download Diff File Expand all files Collapse all files

218
src/Concelier/__Libraries/StellaOps.Concelier.Interest/README.md Normal file
View File

@@ -0,0 +1,218 @@
# StellaOps.Concelier.Interest
Interest scoring service for canonical advisories. This module learns which advisories matter to your organization by analyzing SBOM intersections, reachability data, VEX statements, and runtime signals.
## Overview
Interest scoring helps prioritize advisories by computing a relevance score (0.0 to 1.0) based on:
- **SBOM Intersection** (30%): Advisory affects packages in your SBOMs
- **Reachability** (25%): Vulnerable code is reachable from application entrypoints
- **Deployment** (20%): Affected component is deployed in production
- **VEX Status** (15%): No `not_affected` VEX statement exists
- **Recency** (10%): How recently the advisory was seen in builds (decays over 365 days)
## Key Features
### Score Tiers
| Tier | Score Range | Description |
|------|-------------|-------------|
| **High** | ≥ 0.7 | Urgent attention required |
| **Medium** | 0.4 - 0.7 | Should be reviewed |
| **Low** | 0.2 - 0.4 | Lower priority |
| **None** | < 0.2 | Can be ignored or degraded to stub |
### Stub Degradation
Low-interest advisories (score < 0.2) can be automatically degraded to lightweight stubs:
- Only essential fields retained (ID, CVE, severity, title)
- Full details discarded to save storage
- Stubs auto-restore when interest score increases above threshold (0.4)
## Usage
### Computing Scores
```csharp
// Inject the service
var scoringService = serviceProvider.GetRequiredService<IInterestScoringService>();
// Compute score for a canonical advisory
var score = await scoringService.ComputeScoreAsync(canonicalId);
// Or compute from explicit signals
var input = new InterestScoreInput
{
CanonicalId = canonicalId,
SbomMatches = [
new SbomMatch
{
SbomDigest = "sha256:...",
Purl = "pkg:npm/lodash@4.17.21",
IsReachable = true,
IsDeployed = false
}
],
VexStatements = []
};
var score = await scoringService.ComputeScoreAsync(input);
```
### Recording Signals
```csharp
// Record an SBOM match
await scoringService.RecordSbomMatchAsync(
canonicalId,
sbomDigest: "sha256:abc123",
purl: "pkg:npm/lodash@4.17.21",
isReachable: true,
isDeployed: false);
// Record a VEX statement
await scoringService.RecordVexStatementAsync(canonicalId, new VexStatement
{
StatementId = "VEX-2025-001",
Status = VexStatus.NotAffected,
Justification = "Component not used in production"
});
```
### Batch Operations
```csharp
// Update scores for specific canonicals
await scoringService.BatchUpdateAsync(canonicalIds);
// Full recalculation (all active advisories)
await scoringService.RecalculateAllAsync();
```
### Degradation/Restoration
```csharp
// Degrade low-interest advisories to stubs
int degraded = await scoringService.DegradeToStubsAsync(threshold: 0.2);
// Restore stubs when interest increases
int restored = await scoringService.RestoreFromStubsAsync(threshold: 0.4);
```
## API Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/v1/canonical/{id}/score` | GET | Get interest score for a canonical |
| `/api/v1/canonical/{id}/score/compute` | POST | Compute and update score |
| `/api/v1/scores` | GET | Query scores with filtering |
| `/api/v1/scores/distribution` | GET | Get score distribution statistics |
| `/api/v1/scores/recalculate` | POST | Trigger batch/full recalculation |
| `/api/v1/scores/degrade` | POST | Run stub degradation |
| `/api/v1/scores/restore` | POST | Run stub restoration |
### Example API Response
```json
{
"canonicalId": "550e8400-e29b-41d4-a716-446655440000",
"score": 0.75,
"tier": "High",
"reasons": ["in_sbom", "reachable", "deployed"],
"lastSeenInBuild": "b5d2c400-e29b-41d4-a716-446655440000",
"computedAt": "2025-12-26T10:30:00Z"
}
```
## Configuration
```json
{
"InterestScore": {
"EnableCache": true,
"DegradationPolicy": {
"Enabled": true,
"DegradationThreshold": 0.2,
"RestorationThreshold": 0.4,
"MinAgeDays": 30,
"BatchSize": 1000,
"JobInterval": "06:00:00"
},
"Job": {
"Enabled": true,
"Interval": "01:00:00",
"FullRecalculationHour": 3,
"FullRecalculationBatchSize": 1000
},
"Weights": {
"InSbom": 0.30,
"Reachable": 0.25,
"Deployed": 0.20,
"NoVexNotAffected": 0.15,
"Recent": 0.10
}
}
}
```
## Background Jobs
### InterestScoreRecalculationJob
Runs periodically to keep scores up-to-date:
- **Incremental mode** (hourly): Updates scores for recently changed advisories
- **Full mode** (nightly at 3 AM UTC): Recalculates all active advisories
### StubDegradationJob
Runs periodically (default: every 6 hours) to:
1. Degrade advisories with scores below threshold
2. Restore stubs whose scores have increased
## Metrics
| Metric | Type | Description |
|--------|------|-------------|
| `concelier_interest_score_computed_total` | Counter | Total scores computed |
| `concelier_interest_score_distribution` | Histogram | Score value distribution |
| `concelier_stub_degradations_total` | Counter | Total stub degradations |
| `concelier_stub_restorations_total` | Counter | Total stub restorations |
| `concelier_scoring_job_duration_seconds` | Histogram | Job execution time |
| `concelier_scoring_job_errors_total` | Counter | Job execution errors |
## Database Schema
```sql
CREATE TABLE vuln.interest_score (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
canonical_id UUID NOT NULL REFERENCES vuln.advisory_canonical(id),
score NUMERIC(3,2) NOT NULL CHECK (score >= 0 AND score <= 1),
reasons JSONB NOT NULL DEFAULT '[]',
last_seen_in_build UUID,
computed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT uq_interest_score_canonical UNIQUE (canonical_id)
);
-- Indexes for common queries
CREATE INDEX idx_interest_score_score ON vuln.interest_score(score DESC);
CREATE INDEX idx_interest_score_computed ON vuln.interest_score(computed_at DESC);
-- Partial indexes for degradation queries
CREATE INDEX idx_interest_score_high ON vuln.interest_score(canonical_id) WHERE score >= 0.7;
CREATE INDEX idx_interest_score_low ON vuln.interest_score(canonical_id) WHERE score < 0.2;
```
## Testing
Run tests with:
```bash
dotnet test src/Concelier/__Tests/StellaOps.Concelier.Interest.Tests/
dotnet test src/Concelier/__Tests/StellaOps.Concelier.Storage.Postgres.Tests/ --filter "InterestScore"
```
## Sprint Reference
- Sprint: `SPRINT_8200_0013_0002_CONCEL_interest_scoring`
- Tasks: ISCORE-8200-000 through ISCORE-8200-033

19
src/Concelier/__Libraries/StellaOps.Concelier.Testing/ConcelierPostgresFixture.cs
View File

@@ -1,19 +0,0 @@
using System.Reflection;
using StellaOps.Concelier.Storage.Postgres;
using StellaOps.Infrastructure.Postgres.Testing;
using Xunit;
namespace StellaOps.Concelier.Testing;
/// <summary>
/// PostgreSQL integration test fixture for the Concelier module.
/// Runs migrations from embedded resources and provides test isolation via schema truncation.
/// </summary>
public sealed class ConcelierPostgresFixture : PostgresIntegrationFixture, ICollectionFixture<ConcelierPostgresFixture>
{
protected override Assembly? GetMigrationAssembly()
=> typeof(ConcelierDataSource).Assembly;
protected override string GetModuleName() => "Concelier";
}

121
src/Concelier/__Libraries/StellaOps.Concelier.Testing/ConnectorTestHarness.cs
View File

@@ -1,121 +0,0 @@
using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Time.Testing;
using StellaOps.Concelier.Connector.Common;
using StellaOps.Concelier.Connector.Common.Testing;
using StellaOps.Concelier.Connector.Common.Http;
using StellaOps.Concelier.Storage.Postgres;
using StellaOps.Infrastructure.Postgres.Options;
namespace StellaOps.Concelier.Testing;
/// <summary>
/// Provides a reusable container for connector integration tests with canned HTTP responses and PostgreSQL-backed storage.
/// </summary>
public sealed class ConnectorTestHarness : IAsyncDisposable
{
private readonly ConcelierPostgresFixture _fixture;
private readonly DateTimeOffset _initialTime;
private readonly string[] _httpClientNames;
private ServiceProvider? _serviceProvider;
public ConnectorTestHarness(ConcelierPostgresFixture fixture, DateTimeOffset initialTime, params string[] httpClientNames)
{
_fixture = fixture ?? throw new ArgumentNullException(nameof(fixture));
_initialTime = initialTime;
_httpClientNames = httpClientNames.Length == 0
? Array.Empty<string>()
: httpClientNames.Distinct(StringComparer.Ordinal).ToArray();
TimeProvider = CreateTimeProvider(initialTime);
Handler = new CannedHttpMessageHandler();
}
public FakeTimeProvider TimeProvider { get; private set; }
public CannedHttpMessageHandler Handler { get; }
public ServiceProvider ServiceProvider => _serviceProvider ?? throw new InvalidOperationException("Call EnsureServiceProviderAsync first.");
public async Task<ServiceProvider> EnsureServiceProviderAsync(Action<IServiceCollection> configureServices)
{
ArgumentNullException.ThrowIfNull(configureServices);
if (_serviceProvider is not null)
{
return _serviceProvider;
}
await _fixture.TruncateAllTablesAsync(CancellationToken.None);
var services = new ServiceCollection();
services.AddLogging(builder => builder.AddProvider(NullLoggerProvider.Instance));
services.AddSingleton<TimeProvider>(TimeProvider);
services.AddSingleton(TimeProvider);
services.AddSingleton(Handler);
services.AddConcelierPostgresStorage(options =>
{
options.ConnectionString = _fixture.ConnectionString;
options.SchemaName = _fixture.SchemaName;
options.CommandTimeoutSeconds = 5;
});
services.AddSourceCommon();
configureServices(services);
foreach (var clientName in _httpClientNames)
{
services.Configure<HttpClientFactoryOptions>(clientName, options =>
{
options.HttpMessageHandlerBuilderActions.Add(builder =>
{
builder.PrimaryHandler = Handler;
});
});
}
var provider = services.BuildServiceProvider();
_serviceProvider = provider;
return provider;
}
public async Task ResetAsync()
{
if (_serviceProvider is { } provider)
{
if (provider is IAsyncDisposable asyncDisposable)
{
await asyncDisposable.DisposeAsync();
}
else
{
provider.Dispose();
}
_serviceProvider = null;
}
await _fixture.TruncateAllTablesAsync(CancellationToken.None);
Handler.Clear();
TimeProvider = CreateTimeProvider(_initialTime);
}
public async ValueTask DisposeAsync()
{
await ResetAsync();
}
private static FakeTimeProvider CreateTimeProvider(DateTimeOffset now)
=> new(now)
{
AutoAdvanceAmount = TimeSpan.Zero,
};
}

20
src/Concelier/__Libraries/StellaOps.Concelier.Testing/StellaOps.Concelier.Testing.csproj
View File

@@ -1,20 +0,0 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
<IsTestProject>false</IsTestProject>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.TimeProvider.Testing" Version="10.0.0" />
<PackageReference Include="xunit" Version="2.9.3">
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="../StellaOps.Concelier.Connector.Common/StellaOps.Concelier.Connector.Common.csproj" />
<ProjectReference Include="../StellaOps.Concelier.Storage.Postgres/StellaOps.Concelier.Storage.Postgres.csproj" />
<ProjectReference Include="../../../__Libraries/StellaOps.Infrastructure.Postgres.Testing/StellaOps.Infrastructure.Postgres.Testing.csproj" />
</ItemGroup>
</Project>