feat: Add PathViewer and RiskDriftCard components with templates and styles
- Implemented PathViewerComponent for visualizing reachability call paths. - Added RiskDriftCardComponent to display reachability drift results. - Created corresponding HTML templates and SCSS styles for both components. - Introduced test fixtures for reachability analysis in JSON format. - Enhanced user interaction with collapsible and expandable features in PathViewer. - Included risk trend visualization and summary metrics in RiskDriftCard.
This commit is contained in:
master
263
docs/cli/drift-cli.md
Normal file
263
docs/cli/drift-cli.md
Normal file
@@ -0,0 +1,263 @@
|
||||
# Drift CLI Reference
|
||||
|
||||
**Sprint:** SPRINT_3600_0004_0001
|
||||
**Task:** UI-024 - Update CLI documentation for drift commands
|
||||
|
||||
## Overview
|
||||
|
||||
The Drift CLI provides commands for detecting and analyzing reachability drift between scan results. Reachability drift occurs when the call paths to vulnerable code change between builds, potentially altering the risk profile of an application.
|
||||
|
||||
## Commands
|
||||
|
||||
### stellaops drift
|
||||
|
||||
Parent command for reachability drift operations.
|
||||
|
||||
```bash
|
||||
stellaops drift <SUBCOMMAND> [OPTIONS]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### stellaops drift compare
|
||||
|
||||
Compare reachability between two scans or graph snapshots.
|
||||
|
||||
```bash
|
||||
stellaops drift compare [OPTIONS]
|
||||
```
|
||||
|
||||
#### Required Options
|
||||
|
||||
| Option | Alias | Description |
|
||||
|--------|-------|-------------|
|
||||
| `--base <ID>` | `-b` | Base scan/graph ID or commit SHA for comparison |
|
||||
|
||||
#### Optional Options
|
||||
|
||||
| Option | Alias | Description | Default |
|
||||
|--------|-------|-------------|---------|
|
||||
| `--head <ID>` | `-h` | Head scan/graph ID or commit SHA | latest |
|
||||
| `--image <REF>` | `-i` | Container image reference (digest or tag) | - |
|
||||
| `--repo <REPO>` | `-r` | Repository reference (owner/repo) | - |
|
||||
| `--output <FMT>` | `-o` | Output format: `table`, `json`, `sarif` | `table` |
|
||||
| `--min-severity <SEV>` | | Minimum severity: `critical`, `high`, `medium`, `low`, `info` | `medium` |
|
||||
| `--only-increases` | | Only show sinks with increased reachability | `false` |
|
||||
| `--verbose` | | Enable verbose output | `false` |
|
||||
|
||||
#### Examples
|
||||
|
||||
##### Compare by scan IDs
|
||||
|
||||
```bash
|
||||
stellaops drift compare --base abc123 --head def456
|
||||
```
|
||||
|
||||
##### Compare by commit SHAs
|
||||
|
||||
```bash
|
||||
stellaops drift compare --base HEAD~1 --head HEAD --repo myorg/myapp
|
||||
```
|
||||
|
||||
##### Filter to risk increases only
|
||||
|
||||
```bash
|
||||
stellaops drift compare --base abc123 --only-increases --min-severity high
|
||||
```
|
||||
|
||||
##### Output as JSON
|
||||
|
||||
```bash
|
||||
stellaops drift compare --base abc123 --output json > drift.json
|
||||
```
|
||||
|
||||
##### Output as SARIF for CI integration
|
||||
|
||||
```bash
|
||||
stellaops drift compare --base abc123 --output sarif > drift.sarif
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### stellaops drift show
|
||||
|
||||
Display details of a previously computed drift result.
|
||||
|
||||
```bash
|
||||
stellaops drift show [OPTIONS]
|
||||
```
|
||||
|
||||
#### Required Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--id <ID>` | Drift result ID to display |
|
||||
|
||||
#### Optional Options
|
||||
|
||||
| Option | Alias | Description | Default |
|
||||
|--------|-------|-------------|---------|
|
||||
| `--output <FMT>` | `-o` | Output format: `table`, `json`, `sarif` | `table` |
|
||||
| `--expand-paths` | | Show full call paths instead of compressed view | `false` |
|
||||
| `--verbose` | | Enable verbose output | `false` |
|
||||
|
||||
#### Examples
|
||||
|
||||
##### Show drift result
|
||||
|
||||
```bash
|
||||
stellaops drift show --id drift-abc123
|
||||
```
|
||||
|
||||
##### Show with expanded paths
|
||||
|
||||
```bash
|
||||
stellaops drift show --id drift-abc123 --expand-paths
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Output Formats
|
||||
|
||||
### Table Format (Default)
|
||||
|
||||
Human-readable table output using Spectre.Console:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Reachability Drift (abc123) │
|
||||
├───────────────────────────────┬─────────────────────────────┤
|
||||
│ Metric │ Value │
|
||||
├───────────────────────────────┼─────────────────────────────┤
|
||||
│ Trend │ ↑ Increasing │
|
||||
│ Net Risk Delta │ +3 │
|
||||
│ Increased │ 4 │
|
||||
│ Decreased │ 1 │
|
||||
│ New Sinks │ 2 │
|
||||
│ Removed Sinks │ 0 │
|
||||
└───────────────────────────────┴─────────────────────────────┘
|
||||
|
||||
┌──────────────┬──────────────────────┬───────────────┬─────────────────────────┬───────┐
|
||||
│ Severity │ Sink │ CVE │ Bucket Change │ Delta │
|
||||
├──────────────┼──────────────────────┼───────────────┼─────────────────────────┼───────┤
|
||||
│ CRITICAL │ SqlConnection.Open │ CVE-2024-1234 │ Runtime → Entrypoint │ +2 │
|
||||
│ HIGH │ XmlParser.Parse │ CVE-2024-5678 │ Unknown → Direct │ +1 │
|
||||
└──────────────┴──────────────────────┴───────────────┴─────────────────────────┴───────┘
|
||||
```
|
||||
|
||||
### JSON Format
|
||||
|
||||
Structured JSON for programmatic processing:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "abc123",
|
||||
"comparedAt": "2025-12-18T10:30:00Z",
|
||||
"baseGraphId": "base-graph-id",
|
||||
"headGraphId": "head-graph-id",
|
||||
"summary": {
|
||||
"totalSinks": 42,
|
||||
"increasedReachability": 4,
|
||||
"decreasedReachability": 1,
|
||||
"unchangedReachability": 35,
|
||||
"newSinks": 2,
|
||||
"removedSinks": 0,
|
||||
"riskTrend": "increasing",
|
||||
"netRiskDelta": 3
|
||||
},
|
||||
"driftedSinks": [
|
||||
{
|
||||
"sinkSymbol": "SqlConnection.Open",
|
||||
"cveId": "CVE-2024-1234",
|
||||
"severity": "critical",
|
||||
"previousBucket": "runtime",
|
||||
"currentBucket": "entrypoint",
|
||||
"isRiskIncrease": true,
|
||||
"riskDelta": 2
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### SARIF Format
|
||||
|
||||
SARIF 2.1.0 output for CI/CD integration:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "2.1.0",
|
||||
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
|
||||
"runs": [
|
||||
{
|
||||
"tool": {
|
||||
"driver": {
|
||||
"name": "StellaOps Drift",
|
||||
"version": "1.0.0",
|
||||
"informationUri": "https://stellaops.io/docs/drift"
|
||||
}
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"ruleId": "CVE-2024-1234",
|
||||
"level": "error",
|
||||
"message": {
|
||||
"text": "Reachability changed: runtime → entrypoint"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exit Codes
|
||||
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| `0` | Success (no risk increases or within threshold) |
|
||||
| `1` | Error during execution |
|
||||
| `2` | Risk increases detected |
|
||||
| `3` | Critical risk increases detected |
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
- name: Check Reachability Drift
|
||||
run: |
|
||||
stellaops drift compare \
|
||||
--base ${{ github.event.pull_request.base.sha }} \
|
||||
--head ${{ github.sha }} \
|
||||
--repo ${{ github.repository }} \
|
||||
--output sarif > drift.sarif
|
||||
continue-on-error: true
|
||||
|
||||
- name: Upload SARIF
|
||||
uses: github/codeql-action/upload-sarif@v2
|
||||
with:
|
||||
sarif_file: drift.sarif
|
||||
```
|
||||
|
||||
### GitLab CI
|
||||
|
||||
```yaml
|
||||
drift-check:
|
||||
script:
|
||||
- stellaops drift compare --base $CI_MERGE_REQUEST_DIFF_BASE_SHA --head $CI_COMMIT_SHA --output sarif > drift.sarif
|
||||
artifacts:
|
||||
reports:
|
||||
sast: drift.sarif
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Reachability Analysis](../reachability/README.md)
|
||||
- [Smart-Diff CLI](./smart-diff-cli.md)
|
||||
- [VEX Decisioning](../vex/decisioning.md)
|
||||
256
docs/contracts/vuln-surface-v1.md
Normal file
256
docs/contracts/vuln-surface-v1.md
Normal file
@@ -0,0 +1,256 @@
|
||||
# Vuln Surface Contract v1
|
||||
|
||||
**Sprint:** SPRINT_3700_0002_0001
|
||||
**Task:** SURF-024
|
||||
**Schema:** `stella.ops/vulnSurface@v1`
|
||||
|
||||
## Overview
|
||||
|
||||
A Vulnerability Surface represents the specific methods that changed between a vulnerable and fixed version of a package. This enables precise reachability analysis by identifying the exact "trigger" methods that are dangerous rather than treating the entire package as vulnerable.
|
||||
|
||||
## Use Cases
|
||||
|
||||
1. **Noise Reduction** - Only flag findings where code actually calls vulnerable methods
|
||||
2. **Confidence Tiers** - "Confirmed reachable" (calls trigger) vs "Potentially reachable" (uses package)
|
||||
3. **Remediation Guidance** - Show developers exactly which API calls to avoid
|
||||
4. **VEX Precision** - Automatically generate VEX "not_affected" for unreachable triggers
|
||||
|
||||
## Data Model
|
||||
|
||||
### VulnSurface
|
||||
|
||||
Root object representing a computed vulnerability surface.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `surface_id` | integer | Yes | Database ID |
|
||||
| `cve_id` | string | Yes | CVE identifier (e.g., "CVE-2024-12345") |
|
||||
| `package_id` | string | Yes | Package identifier in PURL format |
|
||||
| `ecosystem` | string | Yes | Package ecosystem: `nuget`, `npm`, `maven`, `pypi` |
|
||||
| `vuln_version` | string | Yes | Vulnerable version analyzed |
|
||||
| `fixed_version` | string | Yes | First fixed version used for diff |
|
||||
| `sinks` | VulnSurfaceSink[] | Yes | Changed methods (vulnerability triggers) |
|
||||
| `trigger_count` | integer | Yes | Number of callers to sink methods |
|
||||
| `status` | VulnSurfaceStatus | Yes | Computation status |
|
||||
| `confidence` | number | Yes | Confidence score (0.0-1.0) |
|
||||
| `computed_at` | string | Yes | ISO 8601 timestamp |
|
||||
|
||||
### VulnSurfaceSink
|
||||
|
||||
A method that changed between vulnerable and fixed versions.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `sink_id` | integer | Yes | Database ID |
|
||||
| `method_key` | string | Yes | Fully qualified method signature |
|
||||
| `method_name` | string | Yes | Simple method name |
|
||||
| `declaring_type` | string | Yes | Containing class/module |
|
||||
| `namespace` | string | No | Namespace/package |
|
||||
| `change_type` | MethodChangeType | Yes | How the method changed |
|
||||
| `is_public` | boolean | Yes | Whether method is publicly accessible |
|
||||
| `parameter_count` | integer | No | Number of parameters |
|
||||
| `return_type` | string | No | Return type |
|
||||
| `source_file` | string | No | Source file (from debug symbols) |
|
||||
| `start_line` | integer | No | Starting line number |
|
||||
| `end_line` | integer | No | Ending line number |
|
||||
|
||||
### VulnSurfaceTrigger
|
||||
|
||||
A call site that invokes a vulnerable sink method.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `trigger_id` | integer | Yes | Database ID |
|
||||
| `sink_id` | integer | Yes | Reference to sink |
|
||||
| `scan_id` | UUID | Yes | Scan where trigger was found |
|
||||
| `caller_node_id` | string | Yes | Call graph node ID |
|
||||
| `caller_method_key` | string | Yes | FQN of calling method |
|
||||
| `caller_file` | string | No | Source file of caller |
|
||||
| `caller_line` | integer | No | Line number of call |
|
||||
| `reachability_bucket` | string | Yes | Reachability classification |
|
||||
| `path_length` | integer | No | Shortest path from entrypoint |
|
||||
| `confidence` | number | Yes | Confidence score (0.0-1.0) |
|
||||
| `call_type` | string | Yes | Call type: `direct`, `virtual`, `interface`, `reflection` |
|
||||
| `is_conditional` | boolean | Yes | Whether call is behind a condition |
|
||||
|
||||
## Enums
|
||||
|
||||
### VulnSurfaceStatus
|
||||
|
||||
| Value | Description |
|
||||
|-------|-------------|
|
||||
| `pending` | Surface computation queued |
|
||||
| `computing` | Currently being computed |
|
||||
| `computed` | Successfully computed |
|
||||
| `failed` | Computation failed |
|
||||
| `stale` | Needs recomputation (new version available) |
|
||||
|
||||
### MethodChangeType
|
||||
|
||||
| Value | Description |
|
||||
|-------|-------------|
|
||||
| `added` | Method added in fix (not in vulnerable version) |
|
||||
| `removed` | Method removed in fix (was in vulnerable version) |
|
||||
| `modified` | Method body changed between versions |
|
||||
| `unknown` | Change type could not be determined |
|
||||
|
||||
### Reachability Buckets
|
||||
|
||||
| Bucket | Description | Risk Level |
|
||||
|--------|-------------|------------|
|
||||
| `entrypoint` | Sink is directly exposed as entrypoint | Critical |
|
||||
| `direct` | Reachable from entrypoint with no authentication gates | High |
|
||||
| `runtime` | Reachable but behind runtime conditions/auth | Medium |
|
||||
| `unknown` | Reachability could not be determined | Medium |
|
||||
| `unreachable` | No path from any entrypoint | Low |
|
||||
|
||||
## Fingerprinting Methods
|
||||
|
||||
### cecil-il (NuGet/.NET)
|
||||
|
||||
Uses Mono.Cecil to compute SHA-256 hash of IL instruction sequence:
|
||||
|
||||
```
|
||||
IL_0000: ldarg.0
|
||||
IL_0001: call System.Object::.ctor()
|
||||
IL_0006: ret
|
||||
```
|
||||
|
||||
Normalized to remove:
|
||||
- NOP instructions
|
||||
- Debug sequence points
|
||||
- Local variable indices (replaced with placeholders)
|
||||
|
||||
### babel-ast (npm/Node.js)
|
||||
|
||||
Uses Babel to parse JavaScript/TypeScript and compute hash of normalized AST:
|
||||
|
||||
```javascript
|
||||
function vulnerable(input) {
|
||||
eval(input); // dangerous!
|
||||
}
|
||||
```
|
||||
|
||||
Normalized to remove:
|
||||
- Comments
|
||||
- Whitespace
|
||||
- Variable names (renamed to positional)
|
||||
|
||||
### asm-bytecode (Maven/Java)
|
||||
|
||||
Uses ASM to compute hash of Java bytecode:
|
||||
|
||||
```
|
||||
ALOAD 0
|
||||
INVOKESPECIAL java/lang/Object.<init>()V
|
||||
RETURN
|
||||
```
|
||||
|
||||
Normalized to remove:
|
||||
- Line number tables
|
||||
- Local variable tables
|
||||
- Stack map frames
|
||||
|
||||
### python-ast (PyPI)
|
||||
|
||||
Uses Python's `ast` module to compute hash of normalized AST:
|
||||
|
||||
```python
|
||||
def vulnerable(user_input):
|
||||
exec(user_input) # dangerous!
|
||||
```
|
||||
|
||||
Normalized to remove:
|
||||
- Docstrings
|
||||
- Comments
|
||||
- Variable names
|
||||
|
||||
## Database Schema
|
||||
|
||||
```sql
|
||||
-- Surfaces table
|
||||
CREATE TABLE scanner.vuln_surfaces (
|
||||
id UUID PRIMARY KEY,
|
||||
tenant_id UUID NOT NULL,
|
||||
cve_id TEXT NOT NULL,
|
||||
package_ecosystem TEXT NOT NULL,
|
||||
package_name TEXT NOT NULL,
|
||||
vuln_version TEXT NOT NULL,
|
||||
fixed_version TEXT,
|
||||
fingerprint_method TEXT NOT NULL,
|
||||
total_methods_vuln INTEGER,
|
||||
total_methods_fixed INTEGER,
|
||||
changed_method_count INTEGER,
|
||||
computed_at TIMESTAMPTZ DEFAULT now(),
|
||||
UNIQUE (tenant_id, cve_id, package_ecosystem, package_name, vuln_version)
|
||||
);
|
||||
|
||||
-- Sinks table
|
||||
CREATE TABLE scanner.vuln_surface_sinks (
|
||||
id UUID PRIMARY KEY,
|
||||
surface_id UUID REFERENCES scanner.vuln_surfaces(id) ON DELETE CASCADE,
|
||||
method_key TEXT NOT NULL,
|
||||
method_name TEXT NOT NULL,
|
||||
declaring_type TEXT NOT NULL,
|
||||
change_type TEXT NOT NULL,
|
||||
UNIQUE (surface_id, method_key)
|
||||
);
|
||||
|
||||
-- Triggers table
|
||||
CREATE TABLE scanner.vuln_surface_triggers (
|
||||
id UUID PRIMARY KEY,
|
||||
sink_id UUID REFERENCES scanner.vuln_surface_sinks(id) ON DELETE CASCADE,
|
||||
scan_id UUID NOT NULL,
|
||||
caller_node_id TEXT NOT NULL,
|
||||
reachability_bucket TEXT NOT NULL,
|
||||
confidence REAL NOT NULL,
|
||||
UNIQUE (sink_id, scan_id, caller_node_id)
|
||||
);
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### POST /api/v1/surfaces/compute
|
||||
|
||||
Request surface computation for a CVE + package.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"cveId": "CVE-2024-12345",
|
||||
"ecosystem": "nuget",
|
||||
"packageName": "Newtonsoft.Json",
|
||||
"vulnVersion": "13.0.1",
|
||||
"fixedVersion": "13.0.2"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"surfaceId": "uuid",
|
||||
"status": "pending"
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/v1/surfaces/{surfaceId}
|
||||
|
||||
Get computed surface with sinks.
|
||||
|
||||
### GET /api/v1/surfaces/{surfaceId}/triggers?scanId={scanId}
|
||||
|
||||
Get triggers for a surface in a specific scan.
|
||||
|
||||
## Integration Points
|
||||
|
||||
1. **Concelier** - Feeds CVE + affected version ranges
|
||||
2. **Scanner** - Computes surfaces during SBOM analysis
|
||||
3. **Call Graph** - Provides reachability analysis
|
||||
4. **VEX Lens** - Uses surfaces for automated VEX decisions
|
||||
5. **UI** - Displays surface details and trigger paths
|
||||
|
||||
## References
|
||||
|
||||
- [Vuln Surfaces Sprint](../implplan/SPRINT_3700_0002_0001_vuln_surfaces_core.md)
|
||||
- [Reachability Architecture](../reachability/README.md)
|
||||
- [RichGraph Contract](./richgraph-v1.md)
|
||||
@@ -64,12 +64,40 @@ Before starting, read:
|
||||
| 4 | T4 | DONE | Expose verification settings | Attestor Guild | Add `RekorVerificationOptions` in Configuration/ |
|
||||
| 5 | T5 | DONE | Use verifiers in HTTP client | Attestor Guild | Implement `HttpRekorClient.VerifyInclusionAsync` |
|
||||
| 6 | T6 | DONE | Stub verification behavior | Attestor Guild | Implement `StubRekorClient.VerifyInclusionAsync` |
|
||||
| 7 | T7 | BLOCKED | Wire verification pipeline | Attestor Guild | Requires T8 for offline mode before full pipeline integration |
|
||||
| 8 | T8 | BLOCKED | Add sealed/offline checkpoint mode | Attestor Guild | Depends on finalized offline checkpoint bundle format contract |
|
||||
| 9 | T9 | DONE | Add unit coverage | Attestor Guild | Add unit tests for Merkle proof verification |
|
||||
| 10 | T10 | DONE | Add integration coverage | Attestor Guild | RekorInclusionVerificationIntegrationTests.cs added |
|
||||
| 11 | T11 | DONE | Expose verification counters | Attestor Guild | Added Rekor counters to AttestorMetrics |
|
||||
| 12 | T12 | DONE | Sync docs | Attestor Guild | Added Rekor verification section to architecture.md |
|
||||
| 7 | T6a | TODO | Freeze offline checkpoint/receipt contract | Attestor Guild · AirGap Guild | Publish canonical offline layout + schema for: tlog root key, checkpoint signature, and inclusion proof pack (docs + fixtures) |
|
||||
| 8 | T6b | TODO | Add offline fixtures + validation harness | Attestor Guild | Add deterministic fixtures + parsing helpers so offline mode can be tested without network |
|
||||
| 9 | T7 | BLOCKED | Wire verification pipeline | Attestor Guild | BLOCKED on T8 (and its prerequisites T6a/T6b) before full pipeline integration |
|
||||
| 10 | T8 | BLOCKED | Add sealed/offline checkpoint mode | Attestor Guild | BLOCKED on T6a/T6b (offline checkpoint/receipt contract + fixtures) |
|
||||
| 11 | T9 | DONE | Add unit coverage | Attestor Guild | Add unit tests for Merkle proof verification |
|
||||
| 12 | T10 | DONE | Add integration coverage | Attestor Guild | RekorInclusionVerificationIntegrationTests.cs added |
|
||||
| 13 | T11 | DONE | Expose verification counters | Attestor Guild | Added Rekor counters to AttestorMetrics |
|
||||
| 14 | T12 | DONE | Sync docs | Attestor Guild | Added Rekor verification section to architecture.md |
|
||||
|
||||
---
|
||||
|
||||
## Unblock Task Notes (T6a/T6b)
|
||||
|
||||
### T6a: Freeze offline checkpoint/receipt contract
|
||||
- **Goal:** define the canonical offline inputs required to verify inclusion proofs without network access.
|
||||
- **Use these docs as the baseline (do not invent new shapes):**
|
||||
- `docs/product-advisories/14-Dec-2025 - Rekor Integration Technical Reference.md` (§13)
|
||||
- `docs/product-advisories/14-Dec-2025 - Offline and Air-Gap Technical Reference.md` (§3–4; `evidence/tlog/checkpoint.sig` + `entries/`)
|
||||
- **Minimum deliverables:**
|
||||
- A single canonical contract doc (new or existing) that answers:
|
||||
- Where the **tlog public key** comes from (file path, rotation/versioning)
|
||||
- Where the **signed checkpoint/tree head** lives (file path; signature format)
|
||||
- Where the **inclusion proof pack** lives (file path; entry + hashes; deterministic ordering rules)
|
||||
- How the checkpoint is bound to the proof pack (tree size, root hash)
|
||||
- A schema file (JSON Schema) for the on-disk checkpoint/receipt shape used by Attestor offline verification.
|
||||
|
||||
### T6b: Offline fixtures + validation harness
|
||||
- **Goal:** make offline mode testable and reproducible.
|
||||
- **Minimum deliverables:**
|
||||
- Deterministic fixtures committed under `src/Attestor/StellaOps.Attestor.Tests/Fixtures/` (checkpoint, pubkey, valid/invalid proof material).
|
||||
- Tests that verify:
|
||||
- checkpoint signature verification succeeds/fails as expected
|
||||
- recomputed Merkle root matches checkpoint for valid entries and fails for tampered fixtures
|
||||
- no network calls are required for offline mode
|
||||
|
||||
---
|
||||
|
||||
@@ -285,6 +313,7 @@ public Counter<long> CheckpointVerifyTotal { get; } // attestor.checkpoint_
|
||||
## Interlocks
|
||||
- Rekor public key distribution must be configured via `AttestorOptions` and documented for offline bundles.
|
||||
- Offline checkpoints must be pre-distributed; `AllowOfflineWithoutSignature` policy requires explicit operator intent.
|
||||
- T6a/T6b define the concrete offline checkpoint/receipt contract and fixtures; do not implement T8 until those are published and reviewed.
|
||||
|
||||
---
|
||||
|
||||
@@ -320,6 +349,7 @@ public Counter<long> CheckpointVerifyTotal { get; } // attestor.checkpoint_
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-14 | Normalised sprint file to standard template sections; started implementation and moved `T1` to `DOING`. | Implementer |
|
||||
| 2025-12-18 | Added unblock tasks (T6a/T6b) for offline checkpoint/receipt contract + fixtures; updated T7/T8 to be BLOCKED on them. | Project Mgmt |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -160,11 +160,13 @@ External Dependencies:
|
||||
| **EPSS-3410-011** | Implement outbox event schema | DONE | Agent | 2h | `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Epss/Events/EpssUpdatedEvent.cs` |
|
||||
| **EPSS-3410-012** | Unit tests (parser, detector, flags) | DONE | Agent | 6h | `EpssCsvStreamParserTests.cs`, `EpssChangeDetectorTests.cs` |
|
||||
| **EPSS-3410-013** | Integration tests (Testcontainers) | DONE | Agent | 8h | `EpssRepositoryIntegrationTests.cs` |
|
||||
| **EPSS-3410-014** | Performance test (300k rows) | BLOCKED | Backend | 4h | Requires CI infrastructure for benchmark runs with Testcontainers + 300k row dataset. Repository uses NpgsqlBinaryImporter for bulk insert; expected <120s based on similar workloads. |
|
||||
| **EPSS-3410-013A** | Perf harness + deterministic dataset generator | TODO | Backend | 4h | Add a perf test project and deterministic 310k-row CSV generator (fixed seed, no network). Produce local run instructions and baseline output format. |
|
||||
| **EPSS-3410-013B** | CI perf runner + workflow for EPSS ingest | TODO | DevOps | 4h | Add a Gitea workflow (nightly/manual) + runner requirements so perf tests can run with Docker/Testcontainers; publish runner label/capacity requirements and artifact retention. |
|
||||
| **EPSS-3410-014** | Performance test (300k rows) | BLOCKED | Backend | 4h | BLOCKED on EPSS-3410-013A/013B. Once harness + CI runner exist, execute and record baseline (<120s) with environment details. |
|
||||
| **EPSS-3410-015** | Observability (metrics, logs, traces) | DONE | Agent | 4h | ActivitySource with tags (model_date, row_count, cve_count, duration_ms); structured logging at Info/Warning/Error levels. |
|
||||
| **EPSS-3410-016** | Documentation (runbook, troubleshooting) | DONE | Agent | 3h | Added Operations Runbook (§10) to `docs/modules/scanner/epss-integration.md` with configuration, modes, manual ingestion, troubleshooting, and monitoring guidance. |
|
||||
|
||||
**Total Estimated Effort**: 65 hours (~2 weeks for 1 developer)
|
||||
**Total Estimated Effort**: 73 hours (~2 weeks for 1 developer)
|
||||
|
||||
---
|
||||
|
||||
@@ -604,11 +606,46 @@ public async Task ComputeChanges_DetectsFlags_Correctly()
|
||||
|
||||
---
|
||||
|
||||
### EPSS-3410-013A: Perf Harness + Deterministic Dataset Generator
|
||||
|
||||
**Description**: Add an offline-friendly perf harness for EPSS ingest without committing a huge static dataset.
|
||||
|
||||
**Deliverables**:
|
||||
- New test project: `src/Scanner/__Tests/StellaOps.Scanner.Storage.Performance.Tests/`
|
||||
- Deterministic generator: 310k rows with fixed seed, stable row order, and controlled CVE distribution.
|
||||
- Test tagged so it does not run in default CI (`[Trait("Category","Performance")]` or equivalent).
|
||||
- Local run snippet (exact `dotnet test` invocation + required env vars for Testcontainers).
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Generator produces identical output across runs (same seed ⇒ same SHA-256 of CSV bytes)
|
||||
- [ ] Perf test runs locally in <= 5 minutes on a dev machine (budget validation happens in CI)
|
||||
- [ ] No network required beyond local Docker engine for Testcontainers
|
||||
|
||||
---
|
||||
|
||||
### EPSS-3410-013B: CI Perf Runner + Workflow
|
||||
|
||||
**Description**: Enable deterministic perf execution in CI with known hardware + reproducible logs.
|
||||
|
||||
**Deliverables**:
|
||||
- Gitea workflow (nightly + manual): `.gitea/workflows/epss-perf.yml`
|
||||
- Runner requirements documented (label, OS/arch, CPU/RAM, Docker/Testcontainers support).
|
||||
- Artifacts retained: perf logs + environment metadata (CPU model, cores, memory, Docker version, image digests).
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] CI job can spin up PostgreSQL via Testcontainers reliably
|
||||
- [ ] Perf test output includes total duration + phase breakdowns (parse/insert/changes/current)
|
||||
- [ ] Budgets enforced only in this workflow (does not break default PR CI)
|
||||
|
||||
---
|
||||
|
||||
### EPSS-3410-014: Performance Test (300k rows)
|
||||
|
||||
**Description**: Verify ingestion meets performance budget.
|
||||
|
||||
**File**: `src/Concelier/__Tests/StellaOps.Concelier.Epss.Performance.Tests/EpssIngestPerformanceTests.cs`
|
||||
**BLOCKED ON:** EPSS-3410-013A, EPSS-3410-013B
|
||||
|
||||
**File**: `src/Scanner/__Tests/StellaOps.Scanner.Storage.Performance.Tests/EpssIngestPerformanceTests.cs` (new project)
|
||||
|
||||
**Requirements**:
|
||||
- Synthetic CSV: 310,000 rows (close to real-world)
|
||||
@@ -865,11 +902,12 @@ concelier:
|
||||
| 2025-12-18 | Completed EPSS-3410-015: Verified ActivitySource tracing with model_date, row_count, cve_count, duration_ms tags; structured logging in place. | Agent |
|
||||
| 2025-12-18 | Completed EPSS-3410-016: Added Operations Runbook (§10) to docs/modules/scanner/epss-integration.md covering config, online/bundle modes, manual trigger, troubleshooting, monitoring. | Agent |
|
||||
| 2025-12-18 | BLOCKED EPSS-3410-014: Performance test requires CI infrastructure and 300k row dataset. BULK INSERT uses NpgsqlBinaryImporter; expected to meet <120s budget. | Agent |
|
||||
| 2025-12-18 | Added unblock tasks EPSS-3410-013A/013B; EPSS-3410-014 remains BLOCKED until harness + CI perf runner/workflow are available. | Project Mgmt |
|
||||
|
||||
## Next Checkpoints
|
||||
|
||||
- Unblock performance test (014) when CI infrastructure is available.
|
||||
- Unblock performance test (EPSS-3410-014) by completing EPSS-3410-013A (harness) and EPSS-3410-013B (CI perf runner/workflow).
|
||||
- Close Scanner integration (SPRINT_3410_0002_0001).
|
||||
|
||||
**Sprint Status**: BLOCKED (1 task pending CI infrastructure)
|
||||
**Sprint Status**: BLOCKED (EPSS-3410-014 pending EPSS-3410-013B CI perf runner/workflow)
|
||||
**Approval**: _____________________ Date: ___________
|
||||
|
||||
@@ -47,8 +47,8 @@ Integrate EPSS v4 data into the Scanner WebService for vulnerability scoring and
|
||||
| 2 | EPSS-SCAN-002 | DONE | Agent | 2h | Create `EpssEvidence` record type |
|
||||
| 3 | EPSS-SCAN-003 | DONE | Agent | 4h | Implement `IEpssProvider` interface |
|
||||
| 4 | EPSS-SCAN-004 | DONE | Agent | 4h | Implement `EpssProvider` with PostgreSQL lookup |
|
||||
| 5 | EPSS-SCAN-005 | TODO | Backend | 2h | Add optional Valkey cache layer |
|
||||
| 6 | EPSS-SCAN-006 | TODO | Backend | 4h | Integrate EPSS into `ScanProcessor` |
|
||||
| 5 | EPSS-SCAN-005 | DONE | Agent | 2h | Add optional Valkey cache layer |
|
||||
| 6 | EPSS-SCAN-006 | DONE | Agent | 4h | Integrate EPSS into `ScanProcessor` via EpssEnrichmentStageExecutor |
|
||||
| 7 | EPSS-SCAN-007 | DONE | — | 2h | Add EPSS weight to scoring configuration (EpssMultiplier in ScoreExplanationWeights) |
|
||||
| 8 | EPSS-SCAN-008 | DONE | Agent | 4h | Implement `GET /epss/current` bulk lookup API |
|
||||
| 9 | EPSS-SCAN-009 | DONE | Agent | 2h | Implement `GET /epss/history` time-series API |
|
||||
@@ -132,6 +132,7 @@ scoring:
|
||||
| 2025-12-17 | Sprint created from advisory processing | Agent |
|
||||
| 2025-12-17 | EPSS-SCAN-001: Created 008_epss_integration.sql in Scanner Storage | Agent |
|
||||
| 2025-12-17 | EPSS-SCAN-012: Created docs/modules/scanner/epss-integration.md | Agent |
|
||||
| 2025-12-18 | EPSS-SCAN-005: Implemented CachingEpssProvider with Valkey cache layer. Created EpssServiceCollectionExtensions for DI registration. | Agent |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -37,14 +37,14 @@ This sprint implements live EPSS enrichment for existing vulnerability instances
|
||||
|
||||
| # | Status | Task | Notes |
|
||||
|---|--------|------|-------|
|
||||
| 1 | TODO | Implement `EpssEnrichmentJob` service | Core enrichment logic |
|
||||
| 2 | TODO | Create `vuln_instance_triage` schema updates | Add `current_epss_*` columns |
|
||||
| 1 | DONE | Implement `EpssEnrichmentJob` service | Created EpssEnrichmentJob.cs with background processing |
|
||||
| 2 | DONE | Create `vuln_instance_triage` schema updates | Created 014_epss_triage_columns.sql with EPSS columns and batch_update_epss_triage() |
|
||||
| 3 | DONE | Implement `epss_changes` flag logic | `EpssChangeFlags` enum with NEW_SCORED, CROSSED_HIGH, BIG_JUMP, DROPPED_LOW |
|
||||
| 4 | TODO | Add efficient targeting filter | Only update instances with flags set |
|
||||
| 4 | DONE | Add efficient targeting filter | Added GetChangesAsync() to IEpssRepository; EpssEnrichmentJob uses flag filtering |
|
||||
| 5 | DONE | Implement priority band calculation | `EpssPriorityCalculator` maps percentile to CRITICAL/HIGH/MEDIUM/LOW |
|
||||
| 6 | TODO | Emit `vuln.priority.changed` event | Only when band changes |
|
||||
| 6 | DONE | Emit `vuln.priority.changed` event | Added IEpssSignalPublisher.PublishPriorityChangedAsync() in EpssEnrichmentJob |
|
||||
| 7 | DONE | Add configurable thresholds | `EpssEnrichmentOptions` with HighPercentile, HighScore, BigJumpDelta, etc. |
|
||||
| 8 | TODO | Implement bulk update optimization | Batch updates for performance |
|
||||
| 8 | DONE | Implement bulk update optimization | Added batch_update_epss_triage() PostgreSQL function |
|
||||
| 9 | DONE | Add `EpssEnrichmentOptions` configuration | Environment-specific settings in Scanner.Core.Configuration |
|
||||
| 10 | TODO | Create unit tests for enrichment logic | Flag detection, band calculation |
|
||||
| 11 | TODO | Create integration tests | End-to-end enrichment flow |
|
||||
@@ -58,10 +58,12 @@ This sprint implements live EPSS enrichment for existing vulnerability instances
|
||||
|
||||
| # | Status | Task | Notes |
|
||||
|---|--------|------|-------|
|
||||
| R1 | TODO | Create `epss_raw` table migration | `011_epss_raw_layer.sql` - Full JSONB payload storage |
|
||||
| R2 | TODO | Update `EpssIngestJob` to store raw payload | Decompress CSV, convert to JSONB array, store in `epss_raw` |
|
||||
| R3 | TODO | Add retention policy for raw data | `prune_epss_raw()` function - Keep 365 days |
|
||||
| R4 | TODO | Implement `ReplayFromRawAsync()` method | Re-normalize from stored raw without re-downloading |
|
||||
| R1 | DONE | Create `epss_raw` table migration | `011_epss_raw_layer.sql` - Full JSONB payload storage |
|
||||
| R2 | DONE | Update `EpssIngestJob` to store raw payload | Added StoreRawPayloadAsync(), converts to JSONB, stores in `epss_raw` |
|
||||
| R3 | DONE | Add retention policy for raw data | `prune_epss_raw()` function in migration - Keep 365 days |
|
||||
| R4 | DONE | Implement `ReplayFromRawAsync()` method | Created EpssReplayService with ReplayFromRawAsync() and ReplayRangeAsync() |
|
||||
| R5 | DONE | Implement `IEpssRawRepository` interface | Created with CRUD operations |
|
||||
| R6 | DONE | Implement `PostgresEpssRawRepository` | PostgreSQL implementation with DI registration |
|
||||
|
||||
### Signal-Ready Layer Tasks (S1-S12)
|
||||
|
||||
@@ -69,16 +71,16 @@ This sprint implements live EPSS enrichment for existing vulnerability instances
|
||||
|
||||
| # | Status | Task | Notes |
|
||||
|---|--------|------|-------|
|
||||
| S1 | TODO | Create `epss_signal` table migration | `012_epss_signal_layer.sql` - Tenant-scoped with dedupe_key |
|
||||
| S2 | TODO | Implement `IEpssSignalRepository` interface | Signal CRUD operations |
|
||||
| S3 | TODO | Implement `PostgresEpssSignalRepository` | PostgreSQL implementation |
|
||||
| S4 | TODO | Implement `ComputeExplainHash()` | Deterministic SHA-256 of signal inputs |
|
||||
| S5 | TODO | Create `EpssSignalJob` service | Runs after enrichment, per-tenant |
|
||||
| S6 | TODO | Add "observed CVEs" filter | Only signal for CVEs in tenant's inventory |
|
||||
| S7 | TODO | Implement model version change detection | Compare vs previous day's `model_version_tag` |
|
||||
| S8 | TODO | Add `MODEL_UPDATED` event type | Summary event instead of 300k individual deltas |
|
||||
| S9 | TODO | Connect to Notify/Router | Publish to `signals.epss` topic |
|
||||
| S10 | TODO | Add signal deduplication | Idempotent via `dedupe_key` constraint |
|
||||
| S1 | DONE | Create `epss_signal` table migration | `012_epss_signal_layer.sql` - Tenant-scoped with dedupe_key |
|
||||
| S2 | DONE | Implement `IEpssSignalRepository` interface | Signal CRUD operations with config support |
|
||||
| S3 | DONE | Implement `PostgresEpssSignalRepository` | PostgreSQL implementation with DI registration |
|
||||
| S4 | DONE | Implement `ComputeExplainHash()` | Created EpssExplainHashCalculator with deterministic SHA-256 |
|
||||
| S5 | DONE | Create `EpssSignalJob` service | Created EpssSignalJob.cs with batch processing and tenant support |
|
||||
| S6 | DONE | Add "observed CVEs" filter | Created IObservedCveRepository and PostgresObservedCveRepository; integrated in EpssSignalJob |
|
||||
| S7 | DONE | Implement model version change detection | Added in EpssSignalJob with _lastModelVersion tracking |
|
||||
| S8 | DONE | Add `MODEL_UPDATED` event type | EmitModelUpdatedSignalAsync() creates summary event |
|
||||
| S9 | DONE | Connect to Notify/Router | Created IEpssSignalPublisher interface; EpssSignalJob publishes via PublishBatchAsync() |
|
||||
| S10 | DONE | Add signal deduplication | Idempotent via `dedupe_key` constraint in repository |
|
||||
| S11 | TODO | Unit tests for signal generation | Flag logic, explain hash, dedupe key |
|
||||
| S12 | TODO | Integration tests for signal flow | End-to-end tenant-scoped signal emission |
|
||||
| S13 | TODO | Add Prometheus metrics for signals | `epss_signals_emitted_total{event_type, tenant_id}` |
|
||||
@@ -175,15 +177,36 @@ concelier:
|
||||
|
||||
---
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-18 | Task #1: Implemented `EpssEnrichmentJob` with batch processing, priority band calculation, and trigger mechanism | Agent |
|
||||
| 2025-12-18 | R5-R6: Implemented `IEpssRawRepository` and `PostgresEpssRawRepository` for raw payload storage | Agent |
|
||||
| 2025-12-18 | S2-S3: Implemented `IEpssSignalRepository` and `PostgresEpssSignalRepository` with tenant config support | Agent |
|
||||
| 2025-12-18 | Registered new repositories in DI: `EpssRawRepository`, `EpssSignalRepository` | Agent |
|
||||
| 2025-12-18 | Task #2: Created 014_epss_triage_columns.sql migration with EPSS columns and batch_update_epss_triage() function | Agent |
|
||||
| 2025-12-18 | R2: Updated EpssIngestJob with StoreRawPayloadAsync() to store raw JSONB payload | Agent |
|
||||
| 2025-12-18 | S4: Created EpssExplainHashCalculator with ComputeExplainHash() and ComputeDedupeKey() | Agent |
|
||||
| 2025-12-18 | S5, S7, S8: Created EpssSignalJob with model version detection and MODEL_UPDATED event support | Agent |
|
||||
| 2025-12-18 | EPSS-SCAN-006: Created EpssEnrichmentStageExecutor for scan pipeline integration | Agent |
|
||||
| 2025-12-18 | R4: Created EpssReplayService with ReplayFromRawAsync() and ReplayRangeAsync() | Agent |
|
||||
| 2025-12-18 | S6: Created IObservedCveRepository, PostgresObservedCveRepository; integrated tenant-scoped filtering in EpssSignalJob | Agent |
|
||||
| 2025-12-18 | S9: Created IEpssSignalPublisher interface; integrated PublishBatchAsync() in EpssSignalJob | Agent |
|
||||
| 2025-12-18 | Task #4: Added GetChangesAsync() to IEpssRepository; EpssEnrichmentJob uses flag-based targeting | Agent |
|
||||
| 2025-12-18 | Task #6: Added PublishPriorityChangedAsync() to IEpssSignalPublisher; EpssEnrichmentJob emits events | Agent |
|
||||
|
||||
---
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
- [ ] `EpssEnrichmentJob` updates vuln_instance_triage with current EPSS
|
||||
- [ ] Only instances with material changes are updated (flag-based targeting)
|
||||
- [ ] `vuln.priority.changed` event emitted only when band changes
|
||||
- [ ] Raw payload stored in `epss_raw` for replay capability
|
||||
- [ ] Signals emitted only for observed CVEs per tenant
|
||||
- [ ] Model version changes suppress noisy delta signals
|
||||
- [ ] Each signal has deterministic `explain_hash`
|
||||
- [x] `EpssEnrichmentJob` updates vuln_instance_triage with current EPSS
|
||||
- [x] Only instances with material changes are updated (flag-based targeting)
|
||||
- [x] `vuln.priority.changed` event emitted only when band changes
|
||||
- [x] Raw payload stored in `epss_raw` for replay capability
|
||||
- [x] Signals emitted only for observed CVEs per tenant
|
||||
- [x] Model version changes suppress noisy delta signals
|
||||
- [x] Each signal has deterministic `explain_hash`
|
||||
- [ ] All unit and integration tests pass
|
||||
- [ ] Documentation updated
|
||||
|
||||
@@ -195,17 +218,29 @@ concelier:
|
||||
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/011_epss_raw_layer.sql`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/012_epss_signal_layer.sql`
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Services/EpssSignalJob.cs`
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Services/EpssExplainHashCalculator.cs`
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Repositories/IEpssSignalRepository.cs`
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Repositories/PostgresEpssSignalRepository.cs`
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Repositories/IEpssRawRepository.cs`
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Repositories/PostgresEpssRawRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/014_epss_triage_columns.sql`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Repositories/IEpssSignalRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Repositories/IEpssRawRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Repositories/IObservedCveRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/PostgresEpssSignalRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/PostgresEpssRawRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/PostgresObservedCveRepository.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Epss/EpssReplayService.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Epss/IEpssSignalPublisher.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Epss/CachingEpssProvider.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Epss/EpssExplainHashCalculator.cs`
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/EpssServiceCollectionExtensions.cs`
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/Processing/EpssEnrichmentJob.cs`
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/Processing/EpssEnrichmentStageExecutor.cs`
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/Processing/EpssSignalJob.cs`
|
||||
|
||||
### Existing Files to Update
|
||||
### Existing Files Updated
|
||||
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Jobs/EpssIngestJob.cs` - Store raw payload
|
||||
- `src/Concelier/__Libraries/StellaOps.Concelier.Epss/Jobs/EpssEnrichmentJob.cs` - Add model version detection
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs` - Added EPSS repository registrations
|
||||
- `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/MigrationIds.cs` - Added new migration IDs
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/Processing/ScanStageNames.cs` - Added EpssEnrichment stage
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/Processing/EpssIngestJob.cs` - Added raw payload storage
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/Program.cs` - Registered EpssEnrichmentStageExecutor
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -60,9 +60,9 @@ public sealed record NativeBinaryMetadata {
|
||||
| 2 | BSE-002 | DONE | Create NativeComponentEmitter |
|
||||
| 3 | BSE-003 | DONE | Create NativePurlBuilder |
|
||||
| 4 | BSE-004 | DONE | Create NativeComponentMapper (layer fragment generation) |
|
||||
| 5 | BSE-005 | DONE | Add NativeBinaryMetadata (with Imports/Exports) |
|
||||
| 6 | BSE-006 | TODO | Update CycloneDxComposer |
|
||||
| 7 | BSE-007 | TODO | Add stellaops:binary.* properties |
|
||||
| 5 | BSE-005 | DONE | Add NativeBinaryMetadata (with Imports/Exports/PE/Mach-O fields) |
|
||||
| 6 | BSE-006 | DONE | Update CycloneDxComposer via LayerComponentMapping.ToFragment() |
|
||||
| 7 | BSE-007 | DONE | Add stellaops:binary.* properties in ToComponentRecord() |
|
||||
| 8 | BSE-008 | DONE | Unit tests (22 tests passing) |
|
||||
| 9 | BSE-009 | TODO | Integration tests |
|
||||
|
||||
|
||||
@@ -48,8 +48,30 @@ Extend the Unknowns registry with native binary-specific classification reasons,
|
||||
| 1 | NUC-001 | DONE | Add UnknownKind enum values (MissingBuildId, UnknownBuildId, UnresolvedNativeLibrary, HeuristicDependency, UnsupportedBinaryFormat) |
|
||||
| 2 | NUC-002 | DONE | Create NativeUnknownContext model |
|
||||
| 3 | NUC-003 | DONE | Create NativeUnknownClassifier service |
|
||||
| 4 | NUC-004 | TODO | Integration with native analyzer |
|
||||
| 5 | NUC-005 | TODO | Unit tests |
|
||||
| 4 | NUC-003A | TODO | Approve + add `StellaOps.Unknowns.Core` reference from `src/Scanner/StellaOps.Scanner.Worker` (avoid circular deps; document final dependency direction) |
|
||||
| 5 | NUC-003B | TODO | Wire native analyzer outputs to Unknowns: call `NativeUnknownClassifier` and persist via Unknowns repository/service from scan pipeline |
|
||||
| 6 | NUC-004 | BLOCKED | Integrate with native analyzer (BLOCKED on NUC-003A/NUC-003B) |
|
||||
| 7 | NUC-005 | TODO | Unit tests |
|
||||
|
||||
---
|
||||
|
||||
## Unblock Task Notes (NUC-003A/NUC-003B)
|
||||
|
||||
### NUC-003A: Project reference + dependency direction
|
||||
- **Goal:** make the integration unambiguous: Scanner Worker emits Unknowns during scan; Unknowns.Core provides the domain + classifier.
|
||||
- **Touchpoints (expected):**
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/StellaOps.Scanner.Worker.csproj` (add project reference)
|
||||
- If persistence from Worker is required, also reference `src/Unknowns/__Libraries/StellaOps.Unknowns.Storage.Postgres/` and ensure migrations are applied by Scanner startup.
|
||||
- **Acceptance criteria (minimum):**
|
||||
- `dotnet build src/Scanner/StellaOps.Scanner.Worker/StellaOps.Scanner.Worker.csproj` succeeds with no circular references.
|
||||
|
||||
### NUC-003B: Wiring from native analyzer to Unknowns
|
||||
- **Goal:** convert analyzer-side identification/resolution gaps into first-class Unknowns records.
|
||||
- **Touchpoints (expected):**
|
||||
- `src/Scanner/StellaOps.Scanner.Analyzers.Native/` (where classification context is produced)
|
||||
- `src/Scanner/StellaOps.Scanner.Worker/` (where results are persisted/emitted)
|
||||
- **Acceptance criteria (minimum):**
|
||||
- A missing build-id produces `UnknownKind.MissingBuildId` with a populated `NativeUnknownContext` and is visible via existing Unknowns API surfaces.
|
||||
|
||||
---
|
||||
|
||||
@@ -58,3 +80,11 @@ Extend the Unknowns registry with native binary-specific classification reasons,
|
||||
- [ ] Binaries without build-id create MissingBuildId unknowns
|
||||
- [ ] Build-IDs not in index create UnknownBuildId unknowns
|
||||
- [ ] Unknowns emit to registry, not core SBOM
|
||||
|
||||
---
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-18 | Added unblock tasks NUC-003A/NUC-003B; NUC-004 remains BLOCKED until dependency direction + wiring are implemented. | Project Mgmt |
|
||||
|
||||
@@ -789,12 +789,12 @@ public sealed class DriftSarifGenerator
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | UI-001 | DONE | Create PathNode TypeScript interface | `path-viewer.models.ts` |
|
||||
| 2 | UI-002 | DONE | Create CompressedPath TypeScript interface | `path-viewer.models.ts` |
|
||||
| 3 | UI-003 | TODO | Create PathViewerComponent | Core visualization |
|
||||
| 4 | UI-004 | TODO | Style PathViewerComponent | SCSS styling |
|
||||
| 3 | UI-003 | DONE | Create PathViewerComponent | `components/path-viewer/` |
|
||||
| 4 | UI-004 | DONE | Style PathViewerComponent | SCSS with BEM |
|
||||
| 5 | UI-005 | DONE | Create DriftedSink TypeScript interface | `drift.models.ts` |
|
||||
| 6 | UI-006 | DONE | Create DriftResult TypeScript interface | `drift.models.ts` |
|
||||
| 7 | UI-007 | TODO | Create RiskDriftCardComponent | Summary card |
|
||||
| 8 | UI-008 | TODO | Style RiskDriftCardComponent | SCSS styling |
|
||||
| 7 | UI-007 | DONE | Create RiskDriftCardComponent | `components/risk-drift-card/` |
|
||||
| 8 | UI-008 | DONE | Style RiskDriftCardComponent | SCSS with BEM |
|
||||
| 9 | UI-009 | DONE | Create drift API service | `drift-api.service.ts` |
|
||||
| 10 | UI-010 | TODO | Integrate PathViewer into scan details | Page integration |
|
||||
| 11 | UI-011 | TODO | Integrate RiskDriftCard into PR view | Page integration |
|
||||
@@ -805,12 +805,12 @@ public sealed class DriftSarifGenerator
|
||||
| 16 | UI-016 | TODO | Implement drift attestation service | DSSE signing |
|
||||
| 17 | UI-017 | TODO | Add attestation to drift API | API integration |
|
||||
| 18 | UI-018 | TODO | Unit tests for attestation | Predicate validation |
|
||||
| 19 | UI-019 | TODO | Create DriftCommand for CLI | CLI command |
|
||||
| 20 | UI-020 | TODO | Implement table output | Spectre.Console |
|
||||
| 21 | UI-021 | TODO | Implement JSON output | JSON serialization |
|
||||
| 22 | UI-022 | TODO | Create DriftSarifGenerator | SARIF 2.1.0 |
|
||||
| 23 | UI-023 | TODO | Implement SARIF output for CLI | CLI integration |
|
||||
| 24 | UI-024 | TODO | Update CLI documentation | docs/cli/ |
|
||||
| 19 | UI-019 | DONE | Create DriftCommand for CLI | `Commands/DriftCommandGroup.cs` |
|
||||
| 20 | UI-020 | DONE | Implement table output | Spectre.Console tables |
|
||||
| 21 | UI-021 | DONE | Implement JSON output | JSON serialization |
|
||||
| 22 | UI-022 | DONE | Create DriftSarifGenerator | SARIF 2.1.0 (placeholder) |
|
||||
| 23 | UI-023 | DONE | Implement SARIF output for CLI | `CommandHandlers.Drift.cs` |
|
||||
| 24 | UI-024 | DONE | Update CLI documentation | `docs/cli/drift-cli.md` |
|
||||
| 25 | UI-025 | TODO | Integration tests for CLI | End-to-end |
|
||||
|
||||
---
|
||||
|
||||
@@ -334,20 +334,20 @@ cas://reachability/graphs/{blake3:hash}/
|
||||
|---|---------|--------|-------------|
|
||||
| 1 | RWD-001 | DONE | Create ReachabilityWitnessStatement.cs |
|
||||
| 2 | RWD-002 | DONE | Create ReachabilityWitnessOptions.cs |
|
||||
| 3 | RWD-003 | TODO | Add PredicateTypes.StellaOpsReachabilityWitness |
|
||||
| 3 | RWD-003 | DONE | Add PredicateTypes.StellaOpsReachabilityWitness |
|
||||
| 4 | RWD-004 | DONE | Create ReachabilityWitnessDsseBuilder.cs |
|
||||
| 5 | RWD-005 | DONE | Create IReachabilityWitnessPublisher.cs |
|
||||
| 6 | RWD-006 | DONE | Create ReachabilityWitnessPublisher.cs |
|
||||
| 7 | RWD-007 | TODO | Implement CAS storage integration (placeholder done) |
|
||||
| 8 | RWD-008 | TODO | Implement Rekor submission (placeholder done) |
|
||||
| 9 | RWD-009 | TODO | Integrate with RichGraphWriter |
|
||||
| 10 | RWD-010 | TODO | Add service registration |
|
||||
| 9 | RWD-009 | DONE | Integrate with RichGraphWriter (AttestingRichGraphWriter) |
|
||||
| 10 | RWD-010 | DONE | Add service registration |
|
||||
| 11 | RWD-011 | DONE | Unit tests for DSSE builder (15 tests) |
|
||||
| 12 | RWD-012 | TODO | Unit tests for publisher |
|
||||
| 12 | RWD-012 | DONE | Unit tests for publisher (8 tests) |
|
||||
| 13 | RWD-013 | TODO | Integration tests with Attestor |
|
||||
| 14 | RWD-014 | TODO | Add golden fixture: graph-only.golden.json |
|
||||
| 15 | RWD-015 | TODO | Add golden fixture: graph-with-runtime.golden.json |
|
||||
| 16 | RWD-016 | TODO | Verify deterministic DSSE output |
|
||||
| 14 | RWD-014 | DONE | Add golden fixture: graph-only.golden.json |
|
||||
| 15 | RWD-015 | DONE | Add golden fixture: graph-with-runtime.golden.json |
|
||||
| 16 | RWD-016 | DONE | Verify deterministic DSSE output (4 tests) |
|
||||
|
||||
---
|
||||
|
||||
@@ -356,6 +356,9 @@ cas://reachability/graphs/{blake3:hash}/
|
||||
| Date | Update | Owner |
|
||||
|------|--------|-------|
|
||||
| 2025-12-18 | Created ReachabilityWitnessStatement, ReachabilityWitnessOptions, ReachabilityWitnessDsseBuilder, IReachabilityWitnessPublisher, ReachabilityWitnessPublisher. Created 15 DSSE builder tests. 6/16 tasks DONE. | Agent |
|
||||
| 2025-12-18 | Added PredicateTypes.StellaOpsReachabilityWitness to Signer.Core. Created ReachabilityAttestationServiceCollectionExtensions.cs for DI. Created ReachabilityWitnessPublisherTests.cs (8 tests). 9/16 tasks DONE. | Agent |
|
||||
| 2025-12-18 | Fixed PathExplanationServiceTests.cs (RichGraph/RichGraphEdge constructor updates). Fixed RichGraphWriterTests.cs assertion. All 119 tests pass. | Agent |
|
||||
| 2025-12-18 | Created AttestingRichGraphWriter.cs for integrated attestation. Created golden fixtures. Created AttestingRichGraphWriterTests.cs (4 tests). 13/16 tasks DONE. All 123 tests pass. | Agent |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# SPRINT_3700_0001_0001 - Witness Foundation
|
||||
|
||||
**Status:** BLOCKED (2 tasks pending integration: WIT-008, WIT-009)
|
||||
**Status:** BLOCKED (WIT-008 blocked on WIT-007A/WIT-007B; WIT-009 blocked on WIT-007C/WIT-007D)
|
||||
**Priority:** P0 - CRITICAL
|
||||
**Module:** Scanner, Attestor
|
||||
**Working Directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/`
|
||||
@@ -46,14 +46,38 @@ Before starting, read:
|
||||
| 5 | WIT-005 | DONE | Create PathWitness record model |
|
||||
| 6 | WIT-006 | DONE | Create IPathWitnessBuilder interface |
|
||||
| 7 | WIT-007 | DONE | Implement PathWitnessBuilder service |
|
||||
| 8 | WIT-008 | BLOCKED | Integrate with ReachabilityAnalyzer output - requires ReachabilityAnalyzer refactoring |
|
||||
| 9 | WIT-009 | BLOCKED | Add DSSE envelope generation - requires Attestor service integration |
|
||||
| 10 | WIT-010 | DONE | Create WitnessEndpoints.cs (GET /witness/{id}, list, verify) |
|
||||
| 11 | WIT-011 | DONE | Create 013_witness_storage.sql migration |
|
||||
| 12 | WIT-012 | DONE | Create PostgresWitnessRepository + IWitnessRepository |
|
||||
| 13 | WIT-013 | DONE | Add UsesBlake3HashForDefaultProfile test to RichGraphWriterTests |
|
||||
| 14 | WIT-014 | DONE | Add PathWitnessBuilderTests |
|
||||
| 15 | WIT-015 | DONE | Create docs/contracts/witness-v1.md |
|
||||
| 8 | WIT-007A | TODO | Define ReachabilityAnalyzer → PathWitnessBuilder output contract (types, ordering, limits, fixtures) |
|
||||
| 9 | WIT-007B | TODO | Refactor ReachabilityAnalyzer to surface deterministic paths to sinks (enables witness generation) |
|
||||
| 10 | WIT-007C | TODO | Define witness predicate + DSSE payloadType constants (Attestor) and align `docs/contracts/witness-v1.md` |
|
||||
| 11 | WIT-007D | TODO | Implement DSSE sign+verify for witness payload using `StellaOps.Attestor.Envelope`; add golden fixtures |
|
||||
| 12 | WIT-008 | BLOCKED | Integrate witness generation with ReachabilityAnalyzer output (BLOCKED on WIT-007A, WIT-007B) |
|
||||
| 13 | WIT-009 | BLOCKED | Add DSSE envelope generation (BLOCKED on WIT-007C, WIT-007D) |
|
||||
| 14 | WIT-010 | DONE | Create WitnessEndpoints.cs (GET /witness/{id}, list, verify) |
|
||||
| 15 | WIT-011 | DONE | Create 013_witness_storage.sql migration |
|
||||
| 16 | WIT-012 | DONE | Create PostgresWitnessRepository + IWitnessRepository |
|
||||
| 17 | WIT-013 | DONE | Add UsesBlake3HashForDefaultProfile test to RichGraphWriterTests |
|
||||
| 18 | WIT-014 | DONE | Add PathWitnessBuilderTests |
|
||||
| 19 | WIT-015 | DONE | Create docs/contracts/witness-v1.md |
|
||||
|
||||
---
|
||||
|
||||
## Unblock Task Notes (WIT-007A..WIT-007D)
|
||||
|
||||
### WIT-007A: ReachabilityAnalyzer → witness output contract
|
||||
- **Goal:** define the exact path output shape (entrypoint → sink), including stable ordering and caps (max depth/path count) so witness generation is deterministic.
|
||||
- **Touchpoints (expected):** `src/Scanner/__Libraries/StellaOps.Scanner.CallGraph/Analysis/ReachabilityAnalyzer.cs` and `src/Scanner/__Tests/StellaOps.Scanner.CallGraph.Tests/` (fixtures + determinism assertions).
|
||||
- **Evidence:** fixture graphs + expected path lists committed and validated by tests.
|
||||
|
||||
### WIT-007B: ReachabilityAnalyzer refactor (sink-aware + path export)
|
||||
- **Acceptance criteria (minimum):** analyzer accepts explicit sinks and returns deterministic path(s) per reachable sink without breaking existing tests/behaviour.
|
||||
|
||||
### WIT-007C: Witness predicate + DSSE payloadType constants
|
||||
- **Goal:** remove ambiguity about predicate URI/media type; Scanner/Attestor must sign/verify the same bytes.
|
||||
- **Touchpoints (expected):** `src/Attestor/StellaOps.Attestor/Predicates/` and `docs/contracts/witness-v1.md`.
|
||||
|
||||
### WIT-007D: DSSE signing + verification for witnesses
|
||||
- **Preferred implementation:** use `src/Attestor/StellaOps.Attestor.Envelope/` (serializer + `EnvelopeSignatureService`) for Ed25519 first.
|
||||
- **Evidence:** golden fixture payload + DSSE envelope + public key, plus unit tests proving deterministic serialization and successful verification.
|
||||
|
||||
---
|
||||
|
||||
@@ -345,7 +369,7 @@ public static class WitnessPredicates
|
||||
- [x] All existing RichGraph tests pass
|
||||
- [x] PathWitness model serializes correctly
|
||||
- [x] PathWitnessBuilder generates valid witnesses
|
||||
- [ ] DSSE signatures verify correctly (BLOCKED: WIT-009)
|
||||
- [ ] DSSE signatures verify correctly (BLOCKED: WIT-009; blocked on WIT-007C/WIT-007D)
|
||||
- [x] `/witness/{id}` endpoint returns witness JSON
|
||||
- [x] Documentation complete
|
||||
|
||||
@@ -358,8 +382,8 @@ public static class WitnessPredicates
|
||||
| WIT-DEC-001 | Use Blake3.NET library | Well-tested, MIT license |
|
||||
| WIT-DEC-002 | Store witnesses in Postgres JSONB | Flexible queries, no separate store |
|
||||
| WIT-DEC-003 | Ed25519 signatures only | Simplicity, Ed25519 is default for DSSE |
|
||||
| WIT-DEC-004 | Defer ReachabilityAnalyzer integration | Requires understanding of call flow; new sprint needed |
|
||||
| WIT-DEC-005 | Defer DSSE signing to Attestor sprint | DSSE signing belongs in Attestor module |
|
||||
| WIT-DEC-004 | Convert ReachabilityAnalyzer blocker into explicit tasks | Track contract+refactor as WIT-007A/WIT-007B; keep WIT-008 BLOCKED until complete |
|
||||
| WIT-DEC-005 | Convert DSSE signing blocker into explicit tasks | Track predicate+sign/verify as WIT-007C/WIT-007D; keep WIT-009 BLOCKED until complete |
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
@@ -381,3 +405,4 @@ public static class WitnessPredicates
|
||||
| 2025-12-18 | Completed WIT-010: Created WitnessEndpoints.cs with GET /witnesses/{id}, list (by scan/cve/graphHash), by-hash, verify endpoints | Agent |
|
||||
| 2025-12-18 | Registered MapWitnessEndpoints() in Scanner.WebService Program.cs | Agent |
|
||||
| 2025-12-18 | Completed WIT-013: Added UsesBlake3HashForDefaultProfile test to RichGraphWriterTests.cs | Agent |
|
||||
| 2025-12-18 | Added unblock tasks WIT-007A..WIT-007D and updated WIT-008/WIT-009 dependencies accordingly. | Project Mgmt |
|
||||
|
||||
@@ -101,17 +101,17 @@ Before starting, read:
|
||||
| 11 | SURF-011 | TODO | Implement PythonAstFingerprinter |
|
||||
| 12 | SURF-012 | TODO | Create MethodKey normalizer per ecosystem |
|
||||
| 13 | SURF-013 | DONE | Create MethodDiffEngine service |
|
||||
| 14 | SURF-014 | TODO | Create 011_vuln_surfaces.sql migration |
|
||||
| 14 | SURF-014 | DONE | Create 014_vuln_surfaces.sql migration |
|
||||
| 15 | SURF-015 | DONE | Create VulnSurface, VulnSurfaceSink models |
|
||||
| 16 | SURF-016 | TODO | Create PostgresVulnSurfaceRepository |
|
||||
| 16 | SURF-016 | DONE | Create PostgresVulnSurfaceRepository |
|
||||
| 17 | SURF-017 | DONE | Create VulnSurfaceBuilder orchestrator service |
|
||||
| 18 | SURF-018 | DONE | Create IVulnSurfaceBuilder interface |
|
||||
| 19 | SURF-019 | TODO | Add surface builder metrics |
|
||||
| 20 | SURF-020 | TODO | Create NuGetDownloaderTests |
|
||||
| 21 | SURF-021 | TODO | Create CecilFingerprinterTests |
|
||||
| 22 | SURF-022 | TODO | Create MethodDiffEngineTests |
|
||||
| 19 | SURF-019 | DONE | Add surface builder metrics |
|
||||
| 20 | SURF-020 | DONE | Create NuGetDownloaderTests (9 tests) |
|
||||
| 21 | SURF-021 | DONE | Create CecilFingerprinterTests (7 tests) |
|
||||
| 22 | SURF-022 | DONE | Create MethodDiffEngineTests (8 tests) |
|
||||
| 23 | SURF-023 | TODO | Integration test with real CVE (Newtonsoft.Json) |
|
||||
| 24 | SURF-024 | TODO | Create docs/contracts/vuln-surface-v1.md |
|
||||
| 24 | SURF-024 | DONE | Create docs/contracts/vuln-surface-v1.md |
|
||||
|
||||
---
|
||||
|
||||
@@ -447,3 +447,6 @@ Expected Changed Methods:
|
||||
| Date (UTC) | Update | Owner |
|
||||
|---|---|---|
|
||||
| 2025-12-18 | Created sprint from advisory analysis | Agent |
|
||||
| 2025-12-18 | Created CecilMethodFingerprinterTests.cs (7 tests) and MethodDiffEngineTests.cs (8 tests). 12/24 tasks DONE. All 26 VulnSurfaces tests pass. | Agent |
|
||||
| 2025-12-18 | Created NuGetPackageDownloaderTests.cs (9 tests). Fixed IVulnSurfaceRepository interface/implementation mismatch. Added missing properties to VulnSurfaceSink model. 19/24 tasks DONE. All 35 VulnSurfaces tests pass. | Agent |
|
||||
| 2025-12-18 | Created VulnSurfaceMetrics.cs with counters, histograms, and gauges. Integrated metrics into VulnSurfaceBuilder. 20/24 tasks DONE. | Agent |
|
||||
Reference in New Issue
Block a user