# EPSS v4 Integration Guide ## Overview EPSS (Exploit Prediction Scoring System) v4 is a machine learning-based vulnerability scoring system developed by FIRST.org that predicts the probability a CVE will be exploited in the wild within the next 30 days. StellaOps integrates EPSS as a **probabilistic threat signal** alongside CVSS v4's **deterministic severity assessment**, enabling more accurate vulnerability prioritization. **Key Concepts**: - **EPSS Score**: Probability (0.0-1.0) that a CVE will be exploited in next 30 days - **EPSS Percentile**: Ranking (0.0-1.0) of this CVE relative to all scored CVEs - **Model Date**: Date for which EPSS scores were computed - **Immutable at-scan**: EPSS evidence captured at scan time never changes (deterministic replay) - **Current EPSS**: Live projection for triage (updated daily) --- ## How EPSS Works EPSS uses machine learning to predict exploitation probability based on: 1. **Vulnerability Characteristics**: CVSS metrics, CWE, affected products 2. **Social Signals**: Twitter/GitHub mentions, security blog posts 3. **Exploit Database Entries**: Exploit-DB, Metasploit, etc. 4. **Historical Exploitation**: Past exploitation patterns EPSS is updated **daily** by FIRST.org based on fresh threat intelligence. ### EPSS vs CVSS | Dimension | CVSS v4 | EPSS v4 | |-----------|---------|---------| | **Nature** | Deterministic severity | Probabilistic threat | | **Scale** | 0.0-10.0 (severity) | 0.0-1.0 (probability) | | **Update Frequency** | Static (per CVE version) | Daily (live threat data) | | **Purpose** | Impact assessment | Likelihood assessment | | **Source** | Vendor/NVD | FIRST.org ML model | **Example**: - **CVE-2024-1234**: CVSS 9.8 (Critical) + EPSS 0.01 (1st percentile) - Interpretation: Severe impact if exploited, but very unlikely to be exploited - Priority: **Medium** (deprioritize despite high CVSS) - **CVE-2024-5678**: CVSS 6.5 (Medium) + EPSS 0.95 (98th percentile) - Interpretation: Moderate impact, but actively being exploited - Priority: **High** (escalate despite moderate CVSS) --- ## Architecture Overview ### Data Flow ``` ┌────────────────────────────────────────────────────────────────┐ │ EPSS Data Lifecycle in StellaOps │ └────────────────────────────────────────────────────────────────┘ 1. INGESTION (Daily 00:05 UTC) ┌───────────────────┐ │ FIRST.org │ Daily CSV: epss_scores-YYYY-MM-DD.csv.gz │ (300k CVEs) │ ~15MB compressed └────────┬──────────┘ │ ▼ ┌───────────────────────────────────────────────────────────┐ │ Concelier: EpssIngestJob │ │ - Download/Import CSV │ │ - Parse (handle # comment, validate bounds) │ │ - Bulk insert: epss_scores (partitioned by month) │ │ - Compute delta: epss_changes (flags for enrichment) │ │ - Upsert: epss_current (latest projection) │ │ - Emit event: "epss.updated" │ └────────┬──────────────────────────────────────────────────┘ │ ▼ [PostgreSQL: concelier.epss_*] │ ├─────────────────────────────┐ │ │ ▼ ▼ 2. AT-SCAN CAPTURE (Immutable Evidence) ┌────────────────────────────────────────────────────────────┐ │ Scanner: On new scan │ │ - Bulk query: epss_current for CVE list │ │ - Store immutable evidence: │ │ * epss_score_at_scan │ │ * epss_percentile_at_scan │ │ * epss_model_date_at_scan │ │ * epss_import_run_id_at_scan │ │ - Use in lattice decision (SR→CR if EPSS≥90th) │ └─────────────────────────────────────────────────────────────┘ 3. LIVE ENRICHMENT (Existing Findings) ┌─────────────────────────────────────────────────────────────┐ │ Concelier: EpssEnrichmentJob (on "epss.updated") │ │ - Read: epss_changes WHERE flags IN (CROSSED_HIGH, BIG_JUMP)│ │ - Find impacted: vuln_instance_triage BY cve_id │ │ - Update: current_epss_score, current_epss_percentile │ │ - If priority band changed → emit "vuln.priority.changed" │ └────────┬────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Notify: On "vuln.priority.changed" │ │ - Check tenant notification rules │ │ - Send: Slack / Email / Teams / In-app │ │ - Payload: EPSS delta, threshold crossed │ └─────────────────────────────────────────────────────────────┘ 4. POLICY SCORING ┌─────────────────────────────────────────────────────────────┐ │ Policy Engine: Risk Score Formula │ │ risk_score = (cvss/10) + epss_bonus + kev_bonus + reach_mult│ │ │ │ EPSS Bonus (Simple Profile): │ │ - Percentile ≥99th: +10% │ │ - Percentile ≥90th: +5% │ │ - Percentile ≥50th: +2% │ │ - Percentile <50th: 0% │ │ │ │ VEX Lattice Rules: │ │ - SR + EPSS≥90th → Escalate to CR (Confirmed Reachable) │ │ - DV + EPSS≥95th → Flag for review (vendor denial) │ │ - U + EPSS≥95th → Prioritize for reachability analysis │ └─────────────────────────────────────────────────────────────┘ ``` ### Database Schema **Location**: `concelier` database #### epss_import_runs (Provenance) Tracks each EPSS import with full provenance for audit trail. ```sql CREATE TABLE concelier.epss_import_runs ( import_run_id UUID PRIMARY KEY, model_date DATE NOT NULL UNIQUE, source_uri TEXT NOT NULL, file_sha256 TEXT NOT NULL, row_count INT NOT NULL, model_version_tag TEXT NULL, published_date DATE NULL, status TEXT NOT NULL, -- IN_PROGRESS, SUCCEEDED, FAILED created_at TIMESTAMPTZ NOT NULL ); ``` #### epss_scores (Time-Series, Partitioned) Immutable append-only history of daily EPSS scores. ```sql CREATE TABLE concelier.epss_scores ( model_date DATE NOT NULL, cve_id TEXT NOT NULL, epss_score DOUBLE PRECISION NOT NULL, percentile DOUBLE PRECISION NOT NULL, import_run_id UUID NOT NULL, PRIMARY KEY (model_date, cve_id) ) PARTITION BY RANGE (model_date); ``` **Partitions**: Monthly (e.g., `epss_scores_2025_12`) #### epss_current (Latest Projection) Materialized view of latest EPSS score per CVE for fast lookups. ```sql CREATE TABLE concelier.epss_current ( cve_id TEXT PRIMARY KEY, epss_score DOUBLE PRECISION NOT NULL, percentile DOUBLE PRECISION NOT NULL, model_date DATE NOT NULL, import_run_id UUID NOT NULL, updated_at TIMESTAMPTZ NOT NULL ); ``` **Usage**: Scanner bulk queries this table for new scans. #### epss_changes (Delta Tracking, Partitioned) Tracks material EPSS changes for targeted enrichment. ```sql CREATE TABLE concelier.epss_changes ( model_date DATE NOT NULL, cve_id TEXT NOT NULL, old_score DOUBLE PRECISION NULL, new_score DOUBLE PRECISION NOT NULL, delta_score DOUBLE PRECISION NULL, old_percentile DOUBLE PRECISION NULL, new_percentile DOUBLE PRECISION NOT NULL, delta_percentile DOUBLE PRECISION NULL, flags INT NOT NULL, -- Bitmask PRIMARY KEY (model_date, cve_id) ) PARTITION BY RANGE (model_date); ``` **Flags** (bitmask): - `1` = NEW_SCORED (CVE newly appeared) - `2` = CROSSED_HIGH (percentile ≥95th) - `4` = BIG_JUMP (|Δscore| ≥0.10) - `8` = DROPPED_LOW (percentile <50th) - `16` = SCORE_INCREASED - `32` = SCORE_DECREASED --- ## Configuration ### Scheduler Configuration **File**: `etc/scheduler.yaml` ```yaml scheduler: jobs: - name: epss.ingest schedule: "0 5 0 * * *" # Daily at 00:05 UTC worker: concelier args: source: online date: null # Auto: yesterday timeout: 600s retry: max_attempts: 3 backoff: exponential ``` ### Concelier Configuration **File**: `etc/concelier.yaml` ```yaml concelier: epss: enabled: true online_source: base_url: "https://epss.empiricalsecurity.com/" url_pattern: "epss_scores-{date:yyyy-MM-dd}.csv.gz" timeout: 180s bundle_source: path: "/opt/stellaops/bundles/epss/" thresholds: high_percentile: 0.95 # Top 5% high_score: 0.50 # 50% probability big_jump_delta: 0.10 # 10 percentage points low_percentile: 0.50 # Median enrichment: enabled: true batch_size: 1000 flags_to_process: - NEW_SCORED - CROSSED_HIGH - BIG_JUMP ``` ### Scanner Configuration **File**: `etc/scanner.yaml` ```yaml scanner: epss: enabled: true provider: postgres cache_ttl: 3600 fallback_on_missing: unknown # Options: unknown, zero, skip ``` ### Policy Configuration **File**: `etc/policy.yaml` ```yaml policy: scoring: epss: enabled: true profile: simple # Options: simple, advanced, custom simple_bonuses: percentile_99: 0.10 # +10% percentile_90: 0.05 # +5% percentile_50: 0.02 # +2% lattice: epss_escalation: enabled: true sr_to_cr_threshold: 0.90 # SR→CR if EPSS≥90th percentile ``` --- ## Daily Operation ### Automated Ingestion EPSS data is ingested automatically daily at **00:05 UTC** via Scheduler. **Workflow**: 1. Scheduler triggers `epss.ingest` job at 00:05 UTC 2. Concelier downloads `epss_scores-YYYY-MM-DD.csv.gz` from FIRST.org 3. CSV parsed (comment line → metadata, rows → scores) 4. Bulk insert into `epss_scores` partition (NpgsqlBinaryImporter) 5. Compute delta: `epss_changes` (compare vs `epss_current`) 6. Upsert `epss_current` (latest projection) 7. Emit `epss.updated` event 8. Enrichment job updates impacted vulnerability instances 9. Notifications sent if priority bands changed **Monitoring**: ```bash # Check latest model date stellaops epss status # Output: # EPSS Status: # Latest Model Date: 2025-12-16 # Import Time: 2025-12-17 00:07:32 UTC # CVE Count: 231,417 # Staleness: FRESH (1 day) ``` ### Manual Triggering ```bash # Trigger manual ingest (force re-import) stellaops concelier job trigger epss.ingest --date 2025-12-16 --force # Backfill historical data (last 30 days) stellaops epss backfill --from 2025-11-17 --to 2025-12-16 ``` --- ## Air-Gapped Operation ### Bundle Structure EPSS data for offline deployments is packaged in risk bundles: ``` risk-bundle-2025-12-16/ ├── manifest.json ├── epss/ │ ├── epss_scores-2025-12-16.csv.zst # ZSTD compressed │ └── epss_metadata.json ├── kev/ │ └── kev-catalog.json └── signatures/ └── bundle.dsse.json ``` ### EPSS Metadata **File**: `epss/epss_metadata.json` ```json { "model_date": "2025-12-16", "model_version": "v2025.12.16", "published_date": "2025-12-16", "row_count": 231417, "sha256": "abc123...", "source_uri": "https://epss.empiricalsecurity.com/epss_scores-2025-12-16.csv.gz", "created_at": "2025-12-16T00:00:00Z" } ``` ### Import Procedure ```bash # 1. Transfer bundle to air-gapped system scp risk-bundle-2025-12-16.tar.zst airgap-host:/opt/stellaops/bundles/ # 2. Import bundle stellaops offline import --bundle /opt/stellaops/bundles/risk-bundle-2025-12-16.tar.zst # 3. Verify import stellaops epss status # Output: # EPSS Status: # Latest Model Date: 2025-12-16 # Source: bundle://risk-bundle-2025-12-16 # CVE Count: 231,417 # Staleness: ACCEPTABLE (within 7 days) ``` ### Update Cadence **Recommended**: - **Online**: Daily (automatic) - **Air-gapped**: Weekly (manual bundle import) **Staleness Thresholds**: - **FRESH**: ≤1 day - **ACCEPTABLE**: ≤7 days - **STALE**: ≤14 days - **VERY_STALE**: >14 days (alert, fallback to CVSS-only) --- ## Scanner Integration ### EPSS Evidence in Scan Findings Every scan finding includes **immutable EPSS-at-scan** evidence: ```json { "finding_id": "CVE-2024-12345-pkg:npm/lodash@4.17.21", "cve_id": "CVE-2024-12345", "product": "pkg:npm/lodash@4.17.21", "scan_id": "scan-abc123", "scan_timestamp": "2025-12-17T10:30:00Z", "evidence": { "cvss_v4": { "vector_string": "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:H/SI:H/SA:H", "base_score": 9.3, "severity": "CRITICAL" }, "epss_at_scan": { "epss_score": 0.42357, "percentile": 0.88234, "model_date": "2025-12-16", "import_run_id": "550e8400-e29b-41d4-a716-446655440000" }, "epss_current": { "epss_score": 0.45123, "percentile": 0.89456, "model_date": "2025-12-17", "delta_score": 0.02766, "delta_percentile": 0.01222, "trend": "RISING" } } } ``` **Key Points**: - **epss_at_scan**: Immutable, captured at scan time (deterministic replay) - **epss_current**: Mutable, updated daily for live triage - **Replay**: Historical scans always use `epss_at_scan` for consistent policy evaluation ### Bulk Query Optimization Scanner queries EPSS for all CVEs in a single database call: ```sql SELECT cve_id, epss_score, percentile, model_date, import_run_id FROM concelier.epss_current WHERE cve_id = ANY(@cve_ids); ``` **Performance**: <500ms for 10k CVEs (P95) --- ## Policy Engine Integration ### Risk Score Formula **Simple Profile**: ``` risk_score = (cvss_base / 10) + epss_bonus + kev_bonus ``` **EPSS Bonus Table**: | EPSS Percentile | Bonus | Rationale | |----------------|-------|-----------| | ≥99th | +10% | Top 1% most likely to be exploited | | ≥90th | +5% | Top 10% high exploitation probability | | ≥50th | +2% | Above median moderate risk | | <50th | 0% | Below median no bonus | **Advanced Profile**: Adds: - **KEV synergy**: If in KEV catalog → multiply EPSS bonus by 1.5 - **Uncertainty penalty**: Missing EPSS → -5% - **Temporal decay**: EPSS >30 days stale → reduce bonus by 50% ### VEX Lattice Rules **Escalation**: - **SR (Static Reachable) + EPSS≥90th** → Auto-escalate to **CR (Confirmed Reachable)** - Rationale: High exploit probability warrants confirmation **Review Flags**: - **DV (Denied by Vendor VEX) + EPSS≥95th** → Flag for manual review - Rationale: Vendor denial contradicted by active exploitation signals **Prioritization**: - **U (Unknown) + EPSS≥95th** → Prioritize for reachability analysis - Rationale: High exploit probability justifies effort ### SPL (Stella Policy Language) Syntax ```yaml # Custom policy using EPSS rules: - name: high_epss_escalation condition: | epss.percentile >= 0.95 AND lattice.state == "SR" AND runtime.exposed == true action: escalate_to_cr reason: "High EPSS (top 5%) + Static Reachable + Runtime Exposed" - name: epss_trend_alert condition: | epss.delta_score >= 0.10 AND cvss.base_score >= 7.0 action: notify channels: [slack, email] reason: "EPSS jumped by 10+ points (was {epss.old_score}, now {epss.new_score})" ``` **Available Fields**: - `epss.score` - Current EPSS score (0.0-1.0) - `epss.percentile` - Current percentile (0.0-1.0) - `epss.model_date` - Model date - `epss.delta_score` - Change vs previous scan - `epss.trend` - RISING, FALLING, STABLE - `epss.at_scan.score` - Immutable score at scan time - `epss.at_scan.percentile` - Immutable percentile at scan time --- ## Notification Integration ### Event: vuln.priority.changed Emitted when EPSS change causes priority band shift. **Payload**: ```json { "event_type": "vuln.priority.changed", "vulnerability_id": "CVE-2024-12345", "product_key": "pkg:npm/lodash@4.17.21", "old_priority_band": "medium", "new_priority_band": "high", "reason": "EPSS percentile crossed 95th (was 88th, now 96th)", "epss_change": { "old_score": 0.42, "new_score": 0.78, "delta_score": 0.36, "old_percentile": 0.88, "new_percentile": 0.96, "model_date": "2025-12-16" } } ``` ### Notification Rules **File**: `etc/notify.yaml` ```yaml notify: rules: - name: epss_crossed_high event_type: vuln.priority.changed condition: "payload.epss_change.new_percentile >= 0.95" channels: [slack, email] template: epss_high_alert digest: false # Immediate - name: epss_big_jump event_type: vuln.priority.changed condition: "payload.epss_change.delta_score >= 0.10" channels: [slack] template: epss_rising_threat digest: true digest_time: "09:00" # Daily digest at 9 AM ``` ### Slack Template Example ``` 🚨 **High EPSS Alert** **CVE**: CVE-2024-12345 **Product**: pkg:npm/lodash@4.17.21 **EPSS**: 0.78 (96th percentile) ⬆️ from 0.42 (88th percentile) **Delta**: +0.36 (36 percentage points) **Priority**: Medium → **High** **Action Required**: Review and prioritize remediation. [View in StellaOps →](https://stellaops.example.com/vulns/CVE-2024-12345) ``` --- ## Troubleshooting ### EPSS Data Not Available **Symptom**: Scans show "EPSS: N/A" **Diagnosis**: ```bash # Check EPSS status stellaops epss status # Check import runs stellaops concelier jobs list --type epss.ingest --limit 10 ``` **Resolution**: 1. **No imports**: Trigger manual ingest ```bash stellaops concelier job trigger epss.ingest ``` 2. **Import failed**: Check logs ```bash stellaops concelier logs --job-id --level ERROR ``` 3. **FIRST.org down**: Use air-gapped bundle ```bash stellaops offline import --bundle /path/to/risk-bundle.tar.zst ``` ### Stale EPSS Data **Symptom**: UI shows "EPSS stale (14 days)" **Diagnosis**: ```sql SELECT * FROM concelier.epss_model_staleness; -- Output: days_stale: 14, staleness_status: STALE ``` **Resolution**: 1. **Online**: Check scheduler job status ```bash stellaops scheduler jobs status epss.ingest ``` 2. **Air-gapped**: Import fresh bundle ```bash stellaops offline import --bundle /path/to/latest-bundle.tar.zst ``` 3. **Fallback**: Disable EPSS temporarily (uses CVSS-only) ```yaml # etc/scanner.yaml scanner: epss: enabled: false ``` ### High Memory Usage During Ingest **Symptom**: Concelier worker OOM during EPSS ingest **Diagnosis**: ```bash # Check memory metrics stellaops metrics query 'process_resident_memory_bytes{service="concelier"}' ``` **Resolution**: 1. **Increase worker memory limit**: ```yaml # Kubernetes deployment resources: limits: memory: 1Gi # Was 512Mi ``` 2. **Verify streaming parser** (should not load full CSV into memory): ```bash # Check logs for "EPSS CSV parsed: rows_yielded=" stellaops concelier logs --job-type epss.ingest | grep "CSV parsed" ``` --- ## Best Practices ### 1. Combine Signals (Never Use EPSS Alone) ❌ **Don't**: `if epss > 0.95 then CRITICAL` ✅ **Do**: `if cvss >= 8.0 AND epss >= 0.95 AND runtime_exposed then CRITICAL` ### 2. Review High EPSS Manually Manually review vulnerabilities with EPSS ≥95th percentile, especially if: - CVSS is low (<7.0) but EPSS is high - Vendor VEX denies exploitability but EPSS is high ### 3. Track Trends Monitor EPSS changes over time: - Rising EPSS → increasing threat - Falling EPSS → threat subsiding ### 4. Update Regularly - **Online**: Daily (automatic) - **Air-gapped**: Weekly minimum, daily preferred ### 5. Verify During Audits For compliance audits, use EPSS-at-scan (immutable) not current EPSS: ```sql SELECT epss_score_at_scan, epss_model_date_at_scan FROM scan_findings WHERE scan_id = 'audit-scan-20251217'; ``` --- ## API Reference ### Query Current EPSS ```bash # Single CVE stellaops epss get CVE-2024-12345 # Output: # CVE-2024-12345 # Score: 0.42357 (42.4% probability) # Percentile: 88.2th # Model Date: 2025-12-16 # Status: FRESH ``` ### Batch Query ```bash # From file stellaops epss batch --file cves.txt --output epss-scores.json # cves.txt: # CVE-2024-1 # CVE-2024-2 # CVE-2024-3 ``` ### Query History ```bash # Last 180 days stellaops epss history CVE-2024-12345 --days 180 --format csv # Output: epss-history-CVE-2024-12345.csv # model_date,epss_score,percentile # 2025-12-17,0.45123,0.89456 # 2025-12-16,0.42357,0.88234 # ... ``` ### Top CVEs by EPSS ```bash # Top 100 stellaops epss top --limit 100 --format table # Output: # Rank | CVE | Score | Percentile | CVSS # -----|---------------|--------|------------|------ # 1 | CVE-2024-9999 | 0.9872 | 99.9th | 9.8 # 2 | CVE-2024-8888 | 0.9654 | 99.8th | 8.1 # ... ``` --- ## References - **FIRST EPSS Homepage**: https://www.first.org/epss/ - **EPSS Data & Stats**: https://www.first.org/epss/data_stats - **EPSS API Docs**: https://www.first.org/epss/api - **CVSS v4.0 Spec**: https://www.first.org/cvss/v4.0/specification-document - **StellaOps Policy Guide**: `docs/policy/overview.md` - **StellaOps Reachability Guide**: `docs/modules/scanner/reachability.md` --- **Last Updated**: 2025-12-17 **Version**: 1.0 **Maintainer**: StellaOps Security Team