Refactor code structure for improved readability and maintainability; optimize performance in key functions.
This commit is contained in:
49
docs/modules/concelier/operations/connectors/epss.md
Normal file
49
docs/modules/concelier/operations/connectors/epss.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Concelier EPSS Connector Operations
|
||||
|
||||
This playbook covers deployment and monitoring of the EPSS connector that ingests daily FIRST.org EPSS snapshots.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Network egress to `https://epss.empiricalsecurity.com/` (or a mirrored endpoint).
|
||||
- Updated `concelier.yaml` (or environment variables) with the EPSS source configuration:
|
||||
|
||||
```yaml
|
||||
concelier:
|
||||
sources:
|
||||
epss:
|
||||
baseUri: "https://epss.empiricalsecurity.com/"
|
||||
fetchCurrent: true
|
||||
catchUpDays: 7
|
||||
httpTimeout: "00:02:00"
|
||||
maxRetries: 3
|
||||
airgapMode: false
|
||||
bundlePath: "/var/stellaops/bundles/epss"
|
||||
```
|
||||
|
||||
## 2. Smoke Test (staging)
|
||||
|
||||
1. Restart Concelier workers after configuration changes.
|
||||
2. Trigger a full cycle:
|
||||
- CLI: `stella db jobs run source:epss:fetch --and-then source:epss:parse --and-then source:epss:map`
|
||||
- REST: `POST /jobs/run { "kind": "source:epss:fetch", "chain": ["source:epss:parse", "source:epss:map"] }`
|
||||
3. Verify document status transitions: `pending_parse` -> `pending_map` -> `mapped`.
|
||||
4. Confirm log entries for `Fetched EPSS snapshot` and parse/map summaries.
|
||||
|
||||
## 3. Monitoring
|
||||
|
||||
- **Meter**: `StellaOps.Concelier.Connector.Epss`
|
||||
- **Key counters**:
|
||||
- `epss.fetch.attempts`, `epss.fetch.success`, `epss.fetch.failures`, `epss.fetch.unchanged`
|
||||
- `epss.parse.rows`, `epss.parse.failures`
|
||||
- `epss.map.rows`
|
||||
- **Alert suggestions**:
|
||||
- `rate(epss_fetch_failures_total[15m]) > 0`
|
||||
- `rate(epss_map_rows_total[1h]) == 0` during business hours while other connectors are active
|
||||
|
||||
## 4. Airgap Mode
|
||||
|
||||
- Place snapshots in the bundle directory:
|
||||
- `epss_scores-YYYY-MM-DD.csv.gz`
|
||||
- Optional `manifest.json` listing `name`, `modelVersion`, `sha256`, and `rowCount`.
|
||||
- Set `airgapMode: true` and `bundlePath` to the directory or specific file.
|
||||
- The connector validates the manifest hash when present and logs warnings on mismatch.
|
||||
Reference in New Issue
Block a user