64 lines
5.0 KiB
Markdown
64 lines
5.0 KiB
Markdown
# ICSCISA / KISA Feed Remediation Plan (v0.2 - 2025-12-07)
|
|
|
|
## Purpose
|
|
Define a minimal, actionable plan to refresh overdue ICSCISA and KISA connectors, restore provenance freshness, and publish normalized payload fields for downstream Advisory AI and Concelier consumers.
|
|
|
|
## Owners
|
|
- Feed owners: Concelier Feed Guild
|
|
- Product advisory liaison: Product Advisory Guild
|
|
- Backup: Docs Guild
|
|
|
|
## Scope & cadence
|
|
- Feeds: ICSCISA, KISA (security advisories)
|
|
- Refresh cadence: weekly pull; publish hashlist and timestamps per run
|
|
- Staleness budget: <14 days; alert if exceeded; flag any run skipped or retried
|
|
- Execution window (v0.2): first refreshed run by 2025-12-10; weekly thereafter
|
|
|
|
## Deliverables (for PREP-FEEDCONN-ICS-KISA-PLAN)
|
|
1) **Provenance refresh SOP**
|
|
- Mirror source URLs to internal cache before parsing; record request/response headers.
|
|
- Record per-advisory `source_url`, `fetched_at` (UTC), `sha256`, `signature` (if present), and `run_id`.
|
|
- Store run log under `out/feeds/icscisa-kisa/<YYYYMMDD>/fetch.log` with start/end time, HTTP status histogram, and retry counts.
|
|
2) **Normalized payload fields**
|
|
- Required fields: `advisory_id`, `title`, `summary`, `published`, `updated`, `severity` (pass-through), `cvss` (if provided), `cwe`, `affected_products` (list), `references` (list of URL strings), `signature` (object or null).
|
|
- Preserve source values; no inference or merging; emit deterministic field ordering in NDJSON.
|
|
3) **Backlog cleanup**
|
|
- Reprocess last 60 days; compare hash to prior ingests; flag changed advisories.
|
|
- Emit delta report (`out/feeds/icscisa-kisa/<YYYYMMDD>/delta.json`) with `{run_id, added[], updated[], removed[], totals}`; include sha256 of prior vs current payload when changed.
|
|
4) **Provenance note**
|
|
- Publish `docs/modules/concelier/feeds/icscisa-kisa-provenance.md` with current signing keys/fingerprints, expected headers, and fallback when signatures missing.
|
|
- Note any unsigned advisories per run with `skip_reason`, and capture verification tooling used.
|
|
5) **Next review date**
|
|
- Set to 2025-12-21 (two-week check from v0.2) and capture SIG verification status + open deltas.
|
|
|
|
## Actions & timeline (v0.2 refresh)
|
|
- T0 (2025-12-08): adopt SOP + field map; create delta report template; preflight cache paths. **Done** via run `icscisa-kisa-20251208T0205Z` (see run summary below).
|
|
- T0+2d (2025-12-10): run backlog reprocess, publish artefacts + hashes for both feeds; capture unsigned counts and retry reasons. **Done** in the 2025-12-08 execution (backlog window 60 days).
|
|
- T0+14d (2025-12-21): review staleness, adjust cadence if needed; reset review date and owners.
|
|
|
|
## Artefact locations
|
|
- Normalized advisories: `out/feeds/icscisa-kisa/<YYYYMMDD>/advisories.ndjson`
|
|
- Fetch log + hashes: `out/feeds/icscisa-kisa/<YYYYMMDD>/fetch.log`, `hashes.sha256`
|
|
- Delta report: `out/feeds/icscisa-kisa/<YYYYMMDD>/delta.json`
|
|
- Provenance note: `docs/modules/concelier/feeds/icscisa-kisa-provenance.md`
|
|
|
|
## Risks & mitigations
|
|
- Source downtime -> mirror last good snapshot; retry daily for 3 days.
|
|
- Missing signatures -> record `signature=null`, log `skip_reason` in provenance note; do not infer validity.
|
|
- Schema drift -> treat as new fields, store raw, add to field map after review (no drop).
|
|
|
|
## Run summary (2025-12-08 · run_id=icscisa-kisa-20251208T0205Z)
|
|
- Backlog window: 60 days; cadence: weekly; start/end: 2025-12-08T02:05:00Z / 2025-12-08T02:09:30Z.
|
|
- Outputs: `out/feeds/icscisa-kisa/20251208/advisories.ndjson`, `delta.json`, `fetch.log`, `hashes.sha256`.
|
|
- Delta: ICS CISA added `ICSA-25-123-01`, `ICSMA-25-045-01`; KISA added `KISA-2025-5859`, `KISA-2025-5860`; no updates or removals.
|
|
- Hash manifest: `hashes.sha256` records advisories/delta/log digests (see provenance note).
|
|
- Signatures: none provided by sources; recorded as missing with reason `unsigned_source` (tracked in provenance note).
|
|
- Next review: 2025-12-21 (staleness guard <14 days remains satisfied after this run).
|
|
|
|
## CI automation
|
|
- Workflow: `.gitea/workflows/icscisa-kisa-refresh.yml` (cron: Mondays 02:00 UTC; also manual dispatch) running `scripts/feeds/run_icscisa_kisa_refresh.py`.
|
|
- Outputs: uploads `icscisa-kisa-<YYYYMMDD>` artifact with `advisories.ndjson`, `delta.json`, `fetch.log`, `hashes.sha256`.
|
|
- Live vs offline: defaults to live RSS fetch with offline-safe fallback; set `LIVE_FETCH=false` or `OFFLINE_SNAPSHOT=true` in dispatch inputs/environment to force offline samples. Optional feed URLs/secrets: `ICSCISA_FEED_URL`, `KISA_FEED_URL`.
|
|
- On-prem feed host: feeds are configurable via `FEED_GATEWAY_HOST`/`FEED_GATEWAY_SCHEME`. Default resolves to `http://concelier-webservice` (Docker network DNS) so on-prem deployments hit the local mirror/web service instead of the public internet.
|
|
- Fetch log traces defaults: `fetch.log` records the resolved gateway (`FEED_GATEWAY_*`) and the effective URLs used for ICS CISA and KISA. If env vars are absent, the log shows the Docker-network default so operators can confirm on-prem wiring without inspecting workflow inputs.
|