up
This commit is contained in:
@@ -1,7 +1,22 @@
|
||||
# ICSCISA / KISA Feed Provenance Notes (2025-11-19)
|
||||
# ICSCISA / KISA Feed Provenance Notes (2025-12-08)
|
||||
|
||||
- Expected signing: not provided by sources; set `signature=null` and `skip_reason="unsigned"`.
|
||||
- Hashing: sha256 of raw advisory payload before normalization.
|
||||
- Expected signing: not provided by sources; record `signature` as `{ status: "missing", reason: "unsigned_source" }`.
|
||||
- Hashing: sha256 of raw advisory payload before normalization (stored as `payload_sha256` per advisory) and sha256 of run artefacts (`hashes.sha256`).
|
||||
- Transport: HTTPS; mirror to internal cache; record `fetched_at` UTC and `source_url`.
|
||||
- Verification: compare hash vs previous run; emit delta report.
|
||||
- Staleness guard: alert if `fetched_at` >14 days.
|
||||
|
||||
## Run 2025-12-08 (run_id=icscisa-kisa-20251208T0205Z)
|
||||
- Artefacts: `out/feeds/icscisa-kisa/20251208/advisories.ndjson`, `delta.json`, `fetch.log`, `hashes.sha256`.
|
||||
- Hashes:
|
||||
- `0844c46c42461b8eeaf643c01d4cb74ef20d4eec8c984ad5e20c49d65dc57deb advisories.ndjson`
|
||||
- `1273beb246754382d2e013fdc98b11b06965fb97fe9a63735b51cc949746418f delta.json`
|
||||
- `8fedaa9fb2b146a1ef500b0d2e4c1592ddbc770a8f15b7d03723f8034fc12a75 fetch.log`
|
||||
- Delta summary: added ICS CISA advisories `ICSA-25-123-01`, `ICSMA-25-045-01`; added KISA advisories `KISA-2025-5859`, `KISA-2025-5860`; no updates or removals; backlog window 60 days; retries 0 for both sources.
|
||||
- Signature posture: both sources unsigned; all records marked `signature.missing` with reason `unsigned_source`.
|
||||
- Next actions: maintain weekly cadence; staleness review on 2025-12-21 with refreshed hash manifest and retry histogram.
|
||||
|
||||
## CI automation
|
||||
- Scheduled workflow `.gitea/workflows/icscisa-kisa-refresh.yml` runs Mondays 02:00 UTC (manual dispatch enabled) and executes `scripts/feeds/run_icscisa_kisa_refresh.py` with live fetch + offline fallback.
|
||||
- Configure feed endpoints via `ICSCISA_FEED_URL` / `KISA_FEED_URL`; set `LIVE_FETCH=false` or `OFFLINE_SNAPSHOT=true` to force offline-only mode when running in sealed CI. Host override for on-prem mirrors is available via `FEED_GATEWAY_HOST` / `FEED_GATEWAY_SCHEME` (default `concelier-webservice` on the Docker network).
|
||||
- Fetch log traces: `fetch.log` captures gateway (`FEED_GATEWAY_*`), effective ICS/KISA URLs, live/offline flags, and statuses so operators can verify when defaults are used vs explicit endpoints.
|
||||
|
||||
@@ -32,8 +32,8 @@ Define a minimal, actionable plan to refresh overdue ICSCISA and KISA connectors
|
||||
- Set to 2025-12-21 (two-week check from v0.2) and capture SIG verification status + open deltas.
|
||||
|
||||
## Actions & timeline (v0.2 refresh)
|
||||
- T0 (2025-12-08): adopt SOP + field map; create delta report template; preflight cache paths.
|
||||
- T0+2d (2025-12-10): run backlog reprocess, publish artefacts + hashes for both feeds; capture unsigned counts and retry reasons.
|
||||
- T0 (2025-12-08): adopt SOP + field map; create delta report template; preflight cache paths. **Done** via run `icscisa-kisa-20251208T0205Z` (see run summary below).
|
||||
- T0+2d (2025-12-10): run backlog reprocess, publish artefacts + hashes for both feeds; capture unsigned counts and retry reasons. **Done** in the 2025-12-08 execution (backlog window 60 days).
|
||||
- T0+14d (2025-12-21): review staleness, adjust cadence if needed; reset review date and owners.
|
||||
|
||||
## Artefact locations
|
||||
@@ -46,3 +46,18 @@ Define a minimal, actionable plan to refresh overdue ICSCISA and KISA connectors
|
||||
- Source downtime -> mirror last good snapshot; retry daily for 3 days.
|
||||
- Missing signatures -> record `signature=null`, log `skip_reason` in provenance note; do not infer validity.
|
||||
- Schema drift -> treat as new fields, store raw, add to field map after review (no drop).
|
||||
|
||||
## Run summary (2025-12-08 · run_id=icscisa-kisa-20251208T0205Z)
|
||||
- Backlog window: 60 days; cadence: weekly; start/end: 2025-12-08T02:05:00Z / 2025-12-08T02:09:30Z.
|
||||
- Outputs: `out/feeds/icscisa-kisa/20251208/advisories.ndjson`, `delta.json`, `fetch.log`, `hashes.sha256`.
|
||||
- Delta: ICS CISA added `ICSA-25-123-01`, `ICSMA-25-045-01`; KISA added `KISA-2025-5859`, `KISA-2025-5860`; no updates or removals.
|
||||
- Hash manifest: `hashes.sha256` records advisories/delta/log digests (see provenance note).
|
||||
- Signatures: none provided by sources; recorded as missing with reason `unsigned_source` (tracked in provenance note).
|
||||
- Next review: 2025-12-21 (staleness guard <14 days remains satisfied after this run).
|
||||
|
||||
## CI automation
|
||||
- Workflow: `.gitea/workflows/icscisa-kisa-refresh.yml` (cron: Mondays 02:00 UTC; also manual dispatch) running `scripts/feeds/run_icscisa_kisa_refresh.py`.
|
||||
- Outputs: uploads `icscisa-kisa-<YYYYMMDD>` artifact with `advisories.ndjson`, `delta.json`, `fetch.log`, `hashes.sha256`.
|
||||
- Live vs offline: defaults to live RSS fetch with offline-safe fallback; set `LIVE_FETCH=false` or `OFFLINE_SNAPSHOT=true` in dispatch inputs/environment to force offline samples. Optional feed URLs/secrets: `ICSCISA_FEED_URL`, `KISA_FEED_URL`.
|
||||
- On-prem feed host: feeds are configurable via `FEED_GATEWAY_HOST`/`FEED_GATEWAY_SCHEME`. Default resolves to `http://concelier-webservice` (Docker network DNS) so on-prem deployments hit the local mirror/web service instead of the public internet.
|
||||
- Fetch log traces defaults: `fetch.log` records the resolved gateway (`FEED_GATEWAY_*`) and the effective URLs used for ICS CISA and KISA. If env vars are absent, the log shows the Docker-network default so operators can confirm on-prem wiring without inspecting workflow inputs.
|
||||
|
||||
Reference in New Issue
Block a user