# Feedser CISA ICS Connector Operations

This runbook documents how to provision, rotate, and validate credentials for the CISA Industrial Control Systems (ICS) connector (`source:ics-cisa:*`). Follow it before enabling the connector in staging or offline installations.

## 1. Credential Provisioning

1. **Create a service mailbox** reachable by the Ops crew (shared mailbox recommended).  
2. Browse to `https://public.govdelivery.com/accounts/USDHSCISA/subscriber/new` and subscribe the mailbox to the following GovDelivery topics:
   - `USDHSCISA_16` — ICS-CERT advisories (legacy numbering: `ICSA-YY-###`).
   - `USDHSCISA_19` — ICS medical advisories (`ICSMA-YY-###`).
   - `USDHSCISA_17` — ICS alerts (`IR-ALERT-YY-###`) for completeness.
3. Complete the verification email. After confirmation, note the **personalised subscription code** included in the “Manage Preferences” link. It has the shape `code=AB12CD34EF`.
4. Store the code in the shared secret vault (or Offline Kit secrets bundle) as `feedser/sources/icscisa/govdelivery/code`.

> ℹ️  GovDelivery does not expose a one-time API key; the personalised code is what authenticates the RSS pull. Never commit it to git.

## 2. Feed Validation

Use the following command to confirm the feed is reachable before wiring it into Feedser (substitute `<CODE>` with the personalised value):

```bash
curl -H "User-Agent: StellaOpsFeedser/ics-cisa" \
     "https://content.govdelivery.com/accounts/USDHSCISA/topics/ICS-CERT/feed.rss?format=xml&code=<CODE>"
```

If the endpoint returns HTTP 200 and an RSS payload, record the sample response under `docs/artifacts/icscisa/` (see Task `FEEDCONN-ICSCISA-02-007`). HTTP 403 or 406 usually means the subscription was not confirmed or the code was mistyped.

## 3. Configuration Snippet

Add the connector configuration to `feedser.yaml` (or equivalent environment variables):

```yaml
feedser:
  sources:
    icscisa:
      govDelivery:
        code: "${FEEDSER_ICS_CISA_GOVDELIVERY_CODE}"
        topics:
          - "USDHSCISA_16"
          - "USDHSCISA_19"
          - "USDHSCISA_17"
      rssBaseUri: "https://content.govdelivery.com/accounts/USDHSCISA"
      requestDelay: "00:00:01"
      failureBackoff: "00:05:00"
```

Environment variable example:

```bash
export FEEDSER_SOURCES_ICSCISA_GOVDELIVERY_CODE="AB12CD34EF"
```

Feedser automatically register the host with the Source.Common HTTP allow-list when the connector assembly is loaded.


Optional tuning keys (set only when needed):

- `proxyUri` — HTTP/HTTPS proxy URL used when Akamai blocks direct pulls.
- `requestVersion` / `requestVersionPolicy` — override HTTP negotiation when the proxy requires HTTP/1.1.
- `enableDetailScrape` — toggle HTML detail fallback (defaults to true).
- `captureAttachments` — collect PDF attachments from detail pages (defaults to true).
- `detailBaseUri` — alternate host for detail enrichment if CISA changes their layout.

## 4. Seeding Without GovDelivery

If credentials are still pending, populate the connector with the community CSV dataset before enabling the live fetch:

1. Run `./scripts/fetch-ics-cisa-seed.sh` (or `.ps1`) to download the latest `CISA_ICS_ADV_*.csv` files into `seed-data/ics-cisa/`.
2. Copy the CSVs (and the generated `.sha256` files) into your Offline Kit staging area so they ship alongside the other feeds.
3. Import the kit as usual. The connector can parse the seed data for historical context, but **live GovDelivery credentials are still required** for fresh advisories.
4. Once credentials arrive, update `feedser:sources:icscisa:govDelivery:code` and re-trigger `source:ics-cisa:fetch` so the connector switches to the authorised feed.

> The CSVs are licensed under ODbL 1.0 by the ICS Advisory Project. Preserve the attribution when redistributing them.

## 4. Integration Validation

1. Ensure secrets are in place and restart the Feedser workers.
2. Run a dry-run fetch/parse/map chain against an Akamai-protected topic:
   ```bash
   FEEDSER_SOURCES_ICSCISA_GOVDELIVERY_CODE=... \ 
   FEEDSER_SOURCES_ICSCISA_ENABLEDETAILSCRAPE=1 \ 
   stella db jobs run source:ics-cisa:fetch --and-then source:ics-cisa:parse --and-then source:ics-cisa:map
   ```
3. Confirm logs contain `ics-cisa detail fetch` entries and that new documents/DTOs include attachments (see `docs/artifacts/icscisa`). Canonical advisories should expose PDF links as `references.kind == "attachment"` and affected packages should surface `primitives.semVer.exactValue` for single-version hits.
4. If Akamai blocks direct fetches, set `feedser:sources:icscisa:proxyUri` to your allow-listed egress proxy and rerun the dry-run.

## 4. Rotation & Incident Response

- Review GovDelivery access quarterly. Rotate the personalised code whenever Ops changes the service mailbox password or membership.  
- Revoking the subscription in GovDelivery invalidates the code immediately; update the vault and configuration in the same change.  
- If the code leaks, remove the subscription (`https://public.govdelivery.com/accounts/USDHSCISA/subscriber/manage_preferences?code=<CODE>`), resubscribe, and distribute the new value via the vault.

## 5. Offline Kit Handling

Include the personalised code in `offline-kit/secrets/feedser/icscisa.env`:

```
FEEDSER_SOURCES_ICSCISA_GOVDELIVERY_CODE=AB12CD34EF
```

The Offline Kit deployment script copies this file into the container secret directory mounted at `/run/secrets/feedser`. Ensure permissions are `600` and ownership matches the Feedser runtime user.

## 6. Telemetry & Monitoring

The connector emits metrics under the meter `StellaOps.Feedser.Source.Ics.Cisa`. They allow operators to track Akamai fallbacks, detail enrichment health, and advisory fan-out.

- `icscisa.fetch.*` – counters for `attempts`, `success`, `failures`, `not_modified`, and `fallbacks`, plus histogram `icscisa.fetch.documents` showing documents added per topic pull (tags: `feedser.source`, `icscisa.topic`).
- `icscisa.parse.*` – counters for `success`/`failures` and histograms `icscisa.parse.advisories`, `icscisa.parse.attachments`, `icscisa.parse.detail_fetches` to monitor enrichment workload per feed document.
- `icscisa.detail.*` – counters `success` / `failures` per advisory (tagged with `icscisa.advisory`) to alert when Akamai blocks detail pages.
- `icscisa.map.*` – counters for `success`/`failures` and histograms `icscisa.map.references`, `icscisa.map.packages`, `icscisa.map.aliases` capturing canonical fan-out.

Suggested alerts:

- `increase(icscisa.fetch.failures_total[15m]) > 0` or `increase(icscisa.fetch.fallbacks_total[15m]) > 5` — sustained Akamai or proxy issues.
- `increase(icscisa.detail.failures_total[30m]) > 0` — detail enrichment breaking (potential HTML layout change).
- `histogram_quantile(0.95, rate(icscisa.map.references_bucket[1h]))` trending sharply higher — sudden advisory reference explosion worth investigating.
- Keep an eye on shared HTTP metrics (`feedser.source.http.*{feedser.source="ics-cisa"}`) for request latency and retry patterns.

## 6. Related Tasks

- `FEEDCONN-ICSCISA-02-009` (GovDelivery credential onboarding) — completed once this runbook is followed and secrets are placed in the vault.
- `FEEDCONN-ICSCISA-02-007` (document inventory) — archive the first successful RSS response and any attachment URL schema under `docs/artifacts/icscisa/`.