master
7b5bdcf4d3
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes. - Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes. - Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables. - Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
44 lines
4.8 KiB
Markdown
44 lines
4.8 KiB
Markdown
# Concelier Connector Research – 2025-10-11
|
||
|
||
Snapshot of direct network checks performed on 2025-10-11 (UTC) for the national/vendor connectors in scope. Use alongside each module’s `TASKS.md` notes.
|
||
|
||
## ACSC (Australia)
|
||
- Enumerated feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92). Forcing HTTP/1.1 hangs >600 s and sitemap/HTML fetches fail the same way.
|
||
- Next actions: prototype `SocketsHttpHandler` settings (`RequestVersionOrLower`, allow fallback to relay), capture successful headers from partner vantage (need retention + cache semantics), and keep `FEEDCONN-SHARED-HTTP2-001` open for downgrade work.
|
||
|
||
## CCCS (Canada)
|
||
- JSON endpoint (`https://www.cyber.gc.ca/api/cccs/threats/v1/get?lang=<lang>&content_type=cccs_threat`) returns ~5 100 records per language; `page=<n>` still works for segmented pulls and the earliest `date_created` seen is 2018‑06‑08 (EN) / 2018‑06‑08 (FR). Use an explicit `User-Agent` to avoid 403 responses.
|
||
- Follow-up: telemetry, sanitiser coverage, and backfill procedures are documented in `docs/modules/concelier/operations/connectors/cccs.md` (2025‑10‑15). Adjust `maxEntriesPerFetch` when performing historical sweeps so cursor state remains responsive.
|
||
|
||
## CERT-Bund (Germany)
|
||
- `https://wid.cert-bund.de/content/public/securityAdvisory/rss` responds 200 without cookies (≈250-item window, German taxonomy). Detail links load an Angular SPA that fetches JSON behind the bootstrap session.
|
||
- Confirmed `GET https://wid.cert-bund.de/portal/api/securityadvisory?name=<WID-SEC-…>` returns JSON once the portal cookie container is primed; payload includes severity, CVEs, products, and references used by the connector fixtures.
|
||
- Historical advisories accessible through the SPA search/export endpoints once the `XSRF-TOKEN` cookie (exposed via `GET /portal/api/security/csrf`) is supplied with the `X-XSRF-TOKEN` header:
|
||
- `POST /portal/api/securityadvisory/search` (`{"page":N,"size":100,"sort":["published,desc"]}`) pages data back to 2014.
|
||
- `GET /portal/api/securityadvisory/export?format=json&from=YYYY-MM-DD` emits JSON bundles suitable for Offline Kit mirrors.
|
||
- Locale note: content is German-only; Concelier preserves `language=de` and Docs will publish a CERT-Bund glossary so operators can bridge terminology without machine translation.
|
||
|
||
## KISA / KNVD (Korea)
|
||
- `https://knvd.krcert.or.kr/rss/securityInfo.do` and `/rss/securityNotice.do` return UTF-8 RSS (10-item window) with `detailDos.do?IDX=` links. No cookies required for feed fetch.
|
||
- Detail SPA calls resolve to `rssDetailData.do?IDX=` JSON payloads; connector fetches those directly, sanitises HTML, and records Hangul metadata (NFC). See `docs/dev/kisa_connector_notes.md` for telemetry + localisation guidance.
|
||
|
||
## BDU (Russia / FSTEC)
|
||
- Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml/json`) return 403/404; TLS chain requires Russian Trusted Sub CA and WAF expects additional headers.
|
||
- Next actions: acquire official PEM chain, point `concelier:httpClients:source.bdu:trustedRootPaths` (or `concelier:sources:bdu:http:trustedRootPaths`) at the Offline Kit PEM, keep `allowInvalidCertificates=false`, script session bootstrap, then capture RSS/HTML schema for parser work.
|
||
|
||
## NKTsKI / cert.gov.ru (Russia)
|
||
- `https://cert.gov.ru/rss/advisories.xml` served via Bitrix returns 403/404 even with `Accept-Language: ru-RU`; TLS chain also requires Russian trust anchors.
|
||
- Next actions: source trust store, configure `concelier:httpClients:source.nkcki:trustedRootPaths` (Offline Kit root via `concelier:offline:root`), prepare proxy fallback, and once accessible document taxonomy/retention plus attachment handling.
|
||
|
||
## CISA ICS (United States)
|
||
- `curl -I https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` returns HTTP 403 + `x-reference-error` (Akamai). Same for legacy feed paths.
|
||
- Next actions: secure GovDelivery access, document token rotation, and build HTML/email fallback with throttling.
|
||
|
||
## Cisco PSIRT
|
||
- `https://api.cisco.com/security/advisories/latest` returns `ERR_596_SERVICE_NOT_FOUND` when unauthenticated. openVuln REST requires Mashery OAuth (client credentials) with quotas ~5 req/s, 30/min, 5 000/day; supports `pageIndex/pageSize` pagination.
|
||
- Next actions: register OAuth app, capture pagination/delta parameters, and compare API vs RSS coverage.
|
||
|
||
## Microsoft MSRC
|
||
- REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) requires Azure AD token + `api-version` (current `2024-08-01`) and supports delta filters (`lastModifiedStartDateTime`). CVRF ZIP remains available for offline use.
|
||
- Next actions: finalise AAD app registration, implement token cache, and design combined REST+CVRF ingestion path for determinism.
|