Files

master 7b5bdcf4d3 feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules

- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes.
- Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes.
- Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables.
- Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.

2025-10-30 00:09:39 +02:00

4.1 KiB

Raw Blame History

Concelier KISA Connector Operations

Operational guidance for the Korea Internet & Security Agency (KISA / KNVD) connector (source:kisa:*). Pair this with the engineering brief in docs/dev/kisa_connector_notes.md.

1. Prerequisites

Outbound HTTPS (or mirrored cache) for https://knvd.krcert.or.kr/.
Connector options defined under concelier:sources:kisa:

concelier:
  sources:
    kisa:
      feedUri: "https://knvd.krcert.or.kr/rss/securityInfo.do"
      detailApiUri: "https://knvd.krcert.or.kr/rssDetailData.do"
      detailPageUri: "https://knvd.krcert.or.kr/detailDos.do"
      maxAdvisoriesPerFetch: 10
      requestDelay: "00:00:01"
      failureBackoff: "00:05:00"

Ensure the URIs stay absolute—Concelier adds the feedUri/detailApiUri hosts to the HttpClient allow-list automatically.

2. Staging Smoke Test

Restart the Concelier workers so the KISA options bind.
Run a full connector cycle:
- CLI: stella db jobs run source:kisa:fetch --and-then source:kisa:parse --and-then source:kisa:map
- REST: POST /jobs/run { "kind": "source:kisa:fetch", "chain": ["source:kisa:parse", "source:kisa:map"] }
Confirm telemetry (Meter StellaOps.Concelier.Connector.Kisa):
- kisa.feed.success, kisa.feed.items
- kisa.detail.success / .failures
- kisa.parse.success / .failures
- kisa.map.success / .failures
- kisa.cursor.updates
Inspect logs for structured entries:
- KISA feed returned {ItemCount}
- KISA fetched detail for {Idx} … category={Category}
- KISA mapped advisory {AdvisoryId} (severity={Severity})
- Absence of warnings such as document missing GridFS payload.
Validate MongoDB state:
- raw_documents.metadata has kisa.idx, kisa.category, kisa.title.
- DTO store contains schemaVersion="kisa.detail.v1".
- Advisories include aliases (IDX, CVE) and language="ko".
- source_states entry for kisa shows recent cursor.lastFetchAt.

3. Production Monitoring

Dashboards – Add the following Prometheus/OTEL expressions:
- rate(kisa_feed_items_total[15m]) versus rate(concelier_source_http_requests_total{concelier_source="kisa"}[15m])
- increase(kisa_detail_failures_total{reason!="empty-document"}[1h]) alert at >0
- increase(kisa_parse_failures_total[1h]) for storage/JSON issues
- increase(kisa_map_failures_total[1h]) to flag schema drift
- increase(kisa_cursor_updates_total[6h]) == 0 during active windows → warn
Alerts – Page when rate(kisa_feed_success_total[2h]) == 0 while other connectors are active; back off for maintenance windows announced on https://knvd.krcert.or.kr/.
Logs – Watch for repeated warnings (document missing, DTO missing) or errors with reason tags HttpRequestException, download, parse, map.

4. Localisation Handling

Hangul categories (for example 취약점정보) flow into telemetry tags (category=…) and logs. Dashboards must render UTF‑8 and avoid transliteration.
HTML content is sanitised before storage; translation teams can consume the ContentHtml field safely.
Advisory severity remains as provided by KISA (High, Medium, etc.). Map-level failures include the severity tag for filtering.

5. Fixture & Regression Maintenance

Regression fixtures: src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/Fixtures/kisa-feed.xml and kisa-detail.json.
Refresh via UPDATE_KISA_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/StellaOps.Concelier.Connector.Kisa.Tests.csproj.
The telemetry regression (KisaConnectorTests.Telemetry_RecordsMetrics) will fail if counters/log wiring drifts—treat failures as gating.

6. Known Issues

RSS feeds only expose the latest 10 advisories; long outages require replay via archived feeds or manual IDX seeds.
Detail endpoint occasionally throttles; the connector honours requestDelay and reports failures with reason HttpRequestException. Consider increasing delay for weekend backfills.
If kisa.category tags suddenly appear as unknown, verify KISA has not renamed RSS elements; update the parser fixtures before production rollout.

4.1 KiB Raw Blame History Unescape Escape