Rename Feedser to Concelier
This commit is contained in:
		
							
								
								
									
										74
									
								
								docs/ops/concelier-kisa-operations.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										74
									
								
								docs/ops/concelier-kisa-operations.md
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,74 @@ | ||||
| # Concelier KISA Connector Operations | ||||
|  | ||||
| Operational guidance for the Korea Internet & Security Agency (KISA / KNVD) connector (`source:kisa:*`). Pair this with the engineering brief in `docs/dev/kisa_connector_notes.md`. | ||||
|  | ||||
| ## 1. Prerequisites | ||||
|  | ||||
| - Outbound HTTPS (or mirrored cache) for `https://knvd.krcert.or.kr/`. | ||||
| - Connector options defined under `concelier:sources:kisa`: | ||||
|  | ||||
| ```yaml | ||||
| concelier: | ||||
|   sources: | ||||
|     kisa: | ||||
|       feedUri: "https://knvd.krcert.or.kr/rss/securityInfo.do" | ||||
|       detailApiUri: "https://knvd.krcert.or.kr/rssDetailData.do" | ||||
|       detailPageUri: "https://knvd.krcert.or.kr/detailDos.do" | ||||
|       maxAdvisoriesPerFetch: 10 | ||||
|       requestDelay: "00:00:01" | ||||
|       failureBackoff: "00:05:00" | ||||
| ``` | ||||
|  | ||||
| > Ensure the URIs stay absolute—Concelier adds the `feedUri`/`detailApiUri` hosts to the HttpClient allow-list automatically. | ||||
|  | ||||
| ## 2. Staging Smoke Test | ||||
|  | ||||
| 1. Restart the Concelier workers so the KISA options bind. | ||||
| 2. Run a full connector cycle: | ||||
|    - CLI: `stella db jobs run source:kisa:fetch --and-then source:kisa:parse --and-then source:kisa:map` | ||||
|    - REST: `POST /jobs/run { "kind": "source:kisa:fetch", "chain": ["source:kisa:parse", "source:kisa:map"] }` | ||||
| 3. Confirm telemetry (Meter `StellaOps.Concelier.Source.Kisa`): | ||||
|    - `kisa.feed.success`, `kisa.feed.items` | ||||
|    - `kisa.detail.success` / `.failures` | ||||
|    - `kisa.parse.success` / `.failures` | ||||
|    - `kisa.map.success` / `.failures` | ||||
|    - `kisa.cursor.updates` | ||||
| 4. Inspect logs for structured entries: | ||||
|    - `KISA feed returned {ItemCount}` | ||||
|    - `KISA fetched detail for {Idx} … category={Category}` | ||||
|    - `KISA mapped advisory {AdvisoryId} (severity={Severity})` | ||||
|    - Absence of warnings such as `document missing GridFS payload`. | ||||
| 5. Validate MongoDB state: | ||||
|    - `raw_documents.metadata` has `kisa.idx`, `kisa.category`, `kisa.title`. | ||||
|    - DTO store contains `schemaVersion="kisa.detail.v1"`. | ||||
|    - Advisories include aliases (`IDX`, CVE) and `language="ko"`. | ||||
|    - `source_states` entry for `kisa` shows recent `cursor.lastFetchAt`. | ||||
|  | ||||
| ## 3. Production Monitoring | ||||
|  | ||||
| - **Dashboards** – Add the following Prometheus/OTEL expressions: | ||||
|   - `rate(kisa_feed_items_total[15m])` versus `rate(concelier_source_http_requests_total{concelier_source="kisa"}[15m])` | ||||
|   - `increase(kisa_detail_failures_total{reason!="empty-document"}[1h])` alert at `>0` | ||||
|   - `increase(kisa_parse_failures_total[1h])` for storage/JSON issues | ||||
|   - `increase(kisa_map_failures_total[1h])` to flag schema drift | ||||
|   - `increase(kisa_cursor_updates_total[6h]) == 0` during active windows → warn | ||||
| - **Alerts** – Page when `rate(kisa_feed_success_total[2h]) == 0` while other connectors are active; back off for maintenance windows announced on `https://knvd.krcert.or.kr/`. | ||||
| - **Logs** – Watch for repeated warnings (`document missing`, `DTO missing`) or errors with reason tags `HttpRequestException`, `download`, `parse`, `map`. | ||||
|  | ||||
| ## 4. Localisation Handling | ||||
|  | ||||
| - Hangul categories (for example `취약점정보`) flow into telemetry tags (`category=…`) and logs. Dashboards must render UTF‑8 and avoid transliteration. | ||||
| - HTML content is sanitised before storage; translation teams can consume the `ContentHtml` field safely. | ||||
| - Advisory severity remains as provided by KISA (`High`, `Medium`, etc.). Map-level failures include the severity tag for filtering. | ||||
|  | ||||
| ## 5. Fixture & Regression Maintenance | ||||
|  | ||||
| - Regression fixtures: `src/StellaOps.Concelier.Source.Kisa.Tests/Fixtures/kisa-feed.xml` and `kisa-detail.json`. | ||||
| - Refresh via `UPDATE_KISA_FIXTURES=1 dotnet test src/StellaOps.Concelier.Source.Kisa.Tests/StellaOps.Concelier.Source.Kisa.Tests.csproj`. | ||||
| - The telemetry regression (`KisaConnectorTests.Telemetry_RecordsMetrics`) will fail if counters/log wiring drifts—treat failures as gating. | ||||
|  | ||||
| ## 6. Known Issues | ||||
|  | ||||
| - RSS feeds only expose the latest 10 advisories; long outages require replay via archived feeds or manual IDX seeds. | ||||
| - Detail endpoint occasionally throttles; the connector honours `requestDelay` and reports failures with reason `HttpRequestException`. Consider increasing delay for weekend backfills. | ||||
| - If `kisa.category` tags suddenly appear as `unknown`, verify KISA has not renamed RSS elements; update the parser fixtures before production rollout. | ||||
		Reference in New Issue
	
	Block a user