stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity
Files
40e7f827da764e39ca67fc27a74813a98e9a38d2
git.stella-ops.org/docs/dev/kisa_connector_notes.md
master 2eb6852d34
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details
Add unit tests for SBOM ingestion and transformation 
- Implement `SbomIngestServiceCollectionExtensionsTests` to verify the SBOM ingestion pipeline exports snapshots correctly.
- Create `SbomIngestTransformerTests` to ensure the transformation produces expected nodes and edges, including deduplication of license nodes and normalization of timestamps.
- Add `SbomSnapshotExporterTests` to test the export functionality for manifest, adjacency, nodes, and edges.
- Introduce `VexOverlayTransformerTests` to validate the transformation of VEX nodes and edges.
- Set up project file for the test project with necessary dependencies and configurations.
- Include JSON fixture files for testing purposes.
2025-11-04 07:49:39 +02:00

4.0 KiB
Raw Blame History

KISA Connector Observability & Localisation

The KISA/KNVD connector now ships with structured telemetry, richer logging, and a localisation brief so Docs/QA can extend operator material without reverse-engineering the source.

Telemetry counters

All metrics are emitted from KisaDiagnostics (Meter name StellaOps.Concelier.Connector.Kisa).

Metric Description Tags
kisa.feed.attempts RSS fetch attempts per scheduled job. —
kisa.feed.success Successful RSS fetches (increments even when no new items). —
kisa.feed.failures RSS fetch failures. reason (exception type)
kisa.feed.items Number of items returned by the RSS window. —
kisa.detail.attempts Advisory detail fetch attempts. category (Hangul category from RSS)
kisa.detail.success Detail payloads fetched and persisted. category
kisa.detail.unchanged HTTP 304 responses reused from cache. category
kisa.detail.failures Detail fetch failures or empty payloads. category, reason
kisa.parse.attempts Documents pulled from Mongo for parsing. category
kisa.parse.success Documents parsed into DTOs. category
kisa.parse.failures Download or JSON parse failures. category, reason
kisa.map.success Canonical advisories persisted. severity (e.g. High, unknown)
kisa.map.failures Mapping or DTO hydration failures. severity, reason
kisa.cursor.updates Published cursor advanced after ingest. —

category tags surface the original Hangul labels (for example 취약점정보), normalised to NFC. Downstream dashboards should render them as-is; do not transliterate or trim.

Logging patterns

  • Information level summary when the RSS feed completes (ItemCount), on each persisted detail document (IDX, category, documentId), and when a canonical advisory is written (IDX/severity).
  • Debug level logs capture cache hits (304) and cursor movements (Published timestamp).
  • Warning level emits when a document or DTO is missing so operators can correlate with parse/map counters.
  • Error level retains exception context for feed/detail/parse/map failures; state repository backoffs are still applied.

The messages use structured properties (Idx, Category, DocumentId, Severity) so Grafana/Loki dashboards can filter without regex.

Localisation notes for Docs & QA

  • Hangul fields (title, summary, category, reference.label, product vendor/name) are normalised to NFC before storage. Sample category 취약점정보 roughly translates to “vulnerability information”.
  • Advisory HTML is sanitised via HtmlContentSanitizer, stripping script/style while preserving inline anchors for translation pipelines.
  • Metrics carry Hangul category tags and logging keeps Hangul strings intact; this ensures air-gapped operators can validate native-language content without relying on MT.
  • Fixtures live under src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/Fixtures/. Regenerate with UPDATE_KISA_FIXTURES=1 dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Kisa.Tests/StellaOps.Concelier.Connector.Kisa.Tests.csproj.
  • The regression suite asserts canonical mapping, state cleanup, and telemetry counters (KisaConnectorTests.Telemetry_RecordsMetrics) so QA can track instrumentation drift.
  • When capturing new offline samples, use scripts/kisa_capture_html.py to mirror the RSS feed and write detailDos.do?IDX=… HTML into seed-data/kisa/html/; the SPA now embeds full advisory content in the HTML response while rssDetailData.do returns an error page for unauthenticated clients.
  • 2025-11-03: Connector fetches detailDos.do HTML during the fetch phase and the parser now extracts vendor/product tables directly from the DOM when JSON detail API payloads are unavailable.

For operator docs, link to this brief when documenting Hangul handling or counter dashboards so localisation reviewers have a single reference point.