Rename Concelier Source modules to Connector

src/StellaOps.Concelier.Connector.Common/AGENTS.md

@@ -0,0 +1,31 @@
+# AGENTS
+## Role
+Shared connector toolkit. Provides HTTP clients, retry/backoff, conditional GET (ETag/Last-Modified), schema validation, pagination helpers, clocks, and common DTO utilities for all connectors.
+## Scope
+- Typed HttpClient registrations with allowlisted hosts and timeouts.
+- Request pipeline: retries with jitter, backoff on 429/5xx, rate-limit tracking per source.
+- Conditional GET helpers (If-None-Match, If-Modified-Since), window cursors, and pagination iterators.
+- Validators: JSON Schema, XML Schema (for example XmlSchemaValidator), and sanitizers.
+- Content hashing and raw document capture helpers; metadata extraction (headers, status).
+- HTML sanitization, URL normalization, and PDF-to-text extraction utilities for feeds that require cleanup before validation.
+## Participants
+- Source.* connectors (NVD, Red Hat, JVN, PSIRTs, CERTs, ICS).
+- Storage.Mongo (document/dto repositories using shared shapes).
+- Core (jobs schedule/trigger for connectors).
+- QA (canned HTTP server harness, schema fixtures).
+## Interfaces & contracts
+- All network calls must pass through configured HttpClient with allowlist and sane timeouts; no direct new HttpClient().
+- Validators return detailed errors; invalid payloads quarantined and not mapped.
+- Cursor helpers implement sliding windows and ID-based pagination; rely on IClock/TimeProvider for determinism.
+- Strict provenance tags for extraction method: parser, oval, package.nevra, llm (gated).
+## In/Out of scope
+In: HTTP plumbing, validators, cursor/backoff utilities, hashing.
+Out: connector-specific schemas/mapping rules, merge precedence.
+## Observability & security expectations
+- Metrics: SourceDiagnostics publishes `concelier.source.http.*` counters/histograms tagged with `concelier.source=<connector>` plus retries/failures; connector dashboards slice on that tag instead of bespoke metric names.
+- Logs include uri, status, retries, etag; redact tokens and auth headers.
+- Distributed tracing hooks and per-connector counters should be wired centrally for consistent observability.
+## Tests
+- Author and review coverage in `../StellaOps.Concelier.Connector.Common.Tests`.
+- Shared fixtures (e.g., `MongoIntegrationFixture`, `ConnectorTestHarness`) live in `../StellaOps.Concelier.Testing`.
+- Keep fixtures deterministic; match new cases to real-world advisories or regression scenarios.