Refactor code structure for improved readability and maintainability; removed redundant code blocks and optimized function calls.

2025-11-20 07:50:52 +02:00
parent 616ec73133
commit 10212d67c0
473 changed files with 316758 additions and 388 deletions
--- a/docs/modules/concelier/bridges/vuln-29-001.md
+++ b/docs/modules/concelier/bridges/vuln-29-001.md
@@ -0,0 +1,81 @@
+# CONCELIER-VULN-29-001 · Concelier ↔ Vuln Explorer bridge (contract v0.1)
+
+Purpose: unblock PREP-CONCELIER-VULN-29-001 by defining the request/response contract Concelier must expose for Vuln Explorer and VEX Lens to consume advisory evidence without merge semantics.
+
+## API shape (Concelier WebService)
+- Endpoint: `POST /v1/advisories/search`
+- Auth: tenant-scoped token (`concelier.read` scope) with `mergeAllowed=false`.
+- Request body:
+  ```json
+  {
+    "query": {
+      "advisory_keys": ["GHSA-xxxx-xxxx"],
+      "purls": ["pkg:maven/org.example/app@1.2.3"],
+      "component_hashes": ["sha256:..."],
+      "has_vex": true
+    },
+    "page": {
+      "size": 50,
+      "cursor": null
+    }
+  }
+  ```
+- Response body:
+  ```json
+  {
+    "results": [
+      {
+        "advisory_key": "GHSA-xxxx-xxxx",
+        "source": "ghsa",
+        "observations": [{
+          "id": "obs-123",
+          "purl": "pkg:maven/org.example/app@1.2.3",
+          "version_range": "[1.2.0,2.0.0)",
+          "status": "affected",
+          "published": "2025-11-01T00:00:00Z",
+          "fixed": null,
+          "provenance": {
+            "source": "ghsa",
+            "retrieved_at": "2025-11-18T12:00:00Z",
+            "hash": "sha256:..."
+          }
+        }],
+        "vex": [{
+          "statement_id": "st-001",
+          "status": "not_affected",
+          "justification": "component_not_present",
+          "impact_statement": "Library not shipped",
+          "provenance": {
+            "source": "vendor-vex",
+            "retrieved_at": "2025-11-18T12:00:00Z",
+            "hash": "sha256:..."
+          }
+        }]
+      }
+    ],
+    "page": {"cursor": "next-token", "size": 50}
+  }
+  ```
+
+## Determinism requirements
+- Sort observations by `purl`, then `published`, then `id`.
+- Sort VEX statements by `statement_id`.
+- Timestamps UTC ISO-8601 with `Z`.
+- Omit merge-derived fields; only surface source-provided values.
+
+## Fields Vuln Explorer expects
+- `advisory_key`, `source`, `observations[*].purl`, `observations[*].status`, `observations[*].provenance.hash`.
+- VEX fields: `status`, `justification`, `impact_statement`, `provenance.hash`.
+- Optional: `fix_version` when present; keep absent otherwise (no empty strings).
+
+## Test fixtures
+- Location: `docs/samples/console/console-vex-30-001.json` already includes VEX sample keyed by advisory; add Concelier response sample to `docs/samples/console/concelier-vuln-29-001.json` (to be generated alongside implementation).
+
+## Owners
+- Concelier WebService Guild (producer)
+- Vuln Explorer Guild / VEX Lens Guild (consumers)
+
+## Next actions to unblock work
+- Implement endpoint stub with deterministic ordering and fixture from above sample.
+- Hook to Link-Not-Merge storage (fact-only) once available.
+- Add contract reference to OAS (Sprint 0114 CONCELIER-OAS-61-001 depends on this).
--- a/docs/modules/concelier/feeds/icscisa-kisa-provenance.md
+++ b/docs/modules/concelier/feeds/icscisa-kisa-provenance.md
@@ -0,0 +1,7 @@
+# ICSCISA / KISA Feed Provenance Notes (2025-11-19)
+
+- Expected signing: not provided by sources; set `signature=null` and `skip_reason="unsigned"`.
+- Hashing: sha256 of raw advisory payload before normalization.
+- Transport: HTTPS; mirror to internal cache; record `fetched_at` UTC and `source_url`.
+- Verification: compare hash vs previous run; emit delta report.
+- Staleness guard: alert if `fetched_at` >14 days.
--- a/docs/modules/concelier/feeds/icscisa-kisa.md
+++ b/docs/modules/concelier/feeds/icscisa-kisa.md
@@ -0,0 +1,46 @@
+# ICSCISA / KISA Feed Remediation Plan (v0.1 · 2025-11-19)
+
+## Purpose
+Define a minimal, actionable plan to refresh overdue ICSCISA and KISA connectors, restore provenance freshness, and publish normalized payload fields for downstream Advisory AI and Concelier consumers.
+
+## Owners
+- Feed owners: Concelier Feed Guild
+- Product advisory liaison: Product Advisory Guild
+- Backup: Docs Guild
+
+## Scope & cadence
+- Feeds: ICSCISA, KISA (security advisories)
+- Refresh cadence: weekly pull; publish hashlist and timestamps per run
+- Staleness budget: <14 days; alert if exceeded
+
+## Deliverables (for PREP-FEEDCONN-ICS-KISA-PLAN)
+1) **Provenance refresh SOP**
+   - Mirror source URLs to internal cache
+   - Record `source_url`, `fetched_at` (UTC), `sha256`, `signature` (if present)
+   - Store run log under `out/feeds/icscisa-kisa/<YYYYMMDD>/fetch.log`
+2) **Normalized payload fields**
+   - `advisory_id`, `title`, `summary`, `published`, `updated`, `severity` (pass-through), `cvss` (if provided), `cwe`, `affected_products` (list), `references` (list of URL strings), `signature` (object or null)
+   - Preserve source values; no inference or merging
+3) **Backlog cleanup**
+   - Reprocess last 60 days; compare hash to prior ingests; flag changed advisories
+   - Emit delta report (`out/feeds/icscisa-kisa/<YYYYMMDD>/delta.json`): added/updated/removed ids, counts
+4) **Provenance note**
+   - Publish `docs/modules/concelier/feeds/icscisa-kisa-provenance.md` with current signing keys/fingerprints, expected headers, and fallback when signatures missing
+5) **Next review date**
+   - Set to 2025-12-03 (two-week check) and capture SIG verification status
+
+## Actions & timeline
+- T0 (2025-11-19): adopt SOP + field map; create delta report template
+- T0+2d (2025-11-21): run backlog reprocess, publish artefacts + hashes
+- T0+14d (2025-12-03): review staleness, adjust cadence if needed
+
+## Artefact locations
+- Normalized advisories: `out/feeds/icscisa-kisa/<YYYYMMDD>/advisories.ndjson`
+- Fetch log + hashes: `out/feeds/icscisa-kisa/<YYYYMMDD>/fetch.log`, `hashes.sha256`
+- Delta report: `out/feeds/icscisa-kisa/<YYYYMMDD>/delta.json`
+- Provenance note: `docs/modules/concelier/feeds/icscisa-kisa-provenance.md`
+
+## Risks & mitigations
+- Source downtime → mirror last good snapshot; retry daily for 3 days.
+- Missing signatures → record `signature=null`, log `skip_reason` in provenance note; do not infer validity.
+- Schema drift → treat as new fields, store raw, add to field map after review (no drop).
--- a/docs/modules/concelier/operations/cache.md
+++ b/docs/modules/concelier/operations/cache.md
@@ -0,0 +1,40 @@
+# Concelier Advisory Chunk Cache
+
+Status: draft · aligned with LNM v1 (frozen 2025-11-17)
+
+## Purpose
+- Reduce repeated `/advisories/{key}/chunks` rebuilds while preserving determinism and provenance integrity.
+- Provide consoles and debugging tools with transparent cache provenance (key material and TTL) to keep evidence chains auditable.
+
+## Cache key (deterministic)
+- Inputs (pipe-delimited, all lower-case where applicable):
+  - `tenant`
+  - `advisoryKey`
+  - `chunkLimit`
+  - `observationLimit`
+  - `minimumLength`
+  - `sectionFilter` (sorted, case-insensitive)
+  - `formatFilter` (sorted, case-insensitive)
+  - advisory fingerprint (hash of canonical advisory)
+  - ordered observations: each `observationId@contentHash@createdAtTicks@format`
+- Implementation: `AdvisoryChunkCacheKey.Create` (WebService). Normalized ordering ensures replayability across nodes.
+- External disclosure: headers expose a hashed cache key (see below) rather than the full raw key.
+
+## Response transparency headers
+- `X-Stella-Cache-Key`: SHA-256 hex of the cache key (default 16 chars) for tamper-evident correlation.
+- `X-Stella-Cache-Hit`: `1` on hit, `0` on miss.
+- `X-Stella-Cache-Ttl`: configured TTL in seconds (0 when caching disabled).
+- Consumers may log these headers; do **not** treat them as stable API contract for filtering.
+
+## Determinism & safety
+- No PII in keys or headers; key inputs are tenant/advisory identifiers already present in requests.
+- Timestamps use UTC ticks; content hashes retain upstream digest prefix (e.g., `sha256:`) to distinguish sources.
+- Cache TTL defaults to 0 (disabled) unless configured; safe for offline/air-gapped deployments.
+
+## Testing notes
+- Unit coverage: `AdvisoryChunkCacheKeyTests` (ordering, filter casing) and `AdvisoryChunkBuilderTests` (observationPath pointers influence chunk IDs).
+- Integration tests should assert headers when cache is enabled; disable cache for tests that assert body-only determinism.
+
+## TODOs / follow-ups
+- Add integration test that exercises a cache hit path and validates transparency headers.
+- Document cache configuration knobs in `appsettings.*` once finalized.