commit and up

2025-10-07 08:33:54 +03:00
parent bb7eda17a8
commit 304118b665
585 changed files with 3138 additions and 1096 deletions
--- a/docs/07_HIGH_LEVEL_ARCHITECTURE.md
+++ b/docs/07_HIGH_LEVEL_ARCHITECTURE.md
@@ -140,7 +140,7 @@ intra‑component reach‑ins.
 | `identity`      | Embedded OAuth2/OIDC (OpenIddict 6)                | MIT OpenIddict          | `IIdentityProvider` for LDAP/SAML/JWT gateway     |
 | `pluginloader`  | Discover DLLs, SemVer gate, optional Cosign verify | Reflection + Cosign     | `IPluginLifecycleHook` for telemetry              |
 | `scanning`      | SBOM‑ & image‑flow orchestration; runner pool      | Trivy CLI (default)     | `IScannerRunner` – e.g., Grype, Copacetic, Clair  |
-| `feedmerge`    | Nightly NVD merge & feed enrichment                | Hangfire job            | drop‑in `*.Schedule.dll` for OSV, GHSA, NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU feeds |
+| `feedser` (vulnerability ingest/merge/export service) | Nightly NVD merge & feed enrichment                | Hangfire job            | drop-in `*.Schedule.dll` for OSV, GHSA, NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU feeds |
 | `tls`           | TLS provider abstraction                           | OpenSSL                 | `ITlsProvider` for custom suites (incl. **SM2**, where law or security requires it) |
 | `reporting`     | Render HTML/PDF reports                            | RazorLight              | `IReportRenderer`                                 |
 | `ui`            | Angular SPA & i18n                                 | Angular {{ angular }}   | new locales via `/locales/{lang}.json`            |
@@ -152,7 +152,7 @@ classDiagram
    class identity
    class pluginloader
    class scanning
-    class feedmerger
+    class feedser
    class tls
    class reporting
    class ui
@@ -163,13 +163,13 @@ classDiagram
    configuration ..> identity : Uses
    identity ..> pluginloader : Authenticates Plugins
    pluginloader ..> scanning : Loads Scanner Runners
-    scanning ..> feedmerger : Triggers Feed Merges
+    scanning ..> feedser : Triggers Feed Merges
    tls ..> AllModules : Provides TLS Abstraction
    reporting ..> ui : Renders Reports for UI
-    scheduling ..> feedmerger : Schedules Nightly Jobs
+    scheduling ..> feedser : Schedules Nightly Jobs

    note for scanning "Pluggable: ISScannerRunner<br>e.g., Trivy, Grype"
-    note for feedmerger "Pluggable: *.Schedule.dll<br>e.g., OSV, GHSA Feeds"
+    note for feedser "Pluggable: *.Schedule.dll<br>e.g., OSV, GHSA Feeds"
    note for identity "Pluggable: IIdentityProvider<br>e.g., LDAP, SAML"
    note for reporting "Pluggable: IReportRenderer<br>e.g., Custom PDF"
 ```
@@ -220,30 +220,29 @@ Builder collects layer digests.
 `POST /layers/missing` → Redis SDIFF → missing layer list (< 20 ms).
 SBOM generated only for those layers and uploaded.

-### 4.3 Feed Enrichment
-
-```mermaid
-sequenceDiagram
-    participant CRON as Nightly Cron (Hangfire)
-    participant FM as Feed Merger
-    participant NVD as NVD Feed
-    participant OSV as OSV Plugin (Optional)
-    participant GHSA as GHSA Plugin (Optional)
-    participant REGC as Regional Catalogue Plugin (Optional)
-    participant REDIS as Redis (Merged Feed Storage)
-    participant UI as Web UI
-
-    CRON->>FM: Trigger at 00:59
-    FM->>NVD: Fetch & Merge NVD Data
-    alt Optional Plugins
-        FM->>OSV: Merge OSV Feed
-        FM->>GHSA: Merge GHSA Feed
-        FM->>REGC: Merge Regional Catalogue Feed
-    end
-    FM->>REDIS: Persist Merged Feed
-    REDIS-->>UI: Update Feed Freshness
-    UI->>UI: Display Green 'Feed Age' Tile
-```
+### 4.3 Feedser Harvest & Export
+
+```mermaid
+sequenceDiagram
+    participant SCHED as Feedser Scheduler
+    participant CONN as Source Connector Plug-in
+    participant FEEDSER as Feedser Core
+    participant MONGO as MongoDB (Canonical Advisories)
+    participant EXPORT as Exporter (JSON / Trivy DB)
+    participant ART as Artifact Store / Offline Kit
+
+    SCHED->>CONN: Trigger window (init/resume)
+    CONN->>CONN: Fetch source documents + metadata
+    CONN->>FEEDSER: Submit raw document for parsing
+    FEEDSER->>FEEDSER: Parse & normalize to DTO
+    FEEDSER->>FEEDSER: Merge & deduplicate canonical advisory
+    FEEDSER->>MONGO: Write advisory, provenance, merge_event
+    FEEDSER->>EXPORT: Queue export delta request
+    EXPORT->>MONGO: Read canonical snapshot/deltas
+    EXPORT->>EXPORT: Build deterministic JSON & Trivy DB artifacts
+    EXPORT->>ART: Publish artifacts / Offline Kit bundle
+    ART-->>FEEDSER: Record export state + digests
+```

 ### 4.4 Identity & Auth Flow

@@ -264,15 +263,15 @@ without Core changes.

 | Store          | Primary Use                                   | Why chosen                     |
 |----------------|-----------------------------------------------|--------------------------------|
-| **Redis 7**    | Queue, SBOM cache, Trivy DB mirror            | Sub‑1 ms P99 latency           |
-| **MongoDB**    | History > 180 d, audit logs, policy versions  | Optional; document‑oriented    |
-| **Local tmpfs**| Trivy layer cache (`/var/cache/trivy`)        | Keeps disk I/O off hot path    |
+| **MongoDB**    | Feedser canonical advisories, merge events, export state | Deterministic canonical store with flexible schema |
+| **Redis 7**    | CLI quotas, short-lived job scheduling, layer diff cache | Sub-1 ms P99 latency for hot-path coordination |
+| **Local tmpfs**| Trivy layer cache (`/var/cache/trivy`)        | Keeps disk I/O off hot path    |

 ```mermaid
 flowchart LR
    subgraph "Persistence Layers"
-        REDIS[(Redis: Fast Cache/Queues<br>Sub-1ms P99)]
-        MONGO[(MongoDB: Optional Audit/History<br>>180 Days)]
+        REDIS[(Redis: Quotas & Short-lived Queues<br>Sub-1ms P99)]
+        MONGO[(MongoDB: Canonical Advisories<br>Merge Events & Export State)]
        TMPFS[(Local tmpfs: Trivy Layer Cache<br>Low I/O Overhead)]
    end

@@ -294,7 +293,7 @@ flowchart LR
 | **S‑1** | Pipeline Scan & Alert     | Stella CLI → SBOM → `/scan` → policy verdict → CI exit code & link to *Scan Detail*                |
 | **S‑2** | Mute Noisy CVE            | Dev toggles **Mute** in UI → rule stored in Redis → next build passes                           |
 | **S‑3** | Nightly Re‑scan           | `SbomNightly.Schedule` re‑queues SBOMs (mask‑filter) → dashboard highlights new Criticals       |
-| **S‑4** | Feed Update Cycle         | `FeedMerge Service` merges feeds → UI *Feed Age* tile turns green                                      |
+| **S‑4** | Feed Update Cycle         | `Feedser (vulnerability ingest/merge/export service)` refreshes feeds → UI *Feed Age* tile turns green |
 | **S‑5** | Custom Report Generation  | Plug‑in registers `IReportRenderer` → `/report/custom/{digest}` → CI downloads artifact         |

 ```mermaid
--- a/docs/08_MODULE_SPECIFICATIONS.md
+++ b/docs/08_MODULE_SPECIFICATIONS.md
@@ -1,371 +1,201 @@
-# 8 · Detailed Module Specifications — **Stella Ops**
-_This document defines every backend/agent module that composes Stella Ops,
-their public contracts, configuration keys and extension points._
-
---
-
-## 0 Scope  
-
-Describes **every .NET, and Angular project** that ships in the OSS Core, the plug‑in contracts they expose, and the runtime artefacts (Dockerfiles, Compose files) used to build and operate them.  Commercial capabilities appear *only* as extension points.
-
---
-
-## 1 Repository Layout (flat)  
-
-~~~text
-src/
- │ docker-compose.yml
- └─ docker-compose-library/
- │    ├─ docker-compose.no-deps.yml
- │    ├─ docker-compose.dep.redis.yml
- │    ├─ docker-compose.dep.mongo.yml
- │    ├─ docker-compose.dep.proxy.yml
- │    ├─ docker-compose.dep.repository.yml
- │    └─ docker-compose.local.yml
- └─ backend/
- │    ├─ Dockerfile
- │    ├─ StellaOps.Web/
- │    ├─ StellaOps.Common/
- │    ├─ StellaOps.Plugins/
- │    ├─ StellaOps.Configuration/
- │    ├─ StellaOps.Localization/
- │    ├─ StellaOps.TlsProvider.OpenSSL/
- │    ├─ StellaOps.TlsProvider.OpenSSL.LegacyRegional/
- │    ├─ StellaOps.TlsProvider.Plugin.CustomTlsVendor/
- │    ├─ StellaOps.VulnerabilityDatabase/
- │    ├─ StellaOps.Scheduling/
- │    ├─ StellaOps.Scheduling.SbomsRescan/
- │    ├─ StellaOps.Scheduling.MutesExpire/
- │    ├─ StellaOps.Scheduling.Plugin.CommonCveFeed/
- │    ├─ StellaOps.Scheduling.Plugin.RegionalCatalogueFeed/
- │    ├─ StellaOps.Scanners.Trivy/
- │    ├─ StellaOps.Quota/
- │    ├─ StellaOps.Reporting/
- │    ├─ StellaOps.Notifications/
- │    ├─ StellaOps.Notifications.Email/
- │    ├─ StellaOps.Notifications.Plugin.MsTeams/
- │    ├─ StellaOps.Authority/
- │    ├─ StellaOps.Authority.AD/
- │    ├─ StellaOps.Cli/
- │    └─ StellaOps.Agent.Zastava/
- └─ frontend/
-      ├─ Dockerfile
-      ├─ angular.json
-      ├─ stella-ops-ui/ 
-      └─ libs/
-           ├─ dashboard/
-           ├─ scans/
-           ├─ settings/
-           ├─ core-ui/
-           └─ i18n/
-~~~
-
-All projects are referenced by **`StellaOps.sln`**; `dotnet publish -c Release -p:PublishSingleFile=true` builds a self‑contained **`StellaOps.Api`** binary (plug‑ins load at runtime).
-
---
-
-## 2 Shared Libraries  
-
-| Project | Purpose | Key Interfaces |
-|---------|---------|----------------|
-| `StellaOps.Common` | Serilog sinks, Redis key helpers, DTO primitives. | `RedisKeys`, `Result<T>` |
-| `StellaOps.Plugins` | Plug‑in contracts + Cosign verification. | `IStellaPlugin`, `IScannerRunner`, `ITlsProvider`, `IScheduleJob` |
-| `StellaOps.Localization` | Loads JSON locale bundles (backend & Angular). | `ILocaleProvider`, `CultureMiddleware` |
-
-Angular JSON‑bundle workflow matches the official i18n guide .
-
---
-
-## 3 Core Back‑end Projects  
-
-| Project | Responsibility | Extensibility |
-|---------|----------------|---------------|
-| **`StellaOps.Api`** | ASP.NET host; source‑gen auto‑wires module endpoints. | Attributes `[MapRestController]`, `[MapHealth]`. |
-| **`StellaOps.Configuration`** | Bind `appsettings.json` → typed options; `/health`. | `IConfigValidator`. |
-| **`StellaOps.Quota`** | Enforces **Free‑tier quota** ({{ quota_token }}s scans/day) with early‑warning banner, 5 s soft back‑off, 60 s wait‑wall. | Swappable via `IQuotaStore` (e.g., Postgres). |
-| **`StellaOps.JwtIssuer` *(new)* | Issues, refreshes and validates **Client‑JWTs**. For offline sites it produces a 30‑day token during OUK build and again on every OUK import. | `ITokenSigner` (e.g., HSM) |
-| **`StellaOps.TlsProvider.OpenSSL`** | Default TLS suites. | New suites via `ITlsProvider` plug‑in. |
-| **`StellaOps.TlsProvider.OpenSSL.LegacyRegional`** | . | — |
-| **`StellaOps.VulnerabilityDatabase`** | Feed‑merge CLI writing Redis. | `IAdditionalFeedSource` (OSV, GHSA, regional catalogues). |
-| **`StellaOps.Scheduling`** | Hangfire host inside API . | Jobs via `IScheduleJob`. |
-| **`StellaOps.Scheduling.SbomsRescan`** | Nightly SBOM re‑scan (`0 2 * * *`). | — |
-| **`StellaOps.Scheduling.MutesExpire`** | Daily mute expiry cleanup. | — |
-| **`StellaOps.Scanners.Trivy`** | Trivy CLI for SBOM & image scans . | Other engines implement `IScannerRunner`. |
-| **`StellaOps.Reporting`** | RazorLight HTML reports. | `IReportRenderer` for SARIF, CycloneDX. |
-| **`StellaOps.Notifications`** | DI contracts for alerts. | `INotifier`. |
-| **`StellaOps.Notifications.Email`** | SMTP channel. | — |
-| **`StellaOps.Authority`** | OAuth2 / OIDC via OpenIddict 4 . | External IdPs via plug‑in. |
-| **`StellaOps.Registry`**        | read‑only Docker registry for agents + SBOM‑builder      | Registry v2 (nginx‑hardened)      | `IRegistryProvider` |
-| **`StellaOps.MutePolicies`**   | store YAML / Rego policies, validate & version           | MongoDB + Redis                   | `IPolicyStore` |
-| **`StellaOps.Attestor`** *(TODO)*| SLSA provenance + Rekor verification                      | Sigstore Rekor                    | `IAttestor` |
-
-## 3 · Module Details  
-
-> _Only contracts and configuration that may change in the next two quarters are shown; for stable, unchanging keys see the inline XML‑doc in the codebase._
-
-### 3.1. StellaOps.Configuration  
-
-* **Responsibility** – parse environment variables or `appsettings.json`; expose `/health`, `/metrics`.  
-* **Key extension point** – `IConfigValidator` → validate & normalise custom settings before DI builds.  
-
-### 3.2. StellaOps.Authority  
-
-* **Responsibility** – ship with OpenIddict 6, supporting *client‑credentials* and *password* grants.  
-* `IIdentityProvider` plug‑in can delegate token issuance to LDAP, SAML, Keycloak …  
-
-
-### 3.3. StellaOps.Scanners  
-
-* **Primary flow** – SBOM‑first; falls back to image‑unpack if SBOM absent.  
-* **Multi‑Format Support** – side‑car `.sbom.type` file; auto‑detects (`SPDXID:` or `bomFormat` heuristics).  
-* **Delta Layer Workflow** – `POST /layers/missing` (`SET DIFF` on Redis) responds < 20 ms; Stella CLI passes only new layers.  
-* **Plug‑in contract evolution**  
-
-```csharp
-// current
-Task<ScanResult> RunAsync(Stream sbomJson, CancellationToken ct);
-
-// v2 (preferred)
-Task<ScanResult> RunAsync(Stream sbom, SbomFormat fmt, CancellationToken ct);
-```
-
-### 3.5 StellOps.Registry  
-
-* **Purpose** – internal, anonymous **read‑only** Docker registry to avoid GHCR / Docker Hub pulls.  
-* **Deployment** – container `stellops.registry:2`; mounted volume `/var/lib/registry`; optional TLS via env vars.  
-
-| Key                              | Default | Notes                           |
-|----------------------------------|---------|---------------------------------|
-| `REGISTRY_READONLY`              | `true`  | Forces 403 on PUT, 405 on DELETE |
-| `REGISTRY_STORAGE_DELETE_ENABLED`| `false` | Immutable tags                   |
-
-**Plug‑in contract** — `IRegistryProvider.PullAsync(string imageRef)` for mapping to Artifactory, Harbor, etc.
-
---
-
-### 3.6 StellaOps.MutePolicies  
-
-* **Purpose** – central Policy‑as‑Code store (YAML v1 now, Rego soon).  
-* **Persistence** – current live rules in Redis (`policies:active`); immutable commits in Mongo `policies_history`.  
-
-| REST verb | Path                | Description               |
-|-----------|---------------------|---------------------------|
-| `GET`     | `/policy/export`    | download active YAML      |
-| `POST`    | `/policy/import`    | upload YAML / Rego file   |
-| `POST`    | `/policy/validate`  | lint without persisting   |
-
-**CLI** – Stella CLI gains `--policy-file scan-policy.yaml`.
-
-**Plug‑in contract** — `IPolicyStore` for GitOps back‑ends, Vault, etc.
-
---
-
-### 3.7. StellaOps.Attestor *(Planned – Q1‑2026)*  
-
-Handles SLSA provenance docs and Rekor log verification.
-
-```csharp
-public interface IAttestor {
-    Task<ProvenanceDoc> CreateAsync(ImageRef img, Sbom sbom);
-    Task<bool> VerifyAsync(ProvenanceDoc doc);
-}
-```
-
-### 3.7. StellaOps.FeedMerge.Service
-
-Nightly Hangfire job (01:00) merges NVD JSON; plug‑ins can provide ISourceFeed for OSV, GHSA, NVD, CNNVD, CNVD, ENISA and BDU feeds.
-
-### 3.8. StellOps.Tls
-
-Abstracts TLS stack; default OpenSSL; `ITlsProvider` lets enterprises swap in custom suites—**including SM2, where law or security requires it**.
-
-### 3.9. StellaOps.Reporting
-
-HTML / PDF generation via RazorLight; custom renderers via IReportRenderer.
-
-### 3.10 UI
-
-Angular 17 SPA; lazy‑loaded feature modules, standalone component routes for UI plug‑ins.
-
-Static Go daemon / k8s DaemonSet; watches Docker/CRI‑O events; uploads SBOMs; optional enforce mode via policy plug‑in.
-
-### 3.11 StellaOps.Quota — **Free‑Tier Daily Quota Service**
-
-**Responsibility**
-
-* Track per‑token scan count (`quota:<token>` key in Redis).
-* Reset counters at **00:00 UTC** with key TTL.
-* Inject HTTP headers  
-  * `X‑Stella‑Quota‑Remaining`  
-  * `X‑Stella‑Reset`
-* Apply adaptive throttling:  
-  * scans 90% of {{ quota_token }};  
-  * scans 10% of the max daily → UI banner flag `X‑Stella‑Quota‑Warn:true`;  
-  * scans ≥ {{ quota_token }}  → reply is slower.
-* **Offline token awareness** — if `token.valid == false` and
-  `OfflineMode == true`, return HTTP *451 ComplianceBlock* so that CLI gives a
-  clear actionable error.
-* New config:
-
-```json
-"Quota": {
-  "OfflineGraceDays": 7   // show banner this many days before token expiry
-}
-```
-
-**Interface**
-
-```csharp
-public interface IQuotaService
-{
-    /// <summary>Returns true when the call is allowed.</summary>
-    Task<QuotaVerdict> CheckAsync(string token, CancellationToken ct);
-}
-
-public readonly record struct QuotaVerdict(
-    bool IsAllowed,
-    int Remaining,
-    DateTimeOffset ResetUtc,
-    TimeSpan RetryAfter);
-```
-
-**Configuration** (`appsettings.json` keys)
-
-```json
-"Quota": {
-  "FreeTierDailyLimit": {{ quota_token }} ,
-  "WarnThreshold": 200,
-  "SoftRetrySeconds": 5,
-  "HardRetrySeconds": 60
-}
-```
-
-**Extensibility**
-
-* Override storage by providing an `IQuotaStore` plug‑in for Postgres or Mongo.
-* UI plug‑ins can subscribe to SSE `/quota/events` for custom dashboards.
-
-### 3.12 StellaOps.JwtIssuer — new section
-
-|API	|Path|	Notes|
-|-----|----|-------|
-|`POST	/token/offline`	| Admin‑only. | Generates a 30 d Client‑JWT for air‑gapped clusters; returns ZIP that the admin can copy to the target host.
-
-*OUK hook*
-
-* OUK builder calls JwtIssuer.SignOfflineToken(exp=+30d).
-* Drops client.jwt into ouk/root/.
-* Backend OUK importer places file under /var/lib/stella/tokens/.
-
---
-
-## 4 · Compose / Helm Snippet (reference)
-
-```yaml
-services:
-  registry:
-    image: stellops.registry:2
-    restart: unless-stopped
-    environment:
-      REGISTRY_READONLY: "true"
-    volumes:
-      - ./_registry:/var/lib/registry
-    ports:
-      - "5000:5000"
-
-  backend:
-    image: registry.local/stellops/backend:${TAG}
-    depends_on: [registry, redis]
-```
-
---
-
-## 4 Plug‑ins (sign‑required)  
-
-| Plug‑in | Contract | Notes |
-|---------|----------|-------|
-| `StellaOps.Notifications.Plugin.MsTeams` | `INotifier` | Sends cards to Teams webhooks. |
-| `StellaOps.Authority.AD` | `IIdentityProvider` | LDAP/Active‑Directory token issue. |
-| `StellaOps.Scheduling.Plugin.CommonCveFeed` | `IScheduleJob` | Merges OSV & NVD JSON hourly . |
-| `StellaOps.Scheduling.Plugin.RegionalCatalogueFeed` | `IScheduleJob` | Imports NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU XML daily. |
-| `StellaOps.TlsProvider.Plugin.CustomTlsVendor` | `ITlsProvider` | Binds regional specific shared libs. |
-
-Cosign signatures are mandatory; loader rejects unsigned DLLs when `DisableUnsigned=false`.
-
---
-
-## 5 Agents  
-
-### 5.1 `StellaOps.Cli`  
-
-Distroless CLI;
-Returns exit‑code 1 on policy violation, enabling CI blocking.
-* **Role** – CI helper: Build SBOM, call `/scan`, exit non‑zero on high severity.  
-* **Flags** – `--engine`, `--threshold`, `--registry-pull-token`, `--pdf-out`, `--delta`, `--sbom-type`, `--policy-file.`.  
-* **Auth** – OAuth2 *scanner* scope.
-
-### 5.2 `StellaOps.Agent.Zastava`  
- 
-* **Role** – Passive container inventory → uploads SBOMs via `/agent/sbom`.  
-* **Modes** – `off`, `inventory` (Core default).  
-* No kernel driver (unlike Falco) .
-
---
-
-## 6 Angular Front‑end  
-
-| Package | Path | Feature | Lazy |
-|---------|------|---------|------|
-| **App** | `frontend/stella-ops-ui/` | Shell, auth guards. | — |
-| `dashboard` | `libs/dashboard/` | Live metrics tiles. | ✔ |
-| `scans` | `libs/scans/` | List, detail, mute, diff. | ✔ |
-| `settings` | `libs/settings/` | Feed cron, workers, TLS switch. | ✔ |
-| `core-ui` | `libs/core-ui/` | Tailwind components. | — |
-| `i18n` | `libs/i18n/` | Runtime locale switch, pipe. | — |
-
-Lazy loading of workspace libs follows Nx/Angular guidance .
-
---
-
-## 7 Docker Artefacts  
-
-### 7.1 Dockerfiles  
-
-* **`backend/Dockerfile`** – multi‑stage .NET {{ dotnet }}; single‑file publish; distroless runtime .  
-* **`frontend/Dockerfile`** – Node 20 build → Nginx static serve.  
-* Every plug‑in repo may include its own Dockerfile when shipping side‑cars (e.g., custom scanner).
-
-### 7.2 Compose Stacks  
-
-* **`docker-compose.yml`**  
-  * Extends above with Redis 7 and Mongo 7 for small on‑prem installs.
-
-* **`docker-compose.no-deps.yml`**  
-  * backend, frontend, Trivy, Maven proxy.  
-  * Assumes external Redis & Mongo.
-
-* **`docker-compose.local.yml`**  
-  * Build images from local source and bring up backend, frontend, Redis, Mongo, Trivy, Maven proxy for dev‑loop.
-
-Docker Compose override precedence matches official docs .
-
---
-
-## 8 Performance Budget  
-
-| Flow | P95 target | Bottleneck |
-|------|-----------:|-----------|
-| SBOM fast‑path | ≤ 5 s | Redis queue depth (keep P99 < 1 ms)  |
-| Image‑unpack | ≤ 10 s | Trivy layer unpack. |
-| Nightly re‑scan | 80 SBOM/s | Runner CPU. |
-
---
-
-## Change Log  
-
-| Version | Date | Notes |
-|---------|------|-------|
-| **v2.2** | 2025‑07‑11 | Flat layout; stella‑ops‑ui naming; Dockerfiles & 3 Compose stacks; agents and localisation library. |
-| v2.1 | 2025‑07‑11 | First flat‑structure draft. |
-
-*(End of Module Specifications v2.2‑core)*
+# 8 · Detailed Module Specifications — **Stella Ops Feedser**
+_This document describes the Feedser service, its supporting libraries, connectors, exporters, and test assets that live in the OSS repository._
+
+---
+
+## 0 Scope  
+
+Feedser is the vulnerability ingest/merge/export subsystem of Stella Ops. It
+fetches primary advisories, normalizes and deduplicates them into MongoDB, and
+produces deterministic JSON and Trivy DB exports. This document lists the
+projects that make up that workflow, the extension points they expose, and the
+artefacts they ship.
+
+---
+
+## 1 Repository layout (current)  
+
+```text
+src/
+ ├─ Directory.Build.props / Directory.Build.targets
+ ├─ StellaOps.Plugin/
+ ├─ StellaOps.Feedser.Core/
+ ├─ StellaOps.Feedser.Core.Tests/
+ ├─ StellaOps.Feedser.Models/ (+ .Tests/)
+ ├─ StellaOps.Feedser.Normalization/ (+ .Tests/)
+ ├─ StellaOps.Feedser.Merge/ (+ .Tests/)
+ ├─ StellaOps.Feedser.Storage.Mongo/ (+ .Tests/)
+ ├─ StellaOps.Feedser.Exporter.Json/ (+ .Tests/)
+ ├─ StellaOps.Feedser.Exporter.TrivyDb/ (+ .Tests/)
+ ├─ StellaOps.Feedser.Source.* / StellaOps.Feedser.Source.*.Tests/
+ ├─ StellaOps.Feedser.Testing/
+ ├─ StellaOps.Feedser.Tests.Shared/
+ ├─ StellaOps.Feedser.WebService/ (+ .Tests/)
+ ├─ PluginBinaries/
+ └─ StellaOps.Feedser.sln
+```
+
+Each folder is a .NET project (or set of projects) referenced by
+`StellaOps.Feedser.sln`. Build assets are shared through the root
+`Directory.Build.props/targets` so conventions stay consistent.
+
+---
+
+## 2 Shared libraries  
+
+| Project | Purpose | Key extension points |
+|---------|---------|----------------------|
+| `StellaOps.Plugin` | Base contracts for connectors, exporters, and DI routines plus Cosign validation helpers. | `IFeedConnector`, `IExporterPlugin`, `IDependencyInjectionRoutine` |
+| `StellaOps.DependencyInjection` | Composable service registrations for Feedser and plug-ins. | `IDependencyInjectionRoutine` discovery |
+| `StellaOps.Feedser.Testing` | Common fixtures, builders, and harnesses for integration/unit tests. | `FeedserMongoFixture`, test builders |
+| `StellaOps.Feedser.Tests.Shared` | Shared assembly metadata and fixtures wired in via `Directory.Build.props`. | Test assembly references |
+
+---
+
+## 3 Core projects  
+
+| Project | Responsibility | Extensibility |
+|---------|----------------|---------------|
+| `StellaOps.Feedser.WebService` | ASP.NET Core minimal API hosting Feedser jobs, status endpoints, and scheduler. | DI-based plug-in discovery; configuration binding | 
+| `StellaOps.Feedser.Core` | Job orchestration, connector pipelines, merge workflows, export coordination. | `IFeedConnector`, `IExportJob`, deterministic merge policies |
+| `StellaOps.Feedser.Models` | Canonical advisory DTOs and enums persisted in MongoDB and exported artefacts. | Partial classes for source-specific metadata |
+| `StellaOps.Feedser.Normalization` | Version comparison, CVSS normalization, text utilities for canonicalization. | Helpers consumed by connectors/merge |
+| `StellaOps.Feedser.Merge` | Precedence evaluation, alias graph maintenance, merge-event hashing. | Policy extensions via DI |
+| `StellaOps.Feedser.Storage.Mongo` | Repository layer for documents, DTOs, advisories, merge events, export state. | Connection string/config via options |
+| `StellaOps.Feedser.Exporter.Json` | Deterministic vuln-list JSON export pipeline. | Dependency injection for storage + plugin to host | 
+| `StellaOps.Feedser.Exporter.TrivyDb` | Builds Trivy DB artefacts from canonical advisories. | Optional ORAS push routines |
+
+### 3.1 StellaOps.Feedser.WebService  
+
+* Hosts minimal API endpoints (`/health`, `/status`, `/jobs`).
+* Runs the scheduler that triggers connectors and exporters according to
+  configured windows.
+* Applies dependency-injection routines from `PluginBinaries/` at startup only
+  (restart-time plug-ins).
+
+### 3.2 StellaOps.Feedser.Core  
+
+* Defines job primitives (fetch, parse, map, merge, export) used by connectors.
+* Coordinates deterministic merge flows and writes `merge_event` documents.
+* Provides telemetry/log scopes consumed by WebService and exporters.
+
+### 3.3 StellaOps.Feedser.Storage.Mongo  
+
+* Persists raw documents, DTO records, canonical advisories, aliases, affected
+  packages, references, merge events, export state, and job leases.
+* Exposes repository helpers for exporters to stream full/delta snapshots.
+
+### 3.4 StellaOps.Feedser.Exporter.*  
+
+* `Exporter.Json` mirrors the Aqua vuln-list tree with canonical ordering.
+* `Exporter.TrivyDb` builds Trivy DB Bolt archives and optional OCI bundles.
+* Both exporters honour deterministic hashing and respect export cursors.
+
+---
+
+## 4 Source connectors  
+
+Connectors live under `StellaOps.Feedser.Source.*` and conform to the interfaces
+in `StellaOps.Plugin`.
+
+| Family | Project(s) | Notes |
+|--------|------------|-------|
+| Distro PSIRTs | `StellaOps.Feedser.Source.Distro.*` | Debian, Red Hat, SUSE, Ubuntu connectors with NEVRA/EVR helpers. |
+| Vendor PSIRTs | `StellaOps.Feedser.Source.Vndr.*` | Adobe, Apple, Cisco, Chromium, Microsoft, Oracle, VMware. |
+| Regional CERTs | `StellaOps.Feedser.Source.Cert*`, `Source.Ru.*`, `Source.Ics.*`, `Source.Kisa` | Provide enrichment metadata while preserving vendor precedence. |
+| OSS ecosystems | `StellaOps.Feedser.Source.Ghsa`, `Source.Osv`, `Source.Cve`, `Source.Kev`, `Source.Acsc`, `Source.Cccs`, `Source.Jvn` | Emit SemVer/alias-rich advisories. |
+
+Each connector ships fixtures/tests under the matching `*.Tests` project.
+
+---
+
+## 5 · Module Details  
+
+> _Focus on the Feedser-specific services that replace the legacy FeedMerge cron._
+
+### 5.1 Feedser.Core  
+
+* Owns the fetch → parse → merge → export job pipeline and enforces deterministic
+  merge hashes (`merge_event`).
+* Provides `JobSchedulerBuilder`, job coordinator, and telemetry scopes consumed
+  by the WebService and exporters.
+
+### 5.2 Feedser.Storage.Mongo  
+
+* Bootstrapper creates collections/indexes (documents, dto, advisory, alias,
+  affected, merge_event, export_state, jobs, locks).
+* Repository APIs surface full/delta advisory reads for exporters, plus
+  SourceState and job lease persistence.
+
+### 5.3 Feedser.Exporter.Json / Feedser.Exporter.TrivyDb  
+
+* JSON exporter mirrors vuln-list layout with per-file digests and manifest.
+* Trivy DB exporter shells or native-builds Bolt archives, optionally pushes OCI
+  layers, and records export cursors.
+
+### 5.4 Feedser.WebService  
+
+* Minimal API host exposing `/health`, `/ready`, `/jobs` and wiring telemetry.
+* Loads restart-time plug-ins from `PluginBinaries/`, executes Mongo bootstrap,
+  and registers built-in connectors/exporters with the scheduler.
+
+### 5.5 Plugin host & DI bridge  
+
+* `StellaOps.Plugin` + `StellaOps.DependencyInjection` provide the contracts and
+  helper routines for connectors/exporters to integrate with the WebService.
+
+---
+
+## 6 · Plug-ins & Agents  
+
+* **Plug-in discovery** – restart-only; the WebService enumerates
+  `PluginBinaries/` (or configured directories) and executes the contained
+  `IDependencyInjectionRoutine` implementations.
+* **Connector/exporter packages** – each source/exporter can ship as a plug-in
+  assembly with its own options and HttpClient configuration, keeping the core
+  image minimal.
+* **Stella CLI (agent)** – triggers feed-related jobs (`stella db fetch/merge/export`)
+  and consumes the exported JSON/Trivy DB artefacts, aligning with the SBOM-first
+  workflow described in `AGENTS.md`.
+* **Offline Kit** – bundles Feedser plug-ins, JSON tree, Trivy DB, and export
+  manifests so air-gapped sites can load the latest vulnerability data without
+  outbound connectivity.
+
+---
+
+## 7 · Docker & Distribution Artefacts  
+
+| Artefact | Path / Identifier | Notes |
+|----------|-------------------|-------|
+| Feedser WebService image | `containers/feedser/Dockerfile` (built via CI) | Self-contained ASP.NET runtime hosting scheduler/endpoints. |
+| Plugin bundle | `PluginBinaries/` | Mounted or baked-in assemblies for connectors/exporters. |
+| Offline Kit tarball | Produced by CI release pipeline | Contains JSON tree, Trivy DB OCI layout, export manifest, and plug-ins. |
+| Local dev compose | `scripts/` + future compose overlays | Developers can run MongoDB, Redis (optional), and WebService locally. |
+
+---
+
+## 8 · Performance Budget  
+
+| Scenario | Budget | Source |
+|----------|--------|--------|
+| Advisory upsert (large advisory) | ≤ 500 ms/advisory | `AdvisoryStorePerformanceTests` (Mongo) |
+| Advisory fetch (`GetRecent`) | ≤ 200 ms/advisory | Same performance test harness |
+| Advisory point lookup (`Find`) | ≤ 200 ms/advisory | Same performance test harness |
+| Bulk upsert/fetch cycle | ≤ 28 s total for 30 large advisories | Same performance test harness |
+| Feedser job scheduling | Deterministic cron execution via `JobSchedulerHostedService` | `StellaOps.Feedser.Core` tests |
+| Trivy DB export | Deterministic digests across runs (ongoing TODO for end-to-end test) | `Exporter.TrivyDb` backlog |
+
+Budgets are enforced in automated tests where available; outstanding TODO/DOING
+items (see task boards) continue tracking gaps such as exporter determinism.
+
+---
+
+## 9 Testing  
+
+* Unit and integration tests live alongside each component (`*.Tests`).
+* Shared fixtures come from `StellaOps.Feedser.Testing` and
+  `StellaOps.Feedser.Tests.Shared` (linked via `Directory.Build.props`).
+* Integration suites use ephemeral MongoDB and Redis via Testcontainers to
+  validate end-to-end flow without external dependencies.
+
+---
--- a/docs/13_SECURITY_POLICY.md
+++ b/docs/13_SECURITY_POLICY.md
@@ -81,7 +81,7 @@ cosign verify \

 ## 5 · Private‑feed mirrors 🌐

-The **FeedMerge** service provides a signed SQLite snapshot merging:
+The **Feedser (vulnerability ingest/merge/export service)** provides signed JSON and Trivy DB snapshots that merge:

 * OSV + GHSA
 * (optional) NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU  regionals
@@ -98,4 +98,4 @@ We are grateful to the researchers who help keep Stella Ops safe:
 | ------- | ------------------ | ------------ |
 | *empty* | *(your name here)* |              |

---
+---
--- a/docs/14_GLOSSARY_OF_TERMS.md
+++ b/docs/14_GLOSSARY_OF_TERMS.md
@@ -20,7 +20,7 @@ open a PR and append it alphabetically.*
 | **ADR** | *Architecture Decision Record* – lightweight Markdown file that captures one irreversible design decision. | ADR template lives at `/docs/adr/` |
 | **AIRE** | *AI Risk Evaluator* – optional Plus/Pro plug‑in that suggests mute rules using an ONNX model. | Commercial feature |
 | **Azure‑Pipelines** | CI/CD service in Microsoft Azure DevOps. | Recipe in Pipeline Library |
-| **BDU** | Russian (FSTEC) national vulnerability database: *База данных уязвимостей*. | Merged with NVD by FeedMerge Service |
+| **BDU** | Russian (FSTEC) national vulnerability database: *База данных уязвимостей*. | Merged with NVD by Feedser (vulnerability ingest/merge/export service) |
 | **BuildKit** | Modern Docker build engine with caching and concurrency. | Needed for layer cache patterns |
 | **CI** | *Continuous Integration* – automated build/test pipeline. | Stella integrates via CLI |
 | **Cosign** | Open‑source Sigstore tool that signs & verifies container images **and files**. | Images & OUK tarballs |
@@ -36,7 +36,7 @@ open a PR and append it alphabetically.*
 | **Digest (image)** | SHA‑256 hash uniquely identifying a container image or layer. | Pin digests for reproducible builds |
 | **Docker‑in‑Docker (DinD)** | Running Docker daemon inside a CI container. | Used in GitHub / GitLab recipes |
 | **DTO** | *Data Transfer Object* – C# record serialised to JSON. | Schemas in doc 11 |
-| **FeedMerge service** | Background job that merges OVN, GHSA and NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU XML into Redis. | Cron default `0 1 * * *` |
+| **Feedser** | Vulnerability ingest/merge/export service consolidating OVN, GHSA, NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU feeds into the canonical MongoDB store and export artifacts. | Cron default `0 1 * * *` |
 | **FSTEC** | Russian regulator issuing SOBIT certificates. | Pro GA target |
 | **Gitea** | Self‑hosted Git service – mirrors GitHub repo. | OSS hosting |
 | **GOST TLS** | TLS cipher‑suites defined by Russian GOST R 34.10‑2012 / 34.11‑2012. | Provided by `OpenSslGost` or CryptoPro |
--- a/docs/17_SECURITY_HARDENING_GUIDE.md
+++ b/docs/17_SECURITY_HARDENING_GUIDE.md
@@ -150,7 +150,7 @@ cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
 | Layer                | Cadence                                                  | Method                         |
 | -------------------- | -------------------------------------------------------- | ------------------------------ |
 | Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d |
-| Trivy DB             | 24 h cron via FeedMerge Service                                 | configurable (FeedMerge.Cron) |
+| Trivy DB             | 24 h scheduler via Feedser (vulnerability ingest/merge/export service) | configurable via Feedser scheduler options |
 | Docker Engine        | vendor LTS                                               | distro package manager         |
 | Host OS              | security repos enabled                                   | unattended‑upgrades            |

--- a/docs/19_TEST_SUITE_OVERVIEW.md
+++ b/docs/19_TEST_SUITE_OVERVIEW.md
@@ -16,7 +16,7 @@ contributors who need to extend coverage or diagnose failures.
 | **1. Unit** | `xUnit` (<code>dotnet test</code>) | `*.Tests.csproj` | per PR / push |
 | **2. Property‑based** | `FsCheck` | `SbomPropertyTests` | per PR |
 | **3. Integration (API)** | `Testcontainers` suite | `test/Api.Integration` | per PR + nightly |
-| **4. Integration (DB‑merge)** | in‑memory Mongo + Redis | `FeedMerge.Integration` | per PR |
+| **4. Integration (DB-merge)** | in-memory Mongo + Redis | `Feedser.Integration` (vulnerability ingest/merge/export service) | per PR |
 | **5. Contract (gRPC)** | `Buf breaking` | `buf.yaml` files | per PR |
 | **6. Front‑end unit** | `Jest` | `ui/src/**/*.spec.ts` | per PR |
 | **7. Front‑end E2E** | `Playwright` | `ui/e2e/**` | nightly |
@@ -70,7 +70,7 @@ flowchart LR
  I1 --> FE[Jest]
  FE --> E2E[Playwright]
  E2E --> Lighthouse
-  Lighthouse --> INTEG2[FeedMerge]
+  Lighthouse --> INTEG2[Feedser]
  INTEG2 --> LOAD[k6]
  LOAD --> CHAOS[pumba]
  CHAOS --> RELEASE[Attestation diff]
--- a/docs/40_ARCHITECTURE_OVERVIEW.md
+++ b/docs/40_ARCHITECTURE_OVERVIEW.md
@@ -32,7 +32,7 @@ why the system leans *monolith‑plus‑plug‑ins*, and where extension points
 graph TD
    A(API Gateway)
    B1(Scanner Core<br/>.NET latest LTS)
-    B2(FeedMerge service)
+    B2(Feedser service\n(vuln ingest/merge/export))
    B3(Policy Engine OPA)
    C1(Redis 7)
    C2(MongoDB 7)
@@ -53,7 +53,7 @@ graph TD
 | ---------------------------- | --------------------- | ---------------------------------------------------- |
 | **API Gateway**              | ASP.NET Minimal API   | Auth (JWT), quotas, request routing                  |
 | **Scanner Core**             | C# 12, Polly          | Layer diffing, SBOM generation, vuln correlation     |
-| **FeedMerge**                | C# source‑gen workers | Consolidate NVD + regional CVE feeds into one SQLite |
+| **Feedser (vulnerability ingest/merge/export service)** | C# source-gen workers | Consolidate NVD + regional CVE feeds into the canonical MongoDB store and drive JSON / Trivy DB exports |
 | **Policy Engine**            | OPA (Rego)            | admission decisions, custom org rules                |
 | **Redis 7**                  | Key‑DB compatible     | LRU cache, quota counters                            |
 | **MongoDB 7**                | WiredTiger            | SBOM & findings storage                              |
@@ -121,7 +121,7 @@ Hot‑plugging is deferred until after v 1.0 for security review.
 Although the default deployment is a single container, each sub‑service can be
 extracted:

-* FeedMerge → standalone cron pod.
+* Feedser → standalone cron pod.
 * Policy Engine → side‑car (OPA) with gRPC contract.
 * ResultSink → queue worker (RabbitMQ or Azure Service Bus).

--- a/docs/ARCHITECTURE_FEEDSER.md
+++ b/docs/ARCHITECTURE_FEEDSER.md
@@ -0,0 +1,190 @@
+# ARCHITECTURE.md — **StellaOps.Feedser**
+
+> **Goal**: Build a sovereign-ready, self-hostable **feed-merge service** that ingests authoritative vulnerability sources, normalizes and de-duplicates them into **MongoDB**, and exports **JSON** and **Trivy-compatible DB** artifacts.
+> **Form factor**: Long-running **Web Service** with **REST APIs** (health, status, control) and an embedded **internal cron scheduler**. Controllable by StellaOps.Cli (# stella db ...)
+> **No signing inside Feedser** (signing is a separate pipeline step).
+> **Runtime SDK baseline**: .NET 10 Preview 7 (SDK 10.0.100-preview.7.25380.108) targeting `net10.0`, aligned with the deployed api.stella-ops.org service.
+> **Four explicit stages**:
+>
+> 1. **Source Download** → raw documents.
+> 2. **Parse & Normalize** → schema-validated DTOs enriched with canonical identifiers.
+> 3. **Merge & Deduplicate** → precedence-aware canonical records persisted to MongoDB.
+> 4. **Export** → JSON or TrivyDB (full or delta), then (externally) sign/publish.
+
+---
+
+## 1) Naming & Solution Layout
+
+**Source connectors** namespace prefix: `StellaOps.Feedser.Source.*`
+**Exporters**:
+
+* `StellaOps.Feedser.Exporter.Json`
+* `StellaOps.Feedser.Exporter.TrivyDb`
+
+**Projects** (`/src`):
+
+```
+StellaOps.Feedser.WebService/        # ASP.NET Core (Minimal API, net10.0 preview) WebService + embedded scheduler
+StellaOps.Feedser.Core/              # Domain models, pipelines, merge/dedupe engine, jobs orchestration
+StellaOps.Feedser.Models/            # Canonical POCOs, JSON Schemas, enums
+StellaOps.Feedser.Storage.Mongo/     # Mongo repositories, GridFS access, indexes, resume "flags"
+StellaOps.Feedser.Source.Common/     # HTTP clients, rate-limiters, schema validators, parsers utils
+StellaOps.Feedser.Source.Cve/
+StellaOps.Feedser.Source.Nvd/
+StellaOps.Feedser.Source.Ghsa/
+StellaOps.Feedser.Source.Osv/
+StellaOps.Feedser.Source.Jvn/
+StellaOps.Feedser.Source.CertCc/
+StellaOps.Feedser.Source.Kev/
+StellaOps.Feedser.Source.Kisa/
+StellaOps.Feedser.Source.CertIn/
+StellaOps.Feedser.Source.CertFr/
+StellaOps.Feedser.Source.CertBund/
+StellaOps.Feedser.Source.Acsc/
+StellaOps.Feedser.Source.Cccs/
+StellaOps.Feedser.Source.Ru.Bdu/     # HTML→schema with LLM fallback (gated)
+StellaOps.Feedser.Source.Ru.Nkcki/   # PDF/HTML bulletins → structured
+StellaOps.Feedser.Source.Vndr.Msrc/
+StellaOps.Feedser.Source.Vndr.Cisco/
+StellaOps.Feedser.Source.Vndr.Oracle/
+StellaOps.Feedser.Source.Vndr.Adobe/
+StellaOps.Feedser.Source.Vndr.Apple/
+StellaOps.Feedser.Source.Vndr.Chromium/
+StellaOps.Feedser.Source.Vndr.Vmware/
+StellaOps.Feedser.Source.Distro.RedHat/
+StellaOps.Feedser.Source.Distro.Ubuntu/
+StellaOps.Feedser.Source.Distro.Debian/
+StellaOps.Feedser.Source.Distro.Suse/
+StellaOps.Feedser.Source.Ics.Cisa/
+StellaOps.Feedser.Source.Ics.Kaspersky/
+StellaOps.Feedser.Normalization/     # Canonical mappers, validators, version-range normalization
+StellaOps.Feedser.Merge/             # Identity graph, precedence, deterministic merge
+StellaOps.Feedser.Exporter.Json/
+StellaOps.Feedser.Exporter.TrivyDb/
+StellaOps.Feedser.<Component>.Tests/  # Component-scoped unit/integration suites (Core, Storage.Mongo, Source.*, Exporter.*, WebService, etc.)
+```
+
+---
+
+## 2) Runtime Shape
+
+**Process**: single service (`StellaOps.Feedser.WebService`)
+
+* `Program.cs`: top-level entry using **Generic Host**, **DI**, **Options** binding from `appsettings.json` + environment + optional `feedser.yaml`.
+* Built-in **scheduler** (cron-like) + **job manager** with **distributed locks** in Mongo to prevent overlaps, enforce timeouts, allow cancel/kill.
+* **REST APIs** for health/readiness/progress/trigger/kill/status.
+
+**Key NuGet concepts** (indicative): `MongoDB.Driver`, `Polly` (retry/backoff), `System.Threading.Channels`, `Microsoft.Extensions.Http`, `Microsoft.Extensions.Hosting`, `Serilog`, `OpenTelemetry`.
+
+---
+
+## 3) Data Storage — **MongoDB** (single source of truth)
+
+**Database**: `feedser`
+**Write concern**: `majority` for merge/export state, `acknowledged` for raw docs.
+**Collections** (with “flags”/resume points):
+
+* `source`
+  * `_id`, `name`, `type`, `baseUrl`, `auth`, `notes`.
+* `source_state`
+  * Keys: `sourceName` (unique), `enabled`, `cursor`, `lastSuccess`, `failCount`, `backoffUntil`, `paceOverrides`, `paused`.
+  * Drives incremental fetch/parse/map resume and operator pause/pace controls.
+* `document`
+  * `_id`, `sourceName`, `uri`, `fetchedAt`, `sha256`, `contentType`, `status`, `metadata`, `gridFsId`, `etag`, `lastModified`.
+  * Index `{sourceName:1, uri:1}` unique; optional TTL for superseded versions.
+* `dto`
+  * `_id`, `sourceName`, `documentId`, `schemaVer`, `payload` (BSON), `validatedAt`.
+  * Index `{sourceName:1, documentId:1}`.
+* `advisory`
+  * `_id`, `advisoryKey`, `title`, `summary`, `lang`, `published`, `modified`, `severity`, `exploitKnown`.
+  * Unique `{advisoryKey:1}` plus indexes on `modified` and `published`.
+* `alias`
+  * `advisoryId`, `scheme`, `value` with index `{scheme:1, value:1}`.
+* `affected`
+  * `advisoryId`, `platform`, `name`, `versionRange`, `cpe`, `purl`, `fixedBy`, `introducedVersion`.
+  * Index `{platform:1, name:1}`, `{advisoryId:1}`.
+* `reference`
+  * `advisoryId`, `url`, `kind`, `sourceTag` (e.g., advisory/patch/kb).
+* Flags collections: `kev_flag`, `ru_flags`, `jp_flags`, `psirt_flags` keyed by `advisoryId`.
+* `merge_event`
+  * `_id`, `advisoryKey`, `beforeHash`, `afterHash`, `mergedAt`, `inputs` (document ids).
+* `export_state`
+  * `_id` (`json`/`trivydb`), `baseExportId`, `baseDigest`, `lastFullDigest`, `lastDeltaDigest`, `exportCursor`, `targetRepo`, `exporterVersion`.
+* `locks`
+  * `_id` (`jobKey`), `holder`, `acquiredAt`, `heartbeatAt`, `leaseMs`, `ttlAt` (TTL index cleans dead locks).
+* `jobs`
+  * `_id`, `type`, `args`, `state`, `startedAt`, `endedAt`, `error`, `owner`, `heartbeatAt`, `timeoutMs`.
+
+**GridFS buckets**: `fs.documents` for raw large payloads; referenced by `document.gridFsId`.
+
+---
+
+## 4) Job & Scheduler Model
+
+* Scheduler stores cron expressions per source/exporter in config; persists next-run pointers in Mongo.
+* Jobs acquire locks (`locks` collection) to ensure singleton execution per source/exporter.
+* Supports manual triggers via API endpoints (`POST /jobs/{type}`) and pause/resume toggles per source.
+
+---
+
+## 5) Connector Contracts
+
+Connectors implement:
+
+```csharp
+public interface IFeedConnector {
+    string SourceName { get; }
+    Task FetchAsync(IServiceProvider sp, CancellationToken ct);
+    Task ParseAsync(IServiceProvider sp, CancellationToken ct);
+    Task MapAsync(IServiceProvider sp, CancellationToken ct);
+}
+```
+
+* Fetch populates `document` rows respecting rate limits, conditional GET, and `source_state.cursor`.
+* Parse validates schema (JSON Schema, XSD) and writes sanitized DTO payloads.
+* Map produces canonical advisory rows + provenance entries; must be idempotent.
+* Base helpers in `StellaOps.Feedser.Source.Common` provide HTTP clients, retry policies, and watermark utilities.
+
+---
+
+## 6) Merge & Normalization
+
+* Canonical model stored in `StellaOps.Feedser.Models` with serialization contracts used by storage/export layers.
+* `StellaOps.Feedser.Normalization` handles NEVRA/EVR/PURL range parsing, CVSS normalization, localization.
+* `StellaOps.Feedser.Merge` builds alias graphs keyed by CVE first, then falls back to vendor/regional IDs.
+* Precedence rules: PSIRT/OVAL overrides generic ranges; KEV only toggles exploitation; regional feeds enrich severity but don’t override vendor truth.
+* Determinism enforced via canonical JSON hashing logged in `merge_event`.
+
+---
+
+## 7) Exporters
+
+* JSON exporter mirrors `aquasecurity/vuln-list` layout with deterministic ordering and reproducible timestamps.
+* Trivy DB exporter initially shells out to `trivy-db` builder; later will emit BoltDB directly.
+* `StellaOps.Feedser.Storage.Mongo` provides cursors for delta exports based on `export_state.exportCursor`.
+* Export jobs produce OCI tarballs (layer media type `application/vnd.aquasec.trivy.db.layer.v1.tar+gzip`) and optionally push via ORAS.
+
+---
+
+## 8) Observability
+
+* Serilog structured logging with enrichment fields (`source`, `uri`, `stage`, `durationMs`).
+* OpenTelemetry traces around fetch/parse/map/export; metrics for rate limit hits, schema failures, dedupe ratios, package size.
+* Prometheus scraping endpoint served by WebService.
+
+---
+
+## 9) Security Considerations
+
+* Offline-first: connectors only reach allowlisted hosts.
+* BDU LLM fallback gated by config flag; logs audit trail with confidence score.
+* No secrets written to logs; secrets loaded via environment or mounted files.
+* Signing handled outside Feedser pipeline.
+
+---
+
+## 10) Deployment Notes
+
+* Default storage MongoDB; for air-gapped, bundle Mongo image + seeded data backup.
+* Horizontal scale achieved via multiple web service instances sharing Mongo locks.
+* Provide `feedser.yaml` template describing sources, rate limits, and export settings.