git.stella-ops.org/docs/08_MODULE_SPECIFICATIONS.md

# 8 · Detailed Module Specifications — Stella Ops Feedser This document describes the Feedser service, its supporting libraries, connectors, exporters, and test assets that live in the OSS repository.

---

## 0 Scope

Feedser is the vulnerability ingest/merge/export subsystem of Stella Ops. It fetches primary advisories, normalizes and deduplicates them into MongoDB, and produces deterministic JSON and Trivy DB exports. This document lists the projects that make up that workflow, the extension points they expose, and the artefacts they ship.

---

## 1 Repository layout (current)

src/
 ├─ Directory.Build.props / Directory.Build.targets
 ├─ StellaOps.Plugin/
 ├─ StellaOps.Feedser.Core/
 ├─ StellaOps.Feedser.Core.Tests/
 ├─ StellaOps.Feedser.Models/ (+ .Tests/)
 ├─ StellaOps.Feedser.Normalization/ (+ .Tests/)
 ├─ StellaOps.Feedser.Merge/ (+ .Tests/)
 ├─ StellaOps.Feedser.Storage.Mongo/ (+ .Tests/)
 ├─ StellaOps.Feedser.Exporter.Json/ (+ .Tests/)
 ├─ StellaOps.Feedser.Exporter.TrivyDb/ (+ .Tests/)
 ├─ StellaOps.Feedser.Source.* / StellaOps.Feedser.Source.*.Tests/
 ├─ StellaOps.Feedser.Testing/
 ├─ StellaOps.Feedser.Tests.Shared/
 ├─ StellaOps.Feedser.WebService/ (+ .Tests/)
 ├─ PluginBinaries/
 └─ StellaOps.Feedser.sln

Each folder is a .NET project (or set of projects) referenced by StellaOps.Feedser.sln. Build assets are shared through the root Directory.Build.props/targets so conventions stay consistent.

---

## 2 Shared libraries

Project	Purpose	Key extension points
StellaOps.Plugin	Base contracts for connectors, exporters, and DI routines plus Cosign validation helpers.	IFeedConnector, IExporterPlugin, IDependencyInjectionRoutine
StellaOps.DependencyInjection	Composable service registrations for Feedser and plug-ins.	IDependencyInjectionRoutine discovery
StellaOps.Feedser.Testing	Common fixtures, builders, and harnesses for integration/unit tests.	FeedserMongoFixture, test builders
StellaOps.Feedser.Tests.Shared	Shared assembly metadata and fixtures wired in via Directory.Build.props.	Test assembly references

---

## 3 Core projects

Project	Responsibility	Extensibility
StellaOps.Feedser.WebService	ASP.NET Core minimal API hosting Feedser jobs, status endpoints, and scheduler.	DI-based plug-in discovery; configuration binding
StellaOps.Feedser.Core	Job orchestration, connector pipelines, merge workflows, export coordination.	IFeedConnector, IExportJob, deterministic merge policies
StellaOps.Feedser.Models	Canonical advisory DTOs and enums persisted in MongoDB and exported artefacts.	Partial classes for source-specific metadata
StellaOps.Feedser.Normalization	Version comparison, CVSS normalization, text utilities for canonicalization.	Helpers consumed by connectors/merge
StellaOps.Feedser.Merge	Precedence evaluation, alias graph maintenance, merge-event hashing.	Policy extensions via DI
StellaOps.Feedser.Storage.Mongo	Repository layer for documents, DTOs, advisories, merge events, export state.	Connection string/config via options
StellaOps.Feedser.Exporter.Json	Deterministic vuln-list JSON export pipeline.	Dependency injection for storage + plugin to host
StellaOps.Feedser.Exporter.TrivyDb	Builds Trivy DB artefacts from canonical advisories.	Optional ORAS push routines

### 3.1 StellaOps.Feedser.WebService

• Hosts minimal API endpoints (/health, /status, /jobs).
• Runs the scheduler that triggers connectors and exporters according to configured windows.
• Applies dependency-injection routines from PluginBinaries/ at startup only (restart-time plug-ins).

### 3.2 StellaOps.Feedser.Core

• Defines job primitives (fetch, parse, map, merge, export) used by connectors.
• Coordinates deterministic merge flows and writes merge_event documents.
• Provides telemetry/log scopes consumed by WebService and exporters.

### 3.3 StellaOps.Feedser.Storage.Mongo

• Persists raw documents, DTO records, canonical advisories, aliases, affected packages, references, merge events, export state, and job leases.
• Exposes repository helpers for exporters to stream full/delta snapshots.

### 3.4 StellaOps.Feedser.Exporter.*

• Exporter.Json mirrors the Aqua vuln-list tree with canonical ordering.
• Exporter.TrivyDb builds Trivy DB Bolt archives and optional OCI bundles.
• Both exporters honour deterministic hashing and respect export cursors.

---

## 4 Source connectors

Connectors live under StellaOps.Feedser.Source.* and conform to the interfaces in StellaOps.Plugin.

Family	Project(s)	Notes
Distro PSIRTs	StellaOps.Feedser.Source.Distro.*	Debian, Red Hat, SUSE, Ubuntu connectors with NEVRA/EVR helpers.
Vendor PSIRTs	StellaOps.Feedser.Source.Vndr.*	Adobe, Apple, Cisco, Chromium, Microsoft, Oracle, VMware.
Regional CERTs	StellaOps.Feedser.Source.Cert*, Source.Ru.*, Source.Ics.*, Source.Kisa	Provide enrichment metadata while preserving vendor precedence.
OSS ecosystems	StellaOps.Feedser.Source.Ghsa, Source.Osv, Source.Cve, Source.Kev, Source.Acsc, Source.Cccs, Source.Jvn	Emit SemVer/alias-rich advisories.

Each connector ships fixtures/tests under the matching *.Tests project.

---

## 5 · Module Details

Focus on the Feedser-specific services that replace the legacy FeedMerge cron.

### 5.1 Feedser.Core

• Owns the fetch → parse → merge → export job pipeline and enforces deterministic merge hashes (merge_event).
• Provides JobSchedulerBuilder, job coordinator, and telemetry scopes consumed by the WebService and exporters.

### 5.2 Feedser.Storage.Mongo

• Bootstrapper creates collections/indexes (documents, dto, advisory, alias, affected, merge_event, export_state, jobs, locks).
• Repository APIs surface full/delta advisory reads for exporters, plus SourceState and job lease persistence.

### 5.3 Feedser.Exporter.Json / Feedser.Exporter.TrivyDb

• JSON exporter mirrors vuln-list layout with per-file digests and manifest.
• Trivy DB exporter shells or native-builds Bolt archives, optionally pushes OCI layers, and records export cursors. Delta runs reuse unchanged blobs from the previous full baseline, annotating metadata.json with mode, baseExportId, baseManifestDigest, resetBaseline, and delta.changedFiles[]/delta.removedPaths[]. ORAS pushes honour publishFull / publishDelta, and offline bundles respect includeFull / includeDelta for air-gapped syncs.

### 5.4 Feedser.WebService

• Minimal API host exposing /health, /ready, /jobs and wiring telemetry.
• Loads restart-time plug-ins from PluginBinaries/, executes Mongo bootstrap, and registers built-in connectors/exporters with the scheduler.

### 5.5 Plugin host & DI bridge

• StellaOps.Plugin + StellaOps.DependencyInjection provide the contracts and helper routines for connectors/exporters to integrate with the WebService.

---

## 6 · Plug-ins & Agents

• Plug-in discovery – restart-only; the WebService enumerates PluginBinaries/ (or configured directories) and executes the contained IDependencyInjectionRoutine implementations.
• Connector/exporter packages – each source/exporter can ship as a plug-in assembly with its own options and HttpClient configuration, keeping the core image minimal.
• StellaOps CLI (agent) – new StellaOps.Cli module that exposes scanner, scan, and db verbs (via System.CommandLine 2.0) to download scanner container bundles, install them locally, execute scans against target directories, automatically upload results, and trigger Feedser jobs (db fetch/merge/export) aligned with the SBOM-first workflow described in AGENTS.md.
• Offline Kit – bundles Feedser plug-ins, JSON tree, Trivy DB, and export manifests so air-gapped sites can load the latest vulnerability data without outbound connectivity.

---

## 7 · Docker & Distribution Artefacts

Artefact	Path / Identifier	Notes
Feedser WebService image	containers/feedser/Dockerfile (built via CI)	Self-contained ASP.NET runtime hosting scheduler/endpoints.
Plugin bundle	PluginBinaries/	Mounted or baked-in assemblies for connectors/exporters.
Offline Kit tarball	Produced by CI release pipeline	Contains JSON tree, Trivy DB OCI layout, export manifest, and plug-ins.
Local dev compose	scripts/ + future compose overlays	Developers can run MongoDB, Redis (optional), and WebService locally.

---

## 8 · Performance Budget

Scenario	Budget	Source
Advisory upsert (large advisory)	≤ 500 ms/advisory	AdvisoryStorePerformanceTests (Mongo)
Advisory fetch (GetRecent)	≤ 200 ms/advisory	Same performance test harness
Advisory point lookup (Find)	≤ 200 ms/advisory	Same performance test harness
Bulk upsert/fetch cycle	≤ 28 s total for 30 large advisories	Same performance test harness
Feedser job scheduling	Deterministic cron execution via JobSchedulerHostedService	StellaOps.Feedser.Core tests
Trivy DB export	Deterministic digests across runs (ongoing TODO for end-to-end test)	Exporter.TrivyDb backlog

Budgets are enforced in automated tests where available; outstanding TODO/DOING items (see task boards) continue tracking gaps such as exporter determinism.

---

## 9 Testing

• Unit and integration tests live alongside each component (*.Tests).
• Shared fixtures come from StellaOps.Feedser.Testing and StellaOps.Feedser.Tests.Shared (linked via Directory.Build.props).
• Integration suites use ephemeral MongoDB and Redis via Testcontainers to validate end-to-end flow without external dependencies.

---