Add initial documentation
This commit is contained in:
372
docs/08_MODULE_SPECIFICATIONS.md
Normal file
372
docs/08_MODULE_SPECIFICATIONS.md
Normal file
@ -0,0 +1,372 @@
|
||||
# 8 · Detailed Module Specifications — **Stella Ops**
|
||||
_This document defines every backend/agent module that composes Stella Ops,
|
||||
their public contracts, configuration keys and extension points._
|
||||
|
||||
---
|
||||
|
||||
## 0 Scope
|
||||
|
||||
Describes **every .NET, and Angular project** that ships in the OSS Core, the plug‑in contracts they expose, and the runtime artefacts (Dockerfiles, Compose files) used to build and operate them. Commercial capabilities appear *only* as extension points.
|
||||
|
||||
---
|
||||
|
||||
## 1 Repository Layout (flat)
|
||||
|
||||
~~~text
|
||||
src/
|
||||
│ docker-compose.yml
|
||||
└─ docker-compose-library/
|
||||
│ ├─ docker-compose.no-deps.yml
|
||||
│ ├─ docker-compose.dep.redis.yml
|
||||
│ ├─ docker-compose.dep.mongo.yml
|
||||
│ ├─ docker-compose.dep.proxy.yml
|
||||
│ ├─ docker-compose.dep.repository.yml
|
||||
│ └─ docker-compose.local.yml
|
||||
└─ backend/
|
||||
│ ├─ Dockerfile
|
||||
│ ├─ StellaOps.Web/
|
||||
│ ├─ StellaOps.Common/
|
||||
│ ├─ StellaOps.Plugins/
|
||||
│ ├─ StellaOps.Configuration/
|
||||
│ ├─ StellaOps.Localization/
|
||||
│ ├─ StellaOps.TlsProvider.OpenSSL/
|
||||
│ ├─ StellaOps.TlsProvider.OpenSSLGost/
|
||||
│ ├─ StellaOps.TlsProvider.Plugin.CryptoPro/
|
||||
│ ├─ StellaOps.VulnerabilityDatabase/
|
||||
│ ├─ StellaOps.Scheduling/
|
||||
│ ├─ StellaOps.Scheduling.SbomsRescan/
|
||||
│ ├─ StellaOps.Scheduling.MutesExpire/
|
||||
│ ├─ StellaOps.Scheduling.Plugin.CommonCveFeed/
|
||||
│ ├─ StellaOps.Scheduling.Plugin.RussianCveFeed/
|
||||
│ ├─ StellaOps.Scanners.Trivy/
|
||||
│ ├─ StellaOps.Quota/
|
||||
│ ├─ StellaOps.Reporting/
|
||||
│ ├─ StellaOps.Notifications/
|
||||
│ ├─ StellaOps.Notifications.Email/
|
||||
│ ├─ StellaOps.Notifications.Plugin.MsTeams/
|
||||
│ ├─ StellaOps.Authority/
|
||||
│ ├─ StellaOps.Authority.AD/
|
||||
│ ├─ StellaOps.Agent.Santech/
|
||||
│ └─ StellaOps.Agent.Zastava/
|
||||
└─ frontend/
|
||||
├─ Dockerfile
|
||||
├─ angular.json
|
||||
├─ stella-ops-ui/
|
||||
└─ libs/
|
||||
├─ dashboard/
|
||||
├─ scans/
|
||||
├─ settings/
|
||||
├─ core-ui/
|
||||
└─ i18n/
|
||||
~~~
|
||||
|
||||
All projects are referenced by **`StellaOps.sln`**; `dotnet publish -c Release -p:PublishSingleFile=true` builds a self‑contained **`StellaOps.Api`** binary (plug‑ins load at runtime).
|
||||
|
||||
---
|
||||
|
||||
## 2 Shared Libraries
|
||||
|
||||
| Project | Purpose | Key Interfaces |
|
||||
|---------|---------|----------------|
|
||||
| `StellaOps.Common` | Serilog sinks, Redis key helpers, DTO primitives. | `RedisKeys`, `Result<T>` |
|
||||
| `StellaOps.Plugins` | Plug‑in contracts + Cosign verification. | `IStellaPlugin`, `IScannerRunner`, `ITlsProvider`, `IScheduleJob` |
|
||||
| `StellaOps.Localization` | Loads JSON locale bundles (backend & Angular). | `ILocaleProvider`, `CultureMiddleware` |
|
||||
|
||||
Angular JSON‑bundle workflow matches the official i18n guide .
|
||||
|
||||
---
|
||||
|
||||
## 3 Core Back‑end Projects
|
||||
|
||||
| Project | Responsibility | Extensibility |
|
||||
|---------|----------------|---------------|
|
||||
| **`StellaOps.Api`** | ASP.NET host; source‑gen auto‑wires module endpoints. | Attributes `[MapRestController]`, `[MapHealth]`. |
|
||||
| **`StellaOps.Configuration`** | Bind `appsettings.json` → typed options; `/health`. | `IConfigValidator`. |
|
||||
| **`StellaOps.Quota`** | Enforces **Free‑tier quota** (333 scans/day) with early‑warning banner, 5 s soft back‑off, 60 s wait‑wall. | Swappable via `IQuotaStore` (e.g., Postgres). |
|
||||
| **`StellaOps.JwtIssuer` *(new)* | Issues, refreshes and validates **Client‑JWTs**. For offline sites it produces a 30‑day token during OUK build and again on every OUK import. | `ITokenSigner` (e.g., HSM) |
|
||||
| **`StellaOps.TlsProvider.OpenSSL`** | Default TLS suites. | New suites via `ITlsProvider` plug‑in. |
|
||||
| **`StellaOps.TlsProvider.OpenSSLGost`** | GOST suites. | — |
|
||||
| **`StellaOps.VulnerabilityDatabase`** | Feed‑merge CLI writing Redis. | `IAdditionalFeedSource` (OSV, GHSA, BDU). |
|
||||
| **`StellaOps.Scheduling`** | Hangfire host inside API . | Jobs via `IScheduleJob`. |
|
||||
| **`StellaOps.Scheduling.SbomsRescan`** | Nightly SBOM re‑scan (`0 2 * * *`). | — |
|
||||
| **`StellaOps.Scheduling.MutesExpire`** | Daily mute expiry cleanup. | — |
|
||||
| **`StellaOps.Scanners.Trivy`** | Trivy CLI for SBOM & image scans . | Other engines implement `IScannerRunner`. |
|
||||
| **`StellaOps.Reporting`** | RazorLight HTML reports. | `IReportRenderer` for SARIF, CycloneDX. |
|
||||
| **`StellaOps.Notifications`** | DI contracts for alerts. | `INotifier`. |
|
||||
| **`StellaOps.Notifications.Email`** | SMTP channel. | — |
|
||||
| **`StellaOps.Authority`** | OAuth2 / OIDC via OpenIddict 4 . | External IdPs via plug‑in. |
|
||||
| **`StellaOps.Registry`** | read‑only Docker registry for agents + SBOM‑builder | Registry v2 (nginx‑hardened) | `IRegistryProvider` |
|
||||
| **`StellaOps.MutePolicies`** | store YAML / Rego policies, validate & version | MongoDB + Redis | `IPolicyStore` |
|
||||
| **`StellaOps.Attestor`** *(TODO)*| SLSA provenance + Rekor verification | Sigstore Rekor | `IAttestor` |
|
||||
|
||||
## 3 · Module Details
|
||||
|
||||
> _Only contracts and configuration that may change in the next two quarters are shown; for stable, unchanging keys see the inline XML‑doc in the codebase._
|
||||
|
||||
### 3.1. StellaOps.Configuration
|
||||
|
||||
* **Responsibility** – parse environment variables or `appsettings.json`; expose `/health`, `/metrics`.
|
||||
* **Key extension point** – `IConfigValidator` → validate & normalise custom settings before DI builds.
|
||||
|
||||
### 3.2. StellaOps.Authority
|
||||
|
||||
* **Responsibility** – ship with OpenIddict 6, supporting *client‑credentials* and *password* grants.
|
||||
* `IIdentityProvider` plug‑in can delegate token issuance to LDAP, SAML, Keycloak …
|
||||
|
||||
|
||||
### 3.3. StellaOps.Scanners
|
||||
|
||||
* **Primary flow** – SBOM‑first; falls back to image‑unpack if SBOM absent.
|
||||
* **Multi‑Format Support** – side‑car `.sbom.type` file; auto‑detects (`SPDXID:` or `bomFormat` heuristics).
|
||||
* **Delta Layer Workflow** – `POST /layers/missing` (`SET DIFF` on Redis) responds < 20 ms; SanTech passes only new layers.
|
||||
* **Plug‑in contract evolution**
|
||||
|
||||
```csharp
|
||||
// current
|
||||
Task<ScanResult> RunAsync(Stream sbomJson, CancellationToken ct);
|
||||
|
||||
// v2 (preferred)
|
||||
Task<ScanResult> RunAsync(Stream sbom, SbomFormat fmt, CancellationToken ct);
|
||||
```
|
||||
|
||||
### 3.5 StellOps.Registry
|
||||
|
||||
* **Purpose** – internal, anonymous **read‑only** Docker registry to avoid GHCR / Docker Hub pulls.
|
||||
* **Deployment** – container `stellops.registry:2`; mounted volume `/var/lib/registry`; optional TLS via env vars.
|
||||
|
||||
| Key | Default | Notes |
|
||||
|----------------------------------|---------|---------------------------------|
|
||||
| `REGISTRY_READONLY` | `true` | Forces 403 on PUT, 405 on DELETE |
|
||||
| `REGISTRY_STORAGE_DELETE_ENABLED`| `false` | Immutable tags |
|
||||
|
||||
**Plug‑in contract** — `IRegistryProvider.PullAsync(string imageRef)` for mapping to Artifactory, Harbor, etc.
|
||||
|
||||
---
|
||||
|
||||
### 3.6 StellaOps.MutePolicies
|
||||
|
||||
* **Purpose** – central Policy‑as‑Code store (YAML v1 now, Rego soon).
|
||||
* **Persistence** – current live rules in Redis (`policies:active`); immutable commits in Mongo `policies_history`.
|
||||
|
||||
| REST verb | Path | Description |
|
||||
|-----------|---------------------|---------------------------|
|
||||
| `GET` | `/policy/export` | download active YAML |
|
||||
| `POST` | `/policy/import` | upload YAML / Rego file |
|
||||
| `POST` | `/policy/validate` | lint without persisting |
|
||||
|
||||
**CLI** – SanTech gains `--policy-file scan-policy.yaml`.
|
||||
|
||||
**Plug‑in contract** — `IPolicyStore` for GitOps back‑ends, Vault, etc.
|
||||
|
||||
---
|
||||
|
||||
### 3.7. StellaOps.Attestor *(Planned – Q1‑2026)*
|
||||
|
||||
Handles SLSA provenance docs and Rekor log verification.
|
||||
|
||||
```csharp
|
||||
public interface IAttestor {
|
||||
Task<ProvenanceDoc> CreateAsync(ImageRef img, Sbom sbom);
|
||||
Task<bool> VerifyAsync(ProvenanceDoc doc);
|
||||
}
|
||||
```
|
||||
|
||||
### 3.7. StellaOps.FeedMerger
|
||||
|
||||
Nightly Hangfire job (01:00) merges NVD JSON; plug‑ins can provide ISourceFeed for OSV, GHSA, BDU feeds.
|
||||
|
||||
### 3.8. StellOps.Tls
|
||||
|
||||
Abstracts TLS stack; default OpenSSL; ITlsProvider lets enterprises swap in GOST or SM cipher suites.
|
||||
|
||||
### 3.9. StellaOps.Reporting
|
||||
|
||||
HTML / PDF generation via RazorLight; custom renderers via IReportRenderer.
|
||||
|
||||
### 3.10 UI
|
||||
|
||||
Angular 17 SPA; lazy‑loaded feature modules, standalone component routes for UI plug‑ins.
|
||||
|
||||
Static Go daemon / k8s DaemonSet; watches Docker/CRI‑O events; uploads SBOMs; optional enforce mode via policy plug‑in.
|
||||
|
||||
### 3.11 StellaOps.Quota — **Free‑Tier Daily Quota Service**
|
||||
|
||||
**Responsibility**
|
||||
|
||||
* Track per‑token scan count (`quota:<token>` key in Redis).
|
||||
* Reset counters at **00:00 UTC** with key TTL.
|
||||
* Inject HTTP headers
|
||||
* `X‑Stella‑Quota‑Remaining`
|
||||
* `X‑Stella‑Reset`
|
||||
* Apply adaptive throttling:
|
||||
* scans 0‑199 → normal;
|
||||
* scans 200‑332 → UI banner flag `X‑Stella‑Quota‑Warn:true`;
|
||||
* scans ≥ 333 → respond **`429`** with escalating `Retry‑After` values
|
||||
(5 s for first 30 hits, then 60 s).
|
||||
* **Offline token awareness** — if `token.valid == false` and
|
||||
`OfflineMode == true`, return HTTP *451 ComplianceBlock* so that CLI gives a
|
||||
clear actionable error.
|
||||
* New config:
|
||||
|
||||
```json
|
||||
"Quota": {
|
||||
"OfflineGraceDays": 7 // show banner this many days before token expiry
|
||||
}
|
||||
```
|
||||
|
||||
**Interface**
|
||||
|
||||
```csharp
|
||||
public interface IQuotaService
|
||||
{
|
||||
/// <summary>Returns true when the call is allowed.</summary>
|
||||
Task<QuotaVerdict> CheckAsync(string token, CancellationToken ct);
|
||||
}
|
||||
|
||||
public readonly record struct QuotaVerdict(
|
||||
bool IsAllowed,
|
||||
int Remaining,
|
||||
DateTimeOffset ResetUtc,
|
||||
TimeSpan RetryAfter);
|
||||
```
|
||||
|
||||
**Configuration** (`appsettings.json` keys)
|
||||
|
||||
```json
|
||||
"Quota": {
|
||||
"FreeTierDailyLimit": 333,
|
||||
"WarnThreshold": 200,
|
||||
"SoftRetrySeconds": 5,
|
||||
"HardRetrySeconds": 60
|
||||
}
|
||||
```
|
||||
|
||||
**Extensibility**
|
||||
|
||||
* Override storage by providing an `IQuotaStore` plug‑in for Postgres or Mongo.
|
||||
* UI plug‑ins can subscribe to SSE `/quota/events` for custom dashboards.
|
||||
|
||||
### 3.12 StellaOps.JwtIssuer — new section
|
||||
|
||||
|API |Path| Notes|
|
||||
|-----|----|-------|
|
||||
|`POST /token/offline` | Admin‑only. | Generates a 30 d Client‑JWT for air‑gapped clusters; returns ZIP that the admin can copy to the target host.
|
||||
|
||||
*OUK hook*
|
||||
|
||||
* OUK builder calls JwtIssuer.SignOfflineToken(exp=+30d).
|
||||
* Drops client.jwt into ouk/root/.
|
||||
* Backend OUK importer places file under /var/lib/stella/tokens/.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Compose / Helm Snippet (reference)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
registry:
|
||||
image: stellops.registry:2
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
REGISTRY_READONLY: "true"
|
||||
volumes:
|
||||
- ./_registry:/var/lib/registry
|
||||
ports:
|
||||
- "5000:5000"
|
||||
|
||||
backend:
|
||||
image: registry.local/stellops/backend:${TAG}
|
||||
depends_on: [registry, redis]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4 Plug‑ins (sign‑required)
|
||||
|
||||
| Plug‑in | Contract | Notes |
|
||||
|---------|----------|-------|
|
||||
| `StellaOps.Notifications.Plugin.MsTeams` | `INotifier` | Sends cards to Teams webhooks. |
|
||||
| `StellaOps.Authority.AD` | `IIdentityProvider` | LDAP/Active‑Directory token issue. |
|
||||
| `StellaOps.Scheduling.Plugin.CommonCveFeed` | `IScheduleJob` | Merges OSV & NVD JSON hourly . |
|
||||
| `StellaOps.Scheduling.Plugin.RussianCveFeed` | `IScheduleJob` | Imports BDU XML daily. |
|
||||
| `StellaOps.TlsProvider.Plugin.CryptoPro` | `ITlsProvider` | Binds CryptoPro shared libs. |
|
||||
|
||||
Cosign signatures are mandatory; loader rejects unsigned DLLs when `DisableUnsigned=false`.
|
||||
|
||||
---
|
||||
|
||||
## 5 Agents
|
||||
|
||||
### 5.1 `StellaOps.Agent.Santech`
|
||||
|
||||
Distroless CLI;
|
||||
Returns exit‑code 1 on policy violation, enabling CI blocking.
|
||||
* **Role** – CI helper: Build SBOM, call `/scan`, exit non‑zero on high severity.
|
||||
* **Flags** – `--engine`, `--threshold`, `--registry-pull-token`, `--pdf-out`, `--delta`, `--sbom-type`, `--policy-file.`.
|
||||
* **Auth** – OAuth2 *scanner* scope.
|
||||
|
||||
### 5.2 `StellaOps.Agent.Zastava`
|
||||
|
||||
* **Role** – Passive container inventory → uploads SBOMs via `/agent/sbom`.
|
||||
* **Modes** – `off`, `inventory` (Core default).
|
||||
* No kernel driver (unlike Falco) .
|
||||
|
||||
---
|
||||
|
||||
## 6 Angular Front‑end
|
||||
|
||||
| Package | Path | Feature | Lazy |
|
||||
|---------|------|---------|------|
|
||||
| **App** | `frontend/stella-ops-ui/` | Shell, auth guards. | — |
|
||||
| `dashboard` | `libs/dashboard/` | Live metrics tiles. | ✔ |
|
||||
| `scans` | `libs/scans/` | List, detail, mute, diff. | ✔ |
|
||||
| `settings` | `libs/settings/` | Feed cron, workers, TLS switch. | ✔ |
|
||||
| `core-ui` | `libs/core-ui/` | Tailwind components. | — |
|
||||
| `i18n` | `libs/i18n/` | Runtime locale switch, pipe. | — |
|
||||
|
||||
Lazy loading of workspace libs follows Nx/Angular guidance .
|
||||
|
||||
---
|
||||
|
||||
## 7 Docker Artefacts
|
||||
|
||||
### 7.1 Dockerfiles
|
||||
|
||||
* **`backend/Dockerfile`** – multi‑stage .NET 9; single‑file publish; distroless runtime .
|
||||
* **`frontend/Dockerfile`** – Node 20 build → Nginx static serve.
|
||||
* Every plug‑in repo may include its own Dockerfile when shipping side‑cars (e.g., custom scanner).
|
||||
|
||||
### 7.2 Compose Stacks
|
||||
|
||||
* **`docker-compose.yml`**
|
||||
* Extends above with Redis 7 and Mongo 7 for small on‑prem installs.
|
||||
|
||||
* **`docker-compose.no-deps.yml`**
|
||||
* backend, frontend, Trivy, Maven proxy.
|
||||
* Assumes external Redis & Mongo.
|
||||
|
||||
* **`docker-compose.local.yml`**
|
||||
* Build images from local source and bring up backend, frontend, Redis, Mongo, Trivy, Maven proxy for dev‑loop.
|
||||
|
||||
Docker Compose override precedence matches official docs .
|
||||
|
||||
---
|
||||
|
||||
## 8 Performance Budget
|
||||
|
||||
| Flow | P95 target | Bottleneck |
|
||||
|------|-----------:|-----------|
|
||||
| SBOM fast‑path | ≤ 5 s | Redis queue depth (keep P99 < 1 ms) |
|
||||
| Image‑unpack | ≤ 10 s | Trivy layer unpack. |
|
||||
| Nightly re‑scan | 80 SBOM/s | Runner CPU. |
|
||||
|
||||
---
|
||||
|
||||
## Change Log
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---------|------|-------|
|
||||
| **v2.2** | 2025‑07‑11 | Flat layout; stella‑ops‑ui naming; Dockerfiles & 3 Compose stacks; agents and localisation library. |
|
||||
| v2.1 | 2025‑07‑11 | First flat‑structure draft. |
|
||||
|
||||
*(End of Module Specifications v2.2‑core)*
|
||||
Reference in New Issue
Block a user