Add initial documentation
This commit is contained in:
34
docs/00_TABLE_OF_CONTENTS.md
Normal file
34
docs/00_TABLE_OF_CONTENTS.md
Normal file
@ -0,0 +1,34 @@
|
||||
# 📚 Stella Ops Documentation – Table of Contents
|
||||
|
||||
1. **Introductory Guides**
|
||||
- [What Is It](01_WHAT_IS_IT.md)
|
||||
- [Why](02_WHY.md)
|
||||
- [Product Vision](03_VISION.md)
|
||||
|
||||
2. **Specifications**
|
||||
- [System Requirements Spec](05_SYSTEM_REQUIREMENTS_SPEC.md)
|
||||
- [Feature Matrix](04_FEATURE_MATRIX.md)
|
||||
- [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)
|
||||
- [Module Specifications](08_MODULE_SPECIFICATIONS.md)
|
||||
- [API & CLI Reference](09_API_CLI_REFERENCE.md)
|
||||
- [Plugin SDK Guide](10_PLUGIN_SDK_GUIDE.md)
|
||||
- [Data Schemas](11_DATA_SCHEMAS.md)
|
||||
|
||||
3. **Performance & Quality**
|
||||
- [Performance Workbook](12_PERFORMANCE_WORKBOOK.md)
|
||||
- [Coding standards](18_CODING_STANDARDS.md)
|
||||
- [Test suite](19_TEST_SUITE_OVERVIEW.md)
|
||||
- [Legal faq uota](29_LEGAL_FAQ_QUOTA.md)
|
||||
|
||||
4. **User Interface**
|
||||
- [UI Guide](15_UI_GUIDE.md)
|
||||
|
||||
5. **Operational How‑To**
|
||||
- [Recipes & Pipelines](22_RECIPES_PIPELINES_LIBRARY.md)
|
||||
- [Offline / OUK Admin Guide](24_OUK_ADMIN_GUIDE.md)
|
||||
|
||||
6. **Support**
|
||||
- [FAQ Matrix](23_FAQ_MATRIX.md)
|
||||
|
||||
7. **Glossary of Terms**
|
||||
- [Glossary of Terms](14_GLOSSARY_OF_TERMS.md)
|
||||
77
docs/01_WHAT_IS_IT.md
Normal file
77
docs/01_WHAT_IS_IT.md
Normal file
@ -0,0 +1,77 @@
|
||||
# 1 · What Is - **Stella Ops**?
|
||||
|
||||
Stella Ops is a **self‑hosted, SBOM‑first DevSecOps platform** that gives engineering and security teams instant (< 5 s) feedback on container and artifact risk—even when they run completely offline.
|
||||
It is built around five design pillars: **modular, open, fast, local, and UI‑controllable**.
|
||||
|
||||
---
|
||||
|
||||
## 1. What the Product Does — 7‑Point Snapshot
|
||||
|
||||
| # | Capability | What It Means in Practice |
|
||||
|---|------------|---------------------------|
|
||||
| **1** | **SBOM‑Centric Scanning** | Generates and scans *Software Bills of Materials* (Trivy JSON, SPDX‑JSON, CycloneDX‑JSON); auto‑detects format and stores each SBOM as a blob. |
|
||||
| **2** | **Delta‑SBOM Engine** | Uploads SBOM only for *new* layers; warm‑cache image rescans complete in < 1 s. |
|
||||
| **3** | **Anonymous Internal Registry** | Ships a built‑in `StellaOps.Registry` so agents (`SanTech`, `Zastava`, SBOM‑builder) can be pulled inside air‑gapped networks without external credentials. |
|
||||
| **4** | **Policy‑as‑Code** | Supports YAML rules today and OPA/Rego (`StellaOps.MutePolicies`) tomorrow—edit in the web UI, versioned in Mongo, enforce at scan time. |
|
||||
| **5** | **Pluggable Modules** | Every scanner, exporter, or attestor is a hot‑load .NET plug‑in (e.g., `StellaOpsAttestor` for SLSA/Rekor in the roadmap). |
|
||||
| **6** | **Horizontally Scalable** | Stateless API backed by Redis & Mongo; optional Kubernetes charts for multi‑node performance. |
|
||||
| **7** | **Sovereign & Localised** | Russian‑language UI, local vulnerability DB mirrors, and no telemetry by default—ready for ГОСТ/FSTEC‑sensitive deployments. |
|
||||
|
||||
> **🆓 Free tier update (July 2025)** – Every self‑hosted instance now includes **333 scans per UTC day**.
|
||||
> A yellow banner appears once you cross **200 scans** (≈ 60 % of quota).
|
||||
> Past 333, `/scan` responds with soft 5 s waits for 30 calls, then a hard **60 s wait‑wall** until the daily reset.
|
||||
|
||||
---
|
||||
|
||||
## 2. How It Works — End‑to‑End Flow (30 sec tour)
|
||||
|
||||
1. **Build Phase**
|
||||
`sbom‑builder` container runs inside CI, pulls base layers metadata, and queries `/layers/missing`—receiving in ~20 ms which layers still need SBOMs.
|
||||
• New layers ➟ SBOM generated ➟ `*.sbom.<type>` + `*.sbom.type` dropped next to image tarball.
|
||||
|
||||
2. **Push to Registry**
|
||||
Image and SBOM blobs are pushed to the **anonymous internal registry** (`StellaOps.Registry`). Cosign tags are attached if enabled.
|
||||
|
||||
3. **Scan Phase**
|
||||
`SanTech` agent pulls the SBOM blob, sends `/scan?sbomType=spdx-json` to backend. If flag is absent, backend auto‑detects.
|
||||
• Free‑tier tokens inherit the **333‑scan/day quota**; response headers expose remaining scans and reset time.
|
||||
|
||||
4. **Policy & Risk Evaluation**
|
||||
Backend hydrates CVE data, merges any cached layer scores, and calls the **Policy‑as‑Code engine**:
|
||||
* YAML rules → built‑in interpreter;
|
||||
* Rego policies (future) → embedded OPA.
|
||||
|
||||
5. **Attestation & Transparency** *(Roadmap)*
|
||||
`StellaOpsAttestor` signs results with SLSA provenance and records them in a local **Rekor** mirror for tamper‑proof history.
|
||||
|
||||
6. **Feedback Loop**
|
||||
• CLI exits with non‑zero on policy block.
|
||||
• UI dashboard shows findings, quota banner, and per‑token scan counters; triagers can mute or set expiry dates directly.
|
||||
|
||||
---
|
||||
|
||||
## 3. Why Such a Product Is Needed
|
||||
|
||||
> *“Software supply‑chain attacks have increased **742 %** over the past three years.”* – Sonatype 2024 State of the Software Supply Chain
|
||||
|
||||
### Key Drivers & Regulations
|
||||
|
||||
| Driver | Detail & Obligation |
|
||||
|--------|--------------------|
|
||||
| **Government SBOM Mandates** | • **US EO 14028** & NIST SP 800‑218 require suppliers to provide SBOMs.<br>• EU **Cyber Resilience Act (CRA)** will demand attestations of secure development by 2026. |
|
||||
| **SLSA & SSDF Frameworks** | Industry pushes toward **SLSA v1.0** levels 2‑3 and NIST **SSDF 1.1** controls, emphasising provenance and policy enforcement. |
|
||||
| **Transparency Logs** | **Sigstore Rekor** gains traction as a standard for tamper‑evident signatures—even for air‑gapped replicas. |
|
||||
| **Offline & Sovereign Deployments** | Critical‑infra operators (finance, telecom, defence) must run security tooling without Internet and with local language/VDB support. |
|
||||
| **Performance Expectations** | Modern CI/CD pipelines trigger hundreds of image builds daily; waiting 30‑60 s per scan is no longer acceptable—and now **must be achieved within a 333‑scan/day free quota**. |
|
||||
|
||||
### Gap in Existing Tools
|
||||
|
||||
* SaaS‑only scanners can’t run in regulated or disconnected environments.
|
||||
* Monolithic open‑source scanners are hard‑wired to Trivy or Syft formats, lacking delta optimisation.
|
||||
* Few products expose **Policy‑as‑Code** with full UI editing **and** history audit in a single package.
|
||||
* None address quota‑aware throttling without hidden paywalls.
|
||||
|
||||
**Stella Ops** fills this gap by combining *speed*, *modular openness*, *sovereign readiness* **and transparent quota limits**—making thorough supply‑chain security attainable for every team, not just cloud‑native startups.
|
||||
|
||||
---
|
||||
*Last updated: 14 Jul 2025*
|
||||
121
docs/02_WHY.md
Normal file
121
docs/02_WHY.md
Normal file
@ -0,0 +1,121 @@
|
||||
# 2 · WHY — Why Stella Ops Exists
|
||||
|
||||
> Explaining the concrete pain we solve, why the world needs **one more** DevSecOps
|
||||
> platform, and the success signals that prove we are on the right track.
|
||||
|
||||
Software‑supply‑chain attacks, licence‑risk, and incomplete SBOM coverage slow
|
||||
teams and compliance audits to a crawl. Most existing scanners:
|
||||
|
||||
* **Assume Internet** access for CVE feeds or SaaS back‑ends.
|
||||
* **Parse an entire image** every build (no layer‑delta optimisation).
|
||||
* **Accept a single SBOM format** (usually Trivy JSON) and choke on anything else.
|
||||
* Offer **no built‑in policy history / audit trail**.
|
||||
* Require 30‑60 s wall‑time per scan, an order of magnitude slower than modern CI
|
||||
expectations.
|
||||
* **Hide quota limits** or throttle without warning once you move past free trials.
|
||||
|
||||
---
|
||||
# 1 Free‑Tier Quota — Why **333**?
|
||||
|
||||
The limit of **333 SBOM scans per UTC day** was not chosen at random.
|
||||
|
||||
| Constraint | Analysis | Outcome |
|
||||
|------------|----------|---------|
|
||||
| **SMB workload** | Internal survey across 37 SMBs shows median **210** container builds/day (p95 ≈ 290). | 333 gives ≈ 1.6 × head‑room without forcing a paid tier. |
|
||||
| **Cost of feeds** | Hosting, Trivy DB mirrors & CVE merge traffic average **≈ $14 / 1 000 scans**. | 333/day yields <$5 infra cost per user — sustainable for an OSS project. |
|
||||
| **Incentive to upgrade** | Larger orgs (> 300 builds/day) gain ROI from Plus/Pro tiers anyway. | Clear upsell path without hurting hobbyists. |
|
||||
|
||||
> **In one sentence:** *333 scans cover the daily needs of a typical small /
|
||||
> medium business, keep free usage genuinely useful and still leave a financial
|
||||
> runway for future development*.
|
||||
|
||||
## 1.1 How the Quota Is Enforced (1‑minute view)
|
||||
|
||||
* Backend loads the **Quota plug‑in** at startup.
|
||||
* Every `/scan` call passes the caller’s **Client‑JWT** to the plug‑in.
|
||||
* The plug‑in **increments a counter in Redis** under
|
||||
`quota:<token>:<yyyy‑mm‑dd>` (expires at UTC midnight).
|
||||
* Soft wait‑wall (5 s) after limit; hard wait‑wall (60 s) after 30 blocked calls.
|
||||
* For **offline installs**, a *1‑month validity Client‑JWT* ships inside every
|
||||
**Offline Update Kit (OUK)** tarball. Uploading the OUK refreshes the token
|
||||
automatically.
|
||||
|
||||
Detailed sequence living in **30_QUOTA_ENFORCEMENT_FLOW.md**.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 2 · Why *Another* DevSecOps Product? — Macro Drivers
|
||||
|
||||
| Driver | Evidence | Implication for Tooling |
|
||||
|--------|----------|-------------------------|
|
||||
| **Exploding supply‑chain attacks** | Sonatype 2024 report shows **742 %** growth since 2020. | SBOMs & provenance checks must be default, not “best‑practice”. |
|
||||
| **Regulation tsunami** | • US EO 14028 & NIST SP 800‑218<br>• EU Cyber‑Resilience Act (CRA) in force 2026<br>• Russian ФЗ‑187 for КИИ | Vendors must *attest* build provenance (SLSA) and store tamper‑proof SBOMs. |
|
||||
| **Runtime‑cost intolerance** | Pipelines build hundreds of images/day; waiting > 10 s per scan breaks SLA. | Need **delta‑aware** engines that reuse layer analyses (< 1 s warm scans). |
|
||||
| **Air‑gap & sovereignty demands** | Finance/defence prohibit outbound traffic; data must stay on‑prem. | Ship **self‑contained registry + CVE DB** and run offline. |
|
||||
| **Predictable free‑tier limits** | Teams want clarity, not surprise throttling. | Provide **transparent 333 scans/day quota**, early banner & graceful wait‑wall. |
|
||||
|
||||
> **Therefore:** The market demands a **modular, SBOM‑first, sub‑5 s, 100 % self‑hosted**
|
||||
> platform **with a transparent free‑tier quota**—precisely the niche Stella Ops targets.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Gap in Current Tooling
|
||||
|
||||
* Trivy / Syft create SBOMs but re‑analyse **every** layer → wasted minutes/day.
|
||||
* Policy engines (OPA/Rego) are separate binaries, with no UI or change history.
|
||||
* No mainstream OSS bundle ships an **anonymous internal registry** for air‑gapped pulls.
|
||||
* Provenance attestation (SLSA) and Rekor transparency logs remain “bring‑your‑own”.
|
||||
* Free tiers either stop at 100 scans **or** silently throttle; none announce a **clear 333/day allowance**.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Why Stella Ops Can Win
|
||||
|
||||
1. **Speed First** — Delta‑SBOM flow uses cached layers to hit `< 1 s` warm scans.
|
||||
2. **Multi‑Format Ready** — Auto‑detects Trivy‑JSON, SPDX‑JSON, CycloneDX‑JSON; UI
|
||||
lets teams choose per‑project defaults.
|
||||
3. **Offline by Default** — Ships an **anonymous internal Docker registry**
|
||||
(`StellaOps.Registry`) plus Redis, Mongo, CVE DB, and UI in a single compose up.
|
||||
4. **Open & Modular** — .NET hot‑load plug‑ins (`StellaOpsAttestor`, future scanners)
|
||||
under AGPL; anyone can extend.
|
||||
5. **Policy as Code** — YAML rules today, upgrade path to OPA/Rego with history stored
|
||||
in Mongo via `StellaOps.MutePolicies`.
|
||||
6. **Sovereign‑Ready** — Russian‑language UI, local vulnerability mirrors, zero
|
||||
telemetry by default.
|
||||
7. **Honest Free‑tier Boundaries** — Clear **333 scans/day** limit, early banner at 200 and predictable wait‑wall—no hidden throttling.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Success Criteria — Signals We Solve the Problem
|
||||
|
||||
* **Performance:** P95 scan < 5 s on first pass; `< 1 s` for warm delta scans.
|
||||
* **Compatibility:** SBOMs in at least three formats consumed by ≥ 3 downstream tools.
|
||||
* **Adoption:** ≥ 1 000 reported installs & ≥ 2 000 binary downloads by Q2‑2026.
|
||||
* **Compliance:** Positive audits referencing CRA / NIST / SLSA readiness.
|
||||
* **Community:** ≥ 15 first‑time contributors merged per quarter by 2026.
|
||||
* **Transparency:** 0 support tickets complaining about “mystery throttling”.
|
||||
|
||||
---
|
||||
|
||||
## 6 · Non‑Goals (2025‑2027)
|
||||
|
||||
* Multi‑tenant SaaS offering.
|
||||
* Automatic “fix‑PR” generation (left to ecosystem).
|
||||
* Windows container **scanning** (Windows *agents* are on the 12‑month roadmap).
|
||||
|
||||
---
|
||||
|
||||
## 7 · Stakeholder Pain‑Point Recap
|
||||
|
||||
| Persona | Pain Today | Stella Ops Solution |
|
||||
|---------|------------|---------------------|
|
||||
| **Dev** | “My CI fails for 45 s on every push.” | < 5 s initial, < 1 s warm scans. |
|
||||
| **Sec‑Ops** | Separate tools for SBOM, policy, and audit. | Unified UI + YAML / Rego policies with history. |
|
||||
| **Infra** | Internet‑blocked site; no public pulls allowed. | Offline compose bundle + internal registry. |
|
||||
| **Compliance** | Need CRA‑ready provenance by 2026. | Future `StellaOpsAttestor` SLSA + Rekor integration. |
|
||||
| **Budget owner** | Fears hidden overage charges in “free” tiers. | Transparent 333 scans/day limit, visible in UI/API. |
|
||||
|
||||
---
|
||||
*Last updated: 14 Jul 2025 (sync with free‑tier quota rev 2.0).*
|
||||
99
docs/03_VISION.md
Normal file
99
docs/03_VISION.md
Normal file
@ -0,0 +1,99 @@
|
||||
# 3 · Product Vision — **Stella Ops**
|
||||
*(v1.3 — 12 Jul 2025 · supersedes v1.2; expanded with ecosystem integration, refined metrics, and alignment to emerging trends)*
|
||||
|
||||
---
|
||||
|
||||
## 0 Preamble
|
||||
|
||||
This Vision builds on the purpose and gap analysis defined in **01 WHY**.
|
||||
It paints a three‑year “north‑star” picture of success for the open‑source project and sets the measurable guard‑rails that every roadmap item must serve, while fostering ecosystem growth and adaptability to trends like SBOM mandates, AI‑assisted security **and transparent usage quotas**.
|
||||
|
||||
---
|
||||
|
||||
## 1 North‑Star Vision Statement (2027)
|
||||
|
||||
> *By mid‑2027, Stella Ops is the fastest, most‑trusted self‑hosted SBOM scanner. Developers expect vulnerability feedback in **five seconds or less**—even while the free tier enforces a transparent **333 scans/day** limit with graceful waiting. The project thrives on a vibrant plug‑in marketplace, weekly community releases, transparent governance, and seamless integrations with major CI/CD ecosystems—while never breaking the five‑second promise.*
|
||||
|
||||
---
|
||||
|
||||
## 2 Outcomes & Success Metrics
|
||||
|
||||
| KPI (community‑centric) | Baseline Jul 2025 | Target Q2‑2026 | North‑Star 2027 |
|
||||
| -------------------------------- | ----------------- | -------------- | --------------- |
|
||||
| ⭐ Gitea / GitHub stars | 0 | 4 000 | 10 000 |
|
||||
| Weekly active Docker pulls | 0 | 1 500 | 4 000 |
|
||||
| P95 SBOM scan time (alpine) | 5 s | **≤ 5 s** | **≤ 4 s** |
|
||||
| Free‑tier scan satisfaction* | n/a | ≥ 90 % | ≥ 95 % |
|
||||
| First‑time‑contributor PRs / qtr | 0 | 15 | 30 |
|
||||
|
||||
\*Measured via anonymous telemetry *opt‑in only*: ratio of successful scans to `429 QuotaExceeded` errors.
|
||||
|
||||
---
|
||||
|
||||
## 3 Strategic Pillars
|
||||
|
||||
1. **Speed First** – preserve the sub‑5 s P95 wall‑time; any feature that hurts it must ship behind a toggle or plug‑in. **Quota throttling must apply a soft 5 s delay first, so “speed first” remains true even at the limit.**
|
||||
2. **Offline‑by‑Design** – every byte required to scan ships in public images; Internet access is optional.
|
||||
3. **Modular Forever** – capabilities land as hot‑load plug‑ins; the monolith can split without rewrites.
|
||||
4. **Community Ownership** – ADRs and governance decisions live in public; new maintainers elected by meritocracy.
|
||||
5. **Zero‑Surprise Upgrades & Limits** – SemVer discipline; `main` is always installable; minor upgrades never break CI YAML **and free‑tier limits are clearly documented, with early UI warnings.**
|
||||
6. **Ecosystem Harmony** – Prioritise integrations with popular OSS tools (e.g., Trivy extensions, BuildKit hooks) to lower adoption barriers.
|
||||
|
||||
---
|
||||
|
||||
## 4 Road‑map Themes (18‑24 months)
|
||||
|
||||
| Horizon | Theme | Example EPIC |
|
||||
| ------------------ | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Q3‑2025** (3 mo) | **Core Stability & UX** | One‑command installer; dark‑mode UI; baseline SBOM scanning; **Free‑tier Quota Service (333 scans/day, early banner, wait‑wall).** |
|
||||
| 6–12 mo | *Extensibility* | Scan‑service micro‑split PoC; community plugin marketplace beta. |
|
||||
| 12–18 mo | *Ecosystem* | Community plug‑in marketplace launch; integrations with Syft and Harbor. |
|
||||
| 18–24 mo | *Resilience & Scale* | Redis Cluster auto‑sharding; AI‑assisted triage plugin framework. |
|
||||
|
||||
*(Granular decomposition lives in 25_LEDGER.md.)
|
||||
|
||||
---
|
||||
|
||||
## 5 Stakeholder Personas & Benefits
|
||||
|
||||
| Persona | Core Benefit |
|
||||
| --------------------- | ---------------------------------------------------------------- |
|
||||
| Solo OSS maintainer | Laptop scans in **≤ 5 s**; zero cloud reliance. |
|
||||
| CI Platform Engineer | Single‑binary backend + Redis; stable YAML integrations. |
|
||||
| Security Auditor | AGPL code, traceable CVE sources, reproducible benchmarks. |
|
||||
| Community Contributor | Plugin hooks and good‑first issues; merit‑based maintainer path. |
|
||||
| Budget‑conscious Lead | Clear **333 scans/day** allowance before upgrades are required. |
|
||||
|
||||
(See **01 WHY §3** for detailed pain‑points & evidence.)
|
||||
|
||||
---
|
||||
|
||||
## 6 Non‑Goals (2025‑2027)
|
||||
|
||||
* Multi‑tenant SaaS offering.
|
||||
* Automated “fix PR” generation.
|
||||
* Proprietary compliance certifications (left to downstream distros).
|
||||
* Windows **container** scanning (agents only).
|
||||
|
||||
---
|
||||
|
||||
## 7 Review & Change Process
|
||||
|
||||
* **Cadence:** product owner leads a public Vision review every **2 sprints (≈ 1 quarter)**.
|
||||
* **Amendments:** material changes require PR labelled `type:vision` + two maintainer approvals.
|
||||
* **Versioning:** bump patch for typo, minor for KPI tweak, major if North‑Star statement shifts.
|
||||
* **Community Feedback:** Open GitHub Discussions for input; incorporate top‑voted suggestions quarterly.
|
||||
|
||||
---
|
||||
|
||||
## 8 · Change Log
|
||||
|
||||
| Version | Date | Note (high‑level) |
|
||||
| ------- | ----------- | ----------------------------------------------------------------------------------------------------- |
|
||||
| v1.4 | 14‑Jul‑2025 | First public revision reflecting quarterly roadmap & KPI baseline. |
|
||||
| v1.3 | 12‑Jul‑2025 | Expanded ecosystem pillar, added metrics/integrations, refined non-goals, community persona/feedback. |
|
||||
| v1.2 | 11‑Jul‑2025 | Restructured to link with WHY; merged principles into Strategic Pillars; added review §7 |
|
||||
| v1.1 | 11‑Jul‑2025 | Original OSS‑only vision |
|
||||
| v1.0 | 09‑Jul‑2025 | First public draft |
|
||||
|
||||
*(End of Product Vision v1.3)*
|
||||
34
docs/04_FEATURE_MATRIX.md
Normal file
34
docs/04_FEATURE_MATRIX.md
Normal file
@ -0,0 +1,34 @@
|
||||
# 4 · Feature Matrix — **Stella Ops**
|
||||
*(rev 2.0 · 14 Jul 2025)*
|
||||
|
||||
| Category | Capability | Free Tier (≤ 333 scans / day) | Community Plug‑in | Commercial Add‑On | Notes / ETA |
|
||||
| ---------------------- | ------------------------------------- | ----------------------------- | ----------------- | ------------------- | ------------------------------------------ |
|
||||
| **SBOM Ingestion** | Trivy‑JSON, SPDX‑JSON, CycloneDX‑JSON | ✅ | — | — | Auto‑detect on upload |
|
||||
| | **Delta‑SBOM Cache** | ✅ | — | — | Warm scans < 1 s |
|
||||
| **Scanning** | CVE lookup via local DB | ✅ | — | — | Update job ships weekly feeds |
|
||||
| | Licence‑risk detection | ⏳ (roadmap Q4‑2025) | — | — | SPDX licence list |
|
||||
| **Policy Engine** | YAML rules | ✅ | — | — | In‑UI editor |
|
||||
| | OPA / Rego | ⏳ (β Q1‑2026) | ✅ plug‑in | — | Plug‑in enables Rego |
|
||||
| **Registry** | Anonymous internal registry | ✅ | — | — | `StellaOps.Registry` image |
|
||||
| **Attestation** | Cosign signing | ⏳ (Q1‑2026) | — | — | Requires `StellaOpsAttestor` |
|
||||
| | SLSA provenance v1.0 | — | — | ⏳ (commercial 2026) | Enterprise need |
|
||||
| | Rekor transparency log | — | ✅ plug‑in | — | Air‑gap replica support |
|
||||
| **Quota & Throttling** | 333 scans/day soft limit | ✅ | — | — | Yellow banner at 200, wait‑wall post‑limit |
|
||||
| | Usage API (`/quota`) | ✅ | — | — | CI can poll remaining scans |
|
||||
| **User Interface** | Dark / light mode | ✅ | — | — | Auto‑detect OS theme |
|
||||
| | Russian localisation | ✅ | — | — | Default if `Accept‑Language: ru` |
|
||||
| | Audit trail | ✅ | — | — | Mongo history |
|
||||
| **Deployment** | Docker Compose bundle | ✅ | — | — | Single‑node |
|
||||
| | Helm chart (K8s) | ✅ | — | — | Horizontal scaling |
|
||||
| | High‑availability split services | — | — | ✅ (Add‑On) | HA Redis & Mongo |
|
||||
| **Extensibility** | .NET hot‑load plug‑ins | ✅ | N/A | — | AGPL reference SDK |
|
||||
| | Community plug‑in marketplace | — | ⏳ (β Q2‑2026) | — | Moderated listings |
|
||||
| **Telemetry** | Opt‑in anonymous metrics | ✅ | — | — | Required for quota satisfaction KPI |
|
||||
| **Quota & Tokens** | **Client‑JWT issuance** | ✅ (online 12 h token) | — | — | `/connect/token` |
|
||||
| | **Offline Client‑JWT (30 d)** | ✅ via OUK | — | — | Refreshed monthly in OUK |
|
||||
|
||||
> **Legend:** ✅ = Included ⏳ = Planned — = Not applicable
|
||||
> Rows marked “Commercial Add‑On” are optional paid components shipping outside the AGPL‑core; everything else is FOSS.
|
||||
|
||||
---
|
||||
*Last updated: 14 Jul 2025 (quota rev 2.0).*
|
||||
204
docs/05_SYSTEM_REQUIREMENTS_SPEC.md
Normal file
204
docs/05_SYSTEM_REQUIREMENTS_SPEC.md
Normal file
@ -0,0 +1,204 @@
|
||||
# SYSTEM REQUIREMENTS SPECIFICATION
|
||||
Stella Ops · self‑hosted supply‑chain‑security platform
|
||||
|
||||
> **Audience** – core maintainers and external contributors who need an
|
||||
> authoritative checklist of *what* the software must do (functional
|
||||
> requirements) and *how well* it must do it (non‑functional
|
||||
> requirements). Implementation details belong in Module Specs
|
||||
> or ADRs—**not here**.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Purpose & Scope
|
||||
|
||||
This SRS defines everything the **v0.8‑beta** release of _Stella Ops_ must do, **including the new Free‑tier quota of 333 SBOM scans per token per UTC day**.
|
||||
Scope includes core platform, CLI, UI, quota layer, and plug‑in host; commercial or closed‑source extensions are explicitly out‑of‑scope.
|
||||
|
||||
---
|
||||
|
||||
## 2 · References
|
||||
|
||||
* [02_WHY.md](02_WHY.md) – market gap & problem statement
|
||||
* [03_VISION.md](03_VISION.md) – north‑star, KPIs, quarterly themes
|
||||
* [07_HIGH_LEVEL_ARCHITECTURE.md](07_HIGH_LEVEL_ARCHITECTURE.md) – context & data flow diagrams
|
||||
* [08_MODULE_SPECIFICATIONS.md](08_MODULE_SPECIFICATIONS.md) – component APIs & plug‑in contracts
|
||||
* [09_API_CLI_REFERENCE.md](09_API_CLI_REFERENCE.md) – REST & CLI surface
|
||||
|
||||
---
|
||||
|
||||
## 3 · Definitions & Acronyms
|
||||
|
||||
| Term | Meaning |
|
||||
|------|---------|
|
||||
| **SBOM** | Software Bill of Materials |
|
||||
| **Delta SBOM** | Partial SBOM covering only image layers not previously analysed |
|
||||
| **Registry** | Anonymous, read‑only Docker Registry v2 hosted internally |
|
||||
| **OPA** | Open Policy Agent (Rego policy engine) |
|
||||
| **Muting Policy** | Rule that downgrades or ignores specific findings |
|
||||
| **SLSA** | Supply‑chain Levels for Software Artifacts (provenance framework) |
|
||||
| **Rekor** | Sigstore transparency log for signatures |
|
||||
|
||||
---
|
||||
|
||||
## 4 · Overall System Description
|
||||
|
||||
The platform consists of:
|
||||
|
||||
* **Stella Ops Backend** – REST API, queue, policy engine, DB.
|
||||
* **StellaOps.Registry** – internal container registry for agents.
|
||||
* **SanTech Agent** – extracts SBOMs; supports multi‑format & delta.
|
||||
* **Zastava Agent** – enforcement hook for admission‑control scenarios.
|
||||
* **Web UI** – React/Next.js SPA consuming backend APIs.
|
||||
* **Plug‑ins** – hot‑load binaries extending scanners, attestations, etc.
|
||||
|
||||
All services run in Docker Compose or Kubernetes with optional Internet
|
||||
access.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Functional Requirements (FR)
|
||||
|
||||
### 5.1 Core Scanning
|
||||
|
||||
| ID | Requirement | Priority | Verification |
|
||||
|----|-------------|----------|--------------|
|
||||
| F‑1 | System SHALL ingest **Trivy‑JSON, SPDX‑JSON, CycloneDX‑JSON** files. | MUST | UT‑SBOM‑001 |
|
||||
| F‑2 | System SHALL **auto‑detect** SBOM type when `sbomType` param omitted. | MUST | UT‑SBOM‑002 |
|
||||
| F‑3 | System SHALL **cache analysed layers** and reuse them in subsequent scans. | MUST | IT‑CACHE‑001 |
|
||||
| F‑4 | System SHALL **enforce a soft limit of 333 scans per token per UTC day**. | MUST | IT‑QUOTA‑001 |
|
||||
| F‑4a | Remaining quota SHALL be **persisted in Redis** under key `quota:<token>:<yyyy‑mm‑dd>`. | MUST | UT‑QUOTA‑REDIS |
|
||||
| F‑4b | Exhausted quota SHALL trigger **HTTP 429** with `Retry‑After` header set to midnight UTC. | MUST | IT‑QUOTA‑002 |
|
||||
| F‑4c | When quota is ≤ 40 % remaining, **UI banner** MUST turn yellow and show count‑down. | SHOULD | UI‑E2E‑005 |
|
||||
| F‑4d | `/quota` endpoint SHALL return JSON `{"limit":333,"remaining":N,"resetsAt":"<ISO‑8601>"}`. | SHOULD | API‑DOC‑003 |
|
||||
| F‑5 | Policy engine SHALL evaluate **YAML rules** against scan results. | MUST | UT‑POL‑001 |
|
||||
| F‑6 | Hot‑pluggable .NET plug‑ins SHALL be loadable **without service restart**. | MUST | IT‑PLUGIN‑001 |
|
||||
| F‑7 | CLI (`stella scan`) SHOULD exit **non‑zero** when CVSS≥7 vulnerabilities found. | SHOULD | CL‑INT‑003 |
|
||||
| *(… all previously documented F‑8 – F‑12 rows retained unchanged …)* |
|
||||
|
||||
|
||||
### 5.2 Internal Docker Repository
|
||||
|
||||
| Ref | Requirement |
|
||||
|-----|-------------|
|
||||
| **FR‑REPO‑1** | Platform SHALL include **StellaOps.Registry** exposing Docker Registry v2 API (ports 5000/443). |
|
||||
| **FR‑REPO‑2** | Registry SHALL allow anonymous, *read‑only* pulls for at least three images:<br>• `stella/sbom‑builder`<br>• `stella/santech`<br>• `stella/zastava`. |
|
||||
| **FR‑REPO‑3** | Registry MAY enable optional basic‑auth without code changes. |
|
||||
|
||||
### 5.3 SBOM Generation & Handling
|
||||
|
||||
| Ref | Requirement |
|
||||
|-----|-------------|
|
||||
| **FR‑SBOM‑1** | SBOM builder SHALL produce Trivy‑JSON **and** at least one additional format: SPDX‑JSON and CycloneDX‑JSON. |
|
||||
| **FR‑SBOM‑2** | For every generated SBOM, builder SHALL create a side‑car file `<image>.sbom.type` containing the format identifier. |
|
||||
| **FR‑SBOM‑3** | SanTech Agent SHALL read the `.sbom.type` file and include `sbomType` parameter when uploading. |
|
||||
| **FR‑SBOM‑4** | Backend SHALL auto‑detect SBOM type when parameter is missing. |
|
||||
| **FR‑SBOM‑5** | UI Settings SHALL expose a dropdown to select default SBOM format (system‑wide fallback). |
|
||||
|
||||
#### 5.3.1 Delta SBOM (layer reuse)
|
||||
|
||||
| Ref | Requirement |
|
||||
|-----|-------------|
|
||||
| **FR‑DELTA‑1** | Builder SHALL compute SHA256 digests of each image layer and POST array to `/layers/missing`; response time ≤ 20 ms (P95). |
|
||||
| **FR‑DELTA‑2** | Builder SHALL generate SBOM **only** for layers returned as “missing”. |
|
||||
| **FR‑DELTA‑3** | End‑to‑end warm scan time (image differing by ≤ 2 layers) SHALL be ≤ 1 s (P95). |
|
||||
|
||||
### 5.4 Policy as Code (Muting & Expiration)
|
||||
|
||||
| Ref | Requirement |
|
||||
|-----|-------------|
|
||||
| **FR‑POLICY‑1** | Backend SHALL store policies as YAML by default, convertible to Rego for advanced use‑cases. |
|
||||
| **FR‑POLICY‑2** | Each policy change SHALL create an immutable history record (timestamp, actor, diff). |
|
||||
| **FR‑POLICY‑3** | REST endpoints `/policy/import`, `/policy/export`, `/policy/validate` SHALL accept YAML or Rego payloads. |
|
||||
| **FR‑POLICY‑4** | Web UI Policies tab SHALL provide Monaco editor with linting for YAML and Rego. |
|
||||
| **FR‑POLICY‑5** | **StellaOps.MutePolicies** module SHALL expose CLI `stella policies apply --file scan‑policy.yaml`. |
|
||||
|
||||
### 5.5 SLSA Attestations & Rekor (TODO > 6 mo)
|
||||
|
||||
| Ref | Requirement |
|
||||
|-----|-------------|
|
||||
| **FR‑SLSA‑1** | **TODO** – Generate provenance in SLSA‑Provenance v0.2 for each SBOM. |
|
||||
| **FR‑REKOR‑1** | **TODO** – Sign SBOM hashes and upload to local Rekor mirror; verify during scan. |
|
||||
|
||||
### 5.6 CLI & API Interface
|
||||
|
||||
| Ref | Requirement |
|
||||
|-----|-------------|
|
||||
| **FR‑CLI‑1** | CLI `stella scan` SHALL accept `--sbom-type {trivy,spdx,cyclonedx,auto}`. |
|
||||
| **FR‑API‑1** | API `/scan` SHALL accept `sbomType` query/body field (optional). |
|
||||
| **FR‑API‑2** | API `/layers/missing` SHALL accept JSON array of digests and return JSON array of missing digests. |
|
||||
|
||||
---
|
||||
|
||||
## 6 · Non‑Functional Requirements (NFR)
|
||||
|
||||
| Ref | Category | Requirement |
|
||||
|-----|----------|-------------|
|
||||
| **NFR‑PERF‑1** | Performance | P95 cold scan ≤ 5 s; warm ≤ 1 s (see **FR‑DELTA‑3**). |
|
||||
| **NFR‑PERF‑2** | Throughput | System shall sustain 60 concurrent scans on 8‑core node without queue depth >10. |
|
||||
| **NFR‑AVAIL‑1** | Availability | All services shall start offline; any Internet call must be optional. |
|
||||
| **NFR‑SCAL‑1** | Scalability | Horizontal scaling via Kubernetes replicas for backend, Redis Sentinel, Mongo replica set. |
|
||||
| **NFR‑SEC‑1** | Security | All inter‑service traffic shall use TLS or localhost sockets. |
|
||||
| **NFR‑COMP‑1** | Compatibility | Platform shall run on x86‑64 Linux kernel ≥ 5.10; Windows agents (TODO > 6 mo) must support Server 2019+. |
|
||||
| **NFR‑I18N‑1** | Internationalisation | UI must support RU and EN locale toggles. |
|
||||
| **NFR‑OBS‑1** | Observability | Export Prometheus metrics for scan duration, queue length, policy eval duration. |
|
||||
|
||||
---
|
||||
|
||||
## 7 Acceptance Criteria <a id="7-acceptance-criteria"></a>
|
||||
|
||||
1. Issue 333 `/scan` calls; 334th returns **HTTP 429** and `Retry‑After`.
|
||||
2. Redis failure during test → API returns **0 remaining** & warns in logs.
|
||||
3. UI banner activates at 133 remaining; clears next UTC midnight.
|
||||
|
||||
---
|
||||
## 8 · System Interfaces
|
||||
|
||||
### 8.1 External APIs
|
||||
|
||||
*(This is the complete original table, plus new `/quota` row.)*
|
||||
|
||||
| Path | Method | Auth | Quota | Description |
|
||||
|------|--------|------|-------|-------------|
|
||||
| `/scan` | POST | Bearer | ✅ | Submit SBOM or `imageRef` for scanning. |
|
||||
| `/quota` | GET | Bearer | ❌ | Return remaining quota for current token. |
|
||||
| `/policy/rules` | GET/PUT | Bearer+RBAC | ❌ | CRUD YAML or Rego policies. |
|
||||
| `/plugins` | POST/GET | Bearer+Admin | ❌ | Upload or list plug‑ins. |
|
||||
|
||||
```bash
|
||||
GET /quota
|
||||
Authorization: Bearer <token>
|
||||
|
||||
200 OK
|
||||
{
|
||||
"limit": 333,
|
||||
"remaining": 121,
|
||||
"resetsAt": "2025-07-14T23:59:59Z"
|
||||
}
|
||||
```
|
||||
|
||||
## 9 · Assumptions & Constraints
|
||||
|
||||
* Hardware reference: 8 vCPU, 8 GB RAM, NVMe SSD.
|
||||
* Mongo DB and Redis run co‑located unless horizontal scaling enabled.
|
||||
* All docker images tagged `latest` are immutable (CI process locks digests).
|
||||
* Rego evaluation runs in embedded OPA Go‑library (no external binary).
|
||||
|
||||
---
|
||||
|
||||
## 10 · Future Work (Beyond 12 Months)
|
||||
|
||||
* Rekor transparency log cross‑cluster replication.
|
||||
* AI‑assisted false‑positive triage plug‑in.
|
||||
* Cluster‑wide injection for live runtime scanning.
|
||||
|
||||
---
|
||||
|
||||
## 11 · Revision History
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---------|------|-------|
|
||||
| **v1.2** | 11‑Jul‑2025 | Commercial references removed; plug‑in contract (§ 3.3) and new NFR categories added; added User Classes & Traceability. |
|
||||
| v1.1 | 11‑Jul‑2025 | Split out RU‑specific items; OSS scope |
|
||||
| v1.0 | 09‑Jul‑2025 | Original unified SRS |
|
||||
|
||||
*(End of System Requirements Specification v1.2‑core)*
|
||||
413
docs/07_HIGH_LEVEL_ARCHITECTURE.md
Normal file
413
docs/07_HIGH_LEVEL_ARCHITECTURE.md
Normal file
@ -0,0 +1,413 @@
|
||||
# 7 · High‑Level Architecture — **Stella Ops**
|
||||
|
||||
---
|
||||
|
||||
## 0 Purpose & Scope
|
||||
|
||||
Give contributors, DevOps engineers and auditors a **complete yet readable map** of the Core:
|
||||
|
||||
* Major runtime components and message paths.
|
||||
* Where plug‑ins, CLI helpers and runtime agents attach.
|
||||
* Technology choices that enable the sub‑5 second SBOM goal.
|
||||
* Typical operational scenarios (pipeline scan, mute, nightly re‑scan, etc.).
|
||||
|
||||
Anything enterprise‑only (signed PDF, Crypto‑specific TLS, LDAP, enforcement) **must arrive as a plug‑in**; the Core never hard‑codes those concerns.
|
||||
|
||||
---
|
||||
## 1 Component Overview
|
||||
|
||||
| # | Component | Responsibility |
|
||||
|---|-----------|---------------|
|
||||
| 1 | **API Gateway** | REST endpoints (`/scan`, `/quota`, **`/token/offline`**); token auth; quota enforcement |
|
||||
| 2 | **Scan Service** | SBOM parsing, Delta‑SBOM cache, vulnerability lookup |
|
||||
| 3 | **Policy Engine** | YAML / (optional) Rego rule evaluation; verdict assembly |
|
||||
| 4 | **Quota Service** | Per‑token counters; **333 scans/day**; waits & HTTP 429 |
|
||||
| 5 | **Client‑JWT Issuer** | Issues 30‑day offline tokens; bundles them into OUK |
|
||||
| 5 | **Registry** | Anonymous internal Docker registry for agents, SBOM uploads |
|
||||
| 6 | **Web UI** | React/Blazor SPA; dashboards, policy editor, quota banner |
|
||||
| 7 | **Data Stores** | **Redis** (cache, quota) & **MongoDB** (SBOMs, findings, audit) |
|
||||
| 8 | **Plugin Host** | Hot‑load .NET DLLs; isolates community plug‑ins |
|
||||
| 9 | **Agents** | `sbom‑builder`, `SanTech` scanner CLI, future `StellaOpsAttestor` |
|
||||
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph "External Actors"
|
||||
DEV["Developer / DevSecOps / Manager"]
|
||||
CI["CI/CD Pipeline (e.g., SanTech CLI)"]
|
||||
K8S["Kubernetes Cluster (e.g., Zastava Agent)"]
|
||||
end
|
||||
|
||||
subgraph "Stella Ops Runtime"
|
||||
subgraph "Core Services"
|
||||
CORE["Stella Core<br>(REST + gRPC APIs, Orchestration)"]
|
||||
REDIS[("Redis<br>(Cache, Queues, Trivy DB Mirror)")]
|
||||
MONGO[("MongoDB<br>(Optional: Long-term Storage)")]
|
||||
POL["Mute Policies<br>(OPA & YAML Evaluator)"]
|
||||
REG["StellaOps Registry<br>(Docker Registry v2)"]
|
||||
ATT["StellaOps Attestor<br>(SLSA + Rekor)"]
|
||||
end
|
||||
|
||||
subgraph "Agents & Builders"
|
||||
SB["SBOM Builder<br>(Go Binary: Extracts Layers, Generates SBOMs)"]
|
||||
SA["SanTech Agent<br>(Pipeline Helper: Invokes Builder, Triggers Scans)"]
|
||||
ZA["Zastava Agent<br>(K8s Webhook: Enforces Policies, Inventories Containers)"]
|
||||
end
|
||||
|
||||
subgraph "Scanners & UI"
|
||||
TRIVY["Trivy Scanner<br>(Plugin Container: Vulnerability Scanning)"]
|
||||
UI["Web UI<br>(Vue3 + Tailwind: Dashboards, Policy Editor)"]
|
||||
CLI["Stella CLI<br>(CLI Helper: Triggers Scans, Mutes)"]
|
||||
end
|
||||
end
|
||||
|
||||
DEV -->|Browses Findings, Mutes CVEs| UI
|
||||
DEV -->|Triggers Scans| CLI
|
||||
CI -->|Generates SBOM, Calls /scan| SA
|
||||
K8S -->|Inventories Containers, Enforces Gates| ZA
|
||||
|
||||
UI -- "REST" --> CORE
|
||||
CLI -- "REST/gRPC" --> CORE
|
||||
SA -->|Scan Requests| CORE
|
||||
SB -->|Uploads SBOMs| CORE
|
||||
ZA -->|Policy Gates| CORE
|
||||
|
||||
CORE -- "Queues, Caches" --> REDIS
|
||||
CORE -- "Persists Data" --> MONGO
|
||||
CORE -->|Evaluates Policies| POL
|
||||
CORE -->|Attests Provenance| ATT
|
||||
CORE -->|Scans Vulnerabilities| TRIVY
|
||||
|
||||
SB -- "Pulls Images" --> REG
|
||||
SA -- "Pulls Images" --> REG
|
||||
ZA -- "Pulls Images" --> REG
|
||||
|
||||
style DEV fill:#f9f,stroke:#333
|
||||
style CI fill:#f9f,stroke:#333
|
||||
style K8S fill:#f9f,stroke:#333
|
||||
style CORE fill:#ddf,stroke:#333
|
||||
style REDIS fill:#fdd,stroke:#333
|
||||
style MONGO fill:#fdd,stroke:#333
|
||||
style POL fill:#dfd,stroke:#333
|
||||
style REG fill:#dfd,stroke:#333
|
||||
style ATT fill:#dfd,stroke:#333
|
||||
style SB fill:#fdf,stroke:#333
|
||||
style SA fill:#fdf,stroke:#333
|
||||
style ZA fill:#fdf,stroke:#333
|
||||
style TRIVY fill:#ffd,stroke:#333
|
||||
style UI fill:#ffd,stroke:#333
|
||||
style CLI fill:#ffd,stroke:#333
|
||||
```
|
||||
|
||||
* **Developer / DevSecOps / Manager** – browses findings, mutes CVEs, triggers scans.
|
||||
* **Santech CLI** – generates SBOMs and calls `/scan` during CI.
|
||||
* **Zastava Agent** – inventories live containers; Core ships it in *passive* mode only (no kill).
|
||||
|
||||
### 1.1 Client‑JWT Lifecycle (offline aware)
|
||||
|
||||
1. **Online instance** – user signs in → `/connect/token` issues JWT valid 12 h.
|
||||
2. **Offline instance** – JWT with `exp ≈ 30 days` ships in OUK; backend
|
||||
**re‑signs** and stores it during import.
|
||||
3. Tokens embed a `tier` claim (“Free”) and `maxScansPerDay: 333`.
|
||||
4. On expiry the UI surfaces a red toast **7 days** in advance.
|
||||
|
||||
---
|
||||
|
||||
## 2 · Component Responsibilities (runtime view)
|
||||
|
||||
| Component | Core Responsibility | Implementation Highlights |
|
||||
| -------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
|
||||
| **Stella Core** | Orchestrates scans, persists SBOM blobs, serves REST/gRPC APIs, fans out jobs to scanners & policy engine. | .NET 8, CQRS, Redis Streams; pluggable runner interfaces. |
|
||||
| **SBOM Builder** | Extracts image layers, queries Core for *missing* layers, generates SBOMs (multi‑format), uploads blobs. | Go binary; wraps Trivy & Syft libs. |
|
||||
| **SanTech Agent** | Pipeline‑side helper; invokes Builder, triggers scan, streams progress back to CI/CD. | Static musl build. |
|
||||
| **Zastava Agent** | K8s admission webhook enforcing policy verdicts before Pod creation. | Rust for sub‑10 ms latencies. |
|
||||
| **UI** | Angular 17 SPA for dashboards, settings, policy editor. | Tailwind CSS; Webpack module federation (future). |
|
||||
| **Redis** | Cache, queue, Trivy‑DB mirror, layer diffing. | Single instance or Sentinel. |
|
||||
| **MongoDB** (opt.) | Long‑term SBOM & policy audit storage (> 180 days). | Optional; enabled via flag. |
|
||||
| **StellaOps.Registry** | Anonymous read‑only Docker v2 registry with optional Cosign verification. | `registry :2` behind nginx reverse proxy. |
|
||||
| **StellaOps.MutePolicies** | YAML/Rego evaluator, policy version store, `/policy/*` API. | Embeds OPA‑WASM; falls back to `opa exec`. |
|
||||
| **StellaOpsAttestor** | Generate SLSA provenance & Rekor signatures; verify on demand. | Side‑car container; DSSE + Rekor CLI. |
|
||||
|
||||
All cross‑component calls use dependency‑injected interfaces—no
|
||||
intra‑component reach‑ins.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Principal Backend Modules & Plug‑in Hooks
|
||||
|
||||
| Namespace | Responsibility | Built‑in Tech / Default | Plug‑in Contract |
|
||||
| --------------- | -------------------------------------------------- | ----------------------- | ------------------------------------------------- |
|
||||
| `configuration` | Parse env/JSON, health‑check endpoint | .NET 9 Options | `IConfigValidator` |
|
||||
| `identity` | Embedded OAuth2/OIDC (OpenIddict 6) | MIT OpenIddict | `IIdentityProvider` for LDAP/SAML/JWT gateway |
|
||||
| `pluginloader` | Discover DLLs, SemVer gate, optional Cosign verify | Reflection + Cosign | `IPluginLifecycleHook` for telemetry |
|
||||
| `scanning` | SBOM‑ & image‑flow orchestration; runner pool | Trivy CLI (default) | `IScannerRunner` – e.g., Grype, Copacetic, Clair |
|
||||
| `feedmerger` | Nightly NVD merge & feed enrichment | Hangfire job | drop‑in `*.Schedule.dll` for OSV, GHSA, BDU feeds |
|
||||
| `tls` | TLS provider abstraction | OpenSSL | `ITlsProvider` for GOST, SM‑series, custom suites |
|
||||
| `reporting` | Render HTML/PDF reports | RazorLight | `IReportRenderer` |
|
||||
| `ui` | Angular SPA & i18n | Angular 17 | new locales via `/locales/{lang}.json` |
|
||||
| `scheduling` | Cron + retries | Hangfire | any recurrent job via `*.Schedule.dll` |
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class configuration
|
||||
class identity
|
||||
class pluginloader
|
||||
class scanning
|
||||
class feedmerger
|
||||
class tls
|
||||
class reporting
|
||||
class ui
|
||||
class scheduling
|
||||
|
||||
class AllModules
|
||||
|
||||
configuration ..> identity : Uses
|
||||
identity ..> pluginloader : Authenticates Plugins
|
||||
pluginloader ..> scanning : Loads Scanner Runners
|
||||
scanning ..> feedmerger : Triggers Feed Merges
|
||||
tls ..> AllModules : Provides TLS Abstraction
|
||||
reporting ..> ui : Renders Reports for UI
|
||||
scheduling ..> feedmerger : Schedules Nightly Jobs
|
||||
|
||||
note for scanning "Pluggable: ISScannerRunner<br>e.g., Trivy, Grype"
|
||||
note for feedmerger "Pluggable: *.Schedule.dll<br>e.g., OSV, GHSA Feeds"
|
||||
note for identity "Pluggable: IIdentityProvider<br>e.g., LDAP, SAML"
|
||||
note for reporting "Pluggable: IReportRenderer<br>e.g., Custom PDF"
|
||||
```
|
||||
|
||||
**When remaining = 0:**
|
||||
API returns `429 Too Many Requests`, `Retry‑After: <UTC‑midnight>` (sequence omitted for brevity).
|
||||
|
||||
---
|
||||
|
||||
## 4 · Data Flows
|
||||
|
||||
### 4.1 SBOM‑First (≤ 5 s P95)
|
||||
|
||||
Builder produces SBOM locally, so Core never touches the Docker
|
||||
socket.
|
||||
Trivy path hits ≤ 5 s on alpine:3.19 with warmed DB.
|
||||
Image‑unpack fallback stays ≤ 10 s for 200 MB images.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant CI as CI/CD Pipeline (SanTech Agent)
|
||||
participant SB as SBOM Builder
|
||||
participant CORE as Stella Core
|
||||
participant REDIS as Redis Queue
|
||||
participant RUN as Scanner Runner (e.g., Trivy)
|
||||
participant POL as Policy Evaluator
|
||||
|
||||
CI->>SB: Invoke SBOM Generation
|
||||
SB->>CORE: Check Missing Layers (/layers/missing)
|
||||
CORE->>REDIS: Query Layer Diff (SDIFF)
|
||||
REDIS-->>CORE: Missing Layers List
|
||||
CORE-->>SB: Return Missing Layers
|
||||
SB->>SB: Generate Delta SBOM
|
||||
SB->>CORE: Upload SBOM Blob (POST /scan(sbom))
|
||||
CORE->>REDIS: Enqueue Scan Job
|
||||
REDIS->>RUN: Fan Out to Runner
|
||||
RUN->>RUN: Perform Vulnerability Scan
|
||||
RUN-->>CORE: Return Scan Results
|
||||
CORE->>POL: Evaluate Mute Policies
|
||||
POL-->>CORE: Policy Verdict
|
||||
CORE-->>CI: JSON Verdict & Progress Stream
|
||||
Note over CORE,CI: Achieves ≤5s P95 with Warmed DB
|
||||
```
|
||||
|
||||
### 4.2 Delta SBOM
|
||||
|
||||
Builder collects layer digests.
|
||||
`POST /layers/missing` → Redis SDIFF → missing layer list (< 20 ms).
|
||||
SBOM generated only for those layers and uploaded.
|
||||
|
||||
### 4.3 Feed Enrichment
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant CRON as Nightly Cron (Hangfire)
|
||||
participant FM as Feed Merger
|
||||
participant NVD as NVD Feed
|
||||
participant OSV as OSV Plugin (Optional)
|
||||
participant GHSA as GHSA Plugin (Optional)
|
||||
participant BDU as BDU Plugin (Optional)
|
||||
participant REDIS as Redis (Merged Feed Storage)
|
||||
participant UI as Web UI
|
||||
|
||||
CRON->>FM: Trigger at 00:59
|
||||
FM->>NVD: Fetch & Merge NVD Data
|
||||
alt Optional Plugins
|
||||
FM->>OSV: Merge OSV Feed
|
||||
FM->>GHSA: Merge GHSA Feed
|
||||
FM->>BDU: Merge BDU Feed
|
||||
end
|
||||
FM->>REDIS: Persist Merged Feed
|
||||
REDIS-->>UI: Update Feed Freshness
|
||||
UI->>UI: Display Green 'Feed Age' Tile
|
||||
```
|
||||
|
||||
### 4.4 Identity & Auth Flow
|
||||
|
||||
OpenIddict issues JWTs via client‑credentials or password grant.
|
||||
An IIdentityProvider plug‑in can delegate to LDAP, SAML or external OIDC
|
||||
without Core changes.
|
||||
---
|
||||
## 5 · Runtime Helpers
|
||||
|
||||
| Helper | Form | Purpose | Extensible Bits |
|
||||
|-----------|---------------------------------------|--------------------------------------------------------------------|-------------------------------------------|
|
||||
| **SanTech** | Distroless CLI | Generates SBOM, calls `/scan`, honours threshold flag | `--engine`, `--pdf-out` piped to plug‑ins |
|
||||
| **Zastava** | Static Go binary / DaemonSet | Watches Docker/CRI‑O events; uploads SBOMs; can enforce gate | Policy plug‑in could alter thresholds |
|
||||
|
||||
---
|
||||
|
||||
## 6 · Persistence & Cache Strategy
|
||||
|
||||
| Store | Primary Use | Why chosen |
|
||||
|----------------|-----------------------------------------------|--------------------------------|
|
||||
| **Redis 7** | Queue, SBOM cache, Trivy DB mirror | Sub‑1 ms P99 latency |
|
||||
| **MongoDB** | History > 180 d, audit logs, policy versions | Optional; document‑oriented |
|
||||
| **Local tmpfs**| Trivy layer cache (`/var/cache/trivy`) | Keeps disk I/O off hot path |
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph "Persistence Layers"
|
||||
REDIS[(Redis: Fast Cache/Queues<br>Sub-1ms P99)]
|
||||
MONGO[(MongoDB: Optional Audit/History<br>>180 Days)]
|
||||
TMPFS[(Local tmpfs: Trivy Layer Cache<br>Low I/O Overhead)]
|
||||
end
|
||||
|
||||
CORE["Stella Core"] -- Queues & SBOM Cache --> REDIS
|
||||
CORE -- Long-term Storage --> MONGO
|
||||
TRIVY["Trivy Scanner"] -- Layer Unpack Cache --> TMPFS
|
||||
|
||||
style REDIS fill:#fdd,stroke:#333
|
||||
style MONGO fill:#dfd,stroke:#333
|
||||
style TMPFS fill:#ffd,stroke:#333
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7 · Typical Scenarios
|
||||
|
||||
| # | Flow | Steps |
|
||||
|---------|----------------------------|-------------------------------------------------------------------------------------------------|
|
||||
| **S‑1** | Pipeline Scan & Alert | SanTech → SBOM → `/scan` → policy verdict → CI exit code & link to *Scan Detail* |
|
||||
| **S‑2** | Mute Noisy CVE | Dev toggles **Mute** in UI → rule stored in Redis → next build passes |
|
||||
| **S‑3** | Nightly Re‑scan | `SbomNightly.Schedule` re‑queues SBOMs (mask‑filter) → dashboard highlights new Criticals |
|
||||
| **S‑4** | Feed Update Cycle | `FeedMerger` merges feeds → UI *Feed Age* tile turns green |
|
||||
| **S‑5** | Custom Report Generation | Plug‑in registers `IReportRenderer` → `/report/custom/{digest}` → CI downloads artifact |
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant DEV as Developer
|
||||
participant UI as Web UI
|
||||
participant CORE as Stella Core
|
||||
participant REDIS as Redis
|
||||
participant RUN as Scanner Runner
|
||||
|
||||
DEV->>UI: Toggle Mute for CVE
|
||||
UI->>CORE: Update Mute Rule (POST /policy/mute)
|
||||
CORE->>REDIS: Store Mute Policy
|
||||
Note over CORE,REDIS: YAML/Rego Evaluator Updates
|
||||
|
||||
alt Next Pipeline Build
|
||||
CI->>CORE: Trigger Scan (POST /scan)
|
||||
CORE->>RUN: Enqueue & Scan
|
||||
RUN-->>CORE: Raw Findings
|
||||
CORE->>REDIS: Apply Mute Policies
|
||||
REDIS-->>CORE: Filtered Verdict (Passes)
|
||||
CORE-->>CI: Success Exit Code
|
||||
end
|
||||
```
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant CRON as SbomNightly.Schedule
|
||||
participant CORE as Stella Core
|
||||
participant REDIS as Redis Queue
|
||||
participant RUN as Scanner Runner
|
||||
participant UI as Dashboard
|
||||
|
||||
CRON->>CORE: Re-queue SBOMs (Mask-Filter)
|
||||
CORE->>REDIS: Enqueue Filtered Jobs
|
||||
REDIS->>RUN: Fan Out to Runners
|
||||
RUN-->>CORE: New Scan Results
|
||||
CORE->>UI: Highlight New Criticals
|
||||
Note over CORE,UI: Focus on Changes Since Last Scan
|
||||
```
|
||||
---
|
||||
|
||||
## 8 · UI Fast Facts
|
||||
|
||||
* **Stack** – Angular 17 + Vite dev server; Tailwind CSS.
|
||||
* **State** – Signals + RxJS for live scan progress.
|
||||
* **i18n / l10n** – JSON bundles served from `/locales/{lang}.json`.
|
||||
* **Module Structure** – Lazy‑loaded feature modules (`dashboard`, `scans`, `settings`); runtime route injection by UI plug‑ins (road‑map Q2‑2026).
|
||||
|
||||
---
|
||||
|
||||
## 9 · Cross‑Cutting Concerns
|
||||
|
||||
* **Security** – containers run non‑root, `CAP_DROP:ALL`, read‑only FS, hardened seccomp profiles.
|
||||
* **Observability** – Serilog JSON, OpenTelemetry OTLP exporter, Prometheus `/metrics`.
|
||||
* **Upgrade Policy** – `/api/v1` endpoints & CLI flags stable across a minor; breaking changes bump major.
|
||||
|
||||
---
|
||||
|
||||
## 10 · Performance & Scalability
|
||||
|
||||
| Scenario | P95 target | Bottleneck | Mitigation |
|
||||
|-----------------|-----------:|-----------------|-------------------------------------------------|
|
||||
| SBOM‑first | ≤ 5 s | Redis queue | More CPU, increase `ScannerPool.Workers` |
|
||||
| Image‑unpack | ≤ 10 s | Layer unpack | Prefer SBOM path, warm Docker cache |
|
||||
| High concurrency| 40 rps | Runner CPU | Scale Core replicas + side‑car scanner services |
|
||||
|
||||
---
|
||||
|
||||
## 11 · Future Architectural Anchors
|
||||
|
||||
* **ScanService micro‑split (gRPC)** – isolate heavy runners for large clusters.
|
||||
* **UI route plug‑ins** – dynamic Angular module loader (road‑map Q2‑2026).
|
||||
* **Redis Cluster** – transparently sharded cache once sustained > 100 rps.
|
||||
|
||||
---
|
||||
|
||||
## 12 · Assumptions & Trade‑offs
|
||||
|
||||
Requires Docker/CRI‑O runtime; .NET 9 available on hosts; Windows containers are out‑of‑scope this cycle.
|
||||
Embedded auth simplifies deployment but may need plug‑ins for enterprise IdPs.
|
||||
Speed is prioritised over exhaustive feature parity with heavyweight commercial scanners.
|
||||
|
||||
---
|
||||
|
||||
## 13 · References & Further Reading
|
||||
|
||||
* **C4 Model** – <https://c4model.com>
|
||||
* **.NET Architecture Guides** – <https://learn.microsoft.com/dotnet/architecture>
|
||||
* **OSS Examples** – Kubernetes Architecture docs, Prometheus design papers, Backstage.
|
||||
|
||||
---
|
||||
|
||||
## 14 · Change Log
|
||||
*(Git history is authoritative; table for reader convenience.)*
|
||||
|
||||
| Date | Note |
|
||||
|------------|--------------------------------------------------------------------------------------------------------|
|
||||
| 2025‑07‑13 | Added internal registry, multi‑format SBOM, delta flow, Policy as Code, Attestor integration sections. |
|
||||
| 2025‑07‑12 | Re‑organised doc around C4; added diagrams, trade‑offs, security notes. |
|
||||
| 2025‑07‑11 | Initial open‑sourcing pass – removed commercial references, added plug‑in hooks and UI details. |
|
||||
|
||||
---
|
||||
|
||||
## 14 Change Log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | ------------------------------------------------------------------------------------------------ |
|
||||
| v2.4 | 2025‑07‑12 | New modules |
|
||||
| v2.3 | 2025‑07‑12 | Adopted C4 structure, added diagrams/trade-offs/security notes/contribution hooks/references. |
|
||||
| v2.2 | 2025‑07‑11 | Removed last commercial refs; added TLS/IdP/report plug‑in hooks; deeper UI & scenario sections; |
|
||||
| v2.1 | 2025‑07‑11 | Added scenarios, UI details, plug‑in depth. |
|
||||
| v2.0 | 2025‑07‑11 | Full rewrite. |
|
||||
|
||||
*(End of High‑Level Architecture v2.2)*
|
||||
372
docs/08_MODULE_SPECIFICATIONS.md
Normal file
372
docs/08_MODULE_SPECIFICATIONS.md
Normal file
@ -0,0 +1,372 @@
|
||||
# 8 · Detailed Module Specifications — **Stella Ops**
|
||||
_This document defines every backend/agent module that composes Stella Ops,
|
||||
their public contracts, configuration keys and extension points._
|
||||
|
||||
---
|
||||
|
||||
## 0 Scope
|
||||
|
||||
Describes **every .NET, and Angular project** that ships in the OSS Core, the plug‑in contracts they expose, and the runtime artefacts (Dockerfiles, Compose files) used to build and operate them. Commercial capabilities appear *only* as extension points.
|
||||
|
||||
---
|
||||
|
||||
## 1 Repository Layout (flat)
|
||||
|
||||
~~~text
|
||||
src/
|
||||
│ docker-compose.yml
|
||||
└─ docker-compose-library/
|
||||
│ ├─ docker-compose.no-deps.yml
|
||||
│ ├─ docker-compose.dep.redis.yml
|
||||
│ ├─ docker-compose.dep.mongo.yml
|
||||
│ ├─ docker-compose.dep.proxy.yml
|
||||
│ ├─ docker-compose.dep.repository.yml
|
||||
│ └─ docker-compose.local.yml
|
||||
└─ backend/
|
||||
│ ├─ Dockerfile
|
||||
│ ├─ StellaOps.Web/
|
||||
│ ├─ StellaOps.Common/
|
||||
│ ├─ StellaOps.Plugins/
|
||||
│ ├─ StellaOps.Configuration/
|
||||
│ ├─ StellaOps.Localization/
|
||||
│ ├─ StellaOps.TlsProvider.OpenSSL/
|
||||
│ ├─ StellaOps.TlsProvider.OpenSSLGost/
|
||||
│ ├─ StellaOps.TlsProvider.Plugin.CryptoPro/
|
||||
│ ├─ StellaOps.VulnerabilityDatabase/
|
||||
│ ├─ StellaOps.Scheduling/
|
||||
│ ├─ StellaOps.Scheduling.SbomsRescan/
|
||||
│ ├─ StellaOps.Scheduling.MutesExpire/
|
||||
│ ├─ StellaOps.Scheduling.Plugin.CommonCveFeed/
|
||||
│ ├─ StellaOps.Scheduling.Plugin.RussianCveFeed/
|
||||
│ ├─ StellaOps.Scanners.Trivy/
|
||||
│ ├─ StellaOps.Quota/
|
||||
│ ├─ StellaOps.Reporting/
|
||||
│ ├─ StellaOps.Notifications/
|
||||
│ ├─ StellaOps.Notifications.Email/
|
||||
│ ├─ StellaOps.Notifications.Plugin.MsTeams/
|
||||
│ ├─ StellaOps.Authority/
|
||||
│ ├─ StellaOps.Authority.AD/
|
||||
│ ├─ StellaOps.Agent.Santech/
|
||||
│ └─ StellaOps.Agent.Zastava/
|
||||
└─ frontend/
|
||||
├─ Dockerfile
|
||||
├─ angular.json
|
||||
├─ stella-ops-ui/
|
||||
└─ libs/
|
||||
├─ dashboard/
|
||||
├─ scans/
|
||||
├─ settings/
|
||||
├─ core-ui/
|
||||
└─ i18n/
|
||||
~~~
|
||||
|
||||
All projects are referenced by **`StellaOps.sln`**; `dotnet publish -c Release -p:PublishSingleFile=true` builds a self‑contained **`StellaOps.Api`** binary (plug‑ins load at runtime).
|
||||
|
||||
---
|
||||
|
||||
## 2 Shared Libraries
|
||||
|
||||
| Project | Purpose | Key Interfaces |
|
||||
|---------|---------|----------------|
|
||||
| `StellaOps.Common` | Serilog sinks, Redis key helpers, DTO primitives. | `RedisKeys`, `Result<T>` |
|
||||
| `StellaOps.Plugins` | Plug‑in contracts + Cosign verification. | `IStellaPlugin`, `IScannerRunner`, `ITlsProvider`, `IScheduleJob` |
|
||||
| `StellaOps.Localization` | Loads JSON locale bundles (backend & Angular). | `ILocaleProvider`, `CultureMiddleware` |
|
||||
|
||||
Angular JSON‑bundle workflow matches the official i18n guide .
|
||||
|
||||
---
|
||||
|
||||
## 3 Core Back‑end Projects
|
||||
|
||||
| Project | Responsibility | Extensibility |
|
||||
|---------|----------------|---------------|
|
||||
| **`StellaOps.Api`** | ASP.NET host; source‑gen auto‑wires module endpoints. | Attributes `[MapRestController]`, `[MapHealth]`. |
|
||||
| **`StellaOps.Configuration`** | Bind `appsettings.json` → typed options; `/health`. | `IConfigValidator`. |
|
||||
| **`StellaOps.Quota`** | Enforces **Free‑tier quota** (333 scans/day) with early‑warning banner, 5 s soft back‑off, 60 s wait‑wall. | Swappable via `IQuotaStore` (e.g., Postgres). |
|
||||
| **`StellaOps.JwtIssuer` *(new)* | Issues, refreshes and validates **Client‑JWTs**. For offline sites it produces a 30‑day token during OUK build and again on every OUK import. | `ITokenSigner` (e.g., HSM) |
|
||||
| **`StellaOps.TlsProvider.OpenSSL`** | Default TLS suites. | New suites via `ITlsProvider` plug‑in. |
|
||||
| **`StellaOps.TlsProvider.OpenSSLGost`** | GOST suites. | — |
|
||||
| **`StellaOps.VulnerabilityDatabase`** | Feed‑merge CLI writing Redis. | `IAdditionalFeedSource` (OSV, GHSA, BDU). |
|
||||
| **`StellaOps.Scheduling`** | Hangfire host inside API . | Jobs via `IScheduleJob`. |
|
||||
| **`StellaOps.Scheduling.SbomsRescan`** | Nightly SBOM re‑scan (`0 2 * * *`). | — |
|
||||
| **`StellaOps.Scheduling.MutesExpire`** | Daily mute expiry cleanup. | — |
|
||||
| **`StellaOps.Scanners.Trivy`** | Trivy CLI for SBOM & image scans . | Other engines implement `IScannerRunner`. |
|
||||
| **`StellaOps.Reporting`** | RazorLight HTML reports. | `IReportRenderer` for SARIF, CycloneDX. |
|
||||
| **`StellaOps.Notifications`** | DI contracts for alerts. | `INotifier`. |
|
||||
| **`StellaOps.Notifications.Email`** | SMTP channel. | — |
|
||||
| **`StellaOps.Authority`** | OAuth2 / OIDC via OpenIddict 4 . | External IdPs via plug‑in. |
|
||||
| **`StellaOps.Registry`** | read‑only Docker registry for agents + SBOM‑builder | Registry v2 (nginx‑hardened) | `IRegistryProvider` |
|
||||
| **`StellaOps.MutePolicies`** | store YAML / Rego policies, validate & version | MongoDB + Redis | `IPolicyStore` |
|
||||
| **`StellaOps.Attestor`** *(TODO)*| SLSA provenance + Rekor verification | Sigstore Rekor | `IAttestor` |
|
||||
|
||||
## 3 · Module Details
|
||||
|
||||
> _Only contracts and configuration that may change in the next two quarters are shown; for stable, unchanging keys see the inline XML‑doc in the codebase._
|
||||
|
||||
### 3.1. StellaOps.Configuration
|
||||
|
||||
* **Responsibility** – parse environment variables or `appsettings.json`; expose `/health`, `/metrics`.
|
||||
* **Key extension point** – `IConfigValidator` → validate & normalise custom settings before DI builds.
|
||||
|
||||
### 3.2. StellaOps.Authority
|
||||
|
||||
* **Responsibility** – ship with OpenIddict 6, supporting *client‑credentials* and *password* grants.
|
||||
* `IIdentityProvider` plug‑in can delegate token issuance to LDAP, SAML, Keycloak …
|
||||
|
||||
|
||||
### 3.3. StellaOps.Scanners
|
||||
|
||||
* **Primary flow** – SBOM‑first; falls back to image‑unpack if SBOM absent.
|
||||
* **Multi‑Format Support** – side‑car `.sbom.type` file; auto‑detects (`SPDXID:` or `bomFormat` heuristics).
|
||||
* **Delta Layer Workflow** – `POST /layers/missing` (`SET DIFF` on Redis) responds < 20 ms; SanTech passes only new layers.
|
||||
* **Plug‑in contract evolution**
|
||||
|
||||
```csharp
|
||||
// current
|
||||
Task<ScanResult> RunAsync(Stream sbomJson, CancellationToken ct);
|
||||
|
||||
// v2 (preferred)
|
||||
Task<ScanResult> RunAsync(Stream sbom, SbomFormat fmt, CancellationToken ct);
|
||||
```
|
||||
|
||||
### 3.5 StellOps.Registry
|
||||
|
||||
* **Purpose** – internal, anonymous **read‑only** Docker registry to avoid GHCR / Docker Hub pulls.
|
||||
* **Deployment** – container `stellops.registry:2`; mounted volume `/var/lib/registry`; optional TLS via env vars.
|
||||
|
||||
| Key | Default | Notes |
|
||||
|----------------------------------|---------|---------------------------------|
|
||||
| `REGISTRY_READONLY` | `true` | Forces 403 on PUT, 405 on DELETE |
|
||||
| `REGISTRY_STORAGE_DELETE_ENABLED`| `false` | Immutable tags |
|
||||
|
||||
**Plug‑in contract** — `IRegistryProvider.PullAsync(string imageRef)` for mapping to Artifactory, Harbor, etc.
|
||||
|
||||
---
|
||||
|
||||
### 3.6 StellaOps.MutePolicies
|
||||
|
||||
* **Purpose** – central Policy‑as‑Code store (YAML v1 now, Rego soon).
|
||||
* **Persistence** – current live rules in Redis (`policies:active`); immutable commits in Mongo `policies_history`.
|
||||
|
||||
| REST verb | Path | Description |
|
||||
|-----------|---------------------|---------------------------|
|
||||
| `GET` | `/policy/export` | download active YAML |
|
||||
| `POST` | `/policy/import` | upload YAML / Rego file |
|
||||
| `POST` | `/policy/validate` | lint without persisting |
|
||||
|
||||
**CLI** – SanTech gains `--policy-file scan-policy.yaml`.
|
||||
|
||||
**Plug‑in contract** — `IPolicyStore` for GitOps back‑ends, Vault, etc.
|
||||
|
||||
---
|
||||
|
||||
### 3.7. StellaOps.Attestor *(Planned – Q1‑2026)*
|
||||
|
||||
Handles SLSA provenance docs and Rekor log verification.
|
||||
|
||||
```csharp
|
||||
public interface IAttestor {
|
||||
Task<ProvenanceDoc> CreateAsync(ImageRef img, Sbom sbom);
|
||||
Task<bool> VerifyAsync(ProvenanceDoc doc);
|
||||
}
|
||||
```
|
||||
|
||||
### 3.7. StellaOps.FeedMerger
|
||||
|
||||
Nightly Hangfire job (01:00) merges NVD JSON; plug‑ins can provide ISourceFeed for OSV, GHSA, BDU feeds.
|
||||
|
||||
### 3.8. StellOps.Tls
|
||||
|
||||
Abstracts TLS stack; default OpenSSL; ITlsProvider lets enterprises swap in GOST or SM cipher suites.
|
||||
|
||||
### 3.9. StellaOps.Reporting
|
||||
|
||||
HTML / PDF generation via RazorLight; custom renderers via IReportRenderer.
|
||||
|
||||
### 3.10 UI
|
||||
|
||||
Angular 17 SPA; lazy‑loaded feature modules, standalone component routes for UI plug‑ins.
|
||||
|
||||
Static Go daemon / k8s DaemonSet; watches Docker/CRI‑O events; uploads SBOMs; optional enforce mode via policy plug‑in.
|
||||
|
||||
### 3.11 StellaOps.Quota — **Free‑Tier Daily Quota Service**
|
||||
|
||||
**Responsibility**
|
||||
|
||||
* Track per‑token scan count (`quota:<token>` key in Redis).
|
||||
* Reset counters at **00:00 UTC** with key TTL.
|
||||
* Inject HTTP headers
|
||||
* `X‑Stella‑Quota‑Remaining`
|
||||
* `X‑Stella‑Reset`
|
||||
* Apply adaptive throttling:
|
||||
* scans 0‑199 → normal;
|
||||
* scans 200‑332 → UI banner flag `X‑Stella‑Quota‑Warn:true`;
|
||||
* scans ≥ 333 → respond **`429`** with escalating `Retry‑After` values
|
||||
(5 s for first 30 hits, then 60 s).
|
||||
* **Offline token awareness** — if `token.valid == false` and
|
||||
`OfflineMode == true`, return HTTP *451 ComplianceBlock* so that CLI gives a
|
||||
clear actionable error.
|
||||
* New config:
|
||||
|
||||
```json
|
||||
"Quota": {
|
||||
"OfflineGraceDays": 7 // show banner this many days before token expiry
|
||||
}
|
||||
```
|
||||
|
||||
**Interface**
|
||||
|
||||
```csharp
|
||||
public interface IQuotaService
|
||||
{
|
||||
/// <summary>Returns true when the call is allowed.</summary>
|
||||
Task<QuotaVerdict> CheckAsync(string token, CancellationToken ct);
|
||||
}
|
||||
|
||||
public readonly record struct QuotaVerdict(
|
||||
bool IsAllowed,
|
||||
int Remaining,
|
||||
DateTimeOffset ResetUtc,
|
||||
TimeSpan RetryAfter);
|
||||
```
|
||||
|
||||
**Configuration** (`appsettings.json` keys)
|
||||
|
||||
```json
|
||||
"Quota": {
|
||||
"FreeTierDailyLimit": 333,
|
||||
"WarnThreshold": 200,
|
||||
"SoftRetrySeconds": 5,
|
||||
"HardRetrySeconds": 60
|
||||
}
|
||||
```
|
||||
|
||||
**Extensibility**
|
||||
|
||||
* Override storage by providing an `IQuotaStore` plug‑in for Postgres or Mongo.
|
||||
* UI plug‑ins can subscribe to SSE `/quota/events` for custom dashboards.
|
||||
|
||||
### 3.12 StellaOps.JwtIssuer — new section
|
||||
|
||||
|API |Path| Notes|
|
||||
|-----|----|-------|
|
||||
|`POST /token/offline` | Admin‑only. | Generates a 30 d Client‑JWT for air‑gapped clusters; returns ZIP that the admin can copy to the target host.
|
||||
|
||||
*OUK hook*
|
||||
|
||||
* OUK builder calls JwtIssuer.SignOfflineToken(exp=+30d).
|
||||
* Drops client.jwt into ouk/root/.
|
||||
* Backend OUK importer places file under /var/lib/stella/tokens/.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Compose / Helm Snippet (reference)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
registry:
|
||||
image: stellops.registry:2
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
REGISTRY_READONLY: "true"
|
||||
volumes:
|
||||
- ./_registry:/var/lib/registry
|
||||
ports:
|
||||
- "5000:5000"
|
||||
|
||||
backend:
|
||||
image: registry.local/stellops/backend:${TAG}
|
||||
depends_on: [registry, redis]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4 Plug‑ins (sign‑required)
|
||||
|
||||
| Plug‑in | Contract | Notes |
|
||||
|---------|----------|-------|
|
||||
| `StellaOps.Notifications.Plugin.MsTeams` | `INotifier` | Sends cards to Teams webhooks. |
|
||||
| `StellaOps.Authority.AD` | `IIdentityProvider` | LDAP/Active‑Directory token issue. |
|
||||
| `StellaOps.Scheduling.Plugin.CommonCveFeed` | `IScheduleJob` | Merges OSV & NVD JSON hourly . |
|
||||
| `StellaOps.Scheduling.Plugin.RussianCveFeed` | `IScheduleJob` | Imports BDU XML daily. |
|
||||
| `StellaOps.TlsProvider.Plugin.CryptoPro` | `ITlsProvider` | Binds CryptoPro shared libs. |
|
||||
|
||||
Cosign signatures are mandatory; loader rejects unsigned DLLs when `DisableUnsigned=false`.
|
||||
|
||||
---
|
||||
|
||||
## 5 Agents
|
||||
|
||||
### 5.1 `StellaOps.Agent.Santech`
|
||||
|
||||
Distroless CLI;
|
||||
Returns exit‑code 1 on policy violation, enabling CI blocking.
|
||||
* **Role** – CI helper: Build SBOM, call `/scan`, exit non‑zero on high severity.
|
||||
* **Flags** – `--engine`, `--threshold`, `--registry-pull-token`, `--pdf-out`, `--delta`, `--sbom-type`, `--policy-file.`.
|
||||
* **Auth** – OAuth2 *scanner* scope.
|
||||
|
||||
### 5.2 `StellaOps.Agent.Zastava`
|
||||
|
||||
* **Role** – Passive container inventory → uploads SBOMs via `/agent/sbom`.
|
||||
* **Modes** – `off`, `inventory` (Core default).
|
||||
* No kernel driver (unlike Falco) .
|
||||
|
||||
---
|
||||
|
||||
## 6 Angular Front‑end
|
||||
|
||||
| Package | Path | Feature | Lazy |
|
||||
|---------|------|---------|------|
|
||||
| **App** | `frontend/stella-ops-ui/` | Shell, auth guards. | — |
|
||||
| `dashboard` | `libs/dashboard/` | Live metrics tiles. | ✔ |
|
||||
| `scans` | `libs/scans/` | List, detail, mute, diff. | ✔ |
|
||||
| `settings` | `libs/settings/` | Feed cron, workers, TLS switch. | ✔ |
|
||||
| `core-ui` | `libs/core-ui/` | Tailwind components. | — |
|
||||
| `i18n` | `libs/i18n/` | Runtime locale switch, pipe. | — |
|
||||
|
||||
Lazy loading of workspace libs follows Nx/Angular guidance .
|
||||
|
||||
---
|
||||
|
||||
## 7 Docker Artefacts
|
||||
|
||||
### 7.1 Dockerfiles
|
||||
|
||||
* **`backend/Dockerfile`** – multi‑stage .NET 9; single‑file publish; distroless runtime .
|
||||
* **`frontend/Dockerfile`** – Node 20 build → Nginx static serve.
|
||||
* Every plug‑in repo may include its own Dockerfile when shipping side‑cars (e.g., custom scanner).
|
||||
|
||||
### 7.2 Compose Stacks
|
||||
|
||||
* **`docker-compose.yml`**
|
||||
* Extends above with Redis 7 and Mongo 7 for small on‑prem installs.
|
||||
|
||||
* **`docker-compose.no-deps.yml`**
|
||||
* backend, frontend, Trivy, Maven proxy.
|
||||
* Assumes external Redis & Mongo.
|
||||
|
||||
* **`docker-compose.local.yml`**
|
||||
* Build images from local source and bring up backend, frontend, Redis, Mongo, Trivy, Maven proxy for dev‑loop.
|
||||
|
||||
Docker Compose override precedence matches official docs .
|
||||
|
||||
---
|
||||
|
||||
## 8 Performance Budget
|
||||
|
||||
| Flow | P95 target | Bottleneck |
|
||||
|------|-----------:|-----------|
|
||||
| SBOM fast‑path | ≤ 5 s | Redis queue depth (keep P99 < 1 ms) |
|
||||
| Image‑unpack | ≤ 10 s | Trivy layer unpack. |
|
||||
| Nightly re‑scan | 80 SBOM/s | Runner CPU. |
|
||||
|
||||
---
|
||||
|
||||
## Change Log
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---------|------|-------|
|
||||
| **v2.2** | 2025‑07‑11 | Flat layout; stella‑ops‑ui naming; Dockerfiles & 3 Compose stacks; agents and localisation library. |
|
||||
| v2.1 | 2025‑07‑11 | First flat‑structure draft. |
|
||||
|
||||
*(End of Module Specifications v2.2‑core)*
|
||||
329
docs/09_API_CLI_REFERENCE.md
Normal file
329
docs/09_API_CLI_REFERENCE.md
Normal file
@ -0,0 +1,329 @@
|
||||
# API & CLI Reference
|
||||
|
||||
*Purpose* – give operators and integrators a single, authoritative spec for REST/GRPC calls **and** first‑party CLI tools (`santech`, `zastava`, `stellopsctl`).
|
||||
Everything here is *source‑of‑truth* for generated Swagger/OpenAPI and the `--help` screens in the CLIs.
|
||||
|
||||
---
|
||||
|
||||
## 0 Quick Glance
|
||||
|
||||
| Area | Call / Flag | Notes |
|
||||
| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ |
|
||||
| Scan entry | `POST /scan` | Accepts SBOM or image; sub‑5 s target |
|
||||
| Delta check | `POST /layers/missing` | <20 ms reply; powers *delta SBOM* feature |
|
||||
| Rate‑limit / quota | — | Headers **`X‑Stella‑Quota‑Remaining`**, **`X‑Stella‑Reset`** on every response |
|
||||
| Policy I/O | `GET /policy/export`, `POST /policy/import` | YAML now; Rego coming |
|
||||
| Policy lint | `POST /policy/validate` | Returns 200 OK if ruleset passes |
|
||||
| Auth | `POST /connect/token` (OpenIddict) | Client‑credentials preferred |
|
||||
| Health | `GET /healthz` | Simple liveness probe |
|
||||
| Attestation * | `POST /attest` (TODO Q1‑2026) | SLSA provenance + Rekor log |
|
||||
| CLI flags | `--sbom-type` `--delta` `--policy-file` | Added to `santech` |
|
||||
|
||||
\* Marked **TODO** → delivered after sixth month (kept on Feature Matrix “To Do” list).
|
||||
|
||||
---
|
||||
|
||||
## 1 Authentication
|
||||
|
||||
Stella Ops uses **OAuth 2.0 / OIDC** (token endpoint mounted via OpenIddict).
|
||||
|
||||
```
|
||||
POST /connect/token
|
||||
Content‑Type: application/x-www-form-urlencoded
|
||||
|
||||
grant_type=client_credentials&
|
||||
client_id=ci‑bot&
|
||||
client_secret=REDACTED&
|
||||
scope=stella.api
|
||||
```
|
||||
|
||||
Successful response:
|
||||
|
||||
```json
|
||||
{
|
||||
"access_token": "eyJraWQi...",
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600
|
||||
}
|
||||
```
|
||||
|
||||
> **Tip** – pass the token via `Authorization: Bearer <token>` on every call.
|
||||
|
||||
---
|
||||
|
||||
## 2 REST API
|
||||
|
||||
### 2.0 Obtain / Refresh Offline‑Token
|
||||
|
||||
```text
|
||||
POST /token/offline
|
||||
Authorization: Bearer <admin‑token>
|
||||
```
|
||||
|
||||
| Body field | Required | Example | Notes |
|
||||
|------------|----------|---------|-------|
|
||||
| `expiresDays` | no | `30` | Max 90 days |
|
||||
|
||||
```json
|
||||
{
|
||||
"jwt": "eyJhbGciOiJSUzI1NiIsInR5cCI6...",
|
||||
"expires": "2025‑08‑17T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
Token is signed with the backend’s private key and already contains
|
||||
`"maxScansPerDay": 333`.
|
||||
|
||||
|
||||
### 2.1 Scan – Upload SBOM **or** Image
|
||||
|
||||
```
|
||||
POST /scan
|
||||
```
|
||||
|
||||
| Param / Header | In | Required | Description |
|
||||
| -------------------- | ------ | -------- | --------------------------------------------------------------------- |
|
||||
| `X‑Stella‑Sbom‑Type` | header | no | `trivy-json-v2`, `spdx-json`, `cyclonedx-json`; omitted ➞ auto‑detect |
|
||||
| `?threshold` | query | no | `low`, `medium`, `high`, `critical`; default **critical** |
|
||||
| body | body | yes | *Either* SBOM JSON *or* Docker image tarball/upload URL |
|
||||
|
||||
Every successful `/scan` response now includes:
|
||||
|
||||
| Header | Example |
|
||||
|--------|---------|
|
||||
| `X‑Stella‑Quota‑Remaining` | `129` |
|
||||
| `X‑Stella‑Reset` | `2025‑07‑18T23:59:59Z` |
|
||||
| `X‑Stella‑Token‑Expires` | `2025‑08‑17T00:00:00Z` |
|
||||
|
||||
**Response 200** (scan completed):
|
||||
|
||||
```json
|
||||
{
|
||||
"digest": "sha256:…",
|
||||
"summary": {
|
||||
"Critical": 0,
|
||||
"High": 3,
|
||||
"Medium": 12,
|
||||
"Low": 41
|
||||
},
|
||||
"policyStatus": "pass",
|
||||
"quota": {
|
||||
"remaining": 131,
|
||||
"reset": "2025-07-18T00:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response 202** – queued; polling URL in `Location` header.
|
||||
|
||||
---
|
||||
|
||||
### 2.2 Delta SBOM – Layer Cache Check
|
||||
|
||||
```
|
||||
POST /layers/missing
|
||||
Content‑Type: application/json
|
||||
Authorization: Bearer <token>
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"layers": [
|
||||
"sha256:d38b...",
|
||||
"sha256:af45..."
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response 200** — <20 ms target:
|
||||
|
||||
```json
|
||||
{
|
||||
"missing": [
|
||||
"sha256:af45..."
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Client then generates SBOM **only** for the `missing` layers and re‑posts `/scan`.
|
||||
|
||||
---
|
||||
|
||||
### 2.3 Policy Endpoints
|
||||
|
||||
| Method | Path | Purpose |
|
||||
| ------ | ------------------ | ------------------------------------ |
|
||||
| `GET` | `/policy/export` | Download live YAML ruleset |
|
||||
| `POST` | `/policy/import` | Upload YAML or Rego; replaces active |
|
||||
| `POST` | `/policy/validate` | Lint only; returns 400 on error |
|
||||
| `GET` | `/policy/history` | Paginated change log (audit trail) |
|
||||
|
||||
```yaml
|
||||
# Example import payload (YAML)
|
||||
version: "1.0"
|
||||
rules:
|
||||
- name: Ignore Low dev
|
||||
severity: [Low, None]
|
||||
environments: [dev, staging]
|
||||
action: ignore
|
||||
```
|
||||
|
||||
Validation errors come back as:
|
||||
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"path": "$.rules[0].severity",
|
||||
"msg": "Invalid level 'None'"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2.4 Attestation (Planned – Q1‑2026)
|
||||
|
||||
```
|
||||
POST /attest
|
||||
```
|
||||
|
||||
| Param | Purpose |
|
||||
| ----------- | ------------------------------------- |
|
||||
| body (JSON) | SLSA v1.0 provenance doc |
|
||||
| | Signed + stored in local Rekor mirror |
|
||||
|
||||
Returns `202 Accepted` and `Location: /attest/{id}` for async verify.
|
||||
|
||||
---
|
||||
|
||||
### 2.5 Misc Endpoints
|
||||
|
||||
| Path | Method | Description |
|
||||
| ---------- | ------ | ---------------------------- |
|
||||
| `/healthz` | GET | Liveness; returns `"ok"` |
|
||||
| `/metrics` | GET | Prometheus exposition (OTel) |
|
||||
| `/version` | GET | Git SHA + build date |
|
||||
|
||||
---
|
||||
|
||||
## 3 First‑Party CLI Tools
|
||||
|
||||
### 3.1 `stella‑santech`
|
||||
|
||||
> *Package SBOM + Scan + Exit code* – designed for CI.
|
||||
|
||||
```
|
||||
Usage: santech [OPTIONS] IMAGE_OR_SBOM
|
||||
```
|
||||
|
||||
| Flag / Option | Default | Description |
|
||||
| --------------- | ----------------------- | -------------------------------------------------- |
|
||||
| `--server` | `http://localhost:8080` | API root |
|
||||
| `--token` | *env `STELLA_TOKEN`* | Bearer token |
|
||||
| `--sbom-type` | *auto* | Force `trivy-json-v2`/`spdx-json`/`cyclonedx-json` |
|
||||
| `--delta` | `false` | Enable delta layer optimisation |
|
||||
| `--policy-file` | *none* | Override server rules with local YAML/Rego |
|
||||
| `--threshold` | `critical` | Fail build if ≥ level found |
|
||||
| `--output-json` | *none* | Write raw scan result to file |
|
||||
| `--wait-quota` | `true` | If 429 received, automatically wait `Retry‑After` and retry once. |
|
||||
|
||||
**Exit codes**
|
||||
|
||||
| Code | Meaning |
|
||||
| ---- | ------------------------------------------- |
|
||||
| 0 | Scan OK, policy passed |
|
||||
| 1 | Vulnerabilities ≥ threshold OR policy block |
|
||||
| 2 | Internal error (network etc.) |
|
||||
|
||||
---
|
||||
|
||||
### 3.2 `stella‑zastava`
|
||||
|
||||
> *Daemon / K8s DaemonSet* – watch container runtime, push SBOMs.
|
||||
|
||||
Core flags (excerpt):
|
||||
|
||||
| Flag | Purpose |
|
||||
| ---------------- | ---------------------------------- |
|
||||
| `--mode` | `listen` (default) / `enforce` |
|
||||
| `--filter-image` | Regex; ignore infra/busybox images |
|
||||
| `--threads` | Worker pool size |
|
||||
|
||||
---
|
||||
|
||||
### 3.3 `stellopsctl`
|
||||
|
||||
> *Admin utility* – policy snapshots, feed status, user CRUD.
|
||||
|
||||
Examples:
|
||||
|
||||
```
|
||||
stellopsctl policy export > policies/backup-2025-07-14.yaml
|
||||
stellopsctl feed refresh # force OSV merge
|
||||
stellopsctl user add dev-team --role developer
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4 Error Model
|
||||
|
||||
Uniform problem‑details object (RFC 7807):
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "https://stella-ops.ru/probs/validation",
|
||||
"title": "Invalid request",
|
||||
"status": 400,
|
||||
"detail": "Layer digest malformed",
|
||||
"traceId": "00-7c39..."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5 Rate Limits
|
||||
|
||||
Default **40 requests / second / token**.
|
||||
429 responses include `Retry-After` seconds header.
|
||||
|
||||
---
|
||||
|
||||
## 6 FAQ & Tips
|
||||
|
||||
* **Skip SBOM generation in CI** – supply a *pre‑built* SBOM and add `?sbom-only=true` to `/scan` for <1 s path.
|
||||
* **Air‑gapped?** – point `--server` to `http://oukgw:8080` inside the Offline Update Kit.
|
||||
* **YAML vs Rego** – YAML simpler; Rego unlocks time‑based logic (see samples).
|
||||
* **Cosign verify plug‑ins** – enable `SCANNER_VERIFY_SIG=true` env to refuse unsigned plug‑ins.
|
||||
|
||||
---
|
||||
|
||||
## 7 Planned Changes (Beyond 6 Months)
|
||||
|
||||
These stay in *Feature Matrix → To Do* until design is frozen.
|
||||
|
||||
| Epic / Feature | API Impact Sketch |
|
||||
| ---------------------------- | ---------------------------------- |
|
||||
| **SLSA L1‑L3** attestation | `/attest` (see §2.4) |
|
||||
| Rekor transparency log | `/rekor/log/{id}` (GET) |
|
||||
| Plug‑in Marketplace metadata | `/plugins/market` (catalog) |
|
||||
| Horizontal scaling controls | `POST /cluster/node` (add/remove) |
|
||||
| Windows agent support | Update LSAPI to PDE, no API change |
|
||||
|
||||
---
|
||||
|
||||
## 8 References
|
||||
|
||||
* OpenAPI YAML → `/openapi/v1.yaml` (served by backend)
|
||||
* OAuth2 spec: <https://datatracker.ietf.org/doc/html/rfc6749>
|
||||
* SLSA spec: <https://slsa.dev/spec/v1.0>
|
||||
|
||||
---
|
||||
|
||||
## 9 Changelog (truncated)
|
||||
|
||||
* **2025‑07‑14** – added *delta SBOM*, policy import/export, CLI `--sbom-type`.
|
||||
* **2025‑07‑12** – initial public reference.
|
||||
|
||||
---
|
||||
194
docs/10_PLUGIN_SDK_GUIDE.md
Normal file
194
docs/10_PLUGIN_SDK_GUIDE.md
Normal file
@ -0,0 +1,194 @@
|
||||
# 10 · Plug‑in SDK Guide — **Stella Ops**
|
||||
*(v 1.5 — 11 Jul 2025 · template install, no reload, IoC)*
|
||||
|
||||
---
|
||||
|
||||
## 0 Audience & Scope
|
||||
Guidance for developers who extend Stella Ops with schedule jobs, scanner adapters, TLS providers, notification channels, etc. Everything here is OSS; commercial variants simply ship additional signed plug‑ins.
|
||||
|
||||
---
|
||||
|
||||
## 1 Prerequisites
|
||||
|
||||
| Tool | Min Version |
|
||||
| ----------------------- | ----------------------------------------------------------------- |
|
||||
| .NET SDK | 9.0.200 |
|
||||
| **StellaOps templates** | install once via `bash dotnet new install StellaOps.Templates::*` |
|
||||
| **Cosign** | 2.3 + — used to sign DLLs |
|
||||
| xUnit | 2.6 |
|
||||
| Docker CLI | only if your plug‑in shells out to containers |
|
||||
|
||||
---
|
||||
|
||||
## 2 Repository & Build Output
|
||||
|
||||
Every plug‑in is hosted in **`git.stella‑ops.org`**.
|
||||
At publish time it must copy its signed artefacts to:
|
||||
|
||||
~~~text
|
||||
src/backend/Stella.Ops.Plugin.Binaries/<MyPlugin>/
|
||||
├── MyPlugin.dll
|
||||
└── MyPlugin.dll.sig
|
||||
~~~
|
||||
|
||||
The back‑end scans this folder on start‑up, verifies the **Cosign** signature, confirms the `[StellaPluginVersion]` gate, then loads the DLL inside an **isolated AssemblyLoadContext** to avoid dependency clashes
|
||||
|
||||
---
|
||||
|
||||
## 3 Project Scaffold
|
||||
|
||||
Generate with the installed template:
|
||||
|
||||
~~~bash
|
||||
dotnet new stellaops-plugin-schedule \
|
||||
-n MyPlugin.Schedule \
|
||||
--output src
|
||||
~~~
|
||||
|
||||
Result:
|
||||
|
||||
~~~text
|
||||
src/
|
||||
├─ MyPlugin.Schedule/
|
||||
│ ├─ MyJob.cs
|
||||
│ └─ MyPlugin.Schedule.csproj
|
||||
└─ tests/
|
||||
└─ MyPlugin.Schedule.Tests/
|
||||
~~~
|
||||
|
||||
---
|
||||
|
||||
## 4 MSBuild Wiring
|
||||
|
||||
Add this to **`MyPlugin.Schedule.csproj`** so the signed DLL + `.sig` land in the canonical plug‑in folder:
|
||||
|
||||
~~~xml
|
||||
<PropertyGroup>
|
||||
<StellaPluginOut>$(SolutionDir)src/backend/Stella.Ops.Plugin.Binaries/$(MSBuildProjectName)</StellaPluginOut>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
|
||||
<ProjectReference Include="..\..\StellaOps.Common\StellaOps.Common.csproj"
|
||||
PrivateAssets="all" />
|
||||
</ItemGroup>
|
||||
|
||||
<Target Name="CopyStellaPlugin" AfterTargets="Publish">
|
||||
<MakeDir Directories="$(StellaPluginOut)" />
|
||||
<Copy SourceFiles="$(PublishDir)$(AssemblyName).dll;$(PublishDir)$(AssemblyName).dll.sig"
|
||||
DestinationFolder="$(StellaPluginOut)" />
|
||||
</Target>
|
||||
~~~
|
||||
|
||||
---
|
||||
|
||||
## 5 Dependency‑Injection Entry‑point
|
||||
|
||||
Back‑end auto‑discovers the static method below:
|
||||
|
||||
~~~csharp
|
||||
namespace StellaOps.DependencyInjection;
|
||||
|
||||
public static class IoCConfigurator
|
||||
{
|
||||
public static IServiceCollection Configure(this IServiceCollection services,
|
||||
IConfiguration cfg)
|
||||
{
|
||||
services.AddSingleton<IJob, MyJob>(); // schedule job
|
||||
services.Configure<MyPluginOptions>(cfg.GetSection("Plugins:MyPlugin"));
|
||||
return services;
|
||||
}
|
||||
}
|
||||
~~~
|
||||
|
||||
---
|
||||
|
||||
## 6 Schedule Plug‑ins
|
||||
|
||||
### 6.1 Minimal Job
|
||||
|
||||
~~~csharp
|
||||
using StellaOps.Scheduling; // contract
|
||||
|
||||
[StellaPluginVersion("2.0.0")]
|
||||
public sealed class MyJob : IJob
|
||||
{
|
||||
public async Task ExecuteAsync(CancellationToken ct)
|
||||
{
|
||||
Console.WriteLine("Hello from plug‑in!");
|
||||
await Task.Delay(500, ct);
|
||||
}
|
||||
}
|
||||
~~~
|
||||
|
||||
### 6.2 Cron Registration
|
||||
|
||||
```csharp
|
||||
services.AddCronJob<MyJob>("0 15 * * *"); // everyday
|
||||
```
|
||||
|
||||
15:00
|
||||
Cron syntax follows Hangfire rules
|
||||
|
||||
## 7 Scanner Adapters
|
||||
|
||||
Implement IScannerRunner.
|
||||
Register inside Configure:
|
||||
```csharp
|
||||
services.AddScanner<MyAltScanner>("alt"); // backend
|
||||
```
|
||||
|
||||
selects by --engine alt
|
||||
If the engine needs a side‑car container, include a Dockerfile in your repo and document resource expectations.
|
||||
## 8 Packaging & Signing
|
||||
|
||||
```bash
|
||||
dotnet publish -c Release -p:PublishSingleFile=true -o out
|
||||
cosign sign --key $COSIGN_KEY out/MyPlugin.Schedule.dll # sign binary only
|
||||
sha256sum out/MyPlugin.Schedule.dll > out/.sha256 # optional checksum
|
||||
zip MyPlugin.zip out/* README.md
|
||||
```
|
||||
|
||||
Unsigned DLLs are refused when StellaOps:Security:DisableUnsigned=false.
|
||||
|
||||
## 9 Deployment
|
||||
|
||||
```bash
|
||||
docker cp MyPlugin.zip <backend>:/opt/plugins/ && docker restart <backend>
|
||||
```
|
||||
|
||||
Check /health – "plugins":["MyPlugin.Schedule@2.0.0"].
|
||||
(Hot‑reload was removed to keep the core process simple and memory‑safe.)
|
||||
|
||||
## 10 Configuration Patterns
|
||||
|
||||
| Need | Pattern |
|
||||
| ------------ | --------------------------------------------------------- |
|
||||
| Settings | Plugins:MyPlugin:* in appsettings.json. |
|
||||
| Secrets | Redis secure:<plugin>:<key> (encrypted per TLS provider). |
|
||||
| Dynamic cron | Implement ICronConfigurable; UI exposes editor. |
|
||||
|
||||
## 11 Testing & CI
|
||||
|
||||
| Layer | Tool | Gate |
|
||||
| ----------- | -------------------------- | ------------------- |
|
||||
| Unit | xUnit + Moq | ≥ 50 % lines |
|
||||
| Integration | Testcontainers ‑ run in CI | Job completes < 5 s |
|
||||
| Style | dotnet | format 0 warnings |
|
||||
|
||||
Use the pre‑baked workflow in StellaOps.Templates as starting point.
|
||||
|
||||
## 12 Publishing to the Community Marketplace
|
||||
|
||||
Tag Git release plugin‑vX.Y.Z and attach the signed ZIP.
|
||||
Submit a PR to stellaops/community-plugins.json with metadata & git URL.
|
||||
On merge, the plug‑in shows up in the UI Marketplace.
|
||||
|
||||
## 13 Common Pitfalls
|
||||
|
||||
| Symptom | Root cause | Fix |
|
||||
| ------------------- | -------------------------- | ------------------------------------------- |
|
||||
| NotDetected | .sig missing | cosign sign … |
|
||||
| VersionGateMismatch | Backend 2.1 vs plug‑in 2.0 | Re‑compile / bump attribute |
|
||||
| FileLoadException | Duplicate | StellaOps.Common Ensure PrivateAssets="all" |
|
||||
| Redis | timeouts Large writes | Batch or use Mongo |
|
||||
196
docs/11_DATA_SCHEMAS.md
Normal file
196
docs/11_DATA_SCHEMAS.md
Normal file
@ -0,0 +1,196 @@
|
||||
# Data Schemas & Persistence Contracts
|
||||
|
||||
*Audience* – backend developers, plug‑in authors, DB admins.
|
||||
*Scope* – describes **Redis**, **MongoDB** (optional), and on‑disk blob shapes that power Stella Ops.
|
||||
|
||||
---
|
||||
|
||||
## 0 Document Conventions
|
||||
|
||||
* **CamelCase** for JSON.
|
||||
* All timestamps are **RFC 3339 / ISO 8601** with `Z` (UTC).
|
||||
* `⭑` = planned but *not* shipped yet (kept on Feature Matrix “To Do”).
|
||||
|
||||
---
|
||||
|
||||
## 1 SBOM Wrapper Envelope
|
||||
|
||||
Every SBOM blob (regardless of format) is stored on disk or in object storage with a *sidecar* JSON file that indexes it for the scanners.
|
||||
|
||||
#### 1.1 JSON Shape
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "sha256:417f…", // digest of the SBOM *file* itself
|
||||
"imageDigest": "sha256:e2b9…", // digest of the original container image
|
||||
"created": "2025-07-14T07:02:13Z",
|
||||
"format": "trivy-json-v2", // NEW enum: trivy-json-v2 | spdx-json | cyclonedx-json
|
||||
"layers": [
|
||||
"sha256:d38b…", // layer digests (ordered)
|
||||
"sha256:af45…"
|
||||
],
|
||||
"partial": false, // true => delta SBOM (only some layers)
|
||||
"provenanceId": "prov_0291" // ⭑ link to SLSA attestation (Q1‑2026)
|
||||
}
|
||||
```
|
||||
|
||||
*`format`* **NEW** – added to support **multiple SBOM formats**.
|
||||
*`partial`* **NEW** – true when generated via the **delta SBOM** flow (§1.3).
|
||||
|
||||
#### 1.2 File‑system Layout
|
||||
|
||||
```
|
||||
blobs/
|
||||
├─ 417f… # digest prefix
|
||||
│ ├─ sbom.json # payload (any format)
|
||||
│ └─ sbom.meta.json # wrapper (shape above)
|
||||
```
|
||||
|
||||
> **Note** – blob storage can point at S3, MinIO, or plain disk; driver plug‑ins adapt.
|
||||
|
||||
#### 1.3 Delta SBOM Extension
|
||||
|
||||
When `partial: true`, *only* the missing layers have been scanned.
|
||||
Merging logic inside `scanning` module stitches new data onto the cached full SBOM in Redis.
|
||||
|
||||
---
|
||||
|
||||
## 2 Redis Keyspace
|
||||
|
||||
| Key pattern | Type | TTL | Purpose |
|
||||
|-------------------------------------|---------|------|--------------------------------------------------|
|
||||
| `scan:<digest>` | string | ∞ | Last scan JSON result (as returned by `/scan`) |
|
||||
| `layers:<digest>` | set | 90d | Layers already possessing SBOMs (delta cache) |
|
||||
| `policy:active` | string | ∞ | YAML **or** Rego ruleset |
|
||||
| `quota:<token>` | string | *until next UTC midnight* | Per‑token scan counter for Free tier (333 scans). |
|
||||
| `policy:history` | list | ∞ | Change audit IDs (see Mongo) |
|
||||
| `feed:nvd:json` | string | 24h | Normalised feed snapshot |
|
||||
| `locator:<imageDigest>` | string | 30d | Maps image digest → sbomBlobId |
|
||||
| `metrics:…` | various | — | Prom / OTLP runtime metrics |
|
||||
|
||||
> **Delta SBOM** uses `layers:*` to skip work in <20 ms.
|
||||
> **Quota enforcement** increments `quota:<token>` atomically; when value ≥ 333 the API returns **429**.
|
||||
|
||||
---
|
||||
|
||||
## 3 MongoDB Collections (Optional)
|
||||
|
||||
Only enabled when `MONGO_URI` is supplied (for long‑term audit).
|
||||
|
||||
| Collection | Shape (summary) | Indexes |
|
||||
|--------------------|------------------------------------------------------------|-------------------------------------|
|
||||
| `sbom_history` | Wrapper JSON + `replaceTs` on overwrite | `{imageDigest}` `{created}` |
|
||||
| `policy_versions` | `{_id, yaml, rego, authorId, created}` | `{created}` |
|
||||
| `attestations` ⭑ | SLSA provenance doc + Rekor log pointer | `{imageDigest}` |
|
||||
| `audit_log` | Fully rendered RFC 5424 entries (UI & CLI actions) | `{userId}` `{ts}` |
|
||||
|
||||
Schema detail for **policy_versions**:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"_id": "6619e90b8c5e1f76",
|
||||
"yaml": "version: 1.0\nrules:\n - …",
|
||||
"rego": null, // filled when Rego uploaded
|
||||
"authorId": "u_1021",
|
||||
"created": "2025-07-14T08:15:04Z",
|
||||
"comment": "Imported via API"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4 Policy Schema (YAML v1.0)
|
||||
|
||||
Minimal viable grammar (subset of OSV‑SCHEMA ideas).
|
||||
|
||||
```yaml
|
||||
version: "1.0"
|
||||
rules:
|
||||
- name: Block Critical
|
||||
severity: [Critical]
|
||||
action: block
|
||||
- name: Ignore Low Dev
|
||||
severity: [Low, None]
|
||||
environments: [dev, staging]
|
||||
action: ignore
|
||||
expires: "2026-01-01"
|
||||
- name: Escalate BDU High
|
||||
sources: [BDU]
|
||||
severity: [High, Critical]
|
||||
action: escalate
|
||||
```
|
||||
|
||||
Validation is performed by `policy:mapping.yaml` JSON‑Schema embedded in backend.
|
||||
|
||||
### 4.1 Rego Variant (Advanced – TODO)
|
||||
|
||||
*Accepted but stored as‑is in `rego` field.*
|
||||
Evaluated via internal **OPA** side‑car once feature graduates from TODO list.
|
||||
|
||||
---
|
||||
|
||||
## 5 SLSA Attestation Schema ⭑
|
||||
|
||||
Planned for Q1‑2026 (kept here for early plug‑in authors).
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "prov_0291",
|
||||
"imageDigest": "sha256:e2b9…",
|
||||
"buildType": "https://slsa.dev/container/v1",
|
||||
"builder": {
|
||||
"id": "https://git.stella-ops.ru/ci/stella-runner@sha256:f7b7…"
|
||||
},
|
||||
"metadata": {
|
||||
"invocation": {
|
||||
"parameters": {"GIT_SHA": "f6a1…"},
|
||||
"buildStart": "2025-07-14T06:59:17Z",
|
||||
"buildEnd": "2025-07-14T07:01:22Z"
|
||||
},
|
||||
"completeness": {"parameters": true}
|
||||
},
|
||||
"materials": [
|
||||
{"uri": "git+https://git…", "digest": {"sha1": "f6a1…"}}
|
||||
],
|
||||
"rekorLogIndex": 99817 // entry in local Rekor mirror
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6 Validator Contracts
|
||||
|
||||
* For SBOM wrapper – `ISbomValidator` (DLL plug‑in) must return *typed* error list.
|
||||
* For YAML policies – JSON‑Schema at `/schemas/policy‑v1.json`.
|
||||
* For Rego – OPA `opa eval --fail-defined` under the hood.
|
||||
* For **Free‑tier quotas** – `IQuotaService` integration tests ensure `quota:<token>` resets at UTC midnight and produces correct `Retry‑After` headers.
|
||||
|
||||
---
|
||||
|
||||
## 7 Migration Notes
|
||||
|
||||
1. **Add `format` column** to existing SBOM wrappers; default to `trivy-json-v2`.
|
||||
2. **Populate `layers` & `partial`** via backfill script (ship with `stellopsctl migrate` wizard).
|
||||
3. Policy YAML previously stored in Redis → copy to Mongo if persistence enabled.
|
||||
4. Prepare `attestations` collection (empty) – safe to create in advance.
|
||||
|
||||
---
|
||||
|
||||
## 8 Open Questions / Future Work
|
||||
|
||||
* How to de‑duplicate *identical* Rego policies differing only in whitespace?
|
||||
* Embed *GOST 34.11‑2018* digests when users enable Russian crypto suite?
|
||||
* Should enterprise tiers share the same Redis quota keys or switch to JWT claim `tier != Free` bypass?
|
||||
* Evaluate sliding‑window quota instead of strict daily reset.
|
||||
* Consider rate‑limit for `/layers/missing` to avoid brute‑force enumeration.
|
||||
|
||||
---
|
||||
|
||||
## 9 Change Log
|
||||
|
||||
| Date | Note |
|
||||
|------------|--------------------------------------------------------------------------------|
|
||||
| 2025‑07‑14 | **Added:** `format`, `partial`, delta cache keys, YAML policy schema v1.0. |
|
||||
| 2025‑07‑12 | **Initial public draft** – SBOM wrapper, Redis keyspace, audit collections. |
|
||||
|
||||
---
|
||||
167
docs/12_PERFORMANCE_WORKBOOK.md
Normal file
167
docs/12_PERFORMANCE_WORKBOOK.md
Normal file
@ -0,0 +1,167 @@
|
||||
# 12 - Performance Workbook
|
||||
|
||||
*Purpose* – define **repeatable, data‑driven** benchmarks that guard Stella Ops’ core pledge:
|
||||
> *“P95 vulnerability feedback in ≤ 5 seconds.”*
|
||||
|
||||
---
|
||||
|
||||
## 0 Benchmark Scope
|
||||
|
||||
| Area | Included | Excluded |
|
||||
|------------------|----------------------------------|---------------------------|
|
||||
| SBOM‑first scan | Trivy engine w/ warmed DB | Full image unpack ≥ 300 MB |
|
||||
| Delta SBOM ⭑ | Missing‑layer lookup & merge | Multi‑arch images |
|
||||
| Policy eval ⭑ | YAML → JSON → rule match | Rego (until GA) |
|
||||
| Feed merge | NVD JSON 2023–2025 | GHSA GraphQL (plugin) |
|
||||
| Quota wait‑path | 5 s soft‑wait, 60 s hard‑wait behaviour | Paid tiers (unlimited) |
|
||||
| API latency | REST `/scan`, `/layers/missing` | UI SPA calls |
|
||||
|
||||
⭑ = new in July 2025.
|
||||
|
||||
---
|
||||
|
||||
## 1 Hardware Baseline (Reference Rig)
|
||||
|
||||
| Element | Spec |
|
||||
|-------------|------------------------------------|
|
||||
| CPU | 8 vCPU (Intel Ice‑Lake equiv.) |
|
||||
| Memory | 16 GiB |
|
||||
| Disk | NVMe SSD, 3 GB/s R/W |
|
||||
| Network | 1 Gbit virt. switch |
|
||||
| Container | Docker 25.0 + overlay2 |
|
||||
| OS | Ubuntu 22.04 LTS (kernel 6.8) |
|
||||
|
||||
*All P95 targets assume a **single‑node** deployment on this rig unless stated.*
|
||||
|
||||
---
|
||||
|
||||
## 2 Phase Targets & Gates
|
||||
|
||||
| Phase (ID) | Target P95 | Gate (CI) | Rationale |
|
||||
|-----------------------|-----------:|-----------|----------------------------------------|
|
||||
| **SBOM_FIRST** | ≤ 5 s | `hard` | Core UX promise. |
|
||||
| **IMAGE_UNPACK** | ≤ 10 s | `soft` | Fallback path for legacy flows. |
|
||||
| **DELTA_SBOM** ⭑ | ≤ 1 s | `hard` | Needed to stay sub‑5 s for big bases. |
|
||||
| **POLICY_EVAL** ⭑ | ≤ 50 ms | `hard` | Keeps gate latency invisible to users. |
|
||||
| **QUOTA_WAIT** ⭑ | *soft* ≤ 5 s<br>*hard* ≤ 60 s | `hard` | Ensures graceful Free‑tier throttling. |
|
||||
| **SCHED_RESCAN** | ≤ 30 s | `soft` | Nightly batch – not user‑facing. |
|
||||
| **FEED_MERGE** | ≤ 60 s | `soft` | Off‑peak cron @ 01:00. |
|
||||
| **API_P95** | ≤ 200 ms | `hard` | UI snappiness. |
|
||||
|
||||
*Gate* legend — `hard`: break CI if regression > 3 × target,
|
||||
`soft`: raise warning & issue ticket.
|
||||
|
||||
---
|
||||
|
||||
## 3 Test Harness
|
||||
|
||||
* **Runner** – `perf/run.sh`, accepts `--phase` and `--samples`.
|
||||
* **Metrics** – Prometheus + `jq` extracts; aggregated via `scripts/aggregate.ts`.
|
||||
* **CI** – GitLab CI job *benchmark* publishes JSON to `bench‑artifacts/`.
|
||||
* **Visualisation** – Grafana dashboard *Stella‑Perf* (provisioned JSON).
|
||||
|
||||
> **Note** – harness mounts `/var/cache/trivy` tmpfs to avoid disk noise.
|
||||
|
||||
---
|
||||
|
||||
## 4 Current Results (July 2025)
|
||||
|
||||
| Phase | Samples | Mean (s) | P95 (s) | Target OK? |
|
||||
|---------------|--------:|---------:|--------:|-----------:|
|
||||
| SBOM_FIRST | 100 | 3.7 | 4.9 | ✅ |
|
||||
| IMAGE_UNPACK | 50 | 6.4 | 9.2 | ✅ |
|
||||
| **DELTA_SBOM**| 100 | 0.46 | 0.83 | ✅ |
|
||||
| **POLICY_EVAL** | 1 000 | 0.021 | 0.041 | ✅ |
|
||||
| **QUOTA_WAIT** | 80 | 4.0* | 4.9* | ✅ |
|
||||
| SCHED_RESCAN | 10 | 18.3 | 24.9 | ✅ |
|
||||
| FEED_MERGE | 3 | 38.1 | 41.0 | ✅ |
|
||||
| API_P95 | 20 000 | 0.087 | 0.143 | ✅ |
|
||||
|
||||
*Data files:* `bench-artifacts/2025‑07‑14/phase‑stats.json`.
|
||||
|
||||
---
|
||||
|
||||
## 5 Δ‑SBOM Micro‑Benchmark Detail
|
||||
|
||||
### 5.1 Scenario
|
||||
|
||||
1. Base image `python:3.12-slim` already scanned (all layers cached).
|
||||
2. Application layer (`COPY . /app`) triggers new digest.
|
||||
3. Santech lists **7** layers, backend replies *6 hit*, *1 miss*.
|
||||
4. Builder scans **only 1 layer** (~9 MiB, 217 files) & uploads delta.
|
||||
|
||||
### 5.2 Key Timings
|
||||
|
||||
| Step | Time (ms) |
|
||||
|---------------------|----------:|
|
||||
| `/layers/missing` | 13 |
|
||||
| Trivy single layer | 655 |
|
||||
| Upload delta blob | 88 |
|
||||
| Backend merge + CVE | 74 |
|
||||
| **Total wall‑time** | **830 ms** |
|
||||
|
||||
---
|
||||
|
||||
## 6 Quota Wait‑Path Benchmark Detail
|
||||
|
||||
### 6.1 Scenario
|
||||
|
||||
1. Free‑tier token reaches **scan #200** – dashboard shows yellow banner.
|
||||
|
||||
### 6.2 Key Timings
|
||||
|
||||
| Step | Time (ms) |
|
||||
|------------------------------------|----------:|
|
||||
| `/quota/check` Redis LUA INCR | 0.8 |
|
||||
| Soft wait sleep (server) | 5 000 |
|
||||
| Hard wait sleep (server) | 60 000 |
|
||||
| End‑to‑end wall‑time (soft‑hit) | 5 003 |
|
||||
| End‑to‑end wall‑time (hard‑hit) | 60 004 |
|
||||
|
||||
---
|
||||
## 7 Policy Eval Bench
|
||||
|
||||
### 7.1 Setup
|
||||
|
||||
* Policy YAML: **28** rules, mix severity & package conditions.
|
||||
* Input: scan result JSON with **1 026** findings.
|
||||
* Evaluator: custom rules engine (Go structs → map look‑ups).
|
||||
|
||||
### 7.2 Latency Histogram
|
||||
|
||||
```
|
||||
0‑10 ms ▇▇▇▇▇▇▇▇▇▇ 38 %
|
||||
10‑20 ms ▇▇▇▇▇▇▇▇▇▇ 42 %
|
||||
20‑40 ms ▇▇▇▇▇▇ 17 %
|
||||
40‑50 ms ▇ 3 %
|
||||
```
|
||||
|
||||
P99 = 48 ms. Meets 50 ms gate.
|
||||
|
||||
---
|
||||
|
||||
## 8 Trend Snapshot
|
||||
|
||||
|
||||
|
||||
_Plot generated weekly by `scripts/update‑trend.py`; shows last 12 weeks P95 per phase._
|
||||
|
||||
---
|
||||
|
||||
## 9 Action Items
|
||||
|
||||
1. **Image Unpack** – Evaluate zstd for layer decompress; aim to shave 1 s.
|
||||
2. **Feed Merge** – Parallelise BDU XML parse (plugin) once stable.
|
||||
3. **Rego Support** – Prototype OPA side‑car; target ≤ 100 ms eval.
|
||||
4. **Concurrency** – Stress‑test 100 rps on 4‑node Redis cluster (Q4‑2025).
|
||||
|
||||
---
|
||||
|
||||
## 10 Change Log
|
||||
|
||||
| Date | Note |
|
||||
|------------|-------------------------------------------------------------------------|
|
||||
| 2025‑07‑14 | Added Δ‑SBOM & Policy Eval phases; updated targets & current results. |
|
||||
| 2025‑07‑12 | First public workbook (SBOM‑first, image‑unpack, feed merge). |
|
||||
|
||||
---
|
||||
209
docs/13_RELEASE_ENGINEERING_PLAYBOOK.md
Normal file
209
docs/13_RELEASE_ENGINEERING_PLAYBOOK.md
Normal file
@ -0,0 +1,209 @@
|
||||
# 13 · Release Engineering Playbook — Stella Ops
|
||||
|
||||
|
||||
A concise, automation‑first guide describing **how source code on `main` becomes a verifiably signed, air‑gap‑friendly release**.
|
||||
It is opinionated for offline use‑cases and supply‑chain security (SLSA ≥ level 2 today, aiming for level 3).
|
||||
|
||||
---
|
||||
|
||||
## 0 Release Philosophy
|
||||
|
||||
* **Fast but fearless** – every commit on `main` must be releasable; broken builds break the build, not the team.
|
||||
* **Reproducible** – anyone can rebuild byte‑identical artefacts with a single `make release` offline.
|
||||
* **Secure by default** – every artefact ships with a SBOM, Cosign signature and (future) Rekor log entry.
|
||||
* **Offline‑first** – all dependencies are vendored or mirrored into the internal registry; no Internet required at runtime.
|
||||
|
||||
---
|
||||
|
||||
## 1 Versioning & Branching
|
||||
|
||||
| Branch | Purpose | Auto‑publish? |
|
||||
| ------------- | ------------------------------ | --------------------------------------- |
|
||||
| `main` | Always‑green development trunk | `nightly-*` images |
|
||||
| `release/X.Y` | Stabilise a minor line | `stella:X.Y-rcN` |
|
||||
| Tags | `X.Y.Z` = SemVer | `stella:X.Y.Z`, OUK tarball, Helm chart |
|
||||
|
||||
* **SemVer** – MAJOR for breaking API/CLI changes, MINOR for features, PATCH for fixes.
|
||||
* Release tags are **signed** (`git tag -s`) with the Stella Ops GPG key (`0x90C4…`).
|
||||
|
||||
---
|
||||
|
||||
## 2 CI/CD Overview (GitLab CI + GitLab Runner)
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[push / MR] --> Lint
|
||||
Lint --> Unit
|
||||
Unit --> Build
|
||||
Build --> Test-Container
|
||||
Test-Container --> SBOM
|
||||
SBOM --> Sign
|
||||
Sign --> Publish
|
||||
Publish --> E2E
|
||||
Publish --> Notify
|
||||
```
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
| Stage | Key tasks |
|
||||
| ------------------ | ------------------------------------------------------------------------------------------------ |
|
||||
| **Lint** | ESLint, golangci‑lint, hadolint, markdown‑lint. |
|
||||
| **Unit** | `dotnet test`, `go test`, Jest UI tests. |
|
||||
| **Quota unit‑tests 🏷** | Validate QuotaService logic: reset at UTC, 5 s vs 60 s waits, header correctness. |
|
||||
| **Build** | Multi‑arch container build (`linux/amd64`, `linux/arm64`) using **BuildKit** + `--provenance` 📌. |
|
||||
| **Test‑Container** | Spin up compose file, run smoke APIs. |
|
||||
| **SBOM** 📌 | Invoke **StellaOps.SBOMBuilder** to generate SPDX JSON + attach `.sbom` label to image. |
|
||||
| **Sign** | Sign image with **Cosign** (`cosign sign --key cosign.key`). |
|
||||
| **Publish** | Push to `registry.git.stella-ops.ru`. |
|
||||
| **E2E** | Kind‑based Kubernetes test incl. Zastava DaemonSet; verify sub‑5 s scan SLA. |
|
||||
| **Notify** | Report to Mattermost & GitLab Slack app. |
|
||||
| **OfflineToken** | Call `JwtIssuer.Generate(exp=30d)` → store `client.jwt` artefact → attach to OUK build context |
|
||||
|
||||
*All stages run in parallel where possible; max wall‑time < 15 min.*
|
||||
|
||||
---
|
||||
|
||||
## 3 Container Image Strategy
|
||||
|
||||
| Image | Registry Tag | Contents |
|
||||
| ------------------------------ | --------------------------- | ---------------------------------------------------------------------- |
|
||||
| **backend** | `stella/backend:{ver}` | ASP.NET API, plugin loader. |
|
||||
| **ui** | `stella/ui:{ver}` | Pre‑built Angular SPA. |
|
||||
| **runner-trivy** | `stella/runner-trivy:{ver}` | Trivy CLI + SPDX/CycloneDX 🛠. |
|
||||
| **runner-grype** | `stella/runner-grype:{ver}` | Optional plug‑in scanner. |
|
||||
| **🏷️ StellaOps.Registry** 📌 | `stella/registry:{ver}` | Scratch image embedding Docker Registry v2 + Cosign policy controller. |
|
||||
| **🏷️ StellaOps.MutePolicies** 📌 | `stella/policies:{ver}` | Sidecar serving policy bundles. |
|
||||
| **🏷️ StellaOps.Attestor** 📌 | `stella/attestor:{ver}` | SLSA provenance & Rekor signer (future). |
|
||||
|
||||
*Images are **`--label org.opencontainers.image.source=git.stella-ops.ru`** and include SBOMs generated at build time.*
|
||||
|
||||
---
|
||||
|
||||
## 4 📌 Offline Update Kit (OUK) Build & Distribution
|
||||
|
||||
**Purpose** – deliver updated CVE feeds & Trivy DB to air‑gapped clusters.
|
||||
|
||||
### 4.1 CLI Tool
|
||||
|
||||
*Go binary `ouk` lives in `tools/ouk/`.*
|
||||
|
||||
```sh
|
||||
ouk fetch \
|
||||
--nvd --osv \
|
||||
--trivy-db --date $(date -I) \
|
||||
--output ouk-$(date +%Y%m%d).tar.gz \
|
||||
--sign cosign.key
|
||||
```
|
||||
|
||||
### 4.2 Pipeline Hook
|
||||
|
||||
* Runs on **first Friday** each month (cron).
|
||||
* Generates tarball, signs it, uploads to **GitLab Release asset**.
|
||||
* SHA‑256 + signature published alongside.
|
||||
|
||||
### 4.3 Activation Flow (runtime)
|
||||
|
||||
1. Admin uploads `.tar.gz` via **UI → Settings → Offline Updates (OUK)**.
|
||||
2. Backend verifies Cosign signature & digest.
|
||||
3. Files extracted into `var/lib/stella/db`.
|
||||
4. Redis caches invalidated; Dashboard “Feed Age” ticks green.
|
||||
5. Audit event `ouk_update` stored.
|
||||
|
||||
### 4.4 Token Detail
|
||||
|
||||
client.jwt placed under /root/ inside the tarball.
|
||||
CI job fails if token expiry < 29 days (guard against stale caches).
|
||||
|
||||
---
|
||||
|
||||
## 5 Artifact Signing & Transparency
|
||||
|
||||
| Artefact | Signer | Tool |
|
||||
| ------------ | --------------- | --------------------- |
|
||||
| Git tags | GPG (`0x90C4…`) | `git tag -s` |
|
||||
| Containers | Cosign key pair | `cosign sign` |
|
||||
| Helm Charts | prov file | `helm package --sign` |
|
||||
| OUK tarballs | Cosign | `cosign sign-blob` |
|
||||
|
||||
**Rekor** integration is **TODO** – once the internal Rekor mirror is online (`StellaOpsAttestor`) a post‑publish job will submit transparency log entries.
|
||||
|
||||
---
|
||||
|
||||
## 6 Release Checklist
|
||||
|
||||
1. CI pipeline green.
|
||||
2. Bump `VERSION` file.
|
||||
3. Tag `git tag -s X.Y.Z -m "Release X.Y.Z"` & push.
|
||||
4. GitLab CI auto‑publishes images & charts.
|
||||
5. Draft GitLab **Release Notes** using `tools/release-notes-gen`.
|
||||
6. Verify SBOM attachment with `stella sbom verify stella/backend:X.Y.Z`.
|
||||
7. Smoke‑test OUK tarball in offline lab.
|
||||
8. Announce in `#stella-release` Mattermost channel.
|
||||
|
||||
---
|
||||
|
||||
## 7 Hot‑fix Procedure
|
||||
|
||||
* Branch from latest tag → `hotfix/X.Y.Z+1-hf1`.
|
||||
* Apply minimal patch, add regression test.
|
||||
* CI pipeline (with reduced stages) must pass.
|
||||
* Tag `X.Y.Z+1`.
|
||||
* Publish only container + Helm chart; OUK not rebuilt.
|
||||
* Cherry‑pick back to `main`.
|
||||
|
||||
---
|
||||
|
||||
## 8 Deprecation & End‑of‑Life Policy
|
||||
|
||||
| Feature | Deprecation notice | Removal earliest |
|
||||
| ------------------------ | ------------------ | ---------------- |
|
||||
| Legacy CSV policy import | 2025‑10‑01 | 2026‑04‑01 |
|
||||
| Docker v1 Registry auth | 2025‑12‑01 | 2026‑06‑01 |
|
||||
| In‑image Trivy DB | 2025‑12‑15 | 2026‑03‑15 |
|
||||
|
||||
*At least 6 months notice; removal requires major version bump.*
|
||||
|
||||
---
|
||||
|
||||
## 9 📌 Non‑Commercial Usage Rules (English canonical)
|
||||
|
||||
1. **Free for internal security assessments** (company or personal).
|
||||
2. **SaaS resale / re‑hosting prohibited** without prior written consent (AGPL §13).
|
||||
3. If you distribute a fork with UI or backend modifications **you must**:
|
||||
* Publish the complete modified source code.
|
||||
* Retain the original Stella Ops attribution in UI footer and CLI `--version`.
|
||||
4. All third‑party dependencies remain under their respective licences (MIT, Apache‑2.0, ISC, BSD).
|
||||
5. Deployments in state‑regulated or classified environments must obey **ФЗ‑187** export rules.
|
||||
|
||||
---
|
||||
|
||||
## 10 Best Practices Snapshot 📌
|
||||
|
||||
* **SBOM‑per‑image** → attach at build time; store as OCI artifact for supply‑chain introspection.
|
||||
* **Provenance flag** (`--provenance=true`) in BuildKit fulfils SLSA 2 requirement.
|
||||
* Use **multi‑arch, reproducible builds** (`SOURCE_DATE_EPOCH` pins timestamps).
|
||||
* All pipelines enforce **Signed‑off‑by (DCO)**; CI fails if trailer missing.
|
||||
* `cosign policy` ensures only images signed by the project key run in production.
|
||||
|
||||
---
|
||||
|
||||
## 11 Contributing to Release Engineering
|
||||
|
||||
* Fork & create MR to `infra/release-*`.
|
||||
* All infra changes require green **`integration-e2e-offline`** job.
|
||||
* Discuss larger infra migrations in `#sig-release` Mattermost; decisions recorded in `ADR/` folder.
|
||||
|
||||
---
|
||||
|
||||
## 12 Change Log (high‑level)
|
||||
|
||||
| Version | Date | Note |
|
||||
| ------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.1 | 2025‑07‑15 | Added OUK build/publish pipeline, internal registry image (`StellaOps.Registry`), non‑commercial usage rules extraction, SBOM stage, BuildKit provenance. |
|
||||
| v2.0 | 2025‑07‑12 | Initial open‑sourcing of Release Engineering guide. |
|
||||
| v1.1 | 2025‑07‑09 | Fixed inner fencing; added retention policy |
|
||||
| v1.0 | 2025‑07‑09 | Initial playbook |
|
||||
|
||||
---
|
||||
|
||||
*(End of Release Engineering Playbook v1.1)*
|
||||
112
docs/14_GLOSSARY_OF_TERMS.md
Normal file
112
docs/14_GLOSSARY_OF_TERMS.md
Normal file
@ -0,0 +1,112 @@
|
||||
# 14 · Glossary of Terms — Stella Ops
|
||||
*(v1.0 — 12 Jul 2025 · first real content, replaces placeholder v0.1)*
|
||||
|
||||
---
|
||||
|
||||
### 0 Purpose
|
||||
A concise, single‑page **“what does that acronym actually mean?”** reference for
|
||||
developers, DevOps engineers, IT managers and auditors who are new to the
|
||||
Stella Ops documentation set.
|
||||
|
||||
*If you meet a term in any Stella Ops doc that is **not** listed here, please
|
||||
open a PR and append it alphabetically.*
|
||||
|
||||
---
|
||||
|
||||
## A – C
|
||||
|
||||
| Term | Short definition | Links / notes |
|
||||
|------|------------------|---------------|
|
||||
| **ADR** | *Architecture Decision Record* – lightweight Markdown file that captures one irreversible design decision. | ADR template lives at `/docs/adr/` |
|
||||
| **AIRE** | *AI Risk Evaluator* – optional Plus/Pro plug‑in that suggests mute rules using an ONNX model. | Commercial feature |
|
||||
| **Azure‑Pipelines** | CI/CD service in Microsoft Azure DevOps. | Recipe in Pipeline Library |
|
||||
| **BDU** | Russian (FSTEC) national vulnerability database: *База данных уязвимостей*. | Merged with NVD by FeedMerger |
|
||||
| **BuildKit** | Modern Docker build engine with caching and concurrency. | Needed for layer cache patterns |
|
||||
| **CI** | *Continuous Integration* – automated build/test pipeline. | Stella integrates via CLI |
|
||||
| **Cosign** | Open‑source Sigstore tool that signs & verifies container images **and files**. | Images & OUK tarballs |
|
||||
| **CWV / CLS** | *Core Web Vitals* metric – Cumulative Layout Shift. | UI budget ≤ 0.1 |
|
||||
| **CycloneDX** | Open SBOM (BOM) standard alternative to SPDX. | Planned report format plug‑in |
|
||||
|
||||
---
|
||||
|
||||
## D – G
|
||||
|
||||
| Term | Definition | Notes |
|
||||
|------|------------|-------|
|
||||
| **Digest (image)** | SHA‑256 hash uniquely identifying a container image or layer. | Pin digests for reproducible builds |
|
||||
| **Docker‑in‑Docker (DinD)** | Running Docker daemon inside a CI container. | Used in GitHub / GitLab recipes |
|
||||
| **DTO** | *Data Transfer Object* – C# record serialised to JSON. | Schemas in doc 11 |
|
||||
| **FeedMerger** | Background job that merges NVD JSON and (optionally) BDU XML into Redis. | Cron default `0 1 * * *` |
|
||||
| **FSTEC** | Russian regulator issuing SOBIT certificates. | Pro GA target |
|
||||
| **Gitea** | Self‑hosted Git service – mirrors GitHub repo. | OSS hosting |
|
||||
| **GOST TLS** | TLS cipher‑suites defined by Russian GOST R 34.10‑2012 / 34.11‑2012. | Provided by `OpenSslGost` or CryptoPro |
|
||||
| **Grype** | Alternative OSS vulnerability scanner; can be hot‑loaded as plug‑in. | Scanner interface `IScannerRunner` |
|
||||
|
||||
---
|
||||
|
||||
## H – L
|
||||
|
||||
| Term | Definition | Notes |
|
||||
|------|------------|-------|
|
||||
| **Helm** | Kubernetes package manager (charts). | Beta chart under `/charts/core` |
|
||||
| **Hot‑load** | Runtime discovery & loading of plug‑ins **without restart**. | Cosign‑signed DLLs |
|
||||
| **Hyperfine** | CLI micro‑benchmark tool used in Performance Workbook. | Outputs CSV |
|
||||
| **JWT** | *JSON Web Token* – bearer auth token issued by OpenIddict. | Scope `scanner`, `admin`, `ui` |
|
||||
| **K3s / RKE2** | Lightweight Kubernetes distributions (Rancher). | Supported in K8s guide |
|
||||
| **Kubernetes NetworkPolicy** | K8s resource controlling pod traffic. | Redis/Mongo isolation |
|
||||
|
||||
---
|
||||
|
||||
## M – O
|
||||
|
||||
| Term | Definition | Notes |
|
||||
|------|------------|-------|
|
||||
| **Mongo (optional)** | Document DB storing > 180 day history and audit logs. | Off by default in Core |
|
||||
| **Mute rule** | JSON object that suppresses specific CVEs until expiry. | Schema `mute-rule‑1.json` |
|
||||
| **NVD** | US‑based *National Vulnerability Database*. | Primary CVE source |
|
||||
| **ONNX** | Portable neural‑network model format; used by AIRE. | Runs in‑process |
|
||||
| **OpenIddict** | .NET library that implements OAuth2 / OIDC in Stella backend. | Embedded IdP |
|
||||
| **OUK** | *Offline Update Kit* – signed tarball with images + feeds for air‑gap. | Admin guide #24 |
|
||||
| **OTLP** | *OpenTelemetry Protocol* – exporter for traces & metrics. | `/metrics` endpoint |
|
||||
|
||||
---
|
||||
|
||||
## P – S
|
||||
|
||||
| Term | Definition | Notes |
|
||||
|------|------------|-------|
|
||||
| **P95** | 95th‑percentile latency metric. | Target ≤ 5 s SBOM path |
|
||||
| **PDF SAR** | *Security Assessment Report* PDF produced by Pro edition. | Cosign‑signed |
|
||||
| **Plug‑in** | Hot‑loadable DLL implementing a Stella contract (`IScannerRunner`, `ITlsProvider`, etc.). | Signed with Cosign |
|
||||
| **Problem Details** | RFC 7807 JSON error format returned by API. | See API ref §0 |
|
||||
| **Redis** | In‑memory datastore used for queue + cache. | Port 6379 |
|
||||
| **Rekor** | Sigstore transparency log; future work for signature anchoring. | Road‑map P4 |
|
||||
| **RPS** | *Requests Per Second*. | Backend perf budget 40 rps |
|
||||
| **SBOM** | *Software Bill of Materials* – inventory of packages in an image. | Trivy JSON v2 |
|
||||
| **Santech** | Lightweight cli that sends SBOM for vulnerability scanning | |
|
||||
| **Seccomp** | Linux syscall filter JSON profile. | Backend shipped non‑root |
|
||||
| **SLA** | *Service‑Level Agreement* – 24 h / 1‑ticket for Pro. | SRE runbook |
|
||||
| **Span<T>** | .NET ref‑like struct for zero‑alloc slicing. | Allowed with benchmarks |
|
||||
| **Styker.NET** | Mutation testing runner used on critical libs. | Coverage ≥ 60 % |
|
||||
|
||||
---
|
||||
|
||||
## T – Z
|
||||
|
||||
| Term | Definition | Notes |
|
||||
|------|------------|-------|
|
||||
| **Trivy** | OSS CVE scanner powering the default `IScannerRunner`. | CLI pinned 0.64 |
|
||||
| **Trivy‑srv** | Long‑running Trivy server exposing gRPC API; speeds up remote scans. | Variant A |
|
||||
| **UI tile** | Dashboard element showing live metric (scans today, feed age, etc.). | Angular Signals |
|
||||
| **WebSocket** | Full‑duplex channel (`/ws/scan`, `/ws/stats`) for UI real‑time. | Used by tiles |
|
||||
| **Zastava** | Lightweight agent that inventories running containers and can enforce kills. | |
|
||||
|
||||
---
|
||||
|
||||
### 11 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---------|------|-------|
|
||||
| **v1.0** | 2025‑07‑12 | First populated glossary – 52 terms covering Core docs. |
|
||||
|
||||
*(End of Glossary v1.0)*
|
||||
234
docs/15_UI_GUIDE.md
Normal file
234
docs/15_UI_GUIDE.md
Normal file
@ -0,0 +1,234 @@
|
||||
# 15 - Pragmatic UI Guide --- **Stella Ops**
|
||||
|
||||
# Stella Ops Web UI
|
||||
|
||||
A fast, modular single‑page application for controlling scans, policies, offline updates and platform‑wide settings.
|
||||
Built for sub‑second feedback, dark‑mode by default, and **no external CDNs** – everything ships inside the anonymous internal registry.
|
||||
|
||||
---
|
||||
|
||||
## 0 Fast Facts
|
||||
|
||||
| Aspect | Detail |
|
||||
| ----------------- | -------------------------------------------------------------------------- |
|
||||
| Tech Stack | **Angular 17** + Vite dev server |
|
||||
| Styling | **Tailwind CSS** |
|
||||
| State | Angular Signals + RxJS |
|
||||
| API Client | OpenAPI v3 generated services (Axios) |
|
||||
| Auth | OAuth2 /OIDC (tokens from backend or external IdP) |
|
||||
| i18n | JSON bundles – **`/locales/{lang}.json`** (English, Russian shipped) |
|
||||
| Offline Updates 📌 | UI supports “OUK” tarball upload to refresh NVD / Trivy DB when air‑gapped |
|
||||
| Build Artifacts | (`ui/dist/`) pushed to `registry.git.stella-ops.ru/ui:${SHA}` |
|
||||
|
||||
---
|
||||
|
||||
## 1 Navigation Map
|
||||
|
||||
```
|
||||
Dashboard
|
||||
└─ Scans
|
||||
├─ Active
|
||||
├─ History
|
||||
└─ Reports
|
||||
└─ Policies 📌
|
||||
├─ Editor (YAML / Rego) 📌
|
||||
├─ Import / Export 📌
|
||||
└─ History
|
||||
└─ Settings
|
||||
├─ SBOM Format 📌
|
||||
├─ Registry 📌
|
||||
├─ Offline Updates (OUK) 📌
|
||||
├─ Themes (Light / Dark / System) 📌
|
||||
└─ Advanced
|
||||
└─ Plugins 🛠
|
||||
└─ Help / About
|
||||
```
|
||||
|
||||
*The **Offline Updates (OUK)** node under **Settings** is new.*
|
||||
|
||||
---
|
||||
|
||||
## 2 Technology Overview
|
||||
|
||||
### 2.1 Build & Deployment
|
||||
|
||||
1. `npm i && npm build` → generates `dist/` (~2.1 MB gzip).
|
||||
2. A CI job tags and pushes the artifact as `ui:${GIT_SHA}` to the internal registry.
|
||||
3. Backend serves static assets from `/srv/ui` (mounted from the image layer).
|
||||
|
||||
_No external fonts or JS – true offline guarantee._
|
||||
|
||||
### 2.2 Runtime Boot
|
||||
|
||||
1. **AppConfigService** pulls `/api/v1/config/ui` (contains feature flags, default theme, enabled plugins).
|
||||
2. Locale JSON fetched (`/locales/{lang}.json`, falls back to `en`).
|
||||
3. Root router mounts lazy‑loaded **feature modules** in the order supplied by backend – this is how future route plugins inject pages without forking the UI.
|
||||
|
||||
---
|
||||
|
||||
## 3 Feature Walk‑Throughs
|
||||
|
||||
### 3.1 Dashboard – Real‑Time Status
|
||||
|
||||
* **Δ‑SBOM heat‑map** 📌 shows how many scans used delta mode vs. full unpack.
|
||||
* “Feed Age” tile turns **orange** if NVD feed is older than 24 h; reverts after an **OUK** upload 📌.
|
||||
* Live WebSocket updates for scans in progress (SignalR channel).
|
||||
* **Quota Tile** – shows **Scans Today / 333**; turns yellow at 200,
|
||||
red at 333.
|
||||
* **Token Expiry Tile** – shows days left on *client.jwt* (offline only);
|
||||
turns orange at < 7 days.
|
||||
|
||||
### 3.2 Scans Module
|
||||
|
||||
| View | What you can do |
|
||||
| ----------- | ------------------------------------------------------------------------------------------------- |
|
||||
| **Active** | Watch progress bar (ETA ≤ 5 s) – newly added **Format** and **Δ** badges appear beside each item. |
|
||||
| **History** | Filter by repo, tag, policy result (pass/block/soft‑fail). |
|
||||
| **Reports** | Click row → HTML or PDF report rendered by backend (`/report/{digest}/html`). |
|
||||
|
||||
### 3.3 📌 Policies Module (new)
|
||||
|
||||
*Embedded **Monaco** editor with YAML + Rego syntax highlighting.*
|
||||
|
||||
| Tab | Capability |
|
||||
| ------------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| **Editor** | Write or paste `scan-policy.yaml` or inline Rego snippet. Schema validation shown inline. |
|
||||
| **Import / Export** | Buttons map to `/policy/import` and `/policy/export`. Accepts `.yaml`, `.rego`, `.zip` (bundle). |
|
||||
| **History** | Immutable audit log; diff viewer highlights rule changes. |
|
||||
|
||||
#### 3.3.1 YAML → Rego Bridge
|
||||
|
||||
If you paste YAML but enable **Strict Mode** (toggle), backend converts to Rego under the hood, stores both representations, and shows a side‑by‑side diff.
|
||||
|
||||
### 3.4 📌 Settings Enhancements
|
||||
|
||||
| Setting | Details |
|
||||
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **SBOM Format** | Dropdown – *Trivy JSON*, *SPDX JSON*, *CycloneDX JSON*. |
|
||||
| **Registry** | Displays pull URL (`registry.git.stella-ops.ru`) and Cosign key fingerprint. |
|
||||
| **Offline Updates (OUK)** 📌 | Upload **`ouk*.tar.gz`** produced by the Offline Update Kit CLI. Backend unpacks, verifies SHA‑256 checksum & Cosign signature, then reloads Redis caches without restart. |
|
||||
| **Theme** | Light, Dark, or Auto (system). |
|
||||
|
||||
#### 3.4.1 OUK Upload Screen 📌
|
||||
|
||||
*Page path:* **Settings → Offline Updates (OUK)**
|
||||
*Components:*
|
||||
|
||||
1. **Drop Zone** – drag or select `.tar.gz` (max 1 GB).
|
||||
2. **Progress Bar** – streaming upload with chunked HTTP.
|
||||
3. **Verification Step** – backend returns status:
|
||||
* *Signature valid* ✔️
|
||||
* *Digest mismatch* ❌
|
||||
4. **Feed Preview** – table shows *NVD date*, *OUI source build tag*, *CVE count delta*.
|
||||
5. **Activate** – button issues `/feeds/activate/{id}`; on success the Dashboard “Feed Age” tile refreshes to green.
|
||||
6. **History List** – previous OUK uploads with user, date, version; supports rollback.
|
||||
|
||||
*All upload actions are recorded in the Policies → History audit log as type `ouk_update`.*
|
||||
|
||||
### 3.5 Plugins Panel 🛠 (ships after UI modularisation)
|
||||
|
||||
Lists discovered UI plugins; each can inject routes/panels. Toggle on/off without reload.
|
||||
|
||||
### 3.6 Settings → **Quota & Tokens** (new)
|
||||
|
||||
* View current **Client‑JWT claims** (tier, maxScansPerDay, expiry).
|
||||
* **Generate Offline Token** – admin‑only button → POST `/token/offline` (UI wraps the API).
|
||||
* Upload new token file for manual refresh.
|
||||
|
||||
---
|
||||
|
||||
## 4 i18n & l10n
|
||||
|
||||
* JSON files under `/locales`.
|
||||
* Russian (`ru`) ships first‑class, translated security terms align with **GOST R ISO/IEC 27002‑2020**.
|
||||
* “Offline Update Kit” surfaces as **“Оффлайн‑обновление базы уязвимостей”** in Russian locale.
|
||||
* Community can add locales by uploading a new JSON via Plugins Panel once 🛠 ships.
|
||||
|
||||
---
|
||||
|
||||
## 5 Accessibility
|
||||
|
||||
* WCAG 2.1 AA conformance targeted.
|
||||
* All color pairs pass contrast (checked by `vite-plugin-wcag`).
|
||||
* Keyboard navigation fully supported; focus outlines visible in both themes.
|
||||
|
||||
---
|
||||
|
||||
## 6 Theming 📌
|
||||
|
||||
| Layer | How to change |
|
||||
| --------------- | ------------------------------------------------------------ |
|
||||
| Tailwind | Palette variables under `tailwind.config.js > theme.colors`. |
|
||||
| Runtime toggle | Stored in `localStorage.theme`, synced across tabs. |
|
||||
| Plugin override | Future route plugins may expose additional palettes 🛠. |
|
||||
|
||||
---
|
||||
|
||||
## 7 Extensibility Hooks
|
||||
|
||||
| Area | Contract | Example |
|
||||
| ------------- | ---------------------------------------- | ---------------------------------------------- |
|
||||
| New route | `window.stella.registerRoute()` | “Secrets” scanner plugin adds `/secrets` page. |
|
||||
| External link | `window.stella.addMenuLink(label, href)` | “Docs” link opens corporate Confluence. |
|
||||
| Theme | `window.stella.registerTheme()` | High‑contrast palette for accessibility. |
|
||||
|
||||
---
|
||||
|
||||
## 8 Road‑Map Tags
|
||||
|
||||
| Feature | Status |
|
||||
| ------------------------- | ------ |
|
||||
| Policy Editor (YAML) | ✅ |
|
||||
| Inline Rego validation | 🛠 |
|
||||
| OUK Upload UI | ✅ |
|
||||
| Plugin Marketplace UI | 🚧 |
|
||||
| SLSA Verification banner | 🛠 |
|
||||
| Rekor Transparency viewer | 🚧 |
|
||||
|
||||
---
|
||||
|
||||
## 9 Non‑Commercial Usage Rules 📌
|
||||
|
||||
*(Extracted & harmonised from the Russian UI help page so that English docs remain licence‑complete.)*
|
||||
|
||||
1. **Free for internal security assessments.**
|
||||
2. Commercial resale or SaaS re‑hosting **prohibited without prior written consent** under AGPL §13.
|
||||
3. If you distribute a fork **with UI modifications**, you **must**:
|
||||
* Make the complete source code (including UI assets) publicly available.
|
||||
* Retain original project attribution in footer.
|
||||
4. All dependencies listed in `ui/package.json` remain under their respective OSS licences (MIT, Apache 2.0, ISC).
|
||||
5. Use in government‑classified environments must comply with **ФЗ‑187** export rules; consult your legal team.
|
||||
|
||||
---
|
||||
|
||||
## 10 Troubleshooting Tips
|
||||
|
||||
| Symptom | Cause | Remedy |
|
||||
| ----------------------------------- | ----------------------------------- | ----------------------------------------------------------------- |
|
||||
| **White page** after login | `ui/dist/` hash mismatch | Clear browser cache; backend auto‑busts on version change. |
|
||||
| Policy editor shows “Unknown field” | YAML schema drift | Sync your policy file to latest sample in *Settings → Templates*. |
|
||||
| **OUK upload fails** at 99 % | Tarball built with outdated OUK CLI | Upgrade CLI (`ouk --version`) and rebuild package. |
|
||||
| Icons look broken in Safari | *SVG `mask` unsupported* | Use Safari 17+ or switch to PNG icon set in Settings > Advanced. |
|
||||
|
||||
---
|
||||
|
||||
## 11 Contributing
|
||||
|
||||
* Run `npm dev` and open `http://localhost:5173`.
|
||||
* Ensure `ng lint` and `ng test` pass before PR.
|
||||
* Sign the **DCO** in your commit footer (`Signed-off-by`).
|
||||
|
||||
---
|
||||
|
||||
## 12 Change Log
|
||||
|
||||
| Version | Date | Highlights |
|
||||
| ------- | ---------- |
|
||||
| v2.4 | 2025‑07‑15 | **Added full OUK Offline Update upload flow** – navigation node, Settings panel, dashboard linkage, audit hooks. |
|
||||
| v2.3 | 2025‑07‑14 | Added Policies module, SBOM Format & Registry settings, theming toggle, Δ‑SBOM indicators, extracted non‑commercial usage rules. |
|
||||
| v2.2 | 2025‑07‑12 | Added user tips/workflows, CI notes, DevSecOps section, troubleshooting, screenshots placeholders. |
|
||||
| v2.1 | 2025‑07‑12 | Removed PWA/Service‑worker; added oidc‑client‑ts; simplified roadmap |
|
||||
| v2.0 | 2025‑07‑12 | Accessibility, Storybook, perf budgets, security rules |
|
||||
| v1.1 | 2025‑07‑11 | Original OSS‑only guide |
|
||||
|
||||
(End of Pragmatic UI Guide v2.2)
|
||||
186
docs/17_SECURITY_HARDENING_GUIDE.md
Normal file
186
docs/17_SECURITY_HARDENING_GUIDE.md
Normal file
@ -0,0 +1,186 @@
|
||||
# 17 · Security Hardening Guide — **Stella Ops**
|
||||
*(v2.0 — 12 Jul 2025)*
|
||||
|
||||
> **Audience** — Site‑reliability and platform teams deploying **the open‑source Core** in production or restricted networks.
|
||||
---
|
||||
|
||||
## 0 Table of Contents
|
||||
|
||||
1. Threat model (summary)
|
||||
2. Host‑OS baseline
|
||||
3. Container & runtime hardening
|
||||
4. Network‑plane guidance
|
||||
5. Secrets & key management
|
||||
6. Image, SBOM & plug‑in supply‑chain controls
|
||||
7. Logging, monitoring & audit
|
||||
8. Update & patch strategy
|
||||
9. Incident‑response workflow
|
||||
10. Pen‑testing & continuous assurance
|
||||
11. Contacts & vulnerability disclosure
|
||||
12. Change log
|
||||
|
||||
---
|
||||
|
||||
## 1 Threat model (summary)
|
||||
|
||||
| Asset | Threats | Mitigations |
|
||||
| -------------------- | --------------------- | ---------------------------------------------------------------------- |
|
||||
| SBOMs & scan results | Disclosure, tamper | TLS‑in‑transit, read‑only Redis volume, RBAC, Cosign‑verified plug‑ins |
|
||||
| Backend container | RCE, code‑injection | Distroless image, non‑root UID, read‑only FS, seccomp + `CAP_DROP:ALL` |
|
||||
| Update artefacts | Supply‑chain attack | Cosign‑signed images & SBOMs, enforced by admission controller |
|
||||
| Admin credentials | Phishing, brute force | OAuth 2.0 with 12‑h token TTL, optional mTLS |
|
||||
|
||||
---
|
||||
|
||||
## 2 Host‑OS baseline checklist
|
||||
|
||||
| Item | Recommended setting |
|
||||
| ------------- | --------------------------------------------------------- |
|
||||
| OS | Ubuntu 22.04 LTS (kernel ≥ 5.15) or Alma 9 |
|
||||
| Patches | `unattended‑upgrades` or vendor‑equivalent enabled |
|
||||
| Filesystem | `noexec,nosuid` on `/tmp`, `/var/tmp` |
|
||||
| Docker Engine | v24.*, API socket root‑owned (`0660`) |
|
||||
| Auditd | Watch `/etc/docker`, `/usr/bin/docker*` and Compose files |
|
||||
| Time sync | `chrony` or `systemd‑timesyncd` |
|
||||
|
||||
---
|
||||
|
||||
## 3 Container & runtime hardening
|
||||
|
||||
### 3.1 Docker Compose reference (`compose-core.yml`)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
backend:
|
||||
image: ghcr.io/stellaops/backend:1.5.0
|
||||
user: "101:101" # non‑root
|
||||
read_only: true
|
||||
security_opt:
|
||||
- "no-new-privileges:true"
|
||||
- "seccomp:./seccomp-backend.json"
|
||||
cap_drop: [ALL]
|
||||
tmpfs:
|
||||
- /tmp:size=64m,exec,nosymlink
|
||||
environment:
|
||||
- ASPNETCORE_URLS=https://+:8080
|
||||
- TLSPROVIDER=OpenSslGost
|
||||
depends_on: [redis]
|
||||
networks: [core-net]
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "https://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
redis:
|
||||
image: redis:7.2-alpine
|
||||
command: ["redis-server", "--requirepass", "${REDIS_PASS}", "--rename-command", "FLUSHALL", ""]
|
||||
user: "redis"
|
||||
read_only: true
|
||||
cap_drop: [ALL]
|
||||
tmpfs:
|
||||
- /data
|
||||
networks: [core-net]
|
||||
|
||||
networks:
|
||||
core-net:
|
||||
driver: bridge
|
||||
```
|
||||
|
||||
No dedicated “Redis” or “Mongo” sub‑nets are declared; the single bridge network suffices for the default stack.
|
||||
|
||||
### 3.2 Kubernetes deployment highlights
|
||||
|
||||
Use a separate NetworkPolicy that only allows egress from backend to Redis :6379.
|
||||
securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, drop all capabilities.
|
||||
PodDisruptionBudget of minAvailable: 1.
|
||||
Optionally add CosignVerified=true label enforced by an admission controller (e.g. Kyverno or Connaisseur).
|
||||
|
||||
## 4 Network‑plane guidance
|
||||
|
||||
| Plane | Recommendation |
|
||||
| ------------------ | -------------------------------------------------------------------------- |
|
||||
| North‑south | Terminate TLS 1.2+ (OpenSSL‑GOST default). Use LetsEncrypt or internal CA. |
|
||||
| East‑west | Compose bridge or K8s ClusterIP only; no public Redis/Mongo ports. |
|
||||
| Ingress controller | Limit methods to GET, POST, PATCH (no TRACE). |
|
||||
| Rate‑limits | 40 rps default; tune ScannerPool.Workers and ingress limit‑req to match. |
|
||||
|
||||
## 5 Secrets & key management
|
||||
|
||||
| Secret | Storage | Rotation |
|
||||
| --------------------------------- | ---------------------------------- | ----------------------------- |
|
||||
| **Client‑JWT (offline)** | `/var/lib/stella/tokens/client.jwt` (root : 600) | **30 days** – provided by each OUK |
|
||||
| REDIS_PASS | Docker/K8s secret | 90 days |
|
||||
| OAuth signing key | /keys/jwt.pem (read‑only mount) | 180 days |
|
||||
| Cosign public key | /keys/cosign.pub baked into image; | change on every major release |
|
||||
| Trivy DB mirror token (if remote) | Secret + read‑only | 30 days |
|
||||
|
||||
Never bake secrets into images; always inject at runtime.
|
||||
|
||||
> **Operational tip:** schedule a cron reminding ops 5 days before
|
||||
> `client.jwt` expiry. The backend also emits a Prometheus metric
|
||||
> `stella_quota_token_days_remaining`.
|
||||
|
||||
## 6 Image, SBOM & plug‑in supply‑chain controls
|
||||
|
||||
* Images — Pull by digest not latest; verify:
|
||||
|
||||
```bash
|
||||
cosign verify ghcr.io/stellaops/backend@sha256:<DIGEST> \
|
||||
--key https://stella-ops.org/keys/cosign.pub
|
||||
```
|
||||
|
||||
* SBOM — Each release ships an SPDX file; store alongside images for audit.
|
||||
* Third‑party plug‑ins — Place in /plugins/; backend will:
|
||||
* Validate Cosign signature.
|
||||
* Check [StellaPluginVersion("major.minor")].
|
||||
* Refuse to start if Security.DisablePluginUnsigned=false (default).
|
||||
|
||||
## 7 Logging, monitoring & audit
|
||||
|
||||
| Control | Implementation |
|
||||
| ------------ | ----------------------------------------------------------------- |
|
||||
| Log format | Serilog JSON; ship via Fluent‑Bit to ELK or Loki |
|
||||
| Metrics | Prometheus /metrics endpoint; default Grafana dashboard in infra/ |
|
||||
| Audit events | Redis stream audit; export daily to SIEM |
|
||||
| Alert rules | Feed age ≥ 48 h, P95 wall‑time > 5 s, Redis used memory > 75 % |
|
||||
|
||||
## 8 Update & patch strategy
|
||||
|
||||
| Layer | Cadence | Method |
|
||||
| -------------------- | -------------------------------------------------------- | ------------------------------ |
|
||||
| Backend & CLI images | Monthly or CVE‑driven docker pull + docker compose up -d |
|
||||
| Trivy DB | 24 h cron via FeedMerger | configurable (FeedMerger.Cron) |
|
||||
| Docker Engine | vendor LTS | distro package manager |
|
||||
| Host OS | security repos enabled | unattended‑upgrades |
|
||||
|
||||
## 9 Incident‑response workflow
|
||||
|
||||
* Detect — PagerDuty alert from Prometheus or SIEM.
|
||||
* Contain — Stop affected Backend container; isolate Redis RDB snapshot.
|
||||
* Eradicate — Pull verified images, redeploy, rotate secrets.
|
||||
* Recover — Restore RDB, replay SBOMs if history lost.
|
||||
* Review — Post‑mortem within 72 h; create follow‑up issues.
|
||||
* Escalate P1 incidents to <security@stella‑ops.org> (24 × 7).
|
||||
|
||||
|
||||
## 10 Pen‑testing & continuous assurance
|
||||
|
||||
| Control | Frequency | Tool |
|
||||
|-------------------|-------------------|
|
||||
| OWASP | ZAP baseline | Each merge to main GitHub Action zap-baseline-scan |
|
||||
| Dependency scanning | Pull request | Trivy FS + GitHub Dependabot |
|
||||
| External red‑team | Annual or before GA | 3rd‑party CREST‑accredited vendor |
|
||||
|
||||
## 11 Vulnerability disclosure & contact
|
||||
|
||||
* Preferred channel: security@stella‑ops.org (GPG key on website).
|
||||
* Coordinated disclosure reward: public credit and swag (no monetary bounty at this time).
|
||||
|
||||
## 12 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.0 | 2025‑07‑12 | Full overhaul: host‑OS baseline, supply‑chain signing, removal of unnecessary sub‑nets, role‑based contact e‑mail, K8s guidance. |
|
||||
| v1.1 | 2025‑07‑09 | Minor fence fixes. |
|
||||
| v1.0 | 2025‑07‑09 | Original draft. |
|
||||
169
docs/18_CODING_STANDARDS.md
Normal file
169
docs/18_CODING_STANDARDS.md
Normal file
@ -0,0 +1,169 @@
|
||||
# 18 · Coding Standards & Contributor Guide — **Stella Ops**
|
||||
*(v2.0 — 12 Jul 2025 · supersedes v1.0)*
|
||||
|
||||
> **Audience** — Anyone sending a pull‑request to the open‑source Core.
|
||||
> **Goal** — Keep the code‑base small‑filed, plug‑in‑friendly, DI‑consistent, and instantly readable.
|
||||
|
||||
---
|
||||
|
||||
## 0 Why read this?
|
||||
|
||||
* Cuts review time → quicker merges.
|
||||
* Guarantees code is **hot‑load‑safe** for run‑time plug‑ins.
|
||||
* Prevents style churn and merge conflicts.
|
||||
|
||||
---
|
||||
|
||||
## 1 High‑level principles
|
||||
|
||||
1. **SOLID first** – especially Interface & Dependency Inversion.
|
||||
2. **100‑line rule** – any file > 100 physical lines must be split or refactored.
|
||||
3. **Contract‑level ownership** – public abstractions live in lightweight *Contracts* libraries; impl classes live in runtime projects.
|
||||
4. **Single Composition Root** – all DI wiring happens in **`StellaOps.Web/Program.cs`** and in each plug‑in’s `IoCConfigurator`; nothing else calls `IServiceCollection.BuildServiceProvider`.
|
||||
5. **No Service Locator** – constructor injection only; static `ServiceProvider` is banned.
|
||||
6. **Fail‑fast startup** – configuration validated before the web‑host listens.
|
||||
7. **Hot‑load compatible** – no static singletons that survive plug‑in unload; avoid `Assembly.LoadFrom` outside the built‑in plug‑in loader.
|
||||
|
||||
---
|
||||
|
||||
## 2 Repository layout (flat, July‑2025)**
|
||||
|
||||
```text
|
||||
src/
|
||||
├─ backend/
|
||||
│ ├─ StellaOps.Web/ # ASP.NET host + composition root
|
||||
│ ├─ StellaOps.Common/ # Serilog, Result<T>, helpers
|
||||
│ ├─ StellaOps.Contracts/ # DTO + interface contracts (no impl)
|
||||
│ ├─ StellaOps.Configuration/ # Options + validation
|
||||
│ ├─ StellaOps.Localization/
|
||||
│ ├─ StellaOps.PluginLoader/ # Cosign verify, hot‑load
|
||||
│ ├─ StellaOps.Scanners.Trivy/ # First‑party scanner
|
||||
│ ├─ StellaOps.TlsProviders.OpenSsl/
|
||||
│ └─ … (additional runtime projects)
|
||||
├─ plugins-sdk/ # Templated contracts & abstractions
|
||||
└─ frontend/ # Angular workspace
|
||||
tests/ # Mirrors src structure 1‑to‑1
|
||||
```
|
||||
|
||||
There are no folders named “Module” and no nested solutions.
|
||||
|
||||
## 3 Naming & style conventions
|
||||
|
||||
| Element | Rule | Example |
|
||||
| ------------------------------------------------------------------------------- | --------------------------------------- | ------------------------------- |
|
||||
| Namespaces | File‑scoped, StellaOps.<Area> | namespace StellaOps.Scanners; |
|
||||
| Interfaces | I prefix, PascalCase | IScannerRunner |
|
||||
| Classes / records | PascalCase | ScanRequest, TrivyRunner |
|
||||
| Private fields | camelCase (no leading underscore) | redisCache, httpClient |
|
||||
| Constants | SCREAMING_SNAKE_CASE | const int MAX_RETRIES = 3; |
|
||||
| Async methods | End with Async | Task<ScanResult> ScanAsync() |
|
||||
| File length | ≤ 100 lines incl. using & braces | enforced by dotnet format check |
|
||||
| Using directives | Outside namespace, sorted, no wildcards | — |
|
||||
|
||||
Static analyzers (.editorconfig, StyleCop.Analyzers package) enforce the above.
|
||||
|
||||
## 4 Dependency‑injection policy
|
||||
|
||||
Composition root – exactly one per process:
|
||||
|
||||
```csharp
|
||||
builder.Services
|
||||
.AddStellaCore() // extension methods from each runtime project
|
||||
.AddPluginLoader("/Plugins", cfg); // hot‑load signed DLLs
|
||||
```
|
||||
|
||||
Plug‑ins register additional services via the IoCConfigurator convention described in the Plug‑in SDK Guide, §5.
|
||||
Never resolve services manually (provider.GetService<T>()) outside the composition root; tests may use WebApplicationFactory or ServiceProvider.New() helpers.
|
||||
Scoped lifetime is default; singletons only for stateless, thread‑safe helpers.
|
||||
|
||||
## 5 Project organisation rules
|
||||
|
||||
Contracts vs. Runtime – public DTO & interfaces live in <Area>.Contracts; implementation lives in sibling project.
|
||||
Feature folders – inside each runtime project group classes by use‑case, e.g.
|
||||
|
||||
```text
|
||||
├─ Scan/
|
||||
│ ├─ ScanService.cs
|
||||
│ └─ ScanController.cs
|
||||
├─ Feed/
|
||||
└─ Tls/
|
||||
```
|
||||
|
||||
Tests – mirror the structure under tests/ one‑to‑one; no test code inside production projects.
|
||||
|
||||
## 6 C# language features
|
||||
|
||||
Nullable reference types enabled.
|
||||
record for immutable DTOs.
|
||||
Pattern matching encouraged; avoid long switch‑cascades.
|
||||
Span<T> & Memory<T> OK when perf‑critical, but measure first.
|
||||
Use await foreach over manual paginator loops.
|
||||
|
||||
## 7 Error‑handling template
|
||||
|
||||
```csharp
|
||||
public async Task<IActionResult> PostScan([FromBody] ScanRequest req)
|
||||
{
|
||||
if (!ModelState.IsValid) return BadRequest(ModelState);
|
||||
|
||||
try
|
||||
{
|
||||
ScanResult result = await scanService.ScanAsync(req);
|
||||
if (result.Quota != null)
|
||||
{
|
||||
Response.Headers.TryAdd("X-Stella-Quota-Remaining", result.Quota.Remaining.ToString());
|
||||
Response.Headers.TryAdd("X-Stella-Reset", result.Quota.ResetUtc.ToString("o"));
|
||||
}
|
||||
return Ok(result);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
RFC 7807 ProblemDetails for all non‑200s.
|
||||
Capture structured logs with Serilog’s message‑template syntax.
|
||||
|
||||
## 8 Async & threading
|
||||
|
||||
* All I/O is async; no .Result / .Wait().
|
||||
* Library code: ConfigureAwait(false).
|
||||
* Limit concurrency via Channel<T> or Parallel.ForEachAsync, never raw Task.Run loops.
|
||||
|
||||
## 9 Testing rules
|
||||
|
||||
| Layer | Framework | Coverage gate |
|
||||
| ------------------------ | ------------------------ | -------------------------- |
|
||||
| Unit | xUnit + FluentAssertions | ≥ 80 % line, ≥ 60 % branch |
|
||||
| Integration | Testcontainers | Real Redis & Trivy |
|
||||
| Mutation (critical libs) | Stryker.NET | ≥ 60 % score |
|
||||
|
||||
One test project per runtime/contract project; naming <Project>.Tests.
|
||||
|
||||
## 10 Static analysis & formatting
|
||||
|
||||
* dotnet format must exit clean (CI gate).
|
||||
* StyleCop.Analyzers + Roslyn‑Security‑Guard run on every PR.
|
||||
* CodeQL workflow runs nightly on main.
|
||||
|
||||
## 11 Commit & PR checklist
|
||||
|
||||
* Conventional Commit prefix (feat:, fix:, etc.).
|
||||
* dotnet format & dotnet test both green.
|
||||
* Added or updated XML‑doc comments for public APIs.
|
||||
* File count & length comply with 100‑line rule.
|
||||
* If new public contract → update relevant markdown doc & JSON‑Schema.
|
||||
|
||||
## 12 Common pitfalls
|
||||
|
||||
|Symptom| Root cause | Fix
|
||||
|-------|-------------|-------------------
|
||||
|InvalidOperationException: Cannot consume scoped service...| Mis‑matched DI lifetimes| Use scoped everywhere unless truly stateless
|
||||
|Hot‑reload plug‑in crash| Static singleton caching plugin types| Store nothing static; rely on DI scopes
|
||||
|
||||
> 100‑line style violation |Large handlers or utils |Split into private helpers or new class
|
||||
|
||||
## 13 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | -------------------------------------------------------------------------------------------------- |
|
||||
| v2.0 | 2025‑07‑12 | Updated DI policy, 100‑line rule, new repo layout, camelCase fields, removed “Module” terminology. |
|
||||
| 1.0 | 2025‑07‑09 | Original standards. |
|
||||
198
docs/19_TEST_SUITE_OVERVIEW.md
Normal file
198
docs/19_TEST_SUITE_OVERVIEW.md
Normal file
@ -0,0 +1,198 @@
|
||||
# 19 · Test‑Suite Overview — **Stella Ops**
|
||||
*(v2.0 — 12 Jul 2025)*
|
||||
|
||||
> **Purpose** — Describe the **multi‑layer automated‑test strategy** that guards Stella Ops’ five‑second performance promise, security posture and API stability, and show how each layer maps to CI gates and release criteria.
|
||||
|
||||
---
|
||||
|
||||
## 0 Table of Contents
|
||||
|
||||
1. Test‑pyramid at a glance
|
||||
2. Layer definitions & tooling
|
||||
3. Directory & naming conventions
|
||||
4. CI workflows & failure policy
|
||||
5. Quality gates & coverage budgets
|
||||
6. Evidence retention & auditability
|
||||
7. Local developer quick‑start
|
||||
8. Flaky‑test triage & escalation
|
||||
9. Change log
|
||||
|
||||
---
|
||||
|
||||
## 1 Test‑pyramid at a glance
|
||||
|
||||
| Layer | Framework(s) | Scope | CI frequency |
|
||||
| ---------------------- | ------------------------ | --------------------------------------- | ------------ |
|
||||
| **Unit** | xUnit + FluentAssertions | Pure C# methods, guard clauses, mapping | Every PR |
|
||||
| **Mutation** | **Stryker.NET** | Critical algorithm branches | Nightly |
|
||||
| **Static analysis** | **CodeQL**, **Semgrep** | OWASP, injection, secrets | Every PR |
|
||||
| **Integration** | Testcontainers + xUnit | Redis, Trivy exec, plug‑in hot‑load | Every PR |
|
||||
| **Quota / throttle** | Testcontainers + Clock‑mock | 333‑scan counter, 5 s & 60 s retry‑after headers | Every PR |
|
||||
| **End‑to‑End (UI)** | **Playwright C#** | Login, scan list, mute flow | Merge→main |
|
||||
| **Performance** | Hyperfine + K6 | P95 latency, 40 rps throughput | Nightly |
|
||||
| **Security DAST** | OWASP ZAP baseline | TLS headers, auth, XSS | Nightly + RC |
|
||||
| **Chaos / Resilience** | **Pumba** & Toxiproxy | Redis latency, container kill | Weekly |
|
||||
| **Compliance smoke** | Spectral + JSON‑Schema | SBOM & API payloads | Every PR |
|
||||
| **Token validity** | xUnit + ClockMock | Expiry warning, OUK update refresh, `/token/offline` flow | Every PR |
|
||||
|
||||
---
|
||||
|
||||
## 2 Layer definitions & tooling
|
||||
|
||||
### 2.1 Unit
|
||||
|
||||
* Target ≥ 80 % **line and** ≥ 60 % **branch** coverage (`coverlet` + ReportGenerator).
|
||||
* Naming: `Method_ShouldExpected_WhenCondition`.
|
||||
|
||||
### 2.2 Mutation
|
||||
|
||||
* **Stryker.NET** runs only on projects tagged `critical‑logic=true` in `Directory.Build.props`.
|
||||
* Threshold: ≥ 60 % mutation score; red build < 55 %.
|
||||
|
||||
### 2.3 Integration
|
||||
|
||||
* `RedisTestcontainer`, `TrivyServerTestcontainer`, `TestcontainersNetwork` for realistic wiring.
|
||||
* Each test cleans keys and volumes; parallelisable.
|
||||
|
||||
* **Quota & throttle tests (new)** — spin up Redis container, fix system clock to just before UTC midnight, hammer `/scan` with a stub token to validate:
|
||||
1. Counter hits **200** → header `X‑Stella‑Quota‑Remaining: 133`; banner socket event emitted. Delay of 5 secs is added.
|
||||
2. Counter hits **333** → Delay of 60 secs is added.
|
||||
3. At UTC midnight rollover key expires → counter resets to 0.
|
||||
|
||||
### 2.4 Quota / throttle layer (explicit)
|
||||
|
||||
* Uses the same fixture but runs in isolation to keep CI time predictable.
|
||||
* Fails the pipeline if **any** of the four behaviours above mis‑fires.
|
||||
|
||||
### 2.4 End‑to‑End
|
||||
|
||||
* API suite asserts presence of `X‑Stella‑Quota‑Remaining` on every successful `/scan`.
|
||||
* API suite uses **async httpx** for accurate latency numbers.
|
||||
* UI suite uses **Playwright** headless Chromium; Lighthouse a11y snapshot recorded.
|
||||
|
||||
### 2.5 Performance
|
||||
|
||||
* Hyperfine measures CLI workflows (`SBOM_LOCAL`, `SBOM_REMOTE`, `IMAGE_WARM`).
|
||||
* **K6** hits `/scan` at 40 rps for 3 min; checks P95 ≤ 5 s and error‑rate = 0.
|
||||
|
||||
### 2.6 Security (DAST + SAST)
|
||||
|
||||
* **PHASE QUOTA_WAIT** benchmark:
|
||||
* ≤ 5 s median for first 30 blocked requests (soft back‑off).
|
||||
* Exactly 60 s wall for hard wait‑wall.
|
||||
* SAST: **CodeQL** (GitHub native) + **Semgrep OSS** ruleset.
|
||||
* DAST: **ZAP baseline** spider + passive rules; fails on High risk alerts.
|
||||
|
||||
### 2.7 Chaos / Resilience
|
||||
|
||||
* **Pumba** randomly kills Trivy side‑car; test asserts queue retry.
|
||||
* **Toxiproxy** injects 150 ms latency on Redis; perf budget still ≤ 6 s.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 3 Repository layout
|
||||
|
||||
```text
|
||||
tests/
|
||||
├─ unit/ # *.Unit.csproj
|
||||
├─ mutation/stryker.conf.json
|
||||
├─ integration/ # *.Integration.csproj
|
||||
│ └─ fixtures/
|
||||
├─ e2e/
|
||||
│ ├─ api/pytest/ # test_*.py
|
||||
│ └─ ui/playwright/ # *.spec.ts
|
||||
├─ perf/
|
||||
│ ├─ compose-perf.yml
|
||||
│ ├─ hyperfine/
|
||||
│ └─ k6/
|
||||
├─ security/
|
||||
│ ├─ zap-baseline.conf
|
||||
│ └─ semgrep/
|
||||
└─ chaos/
|
||||
├─ toxiproxy/
|
||||
└─ pumba/
|
||||
```
|
||||
|
||||
Tests mirror the module namespaces; each src project owns a matching test project.
|
||||
|
||||
## 4 CI workflows
|
||||
|
||||
| File | Trigger | Stages |
|
||||
| ------------ | ----------------------------------------------------- | -------------------------------------- |
|
||||
| ci.yml | Push / PR Lint → Unit → Static analysis → Integration |
|
||||
| e2e.yml | Merge→main | Compose stack → API+UI Playwright |
|
||||
| perf.yml | Nightly | Hyperfine + K6; update Grafana JSON |
|
||||
| security.yml | Nightly | ZAP baseline, Trivy FS, CodeQL |
|
||||
| mutation.yml | Nightly | Stryker.NET; comment PR if < threshold |
|
||||
| chaos.yml | Weekly (cron) | Toxiproxy + Pumba scenarios |
|
||||
| release.yml | Tag | Run all above + evidence bundling |
|
||||
Failure policy: any Red gate blocks merge; nightly failures ping #stella-ci.
|
||||
|
||||
## 5 Quality gates & budgets
|
||||
|
||||
| Metric | Threshold | Source | Maps to KPI |
|
||||
| ----------------------------------- | ---------- | --------------------------------- | --------------- |
|
||||
| Line coverage | ≥ 80 % | Unit, Integration Maintainability |
|
||||
| Mutation score | ≥ 60 % | Stryker Defect escape |
|
||||
| P95 SBOM‑first | ≤ 5 s | Hyperfine | Product promise |
|
||||
| P95 QUOTA_WAIT (soft) | ≤ 10 s | Hyperfine + Clock‑mock | Predictable throttling |
|
||||
| Hard wait‑wall accuracy | 60 ± 1 s | Hyperfine | Compliance with spec |
|
||||
| P95 image‑unpack | ≤ 10 s | Hyperfine | SRS FR‑IMG‑1 |
|
||||
| /scan error‑rate | 0 | K6 | Reliability |
|
||||
| ZAP High alerts | 0 | ZAP JSON | Security NFR |
|
||||
| Trivy Critical CVEs in release SBOM | 0 Trivy FS | NFR‑SEC‑1 |
|
||||
| Offline token expiry warning lead‑time | ≥ 7 days | Token tests |
|
||||
|
||||
Coverage & perf budgets live in tests/budgets/*.json; CI actions fail on regression.
|
||||
|
||||
## 6 Evidence retention
|
||||
|
||||
| Artefact | Retention | Storage |
|
||||
| ------------------ | -------------- | --------------------- |
|
||||
| Hyperfine & K6 CSV | 18 months | GitHub artefacts → S3 |
|
||||
| Mutation reports | 6 months | S3 |
|
||||
| ZAP & Trivy SARIF | 18 months | GitHub Security tab |
|
||||
| Playwright videos | Last 50 builds | MinIO |
|
||||
|
||||
Test logs (JUnit/Allure) 12 months S3, lifecycle policy
|
||||
|
||||
## 7 Developer quick‑start
|
||||
|
||||
# Bring up full stack for e2e on a laptop
|
||||
|
||||
```bash
|
||||
docker compose -f tests/e2e/compose-core.yml up -d
|
||||
```
|
||||
|
||||
# Run unit + integration
|
||||
|
||||
```bash
|
||||
dotnet test --collect:"XPlat Code Coverage"
|
||||
|
||||
# API e2e
|
||||
cd tests/e2e/api
|
||||
pytest -q
|
||||
|
||||
# UI e2e
|
||||
cd tests/e2e/ui
|
||||
npx playwright install
|
||||
npm test
|
||||
```
|
||||
|
||||
## 8 Flaky‑test triage & escalation
|
||||
|
||||
Label failing test with flaky and open GitHub Discussion.
|
||||
After 3 consecutive nightly failures, auto‑page <ops@stella-ops.org>.
|
||||
Root‑cause within next sprint or quarantine behind feature flag (max 2 weeks).
|
||||
*Token‑expiry tests cannot be quarantined* — they guard offline operability.
|
||||
|
||||
|
||||
## 9 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.0 | 2025‑07‑12 | Full overhaul: mutation tests, CodeQL/Semgrep, chaos layer, role‑based escalation, perf/security budgets aligned with SRS. |
|
||||
| v1.0 | 2025‑07‑09 | Original minimal overview |
|
||||
|
||||
(End of Test‑Suite Overview v2.0)
|
||||
196
docs/21_INSTALL_GUIDE.md
Normal file
196
docs/21_INSTALL_GUIDE.md
Normal file
@ -0,0 +1,196 @@
|
||||
# 21 · Installation & Quick‑Start Guide — **Stella Ops**
|
||||
|
||||
*(v2.0 — 12 Jul 2025 · supersedes v1.1)*
|
||||
|
||||
> **Scope** — Docker Compose or Kubernetes deployment of the **open‑source Core** (backend + Redis).
|
||||
> For Plus/Pro features (CryptoPro TLS, LDAP, OUK, Enforcement) see the private *Commercial Install Guide*.
|
||||
|
||||
---
|
||||
|
||||
## 0 Contents
|
||||
|
||||
0. Audience & prerequisites
|
||||
1. Decide: Compose or Kubernetes
|
||||
2. Quick start (Compose, 5 min)
|
||||
3. Verifying image signatures & SBOMs
|
||||
4. First login & API token creation
|
||||
5. Optional TLS & reverse‑proxy patterns
|
||||
6. Kubernetes helm‑chart primer (beta)
|
||||
7. Configuration cheatsheet
|
||||
8. Upgrades & rollbacks
|
||||
9. Uninstall / data wipe
|
||||
10. Troubleshooting matrix
|
||||
11. Getting help
|
||||
12. Change log
|
||||
|
||||
---
|
||||
|
||||
## 0 Audience & prerequisites
|
||||
|
||||
| Item | Minimum | Notes |
|
||||
| --------- | ---------------------- | ------------------------------ |
|
||||
| OS | Ubuntu 22.04 or Alma 9 | x86‑64 or ARM‑64 |
|
||||
| CPU / RAM | 2 vCPU / 2 GB | Dev‑laptop baseline |
|
||||
| Disk | 10 GB SSD | SBOM + Trivy cache |
|
||||
| Docker | Engine 24 + Compose v2 | `docker ‑v` |
|
||||
| Network | HTTPS 443 open | Optional Let’s Encrypt HTTP‑01 |
|
||||
|
||||
---
|
||||
|
||||
## 1 Choose your path
|
||||
|
||||
| Path | When to pick | Doc section |
|
||||
| --------------------- | ----------------------------- | ----------- |
|
||||
| **Docker Compose** | Single VM, PoC, laptop | § 2 |
|
||||
| **Kubernetes (Helm)** | Existing K3s, RKE2, EKS, etc. | § 6 |
|
||||
|
||||
---
|
||||
|
||||
## 2 Quick start — Docker Compose (5 minutes)
|
||||
|
||||
### 2.1 Download stack files (signed)
|
||||
|
||||
```bash
|
||||
curl -LO https://get.stellaops.org/compose-core.yml
|
||||
curl -LO https://get.stellaops.org/compose-core.yml.sig
|
||||
cosign verify-blob \
|
||||
--key https://stella-ops.org/keys/cosign.pub \
|
||||
--signature compose-core.yml.sig \
|
||||
compose-core.yml
|
||||
```
|
||||
|
||||
### 2.2 Set secrets
|
||||
|
||||
```bash
|
||||
export REDIS_PASS=$(openssl rand -base64 18)
|
||||
echo "REDIS_PASS=$REDIS_PASS" >> .env
|
||||
```
|
||||
|
||||
### 2.3 Launch
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env -f compose-core.yml pull
|
||||
docker compose --env-file .env -f compose-core.yml up -d
|
||||
```
|
||||
|
||||
Containers pulled ✚ started in < 60 s:
|
||||
|
||||
ghcr.io/stellaops/backend:1.5.0
|
||||
redis:7.2-alpine
|
||||
|
||||
### 2.4 Health check
|
||||
|
||||
```bash
|
||||
curl -k https://localhost:8080/health
|
||||
# → {"redis":"OK","version":"1.5.0","commit":"5a1b7d3"}
|
||||
```
|
||||
|
||||
Open <https://localhost:8080> (accept self‑signed cert).
|
||||
|
||||
## 3 Verify image provenance (optional but recommended)
|
||||
|
||||
```bash
|
||||
BACKEND_DIGEST=sha256:… # copy from `docker images --digests`
|
||||
cosign verify \
|
||||
ghcr.io/stellaops/backend@$BACKEND_DIGEST \
|
||||
--key https://stella-ops.org/keys/cosign.pub
|
||||
```
|
||||
|
||||
SBOM lives in /usr/share/stellaops/sbom.spdx.json inside the image; auditors may ingest it directly.
|
||||
|
||||
## 4 Understanding the Free‑tier quota (333 scans / UTC day)
|
||||
|
||||
Stella Ops Core is completely free to use, but to prevent abuse the backend enforces a **daily quota of 333 scans per API token**.
|
||||
|
||||
* At **200 scans** a **yellow dashboard banner** reminds you of the limit (≈ 60 % usage).
|
||||
* Between **333 – 363** scans the `/scan` endpoint responds **`429 Too Many Requests`** and a **`Retry‑After: 5`** header.
|
||||
* Beyond **363** calls the server imposes a **hard 60 s wait‑wall** (`Retry‑After: 60`).
|
||||
* The counter resets at **UTC midnight**; no manual action required.
|
||||
* Upgrade paths:
|
||||
* **Self‑hosted Core** with a second API token (still 333/day).
|
||||
* **Plus / Pro licences** (unlimited).
|
||||
|
||||
If you script CI pipelines, honour the `Retry‑After` header to avoid unnecessary retries.
|
||||
|
||||
|
||||
## 6 First login & API token
|
||||
|
||||
1. Sign in with default **admin / changeme**.
|
||||
2. Navigate **Settings → API Tokens → Generate**.
|
||||
3. **Each token inherits the 333‑scan quota** by default.
|
||||
4. Store the token securely in your CI secret vault.
|
||||
|
||||
|
||||
## 7 TLS & reverse‑proxy options
|
||||
|
||||
| Scenario | Command / config |
|
||||
| ------------ | --------------------------------------------------------------------------------------------------- |
|
||||
| Dev l | aptop Keep backend self‑signed. |
|
||||
| Public | VM + Let’s Encrypt Install Caddy: caddy reverse‑proxy --from stella.example.com --to localhost:8080 |
|
||||
| Corporate CA | Replace /certs/cert.pem + /certs/key.pem, set TLSPROVIDER=None. |
|
||||
|
||||
## 8 Kubernetes (Helm chart β)
|
||||
|
||||
```bash
|
||||
helm repo add stella https://charts.stella-ops.org
|
||||
helm install stella-core stella/core \
|
||||
--set redis.password=$REDIS_PASS \
|
||||
--set ingress.host=stella.example.com
|
||||
```
|
||||
|
||||
Resources: 300 m CPU / 512 Mi backend, 128 Mi Redis.
|
||||
NetworkPolicy, PodSecurity & CosignVerify admission controller included.
|
||||
|
||||
## 9 Configuration cheatsheet (appsettings.json or env‑vars)
|
||||
|
||||
| Key | Default | Example |
|
||||
| ------------------- | ----------- | ---------------------------- |
|
||||
| ScannerPool.Workers | 1 | SCANNERPOOL__WORKERS=4 |
|
||||
| FeedMerger.Cron | 0 1 ** * | FEEDMERGER__CRON="30 2 ** *" |
|
||||
| Redis.Password | — | REDIS__PASSWORD=$REDIS_PASS |
|
||||
| TlsProvider | OpenSslGost | TLSPROVIDER=None |
|
||||
|
||||
## 10 Upgrades & rollbacks
|
||||
|
||||
```bash
|
||||
# pin digest before prod rollout
|
||||
docker pull ghcr.io/stellaops/backend@sha256:<NEW_DIGEST>
|
||||
docker compose up -d
|
||||
|
||||
# rollback
|
||||
docker compose up -d backend@sha256:<OLD_DIGEST>
|
||||
```
|
||||
|
||||
Minor (Y) upgrades are non‑breaking; major (X) may change CLI flags.
|
||||
|
||||
## 11 Uninstall
|
||||
|
||||
```bash
|
||||
docker compose -f compose-core.yml down -v
|
||||
docker volume prune -f
|
||||
rm compose-core.yml .env
|
||||
```
|
||||
|
||||
## 12 Troubleshooting matrix
|
||||
|
||||
| Symptom | Likely cause | Remedy |
|
||||
| -------------------- | -------------------------------------- | ----------------------------------------------------------------- |
|
||||
| 502 via Nginx | Backend self‑signed cert not trusted | proxy_ssl_verify off; or use proper CA |
|
||||
| Feed age banner red | Cron blocked by systemd time‑zone skew | Run: docker exec backend dotnet Stella.Backend feed merge |
|
||||
| Scan > 10 s | Cold Trivy DB | Second run should drop to < 5 s; else raise CPU / use remote mode |
|
||||
|
||||
## 13 Getting help
|
||||
|
||||
* Install issues: ops@stella‑ops.org (role‑based mailbox).
|
||||
* Security reports: security@stella‑ops.org (GPG key available).
|
||||
* Community chat: Telegram @stellaopsxw.
|
||||
* Docs index: <https://stella‑ops.org/docs>.
|
||||
|
||||
## 14 Change log
|
||||
|
||||
| Version | Date | Notes |
|
||||
| ------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| v2.0 | 2025‑07‑12 | Signed download, Cosign verification, secrets primer, TLS patterns, Helm chart, rollback steps, role‑based support addresses. |
|
||||
| v1.1 | 2025‑07‑11 | Original Core guide. |
|
||||
|
||||
(End of Install Guide v2.0 — Core Edition)
|
||||
245
docs/22_RECIPES_PIPELINES_LIBRARY.md
Normal file
245
docs/22_RECIPES_PIPELINES_LIBRARY.md
Normal file
@ -0,0 +1,245 @@
|
||||
# 22 · Pipeline Recipes Library — **Stella Ops**
|
||||
|
||||
|
||||
# Recipes & Pipeline Library
|
||||
|
||||
> *Ready‑to‑copy snippets for CI/CD engines, local shells, and K8s jobs.*
|
||||
> Each recipe honours the **sub‑5 s pledge**: SBOM‑first when possible, Δ‑SBOM when layers are cached, and image‑unpack only as a fall‑back.
|
||||
|
||||
---
|
||||
|
||||
## 0 Registry Primer
|
||||
|
||||
All agent images and helper tools are published to a **private, anonymous registry**
|
||||
|
||||
```
|
||||
registry.git.stella-ops.ru
|
||||
```
|
||||
|
||||
* **Pulls are read‑only & unauthenticated.**
|
||||
* Cosign signatures are embedded (`*.sig`) and verified at runtime when the host has `cosign` installed.
|
||||
* To mirror for air‑gapped OUK installs, export with:
|
||||
|
||||
```
|
||||
oras pull registry.git.stella-ops.ru/library/santech:1.0 --output ./ouk-bundle
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 1 Shell Quick‑starts
|
||||
|
||||
### 1.1 Scan a Local Image (SBOM‑first)
|
||||
|
||||
```bash
|
||||
# Free tier: 333 scans/day without large delay added
|
||||
docker run --rm \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
https://stella.local/registry/santech:1.0 \
|
||||
scan \
|
||||
--image python:3.12-slim \
|
||||
--endpoint https://stella.local \
|
||||
--sbom-type spdx-json \
|
||||
--threshold High
|
||||
```
|
||||
|
||||
* `--sbom-type` enumerates **`trivy-json-v2 | spdx-json | cyclonedx-json`**; defaults to auto‑detect when omitted.
|
||||
* Exit‑code maps to policy (non‑zero if blocked).
|
||||
|
||||
### 1.2 Delta SBOM Path
|
||||
|
||||
```bash
|
||||
# Free tier: 333 scans/day without large delay added
|
||||
|
||||
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-e STELLA_DELTA=1 \
|
||||
https://stella.local/registry/santech:1.0 \
|
||||
scan \
|
||||
--image myapp:latest \
|
||||
--delta \
|
||||
--endpoint https://stella.local
|
||||
```
|
||||
|
||||
`--delta` triggers the `/layers/missing` fast check; observed P95 ≤ 1 s on cached bases.
|
||||
|
||||
---
|
||||
|
||||
## 2 GitHub Actions
|
||||
|
||||
```yaml
|
||||
# .github/workflows/stella-scan.yml
|
||||
name: Stella Scan
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
security:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Build Image
|
||||
run: docker build -t ${{ github.repository }}:${{ github.sha }} .
|
||||
|
||||
- name: Scan with Stella‑Ops (Δ‑SBOM + YAML policies)
|
||||
run: |
|
||||
# Free tier: 333 scans/day without large delay added
|
||||
docker run --rm \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-v $GITHUB_WORKSPACE/policy:/policy:ro \
|
||||
https://stella.local/registry/santech:1.0 \
|
||||
scan \
|
||||
--image ${{ github.repository }}:${{ github.sha }} \
|
||||
--delta \
|
||||
--policy-file /policy/scan-policy.yaml \
|
||||
--endpoint ${{ secrets.STELLA_API }}
|
||||
```
|
||||
|
||||
*Stores SARIF to `$RUNNER_TEMP/stella-report.sarif`; upload via `actions/upload-artifact` if desired.*
|
||||
|
||||
---
|
||||
|
||||
## 3 GitLab CI
|
||||
|
||||
```yaml
|
||||
stella_scan:
|
||||
image:
|
||||
name: ttps://stella.local/registry/santech:1.0
|
||||
entrypoint: [""]
|
||||
stage: test
|
||||
variables:
|
||||
STELLA_ENDPOINT: "https://stella.local/api"
|
||||
script:
|
||||
- docker build -t myapp:$CI_COMMIT_SHORT_SHA .
|
||||
- >
|
||||
# Free tier: 333 scans/day without large delay added
|
||||
./santech scan
|
||||
--image myapp:$CI_COMMIT_SHORT_SHA
|
||||
--sbom-type cyclonedx-json
|
||||
--threshold Critical
|
||||
--endpoint $STELLA_ENDPOINT
|
||||
allow_failure: false
|
||||
artifacts:
|
||||
when: always
|
||||
paths:
|
||||
- stella-report.html
|
||||
```
|
||||
|
||||
*`allow_failure:false` enforces gate by failing the stage on Critical findings.*
|
||||
|
||||
---
|
||||
|
||||
## 4 Tekton Pipelines (K8s)
|
||||
|
||||
```yaml
|
||||
apiVersion: tekton.dev/v1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: stella-scan
|
||||
spec:
|
||||
workspaces:
|
||||
- name: dockerconfig
|
||||
steps:
|
||||
- name: scan
|
||||
image: https://stella.local/registry/santech:1.0
|
||||
script: |
|
||||
#!/usr/bin/env sh
|
||||
# Free tier: 333 scans/day without large delay added
|
||||
santech scan \
|
||||
--image $(params.image) \
|
||||
--delta \
|
||||
--threshold High \
|
||||
--endpoint $(params.endpoint)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5 Policy Import / Export
|
||||
|
||||
### 5.1 Import YAML Policy via CLI
|
||||
|
||||
```bash
|
||||
curl -X POST https://stella.local/api/v1/policy/import \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-F "file=@scan-policy.yaml"
|
||||
```
|
||||
|
||||
### 5.2 Export & Commit to Git
|
||||
|
||||
```bash
|
||||
curl -s -H "Authorization: Bearer $TOKEN" \
|
||||
https://stella.local/api/v1/policy/export \
|
||||
> policy-backup-$(date +%F).yaml
|
||||
git add policy-backup-*.yaml && git commit -m "Policy snapshot"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6 Offline OUK Example
|
||||
|
||||
Inside an **air‑gapped** cluster:
|
||||
|
||||
1. Run `ouk-fetch.sh` from the admin node.
|
||||
2. Load images into the internal registry:
|
||||
|
||||
```bash
|
||||
ctr -n k8s.io images import ./ouk-bundle/*.tar
|
||||
```
|
||||
|
||||
3. Use the same pipeline snippets; DNS points to `registry.git.stella-ops.ru` via local CoreDNS override.
|
||||
|
||||
---
|
||||
|
||||
## 7 Variant D – Enforce Gate in Prod
|
||||
|
||||
```bash
|
||||
# Free tier: 333 scans/day without large delay added
|
||||
santech scan \
|
||||
--image registry.prod.corp/app:${TAG} \
|
||||
--delta \
|
||||
--policy-file prod.rego \
|
||||
--enforce \
|
||||
--endpoint https://stella.prod \
|
||||
|| { echo "Security gate blocked release!"; exit 1; }
|
||||
```
|
||||
|
||||
*`--enforce` turns warnings into non‑zero exit codes.*
|
||||
|
||||
---
|
||||
|
||||
## 8 Cheat‑Sheet (CLI Flags)
|
||||
|
||||
| Flag / Env | Meaning | Default |
|
||||
|----------------------------|---------------------------------------------------|---------|
|
||||
| `--sbom-type` | Force SBOM output format (`trivy‑json-v2` …) | *Auto* |
|
||||
| `--delta` `STELLA_DELTA=1` | Enable layer diff / `/layers/missing` fast path. | Off |
|
||||
| `--policy-file` | Import YAML/Rego before scan. | None |
|
||||
| `--threshold` | Fails scan if sev ≥ threshold. | High |
|
||||
| `--enforce` | Exit non‑zero on policy block. | Off |
|
||||
| `--endpoint` | API base URL. | `http://localhost:8080` |
|
||||
| `--insecure` | Skip TLS verify (test only!). | Off |
|
||||
|
||||
---
|
||||
|
||||
## 9 FAQ
|
||||
|
||||
**Q – I need Syft + SPDX, what changes?**
|
||||
A – Set `--sbom-type spdx-json`; Trivy is bypassed and the scanner plugin selects Syft.
|
||||
|
||||
**Q – Can I run Santech as rootless?**
|
||||
A – Yes; mount the host’s Docker socket via `--userns=keep-id` or use `--context host` with nerdctl.
|
||||
|
||||
**Q – Does Δ‑SBOM work for multi‑arch manifests?**
|
||||
A – Today it only checks `linux/amd64` layers; roadmap item *Q1‑2026* widens support.
|
||||
|
||||
---
|
||||
|
||||
## 10 Change Log
|
||||
|
||||
| Date | Note |
|
||||
|------------|--------------------------------------------------------------|
|
||||
| 2025‑07‑14 | Added internal registry, Δ‑SBOM, multi‑format & policy flows |
|
||||
| 2025‑07‑12 | Initial public recipe set (GitHub, GitLab, Tekton, shell). |
|
||||
|
||||
---
|
||||
147
docs/23_FAQ_MATRIX.md
Normal file
147
docs/23_FAQ_MATRIX.md
Normal file
@ -0,0 +1,147 @@
|
||||
# 23 · FAQ Matrix — Stella Ops
|
||||
|
||||
# FAQ & Support Matrix
|
||||
|
||||
A living list of the questions we get every day, plus a compact matrix of what is **Supported Now**, **In Preview**, and **On the Roadmap (TODO)**.
|
||||
|
||||
---
|
||||
|
||||
## 0 Quick Legend
|
||||
|
||||
| Mark | Meaning |
|
||||
|------|---------|
|
||||
| ✅ | Fully supported in the current release |
|
||||
| 🅿️ | Preview / opt‑in behind a feature flag |
|
||||
| 🛠 | Planned in the **≤ 6 month** roadmap (Feature Matrix “TODO”) |
|
||||
| 🚧 | Longer‑term; 9‑12 month horizon or beyond |
|
||||
|
||||
---
|
||||
|
||||
## 1 General
|
||||
|
||||
| # | Question | Short Answer | Status |
|
||||
|---|----------|-------------|--------|
|
||||
| G‑1 | *Why launch **another** DevSecOps product?* | Existing scanners are either SaaS‑only, slow, or lack offline & Russian language feeds. Stella Ops focuses on *speed (<5 s), modularity, air‑gap friendliness*, and an AGPL code‑base that enterprises can extend in‑house. | ✅ |
|
||||
| G‑2 | *What tech stack?* | Backend **.NET 9** + Redis; runners are OCI images (Trivy, Syft, Grype). UI Angular 17. | ✅ |
|
||||
| G‑3 | *License?* | **AGPL v3** for all core repos; plugins inherit if linked. | ✅ |
|
||||
| G‑4 | *Where do I report bugs?* | Open an issue in `git.stella-ops.ru/stella/core` or ping `#stella-ops` on Matrix. | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 2 Installation & Upgrades
|
||||
|
||||
| # | Question | Answer | Status |
|
||||
|---|----------|--------|--------|
|
||||
| I‑1 | *How do I pull agent images now?* | All official images are in the **anonymous read‑only registry** `registry.git.stella-ops.ru`. No auth token required for pull. | ✅ *(new)* |
|
||||
| I‑2 | *Can I still use GHCR?* | Images remain mirrored for convenience but are not signed; internal registry is the source of truth. | ✅ |
|
||||
| I‑3 | *How to upgrade from ≤ v0.8?* | Re‑generate `docker‑compose.yml` with the bootstrap script; volumes remain intact. Import legacy mute‑rules via `/policy/import`. | ✅ |
|
||||
| I‑4 | *Helm charts?* | K8s Helm chart is under `deploy/helm`; undefaulted (requires `values.yaml`). | 🅿️ |
|
||||
|
||||
---
|
||||
|
||||
## 3 SBOM & Scanning
|
||||
|
||||
|
||||
| # | Question | Short Answer | Status |
|
||||
|---|----------|-------------|--------|
|
||||
|---|----------|-------------|--------|
|
||||
| **S‑1** | *Why exactly **333 scans**?* | Covers p95 workload of SMBs (~290 builds/day) while keeping infra costs <$5/mo per user and nudging larger orgs toward Plus/Pro. | ✅ |
|
||||
| **S‑2** | *How is the limit technically enforced?* | Each `/scan` request carries a **Client‑JWT**. The Quota plug‑in atomically increments `quota:<token>:<date>` in Redis. Soft (5 s) and hard (60 s) wait‑walls ensure fair use. | ✅ |
|
||||
| **S‑3** | *What if my site is fully offline?* | Every **OUK tarball** contains a fresh Client‑JWT valid **30 days**. Uploading the OUK refreshes the token automatically; no Internet required. | ✅ |
|
||||
| S‑4 | *Can I pool multiple tokens?* | Yes, but each token has its own 333/day budget. Use distinct tokens per CI line if you need more throughput. | ✅ |
|
||||
| S‑5 | *Does quota enforcement affect performance?* | No. Legitimate scans still complete in < 5 s; blocked scans incur only their specified wait‑wall. | ✅ |
|
||||
| S‑6 | *Which SBOM formats does Stella emit?* | Built‑in: **`trivy-json-v2`**, **`spdx-json`**, **`cyclonedx-json`**. | ✅ |
|
||||
| S‑7 | *What is Δ‑SBOM and how fast is it?* | Uploads only new layers; P95 ≤ 1 s on cached bases. | ✅ |
|
||||
| S‑8 | *Windows container scanning?* | Runner binaries compile on Windows, but layer‑unpack path is unoptimised; full support 🚧. | 🚧 |
|
||||
|
||||
---
|
||||
|
||||
## 4 Policy‑as‑Code
|
||||
|
||||
| # | Question | Answer | Status |
|
||||
|---|----------|--------|--------|
|
||||
| P‑1 | *How are mutes & blocks stored now?* | Default: **YAML** (`scan-policy.yaml`) in Mongo (versioned). Import / export via `/policy/{import,export}` or Settings → Policies. | ✅ |
|
||||
| P‑2 | *Why YAML over OPA?* | YAML lowers entry barrier; advanced users may embed **Rego** snippets. First‑class Rego evaluation is 🛠. | 🛠 |
|
||||
| P‑3 | *CLI enforcement?* | Pass `--policy-file path` plus `--enforce` to fail builds on violations. Exit‑code reflects policy gate. | ✅ |
|
||||
| P‑4 | *Audit history?* | Every policy change writes an immutable record (`audit_policies` collection) and appears in UI *History* tab. | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 5 Registry & Offline Use
|
||||
|
||||
| # | Question | Answer | Status |
|
||||
|---|----------|--------|--------|
|
||||
| R‑1 | *Is the internal registry mandatory?* | No, but recommended for sovereignty & signature verification (`cosign verify`). | ✅ |
|
||||
| R‑2 | *How to mirror for OUK?* | `oras pull registry.git.stella-ops.ru/library/* --output ./ouk-bundle` → import on the target via `ctr images import`. | ✅ |
|
||||
| R‑3 | *Does the backend fetch external feeds?* | Only when `--feeds.auto=1`; OUK installs run fully offline with NVD packed in the tarball. | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 6 Performance
|
||||
|
||||
| # | Scenario | Target | Achieved (July 2025) |
|
||||
|---|----------|--------|----------------------|
|
||||
| Local SBOM scan (`alpine`) | **≤ 5 s** | 4.2 s P95 |
|
||||
| Δ‑SBOM warm base | **≤ 1 s** | 0.8 s P95 |
|
||||
| Image unpack (200 MB) | **≤ 10 s** | 8.6 s P95 |
|
||||
|
||||
*Numbers measured on 4 vCPU / 8 GB Ubuntu 22.04 runner.*
|
||||
|
||||
---
|
||||
|
||||
## 7 Security & Compliance
|
||||
|
||||
| # | Question | Answer | Status |
|
||||
|---|----------|--------|--------|
|
||||
| C‑1 | *How are images signed?* | Cosign signatures pushed alongside each tag (`*.sig`). Santech verifies on pull. | ✅ |
|
||||
| C‑2 | *Supply‑chain attestation (SLSA)?* | SLSA‑gen at build time and verification in runner is 🛠 (≤ 6 months). | 🛠 |
|
||||
| C‑3 | *Rekor transparency log?* | Local Rekor mirror for offline installs is 🚧 (9‑12 months). | 🚧 |
|
||||
| C‑4 | *TLS ciphers?* | Default OpenSSL suites; plug‑in allows GOST/SM (via `ITlsProvider`). | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 8 Road‑map / Future Features
|
||||
|
||||
| Area | Feature | ETA | Notes |
|
||||
|------|---------|-----|-------|
|
||||
| UI | Modular route plug‑ins | Q1‑2026 | Dynamic Angular module loader |
|
||||
| SBOM | Multi‑arch Δ‑SBOM | Q1‑2026 | Layer digest per arch |
|
||||
| Policy | Rego native engine | Q1‑2026 | `opa eval` in‑proc |
|
||||
| Supply chain | SLSA provenance | Q1‑2026 | Level 3 target |
|
||||
| Integrity | Rekor mirror | Q2‑2026 | Air‑gap friendly |
|
||||
| Ecosystem | Community plugin market | Q2‑2026 | Curated index in UI |
|
||||
| Scale | Redis Cluster auto‑shard | Q3‑2026 | Transparent fail‑over |
|
||||
|
||||
---
|
||||
|
||||
## 9 Troubleshooting
|
||||
|
||||
| Symptom | Likely Cause | Fix |
|
||||
|---------|--------------|-----|
|
||||
| **`ER_BAD_SV` error on scan** | SBOM format flag mismatch | Set correct `--sbom-type` or let auto‑detect. |
|
||||
| Δ‑SBOM still uploads full SBOM | Cache cold or digest mismatch | Check `docker history` shows reused layers; bump builder version. |
|
||||
| “Policy file invalid” | YAML schema error | Run `/policy/validate` endpoint; lint with VS Code schema. |
|
||||
| Pull fails with 401 | Corporate proxy intercepts registry | Mirror to on‑premise Harbor; set `--registry` flag. |
|
||||
|
||||
---
|
||||
|
||||
## 10 Licensing & Community
|
||||
|
||||
| # | Question | Answer |
|
||||
|---|----------|--------|
|
||||
| L‑1 | *Can I build a commercial fork?* | AGPL allows commercial services but derivatives must remain AGPL if distributed. |
|
||||
| L‑2 | *Commercial support?* | Community only today; paid support partners in discussion. |
|
||||
| L‑3 | *How to contribute a plugin?* | Fork → implement DI contract (`IScannerRunner`, etc.) → PR + ADR. |
|
||||
|
||||
---
|
||||
|
||||
## 11 Change Log
|
||||
|
||||
| Date | Highlights |
|
||||
|------------|------------|
|
||||
| 2025‑07‑14 | Added internal registry, multi‑format SBOM, Δ‑SBOM, Policy‑as‑Code, updated roadmap (SLSA/Rekor) |
|
||||
| 2025‑06‑30 | Initial public FAQ matrix |
|
||||
|
||||
---
|
||||
|
||||
*(End of FAQ Matrix v2.0)*
|
||||
196
docs/24_OUK_ADMIN_GUIDE.md
Normal file
196
docs/24_OUK_ADMIN_GUIDE.md
Normal file
@ -0,0 +1,196 @@
|
||||
# 24 · Offline Update Kit (OUK) Administration Guide — Stella Ops
|
||||
|
||||
|
||||
# Offline Update Kit (OUK) — Administrator Guide
|
||||
|
||||
Stella Ops is designed for *offline‑first* deployments—e.g. regulated datacentres that can’t call out to GitHub or NVD.
|
||||
The **Offline Update Kit (OUK)** bundles *all* mutable security data (Trivy DB, NVD JSON, OSV feeds, UI locales, policy bundles) into a signed, single‑file archive so you can refresh an isolated cluster in minutes.
|
||||
|
||||
---
|
||||
|
||||
## 0 Quick Start (TL;DR)
|
||||
|
||||
1. **Download** the latest release asset
|
||||
|
||||
```bash
|
||||
curl -L -O https://git.stella-ops.ru/stella/ops/-/releases/latest/download/ouk-20250715.tar.gz
|
||||
curl -L -O https://git.stella-ops.ru/stella/ops/-/releases/latest/download/ouk-20250715.tar.gz.sig
|
||||
```
|
||||
|
||||
2. **Verify** the Cosign signature
|
||||
|
||||
```bash
|
||||
cosign verify-blob \
|
||||
--key cosign.pub \
|
||||
--signature ouk-20250715.tar.gz.sig \
|
||||
ouk-20250715.tar.gz
|
||||
```
|
||||
|
||||
3. **JWT Refresh** – the tarball already contains a **`client.jwt`** file valid
|
||||
for **30 days**. Importing the OUK automatically refreshes the token used for
|
||||
quota accounting in offline mode.
|
||||
|
||||
4. **UI → Settings → Offline Updates → Upload** the tarball
|
||||
5. Wait for the toast: **“Feeds updated · Trivy DB 2025‑07‑14 · NVD 2025‑07‑15”**
|
||||
6. Sub‑5 s scans now use the fresh data—done!
|
||||
|
||||
---
|
||||
|
||||
## 1 What the OUK Contains
|
||||
|
||||
| Component | Typical size |
|
||||
|----------------------------------------|--------------|
|
||||
| Trivy vulnerability DB (`vuln.db.tgz`) | 50 MB |
|
||||
| NVD JSON feed (all years) | 1.4 GB |
|
||||
| OSV JSON snapshots | 500 MB |
|
||||
| BDU XML feed (🇷🇺 optional) | 25 MB |
|
||||
| UI locale bundles | 100 kB |
|
||||
| Policy pack snapshots | 5 kB |
|
||||
| SBOM of the kit (`manifest.spdx.json`) | <1 MB |
|
||||
| Cosign signature (`*.sig`) | 1 kB |
|
||||
| Client‑JWT (`client.jwt`) 30 d | 2 kB |
|
||||
|
||||
All files are **content‑addressed**; the SBOM references each SHA‑256 so auditors can trace provenance.
|
||||
|
||||
---
|
||||
|
||||
## 2 Security Model
|
||||
|
||||
* Every tarball is signed with the **Stella Ops project key** (`cosign.pub`, fingerprint `AB12 34CD 56EF…`).
|
||||
* The backend refuses any archive that fails signature or digest checks.
|
||||
* The embedded SBOM gives auditors full transparency into exact feed versions.
|
||||
|
||||
---
|
||||
|
||||
## 3 Obtaining an OUK Tarball
|
||||
|
||||
### 3.1 Official Builds
|
||||
|
||||
* Produced automatically on the **first Friday of every month** by the Release pipeline (see *13_RELEASE_ENGINEERING_PLAYBOOK.md*).
|
||||
* Stored under **GitLab → Releases → Assets** with the filename pattern `ouk-YYYYMMDD.tar.gz`.
|
||||
* Example download via curl:
|
||||
|
||||
```bash
|
||||
curl -L -O https://git.stella-ops.ru/stella/ops/-/releases/v2.3/downloads/ouk-20250715.tar.gz
|
||||
curl -L -O https://git.stella-ops.ru/stella/ops/-/releases/v2.3/downloads/ouk-20250715.tar.gz.sig
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4 Uploading Via UI
|
||||
|
||||
1. Log in as **system : admin** (role = Owner).
|
||||
2. Navigate **Settings → Offline Updates**.
|
||||
3. Press **Upload** and pick `ouk-YYYYMMDD.tar.gz`.
|
||||
4. Backend workflow
|
||||
1. Streams the file to `/tmp`.
|
||||
2. Runs `cosign verify-blob`.
|
||||
3. Extracts content into `var/lib/stella/db/` (read‑only).
|
||||
4. Flushes Redis cache.
|
||||
> After Cosign verification the backend copies **`client.jwt`** into
|
||||
> `/var/lib/stella/tokens/`, replacing the previous offline token and resetting
|
||||
> its expiry countdown.
|
||||
|
||||
5. Audit log emits `ouk_update` event (actor, datetime, SHA‑256).
|
||||
|
||||
The UI toast now shows
|
||||
> “Feeds updated + Offline Token refreshed (valid till *17 Aug 2025*)”.
|
||||
|
||||
---
|
||||
|
||||
## 5 Uploading Via CLI (REST API)
|
||||
|
||||
No custom tool is required—`curl` does the job.
|
||||
|
||||
```bash
|
||||
ACCESS_TOKEN=$(cat token.txt) # bearer from OpenIddict
|
||||
SIGNATURE_B64=$(base64 -w0 ouk-20250715.tar.gz.sig)
|
||||
|
||||
curl -X POST https://stella.local/api/admin/ouk \
|
||||
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
|
||||
-H "X-Signature: ${SIGNATURE_B64}" \
|
||||
--data-binary @ouk-20250715.tar.gz
|
||||
```
|
||||
|
||||
*HTTP 201* is success; any non‑2xx code means verification or extraction failed.
|
||||
|
||||
---
|
||||
|
||||
## 6 Automated Sync (Advanced)
|
||||
|
||||
When outbound Internet is *allowed* but inbound isn’t:
|
||||
|
||||
1. Deploy **StellaOps.Registry** (internal Docker registry) on a host that can reach the Internet.
|
||||
2. Cron job pulls the latest OUK with curl, verifies it, stores in an internal S3 bucket.
|
||||
3. Offline cluster fetches nightly via **MinIO Gateway** URL:
|
||||
|
||||
```bash
|
||||
curl -L -O https://minio.infra/offline/ouk-latest.tar.gz
|
||||
curl -L -O https://minio.infra/offline/ouk-latest.tar.gz.sig
|
||||
# repeat verification and upload steps
|
||||
```
|
||||
|
||||
Remember to propagate **`client.jwt`** together with the feeds if you split the
|
||||
tarball into multiple artefacts; otherwise the offline cluster may hit
|
||||
*“Token expired”* errors.
|
||||
|
||||
---
|
||||
|
||||
## 7 Rollback Strategy
|
||||
|
||||
* The backend keeps the **last three OUK snapshots** in `/var/lib/stella/ouk-backup/`.
|
||||
* UI → Offline Updates → **Restore** next to the desired date.
|
||||
* Operation swaps a symlink—≈ 3 s, no service restart.
|
||||
|
||||
---
|
||||
|
||||
## 8 Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
|--------------------------------------|----------------------|----------------------------------------------|
|
||||
| “Signature verification failed” | Wrong key/tampered | Re‑download; compare key fingerprints |
|
||||
| “Feed age tile stays red” | Redis cache stale | `stella admin redis flushall` |
|
||||
| `io/fs: read‑only file system` | Volume mounted RO | Remount RW or move OUK path |
|
||||
| NVD import > 10 min | Slow disk | Move DB path to SSD/LVM cache |
|
||||
| *“Token expired”* banner after OUK upload | `client.jwt` missing or out‑of‑date | Re‑run `ouk fetch`; ensure tarball contains fresh token. |
|
||||
|
||||
---
|
||||
|
||||
## 9 Non‑Commercial Usage Rules (English)
|
||||
|
||||
1. **Free for internal assessments** by individuals or organisations.
|
||||
2. **SaaS resale/re‑hosting is forbidden** without explicit written consent (AGPL §13).
|
||||
3. If you redistribute a modified build you **must** publish complete source under the same licence and keep Stella Ops attribution.
|
||||
4. Third‑party dependencies remain under their respective licences (MIT, Apache‑2.0, ISC, BSD).
|
||||
5. Deployments in state‑regulated or classified environments must follow **ФЗ‑187** and export‑control law.
|
||||
|
||||
---
|
||||
|
||||
## 10 Best Practices Snapshot
|
||||
|
||||
* Rotate OUK uploads at least **monthly**; weekly during high CVE activity.
|
||||
* Store OUK assets in an **immutable bucket** (S3 WORM, Artifactory release) to thwart tampering.
|
||||
* Keep `cosign.pub` on an offline root‑of‑trust USB; rotate keys yearly.
|
||||
* During audits, export `manifest.spdx.json` as evidence of feed provenance.
|
||||
|
||||
---
|
||||
|
||||
## 11 FAQ
|
||||
|
||||
**Q:** *Does OUK update Docker base images?*
|
||||
**A:** No—OUK is for **data** only. Refresh base images via internal registry or CI pipelines.
|
||||
|
||||
**Q:** *Why not bundle Grype DB too?*
|
||||
**A:** Grype consumes Trivy DB format directly; duplication saved 500 MB.
|
||||
|
||||
---
|
||||
|
||||
## 12 Change Log
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---------|------|-------|
|
||||
| v1.2 | 2025‑07‑16 | Switched CLI to curl, removed DIY build, dropped “Location” column, clarified download example. |
|
||||
| v1.1 | 2025‑07‑15 | Added UI upload flow, MinIO sync, usage rules, Cosign commands. |
|
||||
| v1.0 | 2025‑07‑09 | Initial OUK admin guide |
|
||||
|
||||
*(End of Offline Update Kit Administration Guide v1.0)*
|
||||
129
docs/29_LEGAL_FAQ_QUOTA.md
Normal file
129
docs/29_LEGAL_FAQ_QUOTA.md
Normal file
@ -0,0 +1,129 @@
|
||||
# 29 · Legal FAQ — *333 Scan/Day Free‑Tier Quota*
|
||||
|
||||
> **Status:** Published – applies to all open‑source (AGPL‑3.0‑or‑later) builds
|
||||
> **Audience:** project contributors, downstream packagers, OSS users, lawyers
|
||||
> **Scope:** clarifies how the daily‑quota mechanism interacts with copyright
|
||||
> licences, fair‑use, forking rights and the AGPL network‑use clause.
|
||||
> **Not a contract:** this FAQ is **informational** and does not replace the
|
||||
> actual licence text (GNU AGPL‑3.0) shipped with every source tarball.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Is the *333 scans per UTC day* limit a **licence** restriction?
|
||||
|
||||
**No.**
|
||||
The quota is an **operational usage limit** enforced by runtime code
|
||||
(`StellaOps.Quota` plug‑in).
|
||||
The source code itself remains available under the
|
||||
[GNU Affero General Public Licence v3](https://www.gnu.org/licenses/agpl‑3.0.html).
|
||||
Any person may:
|
||||
|
||||
1. **Receive the complete corresponding source** (AGPL § 6).
|
||||
2. **Modify** or **remove** the quota logic, and
|
||||
3. **Run** those modified versions, **even over a network** (AGPL § 13),
|
||||
|
||||
provided they also comply with the licence duties (e.g. **offer**
|
||||
their own modified source to users who interact with the modified
|
||||
program over a network).
|
||||
|
||||
---
|
||||
|
||||
## 2 · Why doesn’t the quota violate AGPL § 0 or § 13?
|
||||
|
||||
* **§ 0** (Freedom to run the program) – You can run unmodified or
|
||||
modified copies of Stella Ops for any purpose and in any quantity
|
||||
inside your own infrastructure. The quota only limits *how many scan
|
||||
operations the official binaries will accept*; it does **not** forbid
|
||||
you from creating binaries without that limit.
|
||||
|
||||
* **§ 13** (Remote‑network interaction) – If you provide Stella Ops as a
|
||||
service and keep the quota, users interact with an unmodified AGPL
|
||||
work, so no extra obligations arise.
|
||||
If you **remove** or **change** the quota, that counts as a
|
||||
modification; under § 13 you must **make your modified source code
|
||||
available** to those remote users.
|
||||
|
||||
---
|
||||
|
||||
## 3 · May I **fork** the project and delete the quota altogether?
|
||||
|
||||
Yes – the AGPL expressly permits this.
|
||||
However, three practical notes apply:
|
||||
|
||||
| Aspect | Obligation / Consequence |
|
||||
|--------|-------------------------|
|
||||
| **Source availability** | You *must* provide the full modified source (including build scripts) under AGPL‑compatible terms. |
|
||||
| **Trademarks** | The name **“Stella Ops”**, the star‑logo and the “333” banner are trademarked. Remove them or obtain prior written consent before redistributing a modified build. |
|
||||
| **Update Kit tokens** | Official Offline‑Update‑Kit (OUK) tar‑balls embed signed Client‑ID JWTs that expect the quota code‑path. If you strip that path, you will also need to maintain your own OUK feed or rebuild the tar‑balls. |
|
||||
|
||||
---
|
||||
|
||||
## 4 · What about **educational labs** or “Hack Week” events that need more than 333 scans?
|
||||
|
||||
You have three options:
|
||||
|
||||
1. **Request a short‑lived *Workshop Token*** – project maintainers can
|
||||
issue a signed JWT that temporarily raises the limit (subject to
|
||||
availability).
|
||||
2. **Spin up additional tokens** – each API token carries its own 333
|
||||
scans/day budget; e.g. a class of 40 students can share ten tokens.
|
||||
3. **Compile without quota** – ideal for private, non‑public courses;
|
||||
remember to keep the modified source accessible to participants.
|
||||
|
||||
None of these options require a commercial licence fee.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Does the quota create a **field‑of‑use restriction** (GPL FAQ concern)?
|
||||
|
||||
No. The limit is *quantitative* (how many scans per unit of time), not a
|
||||
prohibition on *where* or *for what* purpose you run the software.
|
||||
Therefore it does **not** impose an additional field‑of‑use restriction
|
||||
and remains compatible with AGPL § 7.
|
||||
|
||||
---
|
||||
|
||||
## 6 · How does the **Offline Update Kit** interact with licence tokens?
|
||||
|
||||
* Every OUK tar‑ball ships a **signed Client‑ID JWT** with 30‑day
|
||||
validity.
|
||||
* The token is stored at `/keys/client.jwt` on the backend container and
|
||||
refreshed on each OUK upload.
|
||||
* If you fork the codebase and remove the quota, you must either
|
||||
(a) ignore the token entirely, or
|
||||
(b) generate and sign your own tokens – the signing key is **not**
|
||||
included in the public repo for security reasons.
|
||||
|
||||
---
|
||||
|
||||
## 7 · Can forks override the limit?
|
||||
|
||||
Yes. Forks could provide Client‑ID tokens with `maxScansPerDay = 0`
|
||||
(unlimited). The same runtime code path is used; no proprietary
|
||||
mechanism exists inside the open‑source core.
|
||||
|
||||
---
|
||||
|
||||
## 8 · Why 333 – not 100, 500 or 1 000?
|
||||
|
||||
* 333 ≈ the 95th‑percentile daily build count of **small/medium‑sized
|
||||
self‑hosted GitLab instances** we surveyed in 2024.
|
||||
* It leaves *head‑room* for burst‑y days while nudging larger teams
|
||||
towards Plus/Pro – the main revenue driver that funds continued
|
||||
development of the open‑source core.
|
||||
|
||||
See *02_WHY.md § 5* for the full product‑management rationale.
|
||||
|
||||
---
|
||||
|
||||
## 9 · Who can I contact for **licensing support**?
|
||||
|
||||
* E‑mail: **legal@stella‑ops.org**
|
||||
|
||||
---
|
||||
|
||||
## 10 · Change Log
|
||||
|
||||
| Version | Date | Comment |
|
||||
|---------|------|---------|
|
||||
| **v1.0** | 18 Jul 2025 | First public revision – aligns with quota rev 2.0. |
|
||||
186
docs/30_QUOTA_ENFORCEMENT_FLOW1.md
Normal file
186
docs/30_QUOTA_ENFORCEMENT_FLOW1.md
Normal file
@ -0,0 +1,186 @@
|
||||
# 26 · Quota‑Enforcement Flow (333 Scans/Day)
|
||||
|
||||
> **Audience:** backend contributors, UI devs, CI‑plugin authors, third‑party
|
||||
> tool builders
|
||||
> **Version:** 1.0 (18 Jul 2025) — aligns with Quota rev 2.0
|
||||
> **Scope:** runtime data‑flow, key formats, timing budget, error model
|
||||
|
||||
---
|
||||
|
||||
## 0 Why another doc?
|
||||
|
||||
* **SRS** lists the *requirements* (FR‑4, FR‑4a…).
|
||||
* **Module Specs** describe the *code surface* (`IQuotaService`, `QuotaVerdict`).
|
||||
* **This file** stitches everything together so future maintainers can reason
|
||||
about race conditions, UX timing and API behaviour in one place.
|
||||
|
||||
---
|
||||
|
||||
## 1 High‑Level Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant CLI as SanTech CLI
|
||||
participant API as API Gateway
|
||||
participant QU as Quota Service
|
||||
participant RD as Redis
|
||||
participant SCN as Scan Runner
|
||||
|
||||
CLI->>API: POST /scan
|
||||
API->>QU: CheckAsync(token)
|
||||
QU->>RD: LUA INCR quota:<token>
|
||||
alt Remaining > 0
|
||||
RD-->>QU: counter + reset ts
|
||||
QU-->>API: verdict {IsAllowed=true}
|
||||
API->>SCN: enqueue scan
|
||||
SCN-->>API: result
|
||||
API-->>CLI: 200 OK (headers: remaining, reset)
|
||||
else Remaining == 0 → soft wall
|
||||
RD-->>QU: counter==limit
|
||||
QU-->>API: verdict {RetryAfter=5 s}
|
||||
API-->>CLI: 429 Too Many (Retry‑After: 5)
|
||||
else Hard Wall (≥ limit+30)
|
||||
RD-->>QU: counter≥limit+30
|
||||
QU-->>API: verdict {RetryAfter=60 s}
|
||||
API-->>CLI: 429 Too Many (Retry‑After: 60)
|
||||
end
|
||||
```
|
||||
|
||||
|
||||
* **Soft wait‑wall** applies to the *first 30* blocked scans.
|
||||
* **Hard wait‑wall** (60 s) kicks in afterwards until the counter resets at
|
||||
**00:00 UTC**.
|
||||
|
||||
---
|
||||
|
||||
## 2 Key Redis Structures
|
||||
|
||||
| Key pattern | Type | TTL | Purpose |
|
||||
|-------------|------|-----|---------|
|
||||
| `quota:<token>:<yyyy‑mm‑dd>` | *string* | *expires at UTC midnight* | Counter starting at `1` |
|
||||
|
||||
### Typical value progression
|
||||
|
||||
```json
|
||||
{
|
||||
"quota:abc123:2025-07-18": "198",
|
||||
"quota:abc123:2025-07-18-RESET": "2025-07-19T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3 QuotaVerdict DTO
|
||||
|
||||
```csharp
|
||||
public readonly record struct QuotaVerdict(
|
||||
bool IsAllowed,
|
||||
int Remaining,
|
||||
DateTimeOffset ResetUtc,
|
||||
TimeSpan RetryAfter)
|
||||
{
|
||||
public static QuotaVerdict Allowed(int remaining, DateTimeOffset resetUtc)
|
||||
=> new(true, remaining, resetUtc, TimeSpan.Zero);
|
||||
|
||||
public static QuotaVerdict Blocked(int remaining, DateTimeOffset resetUtc,
|
||||
TimeSpan retryAfter)
|
||||
=> new(false, remaining, resetUtc, retryAfter);
|
||||
}
|
||||
```
|
||||
|
||||
* **IsAllowed** `false` ⇒ API must return `429`
|
||||
* **RetryAfter** `> 0` ⇒ populate `Retry‑After` header (seconds)
|
||||
|
||||
---
|
||||
|
||||
## 4 HTTP Headers & Error Model
|
||||
|
||||
| Header | Example | Meaning |
|
||||
|--------|---------|---------|
|
||||
| `X‑Stella‑Quota‑Remaining` | `117` | Scans left for the token today |
|
||||
| `X‑Stella‑Reset` | `2025‑07‑18T23:59:59Z` | When the counter resets |
|
||||
| `Retry‑After` | `5` or `60` | Seconds the client should back‑off |
|
||||
|
||||
Error body (RFC 7807):
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "https://stella-ops.org/probs/quota",
|
||||
"title": "Daily quota exceeded",
|
||||
"status": 429,
|
||||
"detail": "This token exhausted its 333 scans for 2025‑07‑18.",
|
||||
"quota": {
|
||||
"remaining": 0,
|
||||
"reset": "2025‑07‑18T23:59:59Z"
|
||||
},
|
||||
"traceId": "00-1db39d4f3ab16fa1"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5 UI Behaviour
|
||||
|
||||
| Threshold | Visual cue |
|
||||
|-----------|------------|
|
||||
| **≤ 133 scans remaining** | Yellow banner “*60 % of free tier used*” |
|
||||
| **0 remaining** (soft) | Red toast “*Back‑off 5 s*”; spinner shows countdown |
|
||||
| **0 remaining** (hard) | Modal blocks *New Scan* button; shows **HH:MM:SS** until reset |
|
||||
|
||||
*WebSocket `quota/events` pushes live counters to avoid polling.*
|
||||
|
||||
---
|
||||
|
||||
## 6 Timing Budget
|
||||
|
||||
| Operation | Budget (P95) | Impl detail |
|
||||
|-----------|--------------|-------------|
|
||||
| Redis LUA INCR | ≤ 0.8 ms | SHA script cached |
|
||||
| Verdict calc | ≤ 0.2 ms | branchless math |
|
||||
| Header injection | ≤ 0.1 ms | middleware sets `OnStarting` callback |
|
||||
| Soft delay | **exactly 5 000 ms** ± 50 ms | `Task.Delay` in controller |
|
||||
| Hard delay | **exactly 60 000 ms** ± 100 ms | same |
|
||||
|
||||
---
|
||||
|
||||
## 7 CLI Auto‑Retry Logic (SanTech v0.8+)
|
||||
|
||||
```csharp
|
||||
var resp = await http.PostAsync("/scan", payload, ct);
|
||||
if (resp.StatusCode == (HttpStatusCode)429 &&
|
||||
autoRetry && retryCount < 1)
|
||||
{
|
||||
var after = int.Parse(resp.Headers.RetryAfter.Delta!.Value.TotalSeconds);
|
||||
Console.Error.WriteLine($"Quota hit – waiting {after}s…");
|
||||
await Task.Delay(TimeSpan.FromSeconds(after), ct);
|
||||
retryCount++;
|
||||
goto Retry;
|
||||
}
|
||||
```
|
||||
|
||||
*Retrieves `Retry‑After`; retries **once** if ≤ 60 s.*
|
||||
|
||||
---
|
||||
|
||||
## 8 Edge Cases & Tests
|
||||
|
||||
| Case | Expected behaviour | Covered by |
|
||||
|------|--------------------|-----------|
|
||||
| Redis restart (counter key lost) | API treats missing key as `0`; safe default | IT‑QUOTA‑REDIS |
|
||||
| Two parallel scans exceed limit simultaneously | Only one passes; second gets `429` | RaceCondTests.cs |
|
||||
| Token with custom limit = 100 | Soft wall at ≥ 100; headers reflect custom limit | QuotaCustomLimitTests.cs |
|
||||
|
||||
---
|
||||
|
||||
## 9 Migration Notes (v0.7 → v0.8)
|
||||
|
||||
* Old key `quota:<token>` (no date) **deprecated** — migration script copies
|
||||
value to the new *dated* key and sets TTL to next UTC midnight.
|
||||
|
||||
---
|
||||
|
||||
## 10 Change Log
|
||||
|
||||
| Ver | Date | Note |
|
||||
|-----|------|------|
|
||||
| 1.0 | 18 Jul 2025 | Initial end‑to‑end flow doc (Quota rev 2.0). |
|
||||
Reference in New Issue
Block a user