Initial commit

docs/19_TEST_SUITE_OVERVIEW.md

@@ -0,0 +1,91 @@
+# Automated Test‑Suite Overview
+
+This document enumerates **every automated check** executed by the Stella Ops
+CI pipeline, from unit level to chaos experiments.  It is intended for
+contributors who need to extend coverage or diagnose failures.
+
+> **Build parameters** – values such as `{{ dotnet }}` (runtime) and
+> `{{ angular }}` (UI framework) are injected at build time.
+
+---
+
+## Layer map
+
+| Layer | Tooling | Entry‑point | Frequency |
+|-------|---------|-------------|-----------|
+| **1. Unit** | `xUnit` (<code>dotnet test</code>) | `*.Tests.csproj` | per PR / push |
+| **2. Property‑based** | `FsCheck` | `SbomPropertyTests` | per PR |
+| **3. Integration (API)** | `Testcontainers` suite | `test/Api.Integration` | per PR + nightly |
+| **4. Integration (DB‑merge)** | in‑memory Mongo + Redis | `FeedMerge.Integration` | per PR |
+| **5. Contract (gRPC)** | `Buf breaking` | `buf.yaml` files | per PR |
+| **6. Front‑end unit** | `Jest` | `ui/src/**/*.spec.ts` | per PR |
+| **7. Front‑end E2E** | `Playwright` | `ui/e2e/**` | nightly |
+| **8. Lighthouse perf / a11y** | `lighthouse-ci` (Chrome headless) | `ui/dist/index.html` | nightly |
+| **9. Load** | `k6` scripted scenarios | `k6/*.js` | nightly |
+| **10. Chaos CPU / OOM** | `pumba` | Docker Compose overlay | weekly |
+| **11. Dependency scanning** | `Trivy fs` + `dotnet list package --vuln` | root | per PR |
+| **12. License compliance** | `LicenceFinder` | root | per PR |
+| **13. SBOM reproducibility** | `in‑toto attestation` diff | GitLab job | release tags |
+
+---
+
+## Quality gates
+
+| Metric | Budget | Gate |
+|--------|--------|------|
+| API unit coverage | ≥ 85 % lines | PR merge |
+| API response P95 | ≤ 120 ms | nightly alert |
+| Δ‑SBOM warm scan P95 (4 vCPU) | ≤ 5 s | nightly alert |
+| Lighthouse performance score | ≥ 90 | nightly alert |
+| Lighthouse accessibility score | ≥ 95 | nightly alert |
+| k6 sustained RPS drop | &lt; 5 % vs baseline | nightly alert |
+
+---
+
+## Local runner
+
+```bash
+# minimal run: unit + property + frontend tests
+./scripts/dev-test.sh
+
+# full stack incl. Playwright and lighthouse
+./scripts/dev-test.sh --full
+````
+
+The script spins up MongoDB/Redis via Testcontainers and requires:
+
+* Docker ≥ 25
+* Node 20 (for Jest/Playwright)
+
+---
+
+## CI job layout
+
+```mermaid
+flowchart LR
+  subgraph fast-path
+    U[xUnit] --> P[FsCheck] --> I1[Testcontainer API]
+  end
+
+  I1 --> FE[Jest]
+  FE --> E2E[Playwright]
+  E2E --> Lighthouse
+  Lighthouse --> INTEG2[FeedMerge]
+  INTEG2 --> LOAD[k6]
+  LOAD --> CHAOS[pumba]
+  CHAOS --> RELEASE[Attestation diff]
+```
+
+---
+
+## Adding a new test layer
+
+1. Extend `scripts/dev-test.sh` so local contributors get the layer by default.
+2. Add a dedicated GitLab job in `.gitlab-ci.yml` (stage `test` or `nightly`).
+3. Register the job in `docs/19_TEST_SUITE_OVERVIEW.md` *and* list its metric
+   in `docs/metrics/README.md`.
+
+---
+
+*Last updated {{ "now" | date: "%Y‑%m‑%d" }}*
+