Automated Test‑Suite Overview

This document enumerates every automated check executed by the Stella Ops CI pipeline, from unit level to chaos experiments. It is intended for contributors who need to extend coverage or diagnose failures.

Build parameters – values such as {{ dotnet }} (runtime) and {{ angular }} (UI framework) are injected at build time.

Layer map

Layer	Tooling	Entry‑point	Frequency
1. Unit	`xUnit` (`dotnet test`)	`*.Tests.csproj`	per PR / push
2. Property‑based	`FsCheck`	`SbomPropertyTests`	per PR
3. Integration (API)	`Testcontainers` suite	`test/Api.Integration`	per PR + nightly
4. Integration (DB‑merge)	in‑memory Mongo + Redis	`FeedMerge.Integration`	per PR
5. Contract (gRPC)	`Buf breaking`	`buf.yaml` files	per PR
6. Front‑end unit	`Jest`	`ui/src/*/.spec.ts`	per PR
7. Front‑end E2E	`Playwright`	`ui/e2e/**`	nightly
8. Lighthouse perf / a11y	`lighthouse-ci` (Chrome headless)	`ui/dist/index.html`	nightly
9. Load	`k6` scripted scenarios	`k6/*.js`	nightly
10. Chaos CPU / OOM	`pumba`	Docker Compose overlay	weekly
11. Dependency scanning	`Trivy fs` + `dotnet list package --vuln`	root	per PR
12. License compliance	`LicenceFinder`	root	per PR
13. SBOM reproducibility	`in‑toto attestation` diff	GitLab job	release tags

Quality gates

Metric	Budget	Gate
API unit coverage	≥ 85 % lines	PR merge
API response P95	≤ 120 ms	nightly alert
Δ‑SBOM warm scan P95 (4 vCPU)	≤ 5 s	nightly alert
Lighthouse performance score	≥ 90	nightly alert
Lighthouse accessibility score	≥ 95	nightly alert
k6 sustained RPS drop	< 5 % vs baseline	nightly alert

Local runner

# minimal run: unit + property + frontend tests
./scripts/dev-test.sh

# full stack incl. Playwright and lighthouse
./scripts/dev-test.sh --full

The script spins up MongoDB/Redis via Testcontainers and requires:

Docker ≥ 25
Node 20 (for Jest/Playwright)

CI job layout

flowchart LR
  subgraph fast-path
    U[xUnit] --> P[FsCheck] --> I1[Testcontainer API]
  end

  I1 --> FE[Jest]
  FE --> E2E[Playwright]
  E2E --> Lighthouse
  Lighthouse --> INTEG2[FeedMerge]
  INTEG2 --> LOAD[k6]
  LOAD --> CHAOS[pumba]
  CHAOS --> RELEASE[Attestation diff]

Adding a new test layer

Extend scripts/dev-test.sh so local contributors get the layer by default.
Add a dedicated GitLab job in .gitlab-ci.yml (stage test or nightly).
Register the job in docs/19_TEST_SUITE_OVERVIEW.md and list its metric in docs/metrics/README.md.

Last updated {{ "now" | date: "%Y‑%m‑%d" }}

3.0 KiB Executable File Raw Blame History Unescape Escape