# VEX Lens CI + Load/Obs Plan (DEVOPS-VEX-30-001)

Scope: CI jobs, load/perf tests, dashboards, and alerts for VEX Lens API and Issuer Directory.
Assumptions: offline-friendly mirrors available; VEX Lens uses Mongo + Redis; Issuer Directory uses Mongo + OIDC.

## CI Jobs (Gitea workflow template)
- `build-vex`: dotnet restore/build for `src/VexLens/StellaOps.VexLens`, cache `local-nugets/`, set `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1`.
- `test-vex`: `dotnet test` VexLens and Issuer Directory tests with `DOTNET_DISABLE_BUILTIN_GRAPH=1` to avoid graph fan-out; publish TRX + coverage.
- `lint-spec`: validate VEX OpenAPI/JSON schema snapshots (run `dotnet tool run spec-validation`).
- `sbom+attest`: reuse `ops/devops/docker/sbom_attest.sh` after image build; push attestations.
- `loadtest`: run k6 (or oha) scenario against ephemeral stack via compose profile:
  - startup with Mongo/Redis fixtures from `samples/vex/fixtures/*.json`.
  - endpoints: `/vex/entries?tenant=…`, `/issuer-directory/issuers`, `/issuer-directory/statistics`.
  - SLOs: p95 < 250ms for reads, error rate < 0.5%.
  - artifacts: `results.json` + Prometheus remote-write if enabled.

## Load Test Shape (k6 sketch)
- 5 min ramp to 200 VUs, 10 min steady, 2 min ramp-down.
- Mix: 70% list queries (pagination), 20% filtered queries (product, severity), 10% issuer stats.
- Headers: tenant header (`X-StellaOps-Tenant`), auth token from seeded issuer.
- Fixtures: seed 100k VEX statements, 5k issuers, mixed disputed/verified statuses.

## Dashboards (Grafana)
Panels to add under folder `StellaOps / VEX`:
- API latency: p50/p95/p99 for `/vex/entries`, `/issuer-directory/*`.
- Error rates by status code and tenant.
- Query volume and cache hit rate (Redis, if used).
- Mongo metrics: `mongodb_driver_commands_seconds` (p95), connection pool usage.
- Background jobs: ingestion/GC queue latency and failures.

## Alerts
- `vex_api_latency_p95_gt_250ms` for 5m.
- `vex_api_error_rate_gt_0.5pct` for 5m.
- `issuer_directory_cache_miss_rate_gt_20pct` for 15m (if cache enabled).
- `mongo_pool_exhausted` when pool usage > 90% for 5m.

## Offline / air-gap posture
- Use mirrored images and `local-nugets/` only; no outbound fetch in CI jobs.
- k6 binary vendored under `tools/k6/` (add to cache) or use `oha` from `tools/oha/`.
- Load test fixtures stored in repo under `samples/vex/fixtures/` to avoid network pulls.

## How to run locally
```
# build and test
DOTNET_DISABLE_BUILTIN_GRAPH=1 dotnet test src/VexLens/StellaOps.VexLens.Tests/StellaOps.VexLens.Tests.csproj
# run loadtest (requires docker + k6)
make -f ops/devops/Makefile vex-loadtest
```

## Evidence to attach
- TRX + coverage
- k6 `results.json`/`summary.txt`
- Grafana dashboard JSON export (`dashboards/vex/*.json`)
- Alert rules file (`ops/devops/vex/alerts.yaml` when created)