885ce86af4
- Introduced a comprehensive CI job structure for VEX Lens, including build, test, linting, and load testing. - Defined load test parameters and SLOs for VEX Lens API and Issuer Directory. - Created Grafana dashboards and alerting mechanisms for monitoring API performance and error rates. - Established offline posture guidelines for CI jobs and load testing. feat: Implement deterministic projection verification script - Added `verify_projection.sh` script for verifying the integrity of projection exports against expected hashes. - Ensured robust error handling for missing files and hash mismatches. feat: Develop Vuln Explorer CI and Ops Plan - Created CI jobs for Vuln Explorer, including build, test, and replay verification. - Implemented backup and disaster recovery strategies for MongoDB and Redis. - Established Merkle anchoring verification and automation for ledger projector. feat: Introduce EventEnvelopeHasher for hashing event envelopes - Implemented `EventEnvelopeHasher` to compute SHA256 hashes for event envelopes. feat: Add Risk Store and Dashboard components - Developed `RiskStore` for managing risk data and state. - Created `RiskDashboardComponent` for displaying risk profiles with filtering capabilities. - Implemented unit tests for `RiskStore` and `RiskDashboardComponent`. feat: Enhance Vulnerability Detail Component - Developed `VulnerabilityDetailComponent` for displaying detailed information about vulnerabilities. - Implemented error handling for missing vulnerability IDs and loading failures.
55 lines
2.8 KiB
Markdown
55 lines
2.8 KiB
Markdown
# VEX Lens CI + Load/Obs Plan (DEVOPS-VEX-30-001)
|
|
|
|
Scope: CI jobs, load/perf tests, dashboards, and alerts for VEX Lens API and Issuer Directory.
|
|
Assumptions: offline-friendly mirrors available; VEX Lens uses Mongo + Redis; Issuer Directory uses Mongo + OIDC.
|
|
|
|
## CI Jobs (Gitea workflow template)
|
|
- `build-vex`: dotnet restore/build for `src/VexLens/StellaOps.VexLens`, cache `local-nugets/`, set `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1`.
|
|
- `test-vex`: `dotnet test` VexLens and Issuer Directory tests with `DOTNET_DISABLE_BUILTIN_GRAPH=1` to avoid graph fan-out; publish TRX + coverage.
|
|
- `lint-spec`: validate VEX OpenAPI/JSON schema snapshots (run `dotnet tool run spec-validation`).
|
|
- `sbom+attest`: reuse `ops/devops/docker/sbom_attest.sh` after image build; push attestations.
|
|
- `loadtest`: run k6 (or oha) scenario against ephemeral stack via compose profile:
|
|
- startup with Mongo/Redis fixtures from `samples/vex/fixtures/*.json`.
|
|
- endpoints: `/vex/entries?tenant=…`, `/issuer-directory/issuers`, `/issuer-directory/statistics`.
|
|
- SLOs: p95 < 250ms for reads, error rate < 0.5%.
|
|
- artifacts: `results.json` + Prometheus remote-write if enabled.
|
|
|
|
## Load Test Shape (k6 sketch)
|
|
- 5 min ramp to 200 VUs, 10 min steady, 2 min ramp-down.
|
|
- Mix: 70% list queries (pagination), 20% filtered queries (product, severity), 10% issuer stats.
|
|
- Headers: tenant header (`X-StellaOps-Tenant`), auth token from seeded issuer.
|
|
- Fixtures: seed 100k VEX statements, 5k issuers, mixed disputed/verified statuses.
|
|
|
|
## Dashboards (Grafana)
|
|
Panels to add under folder `StellaOps / VEX`:
|
|
- API latency: p50/p95/p99 for `/vex/entries`, `/issuer-directory/*`.
|
|
- Error rates by status code and tenant.
|
|
- Query volume and cache hit rate (Redis, if used).
|
|
- Mongo metrics: `mongodb_driver_commands_seconds` (p95), connection pool usage.
|
|
- Background jobs: ingestion/GC queue latency and failures.
|
|
|
|
## Alerts
|
|
- `vex_api_latency_p95_gt_250ms` for 5m.
|
|
- `vex_api_error_rate_gt_0.5pct` for 5m.
|
|
- `issuer_directory_cache_miss_rate_gt_20pct` for 15m (if cache enabled).
|
|
- `mongo_pool_exhausted` when pool usage > 90% for 5m.
|
|
|
|
## Offline / air-gap posture
|
|
- Use mirrored images and `local-nugets/` only; no outbound fetch in CI jobs.
|
|
- k6 binary vendored under `tools/k6/` (add to cache) or use `oha` from `tools/oha/`.
|
|
- Load test fixtures stored in repo under `samples/vex/fixtures/` to avoid network pulls.
|
|
|
|
## How to run locally
|
|
```
|
|
# build and test
|
|
DOTNET_DISABLE_BUILTIN_GRAPH=1 dotnet test src/VexLens/StellaOps.VexLens.Tests/StellaOps.VexLens.Tests.csproj
|
|
# run loadtest (requires docker + k6)
|
|
make -f ops/devops/Makefile vex-loadtest
|
|
```
|
|
|
|
## Evidence to attach
|
|
- TRX + coverage
|
|
- k6 `results.json`/`summary.txt`
|
|
- Grafana dashboard JSON export (`dashboards/vex/*.json`)
|
|
- Alert rules file (`ops/devops/vex/alerts.yaml` when created)
|