2d08f52715
- Introduced README.md for Zastava Evidence Locker Plan detailing artifacts to sign and post-signing steps. - Added example JSON schemas for observer events and webhook admissions. - Updated implementor guidelines with checklist for CI linting, determinism, secrets management, and schema control. - Created alert rules for Vuln Explorer to monitor API latency and projection errors. - Developed analytics ingestion plan for Vuln Explorer, focusing on telemetry and PII guardrails. - Implemented Grafana dashboard configuration for Vuln Explorer metrics visualization. - Added expected projection SHA256 for vulnerability events. - Created k6 load testing script for Vuln Explorer API. - Added sample projection and replay event data for testing. - Implemented ReplayInputsLock for deterministic replay inputs management. - Developed tests for ReplayInputsLock to ensure stable hash computation. - Created SurfaceManifestDeterminismVerifier to validate manifest determinism and integrity. - Added unit tests for SurfaceManifestDeterminismVerifier to ensure correct functionality. - Implemented Angular tests for VulnerabilityHttpClient and VulnerabilityDetailComponent to verify API interactions and UI rendering.
27 lines
1.6 KiB
Markdown
27 lines
1.6 KiB
Markdown
# Vuln Explorer analytics pipeline plan (DEVOPS-VULN-29-003)
|
|
|
|
Goals: instrument analytics ingestion (query hashes, privacy/PII guardrails), update observability docs, and supply deployable configs.
|
|
|
|
## Instrumentation tasks
|
|
- Expose Prometheus counters/histograms in API:
|
|
- `vuln_query_hashes_total{tenant,query_hash}` increment on cached/served queries.
|
|
- `vuln_api_latency_seconds` histogram (already present; ensure labels avoid PII).
|
|
- `vuln_api_payload_bytes` histogram for request/response sizes.
|
|
- Redact/avoid PII:
|
|
- Hash query bodies server-side (SHA256 with salt per deployment) before logging/metrics; store only hash+shape, not raw filters.
|
|
- Truncate any request field names/values in logs to 128 chars and drop known PII fields (email/userId).
|
|
- Telemetry export:
|
|
- OTLP metrics/logs via existing collector profile; add `service=\"vuln-explorer\"` resource attrs.
|
|
|
|
## Pipelines/configs
|
|
- Grafana dashboard will read from Prometheus metrics already defined in `ops/devops/vuln/dashboards/vuln-explorer.json`.
|
|
- Alert rules already in `ops/devops/vuln/alerts.yaml`; ensure additional rules for PII drops are not required (logs-only).
|
|
|
|
## Docs
|
|
- Update deploy docs (`deploy/README.md`) to mention PII-safe logging in Vuln Explorer and query-hash metrics.
|
|
- Add runbook entry under `docs/modules/vuln-explorer/observability.md` (if absent, create) summarizing metrics and how to interpret query hashes.
|
|
|
|
## CI checks
|
|
- Unit test to assert logging middleware hashes queries and strips PII (to be implemented in API tests).
|
|
- Add static check in pipeline ensuring `vuln_query_hashes_total` and payload histograms are scraped (Prometheus snapshot test).
|