c1acd04249
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
This commit introduces a new script `tenant_isolation_smoke.py` that performs smoke tests to validate tenant isolation in the telemetry storage stack (Tempo + Loki) with mutual TLS enabled. The script checks that traces and logs pushed with specific tenant headers are only accessible to the corresponding tenants, ensuring proper enforcement of multi-tenancy. The tests include pushing a trace and a log entry, followed by assertions to verify access restrictions based on tenant IDs.
Telemetry Collector Assets
These assets provision the default OpenTelemetry Collector instance required by
DEVOPS-OBS-50-001. The collector acts as the secured ingest point for traces,
metrics, and logs emitted by Stella Ops services.
Contents
| File | Purpose |
|---|---|
otel-collector-config.yaml |
Baseline collector configuration (mutual TLS, OTLP receivers, Prometheus exporter). |
storage/prometheus.yaml |
Prometheus scrape configuration tuned for the collector and service tenants. |
storage/tempo.yaml |
Tempo configuration with multitenancy, WAL, and compaction settings. |
storage/loki.yaml |
Loki configuration enabling multitenant log ingestion with retention policies. |
storage/tenants/*.yaml |
Per-tenant overrides for Tempo and Loki rate/retention controls. |
Development workflow
- Generate development certificates (collector + client) using
ops/devops/telemetry/generate_dev_tls.sh. - Launch the collector via
docker compose -f docker-compose.telemetry.yaml up. - Launch the storage backends (Prometheus, Tempo, Loki) via
docker compose -f docker-compose.telemetry-storage.yaml up. - Run the smoke test:
python ops/devops/telemetry/smoke_otel_collector.py. - Explore the storage configuration (
storage/README.md) to tune retention/limits.
The smoke test sends OTLP traffic over TLS and asserts the collector accepted traces, metrics, and logs by scraping the Prometheus metrics endpoint.
Kubernetes
The Helm chart consumes the same configuration (see values.yaml). Provide TLS
material via a secret referenced by telemetry.collector.tls.secretName,
containing ca.crt, tls.crt, and tls.key. Client certificates are required
for ingestion and should be issued by the same CA.