Add tenant isolation smoke test for telemetry stack
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

This commit introduces a new script `tenant_isolation_smoke.py` that performs smoke tests to validate tenant isolation in the telemetry storage stack (Tempo + Loki) with mutual TLS enabled. The script checks that traces and logs pushed with specific tenant headers are only accessible to the corresponding tenants, ensuring proper enforcement of multi-tenancy. The tests include pushing a trace and a log entry, followed by assertions to verify access restrictions based on tenant IDs.
This commit is contained in:
master
2025-11-05 15:09:54 +02:00
parent 90c244948a
commit c1acd04249
20 changed files with 890 additions and 574 deletions

View File

@@ -1,32 +1,45 @@
# StellaOps Graph
Graph module (upcoming) will power graph-indexed queries for SBOM relationships, lineage, and blast-radius analysis.
## Responsibilities
- Model SBOM and advisory entities as a navigable graph.
- Provide APIs for dependency impact, provenance chains, and reachability analysis.
- Integrate with Scheduler/Policy for graph-driven re-evaluation.
- Expose tooling for offline explorers.
- Maintain [Graph Index Canonical Schema](schema.md) with deterministic identities, fixtures, and attribute dictionary.
### Domain highlights (Epic5)
- **Nodes:** artifacts/images, SBOM components, packages/versions, files/paths, licences, advisories, VEX statements, provenance attestations, policy versions.
- **Edges:** `depends_on`, `contains`, `built_from`, `declared_in`, `affected_by`, `vex_exempts`, `governs_with`, `produced_by`, each timestamped and tenant-scoped.
- **Overlays:** policy verdict overlays, VEX consensus, runtime telemetry, and export-ready snapshots with diff support.
- **Queries:** reachability (`impact(graph, advisory)`), blast radius (`reverseDepends(component)`), provenance timeline, saved query library with semantic zoom for Console.
## Key components
- Planned services documented in implementation plan (to be delivered).
## Integrations & dependencies
- SBOM Service / Cartographer for data ingestion.
- Policy & CLI for query surfaces.
## Operational notes
- Pending — see implementation plan for staged milestones.
## Backlog references
- DOCS-GRAPH-24-003 (architecture index) and SCHED-MODELS-21-001 tasks.
## Epic alignment
- **Epic 5 SBOM Graph Explorer:** deliver graph indexer, API, Console explorer, saved queries, overlays, and exports.
# StellaOps Graph
Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers blast-radius analysis, provenance timelines, and saved-query automation across StellaOps. Cartographer has been retired as of 2025-10-30 (see `docs/updates/2025-10-30-devops-governance.md`); this module now owns ingestion, storage, overlays, and query surfaces for graph data.
## Scope & responsibilities
- Ingest SBOM snapshots, advisory/VEX events, policy overlays, and runtime signals to maintain a first-party graph representation with deterministic node/edge identities.
- Serve APIs and saved-query tooling for impact analysis, dependency traversal, diffing, and policy/VEX overlays with explainable provenance.
- Supply Graph Explorer UI/CLI experiences, plus Offline Kit exports (`nodes.jsonl`, `edges.jsonl`, `overlays/`) with DSSE manifests for air-gapped replay.
- Maintain the [Graph Index Canonical Schema](schema.md) and coordinate query/overlay lifecycle with Scheduler, Policy Engine, Vulnerability Explorer, and Export Center.
## Architecture snapshot (Sprint 30 groundwork)
- **Graph Indexer service** — consumes SBOM (`sbom_snapshot`), advisory, and VEX events; normalises identifiers; persists into `graph_nodes`, `graph_edges`, `graph_snapshots`, and overlay caches with tenant partitions.
- **Graph API service** — exposes `GET /graph/nodes`, `/graph/impact/{advisory}`, `/graph/query/saved`, `/graph/diff`, and overlay endpoints with RBAC scopes defined in Authority (`docs/updates/2025-10-26-authority-graph-scopes.md`).
- **Overlay & diff workers** — materialise impact lists, saved-query caches, and signed diff manifests; feed Scheduler `GraphBuildJob`/`GraphOverlayJob` contracts (`docs/updates/2025-10-26-scheduler-graph-jobs.md`).
- **Console & CLI integrations** — planned modules deliver WebGL explorer, timeline viz, and CLI `stella sbom graph ...` commands aligned with implementation plan phases.
- **Storage abstraction** — supports document + adjacency (Mongo) or pluggable graph engine; both paths enforce deterministic ordering and export manifests.
## Current workstreams (Q42025)
- `GRAPH-SVC-30-00x` (in `src/Graph/StellaOps.Graph.Indexer/TASKS.md`) — stand up Graph Indexer pipeline, identity registry, snapshot exports.
- `GRAPH-API-30-00x` — draft API planner/cost guard, streaming responses, and Authority scope integration.
- `DOCS-GRAPH-24-003` & related backlog — author overview/API/query language docs; update this README again once those deliverables land.
- Deployment/DevOps follow-ups (`DEVOPS-VEX-30-001`, `DEPLOY-VEX-30-001`) coordinate dashboards, load tests, and Helm/Compose overlays for the graph stack.
## Integrations & dependencies
- **SBOM Service** (Scanner WebService + Worker) produce `sbom_snapshot` events consumed by Graph Indexer.
- **Concelier/Excititor** contribute advisory + VEX edges; VEX Lens consensus overlays attach to graph nodes as attributes.
- **Policy Engine & Scheduler** trigger recompute jobs and consume overlays for risk/impact automation.
- **Vulnerability Explorer & Console** surface graph queries, saved views, and diff visualisations.
- **Authority** defines scopes (`graph.viewer`, `graph.operator`) and client registrations; secrets managed via existing platform patterns.
## Data, observability & offline
- Collections/tables: `graph_nodes`, `graph_edges`, `graph_snapshots`, `graph_saved_queries`, `graph_overlays_cache`, append-only change logs for replay.
- Metrics: `graph_ingest_lag_seconds`, `graph_nodes_total`, `graph_query_latency_seconds{queryId}`, overlay/diff duration counters.
- Logs/traces: structured ETL logs, query planner traces, WebGL interaction telemetry (once UI lands).
- Offline bundles: deterministic `nodes.jsonl`, `edges.jsonl`, overlay manifests + DSSE signatures, consumable by Export Center and CLI mirroring.
## Key docs & updates
- [`architecture.md`](architecture.md) — inputs, pipelines, APIs, storage choices, observability, offline handling.
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap, work breakdown, risks, test strategy.
- [`schema.md`](schema.md) — canonical node/edge schema and attribute dictionary (keep in sync with indexer code).
- Updates: `docs/updates/2025-10-26-scheduler-graph-jobs.md`, `docs/updates/2025-10-26-authority-graph-scopes.md`, `docs/updates/2025-10-30-devops-governance.md` for the latest decisions/dependencies.
## Epic alignment
- **Epic 5 SBOM Graph Explorer:** Graph Indexer, Graph API, saved queries, overlays, Console/CLI experiences, Offline Kit parity.
- Cross-epic ties: Policy reasoning (explain overlays), Scheduler recompute, Notify/Task Runner integration for graph incidents.

View File

@@ -4,6 +4,7 @@
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| GRAPH-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| GRAPH-DOCS-0001 | DONE (2025-11-05) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | README refreshed with Sprint 30 roadmap, integrations, and references. |
| GRAPH-DOCS-0002 | TODO | Docs Guild | Fold new API/query language docs into module README once DOCS-GRAPH-24-003 lands. | Blocked on DOCS-GRAPH-24-003; add cross-links and imposed rule reminder. |
| GRAPH-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| GRAPH-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against `/docs/implplan/SPRINT_*.md`. | Update status via ./AGENTS.md workflow |

View File

@@ -1,35 +1,46 @@
# StellaOps Notify
Notify evaluates operator-defined rules against platform events and dispatches channel-specific payloads with full auditability.
## Responsibilities
- Process event streams and apply tenant-scoped routing rules.
- Render connector-specific payloads (email, Slack, Teams, webhook, custom).
- Enforce throttling, digests, and delivery retries.
- Surface delivery/audit data for UI and CLI consumers.
## Key components
- `StellaOps.Notify.WebService` (rules API + preview).
- `StellaOps.Notify.Worker` (delivery engine).
- Connector libraries under `StellaOps.Notify.Connectors.*`.
## Integrations & dependencies
- MongoDB for rule/channel storage.
- Redis/NATS for delivery queues.
- CLI/UI for authoring and monitoring notifications.
## Operational notes
- Schema fixtures in ./resources/schemas & ./resources/samples.
- Connector-specific monitoring dashboards.
- Offline runner guidance inside operations playbook.
## Related resources
- ./resources/schemas
- ./resources/samples
## Backlog references
- NOTIFY-SVC-38..40 (Notify backlog) referenced in `docs/README.md`.
- DOCS-NOTIFY updates tracked in ../../TASKS.md when available.
## Epic alignment
- **Epic 11 Notifications Studio:** deliver notifications workspace, preview tooling, immutable delivery ledger, and tenant-scoped throttling/digest controls.
# StellaOps Notify
Notify (Notifications Studio) converts platform events into tenant-scoped alerts with deterministic delivery, offline parity, and a full audit trail. The service is split between the reusable tooling in `src/Notify/*` and the runtime host in `src/Notifier/*` (decision recorded 2025-11-02) so downstream systems can embed the rules engine without inheriting the Studio UI.
## Scope & responsibilities
- Apply tenant-scoped rules to events from Scanner, Scheduler, VEX Lens, Attestor, Task Runner, and Zastava.
- Render channel-specific payloads (Slack, Teams, Email, webhook) using deterministic templates with localisation safeguards.
- Enforce throttling, digests, and quiet-hour calendars so bursts stay explainable and recoverable.
- Persist deliveries, attempts, throttles, and DSSE hashes for CLI/UI investigation and compliance export.
## Current capabilities (Sprint 38 foundations)
- **Rules + channels API:** `StellaOps.Notify.WebService` exposes CRUD, previews, and health probes secured by Authority scopes.
- **Worker pipeline:** `StellaOps.Notify.Worker` ingests bus events, evaluates match predicates, applies per-tenant throttles, and dispatches deliveries.
- **Connector plug-ins:** Restart-time plug-ins under `StellaOps.Notify.Connectors.*` (Slack, Teams, Email, generic webhook) with health checks and retry policy hints declared in `notify-plugin.json`.
- **Template engine:** Deterministic rendering with safe helpers, locale bundles, and redaction defaults that keep Offline Kit parity.
- **Delivery ledger:** Mongo-backed ledger storing hashed payloads, attempts, throttled/digested markers, and provenance links for audit + exports.
## In progress / upcoming (Sprint 39 focus)
- `NOTIFY-SVC-39-001` correlation engine with token-bucket throttles, incident lifecycle, and quiet-hours evaluator.
- `NOTIFY-SVC-39-002` digest generator with schedule runner, ledger queries, and distribution across existing channels.
- `NOTIFY-SVC-39-003` simulation API for rule dry-runs against historical events.
- `NOTIFY-SVC-39-004` quiet-hour calendar integration and default throttles with audit logging.
Status for these items is tracked in `src/Notifier/StellaOps.Notifier/TASKS.md` and sprint plans; update this README once tasks merge.
## Key docs & release alignment
- [`docs/notifications/overview.md`](../../notifications/overview.md) — summary of capabilities, imposed rules, and customer journey.
- [`docs/notifications/architecture.md`](../../notifications/architecture.md) — Notifications Studio runtime view (published 2025-10-29).
- [`docs/notifications/rules.md`](../../notifications/rules.md) — declarative matcher syntax and evaluation order.
- [`docs/notifications/digests.md`](../../notifications/digests.md) — digest windows, coalescing logic, and delivery samples.
- [`docs/notifications/templates.md`](../../notifications/templates.md) — template helpers, localisation, and redaction guidelines.
- [`docs/updates/2025-10-29-notify-docs.md`](../../updates/2025-10-29-notify-docs.md) — latest release note; follow-ups remain to validate connector metadata, quiet-hours semantics, and simulation payloads once Sprint 39 drops land.
## Integrations & dependencies
- **Storage:** MongoDB (`rules`, `channels`, `deliveries`, `digests`, `throttles`) with change streams for worker snapshots.
- **Queues:** Redis Streams or NATS JetStream for ingestion, throttling, and DLQs (`notify.dlq`).
- **Authority:** OpTok-protected APIs, DPoP-backed CLI/UI scopes (`notify.viewer`, `notify.operator`, `notify.admin`), and secret references for channel credentials.
- **Observability:** Prometheus metrics (`notify.sent_total`, `notify.failed_total`, `notify.digest_coalesced_total`, etc.), OTEL traces, and dashboards documented in `docs/notifications/architecture.md#12-observability-prometheus--otel`.
## Operational notes
- Schema fixtures live in `./resources/schemas`; event and delivery samples live in `./resources/samples` for contract tests and UI mocks.
- Offline Kit bundles ship plug-ins, default templates, and seed rules; update manifests under `ops/offline-kit/` when connectors change.
- Dashboards and alert references depend on `DEVOPS-NOTIFY-39-002`; coordinate before renaming metrics or labels.
- When releasing new rule or connector features, mirror guidance into `docs/notifications/*.md` and checklists in `docs/updates/2025-10-29-notify-docs.md` until the follow-ups are closed.
## Epic alignment
- **Epic 11 Notifications Studio:** notifications workspace, preview tooling, immutable delivery ledger, throttling/digest controls, and forthcoming correlation/simulation features.

View File

@@ -4,6 +4,7 @@
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| NOTIFY-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| NOTIFY-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| NOTIFY-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against `/docs/implplan/SPRINT_*.md`. | Update status via ./AGENTS.md workflow |
| NOTIFY-DOCS-0001 | DONE (2025-11-05) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | README refreshed to match 2025-10-29 release note and reference follow-ups. |
| NOTIFY-DOCS-0002 | TODO | Docs Guild | Document correlation engine, digest generator, simulation API, and quiet-hour calendars once NOTIFY-SVC-39-001..004 merge. | Blocked on NOTIFY-SVC-39-001..004 landing; update README + notifications/* docs. |
| NOTIFY-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| NOTIFY-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against `/docs/implplan/SPRINT_*.md`. | Update status via ./AGENTS.md workflow |

View File

@@ -28,6 +28,10 @@ docker compose -f docker-compose.telemetry.yaml up -d
docker compose -f docker-compose.telemetry-storage.yaml up -d
python ../../ops/devops/telemetry/smoke_otel_collector.py --host localhost
python ../../ops/devops/telemetry/validate_storage_stack.py
python ../../ops/devops/telemetry/tenant_isolation_smoke.py \
--collector https://localhost:4318/v1 \
--tempo https://localhost:3200 \
--loki https://localhost:3100
```
Configuration files live in `deploy/telemetry/storage/`. Adjust the overrides before shipping to staging/production.

View File

@@ -1,28 +1,44 @@
# StellaOps VEX Consensus Lens
VEX Lens computes deterministic consensus across conflicting VEX statements while preserving raw provenance.
## Responsibilities
- Ingest VEX evidence from Excititor and align it to SBOM inventory.
- Apply issuer trust weights, freshness rules, and policy-defined tie breakers.
- Publish consensus snapshots and disagreement metadata for Policy Engine and Explorer surfaces.
- Expose APIs for explainability and offline bundle exports.
## Key components
- Consensus computation service and job pipeline.
- Consensus store with versioned snapshots.
- Explain trace generator for disagreements.
## Integrations & dependencies
- Excititor for raw VEX ingestion.
- Policy Engine for applying consensus in suppression flows.
- Vulnerability Explorer and Advisory AI for evidence overlays.
## Operational notes
- Trust model configuration and issuer scoring dashboards.
- Offline kit packaging of consensus snapshots.
- Telemetry on issuer coverage and disagreement counts.
## Epic alignment
- Epic 7: VEX Consensus Lens.
- Lens implementation stories tracked in ../../TASKS.md.
# StellaOps VEX Consensus Lens
VEX Lens produces a deterministic, provenance-rich consensus view of VEX statements so downstream policy and exploration surfaces can trust what “affected” or “not affected” means per artifact. It separates raw ingestion (Excititor) from adjudication, weighs issuers cryptographically, and keeps conflicts transparent for auditors and tooling.
## Scope & responsibilities
- Normalise OpenVEX, CSAF VEX, and CycloneDX VEX documents from Excititor into lens-ready tuples without violating the Aggregation-Only Contract.
- Apply issuer weights, signature verification results, freshness decay, and policy overrides to compute consensus and confidence for each `(artifact, advisory)` pair.
- Emit consensus snapshots, conflict summaries, and DSSE attestations consumed by Policy Engine, Vulnerability Explorer, Advisory AI, and Task Runner workflows.
- Provide simulation/export APIs and Offline Kit bundles so tenants can rehearse policy changes and mirror consensus data in air-gapped environments.
## Architecture snapshot (Sprint 30 groundwork)
- **StellaOps.VexLens service & workers** — orchestrate normalisation, trust weighting, lattice join, and persistence into `vex_consensus`, `vex_consensus_history`, and `vex_conflict_queue` collections.
- **Issuer Directory integration** — maintains publisher metadata, keys, and trust tiers that feed weighting engines and revocation workflows.
- **Consensus APIs** — `/v1/vex/consensus`, `/v1/vex/conflicts`, `/v1/vex/trust/weights`, and export streams with DSSE manifests for Offline Kit + Export Center.
- **Explainability traces** — capture derived-from chains, conflicting issuers, and trust deltas to power UI drilldowns and CLI audits.
- **Recompute orchestration** — Orchestrator jobs trigger recompute on Excititor deltas, issuer updates, or policy knob changes with deterministic ordering and SRM manifests.
## Current workstreams (Q42025)
- `VEXLENS-30-001..004` — build normalisation pipeline, product mapping library, and trust weighting engine (in progress; dependencies captured in src/VexLens/StellaOps.VexLens/TASKS.md).
- `VEXLENS-30-005..007` — expose consensus APIs and export flows, aligning docs with future `/docs/vex/consensus-*.md` deliverables.
- `DOCS-VEX-30-001..004` — broader documentation set (overview, algorithm, API) tracked in `docs/TASKS.md`; update this README again once those artefacts merge.
## Integrations & dependencies
- **Excititor** supplies signature-verified VEX observations and issuer hints.
- **Policy Engine** consumes consensus verdicts for suppression/waiver logic and exposes trust controls to operators.
- **Vulnerability Explorer & Advisory AI** render consensus badges, conflicts, and rationale in investigative workflows.
- **Orchestrator & Scheduler** run backfills, recomputes, and incident routing driven by Lens change streams.
- **Notify / Task Runner** receive conflict and override events for operator actions once notification bridges ship.
## Data & observability
- Collections: `vex_consensus`, `vex_consensus_history`, `vex_conflict_queue`, plus issuer registry tables managed with tenant isolation and deterministic indexes.
- Metrics: `vex_consensus_conflicts_total`, `vex_consensus_latency_seconds`, `vex_consensus_recompute_seconds{reason}`, signature failure counters.
- Traces/logs: `consensus.group`, `consensus.join`, `consensus.persist` spans with correlation IDs and issuer details; structured logs capture trust adjustments and reconciliation outcomes.
- Offline bundles include `consensus.jsonl`, `conflicts.jsonl`, manifest + DSSE signatures, enabling mirror deployments and replay validation.
## Key docs & references
- [`architecture.md`](architecture.md) — implementation-ready blueprint covering inputs, algorithm, APIs, storage, observability, and exports.
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap and acceptance criteria.
- [`../../vex/aggregation.md`](../../vex/aggregation.md) — Aggregation-Only Contract boundaries for VEX ingestion and downstream consumers.
- Sprint tracking in `docs/implplan/SPRINT_200_documentation_process.md` and module engineering tasks in `src/VexLens/StellaOps.VexLens/TASKS.md`.
## Epic alignment
- Epic7 — VEX Consensus Lens: deterministic VEX adjudication, issuer directory, consensus exports.
- Related epics: Issuer Directory (keys & trust), Vulnerability Explorer overlays, Policy trust tuning.

View File

@@ -4,6 +4,7 @@
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| VEX-CONSENSUS-LENS-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Ensure ./README.md reflects the latest epic deliverables. | Align with ./AGENTS.md |
| VEX-CONSENSUS-LENS-DOCS-0001 | DONE (2025-11-05) | Docs Guild | Ensure ./README.md reflects the latest epic deliverables. | README expanded with Sprint 30 roadmap, integrations, and references. |
| VEX-CONSENSUS-LENS-DOCS-0002 | TODO | Docs Guild | Fold consensus overview/algorithm/API docs into module README once DOCS-VEX-30-001..004 merge. | Blocked on DOCS-VEX-30-001..004; add cross-links and imposed rule reminder. |
| VEX-CONSENSUS-LENS-ENG-0001 | TODO | Module Team | Break down epic milestones into actionable stories. | Sync into ../../TASKS.md |
| VEX-CONSENSUS-LENS-OPS-0001 | TODO | Ops Guild | Prepare runbooks/observability assets once MVP lands. | Document outputs in ./README.md |