feat: Add UI benchmark driver and scenarios for graph interactions
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
- Introduced `ui_bench_driver.mjs` to read scenarios and fixture manifest, generating a deterministic run plan. - Created `ui_bench_plan.md` outlining the purpose, scope, and next steps for the benchmark. - Added `ui_bench_scenarios.json` containing various scenarios for graph UI interactions. - Implemented tests for CLI commands, ensuring bundle verification and telemetry defaults. - Developed schemas for orchestrator components, including replay manifests and event envelopes. - Added mock API for risk management, including listing and statistics functionalities. - Implemented models for risk profiles and query options to support the new API.
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# Telemetry agent guide
|
||||
|
||||
## Mission
|
||||
Telemetry module captures deployment and operations guidance for the shared observability stack (collectors, storage, dashboards).
|
||||
Telemetry module captures deployment and operations guidance for the shared observability stack (collectors, storage, dashboards). Telemetry outputs must be deterministic, sealed-mode safe, and verifiable via DSSE/offline bundles.
|
||||
|
||||
## Advisory Handling
|
||||
- Any new/updated advisory triggers immediate doc + sprint updates; no approval.
|
||||
@@ -16,6 +16,8 @@ Telemetry module captures deployment and operations guidance for the shared obse
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
- [Observability runbook](./operations/observability.md) (offline import friendly)
|
||||
- [Telemetry gaps remediation](./contracts/telemetry-gaps-remediation.md)
|
||||
- Schemas: `./schemas/telemetry-config.schema.json`, `./schemas/telemetry-bundle.schema.json`
|
||||
|
||||
## How to get started
|
||||
1. Open sprint file `/docs/implplan/SPRINT_*.md` and locate the stories referencing this module.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Telemetry Gap Remediation (TO1–TO10) — v1 · 2025-12-01
|
||||
|
||||
Source: `docs/product-advisories/31-Nov-2025 FINDINGS.md` (Telemetry gaps TO1–TO10).
|
||||
Scope: telemetry core (collectors/SDK defaults/bundles) across services; applicable to default/forensic/airgap profiles.
|
||||
|
||||
## Decisions (mapped to gaps)
|
||||
- **TO1 Canonical schemas & hashing**: Published versioned schemas
|
||||
- `telemetry-config.schema.json` for collector/SDK profile configs (signed, canonical JSON, stable ordering)
|
||||
- `telemetry-bundle.schema.json` for offline bundle manifests
|
||||
- Hash recipe: SHA-256 over normalized (UTF-8, LF, sorted keys) JSON; test vectors to follow.
|
||||
- **TO2 Provenance & DSSE**: Bundles and profile activations must include DSSE envelope (`*.dsse.json`) with predicate fields: profileHash, collectorVersion, exporters, redactionPolicyUri, cryptoProfile.
|
||||
- **TO3 Determinism & sampling stability**: Sampling policies must declare deterministic seed, ordered rules, and backpressure policy. Logs/traces ordered by (timestamp, traceId). Multi-run hash check recommended in CI.
|
||||
- **TO4 Sealed mode / egress guards**: Sealed mode blocks all non-loopback exporters unless explicitly allowlisted; DNS pinning required; failure is fail-closed. Seal status recorded as DSSE event.
|
||||
- **TO5 Redaction policy & PII tests**: Redaction catalog/allowlist required; bundle must include `redaction-manifest.json` listing rules applied and violations=0. CI must run PII/secret test suite before export.
|
||||
- **TO6 Tenant isolation & quotas**: OTLP signals include `tenant.id` and `project.id`; collector routes by tenant pipeline; per-tenant quotas/limits enforced with counters and alerts.
|
||||
- **TO7 Forensic triggers governance**: Forensic mode requires dual approval, DSSE activation record, expiry timestamp, and auto-rollback; alert if forensic mode active > configured window.
|
||||
- **TO8 Offline bundle schema & verify**: Bundles must follow `telemetry-bundle.schema.json`, created with deterministic tar flags, include hash manifest + DSSE + RFC3161 time-anchor; verifier script provided (`ops/devops/telemetry/verify-telemetry-bundle.sh`).
|
||||
- **TO9 Observability of observability**: Add SLOs + alerts for collector/exporter health, queue backpressure, bundle success rate; scheduled self-test emits DSSE result.
|
||||
- **TO10 CLI/pack contracts**: CLI/pack contract tracked in `cli-spec-v1.yaml`; telemetry exports must respect exit codes and checksum policy (reuse 21/22 for checksum missing/mismatch).
|
||||
|
||||
## Artifacts
|
||||
- Schemas: `docs/modules/telemetry/schemas/telemetry-config.schema.json`, `telemetry-bundle.schema.json`.
|
||||
- Hash recipe: in-line within schemas (canonical JSON, SHA-256).
|
||||
- Verify script: `ops/devops/telemetry/verify-telemetry-bundle.sh`.
|
||||
|
||||
## Adoption notes
|
||||
- Profile and bundle producers must validate against schemas and sign DSSE envelopes before distribution.
|
||||
- Air-gap/forensic profiles MUST set sealed mode and include redaction manifest.
|
||||
- CI should add a multi-run hash test for telemetry exporter output and fail on drift.
|
||||
46
docs/modules/telemetry/schemas/telemetry-bundle.schema.json
Normal file
46
docs/modules/telemetry/schemas/telemetry-bundle.schema.json
Normal file
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "StellaOps Telemetry Bundle Manifest",
|
||||
"version": "1.0.0",
|
||||
"type": "object",
|
||||
"required": ["schemaVersion", "bundleId", "createdAt", "artifacts", "manifestHashAlgorithm", "timeAnchor"],
|
||||
"properties": {
|
||||
"schemaVersion": { "type": "string", "const": "1.0.0" },
|
||||
"bundleId": { "type": "string", "format": "uuid" },
|
||||
"createdAt": { "type": "string", "format": "date-time" },
|
||||
"profileHash": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"collectorVersion": { "type": "string" },
|
||||
"sealedMode": { "type": "boolean" },
|
||||
"redactionManifest": { "type": "string" },
|
||||
"manifestHashAlgorithm": { "type": "string", "enum": ["sha256"] },
|
||||
"timeAnchor": {
|
||||
"type": "object",
|
||||
"required": ["type", "value"],
|
||||
"properties": {
|
||||
"type": { "type": "string", "enum": ["rfc3161", "roughtime"] },
|
||||
"value": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"artifacts": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["path", "sha256", "mediaType"],
|
||||
"properties": {
|
||||
"path": { "type": "string" },
|
||||
"sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"mediaType": { "type": "string" },
|
||||
"size": { "type": "integer", "minimum": 0 }
|
||||
}
|
||||
}
|
||||
},
|
||||
"dsseEnvelope": {
|
||||
"type": "object",
|
||||
"required": ["hash"],
|
||||
"properties": {
|
||||
"hash": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"location": { "type": "string" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
75
docs/modules/telemetry/schemas/telemetry-config.schema.json
Normal file
75
docs/modules/telemetry/schemas/telemetry-config.schema.json
Normal file
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "StellaOps Telemetry Config",
|
||||
"version": "1.0.0",
|
||||
"type": "object",
|
||||
"required": ["schemaVersion", "profiles"],
|
||||
"properties": {
|
||||
"schemaVersion": { "type": "string", "const": "1.0.0" },
|
||||
"hashAlgorithm": { "type": "string", "enum": ["sha256"] },
|
||||
"profiles": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["name", "collectorVersion", "exporters", "redactionPolicyUri", "sampling"],
|
||||
"properties": {
|
||||
"name": { "type": "string" },
|
||||
"description": { "type": "string" },
|
||||
"collectorVersion": { "type": "string" },
|
||||
"cryptoProfile": { "type": "string" },
|
||||
"sealedMode": { "type": "boolean" },
|
||||
"allowlistedEndpoints": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "format": "uri" }
|
||||
},
|
||||
"exporters": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["type", "endpoint"],
|
||||
"properties": {
|
||||
"type": { "type": "string", "enum": ["otlp", "file", "stdout", "null"] },
|
||||
"endpoint": { "type": "string" },
|
||||
"protocol": { "type": "string", "enum": ["grpc", "http"] },
|
||||
"compression": { "type": "string", "enum": ["none", "gzip"] },
|
||||
"enabled": { "type": "boolean", "default": true }
|
||||
}
|
||||
}
|
||||
},
|
||||
"redactionPolicyUri": { "type": "string", "format": "uri" },
|
||||
"sampling": {
|
||||
"type": "object",
|
||||
"required": ["strategy", "seed"],
|
||||
"properties": {
|
||||
"strategy": { "type": "string", "enum": ["always_on", "always_off", "traceidratio", "tail"] },
|
||||
"seed": { "type": "string", "pattern": "^[0-9a-fA-F]{16}$" },
|
||||
"rules": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["match", "priority"],
|
||||
"properties": {
|
||||
"match": { "type": "string" },
|
||||
"priority": { "type": "integer", "minimum": 0 },
|
||||
"sampleRate": { "type": "number", "minimum": 0, "maximum": 1 }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"tenantRouting": {
|
||||
"type": "object",
|
||||
"required": ["attribute"],
|
||||
"properties": {
|
||||
"attribute": { "type": "string", "const": "tenant.id" },
|
||||
"quotasPerTenant": {
|
||||
"type": "object",
|
||||
"additionalProperties": { "type": "integer", "minimum": 0 }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user