feat: Add UI benchmark driver and scenarios for graph interactions
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
- Introduced `ui_bench_driver.mjs` to read scenarios and fixture manifest, generating a deterministic run plan. - Created `ui_bench_plan.md` outlining the purpose, scope, and next steps for the benchmark. - Added `ui_bench_scenarios.json` containing various scenarios for graph UI interactions. - Implemented tests for CLI commands, ensuring bundle verification and telemetry defaults. - Developed schemas for orchestrator components, including replay manifests and event envelopes. - Added mock API for risk management, including listing and statistics functionalities. - Implemented models for risk profiles and query options to support the new API.
This commit is contained in:
19
docs/modules/cli/contracts/output-determinism.md
Normal file
19
docs/modules/cli/contracts/output-determinism.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# CLI Output Determinism Policy (v1) — 2025-12-01
|
||||
|
||||
Scope: `StellaOps.Cli` JSON/table/markdown outputs for advisory verbs and auth doctor.
|
||||
|
||||
Rules
|
||||
- Time: All timestamps UTC; no local timezone conversion.
|
||||
- Locale: `en-US`, `InvariantCulture` for number/date formatting.
|
||||
- Ordering: Sort collections by stable key (id/name) before rendering; JSON keys stable-sorted.
|
||||
- Width: Table renderer clamps to width 80; no ANSI when `--output json` or non-TTY.
|
||||
- Seeds: Randomness forbidden; no wall-clock in hashes; use provided deterministic IDs.
|
||||
|
||||
Tests
|
||||
- Golden fixtures stored under `src/Cli/__Tests/StellaOps.Cli.Tests/Fixtures/output-determinism/`.
|
||||
- Hash check: two consecutive runs of the same command with identical inputs must produce identical SHA256 of stdout.
|
||||
- Locale guard: integration test forces `CultureInfo("fr-FR")` and asserts output matches fixtures.
|
||||
|
||||
Failure handling
|
||||
- Any drift fails CI; diff is printed with unified format.
|
||||
- Add new fields behind explicit versioned spec entry in `cli-spec-v1.yaml`.
|
||||
Reference in New Issue
Block a user