27 lines
2.3 KiB
Markdown
27 lines
2.3 KiB
Markdown
# Acceptance Test Packs with Guardrails
|
|
|
|
## Module
|
|
__Tests
|
|
|
|
## Status
|
|
IMPLEMENTED
|
|
|
|
## Description
|
|
Acceptance test packs with guardrail definitions exist under the test fixtures with expected output validation.
|
|
|
|
## Implementation Details
|
|
- **Acceptance Test Directory**: `src/__Tests/acceptance/` -- acceptance test suite containing end-to-end scenarios with expected output validation and guardrail definitions.
|
|
- **Test Evidence Service**: `src/__Tests/__Libraries/StellaOps.Testing.Evidence/TestEvidenceService.cs` (implements `ITestEvidenceService`) -- captures test execution evidence (inputs, outputs, assertions) for acceptance validation.
|
|
- **Explainability Assertions**: `src/__Tests/__Libraries/StellaOps.Testing.Explainability/ExplainabilityAssertions.cs` -- assertion library for verifying decision explainability in acceptance tests, ensuring verdicts include human-readable rationale.
|
|
- **Explainable Decision Interface**: `src/__Tests/__Libraries/StellaOps.Testing.Explainability/IExplainableDecision.cs` -- contract for decisions that must provide explanations as part of guardrail verification.
|
|
- **Policy Regression Test Base**: `src/__Tests/__Libraries/StellaOps.Testing.Policy/PolicyRegressionTestBase.cs` -- base class for policy acceptance tests with guardrail enforcement.
|
|
- **Policy Diff Engine**: `src/__Tests/__Libraries/StellaOps.Testing.Policy/PolicyDiffEngine.cs` -- diffs policy evaluation outcomes between test runs to detect regressions.
|
|
- **Tests**: `src/__Tests/__Libraries/StellaOps.Testing.Evidence.Tests/TestEvidenceServiceTests.cs`
|
|
|
|
## E2E Test Plan
|
|
- [ ] Run the acceptance test suite and verify all test packs pass with expected outputs matching guardrail definitions
|
|
- [ ] Verify explainability guardrail: run an acceptance test that produces a verdict and confirm the decision includes a human-readable explanation via `ExplainabilityAssertions`
|
|
- [ ] Verify regression detection: modify a policy rule, re-run acceptance tests, and confirm `PolicyDiffEngine` detects the outcome change
|
|
- [ ] Verify evidence capture: run an acceptance test and confirm `TestEvidenceService` captures the full input/output evidence for audit review
|
|
- [ ] Verify guardrail enforcement: introduce a test that violates a guardrail (e.g., missing explanation) and confirm the test fails with a descriptive error
|