qa(advisoryai): verify orchestrator, guardrails, and action-policy features

docs-archived/implplan/SPRINT_20260211_016_AdvisoryAI_unchecked_feature_verification.md

@@ -0,0 +1,86 @@
+# Sprint 20260211_016 - AdvisoryAI Unchecked Feature Verification
+
+## Topic & Scope
+- Verify unchecked `advisoryai` feature docs against current implementation under `src/AdvisoryAI`.
+- Execute Tier 0, Tier 1, and Tier 2 checks per FLOW and move terminal features accordingly.
+- Enforce problems-first ordering before starting any new queued feature.
+- Working directory: `src/AdvisoryAI`.
+- Expected evidence: run artifacts under `docs/qa/feature-checks/runs/advisoryai/**`, state ledger updates, and feature-file moves.
+
+## Dependencies & Concurrency
+- Depends on `src/AdvisoryAI/**` source/tests and `docs/qa/feature-checks/FLOW.md`.
+- Cross-directory updates in `docs/qa/feature-checks/**`, `docs/features/**`, and `docs/implplan/**` are allowed for auditability.
+
+## Documentation Prerequisites
+- `docs/qa/feature-checks/FLOW.md`
+- `src/AdvisoryAI/AGENTS.md`
+- `docs/modules/advisory-ai/architecture.md`
+
+## Delivery Tracker
+
+### QA-AIAI-VERIFY-001 - Verify `advisoryai-orchestrator`
+Status: DONE
+Dependency: none
+Owners: QA / Test Automation
+Task description:
+- Verify `advisoryai-orchestrator` implementation claims in source and tests.
+- Capture Tier 0 source evidence, Tier 1 build/test evidence, and Tier 2 user-surface API behavior evidence.
+- Move feature file from `docs/features/unchecked/advisoryai/` to `docs/features/checked/advisoryai/` on success.
+
+Completion criteria:
+- [x] Tier 0 source verification completed.
+- [x] Tier 1 build and test verification completed.
+- [x] Tier 2 API/behavioral verification completed with fresh run evidence.
+- [x] State file updated and feature moved to checked.
+
+### QA-AIAI-VERIFY-002 - Verify `advisoryai-pipeline-with-guardrails`
+Status: DONE
+Dependency: QA-AIAI-VERIFY-001
+Owners: QA / Test Automation
+Task description:
+- Verify guardrail pipeline, retrievers, and action/idempotency implementation claims for `advisoryai-pipeline-with-guardrails`.
+- Capture Tier 0 source evidence, Tier 1 build/test evidence, and Tier 2 behavioral verification evidence.
+- Move feature file from `docs/features/unchecked/advisoryai/` to `docs/features/checked/advisoryai/` on success, or to `unimplemented` if Tier 0 criteria require terminal downgrade.
+
+Completion criteria:
+- [x] Tier 0 source verification completed.
+- [x] Tier 1 build and test verification completed (if Tier 0 allows progression).
+- [x] Tier 2 behavioral verification completed with fresh run evidence (if Tier 0 allows progression).
+- [x] State file updated and feature moved to checked/unimplemented.
+
+### QA-AIAI-VERIFY-003 - Verify `ai-action-policy-gate`
+Status: DONE
+Dependency: QA-AIAI-VERIFY-002
+Owners: QA / Test Automation
+Task description:
+- Verify action-policy gate behavior claims (policy decisions, approval workflow path, audit recording, idempotency behavior) for `ai-action-policy-gate`.
+- Capture Tier 0 source evidence, Tier 1 build/test evidence, and Tier 2 behavioral evidence for action-policy workflows.
+- Move feature file from `docs/features/unchecked/advisoryai/` to `docs/features/checked/advisoryai/` on success, or classify terminal non-pass state per FLOW.
+
+Completion criteria:
+- [x] Tier 0 source verification completed.
+- [x] Tier 1 build and test verification completed (if Tier 0 allows progression).
+- [x] Tier 2 behavioral verification completed with fresh run evidence (if Tier 0 allows progression).
+- [x] State file updated and feature moved to checked/unimplemented.
+
+## Execution Log
+| Date (UTC) | Update | Owner |
+| --- | --- | --- |
+| 2026-02-11 | Sprint created; started `advisoryai-orchestrator` verification with run-001. | QA |
+| 2026-02-11 | `advisoryai-orchestrator` verified (Tier 0/1/2 pass) and moved to `docs/features/checked/advisoryai/`. | QA |
+| 2026-02-11 | Started `advisoryai-pipeline-with-guardrails` verification (run-001) after confirming concurrent replay problem state had reached terminal outcomes. | QA |
+| 2026-02-11 | `advisoryai-pipeline-with-guardrails` verified (Tier 0/1/2 pass) and moved to `docs/features/checked/advisoryai/` with run-001 artifacts and state update. | QA |
+| 2026-02-11 | Started `ai-action-policy-gate` verification (run-001); state transitioned to checking before Tier 0. | QA |
+| 2026-02-11 | `ai-action-policy-gate` verified (Tier 0/1/2 pass) and moved to `docs/features/checked/advisoryai/` with run-001 artifacts and state update. | QA |
+
+## Decisions & Risks
+- Decision: Selected `advisoryai` as the next unclaimed module missing from checked-state coverage.
+- Decision: Tier 2 used endpoint integration tests (`AdvisoryChatEndpointsIntegrationTests`) plus run lifecycle integration tests (`RunServiceIntegrationTests`) as user-surface behavioral evidence.
+- Risk: `dotnet test --filter` emits `MTP0001` (filter ignored under Microsoft.Testing.Platform), which can broaden executed scope.
+- Mitigation: capture command logs and record warning in run artifacts; treat filter behavior as residual risk while preserving fresh Tier 2 evidence.
+- Decision: Concurrent replay problem-state work was completed by another agent; no contested-file takeover was required before continuing AdvisoryAI verification.
+- Decision: For `advisoryai-pipeline-with-guardrails`, Tier 2 evidence was captured via guardrail/pipeline/retriever/action behavioral integration tests with fresh run-001 logs.
+- Decision: For `ai-action-policy-gate`, Tier 2 evidence used action governance integration tests covering allow/deny/approval and idempotency behavior with fresh run-001 logs.
+
+## Next Checkpoints
+- If no cross-module problem states are present, continue with next queued `advisoryai` feature per FLOW deterministic ordering.