stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

qa(advisoryai): verify orchestrator, guardrails, and action-policy features

Browse Source
This commit is contained in:
master
2026-02-11 13:48:23 +02:00
parent e716bc6adc
commit 4e5300660d
40 changed files with 7770 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/features/unchecked/advisoryai/advisoryai-orchestrator.md
View File

@@ -1,34 +0,0 @@
# AdvisoryAI Orchestrator (Chat + Workbench + Runs)
## Module
AdvisoryAI
## Status
IMPLEMENTED
## Description
The AdvisoryAI module provides a chat orchestrator with session management, run tracking (with artifacts and events), and tool routing. Backend web service with chat and run endpoints is operational.
## Implementation Details
- **Modules**: `src/AdvisoryAi/StellaOps.AdvisoryAI/`, `src/AdvisoryAi/StellaOps.AdvisoryAI.WebService/`, `src/AdvisoryAi/StellaOps.AdvisoryAI.Worker/`
- **Key Classes**:
- `AdvisoryPipelineOrchestrator` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Orchestration/AdvisoryPipelineOrchestrator.cs`) - main pipeline orchestrator coordinating task plans and execution
- `AdvisoryPipelineExecutor` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Execution/AdvisoryPipelineExecutor.cs`) - executes advisory pipeline stages
- `AdvisoryChatService` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Chat/Services/AdvisoryChatService.cs`) - chat session orchestration service
- `ConversationService` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Chat/ConversationService.cs`) - manages conversation state and context
- `RunService` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Runs/RunService.cs`) - tracks runs with artifacts and events
- `InMemoryRunStore` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Runs/InMemoryRunStore.cs`) - in-memory storage for run data
- `AdvisoryChatIntentRouter` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Chat/Routing/AdvisoryChatIntentRouter.cs`) - routes chat intents to appropriate handlers
- `ChatEndpoints` (`src/AdvisoryAi/StellaOps.AdvisoryAI.WebService/Endpoints/ChatEndpoints.cs`) - REST endpoints for chat operations
- `RunEndpoints` (`src/AdvisoryAi/StellaOps.AdvisoryAI.WebService/Endpoints/RunEndpoints.cs`) - REST endpoints for run tracking
- `AdvisoryTaskWorker` (`src/AdvisoryAi/StellaOps.AdvisoryAI.Worker/Services/AdvisoryTaskWorker.cs`) - background worker processing advisory tasks
- **Interfaces**: `IAdvisoryPipelineOrchestrator`, `IRunService`, `IRunStore`
- **Source**: Feature matrix scan
## E2E Test Plan
- [ ] Submit a chat message via `ChatEndpoints` and verify `AdvisoryChatService` processes it with correct conversation context
- [ ] Create a run via `RunEndpoints` and verify `RunService` tracks artifacts and events in `InMemoryRunStore`
- [ ] Verify `AdvisoryChatIntentRouter` routes different intent types (explain, remediate, policy) to correct handlers
- [ ] Verify `AdvisoryPipelineOrchestrator` creates and executes task plans with `AdvisoryPipelineExecutor`
- [ ] Verify `AdvisoryTaskWorker` picks up queued tasks and processes them to completion
- [ ] Verify conversation context is maintained across multiple messages in the same session via `ConversationService`

36
docs/features/unchecked/advisoryai/advisoryai-pipeline-with-guardrails.md
View File

@@ -1,36 +0,0 @@
# AdvisoryAI Pipeline with Guardrails
## Module
AdvisoryAI
## Status
IMPLEMENTED
## Description
Full advisory AI pipeline with guardrails, chat interface, action execution, and idempotency handling. Includes retrieval, structured/vector retrievers, and SBOM context retrieval.
## Implementation Details
- **Modules**: `src/AdvisoryAi/StellaOps.AdvisoryAI/`, `src/AdvisoryAi/StellaOps.AdvisoryAI.Hosting/`
- **Key Classes**:
- `AdvisoryGuardrailPipeline` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Guardrails/AdvisoryGuardrailPipeline.cs`) - guardrail pipeline filtering AI inputs and outputs
- `AdvisoryPipelineOrchestrator` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Orchestration/AdvisoryPipelineOrchestrator.cs`) - orchestrates pipeline stages with guardrail checks
- `AdvisoryPipelineExecutor` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Execution/AdvisoryPipelineExecutor.cs`) - executes pipeline with pre/post guardrails
- `AdvisoryStructuredRetriever` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Retrievers/AdvisoryStructuredRetriever.cs`) - retrieves structured advisory data
- `AdvisoryVectorRetriever` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Retrievers/AdvisoryVectorRetriever.cs`) - vector-based semantic retrieval
- `SbomContextRetriever` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Retrievers/SbomContextRetriever.cs`) - retrieves SBOM context for vulnerability analysis
- `ActionExecutor` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ActionExecutor.cs`) - executes AI-proposed actions
- `IdempotencyHandler` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/IdempotencyHandler.cs`) - ensures idempotent action execution
- `GuardrailAllowlistLoader` (`src/AdvisoryAi/StellaOps.AdvisoryAI.Hosting/GuardrailAllowlistLoader.cs`) - loads guardrail allowlists from configuration
- `GuardrailPhraseLoader` (`src/AdvisoryAi/StellaOps.AdvisoryAI.Hosting/GuardrailPhraseLoader.cs`) - loads guardrail phrase filters
- `AdvisoryAiGuardrailOptions` (`src/AdvisoryAi/StellaOps.AdvisoryAI.Hosting/AdvisoryAiGuardrailOptions.cs`) - guardrail configuration options
- **Interfaces**: `IAdvisoryStructuredRetriever`, `IAdvisoryVectorRetriever`, `ISbomContextRetriever`, `IActionExecutor`, `IIdempotencyHandler`
- **Source**: Feature matrix scan
## E2E Test Plan
- [ ] Submit a prompt through `AdvisoryGuardrailPipeline` and verify guardrails filter prohibited content before reaching LLM
- [ ] Verify `AdvisoryStructuredRetriever` returns relevant CVE/advisory data for a given vulnerability query
- [ ] Verify `AdvisoryVectorRetriever` performs semantic search and returns ranked results
- [ ] Verify `SbomContextRetriever` enriches prompts with SBOM component context
- [ ] Execute an action through `ActionExecutor` and verify `IdempotencyHandler` prevents duplicate execution
- [ ] Verify `GuardrailAllowlistLoader` and `GuardrailPhraseLoader` correctly load and enforce content filters
- [ ] Verify the full pipeline flow: retrieval -> guardrail check -> LLM inference -> output guardrail -> response

31
docs/features/unchecked/advisoryai/ai-action-policy-gate.md
View File

@@ -1,31 +0,0 @@
# AI Action Policy Gate (K4 Lattice Governance for AI-Proposed Actions)
## Module
AdvisoryAI
## Status
IMPLEMENTED
## Description
Connects AI-proposed actions to the Policy Engine's K4 lattice for governance-aware automation. Moves beyond simple role checks to VEX-aware policy gates with approval workflows, idempotency tracking, and action audit ledger. Enables "AI that acts" with governance guardrails.
## Implementation Details
- **Modules**: `src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/`
- **Key Classes**:
- `ActionPolicyGate` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ActionPolicyGate.cs`) - evaluates AI-proposed actions against K4 lattice policy rules
- `ActionRegistry` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ActionRegistry.cs`) - registry of available AI actions with metadata and policy requirements
- `ActionExecutor` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ActionExecutor.cs`) - executes approved actions with policy gate checks
- `ActionAuditLedger` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ActionAuditLedger.cs`) - immutable audit trail of all action decisions and executions
- `ApprovalWorkflowAdapter` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ApprovalWorkflowAdapter.cs`) - integrates with approval workflows for gated actions
- `IdempotencyHandler` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/IdempotencyHandler.cs`) - ensures actions are not duplicated
- `ActionDefinition` (`src/AdvisoryAi/StellaOps.AdvisoryAI/Actions/ActionDefinition.cs`) - defines an action's capabilities, constraints, and policy metadata
- **Interfaces**: `IActionPolicyGate`, `IActionRegistry`, `IActionExecutor`, `IActionAuditLedger`, `IApprovalWorkflowAdapter`, `IIdempotencyHandler`, `IGuidGenerator`
- **Source**: SPRINT_20260109_011_004_BE_policy_action_integration.md
## E2E Test Plan
- [ ] Register an action in `ActionRegistry` and verify `ActionPolicyGate` evaluates it against K4 lattice policy rules
- [ ] Submit an action requiring approval and verify `ApprovalWorkflowAdapter` creates an approval request
- [ ] Execute a gated action after approval and verify `ActionAuditLedger` records the decision, approval, and execution
- [ ] Submit a duplicate action and verify `IdempotencyHandler` prevents re-execution
- [ ] Submit an action that violates policy and verify `ActionPolicyGate` rejects it with a policy violation reason
- [ ] Verify `ActionDefinition` metadata (risk level, required approvals, allowed scopes) is enforced during gate evaluation