save progress

src/AdvisoryAI/StellaOps.AdvisoryAI.Hosting/AGENTS.md

@@ -0,0 +1,48 @@
+# AdvisoryAI Hosting Agent Charter
+
+## Mission
+- Provide hosting extensions and DI registration for AdvisoryAI services.
+- Wire configuration, options validation, and infrastructure components.
+
+## Responsibilities
+- Maintain `ServiceCollectionExtensions` for clean DI registration.
+- Keep options classes (`AdvisoryAiServiceOptions`, `AdvisoryAiGuardrailOptions`) well-documented and validated.
+- Ensure file-system-based implementations (queue, cache, output store) remain deterministic and offline-safe.
+- Coordinate with guardrail phrase loading and metric wiring.
+
+## Required Reading
+- docs/README.md
+- docs/07_HIGH_LEVEL_ARCHITECTURE.md
+- docs/modules/platform/architecture-overview.md
+- docs/modules/advisory-ai/architecture.md
+- src/AdvisoryAI/AGENTS.md (parent module charter)
+- docs/policy/assistant-parameters.md (guardrail and ops knobs)
+
+## Working Directory & Scope
+- Primary: src/AdvisoryAI/StellaOps.AdvisoryAI.Hosting/
+- Dependencies: StellaOps.AdvisoryAI core library for contracts and abstractions.
+- Shared libraries allowed only when referenced by this project.
+
+## Key Components
+- `ServiceCollectionExtensions` — DI registration entry point for WebService and Worker hosts.
+- `AdvisoryAiServiceOptions` — main configuration container with nested queue/storage/inference/guardrail options.
+- `AdvisoryAiServiceOptionsValidator` — startup validation ensuring required config is present.
+- `FileSystemAdvisoryTaskQueue` — file-based task queue for offline-capable environments.
+- `FileSystemAdvisoryPlanCache` — file-based plan caching with hash keys.
+- `FileSystemAdvisoryOutputStore` — content-addressed output storage.
+- `GuardrailPhraseLoader` — loads guardrail phrases from configuration or files.
+- `AdvisoryAiMetrics` — meter and counter definitions.
+
+## Testing Expectations
+- Unit tests in `__Tests/StellaOps.AdvisoryAI.Tests` cover hosting registration paths.
+- Test options validation for missing/invalid config scenarios.
+- Verify file-system implementations with deterministic (seeded) data; no wall-clock timing.
+- Integration tests should use in-memory or temp directories to avoid flakiness.
+
+## Working Agreement
+- Determinism: stable ordering, content-addressed caches, UTC ISO-8601 timestamps.
+- Offline-friendly: no hardcoded external endpoints; respect BYO trust roots.
+- Observability: structured logs with event ids; expose counters via `AdvisoryAiMetrics`.
+- Configuration: prefer `IOptions<T>` with data annotations; validate on startup.
+- Update sprint status in docs/implplan/SPRINT_*.md when starting/completing work.
+- Mirror decisions in sprint Decisions & Risks section.

src/AdvisoryAI/StellaOps.AdvisoryAI.Hosting/FileSystemAdvisoryPlanCache.cs

@@ -161,12 +161,18 @@ internal sealed class FileSystemAdvisoryPlanCache : IAdvisoryPlanCache
 
     private static string Sanitize(string value)
     {
+        const int MaxFileNameLength = 200; // Leave room for extension and path prefixes
         var invalid = Path.GetInvalidFileNameChars();
-        var builder = new char[value.Length];
+        var builder = new char[Math.Min(value.Length, MaxFileNameLength)];
         var length = 0;
 
         foreach (var ch in value)
         {
+            if (length >= MaxFileNameLength)
+            {
+                break;
+            }
+
             builder[length++] = invalid.Contains(ch) ? '_' : ch;
         }
 

src/AdvisoryAI/StellaOps.AdvisoryAI.Hosting/FileSystemAdvisoryTaskQueue.cs

@@ -1,3 +1,4 @@
+using System.Globalization;
 using System.Linq;
 using System.Text.Json;
 using Microsoft.Extensions.Logging;
@@ -10,8 +11,12 @@ namespace StellaOps.AdvisoryAI.Hosting;
 internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
 {
     private const string FileExtension = ".json";
+    private const string QuarantineFolder = ".quarantine";
 
     private readonly string _queueDirectory;
+    private readonly string _quarantineDirectory;
+    private readonly TimeProvider _timeProvider;
+    private readonly IGuidProvider _guidProvider;
     private readonly ILogger<FileSystemAdvisoryTaskQueue> _logger;
     private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web)
     {
@@ -20,7 +25,9 @@ internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
 
     public FileSystemAdvisoryTaskQueue(
         IOptions<AdvisoryAiServiceOptions> options,
-        ILogger<FileSystemAdvisoryTaskQueue> logger)
+        ILogger<FileSystemAdvisoryTaskQueue> logger,
+        TimeProvider? timeProvider = null,
+        IGuidProvider? guidProvider = null)
     {
         ArgumentNullException.ThrowIfNull(options);
         _logger = logger ?? throw new ArgumentNullException(nameof(logger));
@@ -28,7 +35,12 @@ internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
         var serviceOptions = options.Value ?? throw new InvalidOperationException("Advisory AI options are required.");
         AdvisoryAiServiceOptionsValidator.Validate(serviceOptions);
         _queueDirectory = serviceOptions.ResolveQueueDirectory(AppContext.BaseDirectory);
+        _quarantineDirectory = Path.Combine(_queueDirectory, QuarantineFolder);
         Directory.CreateDirectory(_queueDirectory);
+        Directory.CreateDirectory(_quarantineDirectory);
+
+        _timeProvider = timeProvider ?? TimeProvider.System;
+        _guidProvider = guidProvider ?? SystemGuidProvider.Instance;
     }
 
     public async ValueTask EnqueueAsync(AdvisoryTaskQueueMessage message, CancellationToken cancellationToken)
@@ -38,7 +50,9 @@ internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
         var envelope = FileQueueEnvelope.FromMessage(message);
         var payload = JsonSerializer.Serialize(envelope, _serializerOptions);
 
-        var fileName = $"{DateTimeOffset.UtcNow:yyyyMMddTHHmmssfff}_{Guid.NewGuid():N}{FileExtension}";
+        var timestamp = _timeProvider.GetUtcNow().ToString("yyyyMMddTHHmmssfff", CultureInfo.InvariantCulture);
+        var guid = _guidProvider.NewGuid().ToString("N", CultureInfo.InvariantCulture);
+        var fileName = $"{timestamp}_{guid}{FileExtension}";
         var tempPath = Path.Combine(_queueDirectory, $"{fileName}.tmp");
         var targetPath = Path.Combine(_queueDirectory, fileName);
 
@@ -60,6 +74,7 @@ internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
             foreach (var file in files)
             {
                 AdvisoryTaskQueueMessage? message = null;
+                var shouldQuarantine = false;
 
                 try
                 {
@@ -73,12 +88,19 @@ internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
                 catch (IOException)
                 {
                     // File locked by another process; skip and retry.
+                    continue;
                 }
                 catch (JsonException ex)
                 {
-                    _logger.LogWarning(ex, "Failed to deserialize advisory task queue file {File}", file);
+                    _logger.LogWarning(ex, "Failed to deserialize advisory task queue file {File}; moving to quarantine", file);
+                    shouldQuarantine = true;
                 }
-                finally
+
+                if (shouldQuarantine)
+                {
+                    TryQuarantine(file);
+                }
+                else
                 {
                     TryDelete(file);
                 }
@@ -108,6 +130,22 @@ internal sealed class FileSystemAdvisoryTaskQueue : IAdvisoryTaskQueue
         }
     }
 
+    private void TryQuarantine(string path)
+    {
+        try
+        {
+            var fileName = Path.GetFileName(path);
+            var quarantinePath = Path.Combine(_quarantineDirectory, fileName);
+            File.Move(path, quarantinePath, overwrite: true);
+            _logger.LogDebug("Moved corrupt queue file {File} to quarantine", path);
+        }
+        catch (IOException ex)
+        {
+            _logger.LogDebug(ex, "Failed to quarantine queue file {File}; attempting delete", path);
+            TryDelete(path);
+        }
+    }
+
     private sealed record FileQueueEnvelope(string PlanCacheKey, AdvisoryTaskRequestEnvelope Request)
     {
         public static FileQueueEnvelope FromMessage(AdvisoryTaskQueueMessage message)

src/AdvisoryAI/StellaOps.AdvisoryAI.Hosting/IGuidProvider.cs

@@ -0,0 +1,26 @@
+namespace StellaOps.AdvisoryAI.Hosting;
+
+/// <summary>
+/// Abstraction for GUID generation to support deterministic testing.
+/// </summary>
+public interface IGuidProvider
+{
+    /// <summary>
+    /// Generates a new GUID.
+    /// </summary>
+    Guid NewGuid();
+}
+
+/// <summary>
+/// Default implementation using <see cref="Guid.NewGuid"/>.
+/// </summary>
+public sealed class SystemGuidProvider : IGuidProvider
+{
+    /// <summary>
+    /// Shared instance.
+    /// </summary>
+    public static readonly SystemGuidProvider Instance = new();
+
+    /// <inheritdoc />
+    public Guid NewGuid() => Guid.NewGuid();
+}

src/AdvisoryAI/StellaOps.AdvisoryAI.Hosting/ServiceCollectionExtensions.cs

@@ -97,6 +97,10 @@ public static class ServiceCollectionExtensions
                 ApplyGuardrailConfiguration(options, aiOptions.Value.Guardrails, environment);
             });
 
+        // Register deterministic providers (allow test injection)
+        services.TryAddSingleton<IGuidProvider>(SystemGuidProvider.Instance);
+        services.TryAddSingleton(TimeProvider.System);
+
         services.Replace(ServiceDescriptor.Singleton<IAdvisoryTaskQueue, FileSystemAdvisoryTaskQueue>());
         services.Replace(ServiceDescriptor.Singleton<IAdvisoryPlanCache, FileSystemAdvisoryPlanCache>());
         services.Replace(ServiceDescriptor.Singleton<IAdvisoryOutputStore, FileSystemAdvisoryOutputStore>());

src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/AGENTS.md

@@ -0,0 +1,64 @@
+# AdvisoryAI WebService Agent Charter
+
+## Mission
+- Expose HTTP API endpoints for Advisory AI interactions.
+- Handle request validation, rate limiting, and response formatting.
+- Coordinate with consent, justification, and orchestration services.
+
+## Responsibilities
+- Maintain API endpoint definitions in Program.cs (minimal APIs).
+- Keep request/response contracts stable and documented.
+- Enforce rate limiting, consent checks, and proper error handling.
+- Wire hosting extensions and router integration.
+
+## Required Reading
+- docs/README.md
+- docs/07_HIGH_LEVEL_ARCHITECTURE.md
+- docs/modules/platform/architecture-overview.md
+- docs/modules/advisory-ai/architecture.md
+- src/AdvisoryAI/AGENTS.md (parent module charter)
+- docs/policy/assistant-parameters.md (guardrail and ops knobs)
+- docs/modules/advisory-ai/deployment.md (service configuration)
+
+## Working Directory & Scope
+- Primary: src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/
+- Dependencies: StellaOps.AdvisoryAI, StellaOps.AdvisoryAI.Hosting
+- Shared libraries: Router.AspNet for Stella Router integration
+
+## Key Components
+- `Program.cs` — WebApplication setup, endpoint mapping, middleware pipeline.
+- `Contracts/` — Request/response DTOs for API endpoints:
+  - `AdvisoryPlanRequest/Response` — plan generation
+  - `AdvisoryExecuteRequest` — execution trigger
+  - `AdvisoryQueueRequest/Response` — queue management
+  - `ExplainRequest/Response` — explanation endpoints
+  - `ConsentContracts` — AI consent management (VEX-AI-016)
+  - `JustifyContracts` — justification generation
+  - `PolicyStudioContracts` — policy studio integration
+  - `RemediationContracts` — remediation plan endpoints
+- `Services/` — Service implementations:
+  - `IAiConsentStore` / `InMemoryAiConsentStore` — consent tracking
+  - `IAiJustificationGenerator` / `DefaultAiJustificationGenerator` — justification generation
+
+## API Endpoints
+- POST /api/advisory/plan — Generate advisory plan
+- POST /api/advisory/execute — Execute advisory plan
+- POST /api/advisory/queue — Queue advisory task
+- GET /api/advisory/output/{id} — Retrieve advisory output
+- POST /api/advisory/explain — Generate explanation
+- Consent and justification endpoints per VEX-AI-016
+
+## Testing Expectations
+- Unit tests in `__Tests/StellaOps.AdvisoryAI.Tests` cover endpoint logic.
+- Integration tests use WebApplicationFactory for full pipeline testing.
+- Test rate limiting behavior, consent enforcement, and error responses.
+- Verify request validation and contract serialization.
+
+## Working Agreement
+- Determinism: stable response ordering, content-addressed output IDs.
+- Offline-friendly: endpoints must degrade gracefully when inference is unavailable.
+- Observability: structured logs with request correlation ids; expose rate limiter metrics.
+- Configuration: bind from appsettings.json and environment variables (ADVISORYAI__ prefix).
+- Security: validate all input, enforce consent where required, no embedding secrets.
+- Update sprint status in docs/implplan/SPRINT_*.md when starting/completing work.
+- Mirror decisions in sprint Decisions & Risks section.

src/AdvisoryAI/StellaOps.AdvisoryAI.Worker/AGENTS.md

@@ -0,0 +1,59 @@
+# AdvisoryAI Worker Agent Charter
+
+## Mission
+- Execute queued advisory tasks asynchronously as a background service.
+- Process advisory plans, orchestrate pipeline execution, and record metrics.
+
+## Responsibilities
+- Maintain `AdvisoryTaskWorker` background service for queue consumption.
+- Coordinate plan creation/caching and pipeline execution.
+- Track metrics for plan creation, execution latency, and cache hit rates.
+- Handle graceful shutdown and task cancellation.
+
+## Required Reading
+- docs/README.md
+- docs/07_HIGH_LEVEL_ARCHITECTURE.md
+- docs/modules/platform/architecture-overview.md
+- docs/modules/advisory-ai/architecture.md
+- docs/modules/advisory-ai/orchestration-pipeline.md
+- src/AdvisoryAI/AGENTS.md (parent module charter)
+- docs/policy/assistant-parameters.md (guardrail and ops knobs)
+
+## Working Directory & Scope
+- Primary: src/AdvisoryAI/StellaOps.AdvisoryAI.Worker/
+- Dependencies: StellaOps.AdvisoryAI, StellaOps.AdvisoryAI.Hosting
+- Coordinates with: WebService (queues tasks), core orchestrator (executes plans)
+
+## Key Components
+- `Program.cs` — Host builder setup with AdvisoryAI core and hosted service registration.
+- `Services/AdvisoryTaskWorker.cs` — BackgroundService that:
+  - Dequeues tasks from `IAdvisoryTaskQueue`
+  - Checks `IAdvisoryPlanCache` for existing plans
+  - Creates new plans via `IAdvisoryPipelineOrchestrator`
+  - Executes plans via `IAdvisoryPipelineExecutor`
+  - Records metrics via `AdvisoryPipelineMetrics`
+  - Uses injected `TimeProvider` for deterministic timing
+
+## Processing Flow
+1. Dequeue message from `IAdvisoryTaskQueue`
+2. Check plan cache by `PlanCacheKey` (unless `ForceRefresh`)
+3. If cache miss, create plan via orchestrator and cache it
+4. Execute plan via executor
+5. Record metrics and log completion
+6. Loop until cancellation
+
+## Testing Expectations
+- Unit tests in `__Tests/StellaOps.AdvisoryAI.Tests` cover worker logic.
+- Test with mocked queue, cache, orchestrator, and executor.
+- Verify cancellation handling and graceful shutdown.
+- Test cache hit/miss scenarios and ForceRefresh behavior.
+- Use FakeTimeProvider for deterministic timing tests.
+
+## Working Agreement
+- Determinism: use injected TimeProvider, stable plan caching keys, ordered execution.
+- Offline-friendly: worker must handle unavailable inference gracefully.
+- Observability: structured logs with activity tracing, expose pipeline metrics.
+- Configuration: bind from appsettings.json and environment variables (ADVISORYAI__ prefix).
+- Queue/cache: respect bounded capacities and TTLs configured via options.
+- Update sprint status in docs/implplan/SPRINT_*.md when starting/completing work.
+- Mirror decisions in sprint Decisions & Risks section.