feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules

- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes. - Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes. - Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables. - Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
2025-10-30 00:09:39 +02:00
parent 3154c67978
commit 7b5bdcf4d3
503 changed files with 16136 additions and 54638 deletions
--- a/docs/modules/advisory-ai/AGENTS.md
+++ b/docs/modules/advisory-ai/AGENTS.md
@@ -0,0 +1,22 @@
+# Advisory AI agent guide
+
+## Mission
+Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance.
+
+## Key docs
+- [Module README](./README.md)
+- [Architecture](./architecture.md)
+- [Implementation plan](./implementation_plan.md)
+- [Task board](./TASKS.md)
+
+## How to get started
+1. Review ./architecture.md for retrieval pipeline, guardrails, and profile support.
+2. Open ../../implplan/SPRINTS.md and locate stories for this component.
+3. Check ./TASKS.md and update status before/after work.
+4. Read README/architecture for design context and update as the implementation evolves.
+
+## Guardrails
+- Uphold Aggregation-Only Contract boundaries when consuming ingestion data.
+- Preserve determinism and provenance in all derived outputs.
+- Document offline/air-gap pathways for any new feature.
+- Update telemetry/observability assets alongside feature work.
--- a/docs/modules/advisory-ai/README.md
+++ b/docs/modules/advisory-ai/README.md
@@ -0,0 +1,29 @@
+# StellaOps Advisory AI
+
+Advisory AI is the retrieval-augmented assistant that synthesizes advisory and VEX evidence into operator-ready summaries, conflict explanations, and remediation plans with strict provenance.
+
+## Responsibilities
+- Generate policy-aware advisory summaries with citations back to Conseiller and Excititor evidence.
+- Explain conflicting advisories/VEX statements using weights from VEX Lens and Policy Engine.
+- Propose remediation hints aligned with Offline Kit staging and export bundles.
+- Expose API/UI surfaces with guardrails on model prompts, outputs, and retention.
+
+## Key components
+- RAG pipeline drawing from Conseiller, Excititor, VEX Lens, Policy Engine, and SBOM Service data.
+- Prompt templates and guard models enforcing provenance and redaction policies.
+- Vercel/offline inference workers with deterministic caching of generated artefacts.
+
+## Integrations & dependencies
+- Authority for tenant-aware access control.
+- Policy Engine for context-specific decisions and explain traces.
+- Console/CLI for interaction surfaces.
+- Export Center/Vuln Explorer for embedding generated briefs.
+
+## Operational notes
+- Model cache management and offline bundle packaging per Epic 8 requirements.
+- Usage/latency dashboards for prompt/response monitoring.
+- Redaction policies validated against security/LLM guardrail tests.
+
+## Epic alignment
+- Epic 8: Advisory AI Assistant.
+- DOCS-AI stories to be tracked in ../../TASKS.md.
--- a/docs/modules/advisory-ai/TASKS.md
+++ b/docs/modules/advisory-ai/TASKS.md
@@ -0,0 +1,9 @@
+# Task board — Advisory AI
+
+> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
+
+| ID | Status | Owner(s) | Description | Notes |
+|----|--------|----------|-------------|-------|
+| ADVISORY-AI-DOCS-0001 | TODO | Docs Guild | Ensure ./README.md reflects the latest epic deliverables. | Align with ./AGENTS.md |
+| ADVISORY-AI-ENG-0001 | TODO | Module Team | Break down epic milestones into actionable stories. | Sync into ../../TASKS.md |
+| ADVISORY-AI-OPS-0001 | TODO | Ops Guild | Prepare runbooks/observability assets once MVP lands. | Document outputs in ./README.md |
--- a/docs/modules/advisory-ai/architecture.md
+++ b/docs/modules/advisory-ai/architecture.md
@@ -0,0 +1,100 @@
+# Advisory AI architecture
+
+> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
+
+## 1) Goals
+
+- Summarise advisories/VEX evidence into operator-ready briefs with citations.
+- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
+- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
+- Operate deterministically where possible; cache generated artefacts with digests for audit.
+
+## 2) Pipeline overview
+
+```
+                       +---------------------+
+   Concelier/VEX Lens  |  Evidence Retriever |
+   Policy Engine ----> |  (vector + keyword) | ---> Context Pack (JSON)
+   Zastava runtime     +---------------------+
+                               |
+                               v
+                        +-------------+
+                        | Prompt      |
+                        | Assembler   |
+                        +-------------+
+                               |
+                               v
+                        +-------------+
+                        | Guarded LLM |
+                        | (local/host)|
+                        +-------------+
+                               |
+                               v
+                        +-----------------+
+                        | Citation &     |
+                        | Validation      |
+                        +-----------------+
+                               |
+                               v
+                        +----------------+
+                        | Output cache   |
+                        | (hash, bundle) |
+                        +----------------+
+```
+
+## 3) Retrieval & context
+
+- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
+- Context packs include:
+  - Advisory raw excerpts with highlighted sections and source URLs.
+  - VEX statements (normalized tuples + trust metadata).
+  - Policy explain traces for the affected finding.
+  - Runtime/impact hints from Zastava (exposure, entrypoints).
+  - Export-ready remediation data (fixed versions, patches).
+
+All context references include `content_hash` and `source_id` enabling verifiable citations.
+
+## 4) Guardrails
+
+- Prompt templates enforce structure: summary, conflicts, remediation, references.
+- Response validator ensures:
+  - No hallucinated advisories (every fact must map to input context).
+  - Citations follow `[n]` indexing referencing actual sources.
+  - Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
+- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
+
+## 5) Output persistence
+
+- Cached artefacts stored in `advisory_ai_outputs` with fields:
+  - `output_hash` (sha256 of JSON response).
+  - `input_digest` (hash of context pack).
+  - `summary`, `conflicts`, `remediation`, `citations`.
+  - `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
+  - `signatures` (optional DSSE if run in deterministic mode).
+- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
+
+## 6) Profiles & sovereignty
+
+- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
+- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
+
+## 7) APIs
+
+- `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`.
+- `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking.
+- `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps.
+- `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center).
+
+All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
+
+## 8) Observability
+
+- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
+- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
+- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
+
+## 9) Operational controls
+
+- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
+- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
+- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.
--- a/docs/modules/advisory-ai/implementation_plan.md
+++ b/docs/modules/advisory-ai/implementation_plan.md
@@ -0,0 +1,19 @@
+# Implementation plan — Advisory AI
+
+## Current objectives
+- Deliver Epic milestones summarised below while maintaining determinism and offline parity.
+- Keep documentation, telemetry, and runbooks aligned with sprint outcomes.
+
+## Workstreams
+- Roadmap: reconcile open stories in ../../TASKS.md with module backlog.
+- Delivery: ship features outlined in the epic while preserving AOC guardrails.
+- Validation: extend tests/fixtures to guarantee reproducibility and provenance.
+
+## Epic milestones
+- Epic 8: Advisory AI Assistant.
+- DOCS-AI stories to be tracked in ../../TASKS.md.
+
+## Coordination
+- Review ./AGENTS.md before picking up work.
+- Sync with owners listed in docs/implplan/SPRINTS.md.
+- Update this plan whenever scope, dependencies, or guardrails change.