Refactor code structure and optimize performance across multiple modules
This commit is contained in:
StellaOps Bot
397
docs/modules/advisory-ai/guides/explanation-api.md
Normal file
397
docs/modules/advisory-ai/guides/explanation-api.md
Normal file
@@ -0,0 +1,397 @@
|
||||
# Explanation API and Replay Semantics
|
||||
|
||||
> **Sprint:** SPRINT_20251226_015_AI_zastava_companion
|
||||
> **Task:** ZASTAVA-21
|
||||
|
||||
This guide documents the Zastava Companion explanation API, attestation format, and replay semantics for evidence-grounded AI explanations.
|
||||
|
||||
## Overview
|
||||
|
||||
The Explanation API provides evidence-anchored explanations answering:
|
||||
- **What** is this vulnerability?
|
||||
- **Why** does it matter in this context?
|
||||
- **Evidence**: What supports exploitability?
|
||||
- **Counterfactual**: What would change the verdict?
|
||||
|
||||
All explanations are anchored to verifiable evidence nodes (SBOM, reachability, runtime, VEX, patches).
|
||||
|
||||
## Explanation Types
|
||||
|
||||
| Type | Purpose | Example Output |
|
||||
|------|---------|----------------|
|
||||
| `What` | Technical description | "CVE-2024-1234 is a remote code execution vulnerability in lodash's merge function..." |
|
||||
| `Why` | Contextual relevance | "This matters because your service uses lodash@4.17.20 in the request handler path..." |
|
||||
| `Evidence` | Exploitability proof | "Reachability analysis shows the vulnerable function is called from /api/users endpoint..." |
|
||||
| `Counterfactual` | Verdict change conditions | "The verdict would change to 'not affected' if the VEX statement confirmed non-exploitability..." |
|
||||
| `Full` | Comprehensive explanation | All of the above in a structured format |
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Generate Explanation
|
||||
|
||||
```http
|
||||
POST /api/v1/advisory-ai/explain
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"findingId": "finding-abc123",
|
||||
"artifactDigest": "sha256:abcdef...",
|
||||
"scope": "service",
|
||||
"scopeId": "payment-service",
|
||||
"explanationType": "Full",
|
||||
"vulnerabilityId": "CVE-2024-1234",
|
||||
"componentPurl": "pkg:npm/lodash@4.17.20",
|
||||
"plainLanguage": true,
|
||||
"maxLength": 2000
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"explanationId": "expl-20251226-001",
|
||||
"content": "## What is CVE-2024-1234?\n\nCVE-2024-1234 is a critical remote code execution vulnerability...[1]\n\n## Why It Matters\n\nYour payment-service uses lodash@4.17.20 which is affected...[2]\n\n## Evidence\n\n- Reachability: The vulnerable `merge()` function is called from `/api/checkout`...[3]\n- Runtime: No WAF protection detected for this endpoint...[4]\n\n## What Would Change the Verdict\n\nThe verdict would change to 'not affected' if:\n- A VEX statement confirms non-exploitability...[5]\n- The function call is removed from the code path...[6]",
|
||||
"summary": {
|
||||
"line1": "Critical RCE in lodash affecting payment-service",
|
||||
"line2": "Reachable via /api/checkout with no WAF protection",
|
||||
"line3": "Upgrade to lodash@4.17.21 or add VEX exception"
|
||||
},
|
||||
"citations": [
|
||||
{
|
||||
"claimText": "CVE-2024-1234 is a critical remote code execution vulnerability",
|
||||
"evidenceId": "nvd:CVE-2024-1234",
|
||||
"evidenceType": "advisory",
|
||||
"verified": true,
|
||||
"evidenceExcerpt": "CVSS: 9.8 CRITICAL - Improper input validation in lodash merge..."
|
||||
},
|
||||
{
|
||||
"claimText": "payment-service uses lodash@4.17.20",
|
||||
"evidenceId": "sbom:payment-service:lodash@4.17.20",
|
||||
"evidenceType": "sbom",
|
||||
"verified": true,
|
||||
"evidenceExcerpt": "Component: lodash, Version: 4.17.20, Location: node_modules/lodash"
|
||||
},
|
||||
{
|
||||
"claimText": "vulnerable merge() function is called from /api/checkout",
|
||||
"evidenceId": "reach:payment-service:lodash.merge:/api/checkout",
|
||||
"evidenceType": "reachability",
|
||||
"verified": true,
|
||||
"evidenceExcerpt": "Call path: checkout.js:42 -> utils.js:15 -> lodash.merge()"
|
||||
}
|
||||
],
|
||||
"confidenceScore": 0.92,
|
||||
"citationRate": 0.85,
|
||||
"authority": "EvidenceBacked",
|
||||
"evidenceRefs": [
|
||||
"nvd:CVE-2024-1234",
|
||||
"sbom:payment-service:lodash@4.17.20",
|
||||
"reach:payment-service:lodash.merge:/api/checkout",
|
||||
"runtime:payment-service:waf:none"
|
||||
],
|
||||
"modelId": "claude-sonnet-4-20250514",
|
||||
"promptTemplateVersion": "v2.1.0",
|
||||
"inputHashes": [
|
||||
"sha256:abc123...",
|
||||
"sha256:def456..."
|
||||
],
|
||||
"generatedAt": "2025-12-26T10:30:00Z",
|
||||
"outputHash": "sha256:789xyz..."
|
||||
}
|
||||
```
|
||||
|
||||
### Replay Explanation
|
||||
|
||||
Re-runs the explanation with identical inputs to verify determinism.
|
||||
|
||||
```http
|
||||
GET /api/v1/advisory-ai/explain/{explanationId}/replay
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"original": { "...original explanation..." },
|
||||
"replayed": { "...replayed explanation..." },
|
||||
"identical": true,
|
||||
"similarity": 1.0,
|
||||
"divergenceDetails": null
|
||||
}
|
||||
```
|
||||
|
||||
### Get Explanation
|
||||
|
||||
```http
|
||||
GET /api/v1/advisory-ai/explain/{explanationId}
|
||||
```
|
||||
|
||||
### Validate Explanation
|
||||
|
||||
```http
|
||||
POST /api/v1/advisory-ai/explain/{explanationId}/validate
|
||||
```
|
||||
|
||||
Validates that the explanation's input hashes still match current evidence.
|
||||
|
||||
## Evidence Types
|
||||
|
||||
| Type | Source | Description |
|
||||
|------|--------|-------------|
|
||||
| `advisory` | NVD, GHSA, vendor | Vulnerability advisory data |
|
||||
| `sbom` | Container scan | Software bill of materials component |
|
||||
| `reachability` | Call graph analysis | Function reachability proof |
|
||||
| `runtime` | Signals service | Runtime observations (WAF, network) |
|
||||
| `vex` | VEX documents | Vendor exploitability statements |
|
||||
| `patch` | Package registry | Available fix information |
|
||||
|
||||
## Authority Classification
|
||||
|
||||
Explanations are classified by their evidence backing:
|
||||
|
||||
| Authority | Criteria | Display |
|
||||
|-----------|----------|---------|
|
||||
| `EvidenceBacked` | ≥80% citation rate, all citations verified | Green badge: "Evidence-backed" |
|
||||
| `Suggestion` | <80% citation rate or unverified citations | Yellow badge: "AI suggestion" |
|
||||
|
||||
```csharp
|
||||
public enum ExplanationAuthority
|
||||
{
|
||||
EvidenceBacked, // All claims anchored to verified evidence
|
||||
Suggestion // AI suggestion requiring human review
|
||||
}
|
||||
```
|
||||
|
||||
## Attestation Format
|
||||
|
||||
Explanations are wrapped in DSSE (Dead Simple Signing Envelope) attestations:
|
||||
|
||||
### Predicate Type
|
||||
|
||||
```
|
||||
https://stellaops.org/attestation/ai-explanation/v1
|
||||
```
|
||||
|
||||
### Predicate Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"_type": "https://stellaops.org/attestation/ai-explanation/v1",
|
||||
"explanationId": "expl-20251226-001",
|
||||
"explanationType": "Full",
|
||||
"authority": "EvidenceBacked",
|
||||
"finding": {
|
||||
"findingId": "finding-abc123",
|
||||
"vulnerabilityId": "CVE-2024-1234",
|
||||
"componentPurl": "pkg:npm/lodash@4.17.20"
|
||||
},
|
||||
"model": {
|
||||
"modelId": "claude-sonnet-4-20250514",
|
||||
"promptTemplateVersion": "v2.1.0"
|
||||
},
|
||||
"inputs": {
|
||||
"inputHashes": ["sha256:abc123...", "sha256:def456..."],
|
||||
"evidenceRefs": ["nvd:CVE-2024-1234", "sbom:..."]
|
||||
},
|
||||
"output": {
|
||||
"contentHash": "sha256:789xyz...",
|
||||
"confidenceScore": 0.92,
|
||||
"citationRate": 0.85,
|
||||
"citationCount": 6
|
||||
},
|
||||
"generatedAt": "2025-12-26T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### DSSE Envelope
|
||||
|
||||
```json
|
||||
{
|
||||
"payloadType": "application/vnd.stellaops.ai-explanation+json",
|
||||
"payload": "<base64-encoded-predicate>",
|
||||
"signatures": [
|
||||
{
|
||||
"keyId": "stellaops-ai-signer-2025",
|
||||
"sig": "<base64-signature>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### OCI Attachment
|
||||
|
||||
Attestations are pushed as OCI referrers:
|
||||
|
||||
```
|
||||
Artifact: sha256:imagedigest
|
||||
└── Referrer: application/vnd.stellaops.ai-explanation+json
|
||||
└── expl-20251226-001.dsse.json
|
||||
```
|
||||
|
||||
## Replay Semantics
|
||||
|
||||
### Replay Manifest
|
||||
|
||||
Every explanation includes a replay manifest enabling deterministic reproduction:
|
||||
|
||||
```json
|
||||
{
|
||||
"manifestVersion": "1.0",
|
||||
"explanationId": "expl-20251226-001",
|
||||
"model": {
|
||||
"modelId": "claude-sonnet-4-20250514",
|
||||
"weightsDigest": "sha256:modelweights...",
|
||||
"promptTemplateVersion": "v2.1.0"
|
||||
},
|
||||
"inputs": {
|
||||
"findingId": "finding-abc123",
|
||||
"artifactDigest": "sha256:abcdef...",
|
||||
"evidenceHashes": {
|
||||
"advisory": "sha256:111...",
|
||||
"sbom": "sha256:222...",
|
||||
"reachability": "sha256:333..."
|
||||
}
|
||||
},
|
||||
"parameters": {
|
||||
"temperature": 0.0,
|
||||
"seed": 42,
|
||||
"maxTokens": 4096
|
||||
},
|
||||
"output": {
|
||||
"contentHash": "sha256:789xyz...",
|
||||
"generatedAt": "2025-12-26T10:30:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Determinism Requirements
|
||||
|
||||
For replay to produce identical output:
|
||||
|
||||
| Parameter | Required Value | Purpose |
|
||||
|-----------|---------------|---------|
|
||||
| `temperature` | `0.0` | No randomness in generation |
|
||||
| `seed` | `42` (fixed) | Reproducible sampling |
|
||||
| `maxTokens` | Same as original | Consistent truncation |
|
||||
| Model version | Exact match | Same weights |
|
||||
| Prompt template | Exact match | Same prompt structure |
|
||||
|
||||
### Divergence Detection
|
||||
|
||||
When replay produces different output:
|
||||
|
||||
```json
|
||||
{
|
||||
"diverged": true,
|
||||
"similarity": 0.94,
|
||||
"originalHash": "sha256:789xyz...",
|
||||
"replayedHash": "sha256:different...",
|
||||
"divergencePoints": [
|
||||
{
|
||||
"position": 1234,
|
||||
"original": "...uses lodash@4.17.20...",
|
||||
"replayed": "...uses lodash version 4.17.20..."
|
||||
}
|
||||
],
|
||||
"likelyCause": "model_update"
|
||||
}
|
||||
```
|
||||
|
||||
### Divergence Causes
|
||||
|
||||
| Cause | Detection | Resolution |
|
||||
|-------|-----------|------------|
|
||||
| Model update | Weights digest mismatch | Pin model version |
|
||||
| Non-zero temperature | Parameter check | Set temperature=0 |
|
||||
| Evidence change | Input hash mismatch | Re-generate explanation |
|
||||
| Prompt template change | Template version mismatch | Pin template version |
|
||||
|
||||
## CLI Commands
|
||||
|
||||
```bash
|
||||
# Generate explanation
|
||||
stella advisory explain finding-abc123 \
|
||||
--type full \
|
||||
--plain-language \
|
||||
--attest --sign
|
||||
|
||||
# Replay explanation
|
||||
stella advisory replay expl-20251226-001
|
||||
|
||||
# Verify explanation attestation
|
||||
stella attest verify expl-20251226-001.dsse.json
|
||||
|
||||
# Check for divergence
|
||||
stella advisory replay expl-20251226-001 --detect-divergence
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
```yaml
|
||||
advisoryAi:
|
||||
explanation:
|
||||
# Default explanation type
|
||||
defaultType: Full
|
||||
|
||||
# Plain language by default
|
||||
plainLanguage: true
|
||||
|
||||
# Maximum explanation length
|
||||
maxLength: 4000
|
||||
|
||||
# Minimum citation rate for EvidenceBacked authority
|
||||
minCitationRate: 0.80
|
||||
|
||||
# Generate attestation for each explanation
|
||||
generateAttestation: true
|
||||
|
||||
# Sign attestations
|
||||
signAttestation: true
|
||||
|
||||
# Determinism settings for replay
|
||||
inference:
|
||||
temperature: 0.0
|
||||
seed: 42
|
||||
maxTokens: 4096
|
||||
```
|
||||
|
||||
## 3-Line Summary Format
|
||||
|
||||
Every explanation includes a 3-line summary following the AI UX pattern:
|
||||
|
||||
| Line | Purpose | Example |
|
||||
|------|---------|---------|
|
||||
| Line 1 | What changed / what is it | "Critical RCE in lodash affecting payment-service" |
|
||||
| Line 2 | Why it matters | "Reachable via /api/checkout with no WAF protection" |
|
||||
| Line 3 | Next action | "Upgrade to lodash@4.17.21 or add VEX exception" |
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Generation Errors
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "evidence_retrieval_failed",
|
||||
"message": "Unable to retrieve SBOM for artifact sha256:abc...",
|
||||
"recoverable": true,
|
||||
"suggestion": "Ensure the artifact has been scanned before requesting explanation"
|
||||
}
|
||||
```
|
||||
|
||||
### Validation Errors
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "citation_verification_failed",
|
||||
"message": "Citation [2] references evidence that no longer exists",
|
||||
"invalidCitations": ["sbom:payment-service:lodash@4.17.20"],
|
||||
"suggestion": "Re-generate explanation with current evidence"
|
||||
}
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [AI Attestations](./ai-attestations.md)
|
||||
- [LLM Provider Plugins](./llm-provider-plugins.md)
|
||||
- [Offline Model Bundles](./offline-model-bundles.md)
|
||||
- [Advisory AI Architecture](../architecture.md)
|
||||
Reference in New Issue
Block a user