Refactor code structure and optimize performance across multiple modules
This commit is contained in:
StellaOps Bot
373
docs/modules/advisory-ai/guides/ai-attestations.md
Normal file
373
docs/modules/advisory-ai/guides/ai-attestations.md
Normal file
@@ -0,0 +1,373 @@
|
||||
# AI Attestations and Replay Semantics
|
||||
|
||||
> **Sprint:** SPRINT_20251226_018_AI_attestations
|
||||
> **Task:** AIATTEST-23
|
||||
|
||||
This guide documents the AI attestation schemas, authority classification, and deterministic replay semantics.
|
||||
|
||||
## Overview
|
||||
|
||||
AI-generated artifacts in StellaOps are wrapped in cryptographic attestations that:
|
||||
1. Capture the exact inputs (prompts, context, model parameters)
|
||||
2. Prove the generation chain (model ID, weights digest, configuration)
|
||||
3. Enable deterministic replay for compliance verification
|
||||
4. Support divergence detection across environments
|
||||
|
||||
## Attestation Types
|
||||
|
||||
### AI Artifact Predicate
|
||||
|
||||
```json
|
||||
{
|
||||
"_type": "https://stellaops.org/attestation/ai-artifact/v1",
|
||||
"artifactId": "ai-artifact-20251226-001",
|
||||
"artifactType": "explanation",
|
||||
"authority": "ai-generated",
|
||||
"generatedAt": "2025-12-26T10:30:00Z",
|
||||
"model": {
|
||||
"modelId": "llama3-8b-q4km",
|
||||
"weightsDigest": "sha256:a1b2c3...",
|
||||
"promptTemplateVersion": "v2.1.0"
|
||||
},
|
||||
"inputs": {
|
||||
"systemPromptHash": "sha256:abc123...",
|
||||
"userPromptHash": "sha256:def456...",
|
||||
"contextHashes": ["sha256:111...", "sha256:222..."]
|
||||
},
|
||||
"parameters": {
|
||||
"temperature": 0.0,
|
||||
"seed": 42,
|
||||
"maxTokens": 2048,
|
||||
"topK": 1
|
||||
},
|
||||
"output": {
|
||||
"contentHash": "sha256:789xyz...",
|
||||
"tokenCount": 847
|
||||
},
|
||||
"replayManifest": {
|
||||
"manifestId": "replay-20251226-001",
|
||||
"manifestHash": "sha256:manifest..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Artifact Types
|
||||
|
||||
| Type | Description | Authority |
|
||||
|------|-------------|-----------|
|
||||
| `explanation` | Vulnerability explanation for humans | `ai-generated` |
|
||||
| `remediation` | Fix plan with upgrade paths | `ai-generated` |
|
||||
| `vex_draft` | Draft VEX statement | `ai-draft-requires-review` |
|
||||
| `policy_draft` | Draft policy rules | `ai-draft-requires-review` |
|
||||
| `triage_suggestion` | Triage action suggestions | `ai-suggestion` |
|
||||
|
||||
### Authority Classification
|
||||
|
||||
AI outputs are classified by their authority level:
|
||||
|
||||
```
|
||||
ai-generated → Informational only, human review optional
|
||||
ai-draft-requires-review → Draft requires explicit human approval
|
||||
ai-suggestion → Suggestion, user decides action
|
||||
ai-verified → AI output verified against ground truth
|
||||
human-approved → AI output approved by human reviewer
|
||||
```
|
||||
|
||||
## Replay Manifest
|
||||
|
||||
The replay manifest captures everything needed to reproduce an AI generation:
|
||||
|
||||
```json
|
||||
{
|
||||
"manifestVersion": "1.0",
|
||||
"artifactId": "ai-artifact-20251226-001",
|
||||
"artifactType": "explanation",
|
||||
|
||||
"model": {
|
||||
"modelId": "llama3-8b-q4km",
|
||||
"weightsDigest": "sha256:a1b2c3d4e5f6...",
|
||||
"promptTemplateVersion": "v2.1.0"
|
||||
},
|
||||
|
||||
"prompts": {
|
||||
"systemPrompt": "You are a security analyst...",
|
||||
"userPrompt": "Explain CVE-2024-1234 affecting lodash@4.17.20...",
|
||||
"systemPromptHash": "sha256:abc123...",
|
||||
"userPromptHash": "sha256:def456..."
|
||||
},
|
||||
|
||||
"context": {
|
||||
"contextPack": [...],
|
||||
"contextHashes": ["sha256:111...", "sha256:222..."]
|
||||
},
|
||||
|
||||
"parameters": {
|
||||
"temperature": 0.0,
|
||||
"seed": 42,
|
||||
"maxTokens": 2048,
|
||||
"topK": 1,
|
||||
"topP": 1.0
|
||||
},
|
||||
|
||||
"output": {
|
||||
"content": "CVE-2024-1234 is a critical vulnerability...",
|
||||
"contentHash": "sha256:789xyz...",
|
||||
"tokenCount": 847
|
||||
},
|
||||
|
||||
"metadata": {
|
||||
"generatedAt": "2025-12-26T10:30:00Z",
|
||||
"replayable": true,
|
||||
"deterministicSettings": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Deterministic Requirements
|
||||
|
||||
For an AI artifact to be replayable:
|
||||
|
||||
1. **Temperature must be 0**: No randomness in token selection
|
||||
2. **Seed must be fixed**: Same seed across replays (default: 42)
|
||||
3. **Model weights must match**: Verified by weights digest
|
||||
4. **Prompts must match**: Verified by prompt hashes
|
||||
5. **Context must match**: All input hashes must verify
|
||||
|
||||
### Configuration for Determinism
|
||||
|
||||
```yaml
|
||||
advisoryAi:
|
||||
attestations:
|
||||
requireDeterminism: true
|
||||
defaultSeed: 42
|
||||
|
||||
inference:
|
||||
local:
|
||||
temperature: 0.0
|
||||
seed: 42
|
||||
topK: 1
|
||||
topP: 1.0
|
||||
```
|
||||
|
||||
## Replay Workflow
|
||||
|
||||
### Replay Execution
|
||||
|
||||
```csharp
|
||||
// Load replay manifest
|
||||
var manifest = await LoadManifestAsync("replay-20251226-001.json");
|
||||
|
||||
// Create replayer with same model
|
||||
var replayer = replayerFactory.Create(manifest.Model.ModelId);
|
||||
|
||||
// Execute replay
|
||||
var result = await replayer.ReplayAsync(manifest, cancellationToken);
|
||||
|
||||
// Check if output is identical
|
||||
if (result.Identical)
|
||||
{
|
||||
Console.WriteLine("Replay successful: output matches original");
|
||||
}
|
||||
else
|
||||
{
|
||||
Console.WriteLine($"Divergence detected: similarity = {result.SimilarityScore:P2}");
|
||||
}
|
||||
```
|
||||
|
||||
### Divergence Detection
|
||||
|
||||
When replay produces different output:
|
||||
|
||||
```json
|
||||
{
|
||||
"diverged": true,
|
||||
"similarityScore": 0.97,
|
||||
"originalHash": "sha256:789xyz...",
|
||||
"replayedHash": "sha256:different...",
|
||||
"details": [
|
||||
{
|
||||
"type": "content_divergence",
|
||||
"description": "Content differs at position",
|
||||
"position": 1842,
|
||||
"originalSnippet": "...vulnerability allows...",
|
||||
"replayedSnippet": "...vulnerability permits..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Common Divergence Causes
|
||||
|
||||
| Cause | Detection | Resolution |
|
||||
|-------|-----------|------------|
|
||||
| Different model weights | Weights digest mismatch | Use exact model version |
|
||||
| Non-zero temperature | Parameter check | Set temperature to 0 |
|
||||
| Different seed | Parameter check | Use same seed |
|
||||
| Prompt template change | Template version mismatch | Pin template version |
|
||||
| Context ordering | Context hash mismatch | Sort context deterministically |
|
||||
|
||||
## Attestation Signing
|
||||
|
||||
### DSSE Envelope Format
|
||||
|
||||
AI attestations use DSSE (Dead Simple Signing Envelope):
|
||||
|
||||
```json
|
||||
{
|
||||
"payloadType": "application/vnd.stellaops.ai-attestation+json",
|
||||
"payload": "<base64-encoded-attestation>",
|
||||
"signatures": [
|
||||
{
|
||||
"keyId": "stellaops-ai-signer-2025",
|
||||
"sig": "<base64-signature>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Signing Configuration
|
||||
|
||||
```yaml
|
||||
advisoryAi:
|
||||
attestations:
|
||||
sign: true
|
||||
keyId: "stellaops-ai-signer-2025"
|
||||
cryptoScheme: ed25519 # ed25519 | ecdsa-p256 | gost3410 | sm2
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Generate with Attestation
|
||||
|
||||
```http
|
||||
POST /api/v1/advisory/explain
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"findingId": "finding-123",
|
||||
"artifactDigest": "sha256:...",
|
||||
"options": {
|
||||
"generateAttestation": true,
|
||||
"signAttestation": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Response includes:
|
||||
|
||||
```json
|
||||
{
|
||||
"explanation": "...",
|
||||
"attestation": {
|
||||
"predicateType": "https://stellaops.org/attestation/ai-artifact/v1",
|
||||
"predicate": {...},
|
||||
"signature": {...}
|
||||
},
|
||||
"replayManifestId": "replay-20251226-001"
|
||||
}
|
||||
```
|
||||
|
||||
### Verify Attestation
|
||||
|
||||
```http
|
||||
POST /api/v1/attestation/verify
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"attestation": {...},
|
||||
"options": {
|
||||
"verifySignature": true,
|
||||
"verifyReplay": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Replay Artifact
|
||||
|
||||
```http
|
||||
POST /api/v1/advisory/replay
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"manifestId": "replay-20251226-001"
|
||||
}
|
||||
```
|
||||
|
||||
## CLI Commands
|
||||
|
||||
```bash
|
||||
# Generate explanation with attestation
|
||||
stella advisory explain finding-123 --attest --sign
|
||||
|
||||
# Verify attestation
|
||||
stella attest verify ai-artifact-20251226-001.dsse.json
|
||||
|
||||
# Replay from manifest
|
||||
stella advisory replay --manifest replay-20251226-001.json
|
||||
|
||||
# Check divergence
|
||||
stella advisory replay --manifest replay-20251226-001.json --detect-divergence
|
||||
```
|
||||
|
||||
## Storage and Retrieval
|
||||
|
||||
### Attestation Storage
|
||||
|
||||
Attestations are stored in the Evidence Locker:
|
||||
|
||||
```
|
||||
/evidence/ai-attestations/
|
||||
├── 2025/12/26/
|
||||
│ ├── ai-artifact-20251226-001.json
|
||||
│ ├── ai-artifact-20251226-001.dsse.json
|
||||
│ └── replay-20251226-001.json
|
||||
```
|
||||
|
||||
### Retrieval
|
||||
|
||||
```http
|
||||
GET /api/v1/attestation/ai-artifact-20251226-001
|
||||
|
||||
# Returns attestation + replay manifest
|
||||
```
|
||||
|
||||
## Audit Trail
|
||||
|
||||
AI operations are logged for compliance:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2025-12-26T10:30:00Z",
|
||||
"operation": "ai_generation",
|
||||
"artifactId": "ai-artifact-20251226-001",
|
||||
"artifactType": "explanation",
|
||||
"modelId": "llama3-8b-q4km",
|
||||
"authority": "ai-generated",
|
||||
"user": "system",
|
||||
"inputHashes": ["sha256:..."],
|
||||
"outputHash": "sha256:...",
|
||||
"signed": true,
|
||||
"replayable": true
|
||||
}
|
||||
```
|
||||
|
||||
## Integration with VEX
|
||||
|
||||
AI-drafted VEX statements require human approval:
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[AI generates VEX draft] --> B[Authority: ai-draft-requires-review]
|
||||
B --> C[Human reviews draft]
|
||||
C --> D{Approve?}
|
||||
D -->|Yes| E[Authority: human-approved]
|
||||
D -->|No| F[Draft rejected]
|
||||
E --> G[Publish VEX]
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Advisory AI Architecture](../architecture.md)
|
||||
- [Offline Model Bundles](./offline-model-bundles.md)
|
||||
- [Attestor Module](../../attestor/architecture.md)
|
||||
- [Evidence Locker](../../evidence-locker/architecture.md)
|
||||
Reference in New Issue
Block a user