stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
b4fc66feb6f1c992cf5f0de8925698a9c89e17f6
git.stella-ops.org/docs/modules/advisory-ai/guides/ai-attestations.md

8.4 KiB
Raw Blame History

AI Attestations and Replay Semantics

Sprint: SPRINT_20251226_018_AI_attestations Task: AIATTEST-23

This guide documents the AI attestation schemas, authority classification, and deterministic replay semantics.

Overview

AI-generated artifacts in StellaOps are wrapped in cryptographic attestations that:

  1. Capture the exact inputs (prompts, context, model parameters)
  2. Prove the generation chain (model ID, weights digest, configuration)
  3. Enable deterministic replay for compliance verification
  4. Support divergence detection across environments

Attestation Types

AI Artifact Predicate

{
  "_type": "https://stellaops.org/attestation/ai-artifact/v1",
  "artifactId": "ai-artifact-20251226-001",
  "artifactType": "explanation",
  "authority": "ai-generated",
  "generatedAt": "2025-12-26T10:30:00Z",
  "model": {
    "modelId": "llama3-8b-q4km",
    "weightsDigest": "sha256:a1b2c3...",
    "promptTemplateVersion": "v2.1.0"
  },
  "inputs": {
    "systemPromptHash": "sha256:abc123...",
    "userPromptHash": "sha256:def456...",
    "contextHashes": ["sha256:111...", "sha256:222..."]
  },
  "parameters": {
    "temperature": 0.0,
    "seed": 42,
    "maxTokens": 2048,
    "topK": 1
  },
  "output": {
    "contentHash": "sha256:789xyz...",
    "tokenCount": 847
  },
  "replayManifest": {
    "manifestId": "replay-20251226-001",
    "manifestHash": "sha256:manifest..."
  }
}

Artifact Types

Type Description Authority
explanation Vulnerability explanation for humans ai-generated
remediation Fix plan with upgrade paths ai-generated
vex_draft Draft VEX statement ai-draft-requires-review
policy_draft Draft policy rules ai-draft-requires-review
triage_suggestion Triage action suggestions ai-suggestion

Authority Classification

AI outputs are classified by their authority level:

ai-generated           → Informational only, human review optional
ai-draft-requires-review → Draft requires explicit human approval
ai-suggestion          → Suggestion, user decides action
ai-verified            → AI output verified against ground truth
human-approved         → AI output approved by human reviewer

Replay Manifest

The replay manifest captures everything needed to reproduce an AI generation:

{
  "manifestVersion": "1.0",
  "artifactId": "ai-artifact-20251226-001",
  "artifactType": "explanation",

  "model": {
    "modelId": "llama3-8b-q4km",
    "weightsDigest": "sha256:a1b2c3d4e5f6...",
    "promptTemplateVersion": "v2.1.0"
  },

  "prompts": {
    "systemPrompt": "You are a security analyst...",
    "userPrompt": "Explain CVE-2024-1234 affecting lodash@4.17.20...",
    "systemPromptHash": "sha256:abc123...",
    "userPromptHash": "sha256:def456..."
  },

  "context": {
    "contextPack": [...],
    "contextHashes": ["sha256:111...", "sha256:222..."]
  },

  "parameters": {
    "temperature": 0.0,
    "seed": 42,
    "maxTokens": 2048,
    "topK": 1,
    "topP": 1.0
  },

  "output": {
    "content": "CVE-2024-1234 is a critical vulnerability...",
    "contentHash": "sha256:789xyz...",
    "tokenCount": 847
  },

  "metadata": {
    "generatedAt": "2025-12-26T10:30:00Z",
    "replayable": true,
    "deterministicSettings": true
  }
}

Deterministic Requirements

For an AI artifact to be replayable:

  1. Temperature must be 0: No randomness in token selection
  2. Seed must be fixed: Same seed across replays (default: 42)
  3. Model weights must match: Verified by weights digest
  4. Prompts must match: Verified by prompt hashes
  5. Context must match: All input hashes must verify

Configuration for Determinism

advisoryAi:
  attestations:
    requireDeterminism: true
    defaultSeed: 42

  inference:
    local:
      temperature: 0.0
      seed: 42
      topK: 1
      topP: 1.0

Replay Workflow

Replay Execution

// Load replay manifest
var manifest = await LoadManifestAsync("replay-20251226-001.json");

// Create replayer with same model
var replayer = replayerFactory.Create(manifest.Model.ModelId);

// Execute replay
var result = await replayer.ReplayAsync(manifest, cancellationToken);

// Check if output is identical
if (result.Identical)
{
    Console.WriteLine("Replay successful: output matches original");
}
else
{
    Console.WriteLine($"Divergence detected: similarity = {result.SimilarityScore:P2}");
}

Divergence Detection

When replay produces different output:

{
  "diverged": true,
  "similarityScore": 0.97,
  "originalHash": "sha256:789xyz...",
  "replayedHash": "sha256:different...",
  "details": [
    {
      "type": "content_divergence",
      "description": "Content differs at position",
      "position": 1842,
      "originalSnippet": "...vulnerability allows...",
      "replayedSnippet": "...vulnerability permits..."
    }
  ]
}

Common Divergence Causes

Cause Detection Resolution
Different model weights Weights digest mismatch Use exact model version
Non-zero temperature Parameter check Set temperature to 0
Different seed Parameter check Use same seed
Prompt template change Template version mismatch Pin template version
Context ordering Context hash mismatch Sort context deterministically

Attestation Signing

DSSE Envelope Format

AI attestations use DSSE (Dead Simple Signing Envelope):

{
  "payloadType": "application/vnd.stellaops.ai-attestation+json",
  "payload": "<base64-encoded-attestation>",
  "signatures": [
    {
      "keyId": "stellaops-ai-signer-2025",
      "sig": "<base64-signature>"
    }
  ]
}

Signing Configuration

advisoryAi:
  attestations:
    sign: true
    keyId: "stellaops-ai-signer-2025"
    cryptoScheme: ed25519  # ed25519 | ecdsa-p256 | gost3410 | sm2

API Endpoints

Generate with Attestation

POST /api/v1/advisory/explain
Content-Type: application/json

{
  "findingId": "finding-123",
  "artifactDigest": "sha256:...",
  "options": {
    "generateAttestation": true,
    "signAttestation": true
  }
}

Response includes:

{
  "explanation": "...",
  "attestation": {
    "predicateType": "https://stellaops.org/attestation/ai-artifact/v1",
    "predicate": {...},
    "signature": {...}
  },
  "replayManifestId": "replay-20251226-001"
}

Verify Attestation

POST /api/v1/attestation/verify
Content-Type: application/json

{
  "attestation": {...},
  "options": {
    "verifySignature": true,
    "verifyReplay": true
  }
}

Replay Artifact

POST /api/v1/advisory/replay
Content-Type: application/json

{
  "manifestId": "replay-20251226-001"
}

CLI Commands

# Generate explanation with attestation
stella advisory explain finding-123 --attest --sign

# Verify attestation
stella attest verify ai-artifact-20251226-001.dsse.json

# Replay from manifest
stella advisory replay --manifest replay-20251226-001.json

# Check divergence
stella advisory replay --manifest replay-20251226-001.json --detect-divergence

Storage and Retrieval

Attestation Storage

Attestations are stored in the Evidence Locker:

/evidence/ai-attestations/
  ├── 2025/12/26/
  │   ├── ai-artifact-20251226-001.json
  │   ├── ai-artifact-20251226-001.dsse.json
  │   └── replay-20251226-001.json

Retrieval

GET /api/v1/attestation/ai-artifact-20251226-001

# Returns attestation + replay manifest

Audit Trail

AI operations are logged for compliance:

{
  "timestamp": "2025-12-26T10:30:00Z",
  "operation": "ai_generation",
  "artifactId": "ai-artifact-20251226-001",
  "artifactType": "explanation",
  "modelId": "llama3-8b-q4km",
  "authority": "ai-generated",
  "user": "system",
  "inputHashes": ["sha256:..."],
  "outputHash": "sha256:...",
  "signed": true,
  "replayable": true
}

Integration with VEX

AI-drafted VEX statements require human approval:

graph LR
    A[AI generates VEX draft] --> B[Authority: ai-draft-requires-review]
    B --> C[Human reviews draft]
    C --> D{Approve?}
    D -->|Yes| E[Authority: human-approved]
    D -->|No| F[Draft rejected]
    E --> G[Publish VEX]

Related Documentation