git.stella-ops.org/docs/modules/vexlens/operations/deployment.md

VexLens Deployment Runbook

Operational runbook for deploying and configuring VexLens consensus engine.

---

1) Prerequisites

1.1 Infrastructure Requirements

Component	Requirement	Notes
Runtime	.NET 10.0+	LTS recommended
Database	MongoDB 6.0+	For projections and issuer directory
Cache	Redis 7.0+ (optional)	For caching consensus results
Memory	512MB minimum	2GB recommended for production
CPU	2 cores minimum	4 cores for high throughput

1.2 Dependencies

• Excititor: VEX document ingestion service
• Authority: OIDC token validation
• Policy Engine: (optional) For VEX-aware policy evaluation

---

2) Configuration

2.1 Environment Variables

# Core Settings
VEXLENS_CONSENSUS_DEFAULT_MODE=WeightedVote
VEXLENS_CONSENSUS_MINIMUM_CONFIDENCE=0.1
VEXLENS_CONSENSUS_CONFLICT_THRESHOLD=0.3

# Trust Settings
VEXLENS_TRUST_FRESHNESS_HALFLIFE_DAYS=90
VEXLENS_TRUST_MINIMUM_FRESHNESS=0.3
VEXLENS_TRUST_ALLOW_UNSIGNED=true
VEXLENS_TRUST_UNSIGNED_PENALTY=0.3
VEXLENS_TRUST_ALLOW_UNKNOWN_ISSUERS=true
VEXLENS_TRUST_UNKNOWN_ISSUER_PENALTY=0.5

# Storage
VEXLENS_STORAGE_MONGODB_CONNECTION_STRING=mongodb://localhost:27017
VEXLENS_STORAGE_MONGODB_DATABASE=vexlens
VEXLENS_STORAGE_PROJECTION_RETENTION_DAYS=365
VEXLENS_STORAGE_EVENT_RETENTION_DAYS=90

# Issuer Directory
VEXLENS_ISSUER_DIRECTORY_SOURCE=mongodb
VEXLENS_ISSUER_DIRECTORY_REFRESH_INTERVAL_MINUTES=60

# Observability
VEXLENS_OTEL_EXPORTER_ENDPOINT=http://otel-collector:4317
VEXLENS_OTEL_SERVICE_NAME=vexlens

2.2 Configuration File (vexlens.yaml)

vexlens:
  consensus:
    defaultMode: WeightedVote
    minimumConfidence: 0.1
    conflictThreshold: 0.3
    requireJustificationForNotAffected: false

  trust:
    freshnessHalfLifeDays: 90
    minimumFreshness: 0.3
    allowUnsigned: true
    unsignedPenalty: 0.3
    allowUnknownIssuers: true
    unknownIssuerPenalty: 0.5
    factorWeights:
      IssuerBase: 0.25
      SignatureStatus: 0.15
      Freshness: 0.15
      IssuerCategory: 0.10
      IssuerTier: 0.10
      StatusQuality: 0.10
      TransparencyLog: 0.05
      SourceMatch: 0.05
      ProductAuthority: 0.05

  storage:
    mongodb:
      connectionString: mongodb://localhost:27017
      database: vexlens
      projectionsCollection: consensus_projections
      issuersCollection: issuers
    projectionRetentionDays: 365
    eventRetentionDays: 90

  issuerDirectory:
    source: mongodb
    refreshIntervalMinutes: 60
    seedFile: /etc/vexlens/issuers.json

  observability:
    metrics:
      enabled: true
      exporterEndpoint: http://otel-collector:4317
    tracing:
      enabled: true
      samplingRatio: 0.1
    logging:
      level: Information
      format: json

---

3) Deployment Steps

3.1 Docker Deployment

# Pull the image
docker pull stellaops/vexlens:latest

# Run with configuration
docker run -d \
  --name vexlens \
  -p 8080:8080 \
  -v /etc/vexlens:/etc/vexlens:ro \
  -e VEXLENS_STORAGE_MONGODB_CONNECTION_STRING=mongodb://mongo:27017 \
  stellaops/vexlens:latest

3.2 Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vexlens
  namespace: stellaops
spec:
  replicas: 2
  selector:
    matchLabels:
      app: vexlens
  template:
    metadata:
      labels:
        app: vexlens
    spec:
      containers:
      - name: vexlens
        image: stellaops/vexlens:latest
        ports:
        - containerPort: 8080
        env:
        - name: VEXLENS_STORAGE_MONGODB_CONNECTION_STRING
          valueFrom:
            secretKeyRef:
              name: vexlens-secrets
              key: mongodb-connection-string
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "2000m"
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
        volumeMounts:
        - name: config
          mountPath: /etc/vexlens
          readOnly: true
      volumes:
      - name: config
        configMap:
          name: vexlens-config
---
apiVersion: v1
kind: Service
metadata:
  name: vexlens
  namespace: stellaops
spec:
  selector:
    app: vexlens
  ports:
  - port: 80
    targetPort: 8080

3.3 Helm Deployment

helm install vexlens stellaops/vexlens \
  --namespace stellaops \
  --set mongodb.connectionString=mongodb://mongo:27017 \
  --set replicas=2 \
  --set resources.requests.memory=512Mi \
  --set resources.limits.memory=2Gi

---

4) Issuer Directory Setup

4.1 Seed Issuers File

Create /etc/vexlens/issuers.json:

{
  "issuers": [
    {
      "issuerId": "npm-security",
      "name": "npm Security Team",
      "category": "Vendor",
      "trustTier": "Authoritative",
      "keyFingerprints": [
        {
          "fingerprint": "ABCD1234EFGH5678",
          "keyType": "Pgp",
          "algorithm": "EdDSA"
        }
      ],
      "metadata": {
        "description": "Official npm security advisories",
        "uri": "https://www.npmjs.com/advisories"
      }
    },
    {
      "issuerId": "github-security",
      "name": "GitHub Security Lab",
      "category": "Aggregator",
      "trustTier": "Trusted",
      "metadata": {
        "description": "GitHub Security Advisories",
        "uri": "https://github.com/advisories"
      }
    }
  ]
}

4.2 Register Issuer via API

curl -X POST http://vexlens:8080/api/v1/vexlens/issuers \
  -H "Content-Type: application/json" \
  -H "X-StellaOps-Tenant: tenant-001" \
  -d '{
    "issuerId": "vendor-acme",
    "name": "ACME Corporation",
    "category": "Vendor",
    "trustTier": "Authoritative",
    "initialKeys": [
      {
        "fingerprint": "1234ABCD5678EFGH",
        "keyType": "Pgp",
        "algorithm": "RSA"
      }
    ],
    "metadata": {
      "description": "ACME security advisories",
      "uri": "https://security.acme.example.com"
    }
  }'

---

5) Health Checks

5.1 Liveness Probe

curl http://vexlens:8080/health/live
# Response: {"status": "Healthy"}

5.2 Readiness Probe

curl http://vexlens:8080/health/ready
# Response: {"status": "Healthy", "checks": {"mongodb": "Healthy", "issuerDirectory": "Healthy"}}

5.3 Detailed Health

curl http://vexlens:8080/health/detailed
# Full health check with component details

---

6) Monitoring

6.1 Key Metrics to Monitor

Metric	Alert Threshold	Description
vexlens.consensus.duration_seconds	p99 > 5s	Consensus computation latency
vexlens.consensus.conflicts_total	rate > 100/min	High conflict rate
vexlens.normalization.errors_total	rate > 10/min	Normalization failures
vexlens.projection.query_duration_seconds	p99 > 1s	Slow projection queries

6.2 Grafana Dashboard

Import the VexLens dashboard from deploy/grafana/vexlens-dashboard.json.

6.3 Alerting Rules

groups:
- name: vexlens
  rules:
  - alert: VexLensHighLatency
    expr: histogram_quantile(0.99, rate(vexlens_consensus_duration_seconds_bucket[5m])) > 5
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "VexLens consensus latency is high"

  - alert: VexLensHighConflictRate
    expr: rate(vexlens_consensus_conflicts_total[5m]) > 100
    for: 10m
    labels:
      severity: warning
    annotations:
      summary: "VexLens detecting high conflict rate"

  - alert: VexLensNormalizationErrors
    expr: rate(vexlens_normalization_errors_total[5m]) > 10
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "VexLens normalization errors increasing"

---

7) Backup and Recovery

7.1 Backup Projections

# MongoDB backup
mongodump --uri="mongodb://localhost:27017" \
  --db=vexlens \
  --collection=consensus_projections \
  --out=/backup/vexlens-$(date +%Y%m%d)

7.2 Backup Issuer Directory

# Export issuers to JSON
curl http://vexlens:8080/api/v1/vexlens/issuers?limit=1000 \
  > /backup/issuers-$(date +%Y%m%d).json

7.3 Restore

# Restore MongoDB
mongorestore --uri="mongodb://localhost:27017" \
  --db=vexlens \
  /backup/vexlens-20251206/

# Re-seed issuers if needed
# Issuers are automatically loaded from seed file on startup

---

8) Scaling

8.1 Horizontal Scaling

VexLens is stateless for compute operations. Scale horizontally by adding replicas:

kubectl scale deployment vexlens --replicas=4 -n stellaops

8.2 Performance Tuning

# For high-throughput deployments
vexlens:
  consensus:
    # Enable batch processing
    batchSize: 100
    batchTimeoutMs: 50

  storage:
    mongodb:
      # Connection pool
      maxConnectionPoolSize: 100
      minConnectionPoolSize: 10

  caching:
    enabled: true
    redis:
      connectionString: redis://redis:6379
    consensusTtlMinutes: 5
    issuerTtlMinutes: 60

---

9) Troubleshooting

9.1 Common Issues

Issue	Cause	Resolution
Slow consensus	Many statements	Enable caching, increase batch size
High conflict rate	Inconsistent sources	Review issuer trust tiers
Normalization failures	Invalid VEX format	Check Excititor connector config
Low confidence scores	Missing signatures	Configure issuer keys

9.2 Debug Logging

# Enable debug logging
export VEXLENS_OBSERVABILITY_LOGGING_LEVEL=Debug

9.3 Determinism Verification

# Run determinism harness
curl -X POST http://vexlens:8080/api/v1/vexlens/test/determinism \
  -H "Content-Type: application/json" \
  -d '{"vexContent": "..."}'

---

10) Upgrade Procedure

10.1 Rolling Upgrade

# Update image
kubectl set image deployment/vexlens vexlens=stellaops/vexlens:v1.2.0 -n stellaops

# Monitor rollout
kubectl rollout status deployment/vexlens -n stellaops

10.2 Database Migrations

VexLens uses automatic schema migrations. No manual intervention required for minor versions.

For major version upgrades:

1. Backup all data
2. Review migration notes in release changelog
3. Apply migrations: vexlens migrate --apply
4. Verify: vexlens migrate --verify