Files
git.stella-ops.org/docs/operations/hsm-setup-runbook.md

6.9 KiB

HSM Setup and Configuration Runbook

This runbook provides step-by-step procedures for configuring Hardware Security Module (HSM) integration with Stella Ops.

Overview

Stella Ops supports PKCS#11-compatible HSMs for cryptographic key storage and signing operations. This includes:

  • YubiHSM 2
  • Thales Luna Network HSM
  • AWS CloudHSM
  • SoftHSM2 (development/testing)

Prerequisites

Hardware Requirements

Component Requirement
HSM Device PKCS#11 compatible
Network HSM accessible from Stella Ops services
Backup Secondary HSM for key backup

Software Requirements

# PKCS#11 library for your HSM
# Example for SoftHSM2 (development)
apt-get install softhsm2 opensc

# Verify installation
softhsm2-util --version
pkcs11-tool --version

SoftHSM2 Setup (Development)

Step 1: Initialize SoftHSM

# Create token directory
mkdir -p /var/lib/softhsm/tokens
chmod 700 /var/lib/softhsm/tokens

# Initialize token
softhsm2-util --init-token \
  --slot 0 \
  --label "StellaOps-Dev" \
  --so-pin 12345678 \
  --pin 87654321

# Verify token
softhsm2-util --show-slots

Step 2: Generate Signing Key

# Generate ECDSA P-256 key
pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so \
  --login --pin 87654321 \
  --keypairgen \
  --key-type EC:prime256v1 \
  --id 01 \
  --label "stellaops-signing-2026"

# List keys
pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so \
  --login --pin 87654321 \
  --list-objects

Step 3: Export Public Key

# Export public key for distribution
pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so \
  --login --pin 87654321 \
  --read-object \
  --type pubkey \
  --id 01 \
  --output-file stellaops-signing-2026.pub.der

# Convert to PEM
openssl ec -pubin -inform DER \
  -in stellaops-signing-2026.pub.der \
  -outform PEM \
  -out stellaops-signing-2026.pub.pem

YubiHSM 2 Setup

Step 1: Install YubiHSM SDK

# Download YubiHSM SDK
wget https://developers.yubico.com/YubiHSM2/Releases/yubihsm2-sdk-2023.01-ubuntu2204-amd64.tar.gz
tar xzf yubihsm2-sdk-*.tar.gz
cd yubihsm2-sdk
sudo ./install.sh

# Start connector
sudo systemctl enable yubihsm-connector
sudo systemctl start yubihsm-connector

Step 2: Initialize YubiHSM

# Connect to YubiHSM shell
yubihsm-shell

# Authenticate with default auth key
connect
session open 1 password

# Create authentication key for Stella Ops
generate authkey 0 100 "StellaOps-Auth" 1 generate-asymmetric-key:sign-ecdsa:delete-asymmetric-key

# Generate signing key
generate asymmetric 0 200 "StellaOps-Signing" 1 sign-ecdsa ecp256

# Export public key
get public key 0 200 stellaops-yubihsm.pub

session close 0
quit

Step 3: Configure PKCS#11

# Create PKCS#11 configuration
cat > /etc/yubihsm_pkcs11.conf <<EOF
connector = http://127.0.0.1:12345
EOF

# Test PKCS#11 access
pkcs11-tool --module /usr/lib/libyubihsm_pkcs11.so \
  --list-slots

Stella Ops Configuration

Basic HSM Configuration

# etc/stellaops.yaml
signing:
  provider: "hsm"
  hsm:
    type: "pkcs11"
    libraryPath: "/usr/lib/softhsm/libsofthsm2.so"  # or /usr/lib/libyubihsm_pkcs11.so
    slotId: 0
    tokenLabel: "StellaOps-Dev"
    pin: "${HSM_PIN}"  # Use environment variable
    keyId: "01"
    keyLabel: "stellaops-signing-2026"

    # Connection settings
    connectionTimeoutSeconds: 30
    maxSessions: 10
    sessionIdleTimeoutSeconds: 300

    # Retry settings
    maxRetries: 3
    retryDelayMs: 100

Environment Variables

# Set HSM PIN securely
export HSM_PIN="87654321"

# Or use secrets manager
export HSM_PIN=$(aws secretsmanager get-secret-value \
  --secret-id stellaops/hsm-pin \
  --query SecretString --output text)

Kubernetes Secret

apiVersion: v1
kind: Secret
metadata:
  name: stellaops-hsm
  namespace: stellaops
type: Opaque
stringData:
  HSM_PIN: "87654321"

Verification

Step 1: Connectivity Check

# Run HSM connectivity doctor check
stella doctor --check hsm

# Expected output:
# [PASS] HSM Connectivity
#   - Library loaded: /usr/lib/softhsm/libsofthsm2.so
#   - Slot available: 0 (StellaOps-Dev)
#   - Key found: stellaops-signing-2026
#   - Sign/verify test: PASSED

Step 2: Signing Test

# Test signing operation
stella sign test \
  --message "test message" \
  --key-label "stellaops-signing-2026"

# Expected output:
# Signature: base64...
# Algorithm: ECDSA-P256
# Key ID: 01

Step 3: Integration Test

# Run HSM integration tests
stella test integration --filter "HSM*"

Key Rotation

Step 1: Generate New Key

# Generate new key in HSM
pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so \
  --login --pin ${HSM_PIN} \
  --keypairgen \
  --key-type EC:prime256v1 \
  --id 02 \
  --label "stellaops-signing-2027"

Step 2: Add to Trust Anchor

# Add new key to Stella Ops
stella key add \
  --key-id "stellaops-signing-2027" \
  --algorithm EC-P256 \
  --public-key stellaops-signing-2027.pub.pem

Step 3: Transition Period

# Update configuration for dual-key
signing:
  activeKeyId: "stellaops-signing-2027"
  additionalKeys:
    - keyId: "stellaops-signing-2026"
      keyLabel: "stellaops-signing-2026"

Step 4: Revoke Old Key

# After transition period (2-4 weeks)
stella key revoke \
  --key-id "stellaops-signing-2026" \
  --reason "scheduled-rotation"

Troubleshooting

Common Issues

Issue Symptom Resolution
Library not found PKCS11 library not found Verify libraryPath in config
Slot not available Slot 0 not found Run pkcs11-tool --list-slots
Key not found Key stellaops-signing not found Verify key label with --list-objects
Pin incorrect CKR_PIN_INCORRECT Check HSM_PIN environment variable
Session limit CKR_SESSION_COUNT Increase maxSessions or restart

Debug Logging

# Enable HSM debug logging
logging:
  levels:
    StellaOps.Cryptography.Hsm: Debug

Session Recovery

# If sessions exhausted, restart service
kubectl rollout restart deployment stellaops-signer -n stellaops

Security Best Practices

  1. PIN Management

    • Never hardcode PINs in configuration files
    • Use secrets management (Vault, AWS Secrets Manager)
    • Rotate PINs periodically
  2. Key Backup

    • Configure HSM key backup/replication
    • Test key recovery procedures regularly
    • Document recovery process
  3. Access Control

    • Limit HSM access to required services only
    • Use separate authentication keys per service
    • Audit HSM access logs
  4. Network Security

    • Use TLS for network HSM connections
    • Firewall HSM to authorized hosts only
    • Monitor for unauthorized access attempts