fix tests. new product advisories enhancements

This commit is contained in:
master
2026-01-25 19:11:36 +02:00
parent c70e83719e
commit 6e687b523a
504 changed files with 40610 additions and 3785 deletions

View File

@@ -0,0 +1,248 @@
# StellaOps Trust Bootstrap Guide
> Sprint: SPRINT_20260125_003 - WORKFLOW-001
> Last updated: 2026-01-25
## Overview
This guide covers the initial trust setup for a new StellaOps deployment. Trust
bootstrap establishes the cryptographic foundations for secure attestation and
verification.
## Prerequisites
- StellaOps CLI installed (`stella` command available)
- Network access to TUF repository (or offline trust bundle)
- Sufficient permissions to create keys in `/etc/stellaops/keys/`
- For keyless mode: OIDC identity provider configured
## Quick Start
### Online Bootstrap
```bash
# Initialize trust from organization's TUF repository
./devops/scripts/bootstrap-trust.sh \
--tuf-url https://trust.example.com/tuf/ \
--pin rekor-key-v1
```
### Offline Bootstrap (Air-Gapped)
```bash
# Import pre-packaged trust bundle
./devops/scripts/bootstrap-trust-offline.sh \
/media/usb/trust-bundle-2026-01-25.tar.zst
```
## Detailed Steps
### Step 1: Generate Signing Keys (Optional)
If using local signing keys (not keyless/OIDC):
```bash
# Create key directory
mkdir -p /etc/stellaops/keys
chmod 700 /etc/stellaops/keys
# Generate ECDSA P-256 signing key
stella keys generate \
--type ecdsa-p256 \
--out /etc/stellaops/keys/signing-key.pem
# Or use OpenSSL
openssl ecparam -name prime256v1 -genkey -noout \
-out /etc/stellaops/keys/signing-key.pem
chmod 600 /etc/stellaops/keys/signing-key.pem
```
### Step 2: Initialize TUF Client
```bash
# Initialize with your organization's TUF repository
stella trust init \
--tuf-url https://trust.example.com/tuf/ \
--service-map sigstore-services-v1 \
--pin rekor-key-v1 rekor-key-v2
# Verify initialization
stella trust status
```
The `--pin` option specifies which Rekor keys to trust. Pin multiple keys during
rotation periods.
### Step 3: Verify TUF Metadata
```bash
# Check trust status
stella trust status --show-keys --show-endpoints
# Expected output:
# TUF Repository: https://trust.example.com/tuf/
# Service Map: sigstore-services-v1
# Trusted Keys:
# - rekor-key-v1 (expires: 2027-01-01)
# - rekor-key-v2 (expires: 2028-01-01)
# Endpoints:
# - Rekor: https://rekor.sigstore.dev
# - Fulcio: https://fulcio.sigstore.dev
```
### Step 4: Test Sign/Verify Cycle
```bash
# Create a test payload
echo "StellaOps bootstrap test" > /tmp/test-payload.txt
# Sign with your key
stella sign /tmp/test-payload.txt \
--key /etc/stellaops/keys/signing-key.pem \
--out /tmp/test.sig
# Verify signature
stella verify /tmp/test-payload.txt \
--sig /tmp/test.sig
# Clean up
rm /tmp/test-payload.txt /tmp/test.sig
```
### Step 5: Test Rekor Submission (Online Only)
```bash
# Create and submit an attestation
stella attest create /tmp/test-payload.txt \
--type stellaops.io/predicates/test@v1 \
--rekor-submit
# Verify inclusion in transparency log
stella attest verify /tmp/test-payload.txt \
--check-inclusion
```
## Offline Bootstrap
For air-gapped deployments without network access:
### Create Trust Bundle (Connected System)
On a system with network access, create a trust bundle:
```bash
stella trust snapshot export \
--include-tiles \
--out trust-bundle-$(date +%Y-%m-%d).tar.zst
```
### Transfer and Import (Air-Gapped System)
```bash
# Transfer bundle via USB, DVD, or approved data diode
# Then import:
./devops/scripts/bootstrap-trust-offline.sh \
/media/usb/trust-bundle-2026-01-25.tar.zst
# Optional: Reject stale bundles
./devops/scripts/bootstrap-trust-offline.sh \
/media/usb/trust-bundle-2026-01-25.tar.zst \
--reject-if-stale 7d
```
## Configuration Options
### TUF Client Configuration
After bootstrap, TUF client configuration is stored in:
`~/.local/share/StellaOps/TufCache/`
Key files:
- `root.json` - Root of trust (only updated via ceremony)
- `targets.json` - List of trusted targets
- `snapshot.json` - Point-in-time snapshot of targets
- `timestamp.json` - Freshness guarantee (regularly updated)
### Environment Variables
```bash
# Override cache directory
export STELLAOPS_TUF_CACHE=/custom/path
# Enable debug logging
export STELLAOPS_LOG_LEVEL=debug
# Offline mode (no network calls)
export STELLAOPS_OFFLINE=true
```
## Troubleshooting
### Error: "TUF metadata verification failed"
The TUF root key may have been rotated. Obtain the new root.json from your
security team and re-bootstrap:
```bash
stella trust init \
--tuf-url https://trust.example.com/tuf/ \
--root-json /path/to/new/root.json \
--force
```
### Error: "Rekor connectivity check failed"
1. Verify network access to Rekor endpoint
2. Check firewall rules for HTTPS (port 443)
3. Verify the Rekor URL in service map is correct
4. Try forcing a sync: `stella trust sync --force`
### Error: "Key not found in trust store"
The pinned key may not exist in the TUF repository. Check available keys:
```bash
stella trust status --show-keys
```
### Offline: "Bundle is stale"
The trust bundle exceeds the staleness threshold. Obtain a fresh bundle from a
connected system:
```bash
# On connected system
stella trust snapshot export --out fresh-bundle.tar.zst
# Transfer and import
./devops/scripts/bootstrap-trust-offline.sh fresh-bundle.tar.zst
```
## Maintenance
### Periodic Sync
Set up a cron job to keep TUF metadata fresh:
```bash
# Every 6 hours
0 */6 * * * /usr/local/bin/stella trust sync --quiet
```
### Updating Air-Gap Bundles
For air-gapped systems, schedule regular bundle updates based on your
organization's freshness requirements (typically 7-30 days).
## Next Steps
- Configure CI/CD to use the signing key
- Set up key rotation procedures (see `key-rotation-runbook.md`)
- Configure monitoring for trust state freshness
- For air-gap: Establish bundle transfer schedule
## Related Documentation
- [TUF Integration Guide](../modules/attestor/tuf-integration.md)
- [Key Rotation Runbook](key-rotation-runbook.md)
- [Disaster Recovery](disaster-recovery.md)

View File

@@ -0,0 +1,328 @@
# StellaOps Disaster Recovery Guide
> Sprint: SPRINT_20260125_003 - WORKFLOW-003
> Last updated: 2026-01-25
## Overview
This guide covers disaster recovery procedures for StellaOps trust
infrastructure, including Rekor outages, key compromise, and TUF repository
failures.
## Scenario 1: Rekor Service Outage
### Symptoms
- Attestation submissions failing
- Verification requests timing out
- Circuit breaker reporting OPEN state
### Immediate Actions
1. **Verify the outage**
```bash
# Check Rekor health
curl -sf https://rekor.sigstore.dev/api/v1/log | jq .
# Check circuit breaker state
stella trust status --show-circuit-breaker
```
2. **Check if mirror is active**
```bash
# If mirror failover is enabled, verify it's working
stella trust status --show-backends
```
3. **If mirror is not available, swap endpoints via TUF**
```bash
# On TUF repository admin system
./devops/scripts/disaster-swap-endpoint.sh \
--repo /path/to/tuf \
--new-rekor-url https://rekor-mirror.internal:8080 \
--note "Emergency: Production Rekor outage $(date -u)"
```
4. **Publish the update**
```bash
cd /path/to/tuf
./scripts/sign-metadata.sh # Sign updated metadata
./scripts/publish.sh # Deploy to TUF server
```
5. **Force client sync (optional, for immediate effect)**
```bash
stella trust sync --force
```
### Key Principle
**No client reconfiguration required.** Endpoint changes flow through TUF.
Clients discover new endpoints within their configured refresh interval.
### Recovery
Once the primary Rekor is restored:
1. **Swap back to primary**
```bash
./devops/scripts/disaster-swap-endpoint.sh \
--repo /path/to/tuf \
--new-rekor-url https://rekor.sigstore.dev \
--note "Recovery: Primary Rekor restored"
```
2. **Verify service map published**
```bash
stella trust sync --force
stella trust status --show-endpoints
```
3. **Reset circuit breakers**
```bash
stella trust reset-circuits
```
## Scenario 2: Rekor Key Compromise
### Symptoms
- Security team reports potential key exposure
- Unusual entries in transparency log
- Third-party security advisory
### Immediate Actions
1. **Assess the compromise scope**
- When was the key potentially exposed?
- What entries may be affected?
- Are there signed entries from the compromised period?
2. **Emergency key rotation**
```bash
# Phase 1: Add new key immediately (no grace period)
./devops/scripts/rotate-rekor-key.sh add-key \
--repo /path/to/tuf \
--new-key /secure/new-rekor-key-v2.pub
# Sign and publish immediately
cd /path/to/tuf
./scripts/sign-metadata.sh
./scripts/publish.sh
```
3. **Force all clients to sync**
- Announce emergency update to all teams
- Clients should run: `stella trust sync --force`
4. **Revoke compromised key immediately**
```bash
# Phase 2: Remove old key (skip grace period due to compromise)
./devops/scripts/rotate-rekor-key.sh remove-old \
--repo /path/to/tuf \
--old-key-name rekor-key-v1
# Sign and publish
cd /path/to/tuf
./scripts/sign-metadata.sh
./scripts/publish.sh
```
5. **Document the incident**
- Log rotation time
- Affected key ID and fingerprint
- List of potentially affected entries
- Remediation steps taken
### Forensics
Identify entries signed during the compromise window:
```bash
# Query entries by time range
stella rekor query \
--after "2026-01-20T00:00:00Z" \
--before "2026-01-25T00:00:00Z" \
--key-id compromised-key-id
```
## Scenario 3: TUF Repository Unavailable
### Symptoms
- Clients cannot sync trust metadata
- `stella trust sync` failing with network errors
- TUF timestamp verification failing
### Immediate Actions
1. **Diagnose the issue**
```bash
# Check TUF repository health
curl -sf https://trust.example.com/tuf/timestamp.json | jq .
# Check DNS resolution
nslookup trust.example.com
# Check TLS certificate
openssl s_client -connect trust.example.com:443 -servername trust.example.com
```
2. **For clients - extend offline tolerance**
```bash
# Temporarily allow stale metadata (use with caution)
stella trust sync --allow-stale --max-age 7d
```
3. **Restore TUF server**
- Check hosting infrastructure
- Restore from backup if needed
- Verify metadata integrity
4. **Deploy mirror (if available)**
```bash
# Update DNS or load balancer to point to mirror
# Or update clients directly (less preferred)
stella trust init \
--tuf-url https://trust-mirror.example.com/tuf/ \
--force
```
## Scenario 4: Signing Key Compromise
### Symptoms
- Security team reports key exposure
- Unauthorized attestations appearing
### Immediate Actions
1. **Revoke the compromised key**
```bash
./devops/scripts/rotate-signing-key.sh retire \
--old-key compromised-key-name
```
2. **Generate new signing key**
```bash
./devops/scripts/rotate-signing-key.sh generate \
--key-type ecdsa-p256
```
3. **Update CI/CD immediately**
- Remove compromised key from all pipelines
- Add new key
- Trigger rebuild of recent releases
4. **Notify downstream consumers**
- Announce key rotation
- Provide new public key
- Advise re-verification of recent attestations
## Scenario 5: Root Key Ceremony Required
### When Required
- Scheduled root key rotation (typically annual)
- Root key compromise (emergency)
- Threshold change for root signatures
### Procedure
1. **Schedule ceremony**
- Require M-of-N key holders present
- Air-gapped ceremony machine
- Hardware security modules
2. **Generate new root**
```bash
# On air-gapped ceremony machine
tuf-ceremony init \
--threshold 3 \
--keys 5 \
--algorithm ed25519
```
3. **Sign new root with old keys**
- Requires old threshold of signatures
- Ensures continuous trust chain
4. **Distribute new root**
- Publish to TUF repository
- Update bootstrap documentation
- Notify all operators
### Air-Gap Considerations
For air-gapped deployments after root rotation:
```bash
# Export new trust bundle with updated root
stella trust snapshot export \
--include-root \
--out post-rotation-bundle.tar.zst
# Transfer and import on air-gapped systems
./devops/scripts/bootstrap-trust-offline.sh \
post-rotation-bundle.tar.zst \
--force # Required due to root change
```
## Communication Templates
### Outage Notification
```
Subject: [StellaOps] Rekor Service Disruption - Failover Active
Status: Service Degradation
Impact: Attestation submissions may be delayed
Mitigation: Automatic failover to mirror active
Action Required: None - clients will auto-discover new endpoint
Updates: Monitor status at https://status.example.com
```
### Key Rotation Notice
```
Subject: [StellaOps] Emergency Key Rotation - Action Required
Reason: Security precaution / Scheduled rotation
Affected Key: rekor-key-v1 (fingerprint: abc123...)
New Key: rekor-key-v2 (fingerprint: def456...)
Action Required:
1. Run: stella trust sync --force
2. Verify: stella trust status --show-keys
Timeline: Old key will be revoked at [DATE/TIME UTC]
```
## Monitoring and Alerting
### Key Metrics
- Circuit breaker state changes
- TUF metadata freshness
- Rekor submission latency
- Verification success rate
### Alert Thresholds
| Metric | Warning | Critical |
|--------|---------|----------|
| TUF metadata age | > 12h | > 24h |
| Circuit breaker opens | > 2/hour | > 5/hour |
| Submission failures | > 5% | > 20% |
| Verification failures | > 1% | > 5% |
## Contacts
| Role | Contact | Escalation |
|------|---------|------------|
| TUF Admin | tuf-admin@example.com | On-call |
| Security Team | security@example.com | Immediate |
| Platform Team | platform@example.com | Business hours |
## Related Documentation
- [Bootstrap Guide](bootstrap-guide.md)
- [Key Rotation Runbook](key-rotation-runbook.md)
- [TUF Integration Guide](../modules/attestor/tuf-integration.md)

View File

@@ -421,9 +421,111 @@ groups:
---
## TUF-Based Key Rotation
> Sprint: SPRINT_20260125_003 - WORKFLOW-007
For organizations using TUF-based trust distribution, additional key rotation
procedures apply to Rekor public keys and TUF metadata signing keys.
### Rekor Public Key Rotation
Rekor public keys verify transparency log signatures. Rotation uses a dual-key
grace period to ensure all clients sync the new key before removing the old one.
**Recommended rotation interval:** Annually
**Grace period:** 7-14 days
#### Phase 1: Add New Key
```bash
# Add new Rekor key to TUF repository
./devops/scripts/rotate-rekor-key.sh add-key \
--repo /path/to/tuf \
--new-key rekor-key-v2.pub
# Sign and publish TUF metadata
cd /path/to/tuf
./scripts/sign-metadata.sh
./scripts/publish.sh
```
#### Phase 2: Grace Period
During the grace period (7-14 days):
- Monitor client sync logs
- Verify both keys work for verification
- Confirm all clients have updated
```bash
# Check client trust status
stella trust status --show-keys
# Should show both rekor-key-v1 and rekor-key-v2
```
#### Phase 3: Remove Old Key
```bash
# Remove old key after grace period
./devops/scripts/rotate-rekor-key.sh remove-old \
--repo /path/to/tuf \
--old-key-name rekor-key-v1
# Sign and publish
cd /path/to/tuf
./scripts/sign-metadata.sh
./scripts/publish.sh
```
### TUF Root Key Rotation
TUF root keys are the ultimate trust anchor. Rotation is a high-ceremony
operation requiring M-of-N key holders.
**Recommended rotation interval:** 2-3 years
**Requires:** Key ceremony with multiple signers
See [Disaster Recovery](disaster-recovery.md#scenario-5-root-key-ceremony-required)
for full root key ceremony procedures.
### TUF Metadata Signing Key Rotation
For targets, snapshot, and timestamp keys:
```bash
# Generate new metadata signing key
openssl ecparam -name prime256v1 -genkey -noout \
-out /secure/targets-key-v2.pem
# Update root.json to include new key
tuf update-root --add-targets-key /secure/targets-key-v2.pem
# Sign with both old and new keys during transition
tuf sign targets --key /secure/targets-key-v1.pem
tuf sign targets --key /secure/targets-key-v2.pem
# After grace period, remove old key from root.json
tuf update-root --remove-targets-key /secure/targets-key-v1.pem
```
### Automated Scripts
Use the provided automation scripts:
| Script | Purpose |
|--------|---------|
| `devops/scripts/rotate-rekor-key.sh` | Rekor public key rotation |
| `devops/scripts/rotate-signing-key.sh` | Organization signing key rotation |
| `devops/trust-repo-template/scripts/revoke-target.sh` | Remove target from TUF |
---
## Related Documentation
- [Proof Chain API](../api/proofs.md)
- [Attestor Architecture](../modules/attestor/architecture.md)
- [Signer Architecture](../modules/signer/architecture.md)
- [TUF Integration Guide](../modules/attestor/tuf-integration.md)
- [Bootstrap Guide](bootstrap-guide.md)
- [Disaster Recovery](disaster-recovery.md)
- [NIST SP 800-57](https://csrc.nist.gov/publications/detail/sp/800-57-part-1/rev-5/final) - Key Management Guidelines