95 lines
4.3 KiB
Markdown
95 lines
4.3 KiB
Markdown
# Authority Signing Key Rotation Playbook
|
||
|
||
> **Status:** Authored 2025-10-12 as part of OPS3.KEY-ROTATION rollout.
|
||
> Use together with `docs/11_AUTHORITY.md` (Authority service guide) and the automation shipped under `ops/authority/`.
|
||
|
||
## 1. Overview
|
||
|
||
Authority publishes JWKS and revocation bundles signed with ES256 keys. To rotate those keys without downtime we now provide:
|
||
|
||
- **Automation script:** `ops/authority/key-rotation.sh`
|
||
Shell helper that POSTS to `/internal/signing/rotate`, supports metadata, dry-run, and confirms JWKS afterwards.
|
||
- **CI workflow:** `.gitea/workflows/authority-key-rotation.yml`
|
||
Manual dispatch workflow that pulls environment-specific secrets, runs the script, and records the result. Works across staging/production by passing the `environment` input.
|
||
|
||
This playbook documents the repeatable sequence for all environments.
|
||
|
||
## 2. Pre-requisites
|
||
|
||
1. **Generate a new PEM key (per environment)**
|
||
```bash
|
||
openssl ecparam -name prime256v1 -genkey -noout \
|
||
-out certificates/authority-signing-<env>-<year>.pem
|
||
chmod 600 certificates/authority-signing-<env>-<year>.pem
|
||
```
|
||
2. **Stash the previous key** under the same volume so it can be referenced in `signing.additionalKeys` after rotation.
|
||
3. **Ensure secrets/vars exist in Gitea**
|
||
- `<ENV>_AUTHORITY_BOOTSTRAP_KEY`
|
||
- `<ENV>_AUTHORITY_URL`
|
||
- Optional shared defaults `AUTHORITY_BOOTSTRAP_KEY`, `AUTHORITY_URL`.
|
||
|
||
## 3. Executing the rotation
|
||
|
||
### Option A – via CI workflow (recommended)
|
||
|
||
1. Navigate to **Actions → Authority Key Rotation**.
|
||
2. Provide inputs:
|
||
- `environment`: `staging`, `production`, etc.
|
||
- `key_id`: new `kid` (e.g. `authority-signing-2025-dev`).
|
||
- `key_path`: path as seen by the Authority service (e.g. `../certificates/authority-signing-2025-dev.pem`).
|
||
- Optional `metadata`: comma-separated `key=value` pairs (for audit trails).
|
||
3. Trigger. The workflow:
|
||
- Reads the bootstrap key/URL from secrets.
|
||
- Runs `ops/authority/key-rotation.sh`.
|
||
- Prints the JWKS response for verification.
|
||
|
||
### Option B – manual shell invocation
|
||
|
||
```bash
|
||
AUTHORITY_BOOTSTRAP_KEY=$(cat /secure/authority-bootstrap.key) \
|
||
./ops/authority/key-rotation.sh \
|
||
--authority-url https://authority.example.com \
|
||
--key-id authority-signing-2025-dev \
|
||
--key-path ../certificates/authority-signing-2025-dev.pem \
|
||
--meta rotatedBy=ops --meta changeTicket=OPS-1234
|
||
```
|
||
|
||
Use `--dry-run` to inspect the payload before execution.
|
||
|
||
## 4. Post-rotation checklist
|
||
|
||
1. Update `authority.yaml` (or environment-specific overrides):
|
||
- Set `signing.activeKeyId` to the new key.
|
||
- Set `signing.keyPath` to the new PEM.
|
||
- Append the previous key into `signing.additionalKeys`.
|
||
- Ensure `keySource`/`provider` match the values passed to the script.
|
||
2. Run `stellaops-cli auth revoke export` so revocation bundles are re-signed with the new key.
|
||
3. Confirm `/jwks` lists the new `kid` with `status: "active"` and the previous one as `retired`.
|
||
4. Archive the old key securely; keep it available until all tokens/bundles signed with it have expired.
|
||
|
||
## 5. Development key state
|
||
|
||
For the sample configuration (`etc/authority.yaml.sample`) we minted a placeholder dev key:
|
||
|
||
- Active: `authority-signing-2025-dev` (`certificates/authority-signing-2025-dev.pem`)
|
||
- Retired: `authority-signing-dev`
|
||
|
||
Treat these as examples; real environments must maintain their own PEM material.
|
||
|
||
## 6. References
|
||
|
||
- `docs/11_AUTHORITY.md` – Architecture and rotation SOP (Section 5).
|
||
- `docs/ops/authority-backup-restore.md` – Recovery flow referencing this playbook.
|
||
- `ops/authority/README.md` – CLI usage and examples.
|
||
- `scripts/rotate-policy-cli-secret.sh` – Helper to mint new `policy-cli` shared secrets when policy scope bundles change.
|
||
|
||
## 7. Appendix — Policy CLI secret rotation
|
||
|
||
Scope migrations such as AUTH-POLICY-23-004 require issuing fresh credentials for the `policy-cli` client. Use the helper script committed with the repo to keep secrets deterministic across environments.
|
||
|
||
```bash
|
||
./scripts/rotate-policy-cli-secret.sh --output etc/secrets/policy-cli.secret
|
||
```
|
||
|
||
The script writes a timestamped header and a random secret into the target file. Use `--dry-run` when generating material for external secret stores. After updating secrets in staging/production, recycle the Authority pods and confirm the new client credentials work before the next release freeze.
|