Secret rotation runbook (Phase 5)

Rotation cadences per secret type. All rotation is automated via Infisical; this runbook covers the manual verification and DR procedures.

Rotation schedule

Secret	Cadence	Method
OpenRouter API key	90 days	Infisical rotation policy → update `OPENROUTER_API_KEY` env
MCP bearer token	30 days	Infisical random password generation
Approval HMAC secret	90 days	Infisical random password generation
Matrix access token	90 days	Manual (Matrix does not support automated rotation)
Age DR key (SOPS fallback)	Never	Static — stored offline for DR only

How to rotate a secret

Automated (Infisical)

# Secrets managed by Infisical rotate automatically per the policy above.
# To force an immediate rotation:
infisical secrets rotate --project-id $INFISICAL_PROJECT_ID \
  --secret-name <secret-key> --env dev

# Verify the new value is available:
oikos secret list

Manual (SOPS fallback)

# If Infisical is unavailable, use the SOPS DR fallback:
sops -d secrets/<name>.yaml

# To rotate a SOPS secret:
sops -e --in-place secrets/<name>.yaml   # edit in place

Rotation verification

After any rotation, verify the consuming services still work:

# 1. OpenRouter key: test Hermes query
curl -s -X POST http://localhost:8092/query \
  -H "Content-Type: application/json" \
  -d '{"query":"fleet health"}'

# 2. MCP bearer token: test MCP connection
curl -s -X POST http://localhost:8092/query \
  -H "Content-Type: application/json" \
  -d '{"tool":"list_entities","args":{"limit":1}}'

# 3. Approval HMAC: create a test execution
curl -s -X POST http://localhost:8092/query \
  -H "Content-Type: application/json" \
  -d '{"query":"restart caddy"}'

Disaster recovery

If Infisical is completely unavailable:

# 1. Export SOPS DR fallback
oikos secret export-sops > /tmp/sops-dr-backup.txt

# 2. Configure services to use SOPS fallback
# Set OIKOS_SECRETS_DIR=/opt/homelab-context/secrets
# This switches the secrets manager to SOPS-only mode.

# 3. Restart services
docker compose restart api hermes notifier scheduler

Restore drill

Run monthly:

# 1. Export all secrets from Infisical
oikos secret list

# 2. Simulate Infisical outage: stop the container
docker compose stop infisical

# 3. Verify SOPS fallback works
OIKOS_SECRETS_DIR=./secrets oikos secret list

# 4. Restore Infisical
docker compose start infisical
sleep 5

# 5. Verify Infisical primary works again
oikos secret list

Changelog

2026-07-07 — initial rotation runbook

Phase 5 rotation cadences, verification steps, and DR restore drill.

2.6 KiB Raw Permalink Blame History