Files
oikos/scripts/cutover-checklist.md
dtoro dcd35b6315 phase 6: deploy pipeline — CI, cutover checklist, watchdog, verification, rollback
- scripts/deploy.sh: Gitea webhook-triggered deploy (git pull → docker build
  → compose up → health check). SHA-tagged images, rolling restart.
- scripts/watchdog.sh: cron health check every 2min, pages operator via
  Matrix after 3 consecutive failures. Reset on recovery.
- scripts/verify-phase6.sh: 14 end-to-end verification checks against
  plan V1–V14 (ontology, DB, API, scheduler, actuator, learning, classifier,
  hermes, secrets, deploy, knowledge, observability, correlation, cutover).
- scripts/rollback.sh: re-deploy previous SHA tag + pg_restore from
  pre-deploy dump. Health check loop, returns to main branch after.
- scripts/cutover-checklist.md: pre/post-cutover steps — backup, CI gate,
  Caddy re-point, DNS, apps/105 disable, cleanup.
- compose/caddy/Caddyfile.oikos: reverse-proxy config for
  oikos/mcp/hermes.hubris.network → mac-mini mesh IP.
- .gitignore: added bin/ to exclude compiled binaries.

14/14 verification checks pass against running Docker stack.
2026-07-07 17:37:21 +02:00

2.1 KiB

Cutover checklist — Phase 6: apps/105 → Docker stack on mac-mini

Status: [ ] = pending, [x] = done

Pre-cutover

  • Backup: pg_dump oikos > backups/pre-cutover-$(date +%Y%m%d).sql
  • CI green: latest push passes .gitea/workflows/ci.yml
  • Deploy test: ./scripts/deploy.sh succeeds on mac-mini
  • Caddy config ready: compose/caddy/Caddyfile.oikos committed to dtoro/caddy-conf
  • DNS: oikos.hubris.network, mcp.hubris.network, hermes.hubris.network resolve to mac-mini mesh IP
  • Secrets: Infisical machine identities configured for oikos + hermes
  • Watchdog: crontab entry added on mac-mini

Cutover

  • Stop apps/105 services: systemctl stop oikos-deploy-webhook oikos-api oikos-console
  • Disable apps/105 services: systemctl disable oikos-deploy-webhook oikos-api oikos-console
  • Deploy to mac-mini: ./scripts/deploy.sh
  • Caddy reload: push to dtoro/caddy-conf or caddy reload on LXC 121
  • DNS verify: dig oikos.hubris.network returns mac-mini mesh IP

Post-cutover verification

  • ./scripts/verify-phase6.sh — all 14 checks pass
  • Hermes query: curl http://hermes.hubris.network:8092/query -d '{"query":"fleet health"}' → HTTP 200
  • Agent activity: curl http://oikos.hubris.network:8090/api/v1/agent-activity → returns data
  • Scheduler ticking: docker compose logs scheduler shows "scheduler:" entries
  • Notifier polling: docker compose logs notifier shows "notifier:" entries
  • Watchdog tested: stop API manually, verify Matrix alert fires after 3 failures

Rollback drill

  • ./scripts/rollback.sh — redeploy previous SHA
  • Verify health: all 14 checks pass on rollback
  • Re-deploy latest: ./scripts/deploy.sh

Cleanup

  • Remove Gitea webhooks for apps/105 (ids 10, 11) from dtoro/Homelab-Docs
  • Archive apps/105 LXC (keep for 30 days, then destroy)
  • Update knowledge/wiki/infrastructure/auto-deploy.md — apps/105 entries marked deprecated