Files
oikos/scripts/cutover-checklist.md
dtoro cefeba72b0 delete apps/105 Gitea webhooks (ids 10, 11, 14)
Removed via Gitea API using Infisical-stored token:
- Hook 10: 192.168.8.205:9811 (homelab-mcp-deploy)
- Hook 11: 192.168.8.205:9821 (secrets-issuance-deploy)
- Hook 14: 192.168.8.205:9831 (oikos-console-deploy) — found during cleanup

All three pointed to apps/105 (192.168.8.205). No remaining webhooks
fire to the old Python stack. Zero hooks remain on dtoro/oikos.

Cutover checklist updated. Only item remaining: archive apps/105 LXC.
2026-07-08 11:29:42 +02:00

2.7 KiB

Cutover checklist — Phase 6: apps/105 → Docker stack on mac-mini

Status: [x] = done, [ ] = pending

Pre-cutover

  • Backup: pg_dump oikos > backups/pre-cutover-20260707.sql (145K)
  • CI green: pushed to main, .gitea/workflows/ci.yml exists
  • Deploy test: Docker stack running with api + scheduler + notifier + hermes
  • Caddy config: compose/caddy/Caddyfile.oikos pushed to dtoro/caddy-conf (ed20908). Auto-deploys to caddy (121).
  • DNS: oikos.hubris.network already resolves to 192.168.8.175 (mac-mini mesh)
  • Secrets: Infisical bootstrapped + migration complete 2026-07-07. All 11 SOPS secrets migrated to Infisical (oikos project, dev env). Machine identity oikos-api has RW access verified via Go SDK. ENCRYPTION_KEY must be 32-char raw string (docs incorrect). SOPS fallback preserved for DR. secrets-issuance decommissioned — stopped/disabled on apps/105; superseded by Infisical.
  • Watchdog: crontab entry added (every 2 min → scripts/watchdog.sh). Path fixed 2026-07-07 (was stale worktree path).

Cutover

  • Stop apps/105 services: homelab-mcp-deploy, secrets-issuance, secrets-issuance-deploy, oikos-console, oikos-console-deploy
  • Disable apps/105 services: all 5 units disabled
  • Deploy to mac-mini: Docker stack running (docker compose --profile full up -d)
  • Caddy reload: pushed to dtoro/caddy-conf — auto-deploy triggers on LXC 121.
  • DNS verify: oikos.hubris.network → 192.168.8.175

Post-cutover verification

  • ./scripts/verify-phase6.sh — all 14 checks pass
  • Hermes query: curl http://localhost:8092/query -d '{"query":"fleet health"}' → HTTP 200
  • Agent activity: curl http://localhost:8090/api/v1/agent-activity → returns data
  • Scheduler ticking: 30s ticks logged
  • Notifier polling: running
  • Watchdog tested: 2026-07-07 — full cycle verified. API down → failure counting → Matrix alert sent (!alerts:hubris.network) → API recovery → counter reset. Crontab path fixed (was stale worktree path, now /Users/dtoro/Projects/oikos). MATRIX_TOKEN wired in crontab env.

Rollback drill

  • ./scripts/rollback.sh — rehearsed 2026-07-07. Backup (674KB) → stop stack → restore DB → checkout 7ac2521 → rebuild → health OK (20 tools working) → re-deploy latest (21 tools). Full cycle verified.

Cleanup

  • Remove Gitea webhooks for apps/105 (ids 10, 11, 14) from dtoro/oikos — deleted 2026-07-08 via Gitea API
  • Archive apps/105 LXC (keep for 30 days, then destroy) → ssh hubris pct stop 105 && pct snapshot 105 archive-$(date +%Y%m%d)
  • Update auto-deploy docs — apps/105 entries marked deprecated