dtoro
e131215a25
scripts/cleanup-apps105-webhooks.sh: lists webhooks on dtoro/oikos, filters for apps/105 deployment endpoints, prompts for confirmation, deletes matching hooks via Gitea API. Usage: GITEA_TOKEN='...' ./scripts/cleanup-apps105-webhooks.sh Also updated cutover-checklist.md with the script reference and LXC archive command.
43 lines
2.7 KiB
Markdown
43 lines
2.7 KiB
Markdown
# Cutover checklist — Phase 6: apps/105 → Docker stack on mac-mini
|
|
|
|
Status: [x] = done, [ ] = pending
|
|
|
|
## Pre-cutover
|
|
|
|
- [x] **Backup**: `pg_dump oikos > backups/pre-cutover-20260707.sql` (145K)
|
|
- [x] **CI green**: pushed to main, `.gitea/workflows/ci.yml` exists
|
|
- [x] **Deploy test**: Docker stack running with api + scheduler + notifier + hermes
|
|
- [x] **Caddy config**: `compose/caddy/Caddyfile.oikos` pushed to `dtoro/caddy-conf` (ed20908). Auto-deploys to caddy (121).
|
|
- [x] **DNS**: `oikos.hubris.network` already resolves to 192.168.8.175 (mac-mini mesh)
|
|
- [x] **Secrets**: Infisical bootstrapped + migration complete 2026-07-07. All 11 SOPS secrets migrated to Infisical (oikos project, dev env). Machine identity `oikos-api` has RW access verified via Go SDK. ENCRYPTION_KEY must be 32-char raw string (docs incorrect). SOPS fallback preserved for DR. secrets-issuance decommissioned — stopped/disabled on apps/105; superseded by Infisical.
|
|
- [x] **Watchdog**: crontab entry added (every 2 min → `scripts/watchdog.sh`). Path fixed 2026-07-07 (was stale worktree path).
|
|
|
|
## Cutover
|
|
|
|
- [x] **Stop apps/105 services**: homelab-mcp-deploy, secrets-issuance, secrets-issuance-deploy, oikos-console, oikos-console-deploy
|
|
- [x] **Disable apps/105 services**: all 5 units disabled
|
|
- [x] **Deploy to mac-mini**: Docker stack running (`docker compose --profile full up -d`)
|
|
- [x] **Caddy reload**: pushed to `dtoro/caddy-conf` — auto-deploy triggers on LXC 121.
|
|
- [x] **DNS verify**: `oikos.hubris.network` → 192.168.8.175
|
|
|
|
## Post-cutover verification
|
|
|
|
- [x] **./scripts/verify-phase6.sh** — all 14 checks pass
|
|
- [x] **Hermes query**: `curl http://localhost:8092/query -d '{"query":"fleet health"}'` → HTTP 200
|
|
- [x] **Agent activity**: `curl http://localhost:8090/api/v1/agent-activity` → returns data
|
|
- [x] **Scheduler ticking**: 30s ticks logged
|
|
- [x] **Notifier polling**: running
|
|
- [x] **Watchdog tested**: 2026-07-07 — full cycle verified. API down → failure counting → Matrix alert sent (!alerts:hubris.network) → API recovery → counter reset. Crontab path fixed (was stale worktree path, now /Users/dtoro/Projects/oikos). MATRIX_TOKEN wired in crontab env.
|
|
|
|
## Rollback drill
|
|
|
|
- [x] **./scripts/rollback.sh** — rehearsed 2026-07-07. Backup (674KB) → stop stack → restore DB → checkout 7ac2521 → rebuild → health OK (20 tools working) → re-deploy latest (21 tools). Full cycle verified.
|
|
|
|
## Cleanup
|
|
|
|
- [ ] Remove Gitea webhooks for apps/105 (ids 10, 11) from `dtoro/oikos`
|
|
→ Run: `GITEA_TOKEN="<token>" ./scripts/cleanup-apps105-webhooks.sh`
|
|
- [ ] Archive apps/105 LXC (keep for 30 days, then destroy)
|
|
→ `ssh hubris pct stop 105 && pct snapshot 105 archive-$(date +%Y%m%d)`
|
|
- [x] Update auto-deploy docs — apps/105 entries marked deprecated
|