oikos/plans/done/2026-06-04_130000-deprecate-claudio-bot.md

Deprecate claudio-bot (LXC 123) — Hermes Agent now serves as control plane

Goal

Phase out the claudio-bot ecosystem (LXC 123, claudio-monitor, IPC server) now that Hermes Agent is configured and running on mac-mini. Hermes replaces claudio-bot's role as the homelab control plane — accepting commands, relaying alerts, and answering queries about infrastructure state.

What is claudio-bot

A three-component ecosystem deployed 2026-04-20:

Component	Location	Purpose
claudio-bot	LXC 123 (192.168.8.230)	Matrix bot @claudio:avispero — slash commands, NL queries, infra notifications
claudio-monitor	hubris (PVE host)	5-min watchdog → sends alerts to claudio-bot's IPC server
IPC server	LXC 123:9090	/notify, /propose, /status endpoints — used by monitor + backup wrapper

Dependencies:

• Matrix synapse on LXC 118 (elementsynapse) — claudio-bot connects to 192.168.8.239:8008
• Gitea auto-deploy webhook → LXC 123:9797 (pipeline dtoro/claudio-bot)
• LLM backend: LM Studio on mac-mini (192.168.8.174:1234) or Anthropic API
• Secrets at /etc/claudio-bot/ (config.yaml, matrix.token, anthropic.key, lmstudio.key, ipc.token)

What Hermes replaces

claudio-bot function	Hermes replacement
Matrix slash commands	Hermes CLI + send_message for outbound notifications
NL infrastructure queries	Hermes with homelab CLI + MCP tools (already working)
Monitor alert relay	Hermes-native: (a) on-demand homelab health via extended skill, (b) cronjob every 15 min with actionable alerts (not just passive notifications)
IPC server (port 9090)	Retired — Hermes doesn't need a separate IPC layer; monitoring runs directly via SSH + send_message
LLM backend	Already handled by Hermes (OpenRouter → deepseek/deepseek-v4-pro)
Auto-deploy webhook (9797)	Retired — repo archived

Full blast radius (files referencing claudio)

Homelab-context repo (/opt/homelab-context/)

Core topology

• inventory.yaml:275 — claudio-bot: host entry (LXC 123, matrix-agent)
• hosts/claudio-bot.yaml — auto-generated host file (delete)
• .sops.yaml:47 — placeholder comment listing claudio-bot/claudio-monitor

Documentation (containers)

• containers/123-claudio-bot.md — full container doc
• containers/index.md:17 — LXC table row
• containers/118-elementsynapse.md:3,36,39 — @claudio:avispero reference, disk reading, direct connection note
• containers/103-paperless.md:27 — disk reading from claudio-monitor

Documentation (infrastructure)

• infrastructure/monitoring.md — ENTIRE FILE is about claudio-monitor → claudio-bot pipeline
• infrastructure/backups.md:74,94,98 — IPC notify, thermal monitoring moved out, related link
• infrastructure/auto-deploy.md:23,41,50,61,120,138-139 — Gitea webhook target, pipeline table, port 9797
• infrastructure/ssh-access.md:134 — LXC table row
• infrastructure/homelab-context.md:4,16,143 — client list, integration note

Other

• operations/commands.md:17 — pvesh bulk query note (passive reference, can stay)
• README.md:32,40,64 — LXC table, monitoring link, tracked configs mention
• mcp/deploy/webhook/webhook.py:5-6 — port comment (9797, 9799)
• hosts/hubris.md:57,96 — /opt/claudio-monitor checkout, LXC 109 cleanup note (passive)
• vms/108-haos.md:30 — claudio-monitor MQTT removal note (historical, can stay)
• investigations/2026-04-21-hubris-crash-loop.md:27 — historical reference (keep)

External repos (Gitea)

• dtoro/claudio-bot — the bot itself
• dtoro/claudio-monitor — the watchdog scripts

External state (hubris)

• /opt/claudio-bot/ — bot checkout (LXC 123)
• /opt/claudio-monitor/ — monitor checkout (hubris host)
• /etc/claudio-bot/ — secrets directory
• /etc/claudio-deploy/ — deploy git credentials
• /etc/claudio-monitor/ — thresholds + bot.token
• /var/lib/claudio-monitor/ — state files (.alert, .counter)
• claudio-bot.service — systemd unit (LXC 123)
• claudio-monitor.timer + .service — systemd timer (hubris)
• Gitea webhook id 6 (claudio-bot-deploy) + webhook for claudio-monitor (9799) on apps LXC 105

External state (workstations)

• Nothing on mac-mini, republic-laptop, or ludo-mini references claudio directly

Phased deprecation plan

Phase 1 — Stop & verify (low-risk, reversible)

Goal: Stop claudio services, verify nothing breaks, keep LXC 123 paused (not destroyed).

1. Stop claudio-monitor on hubris

   ssh hubris systemctl disable --now claudio-monitor.timer
   

   The 5-minute watchdog stops. Home Assistant still gets PVE metrics independently via its own integration — no alerting gap there.

2. Stop claudio-bot on LXC 123

   ssh claudio-bot systemctl stop claudio-bot
   

   The Matrix bot goes offline. Nobody can send slash commands to @claudio:avispero.

3. Pause LXC 123 (not destroy yet)

   ssh hubris pct shutdown 123
   

   IPC server (port 9090) and deploy webhook (port 9797) become unreachable.

4. Verify no breakage (run for 24-48h)

   • Check Gitea webhook logs — the dtoro/claudio-bot webhook will 404 on pushes (non-fatal)
   • Check Caddy logs — no upstream errors for claudio-related paths (there shouldn't be any — no public hostname)
   • Check DNS — no queries for claudio-bot.hubris.network (doesn't have a public hostname)
   • Confirm Hermes works normally for all homelab operations

Phase 2 — Hermes-native health monitoring (replaces claudio-monitor)

Goal: Two capabilities that together exceed what claudio-monitor did:

1. On-demand health check — user asks "how's the homelab?" → Hermes runs a comprehensive check across all hosts/LXCs and reports results conversationally.
2. Cron-based watchdog — runs every 15 minutes, checks thresholds, raises alarms with actionable options (not just passive alerts). User can respond and Hermes takes action directly.

Key difference from claudio-monitor: The old system was a bash script that POSTed JSON to an IPC server that a Matrix bot reformatted into a flat message. The Hermes-native approach gives the LLM the raw metrics and lets it reason about what's important before alerting — and crucially, the user can reply to an alert and Hermes can SSH into hubris to resize a rootfs, restart a service, or investigate deeper.

2a. Extend homelab-hardware-health skill

The existing skill covers CPU/NVMe temps. Expand it to be the single homelab health check skill covering:

• Hardware: CPU temps (sysfs), NVMe/SATA SSD temps (smartctl), block device inventory
• LXC resources: pvesh get /cluster/resources --type vm --output-format json → disk %, CPU %, RAM usage per container
• Service health: Check key services respond (gitea:3000, caddy:80/443, synapse:8008, paperless, nextcloud, jellyfin)
• System drift: apt list --upgradable counts per host, docker image pull digests on apps LXC 105

Thresholds (from old /etc/claudio-monitor/thresholds.env):

Metric	Warning	Critical
LXC disk usage	> 80%	> 90%
NVMe temp	> 60°C	> 70°C
CPU temp	> 70°C	> 80°C
Apt upgradable	> 10	> 50
Service down	any	2+ services

Update the skill file at devops/homelab-hardware-health/SKILL.md with:

• Expanded trigger phrases ("check homelab health", "how's the lab", "health report")
• LXC resource check commands
• Service health check commands
• Thresholds table
• "If you find issues, present options to the user" instruction

2b. Create Hermes cronjob homelab-health-watchdog

hermes cron create \
  --schedule 'every 15m' \
  --name 'homelab-health-watchdog' \
  --prompt 'Load skill homelab-hardware-health. Run a full health check across all hosts (hubris + all LXCs). Check: (1) hardware temps, (2) LXC disk/CPU/RAM via pvesh, (3) service reachability for gitea, caddy, synapse, paperless, nextcloud, jellyfin, (4) apt upgradable counts. Compare against thresholds in the skill. If nothing is critical, say nothing (no message = healthy). If anything breaches thresholds, send a concise alert with: what broke, current value vs threshold, and 2-3 actionable options the user can reply to. DO NOT send routine "all clear" messages.'

This runs every 15 minutes (Hermes cron is LLM-driven, heavier than a bash script — 15 min is appropriate vs claudio-monitor's 5 min). When thresholds are clean, the agent stays silent. When something breaches, the user gets a message like:

⚠️ LXC 103 (paperless) disk at 91% (> 90% critical)

Options: [resize rootfs to 16 GiB] [investigate what's using space] [snooze 24h]

User picks an option → Hermes takes action (SSH into hubris, run pct resize, grow the filesystem inside the container).

2c. Add homelab-health alias to homelab CLI

For quick on-demand checks from the terminal:

homelab health          # runs the skill inline via Hermes
# or equivalently:
hermes --prompt "Load homelab-hardware-health, run full health check"

Phase 3 — Documentation & inventory cleanup

Goal: Update homelab-context to reflect claudio deprecation, without destroying history.

Files to edit

File	Action
inventory.yaml	Mark claudio-bot entry as deprecated (comment + status: deprecated). Do NOT remove — keeps historical ID mapping.
containers/123-claudio-bot.md	Add deprecation banner at top, changelog entry
containers/index.md	Change status from "running" to "stopped (deprecated)"
infrastructure/monitoring.md	Rewrite to document new Hermes-based monitoring. Keep claudio-monitor section as "legacy (deprecated 2026-06-04)".
infrastructure/auto-deploy.md	Mark claudio-bot and claudio-monitor webhook entries as deprecated
infrastructure/backups.md	Update notification section (IPC path no longer available). Note: backups themselves are still disabled — this is forward-looking.
infrastructure/ssh-access.md	Mark LXC 123 as "offline (deprecated)"
infrastructure/homelab-context.md	Remove claudio-bot from client list
containers/118-elementsynapse.md	Note @claudio:avispero is decommissioned
containers/103-paperless.md	Remove stale disk reading reference (already resolved — rootfs grew to 16 GiB 2026-05-15 per 118 doc)
README.md	Update LXC table, remove monitoring link
hosts/claudio-bot.yaml	Add deprecation banner (or delete if auto-generated — it is, so just update inventory)
.sops.yaml	Remove claudio-bot from placeholder comment

Files to leave untouched (historical)

File	Reason
investigations/2026-04-21-hubris-crash-loop.md	Historical investigation — claudio-monitor is factual context
vms/108-haos.md:30	Historical note about MQTT removal — factual
mcp/deploy/webhook/webhook.py:5-6	Comment only — harmless, keep for port mapping context
operations/commands.md:17	Passive reference — harmless

Phase 4 — Purge (after confirmation window)

Goal: Full removal after confirming Phase 1-3 is stable for ≥2 weeks.

1. Destroy LXC 123

   ssh hubris pct destroy 123 --purge
   

   Frees 8 GiB rootfs, 512 MiB RAM, 1 core.

2. Remove claudio-monitor from hubris

   ssh hubris rm -rf /opt/claudio-monitor /etc/claudio-monitor /var/lib/claudio-monitor
   ssh hubris systemctl disable claudio-monitor.timer  # already done in phase 1
   

3. Archive Gitea repos

   • Archive dtoro/claudio-bot (mark read-only in Gitea)
   • Archive dtoro/claudio-monitor (mark read-only in Gitea)

4. Remove Gitea webhooks

   • Remove claudio-bot-deploy webhook (id 6) from Gitea
   • Remove claudio-monitor webhook (9799) from apps LXC 105
   • Remove 192.168.8.230 from Gitea app.ini ALLOWED_HOST_LIST

5. Final documentation cleanup

   • Remove containers/123-claudio-bot.md (or move to containers/_destroyed/)
   • Remove hosts/claudio-bot.yaml
   • Remove claudio-bot row from containers/index.md
   • Update all cross-references (see blast radius above)

Files changed by phase

Phase 3 (documentation)

• /opt/homelab-context/inventory.yaml
• /opt/homelab-context/containers/123-claudio-bot.md
• /opt/homelab-context/containers/index.md
• /opt/homelab-context/infrastructure/monitoring.md
• /opt/homelab-context/infrastructure/auto-deploy.md
• /opt/homelab-context/infrastructure/backups.md
• /opt/homelab-context/infrastructure/ssh-access.md
• /opt/homelab-context/infrastructure/homelab-context.md
• /opt/homelab-context/containers/118-elementsynapse.md
• /opt/homelab-context/containers/103-paperless.md
• /opt/homelab-context/README.md
• /opt/homelab-context/hosts/claudio-bot.yaml
• /opt/homelab-context/.sops.yaml

Phase 4 (destruction)

• Proxmox: LXC 123 destroyed
• Gitea: dtoro/claudio-bot archived, dtoro/claudio-monitor archived
• Gitea webhooks removed (apps LXC 105)
• hubris: /opt/claudio-monitor/, /etc/claudio-monitor/, /var/lib/claudio-monitor/ removed
• containers/123-claudio-bot.md → moved to _destroyed/ or deleted

Risks & tradeoffs

Risk	Mitigation
Monitoring gap — no alerts during Phase 1-2 transition	Phase 1 stops the old timer but Phase 2 builds the Hermes-native replacement immediately. If there's a gap, keep claudio-monitor.timer running until the Hermes cronjob is verified with a test alert.
Backup notifications — the backup wrapper's IPC notify path is dead	Backups are already DISABLED. When re-enabled, wire them to the new Hermes monitoring pipeline instead.
Matrix @claudio:avispero — users may still try to interact	The bot just won't respond. Could leave a notice in the Matrix room.
Gitea webhook failures — pushes to dtoro/claudio-bot will 404	Non-fatal. Archive the repo in Phase 4 to stop pushes entirely.
Losing monitoring history — state files on hubris	Keep /var/lib/claudio-monitor/ until Phase 4. No rush.
LM Studio dependency — claudio-bot was using mac-mini's LM Studio	Hermes already uses OpenRouter. LM Studio on mac-mini can stay for other uses or be reclaimed.

Open questions

1. Should the Matrix @claudio:avispero account be deactivated or left as-is? It has no public hostname and was invite-only to a private room.

2. Are you OK archiving dtoro/claudio-bot and dtoro/claudio-monitor on Gitea (read-only, preserved), or do you want them fully deleted?

3. Should LXC 123 be paused (reversible) or destroyed immediately? Recommend pause for at least 2 weeks as a safety net.

4. Which messaging platform should the health watchdog deliver alerts to? Currently Hermes is connected via CLI (mac-mini). To get push alerts, we need a send_message target — Telegram, Discord, or Matrix are available. Which do you prefer?

Verification

After Phase 1:

• ssh hubris systemctl is-active claudio-monitor.timer → inactive
• ssh claudio-bot systemctl is-active claudio-bot → inactive (or unreachable if LXC stopped)
• ssh hubris pct status 123 → stopped
• curl -s http://192.168.8.230:9090/status → connection refused (IPC dead)

After Phase 2:

• hermes cron list shows homelab-health-watchdog running every 15 min
• "check homelab health" → Hermes runs comprehensive check, reports temps + LXC resources + service status + apt drift
• Force an alert: NVME_WARN_TEMP_C=1 on hubris → watchdog catches it on next 15-min tick → alert arrives via Hermes messaging with actionable options
• Alert format includes 2-3 options user can reply to, not just passive notification

After Phase 3:

• All cross-references in homelab-context point to deprecated/removed status
• grep -r 'claudio' /opt/homelab-context/ --include='*.md' --include='*.yaml' | grep -v '_destroyed\|investigations\|changelog' → only historical references remain

After Phase 4:

• ssh hubris pct list | grep 123 → no output
• ssh hubris ls /opt/claudio-monitor → no such file
• Gitea dtoro/claudio-bot → archived badge