dtoro
8a6422bd7d
Problem: node and cross-cutting narratives lived at the repo root
(containers/, vms/, infrastructure/, host .md files), interleaved with the
machine-readable substrate.
Change:
- Move containers/ -> knowledge/wiki/containers/, vms/ -> knowledge/wiki/vms/,
infrastructure/ -> knowledge/wiki/infrastructure/, hosts/{hubris,strong}.md ->
knowledge/wiki/hosts/, infrastructure/references/ -> knowledge/sources/references/,
GLOSSARY.md -> knowledge/GLOSSARY.md.
- Add knowledge/{index.md,log.md,sources/index.md} scaffolding.
- Rewrite all relative links repo-wide via a path-resolving mapper (inbound +
outbound + between-moved-files), including .hermes/, runbooks, operations,
investigations, plans, README, AGENTS.
- Repoint inventory.yaml doc_page fields and regenerate hosts/*.yaml (which
embed doc_page); update oikos/gen-topology.py output path, candidate doc
paths, and footer links; update code-comment doc paths.
Substrate untouched in place: inventory.yaml, hosts/*.yaml (regenerated,
idempotent), oikos/ code, mcp/, secrets/, bin/.
Verification:
- Logical broken-link set identical to pre-move baseline (net 128 -> 127; the
topology regen fixed one, introduced none). Remaining are pre-existing refs
to destroyed/archived nodes, out of scope for this move.
- gen-topology.py --check exit 0 (in sync); cards carry knowledge/wiki/ doc paths.
- build_host_files.py idempotent; all inventory doc_page targets resolve.
- MCP contract verified: get_page/search_docs/get_changelog resolve moved pages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
61 lines
2.4 KiB
Markdown
61 lines
2.4 KiB
Markdown
# Monitoring — Hermes health watchdog
|
|
|
|
Homelab health monitoring via Hermes Agent on mac-mini. Replaced the legacy
|
|
`claudio-monitor` + `claudio-bot` IPC pipeline on 2026-06-04.
|
|
|
|
## Current approach
|
|
|
|
Two layers:
|
|
|
|
1. **On-demand:** ask Hermes "how's the homelab?" or run `homelab health` — loads
|
|
the `homelab-hardware-health` skill, checks hardware temps, LXC resources,
|
|
service reachability, and apt/docker drift across all hosts.
|
|
|
|
2. **Cron watchdog:** `homelab-health-watchdog` runs every 15 minutes via Hermes
|
|
cron. Silent when healthy. When thresholds breach, sends an actionable alert
|
|
to Matrix (`@dtoro:avispero`) with options the user can reply to directly
|
|
(e.g. "resize rootfs", "investigate", "snooze 24h"). Hermes takes action on
|
|
the selected option via SSH.
|
|
|
|
Thresholds: LXC disk >80% warn/>90% critical, NVMe >60°C/>70°C, CPU >70°C/>80°C,
|
|
apt >10/>50 upgradable, services down.
|
|
|
|
Home Assistant pulls PVE metrics independently via its Proxmox VE integration
|
|
(unaffected by this change).
|
|
|
|
## Legacy: claudio-monitor (deprecated 2026-06-04)
|
|
|
|
The old system was a bash watchdog on hubris (`claudio-monitor.timer`, every 5
|
|
min) that POSTed alerts to a Matrix bot (`@claudio:avispero`) via an IPC server
|
|
on LXC 123:9090. All components decommissioned:
|
|
|
|
| Component | Fate |
|
|
|-----------|------|
|
|
| LXC 123 (claudio-bot) | Destroyed 2026-06-04 |
|
|
| `dtoro/claudio-bot` | Archived (read-only) on Gitea |
|
|
| `dtoro/claudio-monitor` | Archived (read-only) on Gitea |
|
|
| `claudio-monitor.timer` | Disabled on hubris |
|
|
| `/opt/claudio-monitor/` | Still on hubris (cleanup pending) |
|
|
| `/etc/claudio-monitor/` | Still on hubris (cleanup pending) |
|
|
|
|
For the full deprecation plan, see `plans/2026-06-04_130000-deprecate-claudio-bot.md`.
|
|
|
|
## Related pages
|
|
- [Hubris host](../hosts/hubris.md)
|
|
- [HAOS VM (108)](../vms/108-haos.md)
|
|
- [Backups (disabled)](backups.md)
|
|
- [Homelab context distribution](homelab-context.md)
|
|
|
|
## Changelog
|
|
|
|
### 2026-06-04 — migrated to Hermes health watchdog
|
|
claudio-monitor + claudio-bot IPC pipeline replaced by Hermes-native monitoring.
|
|
On-demand `homelab health` via extended skill; 15-min cron watchdog with actionable
|
|
Matrix alerts. LXC 123 destroyed, repos archived.
|
|
|
|
### 2026-04-28 — wiki entry created
|
|
Initial documentation.
|
|
|
|
### 2026-04-21 — claudio-monitor stood up; thermal-watch removed
|
|
General health monitor with per-LXC checks. MQTT/REST push paths ripped out.
|