From bbd560bf1929c02d6f9d516b2b71f5408e7e4d3a Mon Sep 17 00:00:00 2001 From: dtoro Date: Tue, 7 Jul 2026 22:39:45 +0200 Subject: [PATCH] =?UTF-8?q?plans:=20flip=20bin/=20migration=20to=20MCP=20t?= =?UTF-8?q?ool=20completion=20=E2=80=94=20agent=20is=20the=20operator=20in?= =?UTF-8?q?terface?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- plans/2026-07-07-migrate-bin-homelab-to-go.md | 210 ++++++++++++------ 1 file changed, 144 insertions(+), 66 deletions(-) diff --git a/plans/2026-07-07-migrate-bin-homelab-to-go.md b/plans/2026-07-07-migrate-bin-homelab-to-go.md index b267d8b..a44459a 100644 --- a/plans/2026-07-07-migrate-bin-homelab-to-go.md +++ b/plans/2026-07-07-migrate-bin-homelab-to-go.md @@ -1,91 +1,169 @@ -# Plan: Migrate `bin/homelab` CLI to Go `oikos homelab` +# Plan: Complete MCP tool surface — Hermes as the primary operator interface -**Status:** Planned (2026-07-07) +**Status:** Planned (2026-07-07, rev 2) — supersedes rev 1 (CLI port was the wrong paradigm) ## Goal -Replace `bin/homelab` (265-line Python CLI) with the Go `oikos homelab` -subcommand already in `cmd/oikos/main.go`. The Python CLI imports the -now-deleted Python kernel modules (`oikos.approve`, `oikos.decide`, etc.) -and reads from `/opt/homelab-context/inventory.yaml` — it's disconnected -from the new DB-backed Go stack. Once the Go CLI matches the Python one, -`bin/homelab` becomes `ln -s "$(which oikos)" bin/homelab` and the `oikos` -binary resolves its role from `os.Args[0]`. +The operator (dtoro) interacts with the homelab through Hermes, the AI agent. +Hermes talks to `oikos api` via MCP protocol at `:8090`. The command-line +`homelab` CLI is an implementation detail — the agent is the interface. -## Current state +This plan completes the MCP tool surface so Hermes can **observe, orient, +decide, and act** on the full homelab without the operator touching a shell. +Once complete, `bin/homelab` (Python) and `bin/oikos` (stale binary) are deleted. -| Layer | Python | Go | Gap | -|-------|--------|-----|-----| -| `list` | inventory.yaml scan | DB-backed `list_entities` | ✅ done | -| `whoami` | hostname → inventory lookup | `hostname()` → DB query | ✅ done | -| `ssh` | resolve mesh IP → `ssh` | same | ✅ done | -| `secret` | `sops -d` wrapper | `secret list/migrate/export-sops` | ✅ done (Infisical) | +## Architecture -## Subcommand migration plan +``` +┌──────────────────────────────────────────────────────────┐ +│ dtoro (operator) │ +│ Interface: Hermes (natural language via chat/matrix) │ +│ Also: Oikos Console (web UI, future) │ +└──────────────────────┬───────────────────────────────────┘ + │ delegates tasks + ▼ +┌──────────────────────────────────────────────────────────┐ +│ Hermes (AI agent, Docker :8092) │ +│ │ +│ Skills: health triage, signal response, execution │ +│ tracking, pattern learning, homelab ops, knowledge │ +│ │ +│ Protocol: MCP streamable HTTP → oikos api :8090 │ +│ Authority: auto-act on reversible_low only; escalates │ +│ config_mutation/destructive to operator via Matrix │ +└──────────────────────┬───────────────────────────────────┘ + │ MCP tools + ▼ +┌──────────────────────────────────────────────────────────┐ +│ oikos api (Go, Docker :8090) │ +│ │ +│ REST API + MCP server + SSE event stream │ +│ Same DB, auth middleware, audit log, policy engine │ +│ │ +│ Actuator (separate container, restricted SSH key) │ +│ Notifier (Matrix alerts) │ +│ Scheduler (OODA loop: probes, signals, patterns) │ +└──────────────────────────────────────────────────────────┘ +``` -### Phase 1 — Operational commands (P0, this week) -Core CLI ops every homelab operator uses daily. +## Current MCP tool surface (15 tools) -| Subcommand | Risk | Notes | -|-----------|------|-------| -| `pct` | low | SSH to Proxmox host + `pct exec/list`. Currently proxies via SSH. Trivial Go wrapper. | -| `logs` | low | `journalctl` SSH wrapper. Reuse ssh path from existing Go code. | -| `restart` | low | `systemctl restart` SSH wrapper. | -| `status` | medium | Ping + HTTP health check of every host/service. Use DB entity data + scheduler health checks. | -| `ssh-config` | low | Generate `~/.ssh/config.d/homelab` from DB hosts list. Output-only. | -| `open` | low | Open a service's URL via `open` (macOS) / `xdg-open` (Linux). | +### Observe — ✅ complete -### Phase 2 — Agentic layer (P1, next 2 weeks) -Oikos agentic commands that already have Go equivalents in `internal/`. +| Tool | What it does | +|------|-------------| +| `get_entity` | Get entity by slug or UUID | +| `list_entities` | List entities by type, state, search | +| `get_relations` | Get relationships for an entity | +| `get_blast_radius` | Entities affected if this entity goes down | +| `get_health_summary` | Current fleet health summary | +| `get_signal_history` | Open and recent signals | +| `get_trend` | Metric trends for an entity | +| `get_event_timeline` | Recent events | +| `query_metrics` | Time-series metric queries | +| `search_knowledge` | Full-text search across docs/runbooks | -| Subcommand | Risk | Notes | -|-----------|------|-------| -| `doctor` | low | Health check enrollment: verify age key, homelab-context clone, mesh connectivity. | -| `service` | medium | Service Console: `explain/health/docs/log/actions/history`. DB-backed — `get_entity_knowledge()` + `tail_log()`. | -| `decide` | low | Policy classification. Go: `internal/policy/classify.go` already exists. | -| `approval` | low | Approval requests. Go: `internal/domain/approval.go` exists. | -| `signal` | medium | Attention layer. Go: `internal/domain/signal.go` exists. | -| `node` | low | Ontology queries. Go: `internal/ontology/` exists. | -| `change` | medium | Mutation workflow. Combine `decide` + `approval` + actuator execution log. | +### Orient — ✅ complete -### Phase 3 — Infrastructure management (P2, 2-4 weeks) +| Tool | What it does | +|------|-------------| +| `get_patterns` | Learned action patterns | +| `get_skills` | Available automation skills | +| `get_audit_trail` | Audit log queries | -| Subcommand | Risk | Notes | -|-----------|------|-------| -| `render-vps-configs` | medium | Renders `vps/*.tmpl` with Infisical secrets. Needs template engine + SSH to VPS. | -| `sync` | low | Trigger homelab-context sync via launchd/systemd timer. | -| `apt-audit` | low | dpkg audit per host. SSH wrapper. | -| `apt-upgrade` | medium | `apt update/upgrade` in detached screen. Needs confirmation. | +### Act — ⚠️ exists, limited scope -### Phase 4 — Cleanup (P3, when ready) +| Tool | What it does | +|------|-------------| +| `request_execution` | Hermes-only mutation path. Takes `target` (entity slug) + `action` (verb). Policy-gated. | -| Action | Risk | Notes | -|--------|------|-------| -| `refresh-creds` | low | **Drop.** Infisical handles secrets now. | -| `nuke` | destructive | Port or drop. Rarely used; destructive — keep as manual SSH. | -| `client add/remove` | medium | Client enrollment. Needs secrets-issuance replacement (manual age-keygen now). | -| `mcp` | low | MCP tool wrapper. Could call the Go MCP client library. | -| Remove `bin/homelab` | **high** | Replace with symlink `oikos` → `bin/homelab`. The `oikos` binary checks `os.Args[0]` for the `homelab` role. | -| Remove `bin/oikos` | low | Stale pre-built arm64 binary. Delete. | +### Self — ✅ complete + +| Tool | What it does | +|------|-------------| +| `get_agent_activity` | Agent self-inspection | + +## What's missing — MCP tools to add + +### Phase 1 — Operational visibility (P0, this week) + +Tools Hermes needs to answer "what's happening right now?" + +| Tool | Input | Output | Implementation | +|------|-------|--------|----------------| +| `tail_log` | `service_slug`, `lines` | Last N log lines via `journalctl` | SSH to host, `journalctl -u -n ` | +| `get_service_status` | `service_slug` | `systemctl is-active` / `is-enabled` + uptime | SSH to host, `systemctl show` | +| `ping_service` | `service_slug` | HTTP/TCP reachability check | HTTP GET health endpoint from oikos scheduler probe data | +| `list_lxcs` | none | All LXCs with ID, host, IP, state | DB query from `entity_status` table | +| `get_lxc_state` | `lxc_slug` | RAM, CPU, disk, uptime | SSH to Proxmox host, `pct status ` | + +### Phase 2 — Execution expansion (P1, next 2 weeks) + +Expand `request_execution` to handle the full operational surface. Currently it writes a generic execution record. It needs to understand action types and route them: + +| Action type | What Hermes says | What actuator does | +|-------------|-----------------|--------------------| +| `restart` | "restart lxc:caddy" | `systemctl restart ` on target host | +| `logs` | "show me caddy logs" | Already handled by `tail_log` tool | +| `pct_exec` | "run apt update on LXC 121" | `pct exec -- ` on Proxmox host | +| `apt_upgrade` | "upgrade packages on lxc:caddy" | `apt update && apt upgrade -y` in detached screen | +| `apt_audit` | "audit packages on all LXCs" | `dpkg -l` + upgradable count per host | +| `systemctl` | "enable service on lxc:foo" | `systemctl ` (gated: enable = config_mutation) | + +Each action type needs: +1. A clear input schema (what params Hermes must provide) +2. Policy classification (risk class, blast radius) +3. Verification step (how to confirm it worked) +4. Ledger entry (what happened, when, by whom) + +### Phase 3 — Agent-to-operator escalation (P1, next 2 weeks) + +When Hermes hits a `config_mutation` or `destructive` action, it escalates to the operator. This path needs to be solid: + +| Component | Current state | Target | +|-----------|--------------|--------| +| Matrix alert | ⚠️ Notifier polls DB, but Hermes doesn't trigger it | Hermes calls `request_execution` → policy gate rejects → notifier sends Matrix alert with approval token | +| Approval flow | ⚠️ `oikos homelab approval` exists in Go but agent can't use it | Operator reacts ✅/❌ on Matrix → webhook → approval token consumed → actuator proceeds | +| Execution tracking | ✅ `request_execution` writes to `executions` table | Add `get_execution_status` MCP tool so Hermes can poll for results | + +### Phase 4 — Cleanup (P2, after all tools work) + +| Action | Notes | +|--------|-------| +| Delete `bin/homelab` | Python CLI. Disconnected from DB, imports deleted modules. Agent handles everything. | +| Delete `bin/oikos` | Stale pre-built arm64 binary. Docker image is the canonical build. | +| Remove `oikos homelab` Go subcommand | The `homelab` CLI role in the Go binary is a dead end. Remove it or keep as debug-only. | +| Update AGENTS.md | Document the complete MCP tool surface. Remove CLI references. | ## Implementation approach -Each subcommand is a separate file in `cmd/oikos/homelab/`. The dispatcher -in `main.go`'s `runHomelabSubcommand()` routes `os.Args` to the right handler. -All subcommands use `internal/config` for DB connection and share the SSH -helper from the existing `ssh` subcommand. +Each new MCP tool is registered in `internal/mcp/server.go` following the existing pattern: -SSH connectivity (for pct, logs, restart) reuses the mesh-address resolution -from `internal/domain/` — entity slug → inventory attributes → mesh IP → -`ssh user@ip`. +```go +register(&mcp.Tool{Name: "tail_log", Description: "...", InputSchema: ...}, + func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) { + // ... + }) +``` + +SSH-based tools (`tail_log`, `get_service_status`, `list_lxcs`, `get_lxc_state`) +reuse the mesh-address resolution already working for `request_execution`: +entity slug → DB lookup → inventory attributes → mesh IP → `ssh user@ip`. + +The actuator container holds the SSH key. MCP tools that need SSH should route +through the actuator's execution queue rather than holding their own SSH key, +maintaining the security boundary: Hermes → MCP → API → execution queue → actuator → SSH. ## Verification -- `oikos homelab` without args prints subcommand list (matching Python parity) -- Each P0 subcommand works end-to-end from mac-mini -- `bin/homelab` → `oikos` symlink; `homelab list` = `oikos homelab list` -- `bin/oikos` removed +- Hermes can answer "show me caddy logs" → `tail_log` returns journal lines +- Hermes can answer "what's the fleet health?" → `get_health_summary` + `list_lxcs` +- Hermes can answer "restart caddy" → `request_execution(action="restart", target="lxc:caddy")` → actuator SSHs → caddy restarts → Hermes confirms +- Hermes can answer "upgrade packages on dns LXC" → `request_execution(action="apt_upgrade", target="lxc:dns")` → gated → actuator runs in screen → Hermes reports result +- `bin/` directory is empty or contains only Docker-related scripts ## Changelog -- 2026-07-07 — plan created. Cataloged 27 Python subcommands vs 4 Go subcommands. \ No newline at end of file +- 2026-07-07 rev 2 — flipped paradigm. Operator interface is Hermes via MCP, not CLI. + Replaced 4-phase CLI port plan with 4-phase MCP completion plan. +- 2026-07-07 rev 1 — cataloged 27 Python CLI subcommands vs 4 Go subcommands. \ No newline at end of file