plans: flip bin/ migration to MCP tool completion — agent is the operator interface
This commit is contained in:
@@ -1,91 +1,169 @@
|
||||
# Plan: Migrate `bin/homelab` CLI to Go `oikos homelab`
|
||||
# Plan: Complete MCP tool surface — Hermes as the primary operator interface
|
||||
|
||||
**Status:** Planned (2026-07-07)
|
||||
**Status:** Planned (2026-07-07, rev 2) — supersedes rev 1 (CLI port was the wrong paradigm)
|
||||
|
||||
## Goal
|
||||
|
||||
Replace `bin/homelab` (265-line Python CLI) with the Go `oikos homelab`
|
||||
subcommand already in `cmd/oikos/main.go`. The Python CLI imports the
|
||||
now-deleted Python kernel modules (`oikos.approve`, `oikos.decide`, etc.)
|
||||
and reads from `/opt/homelab-context/inventory.yaml` — it's disconnected
|
||||
from the new DB-backed Go stack. Once the Go CLI matches the Python one,
|
||||
`bin/homelab` becomes `ln -s "$(which oikos)" bin/homelab` and the `oikos`
|
||||
binary resolves its role from `os.Args[0]`.
|
||||
The operator (dtoro) interacts with the homelab through Hermes, the AI agent.
|
||||
Hermes talks to `oikos api` via MCP protocol at `:8090`. The command-line
|
||||
`homelab` CLI is an implementation detail — the agent is the interface.
|
||||
|
||||
## Current state
|
||||
This plan completes the MCP tool surface so Hermes can **observe, orient,
|
||||
decide, and act** on the full homelab without the operator touching a shell.
|
||||
Once complete, `bin/homelab` (Python) and `bin/oikos` (stale binary) are deleted.
|
||||
|
||||
| Layer | Python | Go | Gap |
|
||||
|-------|--------|-----|-----|
|
||||
| `list` | inventory.yaml scan | DB-backed `list_entities` | ✅ done |
|
||||
| `whoami` | hostname → inventory lookup | `hostname()` → DB query | ✅ done |
|
||||
| `ssh` | resolve mesh IP → `ssh` | same | ✅ done |
|
||||
| `secret` | `sops -d` wrapper | `secret list/migrate/export-sops` | ✅ done (Infisical) |
|
||||
## Architecture
|
||||
|
||||
## Subcommand migration plan
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ dtoro (operator) │
|
||||
│ Interface: Hermes (natural language via chat/matrix) │
|
||||
│ Also: Oikos Console (web UI, future) │
|
||||
└──────────────────────┬───────────────────────────────────┘
|
||||
│ delegates tasks
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ Hermes (AI agent, Docker :8092) │
|
||||
│ │
|
||||
│ Skills: health triage, signal response, execution │
|
||||
│ tracking, pattern learning, homelab ops, knowledge │
|
||||
│ │
|
||||
│ Protocol: MCP streamable HTTP → oikos api :8090 │
|
||||
│ Authority: auto-act on reversible_low only; escalates │
|
||||
│ config_mutation/destructive to operator via Matrix │
|
||||
└──────────────────────┬───────────────────────────────────┘
|
||||
│ MCP tools
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ oikos api (Go, Docker :8090) │
|
||||
│ │
|
||||
│ REST API + MCP server + SSE event stream │
|
||||
│ Same DB, auth middleware, audit log, policy engine │
|
||||
│ │
|
||||
│ Actuator (separate container, restricted SSH key) │
|
||||
│ Notifier (Matrix alerts) │
|
||||
│ Scheduler (OODA loop: probes, signals, patterns) │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Phase 1 — Operational commands (P0, this week)
|
||||
Core CLI ops every homelab operator uses daily.
|
||||
## Current MCP tool surface (15 tools)
|
||||
|
||||
| Subcommand | Risk | Notes |
|
||||
|-----------|------|-------|
|
||||
| `pct` | low | SSH to Proxmox host + `pct exec/list`. Currently proxies via SSH. Trivial Go wrapper. |
|
||||
| `logs` | low | `journalctl` SSH wrapper. Reuse ssh path from existing Go code. |
|
||||
| `restart` | low | `systemctl restart` SSH wrapper. |
|
||||
| `status` | medium | Ping + HTTP health check of every host/service. Use DB entity data + scheduler health checks. |
|
||||
| `ssh-config` | low | Generate `~/.ssh/config.d/homelab` from DB hosts list. Output-only. |
|
||||
| `open` | low | Open a service's URL via `open` (macOS) / `xdg-open` (Linux). |
|
||||
### Observe — ✅ complete
|
||||
|
||||
### Phase 2 — Agentic layer (P1, next 2 weeks)
|
||||
Oikos agentic commands that already have Go equivalents in `internal/`.
|
||||
| Tool | What it does |
|
||||
|------|-------------|
|
||||
| `get_entity` | Get entity by slug or UUID |
|
||||
| `list_entities` | List entities by type, state, search |
|
||||
| `get_relations` | Get relationships for an entity |
|
||||
| `get_blast_radius` | Entities affected if this entity goes down |
|
||||
| `get_health_summary` | Current fleet health summary |
|
||||
| `get_signal_history` | Open and recent signals |
|
||||
| `get_trend` | Metric trends for an entity |
|
||||
| `get_event_timeline` | Recent events |
|
||||
| `query_metrics` | Time-series metric queries |
|
||||
| `search_knowledge` | Full-text search across docs/runbooks |
|
||||
|
||||
| Subcommand | Risk | Notes |
|
||||
|-----------|------|-------|
|
||||
| `doctor` | low | Health check enrollment: verify age key, homelab-context clone, mesh connectivity. |
|
||||
| `service` | medium | Service Console: `explain/health/docs/log/actions/history`. DB-backed — `get_entity_knowledge()` + `tail_log()`. |
|
||||
| `decide` | low | Policy classification. Go: `internal/policy/classify.go` already exists. |
|
||||
| `approval` | low | Approval requests. Go: `internal/domain/approval.go` exists. |
|
||||
| `signal` | medium | Attention layer. Go: `internal/domain/signal.go` exists. |
|
||||
| `node` | low | Ontology queries. Go: `internal/ontology/` exists. |
|
||||
| `change` | medium | Mutation workflow. Combine `decide` + `approval` + actuator execution log. |
|
||||
### Orient — ✅ complete
|
||||
|
||||
### Phase 3 — Infrastructure management (P2, 2-4 weeks)
|
||||
| Tool | What it does |
|
||||
|------|-------------|
|
||||
| `get_patterns` | Learned action patterns |
|
||||
| `get_skills` | Available automation skills |
|
||||
| `get_audit_trail` | Audit log queries |
|
||||
|
||||
| Subcommand | Risk | Notes |
|
||||
|-----------|------|-------|
|
||||
| `render-vps-configs` | medium | Renders `vps/*.tmpl` with Infisical secrets. Needs template engine + SSH to VPS. |
|
||||
| `sync` | low | Trigger homelab-context sync via launchd/systemd timer. |
|
||||
| `apt-audit` | low | dpkg audit per host. SSH wrapper. |
|
||||
| `apt-upgrade` | medium | `apt update/upgrade` in detached screen. Needs confirmation. |
|
||||
### Act — ⚠️ exists, limited scope
|
||||
|
||||
### Phase 4 — Cleanup (P3, when ready)
|
||||
| Tool | What it does |
|
||||
|------|-------------|
|
||||
| `request_execution` | Hermes-only mutation path. Takes `target` (entity slug) + `action` (verb). Policy-gated. |
|
||||
|
||||
| Action | Risk | Notes |
|
||||
|--------|------|-------|
|
||||
| `refresh-creds` | low | **Drop.** Infisical handles secrets now. |
|
||||
| `nuke` | destructive | Port or drop. Rarely used; destructive — keep as manual SSH. |
|
||||
| `client add/remove` | medium | Client enrollment. Needs secrets-issuance replacement (manual age-keygen now). |
|
||||
| `mcp` | low | MCP tool wrapper. Could call the Go MCP client library. |
|
||||
| Remove `bin/homelab` | **high** | Replace with symlink `oikos` → `bin/homelab`. The `oikos` binary checks `os.Args[0]` for the `homelab` role. |
|
||||
| Remove `bin/oikos` | low | Stale pre-built arm64 binary. Delete. |
|
||||
### Self — ✅ complete
|
||||
|
||||
| Tool | What it does |
|
||||
|------|-------------|
|
||||
| `get_agent_activity` | Agent self-inspection |
|
||||
|
||||
## What's missing — MCP tools to add
|
||||
|
||||
### Phase 1 — Operational visibility (P0, this week)
|
||||
|
||||
Tools Hermes needs to answer "what's happening right now?"
|
||||
|
||||
| Tool | Input | Output | Implementation |
|
||||
|------|-------|--------|----------------|
|
||||
| `tail_log` | `service_slug`, `lines` | Last N log lines via `journalctl` | SSH to host, `journalctl -u <service> -n <lines>` |
|
||||
| `get_service_status` | `service_slug` | `systemctl is-active` / `is-enabled` + uptime | SSH to host, `systemctl show` |
|
||||
| `ping_service` | `service_slug` | HTTP/TCP reachability check | HTTP GET health endpoint from oikos scheduler probe data |
|
||||
| `list_lxcs` | none | All LXCs with ID, host, IP, state | DB query from `entity_status` table |
|
||||
| `get_lxc_state` | `lxc_slug` | RAM, CPU, disk, uptime | SSH to Proxmox host, `pct status <vmid>` |
|
||||
|
||||
### Phase 2 — Execution expansion (P1, next 2 weeks)
|
||||
|
||||
Expand `request_execution` to handle the full operational surface. Currently it writes a generic execution record. It needs to understand action types and route them:
|
||||
|
||||
| Action type | What Hermes says | What actuator does |
|
||||
|-------------|-----------------|--------------------|
|
||||
| `restart` | "restart lxc:caddy" | `systemctl restart <service>` on target host |
|
||||
| `logs` | "show me caddy logs" | Already handled by `tail_log` tool |
|
||||
| `pct_exec` | "run apt update on LXC 121" | `pct exec <vmid> -- <cmd>` on Proxmox host |
|
||||
| `apt_upgrade` | "upgrade packages on lxc:caddy" | `apt update && apt upgrade -y` in detached screen |
|
||||
| `apt_audit` | "audit packages on all LXCs" | `dpkg -l` + upgradable count per host |
|
||||
| `systemctl` | "enable service on lxc:foo" | `systemctl <verb> <service>` (gated: enable = config_mutation) |
|
||||
|
||||
Each action type needs:
|
||||
1. A clear input schema (what params Hermes must provide)
|
||||
2. Policy classification (risk class, blast radius)
|
||||
3. Verification step (how to confirm it worked)
|
||||
4. Ledger entry (what happened, when, by whom)
|
||||
|
||||
### Phase 3 — Agent-to-operator escalation (P1, next 2 weeks)
|
||||
|
||||
When Hermes hits a `config_mutation` or `destructive` action, it escalates to the operator. This path needs to be solid:
|
||||
|
||||
| Component | Current state | Target |
|
||||
|-----------|--------------|--------|
|
||||
| Matrix alert | ⚠️ Notifier polls DB, but Hermes doesn't trigger it | Hermes calls `request_execution` → policy gate rejects → notifier sends Matrix alert with approval token |
|
||||
| Approval flow | ⚠️ `oikos homelab approval` exists in Go but agent can't use it | Operator reacts ✅/❌ on Matrix → webhook → approval token consumed → actuator proceeds |
|
||||
| Execution tracking | ✅ `request_execution` writes to `executions` table | Add `get_execution_status` MCP tool so Hermes can poll for results |
|
||||
|
||||
### Phase 4 — Cleanup (P2, after all tools work)
|
||||
|
||||
| Action | Notes |
|
||||
|--------|-------|
|
||||
| Delete `bin/homelab` | Python CLI. Disconnected from DB, imports deleted modules. Agent handles everything. |
|
||||
| Delete `bin/oikos` | Stale pre-built arm64 binary. Docker image is the canonical build. |
|
||||
| Remove `oikos homelab` Go subcommand | The `homelab` CLI role in the Go binary is a dead end. Remove it or keep as debug-only. |
|
||||
| Update AGENTS.md | Document the complete MCP tool surface. Remove CLI references. |
|
||||
|
||||
## Implementation approach
|
||||
|
||||
Each subcommand is a separate file in `cmd/oikos/homelab/`. The dispatcher
|
||||
in `main.go`'s `runHomelabSubcommand()` routes `os.Args` to the right handler.
|
||||
All subcommands use `internal/config` for DB connection and share the SSH
|
||||
helper from the existing `ssh` subcommand.
|
||||
Each new MCP tool is registered in `internal/mcp/server.go` following the existing pattern:
|
||||
|
||||
SSH connectivity (for pct, logs, restart) reuses the mesh-address resolution
|
||||
from `internal/domain/` — entity slug → inventory attributes → mesh IP →
|
||||
`ssh user@ip`.
|
||||
```go
|
||||
register(&mcp.Tool{Name: "tail_log", Description: "...", InputSchema: ...},
|
||||
func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
// ...
|
||||
})
|
||||
```
|
||||
|
||||
SSH-based tools (`tail_log`, `get_service_status`, `list_lxcs`, `get_lxc_state`)
|
||||
reuse the mesh-address resolution already working for `request_execution`:
|
||||
entity slug → DB lookup → inventory attributes → mesh IP → `ssh user@ip`.
|
||||
|
||||
The actuator container holds the SSH key. MCP tools that need SSH should route
|
||||
through the actuator's execution queue rather than holding their own SSH key,
|
||||
maintaining the security boundary: Hermes → MCP → API → execution queue → actuator → SSH.
|
||||
|
||||
## Verification
|
||||
|
||||
- `oikos homelab` without args prints subcommand list (matching Python parity)
|
||||
- Each P0 subcommand works end-to-end from mac-mini
|
||||
- `bin/homelab` → `oikos` symlink; `homelab list` = `oikos homelab list`
|
||||
- `bin/oikos` removed
|
||||
- Hermes can answer "show me caddy logs" → `tail_log` returns journal lines
|
||||
- Hermes can answer "what's the fleet health?" → `get_health_summary` + `list_lxcs`
|
||||
- Hermes can answer "restart caddy" → `request_execution(action="restart", target="lxc:caddy")` → actuator SSHs → caddy restarts → Hermes confirms
|
||||
- Hermes can answer "upgrade packages on dns LXC" → `request_execution(action="apt_upgrade", target="lxc:dns")` → gated → actuator runs in screen → Hermes reports result
|
||||
- `bin/` directory is empty or contains only Docker-related scripts
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2026-07-07 — plan created. Cataloged 27 Python subcommands vs 4 Go subcommands.
|
||||
- 2026-07-07 rev 2 — flipped paradigm. Operator interface is Hermes via MCP, not CLI.
|
||||
Replaced 4-phase CLI port plan with 4-phase MCP completion plan.
|
||||
- 2026-07-07 rev 1 — cataloged 27 Python CLI subcommands vs 4 Go subcommands.
|
||||
Reference in New Issue
Block a user