8950bada44f7d0883d87491965e832c3d0e8a7af
Root cause of "running for 10+ minutes without stopping": a real production execution (TypeType pct_create) was found genuinely stuck 17+ minutes into a single blocking SSH call. The container's post_install script was looping on `getent hosts deb.debian.org`, waiting on a network that could never come up — the operator's static IP config used gw:192.168.8.1, but the actual gateway on that subnet is 192.168.8.2, so every network call hung instead of failing fast (packets dropped, not rejected). Two compounding bugs made this unrecoverable without manual intervention: 1. sshExec (both internal/httpapi/phase3.go and internal/mcp/server.go) had NO execution timeout — `session.CombinedOutput()` blocks until the remote command exits, with no deadline. A hung remote process blocks the Go goroutine forever; the execution can never leave 'running', and the operator has no way to make it stop. Fixed: both now race the SSH call against a 10-minute hard timeout, closing the session/client and returning a clear "timed out after 10m0s" error if exceeded. (The mcp/server.go copy also still had the original "swallowed non-zero exit" bug from before that fix was applied to httpapi's copy only — fixed here too.) 2. provisionScript's DNS-wait loop assumed `getent hosts` fails fast on no connectivity — it doesn't; a black-holed network can make each call hang far past the resolver's nominal timeout, so the documented "~90s" budget was never real. Wrapped every attempt in `timeout 3` so the wall-clock budget is now actually enforced (~2min worst case), and the failure message now suggests checking the net0 gateway. Also fixes the matching UI-side gap (operator's literal question: "is there a way to get more details? it has been running for 10+ minutes without stopping"): - InlineApproval's track() polling loop had its own ~6min ceiling and simply STOPPED polling after that — silently going stale before the backend (now correctly capped at 10min) could ever resolve. Raised to a 14min ceiling with margin, and added a distinct 'stalled' state if that's ever exceeded (explicitly says something's wrong, rather than freezing silently). - The running-card now shows live elapsed time (ticking, from the execution's created_at), the actual command being run, and the execution ID — previously just a static "this can take a minute" with zero information. Also added command display to the destructive pending- approval card for full transparency before confirming. Verified live end-to-end in a real browser (dev server proxying to production): queued a real command via chat, approved via the button, watched the elapsed-time counter tick in real time, and saw it transition to a completed card with real output once the command finished. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
adr: convert all diagrams to Mermaid (sequenceDiagram, stateDiagram-v2, flowchart, erDiagram, graph)
Oikos
Agentic homelab operating system written in Go. Single binary (cmd/oikos),
Docker-deployed on mac-mini, with a standalone Nomos MCP agent gateway
(cmd/nomos). Manages the hubris Proxmox homelab autonomously — observes
state, classifies actions against policy, executes approved procedures over SSH,
learns from outcomes, and escalates when uncertain.
For agents running on enrolled clients: start with AGENTS.md. For client machines: see CLIENTS.md. For developers: see CONTRIBUTING.md.
Quick start
# Dev stack (postgres + api + scheduler + notifier)
docker compose --profile dev up -d
# Full stack (adds Nomos agent gateway)
docker compose --profile full up -d
# Build standalone binary
go build -o bin/oikos -tags timetzdata ./cmd/oikos
# Run all roles in one process (dev mode)
OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" \
go run ./cmd/oikos all
Architecture
┌──────────────────────────────────┐
│ mac-mini (Docker) │
│ │
Workstation ─── │ nomos (8092) ──MCP── api (8090) │
(mesh) │ MCP gateway REST + MCP │
│ │
│ scheduler ── notifier ── postgres │
│ (observe) (Matrix) (Timescale)│
└──────────────────────────────────┘
| Component | Port | Role |
|---|---|---|
oikos api |
8090 | REST API + MCP server (15 tools) |
oikos scheduler |
— | Probe runner, signal lifecycle, metrics |
oikos notifier |
— | Approval tokens, Matrix alerts |
nomos serve |
8092 | MCP client gateway, query routing |
Phases
| Phase | Status | Description |
|---|---|---|
| 1 — Ontology + DB | ✅ | TimescaleDB, migrations, seeds, blast_radius |
| 2 — API | ✅ | OpenAPI-first REST + MCP, auth, SSE, audit |
| 3 — Control loop | ✅ | Scheduler, actuator, learning, classifier, notifier |
| 4 — Nomos agent | ✅ | Standalone MCP client gateway, agent activity |
| 5 — Secrets | ✅ | Infisical backend + SOPS fallback, rotation runbooks |
| 6 — Deploy | ✅ | CI pipeline, cutover checklist, watchdog, rollback |
Full plan: plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md.
Operations
API endpoints
curl http://localhost:8090/api/v1/entities?type=service # fleet
curl http://localhost:8090/api/v1/health # fleet health
curl http://localhost:8090/api/v1/agent-activity # agent log
Nomos queries
# Structured tool call
curl -X POST localhost:8092/query -H "Content-Type: application/json" \
-d '{"tool":"get_blast_radius","args":{"entity_id":"service:authentik"}}'
# Natural language
curl -X POST localhost:8092/query -H "Content-Type: application/json" \
-d '{"query":"what depends on authentik?"}'
CLI
oikos migrate # apply DB migrations
oikos seed # ingest ontology/inventory/policy seeds
oikos export # export DB state to YAML
oikos api # serve REST + MCP
oikos scheduler # run observe loop
oikos notifier # run notification loop
oikos all # all roles in one process
oikos secret list # enumerate SOPS secrets
oikos secret migrate # SOPS → Infisical
Repo layout
cmd/oikos/ Go entry point — single binary
cmd/nomos/ Nomos MCP client gateway
internal/ Go packages (httpapi, mcp, scheduler, actuator, learning,
notifier, policy, secrets, db, config, ontology, domain,
knowledge)
api/openapi.yaml API contract (OpenAPI 3.1)
migrations/ Forward-only SQL migrations (TimescaleDB)
seeds/ Bootstrap YAML (ontology, inventory, policy, knowledge)
compose/ Dockerfiles + Caddy config
scripts/ Deploy, watchdog, verification, rollback
nomos/ Nomos config, persona, skills
.agents/ Agent instruction files, shared conventions, skills
archive/ Historical reference (legacy wiki, plans, SOPS backups)
plans/ Design documents (active + done)
docs/adr/ Architecture decision records
For agents
See AGENTS.md for the full orientation. Quick reference:
- Source of truth: DB (runtime) then seeds (bootstrap). Old wiki is
archived at
archive/knowledge/— use MCPsearch_knowledgeinstead. - Mutations: classify against policy, request approval for
destructive/config_mutation - Secrets: Infisical (primary) or SOPS (fallback) — never hardcode
Related
- OIKOS.md — operating model, OODA loop, ontology
- CLIENTS.md — client onboarding guide
- CONTRIBUTING.md — developer guide
- plans/ — design documents and cutover checklist
- docs/adr/ — architecture decision records
Description
Languages
Go
53.1%
Svelte
25.7%
TypeScript
14%
Shell
3.8%
Python
1.7%
Other
1.5%