dtoro 279549c8c9 fix: scheduler wrote health/metrics/events to probe entities, not targets
Problem: every host/service/lxc/etc. entity_status row was permanently
stuck at 'unknown' since creation. Verified against the live DB:
metric_samples had 17,559 rows, 100% attached to type='check' probe
entities and 0% to any real monitored entity; only 25 check entities
ever had real health written. check_defs.entity_id (the probe's own
bookkeeping entity) and check_defs.target_id (the host/service actually
being observed) were both real fields, but the scheduler wrote
UpsertEntityStatus/InsertMetricSample/emitSchedulerEvent keyed by
entity_id instead of target_id — so every check ran and every result
was real, it just landed on the wrong row. This is the mechanism behind
observed drift: the agent's dashboard/health tools reported the
internal probes' state, never the actual fleet.

Change:
- scheduler.go: runCheck/resolveSignal now resolve targetID from
  cd.TargetID (falling back to the check's own id if unset) and write
  status/metrics/events there. Signals stay keyed by the check entity,
  unchanged, matching their existing resolution logic.
- Added a staleness sweep to housekeeping(): an entity whose last
  observation is older than 3x its fastest enabled check's interval
  (floor 5m) is marked 'stale' and emits health.stale, so a stalled
  scheduler or disabled check_def can no longer look like current data
  forever.
- migrations/016: deletes the now-orphaned check-entity entity_status
  rows so dashboard/fleet-health rollups stop double-counting probes as
  monitored entities. Historical metric_samples on check entities are
  left as-is (time-series data, not safe to reattribute).
- openapi.yaml + regenerated gen code: Entity gains health/last_check_at;
  'stale' added to the health enum everywhere it's used.
- dashboard.go / GetFleetHealth / nomos's get_health_summary MCP tool:
  exclude type='check' entities from rollups.
- nomos/agent.go: replay prior turns' tool_use/tool_result pairs into
  the conversation instead of dropping them (previously only final text
  was replayed, forcing the agent to re-derive fleet state every turn),
  and inject a compact live fleet-health snapshot into the system prompt
  each turn so it starts oriented instead of spending an iteration on
  discovery.

Risk: config_mutation (schema-adjacent — new migration, no destructive
DDL, additive DELETE only on orphaned rows). No behavior change until
oikos-api/oikos-scheduler/nomos are rebuilt and redeployed.

Verification: go build/vet clean across the repo. Ran this worktree's
own API binary against the live dev Postgres on an alternate port
(read-only from the live containers' perspective) and confirmed
/api/v1/entities now returns health/last_check_at, and the dashboard
health rollup dropped from double-counting to an honest 168 unmonitored
entities (matches reality pre-deploy — the live scheduler hasn't run
the fixed code yet). Confirmed check_defs.target_id correctly maps
multiple checks to host:hubris via direct psql query.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-09 00:26:04 +02:00

Oikos

Agentic homelab operating system written in Go. Single binary (cmd/oikos), Docker-deployed on mac-mini, with a standalone Nomos MCP agent gateway (cmd/nomos). Manages the hubris Proxmox homelab autonomously — observes state, classifies actions against policy, executes approved procedures over SSH, learns from outcomes, and escalates when uncertain.

For agents running on enrolled clients: start with AGENTS.md. For client machines: see CLIENTS.md. For developers: see CONTRIBUTING.md.

Quick start

# Dev stack (postgres + api + scheduler + notifier)
docker compose --profile dev up -d

# Full stack (adds Nomos agent gateway)
docker compose --profile full up -d

# Build standalone binary
go build -o bin/oikos -tags timetzdata ./cmd/oikos

# Run all roles in one process (dev mode)
OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" \
  go run ./cmd/oikos all

Architecture

                  ┌──────────────────────────────────┐
                  │         mac-mini (Docker)         │
                  │                                   │
  Workstation ─── │  nomos (8092) ──MCP── api (8090) │
  (mesh)          │    MCP gateway      REST + MCP    │
                  │                                   │
                  │  scheduler ── notifier ── postgres │
                  │  (observe)    (Matrix)   (Timescale)│
                  └──────────────────────────────────┘
Component Port Role
oikos api 8090 REST API + MCP server (15 tools)
oikos scheduler Probe runner, signal lifecycle, metrics
oikos notifier Approval tokens, Matrix alerts
nomos serve 8092 MCP client gateway, query routing

Phases

Phase Status Description
1 — Ontology + DB TimescaleDB, migrations, seeds, blast_radius
2 — API OpenAPI-first REST + MCP, auth, SSE, audit
3 — Control loop Scheduler, actuator, learning, classifier, notifier
4 — Nomos agent Standalone MCP client gateway, agent activity
5 — Secrets Infisical backend + SOPS fallback, rotation runbooks
6 — Deploy CI pipeline, cutover checklist, watchdog, rollback

Full plan: plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md.

Operations

API endpoints

curl http://localhost:8090/api/v1/entities?type=service  # fleet
curl http://localhost:8090/api/v1/health                  # fleet health
curl http://localhost:8090/api/v1/agent-activity          # agent log

Nomos queries

# Structured tool call
curl -X POST localhost:8092/query -H "Content-Type: application/json" \
  -d '{"tool":"get_blast_radius","args":{"entity_id":"service:authentik"}}'

# Natural language
curl -X POST localhost:8092/query -H "Content-Type: application/json" \
  -d '{"query":"what depends on authentik?"}'

CLI

oikos migrate     # apply DB migrations
oikos seed        # ingest ontology/inventory/policy seeds
oikos export      # export DB state to YAML
oikos api         # serve REST + MCP
oikos scheduler   # run observe loop
oikos notifier    # run notification loop
oikos all         # all roles in one process
oikos secret list # enumerate SOPS secrets
oikos secret migrate  # SOPS → Infisical

Repo layout

cmd/oikos/          Go entry point — single binary
cmd/nomos/          Nomos MCP client gateway
internal/           Go packages (httpapi, mcp, scheduler, actuator, learning,
                    notifier, policy, secrets, db, config, ontology, domain,
                    knowledge)
api/openapi.yaml    API contract (OpenAPI 3.1)
migrations/         Forward-only SQL migrations (TimescaleDB)
seeds/              Bootstrap YAML (ontology, inventory, policy, knowledge)
compose/            Dockerfiles + Caddy config
scripts/            Deploy, watchdog, verification, rollback
nomos/              Nomos config, persona, skills
.agents/            Agent instruction files, shared conventions, skills
archive/            Historical reference (legacy wiki, plans, SOPS backups)
plans/              Design documents (active + done)
docs/adr/           Architecture decision records

For agents

See AGENTS.md for the full orientation. Quick reference:

  • Source of truth: DB (runtime) then seeds (bootstrap). Old wiki is archived at archive/knowledge/ — use MCP search_knowledge instead.
  • Mutations: classify against policy, request approval for destructive/config_mutation
  • Secrets: Infisical (primary) or SOPS (fallback) — never hardcode
Description
Agentic OS for running a Homelab
Readme 37 MiB
Languages
Go 53.1%
Svelte 25.7%
TypeScript 14%
Shell 3.8%
Python 1.7%
Other 1.5%