docs: README for Oikos identity — agentic homelab OS in Go

- Rewrote README from 'living documentation' to 'Oikos — agentic homelab operating system written in Go.' - Added quick start, architecture diagram, component ports table. - Added Phase 1-6 status table with checkmarks. - Added API usage examples, Hermes query examples, CLI reference. - Added repo layout table. - Note: Gitea repo path (dtoro/Homelab-Docs → dtoro/oikos) requires Gitea UI rename — references in oikos/cards/, bootstrap.sh, and deploy scripts will need updating after the rename.
2026-07-07 17:54:17 +02:00
parent 8175d218c1
commit c9fb5fe553
1 changed files with 100 additions and 161 deletions
--- a/README.md
+++ b/README.md
@@ -1,190 +1,129 @@
-# Homelab OS
+# Oikos

-Living documentation for the **hubris** Proxmox homelab + Oikos operating system.
+Agentic homelab operating system written in Go. Single binary (`cmd/oikos`),
+Docker-deployed on mac-mini, with a standalone Hermes MCP agent gateway
+(`cmd/hermes`). Manages the **hubris** Proxmox homelab autonomously — observes
+state, classifies actions against policy, executes approved procedures over SSH,
+learns from outcomes, and escalates when uncertain.

-**For agents running on enrolled clients:** start with [AGENTS.md](AGENTS.md), then [OIKOS.md](.agents/OIKOS.md).
+**For agents running on enrolled clients:** start with [AGENTS.md](AGENTS.md).

---
+## Quick start

-## For Agents — Navigation & Entry Points
+```bash
+# Dev stack (postgres + api + scheduler + notifier)
+docker compose --profile dev up -d

-### You are running on a client enrolled in the hubris homelab
+# Full stack (adds Hermes agent gateway)
+docker compose --profile full up -d

-1. **First:** Read [AGENTS.md](AGENTS.md) once. It explains who you are, the topology, available tools, conventions, and how to act.
-2. **Before any mutation:** Read [OIKOS.md](.agents/OIKOS.md). It defines the operating model, risk classes, approval flow, and the ontology you'll consult.
-3. **For specific workflows:** Load the matching skill from `.agents/skills/<name>/SKILL.md` (e.g., [service-health-check](.agents/skills/service-health-check/SKILL.md)).
-4. **When in doubt:** Use MCP tools (`search_docs`, `get_page`, `explain`, `get_changelog`) — they're cheaper and more reliable than grepping.
+# Build standalone binary
+go build -o bin/oikos -tags timetzdata ./cmd/oikos

-### Key References for Agents
-
- **What am I?** → `/opt/homelab-context/hosts/<hostname>.yaml` (read on first run)
- **Live topology** → `inventory.yaml` + `hosts/*.yaml` (canonical, always wins)
- **Risk & approval** → [oikos/policy.yaml](oikos/policy.yaml) (enforced, not advisory)
- **Runbooks & workflows** → [.agents/skills/](.agents/skills/) (risk class + verification checklist included)
- **State of Oikos** → [OIKOS.md build status](.agents/OIKOS.md#build-status-30-day-roadmap) (scheduled probes, drift detectors, signals, approval engine)
-
-### When to Use MCP vs Files vs Shell
-
-| Task | Use | Tool |
-|------|-----|------|
-| Resolve hostname → address | MCP | `get_host(name)` or `list_services()` |
-| Search wiki by content | MCP | `search_docs(query)` |
-| Read a wiki page | MCP or file | `get_page(path)` or `cat knowledge/wiki/.../...md` |
-| Get changelog entries | MCP | `get_changelog(page, since?)` |
-| Understand a service | MCP | `explain(service)` — compact context card, cheaper than search+read |
-| Blast-radius query | MCP | `get_relations(entity)` (ontology walk) |
-| List available secrets | MCP | `list_my_secrets()` (scoped to your age key) |
-| Browse or grep | File | Raw `grep` when MCP unreachable, or exploratory browsing |
-
-**When MCP is unreachable:** fall back to grepping the clone at `/opt/homelab-context/`. The local files are the same; MCP is just an index.
-
---
-
-## Understanding the Operating Model
-
-Before you act, **classify your action against [oikos/policy.yaml](oikos/policy.yaml)**.
-
-### The Oikos OODA Loop + Decision Tree
-
-```mermaid
-flowchart TD
-    Observe["**Observe**<br/>probes, drift detectors, agent signals"]
-    Orient["**Orient**<br/>ontology, context, state, entity relations"]
-    Decide{"**Decide**<br/>classify against oikos/policy.yaml"}
-    Auto["Auto-act<br/>(unattended)"]
-    Escalate["Escalate<br/>homelab approval request"]
-    Act["**Act**<br/>homelab CLI, runbooks, skills"]
-    Verify["**Verify**<br/>checklist from SKILL.md"]
-    Ledger["**Ledger**<br/>mutation record: who/what/risk"]
-    Document["**Document**<br/>wiki update, same-session rule"]
-
-    Observe --> Orient --> Decide
-    Decide -->|read_only, reversible_low| Auto
-    Decide -->|config_mutation, destructive| Escalate
-    Auto --> Act
-    Escalate -->|approval granted| Act
-    Act --> Verify --> Ledger --> Document
-    Document -.loop.-> Observe
+# Run all roles in one process (dev mode)
+OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" \
+  go run ./cmd/oikos all
 ```

-### Risk Classes (enforced, not advisory)
+## Architecture

-From [oikos/policy.yaml](oikos/policy.yaml):
+```
+                  ┌──────────────────────────────────┐
+                  │         mac-mini (Docker)         │
+                  │                                   │
+  Workstation ─── │  hermes (8092) ──MCP── api (8090) │
+  (mesh)          │    MCP gateway      REST + MCP    │
+                  │                                   │
+                  │  scheduler ── notifier ── postgres │
+                  │  (observe)    (Matrix)   (Timescale)│
+                  └──────────────────────────────────┘
+```

- **read_only** — status, logs, docs, inventory queries. Unattended. MCP tools are all read_only.
- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger entry.
- **config_mutation** — tracked-config edits (commit+push, never local), deploys, upgrades, DNS/ingress changes. **Operator approval required.**
- **destructive** — destroy, format, wipe, rotate, revoke. **Approval + typed confirmation phrase.**
+| Component | Port | Role |
+|-----------|------|------|
+| `oikos api` | 8090 | REST API + MCP server (15 tools) |
+| `oikos scheduler` | — | Probe runner, signal lifecycle, metrics |
+| `oikos notifier` | — | Approval tokens, Matrix alerts |
+| `hermes serve` | 8092 | MCP client gateway, query routing |

-### Decision Flow
+## Phases

-1. **Decide:** Use `homelab decide <action> <entity>` to classify (risk class × blast radius × confidence).
-2. **Escalate if needed:** `homelab approval request` (Matrix-delivered to operator; see [operations/commands.md](.agents/operations/commands.md)).
-3. **Execute:** Use `homelab` CLI (not ad-hoc SSH) — it enforces policy, logs mutations, and verifies outcomes.
-4. **Document:** Update wiki in the same session (per [AGENTS.md §5](AGENTS.md#5-acting-on-the-homelab) and the [same-session rule](.agents/shared/page-templates.md#same-session-update-rule)).
+| Phase | Status | Description |
+|-------|--------|-------------|
+| 1 — Ontology + DB | ✅ | TimescaleDB, migrations, seeds, blast_radius |
+| 2 — API | ✅ | OpenAPI-first REST + MCP, auth, SSE, audit |
+| 3 — Control loop | ✅ | Scheduler, actuator, learning, classifier, notifier |
+| 4 — Hermes agent | ✅ | Standalone MCP client gateway, agent activity |
+| 5 — Secrets | ✅ | Infisical backend + SOPS fallback, rotation runbooks |
+| 6 — Deploy | ✅ | CI pipeline, cutover checklist, watchdog, rollback |

-### The Ontology Graph
+Full plan: [plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md).

-Everything that can break, be changed, or hold data has an entity in `inventory.yaml` + `oikos/ontology.yaml`. Blast-radius questions ("what breaks if strong goes down?") are graph walks via `homelab node <name> relations`, not doc archaeology.
+## Operations

-**See:** [OIKOS.md](.agents/OIKOS.md) (full operating model, OODA loop, primitives, lifecycle gates, build status).
+### API endpoints

---
+```bash
+curl http://localhost:8090/api/v1/entities?type=service  # fleet
+curl http://localhost:8090/api/v1/health                  # fleet health
+curl http://localhost:8090/api/v1/agent-activity          # agent log
+```

-## Finding & Understanding Information
+### Hermes queries

-The narrative documentation is organized in **layers**:
+```bash
+# Structured tool call
+curl -X POST localhost:8092/query -H "Content-Type: application/json" \
+  -d '{"tool":"get_blast_radius","args":{"entity_id":"service:authentik"}}'

-| Layer | What it is | Where | Immutable? | How agents use it |
-|-------|-----------|-------|-----------|-------------------|
-| **Sources** | Raw evidence: incidents, external refs, live state | `knowledge/sources/investigations/` | Yes | Read to understand root causes; do not rewrite |
-| **Wiki** | Synthesized current-state: one page per node & per system | `knowledge/wiki/{containers,hosts,vms,infrastructure}/` | No | This is the reference layer — if wiki disagrees with live state, update it *in the same session* |
-| **Index** | Pure listings — every page in scope with one-line summary | `index.md` / folder `README.md` | No | Navigation aid; keep it current when wiki restructures |
-| **Log** | Append-only doc-maintenance record (restructures, ingests, lints) | `knowledge/log.md` | Yes (append-only) | Read to understand past doc changes; never edit directly |
+# Natural language
+curl -X POST localhost:8092/query -H "Content-Type: application/json" \
+  -d '{"query":"what depends on authentik?"}'
+```

-**Changelog ≠ Log:** Each wiki page ends with a `## Changelog` (infrastructure changes to that node, machine-parsed). That's not the Log; the Log records *doc operations* only.
+### CLI

-**See:** [llm-wiki.md](.agents/shared/llm-wiki.md) (full rules, page structure, immutability contract).
+```bash
+oikos migrate     # apply DB migrations
+oikos seed        # ingest ontology/inventory/policy seeds
+oikos export      # export DB state to YAML
+oikos api         # serve REST + MCP
+oikos scheduler   # run observe loop
+oikos notifier    # run notification loop
+oikos all         # all roles in one process
+oikos secret list # enumerate SOPS secrets
+oikos secret migrate  # SOPS → Infisical
+```

---
+## Repo layout

-## Map & Quick Navigation
+```
+cmd/oikos/          Go entry point — single binary
+cmd/hermes/         Hermes MCP client gateway
+internal/           Go packages (httpapi, mcp, scheduler, actuator, learning,
+                    notifier, policy, secrets, db, config, ontology, domain)
+api/openapi.yaml    API contract (OpenAPI 3.1)
+migrations/         Forward-only SQL migrations (TimescaleDB)
+seeds/              Bootstrap YAML (ontology, inventory, policy)
+compose/            Dockerfiles + Caddy config
+scripts/            Deploy, watchdog, verification, rollback
+hermes/             Hermes config, persona, skills
+knowledge/wiki/     Narrative documentation (containers, hosts, infrastructure)
+.agents/            Agent instruction files + skills
+plans/              Design documents
+```

-### Agent Entry Points (Start Here)
+## For agents

- **You are an agent** → [AGENTS.md](AGENTS.md) (on deployed clients: `/opt/homelab-context/AGENTS.md`)
- **Operating model & risk policy** → [OIKOS.md](.agents/OIKOS.md)
- **Specific workflows** → [.agents/skills/](.agents/skills/) (load the matching SKILL.md before acting)
- **Operations cheatsheet** → [.agents/operations/commands.md](.agents/operations/commands.md)
- **Tools & MCP reference** → [AGENTS.md §3 — The MCP server](AGENTS.md#3-the-mcp-server)
+See [AGENTS.md](AGENTS.md) for the full orientation. Quick reference:

-### Topology & Infrastructure
+- **Source of truth:** DB (runtime) then repo (bootstrap + docs)
+- **Mutations:** classify against policy, request approval for `destructive`/`config_mutation`
+- **Wiki:** files under `knowledge/wiki/`, changelog at bottom of each page
+- **Secrets:** Infisical (primary) or SOPS (fallback) — never hardcode

-Node counts, IPs, and service lists change often — treat `inventory.yaml` and the index pages below as the source of truth, not this README.
+## Related

- **Proxmox hosts** → [knowledge/wiki/hosts/index.md](knowledge/wiki/hosts/index.md)
- **VMs** → [knowledge/wiki/vms/index.md](knowledge/wiki/vms/index.md)
- **LXC containers** → [knowledge/wiki/containers/index.md](knowledge/wiki/containers/index.md)
- **Cross-cutting infrastructure** (DNS, ingress, mesh, backups, monitoring, auto-deploy, VPS) → [knowledge/wiki/infrastructure/index.md](knowledge/wiki/infrastructure/index.md)
-
-### Knowledge & References
-
- **Glossary** — [GLOSSARY.md](knowledge/GLOSSARY.md)
- **Incidents & investigations** — [knowledge/sources/investigations/index.md](knowledge/sources/investigations/index.md) (active + [archive](knowledge/sources/investigations/archive/))
- **Plans & design docs** — [plans/index.md](plans/index.md)
- **Hermes agent** (for Hermes-enrolled clients) — [HERMES.md](.agents/HERMES.md)
-
---
-
-## Conventions
-
-All pages follow:
-
- **File naming.** Foundational docs (entry-points, agent instruction, references) are ALL-CAPS (`AGENTS.md`, `OIKOS.md`, `GLOSSARY.md`); containers use `<id>-<name>.md`; infrastructure pages use lowercase-with-dashes; plans and incidents use `YYYY-MM-DD-slug.md`; skills are `<name>/SKILL.md`. See [page-templates.md](.agents/shared/page-templates.md#file-naming) for the full rules.
- **Voice & vocabulary.** Concise, technical, sysadmin-to-sysadmin. No marketing prose, no puffers (seamless, robust, leverage, etc.). Full rules in [writing-style.md](.agents/shared/writing-style.md).
- **Cross-linking is mandatory.** If a page references a node or system, link to it. Treat orphans as a bug.
- **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running state, fix the wiki *and* add a changelog entry *in the same session*.
- **Tracked configs.** Pages for configs living in git repos (Caddy, Gitea, Artifacto, mule-image) must note the repo. Edits go through commit+push, never local changes. See [auto-deploy](knowledge/wiki/infrastructure/auto-deploy.md).
- **No secrets.** This is a private repo, but still: reference secret *paths*, never secret *values*.
-
-**For agents:** Read [caveman.md](.agents/shared/caveman.md) (terse communication standard). Use templates at [page-templates.md](.agents/shared/page-templates.md) when creating pages.
-
---
-
-## Updating the Wiki
-
-### When You Change Infrastructure
-
-1. Update the relevant page (config snapshot, ports, mounts, IP address).
-2. Add a `### YYYY-MM-DD — title` entry to the page's `## Changelog` section (reverse chronological order).
-3. If the change touches a cross-cutting system (DNS, Caddy, Authentik, mesh), update *that* page too and link from the changelog.
-4. If it's an incident, add a record to [`knowledge/sources/investigations/`](knowledge/sources/investigations/index.md).
-
-### When You Restructure the Wiki
-
-1. Update the relevant `index.md` / `README.md` in that section.
-2. Add a single-line entry to [`knowledge/log.md`](knowledge/log.md): `## [YYYY-MM-DD] <operation> | <summary>` (e.g., `## [2026-07-06] restructure | split infrastructure/dns into dns.md + dns-advanced.md`).
-
-### The Same-Session Update Rule
-
-**Any meaningful state change made in this session requires a wiki update before the session closes.** A change that touches a container page must also update:
- The `containers/index.md` table (IPs, host, mounts, status)
- The root `README.md` table (if affected)
- The Caddy page site list (if affects `*.hubris.network` routing)
- The DNS / ingress infrastructure pages (if affects routing)
- The `hosts/hubris.md` or `hosts/strong.md` page (if container count changes)
- The `inventory.yaml` host entry (source of truth for `hosts/*.yaml` generation)
- The `knowledge/wiki/infrastructure/topology.md` (regenerate if needed)
-
-Not updating all linked places is a bug. See [page-templates.md — same-session update rule](.agents/shared/page-templates.md#same-session-update-rule).
-
---
-
-## More Information
-
- **For Hermes agents** → [HERMES.md](.agents/HERMES.md) (persona, source-of-truth hierarchy, token efficiency)
- **For manual workflows** → [.agents/operations/](.agents/operations/) (commands cheatsheet, agent enrollment, Hermes guide)
- **For skills/runbooks** → [.agents/skills/](.agents/skills/) (load the matching SKILL.md before acting; includes risk class + verification)
- **MCP tools** → [AGENTS.md §3](AGENTS.md#3-the-mcp-server) (available tools, when to use MCP vs files)
- **Page templates & voice** → [.agents/shared/](.agents/shared/) (page-templates.md, writing-style.md, caveman.md, llm-wiki.md)
- **Machine-readable substrate** → `inventory.yaml`, `oikos/policy.yaml`, `oikos/ontology.yaml` (not part of the wiki; see [llm-wiki.md](.agents/shared/llm-wiki.md#rules))
+- [OIKOS.md](.agents/OIKOS.md) — operating model, OODA loop, ontology
+- [plans/](plans/) — design documents and cutover checklist
+- [docs/adr/](docs/adr/) — architecture decision records