diff --git a/plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md b/plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md index 2526cf8..a4929bc 100644 --- a/plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md +++ b/plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md @@ -1,21 +1,22 @@ # Plan: Oikos — Docker-based agentic homelab OS on mac-mini -**Status:** Planned (2026-07-06, revised) — supersedes the launchd-based consolidation plan. -**Predecessor:** The original plan re-platformed Oikos services from LXC 105 (apps) to -mac-mini via launchd plists. This revision pivots to a **Docker-based** architecture: -the repo becomes a containerized homelab OS, deployed via `docker compose` on mac-mini. +**Status:** Planned (2026-07-06, rev 2) — supersedes the launchd-based plan and the +Python/Docker rev 1. This revision introduces: Go instead of Python, ontology-first +design with systems modeling, DB-native config (inventory/ontology/policy as graph +metadata, not YAML files), and a feedback loop where the agent learns from execution. ## Vision -Convert this repo into a **Docker-based agentic homelab OS**. The OS is a set of -containerized services that manage the homelab autonomously, with the operator in -control. Two actors: +Convert this repo into a **Docker-based agentic homelab OS** written in **Go**. The OS +is a set of containerized services that manage the homelab autonomously, with the +operator in control. Two actors: - **Operator** (dtoro) — owns the homelab, expresses intent ("install X", "restart Y"), approves destructive actions. Connects from any workstation via remote Hermes or Matrix. - **Agent** — Hermes core + custom homelab skills, running in Docker. Executes orders, - monitors the lab, escalates when unsure. + monitors the lab, escalates when unsure, and **learns from every action** to improve + over time. All OS services run in Docker containers on mac-mini. The OS is deployed by a git push (Gitea webhook → Docker rebuild). It's designed for mac-mini now, with a path @@ -25,22 +26,254 @@ to multi-node later. | Question | Decision | |---|---| -| Repo structure | One repo, reorganize internally. Wiki docs get restructured into the knowledge graph. | -| Hermes runtime | Runs inside Docker as part of the OS stack (gateway mode — connect from any workstation). | -| Agent → homelab access | Hybrid — mounted SSH keys for now, build actuator gateway incrementally. | -| Data storage | Migrate to PostgreSQL. `inventory.yaml` stays as declarative config; signals/ledger/state/knowledge go into DB. | -| Knowledge/context | Structured knowledge graph in Postgres — docs become typed entities linked to inventory entities. | +| Repo structure | One repo, reorganize internally. | +| Language | **Go** — compiled, type-safe, small containers, goroutines for concurrent probes. | +| Hermes runtime | Runs inside Docker as part of the OS stack (gateway mode). | +| Agent → homelab access | Hybrid — mounted SSH keys now, actuator gateway built incrementally. | +| Data storage | PostgreSQL. Inventory/ontology/policy become DB-native graph metadata. | +| Config (inventory/ontology/policy) | **DB-native** — YAML files are seed manifests only (bootstrap + DR). The DB is the runtime source of truth. Editable via API, future frontend. | +| Knowledge/context | Structured knowledge graph in Postgres — ontology-defined entity types, typed relationships, linked to operational data. | | Operator interface | Primary = remote Hermes from any workstation + Matrix. Console/UIs built later for specific tasks. | | Host | mac-mini for now, designed to scale later. | | Deploy | Git push → Gitea webhook → Docker rebuild + restart. | -| `bin/homelab` CLI | Replaced by an API. CLI becomes a thin client that calls the OS API. | -| Secrets | Migrate from SOPS+age to Infisical (open-source, API-first, Docker-native). | +| `bin/homelab` CLI | Replaced by an API. CLI becomes a thin Go client that calls the OS API. | +| Secrets | Migrate from SOPS+age to Infisical. | | Matrix | Keep for now, abstract the notification layer for future channels. | -| MCP server | Merged into the unified API — one service, REST + MCP interfaces. | -| Agent type | Hermes core + custom homelab skills (standard Hermes, purpose-loaded). | +| MCP server | Merged into the unified API — one Go service, REST + MCP interfaces. | +| Agent type | Hermes core + custom homelab skills. | +| Feedback loop | **Agent learns from execution** — outcomes feed back as patterns and skills that inform future decisions. | +| Ontology | **Developed first**, before knowledge graph ingestion. Systems modeling: entities, connections, lifecycles. | | apps/105 | Keep running as fallback until Docker OS is proven. | | mac-mini cleanup | Start fresh in a new directory, clean up old artifacts later. | +## Ontology — the systems model + +The ontology is the foundational layer of the OS. It defines what exists, how things +connect, how they change over time, and how the OS learns about them. It is stored +**in the database** as metadata (entity types, relationship types, lifecycle +definitions). YAML seed files bootstrap it on first deploy; after that, the DB is +authoritative and editable via API. + +### Design principles (systems modeling) + +1. **Three layers** — Infrastructure (the managed world), Governance (who controls + what), Cognition (the OS's own behavior + learning). Dependencies flow upward: + Cognition depends on Governance depends on Infrastructure. +2. **Everything is an entity** — if it can break, be changed, or hold data, it has an + entity type and edges. The OS's own objects (signals, changes, skills) are + first-class entities, not second-class records. +3. **Typed relationships with cardinality** — edges carry semantics. `hosts` is + one-to-many; `depends-on` is many-to-many; `documents` is one-to-one. The graph + is queryable for blast radius, dependency chains, and knowledge lookup. +4. **Lifecycles are state machines** — every entity type has a lifecycle. Infrastructure + entities move through `planned → active → destroyed`. Operational entities have + their own lifecycles (signals, approvals, patterns, skills). Transitions can require + preconditions. +5. **Policy is attached to the ontology** — risk classes and approval rules link to + entity types and actions. The policy IS part of the model, not a separate file. +6. **Learning is modeled** — executions produce outcomes, outcomes accumulate into + patterns, patterns refine skills, skills inform future decisions. This is an + explicit, queryable part of the graph. + +### Layer map + +```mermaid +graph TB + subgraph cognition["Layer 3 — Cognition (the OS's behavior + learning)"] + direction LR + OBS["Observation\nsignal, state-snapshot"] + DEC["Decision\nclassification, risk-assessment"] + ACT["Action\nchange, execution, verification"] + GOV["Governance\napproval-request, approval-decision"] + KNOW["Knowledge\ndocument, runbook, lesson"] + LEARN["Learning\npattern, skill, feedback"] + end + + subgraph governance["Layer 2 — Governance (who controls what)"] + direction LR + IDENT["Identity\nperson, agent, identity-provider"] + SEC["Secrets\nsecret, key, access-grant"] + POL["Policy\nrisk-class, approval-rule, autonomy-setting"] + end + + subgraph infra["Layer 1 — Infrastructure (the managed world)"] + direction LR + PHYS["Physical\nsite, machine, ups, sensor"] + COMP["Compute\nproxmox-host, lxc, vm,\nworkstation, container"] + NET["Network\nlan, mesh, dns-zone,\ningress-route, certificate"] + STOR["Storage\nstorage-pool, volume,\nmount, backup-target"] + SOFT["Software\nservice, application,\nconfig-repo, deploy-pipeline"] + end + + cognition -- "observes, acts on, learns about" --> infra + governance -- "governs access to" --> infra + governance -- "constrains" --> cognition + cognition -- "creates + refines" --> governance +``` + +### Entity relationship diagram — core entities and typed edges + +```mermaid +erDiagram + MACHINE ||--o{ PROXMOX_HOST : "is-a" + PROXMOX_HOST ||--o{ LXC : hosts + PROXMOX_HOST ||--o{ VM : hosts + PROXMOX_HOST ||--o{ WORKSTATION : hosts + LXC ||--o{ SERVICE : provides + VM ||--o{ SERVICE : provides + WORKSTATION ||--o{ SERVICE : provides + LXC ||--o{ MOUNT : has + MOUNT }o--|| STORAGE_POOL : "stores-on" + SERVICE ||--o{ INGRESS_ROUTE : "exposed-by" + INGRESS_ROUTE }o--|| CERTIFICATE : "secured-by" + INGRESS_ROUTE }o--|| IDENTITY_PROVIDER : "secured-by" + SERVICE ||--o{ SERVICE : "depends-on" + CONFIG_REPO ||--|| SERVICE : "configured-by" + DEPLOY_PIPELINE ||--|| SERVICE : "deploys-to" + SERVICE ||--o{ SIGNAL : "monitored-by" + SIGNAL ||--o{ EXECUTION : triggers + EXECUTION ||--|| FEEDBACK : produces + FEEDBACK }o--|| PATTERN : "contributes-to" + PATTERN }o--|| SKILL : "informs" + SKILL ||--o{ CLASSIFICATION : "guides" + CLASSIFICATION ||--|| EXECUTION : "precedes" + EXECUTION ||--o| APPROVAL : "requires" + PERSON ||--o{ APPROVAL : "decides" + DOCUMENT ||--o{ SERVICE : "describes" + RUNBOOK ||--o{ SERVICE : "procedure-for" + AGENT ||--o{ EXECUTION : "performs" + PERSON ||--o{ AGENT : "owns" +``` + +### Infrastructure lifecycle + +```mermaid +stateDiagram-v2 + [*] --> planned : operator creates entity + planned --> provisioning : IP reserved, storage chosen, doc stub + provisioning --> active : mesh joined, health-check answering, doc complete + active --> migrating : preflight + backup verified + migrating --> active : post-verify, caddy checked, mounts checked + active --> deprecated : replacement live or role retired + deprecated --> destroyed : backups verified, secrets revoked, ingress removed + destroyed --> [*] : archaeology entry recorded + + note right of deprecated + Complete only when + zero inbound edges remain + (no depends-on, no routes-to) + end note +``` + +### Signal lifecycle + +```mermaid +stateDiagram-v2 + [*] --> raised : scheduler probe or agent finding + raised --> acknowledged : agent or operator sees it + raised --> muted : operator suppresses (TTL) + acknowledged --> acting : actuator starts execution + acting --> resolved : action succeeded, verification passed + acting --> raised : action failed, re-escalated + raised --> resolved : condition cleared (auto-resolve) + muted --> raised : TTL expired + resolved --> [*] +``` + +### Execution + learning lifecycle + +```mermaid +stateDiagram-v2 + [*] --> proposed : signal + recommended_action + proposed --> approved : operator approves (if gated) + proposed --> auto_approved : risk class allows auto-act + approved --> executing : actuator runs + auto_approved --> executing : actuator runs + executing --> verified : verification command passed + executing --> failed : execution or verification failed + executing --> timed_out : exceeded duration limit + failed --> rolled_back : rollback procedure executed + verified --> [*] : feedback recorded, pattern updated + failed --> [*] : feedback recorded, pattern updated + rolled_back --> [*] : feedback recorded, pattern updated + timed_out --> [*] : feedback recorded, pattern updated +``` + +### Feedback / learning model — the cognition loop + +```mermaid +flowchart TB + subgraph observe["Observe"] + SIG["Signal raised\n(service down, disk full, drift)"] + end + + subgraph decide["Decide"] + CLASS["Classification\nrisk × blast radius × confidence"] + SKILL_LOOKUP["Skill lookup\nbest-known procedure for\nthis entity type + action"] + CLASS --> SKILL_LOOKUP + end + + subgraph act["Act"] + EXEC["Execution\nfollow skill procedure\n(or escalate if no skill)"] + VERIFY["Verification\ncheck if action succeeded"] + EXEC --> VERIFY + end + + subgraph learn["Learn (feedback loop)"] + OUTCOME["Outcome evaluation\nsuccess / failure / partial / unexpected"] + FEEDBACK["Feedback record\nwhat happened vs expected\nextractable lesson"] + PATTERN["Pattern extraction\naccumulate feedback on\nsimilar entity+action pairs"] + SKILL_REFINE["Skill refinement\nupdate or create skill\nbased on validated patterns"] + + OUTCOME --> FEEDBACK + FEEDBACK --> PATTERN + PATTERN --> SKILL_REFINE + end + + SIG --> CLASS + SKILL_LOOKUP --> EXEC + VERIFY --> OUTCOME + SKILL_REFINE -. "informs next decision" .-> SKILL_LOOKUP + + subgraph escalation["Escalation path"] + APPROVAL["Approval request\n→ Matrix ✅/❌"] + end + + CLASS -- "needs approval" --> APPROVAL + APPROVAL -- "approved" --> EXEC + APPROVAL -- "denied" --> RESOLVE["Resolve signal\nnote: denied"] +``` + +### Policy model — how risk attaches to entities + +```mermaid +flowchart LR + subgraph ontology["Ontology (DB metadata)"] + ET["entity_types\nhost, service, lxc, vm, ..."] + RT["relationship_types\nhosts, provides, depends-on, ..."] + LD["lifecycle_defs\nstates + transitions"] + end + + subgraph policy["Policy (DB records)"] + RC["risk_classes\nread_only, reversible_low,\nconfig_mutation, destructive"] + RULES["approval_rules\nentity_type + action → risk_class\n+ approval_required + autonomy_level"] + AUTO["autonomy_settings\nglobal: auto_act on/off\nper-entity: never_auto_act"] + end + + subgraph instances["Instances (DB data)"] + ENT["entities\nhost:hubris, service:caddy, ..."] + REL["relationships\nhubris hosts lxc:apps"] + end + + ET --> RULES + RC --> RULES + RULES --> ENT + AUTO --> ENT + ET --> ENT + RT --> REL + LD --> ENT +``` + ## Target architecture ### Container stack on mac-mini @@ -48,16 +281,15 @@ to multi-node later. ```mermaid graph TB subgraph mac-mini["mac-mini — Docker host, always-on"] - subgraph services["Docker Compose"] - PG["PostgreSQL\n• inventory (synced from YAML)\n• signals • ledger\n• knowledge graph\n• state snapshots\n• approvals"] + subgraph services["Docker Compose — Go binaries"] + PG["PostgreSQL 16\n• entity/relationship store\n• signals • ledger\n• knowledge graph\n• patterns + skills\n• policy + ontology metadata"] INF["Infisical\n(secrets manager)"] HERMES["Hermes Agent — gateway mode\n+ homelab skills\n+ MCP client → API\n+ SSH keys (mounted)"] - API["Oikos API — REST + MCP\n\nMCP: get_host, list_services,\nsearch_knowledge, get_relations\nREST: /hosts, /services, /signals,\n/exec, /approve, /deploy, /events\n\nPolicy enforcement + risk\nclassification + ledger"] - SCHED["Scheduler (Observe)\n+ Actuator (Act)\n• 10-min probes → DB\n• Classify → auto-act or escalate\n• SSH to hubris/strong"] - NOTIFIER["Notifier — abstracted\n• Matrix (current): alerts,\napproval requests\n• Future: webhook, email, etc."] + API["Oikos API — Go (Gin)\n\nMCP: get_host, list_services,\nsearch_knowledge, get_relations\nREST: /hosts, /services, /signals,\n/exec, /approve, /deploy, /events\n\nPolicy enforcement + risk\nclassification + ledger"] + SCHED["Scheduler (Observe) — Go\n+ Actuator (Act) — Go\n• 10-min probes → DB\n• Classify → auto-act or escalate\n• Execution → feedback → patterns\n• SSH to hubris/strong"] + NOTIFIER["Notifier — Go\n• Matrix (current)\n• Future: webhook, email"] end DEPLOY["Gitea webhook →\ndocker compose build + up -d"] - VOLS["Volumes: pg-data, infisical-data,\nssh-keys (ro), hermes-config"] end HERMES -- MCP --> API @@ -71,10 +303,10 @@ graph TB subgraph external["External"] CADDY["Caddy (LXC 121)\n→ mac-mini mesh :8090"] - APPS["apps/105 (fallback)\nkept running until proven"] + APPS["apps/105 (fallback)"] HUBRIS["hubris (PVE)"] STRONG["strong (PVE)"] - GITEA["Gitea (LXC 104)\nwebhook trigger"] + GITEA["Gitea (LXC 104)"] MATRIX["Matrix (LXC 118)"] WS["Any workstation\nHermes remote → gateway"] end @@ -82,42 +314,20 @@ graph TB CADDY -- reverse_proxy --> API GITEA -- webhook --> DEPLOY WS -- "Hermes gateway" --> HERMES - APPS -. "fallback" .- API ``` -### OODA loop — the control flow +### OODA loop — with learning feedback ```mermaid flowchart LR OBSERVE["Observe\nScheduler probes:\n• HTTP health\n• disk usage\n• drift detection"] --> ORIENT["Orient\nRelations graph walk:\n• blast radius\n• lifecycle state\n• runbook match"] - ORIENT --> DECIDE["Decide\nRisk classifier:\nrisk × blast × confidence"] - DECIDE -- "auto-act\n(reversible_low,\ncontained, confident)" --> ACT["Act\nExecute via SSH/API\n→ verify → resolve\n→ ledger entry"] - DECIDE -- "escalate\n(approval needed)" --> APPROVE["Approval request\n→ Matrix ✅/❌\n→ grant or deny"] + ORIENT --> DECIDE["Decide\nRisk classifier +\nskill lookup:\nrisk × blast × confidence"] + DECIDE -- "auto-act" --> ACT["Act\nExecute via SSH\n→ verify → feedback"] + DECIDE -- "escalate" --> APPROVE["Approval\n→ Matrix ✅/❌"] APPROVE -- "approved" --> ACT - APPROVE -- "denied" --> RESOLVE["Resolve signal\nnote: denied"] - ACT --> OBSERVE -``` - -### Knowledge graph — docs to DB - -```mermaid -flowchart TB - subgraph repo["Repo (git-tracked)"] - INV["inventory.yaml\ndeclarative topology"] - DOCS["docs/\n• containers/105-apps.md\n• hosts/hubris.md\n• infrastructure/dns.md\n• investigations/..."] - end - subgraph db["PostgreSQL"] - ENT["entities table\n(host:hubris, service:caddy, ...)"] - REL["relations table\n(hosts, provides, depends-on, ...)"] - KNOW["knowledge_entities table\n(doc:105-apps, doc:hubris, ...)"] - KNOWREL["knowledge_relations table\n(describes, relates-to, runbook-for)"] - end - INGEST["Ingest pipeline\n(runs on deploy)"] - INV --> INGEST --> ENT - INV --> INGEST --> REL - DOCS --> INGEST --> KNOW - DOCS --> INGEST --> KNOWREL - KNOW -. "linked by entity_id" .- ENT + ACT --> LEARN["Learn\nOutcome → feedback\n→ pattern → skill"] + LEARN -. "improves confidence" .-> DECIDE + LEARN --> OBSERVE ``` ### Deploy flow @@ -127,592 +337,734 @@ flowchart LR DEV["Operator\nedits repo"] --> PUSH["git push"] --> GITEA["Gitea\n(LXC 104)"] GITEA -- "webhook" --> MACMINI["mac-mini\ndeploy script"] MACMINI -- "git pull" --> REPO["repo clone"] - MACMINI -- "docker compose\nbuild + up -d" --> STACK["OS containers\nrebuilt + restarted"] - STACK -- "ingest" --> DB["PostgreSQL\nupdated"] + MACMINI -- "go build +\ndocker compose up -d" --> STACK["OS containers\nrebuilt + restarted"] + STACK -- "seed ingest" --> DB["PostgreSQL\nontology + inventory + policy\nsynced from YAML seeds"] ``` -## Repo reorganization +## DB-native configuration -The repo shifts from "wiki + code mixed" to "OS code + declarative config + docs as -ingest source." Proposed layout: +The three YAML files — `inventory.yaml`, `ontology.yaml`, `policy.yaml` — become +**seed manifests**. They bootstrap the DB on first deploy. After that, the DB is the +runtime source of truth, editable via the API. A future frontend can edit all three +directly. + +### How it works + +```mermaid +flowchart TB + subgraph seeds["Seed manifests (git-tracked, YAML)"] + ONTO_YAML["seeds/ontology.yaml\nentity types, relationship types,\nlifecycle definitions"] + INV_YAML["seeds/inventory.yaml\nentity instances (hosts, services,\nnetworks, storage)"] + POL_YAML["seeds/policy.yaml\nrisk classes, approval rules,\nautonomy settings"] + end + + subgraph db["PostgreSQL (runtime source of truth)"] + META["entity_types table\nrelationship_types table\nlifecycle_defs table"] + INST["entities table\nrelationships table"] + POLDB["policies table\nrisk_classes table\nautonomy_settings table"] + end + + INGEST["Seed ingest (on deploy)\nidempotent upsert"] + ONTO_YAML --> INGEST --> META + INV_YAML --> INGEST --> INST + POL_YAML --> INGEST --> POLDB + + API_EDIT["API edits\n(POST/PUT/PATCH)"] + API_EDIT --> META + API_EDIT --> INST + API_EDIT --> POLDB + + EXPORT["Export to YAML\n(for DR / version control)"] + META --> EXPORT + INST --> EXPORT + POLDB --> EXPORT +``` + +### Why DB-native + +- **Querying** — the agent can ask "what services depend on authentik?" as a graph + query, not a YAML parse. Blast-radius walks are SQL, not file reads. +- **Mutation** — adding a service, updating a lifecycle state, changing a policy rule + are DB transactions with audit trail, not file edits + git commits. +- **Consistency** — the ontology, inventory, and policy are always in sync (same DB, + same transaction). No drift between what the YAML says and what the runtime sees. +- **Future frontend** — a UI can edit entities, relationships, and policies directly + via the API. No need to generate/edit YAML files. +- **Version control** — seed YAML files are still git-tracked for bootstrap and DR. + The API can export the current DB state back to YAML for commit. + +## Repo layout (Go project) ``` -/ # repo root -├── docker-compose.yml # the OS stack definition -├── compose/ # per-service Dockerfiles + build context -│ ├── api/ # Oikos API (REST + MCP) — merges mcp/ + bin/homelab -│ │ └── Dockerfile -│ ├── scheduler/ # Observe + Act loop -│ │ └── Dockerfile -│ ├── hermes/ # Hermes agent runtime -│ │ └── Dockerfile -│ ├── postgres/ # init scripts, migrations -│ │ └── init.sql -│ └── infisical/ # Infisical config -│ └── config.yaml -├── oikos/ # Core OS Python package (reused, adapted) -│ ├── api/ # NEW — unified API server -│ │ ├── server.py # FastAPI app, REST routes + MCP protocol adapter -│ │ ├── routes/ # REST route modules -│ │ └── mcp.py # MCP protocol adapter (exposes same tools via MCP) -│ ├── db/ # NEW — database layer -│ │ ├── models.py # SQLAlchemy models: Inventory, Signal, Ledger, Knowledge, Approval -│ │ ├── migrations/ # Alembic migrations -│ │ └── ingest.py # Ingest inventory.yaml + docs into DB on deploy -│ ├── scheduler/ # Adapted from scheduler.py — reads/writes DB -│ ├── actuator/ # NEW — Act stage: reads signals, classifies, executes -│ │ └── act.py -│ ├── policy/ # Adapted — policy.yaml loader + classifier -│ ├── knowledge/ # NEW — knowledge graph ingestion from docs -│ │ └── ingest_docs.py -│ ├── notifier/ # NEW — abstracted notification (Matrix impl) -│ │ ├── base.py # Notifier interface -│ │ └── matrix.py # Matrix implementation -│ ├── decide.py # Reused — risk classifier (reads from DB) -│ ├── signal.py # Adapted — signals in DB instead of JSONL files -│ ├── approve.py # Adapted — approvals in DB -│ ├── ledger.py # Adapted — ledger in DB -│ ├── drift.py # Adapted — drift detection, writes signals to DB -│ ├── relations.py # Reused — ontology graph walk (reads from DB) -│ └── report.py # Adapted — generates reports from DB -├── hermes/ # Hermes agent config + skills (mounted into container) -│ ├── config.yaml # Hermes config (providers, models, gateway) -│ ├── SOUL.md # Agent persona (homelab-specific) -│ └── skills/ # Custom homelab skills +/ # repo root +├── docker-compose.yml # the OS stack definition +├── Makefile # build, test, deploy targets +├── go.mod # Go module definition +├── go.sum +├── cmd/ # binary entrypoints (one per service) +│ ├── api/ # Oikos API server +│ │ └── main.go +│ ├── scheduler/ # Observe + Act loop +│ │ └── main.go +│ └── notifier/ # Notification service +│ └── main.go +├── internal/ # private packages (not importable) +│ ├── db/ # database layer +│ │ ├── queries/ # sqlc SQL queries +│ │ ├── models.go # generated Go types +│ │ └── db.go # connection pool, migrations +│ ├── ontology/ # ontology types + meta-schema +│ │ ├── types.go # EntityType, RelationshipType, LifecycleDef +│ │ ├── graph.go # graph traversal (blast radius, dependencies) +│ │ └── ingest.go # YAML seed → DB ingest +│ ├── api/ # HTTP + MCP server +│ │ ├── server.go # Gin app setup +│ │ ├── routes/ # REST handlers +│ │ │ ├── hosts.go +│ │ │ ├── services.go +│ │ │ ├── signals.go +│ │ │ ├── approvals.go +│ │ │ ├── exec.go +│ │ │ └── knowledge.go +│ │ └── mcp.go # MCP protocol adapter (JSON-RPC over SSE) +│ ├── policy/ # risk classification + approval +│ │ ├── classify.go # risk × blast × confidence +│ │ ├── approve.go # approval request + grant lifecycle +│ │ └── autonomy.go # kill-switch, never-auto-act list +│ ├── scheduler/ # Observe stage +│ │ ├── probe.go # HTTP health, disk, drift +│ │ └── signal.go # raise/resolve signals in DB +│ ├── actuator/ # Act stage +│ │ ├── act.go # read signals, classify, execute or escalate +│ │ ├── execute.go # SSH execution + verification +│ │ └── guard.go # loop-guard, retry caps +│ ├── learning/ # feedback loop (the learning model) +│ │ ├── feedback.go # record outcome + lesson from execution +│ │ ├── pattern.go # extract/validate patterns from feedback +│ │ └── skill.go # create/refine skills from patterns +│ ├── notifier/ # notification abstraction +│ │ ├── notifier.go # interface +│ │ └── matrix.go # Matrix implementation +│ └── config/ # config loading (env, files) +│ └── config.go +├── migrations/ # SQL migrations (golang-migrate format) +│ ├── 001_ontology.up.sql # meta-schema (entity_types, relationship_types, ...) +│ ├── 001_ontology.down.sql +│ ├── 002_instances.up.sql # entities, relationships +│ ├── 003_operations.up.sql # signals, approvals, executions, feedback +│ ├── 004_learning.up.sql # patterns, skills +│ └── 005_policy.up.sql # policies, risk_classes, autonomy +├── seeds/ # YAML seed manifests (bootstrap + DR) +│ ├── ontology.yaml # entity types, relationship types, lifecycles +│ ├── inventory.yaml # entity instances (hosts, services, etc.) +│ └── policy.yaml # risk classes, approval rules, autonomy +├── docs/ # narrative docs (ingested into knowledge graph) +│ ├── containers/ +│ ├── hosts/ +│ ├── infrastructure/ +│ └── investigations/ +├── hermes/ # Hermes agent config + skills +│ ├── config.yaml +│ ├── SOUL.md +│ └── skills/ │ └── homelab-ops/ │ └── SKILL.md -├── inventory.yaml # Declarative config — the "kernel data structure" (stays YAML) -├── ontology.yaml # Systems model (stays YAML) -├── policy.yaml # Risk & approval policy (stays YAML) -├── docs/ # Narrative docs — ingested into knowledge graph on deploy -│ ├── containers/ # LXC/VM documentation -│ ├── hosts/ # Host documentation -│ ├── infrastructure/ # Infrastructure docs -│ └── investigations/ # Incident records -├── secrets/ # SOPS-encrypted secrets (migrated to Infisical, then removed) -└── .sops.yaml # SOPS config (deprecated after Infisical migration) +├── compose/ # Docker build contexts +│ ├── api/Dockerfile +│ ├── scheduler/Dockerfile +│ ├── hermes/Dockerfile +│ └── postgres/init.sql +└── scripts/ # utility scripts + ├── migrate-sops.sh # one-time SOPS → Infisical migration + └── import-legacy.sh # import existing signals/ledger JSONL ``` -**What stays YAML (declarative, human-edited, git-tracked):** -- `inventory.yaml` — topology, the kernel data structure -- `ontology.yaml` — entity types, relationships, lifecycle -- `policy.yaml` — risk classes, approval rules, autonomy settings -- `hermes/config.yaml` — agent configuration -- `hermes/SOUL.md` — agent persona +## Database schema -**What moves to PostgreSQL (mutable, runtime state):** -- `signals/` → `signals` table -- `ledger/` → `ledger_entries` table -- `oikos/state.json` → `state_snapshots` table -- `approvals/` → `approvals` table -- `knowledge/wiki/` narrative docs → `knowledge_entities` + `knowledge_relations` tables +The schema is the ontology made concrete. Five migration groups, each adding a layer. -**What gets removed (no longer needed):** -- `mcp/server.py` — merged into `oikos/api/server.py` -- `bin/homelab` — replaced by the API; thin CLI client can be generated later -- `secrets-issuance/` — replaced by Infisical -- `oikos/systemd/` — replaced by Docker containers -- `oikos/console/` — deferred (built later as a specific UI on top of the API) -- `mcp/deploy/`, `oikos/console/deploy/` — replaced by Docker build -- `hosts/*.yaml` — generated from inventory, now served by the API directly - -## Docker Compose stack - -```yaml -# docker-compose.yml (simplified — full version in the repo) -services: - postgres: - image: postgres:16-alpine - volumes: [pg-data:/var/lib/postgresql/data, ./compose/postgres/init.sql:/docker-entrypoint-initdb.d/init.sql] - environment: - POSTGRES_DB: oikos - POSTGRES_PASSWORD_FILE: /run/secrets/pg_password - secrets: [pg_password] - healthcheck: ... - - infisical: - image: infisical/infisical:latest - volumes: [infisical-data:/var/lib/infisical] - environment: ... - # Secrets manager — replaces SOPS+age for runtime secret access - - api: - build: ./compose/api - depends_on: [postgres, infisical] - environment: - DATABASE_URL: postgresql://oikos@postgres/oikos - HOMELAB_CONTEXT_DIR: /opt/homelab-context - volumes: - - ./inventory.yaml:/app/inventory.yaml:ro - - ./ontology.yaml:/app/ontology.yaml:ro - - ./policy.yaml:/app/policy.yaml:ro - - ./docs:/app/docs:ro - ports: ["8090:8090"] # REST + MCP - # On startup: ingest inventory.yaml + docs into DB if changed - - scheduler: - build: ./compose/scheduler - depends_on: [api, postgres] - environment: - DATABASE_URL: postgresql://oikos@postgres/oikos - SSH_KEY: /run/secrets/ssh_key - volumes: - - ssh-keys:/run/secrets:ro - # Runs observe + act loop on a cron-like schedule (10-min interval) - - hermes: - build: ./compose/hermes - depends_on: [api, infisical] - environment: - OIKOS_API_URL: http://api:8090 - INFISICAL_TOKEN: ... - volumes: - - ./hermes:/root/.hermes:ro - - ssh-keys:/root/.ssh:ro - - hermes-data:/root/.hermes/data - ports: ["8092:8092"] # Hermes gateway port - # Agent runtime — connects to API via MCP, has SSH for direct host access - - notifier: - build: ./compose/scheduler # shares image with scheduler - depends_on: [api, postgres] - # Matrix bot — listens for approval requests, sends alerts - # Abstracted via notifier interface — Matrix impl now, pluggable later - -volumes: - pg-data: - infisical-data: - ssh-keys: # populated from host ~/.ssh at deploy time - hermes-data: -``` - -## Workstreams - -### 1. Database schema + models (`oikos/db/`, new) - -The foundational layer — everything else reads/writes through this. - -**Tables:** +### Migration 1: Ontology meta-schema ```sql --- Inventory entities (synced from inventory.yaml on deploy) -CREATE TABLE entities ( - id TEXT PRIMARY KEY, -- "host:hubris", "service:caddy" - type TEXT NOT NULL, -- host, service, lxc, vm, etc. - name TEXT NOT NULL, - domain TEXT, -- physical, compute, network, etc. - state TEXT DEFAULT 'active', -- provisioning, active, deprecated, destroyed - data JSONB NOT NULL, -- full entity record from inventory.yaml - updated_at TIMESTAMPTZ DEFAULT now() +-- The meta-graph: defines what entity types and relationship types can exist. +-- This IS the ontology, stored in the DB, editable via API. + +CREATE TABLE entity_types ( + name TEXT PRIMARY KEY, -- 'host', 'service', 'signal', 'pattern' + domain TEXT NOT NULL, -- 'physical', 'compute', 'network', ... + layer TEXT NOT NULL, -- 'infrastructure', 'governance', 'cognition' + description TEXT, + lifecycle_id TEXT, -- FK to lifecycle_defs (nullable = no lifecycle) + created_at TIMESTAMPTZ DEFAULT now(), + updated_at TIMESTAMPTZ DEFAULT now() ); --- Entity relationships (the ontology graph) -CREATE TABLE relations ( - source_id TEXT REFERENCES entities(id), - target_id TEXT REFERENCES entities(id), - type TEXT NOT NULL, -- hosts, provides, mounts, depends-on, etc. - data JSONB, +CREATE TABLE relationship_types ( + name TEXT PRIMARY KEY, -- 'hosts', 'provides', 'depends-on' + inverse TEXT, -- 'runs-on', 'provided-by' + source_type TEXT REFERENCES entity_types(name), + target_type TEXT REFERENCES entity_types(name), + cardinality TEXT NOT NULL, -- 'one-to-one', 'one-to-many', 'many-to-many' + description TEXT, + created_at TIMESTAMPTZ DEFAULT now() +); + +CREATE TABLE lifecycle_defs ( + id TEXT PRIMARY KEY, -- 'infrastructure', 'signal', 'execution', ... + states TEXT[] NOT NULL, -- ordered states + default_state TEXT NOT NULL, + transitions JSONB NOT NULL, -- {"from": {"to": {"requires": [...]}}} + created_at TIMESTAMPTZ DEFAULT now() +); +``` + +### Migration 2: Entity instances (the inventory graph) + +```sql +-- Entity instances — the actual hosts, services, signals, patterns, etc. +-- This replaces inventory.yaml as the runtime source of truth. + +CREATE TABLE entities ( + id TEXT PRIMARY KEY, -- 'host:hubris', 'service:caddy', 'sig:2026-07-06-0001' + type TEXT NOT NULL REFERENCES entity_types(name), + name TEXT NOT NULL, -- 'hubris', 'caddy', 'disk-threshold' + state TEXT, -- lifecycle state (e.g. 'active', 'raised') + attributes JSONB NOT NULL DEFAULT '{}', -- type-specific data (IP, mesh addr, port, ...) + parent_id TEXT REFERENCES entities(id), -- for hierarchical entities (LXC on host) + created_at TIMESTAMPTZ DEFAULT now(), + updated_at TIMESTAMPTZ DEFAULT now() +); + +CREATE INDEX idx_entities_type ON entities(type); +CREATE INDEX idx_entities_state ON entities(state); +CREATE INDEX idx_entities_attributes ON entities USING GIN(attributes); + +-- Relationship instances — the typed edges of the graph +CREATE TABLE relationships ( + source_id TEXT NOT NULL REFERENCES entities(id), + target_id TEXT NOT NULL REFERENCES entities(id), + type TEXT NOT NULL REFERENCES relationship_types(name), + attributes JSONB, + created_at TIMESTAMPTZ DEFAULT now(), PRIMARY KEY (source_id, target_id, type) ); --- Signals (replaces signals/*.jsonl) +CREATE INDEX idx_rel_source ON relationships(source_id); +CREATE INDEX idx_rel_target ON relationships(target_id); +CREATE INDEX idx_rel_type ON relationships(type); + +-- Recursive graph traversal function (blast radius, dependency chains) +CREATE OR REPLACE FUNCTION blast_radius(start_id TEXT, max_depth INT DEFAULT 3) +RETURNS TABLE(entity_id TEXT, depth INT) AS $$ + WITH RECURSIVE walk AS ( + SELECT start_id::TEXT AS entity_id, 0::INT AS depth + UNION + SELECT r.target_id::TEXT, w.depth + 1 + FROM relationships r + JOIN walk w ON r.source_id = w.entity_id + WHERE w.depth < max_depth + ) + SELECT DISTINCT entity_id, MIN(depth) FROM walk GROUP BY entity_id; +$$ LANGUAGE sql STABLE; +``` + +### Migration 3: Operations (signals, approvals, ledger, state) + +```sql +-- Signals — now entities in the graph, with a dedicated table for indexed querying +-- (the entity row is the canonical record; this table is a fast lookup) CREATE TABLE signals ( - id TEXT PRIMARY KEY, -- "sig-2026-07-06-0001" - ts TIMESTAMPTZ NOT NULL DEFAULT now(), - kind TEXT NOT NULL, -- service-down, disk-threshold, drift, etc. - severity TEXT NOT NULL, -- info, warning, critical - entity_id TEXT REFERENCES entities(id), - evidence TEXT, - likely_cause TEXT, + entity_id TEXT PRIMARY KEY REFERENCES entities(id), + kind TEXT NOT NULL, + severity TEXT NOT NULL, -- info, warning, critical + target_entity_id TEXT REFERENCES entities(id), -- the infrastructure entity this is about + evidence TEXT, + likely_cause TEXT, recommended_action JSONB, - verification TEXT, - state TEXT NOT NULL DEFAULT 'raised', -- raised, acknowledged, acting, resolved, muted - mute_until TIMESTAMPTZ, - note TEXT, - updated_at TIMESTAMPTZ DEFAULT now() + verification TEXT, + state TEXT NOT NULL DEFAULT 'raised', + mute_until TIMESTAMPTZ, + created_at TIMESTAMPTZ DEFAULT now(), + updated_at TIMESTAMPTZ DEFAULT now() ); --- Change ledger (replaces ledger/*.jsonl) -CREATE TABLE ledger_entries ( - id SERIAL PRIMARY KEY, - ts TIMESTAMPTZ NOT NULL DEFAULT now(), - entity_id TEXT REFERENCES entities(id), - action TEXT NOT NULL, - risk_class TEXT NOT NULL, - result TEXT, -- ok, failed, escalated - approval_id TEXT, -- references approvals if gated - agent TEXT, -- agent identity (age pubkey or hostname) - notes TEXT -); +CREATE INDEX idx_signals_state ON signals(state); +CREATE INDEX idx_signals_target ON signals(target_entity_id); +CREATE INDEX idx_signals_severity ON signals(severity); --- Approvals (replaces approvals/*.jsonl) +-- Approvals CREATE TABLE approvals ( - id TEXT PRIMARY KEY, - ts TIMESTAMPTZ NOT NULL DEFAULT now(), - entity_id TEXT REFERENCES entities(id), - action TEXT NOT NULL, - risk_class TEXT NOT NULL, - status TEXT NOT NULL DEFAULT 'pending', -- pending, approved, denied, expired - ttl INTERVAL NOT NULL DEFAULT '1 hour', - decided_at TIMESTAMPTZ, - decided_by TEXT, + id TEXT PRIMARY KEY, + ts TIMESTAMPTZ DEFAULT now(), + entity_id TEXT REFERENCES entities(id), -- entity to act on + action TEXT NOT NULL, + risk_class TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', -- pending, approved, denied, expired + ttl INTERVAL NOT NULL DEFAULT '1 hour', + decided_at TIMESTAMPTZ, + decided_by TEXT REFERENCES entities(id), -- person entity confirmation_phrase TEXT ); +-- Change ledger (high-level record, links to execution for detail) +CREATE TABLE ledger_entries ( + id SERIAL PRIMARY KEY, + ts TIMESTAMPTZ DEFAULT now(), + entity_id TEXT REFERENCES entities(id), + action TEXT NOT NULL, + risk_class TEXT NOT NULL, + result TEXT, -- ok, failed, escalated + approval_id TEXT REFERENCES approvals(id), + execution_id INTEGER, -- FK to executions (migration 4) + agent_id TEXT REFERENCES entities(id), + notes TEXT +); + -- State snapshots (replaces oikos/state.json) CREATE TABLE state_snapshots ( - id SERIAL PRIMARY KEY, - ts TIMESTAMPTZ NOT NULL DEFAULT now(), - entity_id TEXT REFERENCES entities(id), - health TEXT, -- healthy, degraded, down, unknown - data JSONB -- full probe response -); - --- Knowledge graph (replaces narrative wiki docs) -CREATE TABLE knowledge_entities ( - id TEXT PRIMARY KEY, -- "doc:105-apps", "investigation:2026-07-05-disk" - type TEXT NOT NULL, -- container-doc, host-doc, investigation, plan, runbook - entity_id TEXT REFERENCES entities(id), -- link to inventory entity if applicable - title TEXT NOT NULL, - content TEXT NOT NULL, -- markdown body - tags TEXT[], - source_path TEXT, -- original file path for traceability - updated_at TIMESTAMPTZ DEFAULT now() -); - --- Knowledge relationships (cross-references between docs + entities) -CREATE TABLE knowledge_relations ( - source_id TEXT REFERENCES knowledge_entities(id), - target_id TEXT, -- can be knowledge_entity or inventory entity - type TEXT NOT NULL, -- describes, relates-to, runbook-for, etc. - PRIMARY KEY (source_id, target_id, type) + id SERIAL PRIMARY KEY, + ts TIMESTAMPTZ DEFAULT now(), + entity_id TEXT REFERENCES entities(id), + health TEXT, -- healthy, degraded, down, unknown + data JSONB ); ``` -**Ingest pipeline** (`oikos/db/ingest.py`): -- On startup / deploy: parse `inventory.yaml` → upsert into `entities` + `relations` -- Walk `docs/` directory → parse markdown, extract frontmatter → upsert into `knowledge_entities` -- Link knowledge entities to inventory entities by path convention (`docs/containers/105-apps.md` → `entity:lxc:apps`) -- Idempotent — safe to re-run on every deploy +### Migration 4: Learning model (executions, feedback, patterns, skills) -**Migration from existing data:** -- Write a one-time script to import existing `signals/*.jsonl`, `ledger/*.jsonl`, `approvals/*.jsonl` into the DB -- `oikos/state.json` → `state_snapshots` table +```sql +-- Executions — detailed record of each action the OS performs +CREATE TABLE executions ( + id SERIAL PRIMARY KEY, + ts TIMESTAMPTZ DEFAULT now(), + signal_entity_id TEXT REFERENCES entities(id), -- signal that triggered this + target_entity_id TEXT REFERENCES entities(id), -- entity acted upon + action TEXT NOT NULL, + risk_class TEXT NOT NULL, + approval_id TEXT REFERENCES approvals(id), + agent_id TEXT REFERENCES entities(id), -- who/what executed + skill_id TEXT REFERENCES entities(id), -- skill used (if any) + status TEXT NOT NULL DEFAULT 'queued', -- queued, running, completed, failed, timed-out + result JSONB, -- detailed result data + duration_ms INTEGER, + verified BOOLEAN DEFAULT false, + started_at TIMESTAMPTZ, + completed_at TIMESTAMPTZ +); -### 2. Unified API server (`oikos/api/`, new — merges MCP + homelab CLI) +CREATE INDEX idx_exec_target ON executions(target_entity_id); +CREATE INDEX idx_exec_status ON executions(status); +CREATE INDEX idx_exec_action ON executions(action); -One FastAPI service exposing two interfaces: +-- Feedback — what was learned from an execution +CREATE TABLE feedback ( + id SERIAL PRIMARY KEY, + execution_id INTEGER REFERENCES executions(id), + ts TIMESTAMPTZ DEFAULT now(), + outcome TEXT NOT NULL, -- success, failure, partial, unexpected + observation TEXT, -- what happened vs what was expected + lesson TEXT, -- extractable lesson + unexpected_side_effects TEXT[], + tags TEXT[] +); -**MCP interface** (`oikos/api/mcp.py`): -- Same tool surface as the current `mcp/server.py` (get_host, list_services, search_knowledge, etc.) -- Reads from PostgreSQL instead of filesystem -- `search_docs` → `search_knowledge` (queries the knowledge graph) -- New tools: `get_entity_relations`, `get_signal_history`, `get_ledger` -- Protocol: FastMCP over streamable HTTP (same as current) +CREATE INDEX idx_feedback_execution ON feedback(execution_id); +CREATE INDEX idx_feedback_outcome ON feedback(outcome); -**REST interface** (`oikos/api/routes/`): -- `GET /api/v1/hosts` — list hosts (replaces `homelab list`) -- `GET /api/v1/services` — list services -- `GET /api/v1/hosts/{name}` — host detail (replaces `homelab status`) -- `GET /api/v1/services/{name}/health` — service health (cache-first, `?live=true` for probe) -- `GET /api/v1/signals` — list signals (`?state=raised&severity=warning`) +-- Patterns — generalized rules extracted from accumulated feedback +CREATE TABLE patterns ( + id TEXT PRIMARY KEY, -- 'pat-2026-07-06-001' + ts TIMESTAMPTZ DEFAULT now(), + entity_type TEXT REFERENCES entity_types(name), -- applies to this type + action TEXT NOT NULL, -- 'restart', 'deploy', etc. + pattern TEXT NOT NULL, -- 'service X recovers within 30s after restart' + confidence REAL DEFAULT 0.5, -- 0.0 to 1.0 + evidence_count INTEGER DEFAULT 1, -- how many executions support this + success_count INTEGER DEFAULT 0, + failure_count INTEGER DEFAULT 0, + status TEXT DEFAULT 'hypothesized', -- hypothesized, validated, active, deprecated + last_validated_at TIMESTAMPTZ +); + +CREATE INDEX idx_patterns_type_action ON patterns(entity_type, action); +CREATE INDEX idx_patterns_status ON patterns(status); + +-- Skills — codified procedures refined through feedback +CREATE TABLE skills ( + id TEXT PRIMARY KEY, -- 'skill-restart-service', 'skill-deploy-lxc' + name TEXT NOT NULL, + ts TIMESTAMPTZ DEFAULT now(), + procedure TEXT NOT NULL, -- the codified steps (markdown or structured) + applies_to TEXT REFERENCES entity_types(name), + pattern_ids TEXT[], -- patterns that inform this skill + status TEXT DEFAULT 'drafted', -- drafted, tested, active, refined, deprecated + version INTEGER DEFAULT 1, + success_rate REAL, -- rolling success rate + last_used_at TIMESTAMPTZ +); + +CREATE INDEX idx_skills_type ON skills(applies_to); +CREATE INDEX idx_skills_status ON skills(status); +``` + +### Migration 5: Policy (DB-native risk + approval rules) + +```sql +-- Risk classes — the four-level safety model +CREATE TABLE risk_classes ( + name TEXT PRIMARY KEY, -- 'read_only', 'reversible_low', etc. + description TEXT, + approval_required TEXT NOT NULL DEFAULT 'none', -- none, operator, operator_confirmed + ledger BOOLEAN DEFAULT false, + autonomy_allowed BOOLEAN DEFAULT false -- can agent auto-act at this risk level? +); + +-- Approval rules — entity_type + action → risk_class + requirements +CREATE TABLE approval_rules ( + id SERIAL PRIMARY KEY, + entity_type TEXT REFERENCES entity_types(name), -- applies to this entity type + action TEXT NOT NULL, -- 'restart', 'deploy', 'destroy' + risk_class TEXT NOT NULL REFERENCES risk_classes(name), + autonomy_level TEXT NOT NULL DEFAULT 'auto', -- 'auto', 'escalate', 'never' + scope_entity TEXT REFERENCES entities(id), -- optional: specific entity only + created_at TIMESTAMPTZ DEFAULT now(), + updated_at TIMESTAMPTZ DEFAULT now(), + UNIQUE(entity_type, action) +); + +-- Autonomy settings — global kill-switch + per-entity overrides +CREATE TABLE autonomy_settings ( + key TEXT PRIMARY KEY, -- 'global.auto_act', 'never_auto_act.caddy' + value TEXT NOT NULL, -- 'off', 'reversible_low', 'true', 'false' + updated_at TIMESTAMPTZ DEFAULT now() +); +``` + +## Workstreams + +### 1. Ontology definition + seed manifests (`seeds/`, `internal/ontology/`) + +**Before any code**, finalize the ontology. The current `ontology.yaml` has 8 domains +and 14 relationship types. The new ontology adds: + +- **Layer 3 entities** (cognition): signal, execution, feedback, pattern, skill, + approval, classification, document, runbook +- **New relationships**: `triggers`, `produces`, `contributes-to`, `informs`, `guides`, + `precedes`, `performs`, `procedure-for`, `learned-from` +- **Lifecycle definitions** for operational entities (signals, executions, patterns, + skills) — not just infrastructure + +**Deliverables:** +- `seeds/ontology.yaml` — entity types, relationship types, lifecycle definitions + (seeded into `entity_types`, `relationship_types`, `lifecycle_defs` on deploy) +- `seeds/inventory.yaml` — adapted from current `inventory.yaml` (seeded into + `entities` + `relationships`) +- `seeds/policy.yaml` — adapted from current `policy.yaml` (seeded into `risk_classes`, + `approval_rules`, `autonomy_settings`) +- `internal/ontology/ingest.go` — idempotent seed → DB ingest + +### 2. Database layer + migrations (`migrations/`, `internal/db/`) + +- Write the 5 migrations above +- Set up sqlc for type-safe Go database access +- Connection pool, migration runner +- Graph traversal queries (blast radius, dependency chains, knowledge lookup) +- Import script for existing `signals/*.jsonl`, `ledger/*.jsonl` → DB + +### 3. Unified API server — Go (`cmd/api/`, `internal/api/`) + +One Go binary (Gin web framework) exposing REST + MCP from the same codebase. + +**MCP interface** (`internal/api/mcp.go`): +- JSON-RPC over SSE, compatible with Hermes MCP client +- Tools: `get_host`, `list_services`, `search_knowledge`, `get_entity`, + `get_relations`, `get_blast_radius`, `get_signal_history`, `get_ledger`, + `get_patterns`, `get_skills`, `get_state_snapshot` +- All read from PostgreSQL + +**REST interface** (`internal/api/routes/`): +- `GET /api/v1/entities` — list entities (filter by type, state, domain) +- `GET /api/v1/entities/{id}` — entity detail + relationships +- `POST /api/v1/entities` — create entity (creates inventory entry) +- `PATCH /api/v1/entities/{id}` — update entity (state transition, attributes) +- `GET /api/v1/signals` — list signals (filter by state, severity, entity) - `POST /api/v1/signals/{id}/ack` — acknowledge - `POST /api/v1/signals/{id}/resolve` — resolve - `GET /api/v1/approvals` — pending approvals -- `POST /api/v1/approvals/{id}/decide` — approve/deny (operator only, Authentik-gated) -- `POST /api/v1/exec` — gated execution (the actuator entrypoint) - - Body: `{entity, action, approval_id?}` - - Classifies via `decide.py`, checks approval if needed, executes, records ledger -- `POST /api/v1/deploy/{service}` — trigger a service deploy -- `WS /api/v1/events` — real-time stream (signals, approvals, ledger entries) -- `GET /api/v1/knowledge/{entity_id}` — knowledge graph query for an entity +- `POST /api/v1/approvals/{id}/decide` — approve/deny (Authentik-gated) +- `POST /api/v1/exec` — gated execution (classify → check approval → execute → feedback) +- `GET /api/v1/patterns` — list patterns (filter by entity_type, action, status) +- `GET /api/v1/skills` — list skills (filter by applies_to, status) +- `GET /api/v1/knowledge/{entity_id}` — knowledge graph query - `GET /api/v1/knowledge/search?q=...` — search knowledge graph +- `GET /api/v1/ontology` — list entity types, relationship types, lifecycles +- `POST /api/v1/ontology/entity_types` — create entity type (extend the schema) +- `WS /api/v1/events` — real-time stream (signals, approvals, executions, feedback) -**Policy enforcement** is built into the API: -- Every mutating endpoint classifies the action via `oikos/decide.py` -- `config_mutation`/`destructive` actions require a valid approval ID +**Policy enforcement** (`internal/policy/classify.go`): +- Every mutating endpoint classifies the action via the policy DB +- Risk class → approval check → autonomy check - All mutations write to the ledger automatically **Auth:** - MCP interface: no auth (internal, container-to-container) - REST interface: Authentik OIDC forward-auth (via Caddy) for operator endpoints -- Internal service-to-service: shared secret / mTLS (Docker network) +- Internal: shared secret (Docker network) -### 3. Scheduler + Actuator (`oikos/scheduler/`, `oikos/actuator/`) +### 4. Scheduler + Actuator — Go (`cmd/scheduler/`, `internal/scheduler/`, `internal/actuator/`) -**Scheduler (Observe)** — adapted from `oikos/scheduler.py`: -- Runs as a Docker service on a timer (10-min interval via a loop or cron sidecar) -- Probes service health (HTTP), disk usage (SSH to hubris/strong), drift detection -- Writes signals to PostgreSQL (not JSONL files) -- Writes state snapshots to DB -- No more git commit/push of signals — they're in the DB +**Scheduler (Observe)** — Go service with goroutines for concurrent probes: +- HTTP health probes (concurrent, with timeouts) +- Disk usage probes (SSH to hubris/strong) +- Drift detection (inventory vs live state) +- Writes signals + state snapshots to DB +- Runs on a 10-min ticker -**Actuator (Act)** — new, `oikos/actuator/act.py`: -- Reads open signals from DB where `recommended_action` is set -- For each: classify via `decide.py` - - `auto-act` (reversible_low, contained, confident) → execute via SSH/API, verify, resolve signal, write ledger - - `escalate` → create approval request, notify via Matrix, acknowledge signal -- Loop-guard: check ledger history per (entity, action) to cap auto-retries -- Autonomy kill-switch: `policy.yaml` → `autonomy.auto_act: off|reversible_low`, `never_auto_act: [entity list]` -- **Phase 1**: direct SSH execution (keys mounted) -- **Phase 2**: execution through the API's `/exec` endpoint (gateway pattern) +**Actuator (Act)** — Go service, the control loop: +- Reads open signals with `recommended_action` +- For each: classify via `internal/policy/classify.go` + - **auto-act**: look up skill for (entity_type, action) → follow procedure → execute + via SSH → verify → record execution → generate feedback → update patterns + - **escalate**: create approval request → notify via Matrix → acknowledge signal +- Loop-guard: check execution history per (entity, action) to cap auto-retries + (`SELECT ... FOR UPDATE SKIP LOCKED` for concurrency safety) +- Autonomy kill-switch: check `autonomy_settings` table -### 4. Hermes agent container (`compose/hermes/`) +### 5. Learning engine — Go (`internal/learning/`) -**What it is:** -- Hermes Agent runtime in a Docker container -- Runs in gateway mode — workstations connect to it remotely -- Config: `hermes/config.yaml` (providers, models, gateway port) -- Persona: `hermes/SOUL.md` (homelab-specific) -- Skills: `hermes/skills/homelab-ops/SKILL.md` (how to interact with the homelab) +The feedback loop that makes the agent improve over time. -**What it has access to:** -- **Oikos API** via MCP (container network, no auth needed) -- **Infisical** for secret retrieval (API token) -- **SSH keys** mounted from host (`~/.ssh` → `/root/.ssh:ro`) — for direct host access (hybrid approach) -- **Hermes data volume** — persistent state, session history +**Feedback recording** (`internal/learning/feedback.go`): +- After every execution, evaluate the outcome: + - Did the verification command pass? → success + - Did it fail? → failure + - Did it partially work? → partial + - Did something unexpected happen? → unexpected +- Record a feedback entry with: outcome, observation (what happened vs expected), + lesson (extractable insight), unexpected_side_effects -**How workstations connect:** -- Hermes gateway listens on port 8092 -- Any workstation with Hermes CLI can connect: `hermes gateway connect mac-mini:8092` -- Or via Caddy ingress: `hermes.hubris.network` → mac-mini:8092 (mesh) +**Pattern extraction** (`internal/learning/pattern.go`): +- Periodically scan accumulated feedback for (entity_type, action) pairs +- When N+ executions share a similar outcome, extract a pattern: + - "Restarting service:X typically takes 15s and succeeds" + - "Deploying to LXC:Y via webhook has 30% failure rate, retry helps" +- Patterns start as `hypothesized`, move to `validated` after enough evidence, + then `active` (used by the decision classifier) +- Confidence score = success_count / evidence_count, adjusted by recency -**Homelab skills** (loaded into Hermes): -- `homelab-ops` — how to use the Oikos API, classify actions, request approvals -- `homelab-deploy` — how to deploy/manage services -- `homelab-troubleshoot` — diagnostic procedures -- (Future skills added as needed) +**Skill management** (`internal/learning/skill.go`): +- When a pattern reaches `active` status with confidence > 0.7, create or refine + a skill for that (entity_type, action) pair +- Skills codify the best-known procedure (what steps to take, what to verify, + expected duration, known failure modes) +- Skills are versioned — each refinement increments the version +- The actuator looks up skills before executing: if a skill exists, follow it; + if not, use the default procedure and generate feedback for future pattern extraction -### 5. Infisical secrets migration +**How the classifier uses learning** (`internal/policy/classify.go`): +- Confidence scoring now checks patterns + skills, not just raw ledger history: + - If a pattern exists for (entity_type, action) with high confidence → boost + auto-act confidence + - If patterns show frequent failures → lower confidence, escalate + - If a skill exists → higher confidence (proven procedure available) +- This is the closed loop: **execution → feedback → pattern → skill → classification + → execution** (better informed each time) -**Phase 1 — Stand up Infisical:** -- Add Infisical to the Docker stack -- Configure with PostgreSQL backend (shared with Oikos DB or separate DB) -- Bootstrap admin credentials, create `hubris` project + environments (prod, staging) +### 6. Knowledge graph ingestion (`internal/ontology/ingest.go`) -**Phase 2 — Migrate secrets:** -- Write a migration script: read all `secrets/*.yaml` (SOPS-encrypted), decrypt with age key, import into Infisical -- Map each secret to Infisical's key-value structure -- Verify all secrets are imported +After the ontology is defined and the DB schema is in place: -**Phase 3 — Wire services:** -- All OS services authenticate to Infisical via machine identity (service token) -- Services fetch secrets at startup or on-demand -- Remove `secrets/` directory and `.sops.yaml` once verified -- The age key on mac-mini stays for one-time decryption during migration, then is retired - -**Phase 4 — Retire SOPS:** -- Remove SOPS dependencies from the repo -- Update inventory to reflect Infisical as the secret store -- Document the new secret access pattern - -### 6. Knowledge graph ingestion (`oikos/knowledge/`) - -**What it does:** -- On deploy, walks `docs/` directory -- Parses each markdown file: - - Extracts frontmatter (if any) for metadata - - Infers entity relationships from path conventions: - - `docs/containers/105-apps.md` → links to `entity:lxc:apps` - - `docs/hosts/hubris.md` → links to `entity:host:hubris` - - `docs/infrastructure/dns.md` → links to `entity:service:dns` - - Extracts cross-references (links to other docs) → `knowledge_relations` - - Tags by directory: container-doc, host-doc, infrastructure, investigation, plan -- Upserts into `knowledge_entities` + `knowledge_relations` tables +- On deploy, walk `docs/` directory +- Parse each markdown file: + - Extract frontmatter for metadata (entity type, tags, relations) + - Infer entity relationships from path conventions: + `docs/containers/105-apps.md` → relationship to `entity:lxc:apps` + - Extract cross-references (markdown links) → relationships +- Create knowledge entities in the `entities` table (type = `document`, `runbook`, + `investigation`, etc.) with `documents` / `procedure-for` edges to infrastructure + entities +- Idempotent — safe to re-run on every deploy **Agent access:** -- MCP tool `search_knowledge(query)` — full-text search on knowledge graph +- MCP tool `search_knowledge(query)` — full-text search on knowledge entities - MCP tool `get_entity_knowledge(entity_id)` — all docs related to an entity -- MCP tool `get_relations(entity_id)` — ontology blast-radius query (existing, adapted to DB) +- MCP tool `get_relations(entity_id)` — graph traversal (blast radius, dependencies) -**Docs as source, DB as query layer:** -- Humans still write markdown docs in `docs/` -- The DB is the query/index layer — agents never grep files -- On deploy, docs are re-ingested (idempotent) -- Future: wiki-style editing UI that writes back to `docs/` via git +### 7. Hermes agent container (`compose/hermes/`) -### 7. Notifier abstraction (`oikos/notifier/`) +- Hermes Agent runtime in a Docker container, gateway mode +- Config: `hermes/config.yaml` (providers, models, gateway port) +- Persona: `hermes/SOUL.md` (homelab-specific) +- Skills: `hermes/skills/homelab-ops/SKILL.md` — how to use the Oikos API, classify + actions, request approvals, query the knowledge graph +- Access: Oikos API via MCP (container network) + SSH keys mounted (hybrid) +- Gateway port 8092 — workstations connect remotely +- Hermes data volume for persistent state -**Interface:** -```python -class Notifier(Protocol): - def send_alert(self, signal: Signal) -> None: ... - def send_approval_request(self, approval: Approval) -> None: ... - def listen_for_decisions(self) -> Iterator[ApprovalDecision]: ... +### 8. Infisical secrets migration + +Same as rev 1: +- Stand up Infisical in the Docker stack +- Migrate SOPS secrets (one-time decrypt + import) +- Wire all Go services to Infisical via machine identity +- Retire SOPS + age keys + +### 9. Notifier — Go (`cmd/notifier/`, `internal/notifier/`) + +**Interface (Go):** +```go +type Notifier interface { + SendAlert(ctx context.Context, signal Signal) error + SendApprovalRequest(ctx context.Context, approval Approval) error + ListenForDecisions(ctx context.Context) (<-chan ApprovalDecision, error) +} ``` -**Matrix implementation** (current): +**Matrix implementation:** - Sends alerts to `@dtoro:avispero` via Synapse (LXC 118) - Approval requests as messages with ✅/❌ reactions - Listens for reactions to record decisions -- Reuses existing `oikos/approve.py` Matrix delivery logic +- Pluggable — future implementations (webhook, email) register via config -**Future implementations** (pluggable): -- Webhook, email, Slack, etc. -- Registered in config, selected at runtime +### 10. Docker build + deploy pipeline -### 8. Docker build + deploy pipeline +- Multi-stage Dockerfiles: Go build stage → minimal runtime image (alpine or scratch) +- `docker compose build` from repo root +- Gitea webhook on push to `main` → deploy script on mac-mini +- Deploy script: `git pull && go build ./... && docker compose build && docker compose up -d` +- Seed ingest runs as part of the API startup (idempotent) +- Health checks on each service -**Build:** -- `docker compose build` from the repo root -- Each service has a Dockerfile in `compose//` -- Shared base image: Python 3.12-slim + common deps (postgresql libs, etc.) -- Multi-stage builds for smaller images +### 11. Ingress re-point (Caddy) -**Deploy trigger:** -- Gitea webhook on push to `main` → hits a deploy endpoint on mac-mini -- Deploy script: `git pull && docker compose build && docker compose up -d` -- Alternatively: a "deployer" sidecar container watches Gitea and triggers rebuilds -- The existing 5-min launchd git pull can serve as a fallback deploy mechanism initially - -**Health checks:** -- Each service has a Docker healthcheck -- `docker compose ps` shows health status -- API exposes `/health` for external monitoring - -### 9. Ingress re-point (Caddy) - -- In `dtoro/caddy-conf`: point `mcp.hubris.network` and `oikos.hubris.network` to - mac-mini's mesh address at port 8090 (API) -- Future: `hermes.hubris.network` → mac-mini:8092 (Hermes gateway) +- `dtoro/caddy-conf`: point `mcp.hubris.network` + `oikos.hubris.network` → + mac-mini mesh IP :8090 (API) +- Future: `hermes.hubris.network` → mac-mini:8092 - DNS and public URLs unchanged -- Verify Caddy → mac-mini reachability over netbird first (same as original plan) -### 10. mac-mini host setup +### 12. mac-mini host setup -**Directory:** -- New directory: `~/oikos-os/` (or `/opt/oikos-os/`) — the repo clone -- This is where `docker compose` runs from +- New directory: `~/oikos-os/` — repo clone + `docker compose` working dir - Existing `/opt/homelab-context/` stays untouched until cleanup phase +- Prerequisites: Docker (or OrbStack), Go toolchain (for local dev), SSH keys +- Cleanup (deferred): stop launchd git-sync, remove native Hermes, remove old clone -**Prerequisites:** -- Docker Desktop (or OrbStack) — already installed -- SSH keys accessible at `~/.ssh/` (mounted into containers) -- Age key at `/etc/age/key.txt` (for one-time SOPS migration) +### 13. Decommission apps/105 (deferred) -**Cleanup (deferred, after Docker OS is proven):** -- Stop launchd git-sync for `/opt/homelab-context` -- Remove native Hermes install at `~/.hermes/hermes-agent/` -- Remove `/opt/homelab-context` clone -- Remove any launchd plists for homelab services - -### 11. Decommission apps/105 (deferred) - -Keep apps/105 running all Oikos services (homelab-mcp, secrets-issuance, -oikos-console) as a fallback until the Docker OS is proven end-to-end. - -**Cutover checklist (executed when ready):** -1. Verify Docker OS serves all traffic correctly -2. `systemctl disable --now homelab-mcp secrets-issuance oikos-console` on apps/105 -3. Remove `/opt/homelab-mcp`, `/opt/oikos-console` checkouts on apps/105 -4. Update Caddy backends to point to mac-mini exclusively -5. Remove Gitea webhooks 10, 11, 14 (or repoint to Docker deploy trigger) - -## Reuse (do not reimplement) - -| Existing code | What it becomes | -|---|---| -| `oikos/decide.py` | Risk classifier — adapted to read from DB, logic unchanged | -| `oikos/signal.py` | Signal engine — adapted to DB, lifecycle logic unchanged | -| `oikos/approve.py` | Approval engine — adapted to DB, Matrix delivery reused | -| `oikos/ledger.py` | Ledger — adapted to DB | -| `oikos/policy.py` + `policy.yaml` | Policy loader — unchanged, still reads YAML | -| `oikos/drift.py` | Drift detectors — adapted to write signals to DB | -| `oikos/relations.py` | Ontology graph walk — adapted to DB | -| `oikos/report.py` | Report generator — adapted to read from DB | -| `mcp/server.py` | Merged into `oikos/api/` — MCP protocol adapter on top of DB | -| `bin/homelab` logic | Migrated into API routes — same operations, REST interface | -| `oikos/scheduler.py` | Adapted — probes unchanged, output goes to DB | -| `oikos/approve.py` Matrix delivery | Reused as the Matrix notifier implementation | - -## Risks / trade-offs - -- **Hermes in Docker** — the agent's "world" is the container. Direct file access - to the host is limited to mounted volumes. SSH access is the bridge to the homelab. - Trade-off: less host integration, more isolation. Mitigated by the hybrid approach - (SSH now, actuator gateway later). -- **PostgreSQL as single point of failure** — if Postgres goes down, the OS loses - all state. Mitigated by Docker volume persistence + regular `pg_dump` backups - (can be automated by the scheduler itself once running). -- **Infisical bootstrapping** — migrating from SOPS to Infisical has a transition - period where both systems coexist. Keep SOPS as fallback until all services are - verified against Infisical. -- **Knowledge graph ingestion quality** — markdown-to-structured-data conversion - relies on path conventions and frontmatter. Docs without frontmatter may lose - metadata. Start simple, improve extraction over time. -- **Deploy downtime** — `docker compose up -d` may briefly interrupt services during - container replacement. Use `--scale` + health-gated rollout for zero-downtime - when needed (later). -- **Multi-agent concurrency** — PostgreSQL handles concurrent writers correctly, - but the actuator needs a lock mechanism to prevent two passes from acting on - the same signal simultaneously. Use `SELECT ... FOR UPDATE SKIP LOCKED`. - -## Verification (end to end) - -1. **Database:** `docker compose up postgres` — `psql` shows all tables; ingest - script populates entities from `inventory.yaml` and knowledge from `docs/`. -2. **API:** `curl http://localhost:8090/api/v1/hosts` returns the fleet; - MCP `list_services` tool works via the same endpoint. -3. **Scheduler:** trigger a probe pass — signals appear in DB, state snapshot written. -4. **Actuator:** raise a test `service-down` signal → actuator classifies → - auto-acts (restart) or escalates (Matrix approval request) → ledger entry written. -5. **Hermes:** connect from another workstation via `hermes gateway connect` → - agent responds, can query the API via MCP, can SSH to hubris. -6. **Secrets:** Infisical running, services fetch secrets at startup, SOPS files - removed without breakage. -7. **Deploy:** `git push` → Gitea webhook → `docker compose build + up -d` → - changes live within minutes. -8. **Knowledge:** MCP `search_knowledge("caddy")` returns relevant docs linked - to `entity:service:caddy`. -9. **Notifier:** raise a `critical` signal → Matrix message received → - ✅ reaction → approval granted in DB. -10. **Cutover:** after all above passes, stop services on apps/105, verify - production traffic served only by Docker OS on mac-mini. +Keep apps/105 running as fallback. Cutover checklist when ready: +1. Verify Docker OS serves all traffic +2. `systemctl disable --now` Oikos services on apps/105 +3. Remove old checkouts +4. Update Caddy backends exclusively to mac-mini +5. Remove/retarget Gitea webhooks ## Phasing -**Phase 1 — Foundation (DB + API):** -- PostgreSQL schema + models + migrations -- Ingest pipeline (inventory.yaml → DB, docs → knowledge graph) -- Unified API server (REST + MCP) reading from DB -- Migrate existing signals/ledger/approvals data into DB +**Phase 0 — Ontology design (no code):** +- Finalize entity types, relationship types, lifecycles +- Write seed manifests (`seeds/ontology.yaml`, `seeds/inventory.yaml`, `seeds/policy.yaml`) +- Review diagrams with operator -**Phase 2 — Services (Scheduler + Actuator + Notifier):** -- Adapt scheduler to read/write DB -- Build actuator (Act stage) with kill-switch -- Matrix notifier (abstracted interface + impl) -- SSH-based execution (hybrid approach) +**Phase 1 — Foundation (Go + DB):** +- Go module setup, project structure +- PostgreSQL migrations (all 5) +- Seed ingest pipeline (YAML → DB) +- sqlc queries for core operations +- Import existing signals/ledger data -**Phase 3 — Agent (Hermes container):** -- Hermes Docker image -- Gateway mode config -- Homelab skills (homelab-ops) -- Connect from workstation, verify MCP + SSH access +**Phase 2 — API (Go):** +- Gin server with REST routes +- MCP protocol adapter +- Policy enforcement middleware +- Knowledge graph ingestion from docs/ -**Phase 4 — Secrets (Infisical):** -- Stand up Infisical in the stack -- Migrate SOPS secrets -- Wire all services to Infisical -- Retire SOPS +**Phase 3 — Control loop (Go):** +- Scheduler (Observe) — probes, signals, state snapshots +- Actuator (Act) — classify, execute, verify +- Learning engine — feedback, patterns, skills -**Phase 5 — Deploy pipeline:** -- Docker Compose fully working -- Gitea webhook → rebuild trigger -- Caddy ingress re-point to mac-mini -- Health checks + monitoring +**Phase 4 — Agent (Hermes container):** +- Hermes Docker image, gateway config +- Homelab skills +- Connect from workstation, verify MCP + SSH -**Phase 6 — Cutover + cleanup:** -- Verify end-to-end -- Stop apps/105 services -- Clean up mac-mini old artifacts -- Remove old deploy webhooks +**Phase 5 — Secrets (Infisical):** +- Stand up Infisical, migrate SOPS, wire services + +**Phase 6 — Deploy + cutover:** +- Docker Compose, Gitea webhook, Caddy re-point +- End-to-end verification +- Stop apps/105, clean up mac-mini + +## Reuse (logic carried over, rewritten in Go) + +| Existing Python | What it becomes in Go | +|---|---| +| `oikos/decide.py` | `internal/policy/classify.go` — same scoring logic, reads from DB + patterns | +| `oikos/signal.py` | `internal/scheduler/signal.go` — same lifecycle, DB-backed | +| `oikos/approve.py` | `internal/policy/approve.go` — same grant lifecycle, DB-backed | +| `oikos/ledger.py` | `internal/db/` — ledger_entries table + sqlc queries | +| `oikos/policy.py` + `policy.yaml` | `internal/policy/` + `seeds/policy.yaml` → DB tables | +| `oikos/drift.py` | `internal/scheduler/` — drift detection, writes signals to DB | +| `oikos/relations.py` | `internal/ontology/graph.go` — SQL graph traversal | +| `oikos/report.py` | `internal/api/routes/` — report endpoints, reads from DB | +| `mcp/server.py` | `internal/api/mcp.go` — MCP adapter on top of DB | +| `bin/homelab` logic | `internal/api/routes/` — same operations, REST interface | +| `oikos/scheduler.py` | `internal/scheduler/` — same probes, goroutines for concurrency | +| `oikos/approve.py` Matrix delivery | `internal/notifier/matrix.go` | +| *(new)* | `internal/learning/` — feedback, patterns, skills (no Python equivalent) | + +## Risks / trade-offs + +- **Go rewrite** — the existing Python code (~4400 lines) is replaced. The logic and + design patterns carry over, but it's a full rewrite. Mitigated by the fact that the + Python code is well-documented and the Go structure mirrors it. +- **Hermes in Docker** — agent's world is the container. SSH access is the bridge. + Hybrid approach (mounted keys now, actuator gateway later). +- **PostgreSQL as SPOF** — mitigated by Docker volume persistence + automated + `pg_dump` backups (the scheduler can do this once running). +- **Learning model cold start** — no patterns/skills exist initially. The agent starts + cautious (escalates everything), accumulates feedback, and gradually becomes more + autonomous as patterns validate. This is by design — trust is earned. +- **Infisical bootstrapping** — SOPS coexists during transition. Keep SOPS as fallback. +- **Multi-agent concurrency** — `SELECT ... FOR UPDATE SKIP LOCKED` prevents two + actuator passes from acting on the same signal. +- **Ontology evolution** — as the homelab changes, entity types and relationship types + need to be added/modified. The DB-native approach makes this an API call, not a + file edit + redeploy. + +## Verification (end to end) + +1. **Ontology:** `seeds/ontology.yaml` ingested — `SELECT * FROM entity_types` shows + all types across 3 layers; lifecycle definitions match the state machine diagrams. +2. **DB:** `docker compose up postgres` — all 5 migrations applied; seed ingest + populates entities + relationships from `inventory.yaml`. +3. **API:** `curl http://localhost:8090/api/v1/entities?type=service` returns the + fleet; MCP `list_services` works via the same endpoint. +4. **Scheduler:** trigger a probe pass — signals in DB, state snapshots written. +5. **Actuator:** raise a test `service-down` signal → actuator classifies → + auto-acts (restart) or escalates (Matrix) → execution recorded → feedback generated. +6. **Learning:** after N executions of the same (entity_type, action), a pattern + appears with confidence score; after enough evidence, a skill is created. +7. **Classifier with learning:** set `autonomy.auto_act: reversible_low` → next + similar signal: classifier checks pattern confidence → auto-acts if high, + escalates if low. Kill-switch (`auto_act: off`) → always escalates. +8. **Hermes:** connect from another workstation → agent responds, queries API via + MCP, can SSH to hubris. +9. **Secrets:** Infisical running, Go services fetch secrets, SOPS files removed. +10. **Deploy:** `git push` → Gitea webhook → `docker compose build + up -d` → + changes live, seed ingest syncs any YAML changes to DB. +11. **Knowledge:** MCP `search_knowledge("caddy")` returns docs linked to + `entity:service:caddy`. +12. **Cutover:** stop apps/105, verify production traffic only from Docker OS. ## Out of scope (for now) -- Oikos Console web UI (deferred — built later on top of the API for specific tasks) +- Oikos Console web UI (deferred — built later on top of the API) - Multi-node deployment (designed for, not implemented) -- Vector embeddings / semantic search (knowledge graph is structured-only for now) -- SSH-key-signed approval requests (same backlog as before) +- Vector embeddings / semantic search (structured graph only for now) +- SSH-key-signed approval requests - Prometheus / trend signals -- Actuator gateway pattern (Phase 2 of the hybrid — start with mounted SSH) -- Custom homelab skills beyond `homelab-ops` (add as needed) +- Actuator gateway pattern (Phase 2 of hybrid — start with mounted SSH) +- Automated skill extraction via LLM (patterns extracted statistically for now; + LLM-assisted skill refinement is a future enhancement)