6 Commits

Author SHA1 Message Date
14448a7dd9 chore: add plan 2026-07-06 21:56:56 +02:00
b047c757a7 chore: update title 2026-07-06 21:08:13 +02:00
af14c38fb2 docs: convert OODA loop diagram to real Mermaid syntax
The operating-model diagram in README was ASCII box art in a plain code
fence, not an actual Mermaid diagram — it wouldn't render as a graph on
Gitea/GitHub. Replaced with a `flowchart TD` matching the convention already
used by oikos/gen-topology.py's generated topology.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:49:46 +02:00
0d2093ba3b docs: replace hardcoded infra details in README with pointers to index pages
Problem: README listed specific IPs, container IDs, and per-host counts (e.g.
"hubris (15 active): 102 nfs-export, 103 paperless..."). This duplicates
inventory.yaml and the wiki index pages, and goes stale every time a node
moves, gets added, or is destroyed — exactly what happened during the strong
migration.

Fix: Replaced the Proxmox Hosts / VMs / LXC Containers / Cross-Cutting
Infrastructure subsections with plain pointers to their authoritative index
pages (knowledge/wiki/{hosts,vms,containers,infrastructure}/index.md).
Also dropped the "Last refreshed against live state" date line — another
claim that goes stale without a mechanism to keep it honest.

README's job is navigation, not a live topology snapshot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:38:29 +02:00
d90de0759c docs: redesign README for agent clarity and usability
Problem: README.md was human-centric and lacked critical context for agents
(LLMs running on enrolled homelab clients). Agents needed:
- Explicit entry points (AGENTS.md → OIKOS.md → skills → MCP/files)
- Decision tree for tool selection (when to use MCP vs files vs grep)
- Explanation of operating model (OODA loop, risk classes, layer model)

Solution: Reorganized README with agent-first sections while preserving existing
human-useful content:

NEW SECTIONS:
- "For Agents" (entry points + MCP tool selection table with decision criteria)
- "Understanding the Operating Model" (Mermaid OODA loop diagram, risk classes,
  decision flow: classify → escalate if needed → execute → document)
- "Finding & Understanding Information" (layer model table: sources/wiki/index/log,
  what's immutable vs editable, when to update docs)

REVISED SECTIONS:
- "Map & Quick Navigation" (agent entry points first, then topology)
- "Conventions" (expanded with agent-specific guidance: caveman.md, page-templates.md)
- "Updating the Wiki" (clarified infrastructure changes vs restructuring;
  reinforced same-session update rule with explicit checklist)
- "More Information" (grouped agent-facing resources: HERMES, operations,
  skills, shared conventions)

All links verified. No new files needed — all referenced content already exists.

Verification:
- OODA loop diagram present (visual roadmap for decision flow)
- MCP vs Files vs Shell table shows decision criteria
- Layer model (sources/wiki/index/log) explained with immutability matrix
- All cross-references resolve
- Existing topology + infrastructure content preserved

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:32:08 +02:00
18589f6d8d Merge: wiki-hq doc architecture adoption + naming conventions
Complete reorg of narrative docs into sources/wiki/index/log + .agents/ model.
Six implementation phases + two naming clarification passes.

Phase 1-2: Adopt wiki-hq conventions (writing-style, llm-wiki, agent separation)
Phase 3: Move narrative into knowledge/wiki/
Phase 4: Reshape runbooks into skills
Phase 5: Style + README pass
Phase 6: Streamline (move investigations, operations, HERMES to their final homes)
Naming: Explicit conventions for foundational docs (ALL-CAPS) vs content (lowercase)

All 126 pre-existing broken links fixed. Topology, MCP, substrate untouched.
Verification: docs-lint clean, build_host_files idempotent, all doc_page targets resolve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:20:42 +02:00
2 changed files with 391 additions and 70 deletions

241
README.md
View File

@@ -1,89 +1,190 @@
# Homelab Wiki — `hubris`
# Homelab OS
Living documentation for the **hubris** Proxmox homelab. Every node, every cross-cutting system, and every meaningful incident is its own page; pages are linked so you can start anywhere and walk the graph.
Living documentation for the **hubris** Proxmox homelab + Oikos operating system.
> Last refreshed against live state: **2026-07-06**.
**For agents running on enrolled clients:** start with [AGENTS.md](AGENTS.md), then [OIKOS.md](.agents/OIKOS.md).
## Quick navigation
---
- [Proxmox hosts](#proxmox-hosts) — hubris (primary) + strong (cluster member)
- [VMs](#vms) — ZimaOS, Home Assistant
- [LXC containers](#lxc-containers) — service fleet (17 containers)
- [Cross-cutting infrastructure](#cross-cutting-infrastructure) — DNS, ingress, mesh, storage, auth
- [Investigations](#investigations) — incident timelines
- [Operations](#operations) — cheatsheet, enrollment, runbooks
- [Glossary](knowledge/GLOSSARY.md) — term definitions
- [Conventions](#conventions) — wiki style, changelog hygiene, same-session update rule
## For Agents — Navigation & Entry Points
## Map
### You are running on a client enrolled in the hubris homelab
### Proxmox hosts
- [`hubris`](knowledge/wiki/hosts/hubris.md) — PVE node, GMKtec NucBox M6 Ultra, `192.168.8.77` — primary host, runs [8 LXCs](knowledge/wiki/containers/index.md) + 2 VMs
- [`strong`](knowledge/wiki/hosts/strong.md) — PVE node (cluster hostname `strong`), `192.168.178.181` — 2nd member of `Homelab` cluster. Hosts [7 LXCs](knowledge/wiki/containers/index.md) migrated from hubris (Phase 1+2, 2026-07-05)
1. **First:** Read [AGENTS.md](AGENTS.md) once. It explains who you are, the topology, available tools, conventions, and how to act.
2. **Before any mutation:** Read [OIKOS.md](.agents/OIKOS.md). It defines the operating model, risk classes, approval flow, and the ontology you'll consult.
3. **For specific workflows:** Load the matching skill from `.agents/skills/<name>/SKILL.md` (e.g., [service-health-check](.agents/skills/service-health-check/SKILL.md)).
4. **When in doubt:** Use MCP tools (`search_docs`, `get_page`, `explain`, `get_changelog`) — they're cheaper and more reliable than grepping.
### VMs
- [100 — `zimaos`](knowledge/wiki/vms/100-zimaos.md) — ZimaOS 1.6.1, NAS frontend (evaluation)
- [108 — `haos-16.3`](knowledge/wiki/vms/108-haos.md) — Home Assistant OS
- See [vms/index.md](knowledge/wiki/vms/index.md) for the full table
### Key References for Agents
### LXC containers
See the full table with IPs, hosts, mounts, and status in
[`containers/index.md`](knowledge/wiki/containers/index.md). Quick summary:
- **What am I?** → `/opt/homelab-context/hosts/<hostname>.yaml` (read on first run)
- **Live topology** → `inventory.yaml` + `hosts/*.yaml` (canonical, always wins)
- **Risk & approval** → [oikos/policy.yaml](oikos/policy.yaml) (enforced, not advisory)
- **Runbooks & workflows** → [.agents/skills/](.agents/skills/) (risk class + verification checklist included)
- **State of Oikos** → [OIKOS.md build status](.agents/OIKOS.md#build-status-30-day-roadmap) (scheduled probes, drift detectors, signals, approval engine)
- **hubris** (10 active): 102 nfs-export, 103 paperless, 104 gitea, 105 apps,
114 nextcloud, 119 sophia, 120 mule-images, 121 caddy, 124 authentik (outpost),
128 trmnl, 132 rclone
- **strong** (7 active): 101 jellyfin, 118 elementsynapse, 122 arriman,
129 house, 130 grimmory, 133 seanime, 134 romm
- **Destroyed (archaeology)**: 100 arr, 106 flaresolverr, 107 marimo,
109 syncthing, 110 photoprism, 111 karakeep, 112 immich, 115 reticulum,
123 claudio-bot, 125 seafile, 126 plato, 127 mule-photos-new — see
[containers/index.md](knowledge/wiki/containers/index.md#recently-destroyed-kept-for-archaeology)
### When to Use MCP vs Files vs Shell
### Cross-cutting infrastructure
- [Infrastructure index](knowledge/wiki/infrastructure/index.md) — map of every cross-cutting system
- [Glossary](knowledge/GLOSSARY.md) — term definitions
- [DNS — split-horizon](knowledge/wiki/infrastructure/dns.md)
- [Ingress — Caddy + VPS traefik](knowledge/wiki/infrastructure/ingress.md)
- [Mesh — Tailscale → Netbird migration](knowledge/wiki/infrastructure/mesh.md)
- [Monitoring — Hermes health watchdog](knowledge/wiki/infrastructure/monitoring.md)
- [Media permissions — `media` GID 10000](knowledge/wiki/infrastructure/media-permissions.md)
- [SSH access](knowledge/wiki/infrastructure/ssh-access.md)
- [Backups — rclone → Proton Drive (LXC 132); restic-on-USB deprecated](knowledge/wiki/infrastructure/backups.md)
- [Auto-deploy — gitea-webhook pipelines](knowledge/wiki/infrastructure/auto-deploy.md)
- [VPS hardening — IONOS / netbird control plane](knowledge/wiki/infrastructure/vps-hardening.md)
- [Homelab context distribution](knowledge/wiki/infrastructure/homelab-context.md) — cross-client `/opt/homelab-context` + MCP + secrets-issuance
| Task | Use | Tool |
|------|-----|------|
| Resolve hostname → address | MCP | `get_host(name)` or `list_services()` |
| Search wiki by content | MCP | `search_docs(query)` |
| Read a wiki page | MCP or file | `get_page(path)` or `cat knowledge/wiki/.../...md` |
| Get changelog entries | MCP | `get_changelog(page, since?)` |
| Understand a service | MCP | `explain(service)` — compact context card, cheaper than search+read |
| Blast-radius query | MCP | `get_relations(entity)` (ontology walk) |
| List available secrets | MCP | `list_my_secrets()` (scoped to your age key) |
| Browse or grep | File | Raw `grep` when MCP unreachable, or exploratory browsing |
### Investigations
Time-stamped incident notes / experiments in [`investigations/index.md`](knowledge/sources/investigations/index.md).
Resolved cases move to [`investigations/archive/`](knowledge/sources/investigations/archive/).
**When MCP is unreachable:** fall back to grepping the clone at `/opt/homelab-context/`. The local files are the same; MCP is just an index.
### Operations
- [Command cheatsheet](.agents/operations/commands.md)
- [Agent enrollment](.agents/operations/agent-enrollment.md) — bootstrap a new client (workstation, LXC, VM) into the homelab context system
---
## Understanding the Operating Model
Before you act, **classify your action against [oikos/policy.yaml](oikos/policy.yaml)**.
### The Oikos OODA Loop + Decision Tree
```mermaid
flowchart TD
Observe["**Observe**<br/>probes, drift detectors, agent signals"]
Orient["**Orient**<br/>ontology, context, state, entity relations"]
Decide{"**Decide**<br/>classify against oikos/policy.yaml"}
Auto["Auto-act<br/>(unattended)"]
Escalate["Escalate<br/>homelab approval request"]
Act["**Act**<br/>homelab CLI, runbooks, skills"]
Verify["**Verify**<br/>checklist from SKILL.md"]
Ledger["**Ledger**<br/>mutation record: who/what/risk"]
Document["**Document**<br/>wiki update, same-session rule"]
Observe --> Orient --> Decide
Decide -->|read_only, reversible_low| Auto
Decide -->|config_mutation, destructive| Escalate
Auto --> Act
Escalate -->|approval granted| Act
Act --> Verify --> Ledger --> Document
Document -.loop.-> Observe
```
### Risk Classes (enforced, not advisory)
From [oikos/policy.yaml](oikos/policy.yaml):
- **read_only** — status, logs, docs, inventory queries. Unattended. MCP tools are all read_only.
- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger entry.
- **config_mutation** — tracked-config edits (commit+push, never local), deploys, upgrades, DNS/ingress changes. **Operator approval required.**
- **destructive** — destroy, format, wipe, rotate, revoke. **Approval + typed confirmation phrase.**
### Decision Flow
1. **Decide:** Use `homelab decide <action> <entity>` to classify (risk class × blast radius × confidence).
2. **Escalate if needed:** `homelab approval request` (Matrix-delivered to operator; see [operations/commands.md](.agents/operations/commands.md)).
3. **Execute:** Use `homelab` CLI (not ad-hoc SSH) — it enforces policy, logs mutations, and verifies outcomes.
4. **Document:** Update wiki in the same session (per [AGENTS.md §5](AGENTS.md#5-acting-on-the-homelab) and the [same-session rule](.agents/shared/page-templates.md#same-session-update-rule)).
### The Ontology Graph
Everything that can break, be changed, or hold data has an entity in `inventory.yaml` + `oikos/ontology.yaml`. Blast-radius questions ("what breaks if strong goes down?") are graph walks via `homelab node <name> relations`, not doc archaeology.
**See:** [OIKOS.md](.agents/OIKOS.md) (full operating model, OODA loop, primitives, lifecycle gates, build status).
---
## Finding & Understanding Information
The narrative documentation is organized in **layers**:
| Layer | What it is | Where | Immutable? | How agents use it |
|-------|-----------|-------|-----------|-------------------|
| **Sources** | Raw evidence: incidents, external refs, live state | `knowledge/sources/investigations/` | Yes | Read to understand root causes; do not rewrite |
| **Wiki** | Synthesized current-state: one page per node & per system | `knowledge/wiki/{containers,hosts,vms,infrastructure}/` | No | This is the reference layer — if wiki disagrees with live state, update it *in the same session* |
| **Index** | Pure listings — every page in scope with one-line summary | `index.md` / folder `README.md` | No | Navigation aid; keep it current when wiki restructures |
| **Log** | Append-only doc-maintenance record (restructures, ingests, lints) | `knowledge/log.md` | Yes (append-only) | Read to understand past doc changes; never edit directly |
**Changelog ≠ Log:** Each wiki page ends with a `## Changelog` (infrastructure changes to that node, machine-parsed). That's not the Log; the Log records *doc operations* only.
**See:** [llm-wiki.md](.agents/shared/llm-wiki.md) (full rules, page structure, immutability contract).
---
## Map & Quick Navigation
### Agent Entry Points (Start Here)
- **You are an agent** → [AGENTS.md](AGENTS.md) (on deployed clients: `/opt/homelab-context/AGENTS.md`)
- **Operating model & risk policy** → [OIKOS.md](.agents/OIKOS.md)
- **Specific workflows** → [.agents/skills/](.agents/skills/) (load the matching SKILL.md before acting)
- **Operations cheatsheet** → [.agents/operations/commands.md](.agents/operations/commands.md)
- **Tools & MCP reference** → [AGENTS.md §3 — The MCP server](AGENTS.md#3-the-mcp-server)
### Topology & Infrastructure
Node counts, IPs, and service lists change often — treat `inventory.yaml` and the index pages below as the source of truth, not this README.
- **Proxmox hosts** → [knowledge/wiki/hosts/index.md](knowledge/wiki/hosts/index.md)
- **VMs** → [knowledge/wiki/vms/index.md](knowledge/wiki/vms/index.md)
- **LXC containers** → [knowledge/wiki/containers/index.md](knowledge/wiki/containers/index.md)
- **Cross-cutting infrastructure** (DNS, ingress, mesh, backups, monitoring, auto-deploy, VPS) → [knowledge/wiki/infrastructure/index.md](knowledge/wiki/infrastructure/index.md)
### Knowledge & References
- **Glossary** — [GLOSSARY.md](knowledge/GLOSSARY.md)
- **Incidents & investigations** — [knowledge/sources/investigations/index.md](knowledge/sources/investigations/index.md) (active + [archive](knowledge/sources/investigations/archive/))
- **Plans & design docs** — [plans/index.md](plans/index.md)
- **Hermes agent** (for Hermes-enrolled clients) — [HERMES.md](.agents/HERMES.md)
---
## Conventions
- **File naming.** Foundational docs (entry-points, agent instruction, references) are ALL-CAPS (`AGENTS.md`, `OIKOS.md`, `GLOSSARY.md`); containers use `<id>-<name>.md`; infrastructure pages use lowercase-with-dashes; plans and incidents use `YYYY-MM-DD-slug.md`; skills are `<name>/SKILL.md`. See [page-templates.md](.agents/shared/page-templates.md) for the full rules.
- **Each node page** ends with a `## Changelog` section. Reverse-chronological. Entry format:
```
### YYYY-MM-DD — short title
one or two lines on what changed and why.
```
- **Cross-linking is mandatory.** If a page references another node or system, link to it. Treat orphans as a bug.
- **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running config, fix the wiki *and* note the change in the relevant changelog.
- **Tracked configs.** A node whose config lives in a Gitea repo (Caddy, Gitea customizations, Artifacto, mule-image) is auto-deployed via webhook — see [auto-deploy](knowledge/wiki/infrastructure/auto-deploy.md). Edits there must be pushed, not left local.
- **No secrets.** This is a private repo on `git.hubris.network`, but still: paths to secret files are fine, secret values are not.
All pages follow:
## Maintaining this wiki
- **File naming.** Foundational docs (entry-points, agent instruction, references) are ALL-CAPS (`AGENTS.md`, `OIKOS.md`, `GLOSSARY.md`); containers use `<id>-<name>.md`; infrastructure pages use lowercase-with-dashes; plans and incidents use `YYYY-MM-DD-slug.md`; skills are `<name>/SKILL.md`. See [page-templates.md](.agents/shared/page-templates.md#file-naming) for the full rules.
- **Voice & vocabulary.** Concise, technical, sysadmin-to-sysadmin. No marketing prose, no puffers (seamless, robust, leverage, etc.). Full rules in [writing-style.md](.agents/shared/writing-style.md).
- **Cross-linking is mandatory.** If a page references a node or system, link to it. Treat orphans as a bug.
- **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running state, fix the wiki *and* add a changelog entry *in the same session*.
- **Tracked configs.** Pages for configs living in git repos (Caddy, Gitea, Artifacto, mule-image) must note the repo. Edits go through commit+push, never local changes. See [auto-deploy](knowledge/wiki/infrastructure/auto-deploy.md).
- **No secrets.** This is a private repo, but still: reference secret *paths*, never secret *values*.
When you change a node:
1. Update the relevant page (config snapshot, ports, mounts).
2. Add a changelog entry at the bottom of that page.
3. If the change touches a cross-cutting system (DNS, Caddy, Authentik, mesh), update *that* page too and link it from the changelog entry.
4. If it's an incident, add an entry to [`investigations/`](knowledge/sources/investigations/index.md).
**For agents:** Read [caveman.md](.agents/shared/caveman.md) (terse communication standard). Use templates at [page-templates.md](.agents/shared/page-templates.md) when creating pages.
## See also
---
- [`.agents/shared/page-templates.md`](.agents/shared/page-templates.md) — page templates and tone
- [`.agents/shared/writing-style.md`](.agents/shared/writing-style.md) — prose style, banned vocabulary
## Updating the Wiki
### When You Change Infrastructure
1. Update the relevant page (config snapshot, ports, mounts, IP address).
2. Add a `### YYYY-MM-DD — title` entry to the page's `## Changelog` section (reverse chronological order).
3. If the change touches a cross-cutting system (DNS, Caddy, Authentik, mesh), update *that* page too and link from the changelog.
4. If it's an incident, add a record to [`knowledge/sources/investigations/`](knowledge/sources/investigations/index.md).
### When You Restructure the Wiki
1. Update the relevant `index.md` / `README.md` in that section.
2. Add a single-line entry to [`knowledge/log.md`](knowledge/log.md): `## [YYYY-MM-DD] <operation> | <summary>` (e.g., `## [2026-07-06] restructure | split infrastructure/dns into dns.md + dns-advanced.md`).
### The Same-Session Update Rule
**Any meaningful state change made in this session requires a wiki update before the session closes.** A change that touches a container page must also update:
- The `containers/index.md` table (IPs, host, mounts, status)
- The root `README.md` table (if affected)
- The Caddy page site list (if affects `*.hubris.network` routing)
- The DNS / ingress infrastructure pages (if affects routing)
- The `hosts/hubris.md` or `hosts/strong.md` page (if container count changes)
- The `inventory.yaml` host entry (source of truth for `hosts/*.yaml` generation)
- The `knowledge/wiki/infrastructure/topology.md` (regenerate if needed)
Not updating all linked places is a bug. See [page-templates.md — same-session update rule](.agents/shared/page-templates.md#same-session-update-rule).
---
## More Information
- **For Hermes agents** → [HERMES.md](.agents/HERMES.md) (persona, source-of-truth hierarchy, token efficiency)
- **For manual workflows** → [.agents/operations/](.agents/operations/) (commands cheatsheet, agent enrollment, Hermes guide)
- **For skills/runbooks** → [.agents/skills/](.agents/skills/) (load the matching SKILL.md before acting; includes risk class + verification)
- **MCP tools** → [AGENTS.md §3](AGENTS.md#3-the-mcp-server) (available tools, when to use MCP vs files)
- **Page templates & voice** → [.agents/shared/](.agents/shared/) (page-templates.md, writing-style.md, caveman.md, llm-wiki.md)
- **Machine-readable substrate** → `inventory.yaml`, `oikos/policy.yaml`, `oikos/ontology.yaml` (not part of the wiki; see [llm-wiki.md](.agents/shared/llm-wiki.md#rules))

View File

@@ -0,0 +1,220 @@
# Plan: Consolidate the Oikos control plane onto mac-mini
**Status:** Planned (2026-07-06) — approved, not yet executed.
## Context
The goal is an agentic OS that runs the homelab and keeps the operator in
control. Today the Oikos control plane is scattered and, worse, mis-placed:
`homelab-mcp`, `secrets-issuance`, and `oikos-console` all run on **LXC 105
(apps) — a container inside hubris**, `oikos-scheduler` (the Observe stage) is
undeployed, and the Hermes agent runtime is spread across several clients. Two
problems follow:
- **The observer lives inside the observed.** A scheduler on apps/105 can't
report "hubris is down" — it dies with hubris. The Observe/Act loop needs an
always-on node *outside* both hypervisors.
- **Deploy is fragmented.** Three Gitea webhooks (ids 10/11/14); webhook 14
(console) has been 403-broken for a cause never found, so console never
auto-redeploys.
**Decision:** make **mac-mini** the single Oikos control node — it's always-on,
external to both hubris and strong, on the mesh, can SSH/curl both, and already
runs Hermes. Move *everything* (MCP, secrets-issuance, console, scheduler, the
new actuator, Hermes) there; apps/105 stops running Oikos. Because mac-mini
already has a launchd git-sync (5-min pull), we replace all three Gitea webhooks
with pull-based deploy — which eliminates the webhook-403 problem entirely.
This is a **macOS re-platforming** (systemd → launchd) plus an **ingress
re-point**. The public URLs (`mcp.`/`secrets.`/`oikos.hubris.network`) and DNS
are unchanged — only the Caddy backend and the runtime host move.
## Target architecture
```
Caddy (LXC 121, ingress) ──reverse_proxy over netbird──▶ mac-mini (always-on, macOS)
mcp.hubris.network ─────────────────────────────▶ homelab-mcp :9810
secrets.hubris.network ─────────────────────────────▶ secrets-issuance :9820
oikos.hubris.network ─────────────────────────────▶ oikos-console :8091
mac-mini also runs (no ingress, mesh/SSH outward):
oikos-scheduler + actuator (launchd StartInterval 600s) ──ssh/curl──▶ hubris, strong
Hermes agent runtime (already here)
Single synced checkout: /opt/homelab-context (5-min launchd git pull = deploy trigger)
apps/105: Oikos services stopped + removed; keeps its non-Oikos roles (artifacto, plantuml)
```
## Workstreams
### 1. macOS runtime — launchd plists (`deploy/macos/`, new)
One `LaunchDaemon` plist per service (run at boot, no GUI login), mirroring each
systemd unit's `ExecStart`/`Environment`, replacing `[Timer]` with launchd keys:
| Service | plist | launchd equivalent of |
|---|---|---|
| `homelab-mcp` | `network.hubris.homelab-mcp.plist` | `mcp/deploy/homelab-mcp.service` (KeepAlive) |
| `secrets-issuance` | `network.hubris.secrets-issuance.plist` | its systemd unit (KeepAlive) |
| `oikos-console` | `network.hubris.oikos-console.plist` | `oikos/console/deploy/oikos-console.service` (KeepAlive) |
| `oikos-scheduler` | `network.hubris.oikos-scheduler.plist` | `oikos/systemd/oikos-scheduler.{service,timer}` (`StartInterval` 600) |
- **Single checkout, shared venv.** All services run in-place from
`/opt/homelab-context` with one shared `.venv` (gitignored), instead of the
Linux per-service checkouts (`/opt/homelab-mcp`, `/opt/oikos-console`). This is
the concrete consolidation win: one clone, one sync, N daemons.
- **Confinement gap (tradeoff, document it):** the systemd units use
`ProtectSystem=strict`, `ReadOnlyPaths`, `NoNewPrivileges`, `PrivateTmp`
launchd has no equivalent. Mitigate by running the daemons as a dedicated
non-root `_oikos` user (or `dtoro`) with least-privilege file perms, and note
the reduced isolation in the deploy README + OIKOS.md risk section.
- **`install-macos.sh`** — create `_oikos` user + venv (`pip install` the deps),
copy plists to `/Library/LaunchDaemons`, `launchctl bootstrap system …`.
Idempotent, mirroring `mcp/deploy/deploy.sh`'s shape.
### 2. macOS deploy = the existing pull-sync (retire the 3 webhooks)
- Extend mac-mini's existing launchd git-sync post-pull hook (the enrollment
`tools/post-pull.sh` path) to `launchctl kickstart -k` any service whose files
changed after a pull. Pull-based deploy replaces webhook-push deploy.
- **Retire Gitea webhooks 10, 11, 14** (operator step in Gitea UI). This removes
the never-solved console-webhook 403 by deleting the webhook, not fixing it.
- Update `knowledge/wiki/infrastructure/auto-deploy.md`: mac-mini services deploy
by pull-sync, not webhook; the webhook shape now applies only to remaining
in-lab config repos.
### 3. Build the actuator + autonomy kill-switch (the Act stage)
Still required — it's the control loop, now living on mac-mini.
- **`oikos/act.py`** (new): read open Signals (`oikos_signal.list_signals(
state="raised")`); for each with a `recommended_action`, call
`oikos_decide.classify()`. `auto-act` (reversible_low, contained, confident)
→ `start_acting()` (the existing unused transition), execute via
`homelab restart <svc> -y` (reuses `cmd_restart` + its `_record_change` ledger
write), run the Signal's `verification`, then `resolve()` or leave open with a
`result=failed` ledger entry so the next pass escalates. Everything else →
`oikos_approve.request(...)` + `acknowledge(note="escalated: <id>")`. A
ledger-history loop-guard caps auto-retries per (entity, kind).
- **Autonomy kill-switch** in `oikos/policy.yaml` (`autonomy.auto_act:
off|reversible_low`, `never_auto_act:` entity list) + loaders in
`oikos/policy.py` — the "keep control" guarantee.
- Wire `python3 oikos/act.py run` into the scheduler's launchd pass (after the
observe pass) and commit `signals/` + `ledger/`.
### 4. Ingress re-point (Caddy backends: apps → mac-mini mesh)
- In `dtoro/caddy-conf` (separate repo, tracked-config commit+push): change the
three `reverse_proxy` backends for `mcp.`/`secrets.`/`oikos.hubris.network`
from `192.168.8.205:<port>` to **mac-mini's netbird address**
(`mac-mini-234-17.netbird.selfhosted:<port>`), since Caddy (8.x) and mac-mini
(178.x) are cross-subnet — mesh is the robust path. **DNS and public URLs
unchanged.**
- **Verify Caddy → mac-mini reachability first** (Caddy container must resolve +
reach the netbird name; confirm netbird is on the caddy LXC or route via
hubris). If mesh isn't reachable from Caddy, fall back to the 178.x LAN IP and
confirm hubris routes 8.x↔178.x (it's dual-homed: hubris 178.181, apps 8.205).
### 5. Secrets & SSH keys on mac-mini
- **SOPS recipients:** grant mac-mini's `age_pubkey`
(`age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs`) on every
secret the moved services decrypt at runtime (secrets-issuance's issuing key,
approval HMAC `secrets/oikos-approval-hmac.yaml`, any service creds), then
`sops updatekeys`. Edit `.sops.yaml` recipient lists + re-key; commit via
`homelab client add mac-mini --finalize-pubkey …` path (comment-preserving).
- **SSH keys:** provision on mac-mini the `mcp-reader.key` restricted key the MCP
server SSH-proxies to hubris with (`HOMELAB_MCP_SSH_KEY`), plus root SSH access
to hubris + strong for the scheduler's disk probe (`_ssh` in
`oikos/scheduler.py:102`). Store under `/etc/homelab-mcp/` and `_oikos`'s
`~/.ssh` respectively.
- **Security note (document):** the age-key *issuer* now runs on a workstation.
Record the widened surface in OIKOS.md; least-privilege `_oikos` user +
keychain/file perms are the mitigation.
### 6. Inventory / topology truth (same-session rule)
- `inventory.yaml`: set `services.{homelab_mcp,secrets_issuance}.backend:
mac-mini`; add `oikos_console` + `oikos_scheduler` service entries backed by
mac-mini; give `mac-mini` a `runs:` list (homelab-mcp, secrets-issuance,
oikos-console, oikos-scheduler); remove `homelab-mcp`/`secrets-issuance` from
`apps.runs`. Keep `systemd_unit`→ note launchd label.
- Regenerate derived views: `mcp/build_host_files.py` (hosts/*.yaml) and
`oikos/gen-topology.py` (cards + `topology.md`). Update
`knowledge/wiki/containers/105-apps.md` and add a mac-mini control-node page
under `knowledge/wiki/hosts/` (or note on the workstation).
### 7. Decommission on apps/105 (operator steps, documented not executed)
`systemctl disable --now homelab-mcp secrets-issuance oikos-console` + their
`-deploy` webhook units on apps; remove `/opt/homelab-mcp`, `/opt/oikos-console`
checkouts. Captured in the deploy README as a cutover checklist.
### 8. Docs
`.agents/OIKOS.md` (control node = mac-mini; loop closed; risks),
`auto-deploy.md` (pull-sync), `deploy/macos/README.md` (cutover runbook),
`.agents/skills/autopilot-pass/SKILL.md` (supervised actuator run), host/service
cards (regenerated).
## Critical files
| File | Change |
|---|---|
| `deploy/macos/*.plist` + `install-macos.sh` + `README.md` | **New** — launchd runtime + cutover |
| `oikos/act.py` | **New** — actuator (Act stage) |
| `oikos/policy.yaml`, `oikos/policy.py` | Autonomy kill-switch + loaders |
| `oikos/systemd/run-scheduler.sh` → macOS variant | Observe+Act pass, commit signals/+ledger/ |
| `inventory.yaml` | Move service backends apps→mac-mini; mac-mini `runs:` |
| `hosts/*.yaml`, `oikos/cards/*`, `topology.md` | Regenerated |
| `dtoro/caddy-conf` (external repo) | 3 reverse_proxy backends → mac-mini mesh |
| `.sops.yaml` | Grant mac-mini recipient on moved-service secrets; re-key |
| `bin/homelab` | Optional `autopilot` subcommand |
| Docs: `OIKOS.md`, `auto-deploy.md`, `105-apps.md` | Reflect consolidation |
## Reuse (do not reimplement)
- Deploy scaffolding shape: `mcp/deploy/deploy.sh`, `oikos/console/deploy/`.
- Service definitions to port: `mcp/deploy/homelab-mcp.service`,
`oikos/console/deploy/oikos-console.service`, `oikos/systemd/*`.
- Actuator primitives: `oikos_decide.classify` (`decide.py:64`), signal
transitions incl. unused `start_acting` (`signal.py:125`), `cmd_restart` +
`_record_change` (`bin/homelab:603,190`), `oikos_approve.request`
(`bin/homelab:1370`), `oikos_ledger.history` (`ledger.py:59`).
- Comment-preserving inventory + `.sops.yaml` edits + `sops updatekeys`:
`homelab client add --finalize-pubkey` path in `bin/homelab`.
## Risks / assumptions
- **Caddy → mac-mini reachability** over netbird is unverified — workstream 4
verifies before cutover; LAN 178.x is the fallback (hubris is dual-homed).
- **Lost sandboxing** on macOS (no `ProtectSystem`/`ReadOnlyPaths`) — mitigated
by a least-privilege `_oikos` daemon user, not eliminated.
- **secrets-issuance on a workstation** widens the security surface — accepted
per scope decision; documented.
- **Single-checkout race:** a git pull mid-request could serve half-updated code
— post-pull `kickstart -k` restarts services after the pull completes.
- macOS `/opt/homelab-context` already exists (mac-mini is enrolled); daemons
need it readable by `_oikos`.
## Verification (end to end)
1. **Runtime:** `install-macos.sh` on mac-mini; `launchctl print system/…` shows
all four daemons running; `curl 127.0.0.1:{9810,9820,8091}` responds locally.
2. **Ingress:** after the caddy-conf push + reload, `curl -I
https://mcp.hubris.network/mcp`, `.../` (console → 302 Authentik),
`secrets.hubris.network/health` → 200/302 through Caddy to mac-mini.
3. **Observe+Act loop:** trigger a `service-down` Signal; confirm the launchd
scheduler pass runs it `raised→acting→resolved`, writes a `restart`/`result:
ok` ledger entry, and pushes `signals/`+`ledger/`. Kill-switch: set
`autonomy.auto_act: off` → next pass escalates instead.
4. **Deploy-by-pull:** `git push` a trivial change; within 5 min mac-mini pulls
and `kickstart`s the changed daemon (no webhook involved).
5. **Cutover:** after decommission, `systemctl is-active` on apps shows the three
services `inactive`; production traffic served only by mac-mini; MCP tools +
console + a test enrollment all still work.
## Out of scope (per prior direction)
SSH-signed approval requests, Authentik step-up on the console POST, Prometheus /
temperature trend signals, and the generalize-to-any-homelab refactor.