docs: fix plan/repo drift, retire dead Goose+Nomos and Caveman tooling
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled

Documentation and repo-hygiene pass following the client/server split:

Plan drift (audited all other active plans against current code):
- oikos-gaps-and-improvements.md: mark Section C and D.5 resolved (both
  described cmd/hermes, renamed to cmd/nomos with a real LLM loop since);
  refresh ~10 stale file:line citations; fix tool-count (33, not 28).
- liveness-drift-and-ux-cohesion.md: fix stale default-model claim (now
  deepseek-v4-pro since 2026-07-10) and "not yet deployed" status.
- nomos-agent-code-review.md: fix C1's citation (one unauthenticated route
  to nomos now, not two, after the client/server split).
- wails-desktop-app.md: record the production deploy outcome.

Repo structure: added missing directories to README/CONTRIBUTING layout
tables (checks/, tools/, cmd/webhook/, docs/operations/), fixed a broken
link, added ADR 0015 documenting the auth/CORS/client-split model (there
wasn't one despite CONTRIBUTING's own process requiring it), normalized
ADR 0013/0014's format drift, added an Authentication section to
AGENTS.md/CLIENTS.md (every example call was missing the now-required
bearer header).

Retired the Goose+Nomos workstation flow (bootstrap.sh --with-nomos,
tools/setup-nomos-soul.sh, .agents/operations/nomos-agent.md) and the
Caveman auto-install tooling (tools/setup-caveman.sh, tools/caveman/) —
both superseded by the production containerized Nomos agent, which has
never used either. Kept .agents/shared/caveman.md itself (the terse
writing-style convention agents still follow by reading it).

Deleted the orphaned legacy Python oikos/ directory — nothing imports it,
and bin/homelab (the CLI it was kept for) no longer exists in the repo.

Rewrote .agents/operations/agent-enrollment.md (365 -> ~110 lines) and
commands.md to match the current architecture instead of the retired
`homelab` CLI; migrated the still-true networking prerequisites (Netbird,
split-horizon DNS, SSH key distribution) into the knowledge base as a
runbook via upsert_knowledge rather than duplicating them in markdown.
Updated all 10 .agents/skills/ runbooks referencing the dead CLI with
their real MCP tool / REST API equivalents, or flagged them as needing
verification where no equivalent is confirmed yet.

Two real bugs found and fixed, not just docs:
- The tools/setup-*.sh auto-setup glob was tools/*.setup.sh in THREE
  places (tools/post-pull.sh, bootstrap.sh, and internal/httpapi/impl.go's
  GetClientContext handler) since the mechanism's introduction on
  2026-06-02 — never matched any real filename, so no client has ever
  picked up an auto-setup script via git-pull or the context-poller sync.
  Fixed all three; the Go server-side fix is the one that actually matters
  since it's what the current context-poller mechanism depends on.
- bootstrap.sh removed dead vestigial --gitea-token/--gitea-user flags
  (parsed, never consumed) left over from an earlier clone-based model.

Also flagged, not fixed (documented as an open gap in
client-enrollment/SKILL.md): bootstrap.sh tells a freshly-enrolled client
to call POST /api/v1/clients/{slug}/activate to finish enrollment, but
that route doesn't exist in api/openapi.yaml — EnrollClient sets entities
to provisioning and nothing currently transitions them to active.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-12 18:19:41 +02:00
parent 0c0f35a3a9
commit d80a394b7f
43 changed files with 609 additions and 1430 deletions

View File

@@ -1,19 +1,27 @@
# NOMOS.md — Agent persona for homelab clients
This file is the canonical agent persona for **all** AI agents running on
machines in the **hubris** homelab. It prescribes behaviour, token-efficiency
conventions, and the source-of-truth hierarchy.
This file is the canonical agent persona for AI agents running on machines
in the **hubris** homelab (Claude Code, Codex, or similar). It prescribes
behaviour, token-efficiency conventions, and the source-of-truth hierarchy.
The *production* Nomos agent (`cmd/nomos`, the containerized MCP client
gateway everyone actually talks to) uses a separate, code-adjacent persona —
`nomos/SOUL.md`, baked into its Docker image at build time
(`compose/nomos/Dockerfile`). This file is unrelated to that one; it's for
AI coding agents working *on* a homelab client machine, not the Nomos
service itself.
## Source of truth
The homelab-context repo at `/opt/homelab-context/` is the single source of
truth for:
- Fleet topology (`inventory.yaml`, `inventory.yaml`)
- Service endpoints and credentials (via `homelab secret`)
- Fleet topology (`inventory.yaml`)
- Agent behaviour and conventions
- Everything in this file
When in doubt, check `/opt/homelab-context/` first.
When in doubt, check `/opt/homelab-context/` first, or query the Oikos API/MCP
server directly (see [AGENTS.md](../AGENTS.md) §3-4) — the database is
authoritative at runtime.
## Runbooks — load, don't rediscover
@@ -28,69 +36,9 @@ wiki when a runbook already encodes it. See [OIKOS.md](OIKOS.md) for the
operating model these runbooks execute inside (OODA loop, risk classes,
approval flow, ontology).
## Agent type — how this file gets loaded
## Token efficiency
| Agent | Loading mechanism |
|-------|------------------|
| **Nomos** | `tools/setup-nomos-soul.sh` (auto-setup) → provisions `~/.nomos/SOUL.md` from this file |
| **Goose** | `.goosehints` symlink at `~/.config/goose/.goosehints``/opt/homelab-context/NOMOS.md` |
| **Claude Code / Codex** | Symlink or copy this file into the project's `CLAUDES.md` / `.claude` instructions |
**Do not edit SOUL.md or .goosehints directly.** Edit this file in the
homelab-context repo instead. Changes propagate to all clients on the next
sync (`sudo homelab sync`).
---
## Token efficiency (caveman skill)
All homelab agents use the **Caveman + RTK** token optimization approach from
https://github.com/adityahimaone/hermes-agent-rtk-caveman.
### Before running any CLI command, ask:
1. **Is there a caveman wrapper equivalent?** Use the wrapper for token-efficient
output. Available wrappers (installed at `~/bin/caveman_wrapper.sh`):
- `~/bin/caveman_wrapper.sh git-status` — compact git status
- `~/bin/caveman_wrapper.sh git-log [n]` — compact git log
- `~/bin/caveman_wrapper.sh lint [target]` — compact lint results
- `~/bin/caveman_wrapper.sh test-results [cmd]` — compact test results
2. **If no caveman wrapper exists, pipe through `rtk`** to compress output:
```
rtk <command>
```
RTK (Rust Token Killer) strips redundant whitespace, trims long paths, and
deduplicates repeated lines. This reduces token usage by 60-90% on CLI
operations.
3. **For homelab operations**, prefer the `homelab` CLI or MCP tools over
raw SSH/shell — they're already token-optimized.
### Templates
Caveman templates live at `~/templates/`:
- `git_status.txt` — compact git status format
- `git_log.txt` — compact git log format
- `lint_results.txt` — compact ESLint format
- `test_results.txt` — compact vitest/jest format
### When to skip caveman/rtk
- Interactive commands (editors, prompts) — let human-readable output pass
- Commands with no output — skip entirely
- When you need the exact raw output for post-processing
### Verification
```bash
ls ~/bin/caveman_wrapper.sh && echo "caveman ready"
```
## Important note for Nomos agents
If you are reading this as a Nomos agent, your SOUL.md was auto-provisioned
by `tools/setup-nomos-soul.sh`. This file is the canonical original — you
can verify the content matches or re-provision by running:
bash /opt/homelab-context/tools/setup-nomos-soul.sh
Apply [caveman.md](shared/caveman.md) — terse, fragment-heavy chat responses
(not committed documentation). There's no separate tool to install for
this; it's a response-style convention any agent follows by reading the
file.

View File

@@ -10,6 +10,7 @@ see [CONTRIBUTING.md](../../CONTRIBUTING.md) for a human-friendly version.
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
seed, export, secret, all
cmd/nomos/main.go Nomos MCP client gateway (standalone binary, formerly Hermes)
cmd/webhook/main.go Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
@@ -26,14 +27,19 @@ internal/domain/ Core types: entities, approvals, executions, signals
internal/ontology/ Type hierarchy validation, relationship checks
internal/knowledge/ Knowledge YAML seed ingestion
internal/config/ Config loading from env vars
web/ Control-room SPA (Svelte 5) — standalone static build, not
embedded in the oikos binary (plans/2026-07-12-wails-desktop-app.md)
api/openapi.yaml REST API contract. Source of truth for endpoints.
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
knowledge.yaml. Regenerated from DB via oikos export.
compose/ Dockerfiles. oikos/ (multi-stage), nomos/ (distroless).
compose/ Dockerfiles. oikos/ (2-stage, Go only — SPA is built/deployed
separately), nomos/ (distroless).
Caddy config at compose/caddy/Caddyfile.oikos.
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
checks/ Host health-check scripts run over SSH by the scheduler.
tools/ Client auto-setup scripts (checks).
nomos/ Nomos config.yaml, SOUL.md, skills.
.agents/ Agent instruction files, domains, shared conventions, skills.
plans/ Design documents. active/ + done/.

View File

@@ -1,365 +1,120 @@
# Agent enrollment — bootstrap a client into the homelab context system
# Agent enrollment — operational notes
This walks through enrolling a new machine (workstation, LXC, or VM) so it
joins the cross-client context system: a `/opt/homelab-context/` clone of
this repo that auto-syncs every 5 min, a per-client age key for SOPS
decryption, the `homelab` CLI, and an MCP endpoint in Claude Code's config.
**For the actual enrollment flow, see [CLIENTS.md](../../CLIENTS.md#enrollment)
— it's the current, authoritative version.** This page used to duplicate
that flow in more detail, describing a `homelab` CLI-based two-step
ceremony (`homelab client add` reserves an inventory slot → client
bootstraps → operator finalizes the pubkey). That CLI and that flow don't
exist anymore — enrollment today is one shot: `bootstrap.sh` calls
`POST /api/v1/clients/enroll` directly and gets back an age keypair +
Infisical identity in the same response. What's left here is the handful
of things that are still true and weren't already covered elsewhere.
> Onboarding a Nous-Hermes-powered Goose agent on top of standard enrollment?
> See [nomos-agent.md](nomos-agent.md). It uses the same `bootstrap.sh`
> with an additional `--with-nomos` flag.
Architecture in [project_homelab_context_plan](https://… memory link); the
operational reference is here.
## Prerequisites the client must satisfy
## Prerequisites
| Requirement | Why | How to check |
| --- | --- | --- |
| Hostname matches an entry in `inventory.yaml` | The bootstrap looks up `hosts/$(hostname).yaml`. | `hostname` (Linux) / `scutil --get LocalHostName` (macOS) |
| Hostname matches an entry in `inventory.yaml` | `EnrollClient` looks up the entity by slug derived from hostname; it must exist in `planned`/`provisioning` state. | `hostname` (Linux) / `scutil --get LocalHostName` (macOS) |
| OS is Linux or macOS | bootstrap detects via `uname -s` | `uname -s` |
| On the mesh (Netbird or Tailscale) **or** on the LAN | issuance is gated to mesh + LAN subnets. **For Netbird: use a setup-key, not interactive auth** — see "Getting onto Netbird" below. | `netbird status` / `tailscale status` |
| `git`, `python3`, `python3-yaml`, `age`, `sops` | bootstrap preflight; `homelab` CLI imports yaml | See per-OS commands below |
| Can resolve `*.hubris.network` | bootstrap calls `https://secrets.hubris.network/issue` and writes `https://mcp.hubris.network/mcp` | `dig +short mcp.hubris.network` (should return `192.168.8.175`) |
| On the mesh (Netbird) **or** on the LAN | enrollment validates mesh IP against expected subnets | `netbird status` |
| `curl`, `jq`, `age`, `python3` | bootstrap preflight (`bootstrap.sh:100`) — auto-installed on Fedora/RHEL/Debian/Ubuntu/macOS if missing | `command -v curl jq age python3` |
| Can resolve `*.hubris.network` | bootstrap calls the Oikos API and writes `https://mcp.hubris.network/mcp` | `dig +short mcp.hubris.network` |
### Hostname mismatch is the most common bootstrap failure
If the bootstrap exits with `no hosts/<name>.yaml in the repo`, the
hostname doesn't match any inventory entry. Two fixes:
If the entity for your hostname doesn't exist yet (in `planned` or
`provisioning` state), enrollment 4xxs. Two fixes:
- **Rename the host**: `sudo hostnamectl set-hostname <inventory-name>`
(Linux) or System Preferences → Sharing (macOS), then re-run.
- **Rename the inventory entry**: edit `inventory.yaml` on hubris,
update `inventory.yaml`, push. The next sync (≤5 min) propagates.
- **Rename the host** to match an existing planned entity:
`sudo hostnamectl set-hostname <inventory-name>` (Linux) or System
Preferences → Sharing (macOS), then re-run.
- **Add/rename the inventory entry**: edit `seeds/inventory.yaml`, ingest
via `oikos seed` (or the equivalent MCP/API entity-creation path), then
re-run bootstrap.
### Getting onto Netbird
### Networking prerequisites (Netbird, DNS, SSH key distribution)
Bootstrap auto-installs netbird and drives `netbird up` if the mesh isn't already connected (since commit `<bootstrap-tier1>`). Both paths below produce the same end state: `netbird status` shows `Management: Connected`, peer IP `100.122.x.x/16`.
**Path B — interactive OIDC (default; recommended):**
The new client runs bootstrap straight from a fresh OS. Bootstrap installs netbird (apt/dnf/brew based on the OS), then runs `netbird up --management-url https://netbird.hubris.network --ssh-jwt-cache-ttl 86400`. A device-code URL prints inline. The operator opens it (in a browser logged into Authentik), goes through identification → password → consent, and the CLI returns `Connected`. Bootstrap then proceeds with the rest of preflight.
Pre-condition: the operator must be a registered user in Authentik (typically the lab owner). The first user-login against a netbird account with existing peers is added as `pending_approval=1` and needs an sqlite promotion to `owner` — see [124-authentik.md First-time owner promotion gotcha](../../archive/knowledge/containers/106-auth-outpost.md). Only needed once per account.
**Path A — setup-key (headless/scripted onboarding):**
Useful for headless servers (no browser at all) or unattended cloud-init bootstraps.
1. From an already-enrolled machine, log into the dashboard at `https://netbird.hubris.network/`.
2. **Setup Keys** → Create → set reusable + expiry → copy.
3. On the new client (after installing netbird, OR let bootstrap install it and skip its `netbird up` driver):
```bash
sudo netbird up --setup-key <KEY> \
--management-url https://netbird.hubris.network \
--ssh-jwt-cache-ttl 86400
```
**Why we can't OIDC-login from the public internet (still open as a follow-up):**
`auth.hubris.network` resolves publicly to the VPS (`82.165.190.79`), but Traefik on the VPS doesn't currently route that hostname — only `netbird.hubris.network` is exposed. A brand-new client *off the mesh* hitting `auth.hubris.network` directly gets a Traefik default 404. In practice, Path B works fine because the operator's BROWSER (which clicks the device-code URL) is usually on a network that can reach Authentik through the public IONOS IP via some path. But "fresh laptop in a coffee shop with no prior session anywhere" still gets stuck. Future-session fix: add a Traefik route on the VPS forwarding `auth.hubris.network` via the netbird-routed `192.168.8.0/24` to LXC 124.
### DNS prerequisite
`*.hubris.network` resolves via the split-horizon dnsmasq on LXC 124
([dns.md](../../archive/knowledge/infrastructure/dns.md)) for LAN clients, **but only if the
client uses 192.168.8.180 as its resolver**. Most LXCs and roaming
workstations don't by default. Options:
- **LAN client**: set DNS to 192.168.8.180 (per-interface or
`/etc/resolv.conf`).
- **Off-LAN workstation on Netbird**: configure Netbird DNS forwarder to
point `*.hubris.network` at LXC 124.
- **Hack-fix anywhere**: append to `/etc/hosts`:
```
192.168.8.175 mcp.hubris.network secrets.hubris.network
192.168.8.175 git.hubris.network
```
(192.168.8.175 = caddy on LXC 121, terminates all `*.hubris.network`.)
If DNS isn't an option at all, override the URLs at bootstrap time:
```bash
sudo HOMELAB_GITEA_TOKEN=... \
HOMELAB_REPO_URL=http://192.168.8.121:3000/dtoro/oikos.git \
HOMELAB_ISSUANCE_NETBIRD=http://192.168.8.205:9820/issue \
HOMELAB_MCP_URL=http://192.168.8.205:9810/mcp \
bash /tmp/bootstrap.sh --with-mcp
```
## Install dependencies
Bootstrap auto-installs missing prerequisites (`git`, `python3` + PyYAML, `age`, `sops`, `netbird`) on Fedora/RHEL/Debian/Ubuntu/macOS — no manual `apt`/`dnf`/`brew` needed before running it. The only thing you must have on hand BEFORE the `curl ... | sudo bash` line is `curl` itself (used to pipe the script).
Manual install is still possible (e.g. for air-gapped or unusual platforms); the per-OS recipes are below for reference but optional.
<details>
<summary>Manual recipes (Fedora / Debian / macOS)</summary>
```bash
# Fedora / RHEL / Nobara
sudo dnf install -y git python3-pyyaml age curl
SOPS_VERSION=v3.9.4
sudo curl -fsSL https://github.com/getsops/sops/releases/download/$SOPS_VERSION/sops-$SOPS_VERSION.linux.amd64 \
-o /usr/local/bin/sops && sudo chmod +x /usr/local/bin/sops
# Debian / Ubuntu
sudo apt update && sudo apt install -y git python3-yaml age curl
SOPS_VERSION=v3.9.4
sudo curl -fsSL https://github.com/getsops/sops/releases/download/$SOPS_VERSION/sops-$SOPS_VERSION.linux.amd64 \
-o /usr/local/bin/sops && sudo chmod +x /usr/local/bin/sops
# macOS
brew install git age sops
pip3 install pyyaml # if `python3 -c "import yaml"` fails
```
</details>
## Run the bootstrap
You need a Gitea read-only personal access token for the initial clone
(the in-cluster shared PAT is encrypted at `secrets/gitea-readonly-pat.yaml`
but a new client can't decrypt it before bootstrap — chicken-and-egg).
Ask the operator (or generate in Gitea: Settings → Applications → Generate
New Token → scope `read:repository`).
```bash
TOKEN=... # your Gitea PAT, scope read:repository
# Fetch bootstrap.sh from gitea (HTTPS uses split-DNS → caddy).
curl -fsSL -u "dtoro:$TOKEN" \
https://git.hubris.network/dtoro/oikos/raw/branch/main/bootstrap.sh \
-o /tmp/bootstrap.sh
# Run it.
sudo HOMELAB_GITEA_TOKEN=$TOKEN bash /tmp/bootstrap.sh --with-mcp
```
Flags:
| Flag | Effect |
| --- | --- |
| `--with-mcp` | Merges the homelab MCP server into `~/.claude/.mcp.json` of the invoking user |
| `--no-secrets` | Skips age-key issuance (use when bringing up the first hosts before secrets-issuance exists) |
| `--dry-run` | Prints actions without executing |
The bootstrap is idempotent: re-running on an enrolled client just
verifies state, re-issues the age key only if it doesn't match the
inventory pubkey, and refreshes the sync timer + symlinks.
## Verify
```bash
homelab whoami # prints hosts/$(hostname).yaml
homelab list # shows the full topology
homelab status # ping + HTTP-check across hosts/services
homelab secret hello # decrypt the bootstrap-test secret
systemctl list-timers homelab-context-sync.timer
# next run within ≤5 min
```
For Claude Code: start a new session — the `homelab` MCP server appears
in `~/.claude/.mcp.json` and registers 14 tools (8 context, 5 management,
1 secrets-metadata).
## Post-bootstrap: SSH reachability
A new workstation must be reachable from other workstations and must be
able to reach every host by short hostname. Run these steps after the
bootstrap verify passes:
### 1. Enable SSH server
```bash
# macOS:
sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
# Linux:
sudo systemctl enable --now sshd
```
### 2. Generate SSH key (if missing)
```bash
ls ~/.ssh/id_ed25519.pub 2>/dev/null || ssh-keygen -t ed25519 -a 100
```
### 3. Publish pubkey to the repo
```bash
cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/$(hostname -s).pub
cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add $(hostname -s) pubkey' && git push
```
### 4. Deploy keys to all hosts
From any existing enrolled machine (hubris or another workstation):
```bash
ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
```
This adds the new workstation's pubkey to hubris and every running LXC.
### 5. Generate SSH config
```bash
homelab ssh-config --install
```
Verify:
```bash
ssh hubris hostname # should return "hubris" without password
ssh gitea hostname # should return "gitea" without password
ssh mac-mini hostname # should return "mac-mini" without password (workstation-to-workstation)
```
### 6. Add LAN IP to inventory (if on LAN)
If the workstation has a static or reserved LAN IP, add it to
`inventory.yaml`:
```yaml
hosts:
your-hostname:
lan_ip: 192.168.8.xxx
```
This gives it a primary LAN entry in the generated SSH config (faster
than the Netbird fallback). Commit + push, then:
```bash
cd /opt/homelab-context && git pull --ff-only && homelab ssh-config --install
```
Migrated to a runbook in the knowledge base — query
`search_knowledge("netbird mesh dns")` or `get_entity_knowledge`, or ask
Nomos. Covers: getting onto the Netbird mesh (interactive OIDC vs.
setup-key), why OIDC login can fail from off-mesh, split-horizon DNS
options, and distributing a new workstation's SSH pubkey across the fleet
via `ssh/deploy-keys.sh`.
## Claude Code permissions for fleet ops
By default Claude Code's auto-mode classifier asks for confirmation on every
ssh into the mesh. The bootstrap already installs the ssh ControlMaster block
so subsequent in-session sshes multiplex, but the *first* ssh of each session
still gets classifier-evaluated. Pre-authorize the common fleet ssh patterns
by adding to `~/.claude/settings.json`:
ssh into the mesh. Pre-authorize the common fleet ssh pattern by adding to
`~/.claude/settings.json`:
```json
{
"permissions": {
"defaultMode": "auto",
"allow": [
"Bash(ssh -p 22022 *)",
"Bash(homelab *)"
"Bash(ssh -p 22022 *)"
]
}
}
```
The first rule covers any ssh to a mesh peer on the homelab netbird port; the
second covers all `homelab` CLI invocations. Both are scoped tight enough that
the classifier doesn't gate them but loose enough to handle the variety of
arguments.
This covers any ssh to a mesh peer on the homelab netbird port, scoped tight
enough that the classifier doesn't gate it but loose enough to handle the
variety of arguments.
If you also want the netbird `--ssh-jwt-cache-ttl` flag rationale to be
visible to the classifier (it's not actually durable in 0.71.2, but the
ControlMaster block is — see [runbook-dpkg-interrupted](../skills/runbook-dpkg-interrupted/SKILL.md)
for context), drop a free-text rule into `autoMode.allow` describing the
authorization. Optional.
## Open questions (not verified against current architecture — don't
guess these from the old flow)
## Adding a new client to inventory
The old two-step ceremony had answers for these; the current one-shot
`/api/v1/clients/enroll` flow may handle them differently and this hasn't
been re-verified:
If the hostname you want isn't yet in inventory, enrollment is a two-step
ceremony driven from an existing enrolled client (e.g. hubris). The
`homelab` CLI handles steps 1 + 4; you provide steps 2 + 3.
```bash
# 1. On hubris (or any existing client): add the inventory entry.
homelab client add my-new-machine
# Prompts for kind, os, netbird FQDN, role. Commits + pushes.
# 2. Join the new machine to Netbird (out-of-band, Netbird console / setup key).
# 3. On the new machine: install deps + run bootstrap (above).
# Bootstrap calls /issue, receives a fresh age keypair, and prints the
# public key for the operator to commit back to inventory.
# 4. On hubris: finalize the age public key.
homelab client add my-new-machine --finalize-pubkey age1...
# Updates inventory.yaml hosts.my-new-machine.age_pubkey, regenerates
# inventory.yaml, commits + pushes. The 5-min sync propagates.
```
## Granting a secret to a new client
Adding a client doesn't grant them every secret. Recipients are explicit
per file via `.sops.yaml` glob rules. To grant a client access to (say)
`secrets/hello.yaml`:
1. Edit `.sops.yaml` at the repo root, add the client's `age_pubkey` to
the matching `creation_rules` block.
2. Re-key the existing ciphertext for the new recipient list:
```bash
sops updatekeys -y secrets/hello.yaml
```
3. Commit + push. On the next sync (≤5 min), the client can decrypt.
## Removing a client
```bash
# From any existing client:
homelab client remove my-old-machine
```
This:
1. Removes the inventory entry and `hosts/my-old-machine.yaml`.
2. Runs `sops updatekeys -y` against every file in `secrets/` (operator
must first remove the pubkey from `.sops.yaml` rules).
3. Calls `secrets-issuance` `/revoke` (admin-token-gated, on LXC 105) to
shred the key file and add the hostname to the denylist.
4. Commits + pushes.
The CLI prints a follow-up checklist that the operator must do manually:
- Revoke the peer in the Netbird console (denies future mesh access).
- **Rotate any credentials whose ciphertext the removed client already
has on disk.** The age key revocation only protects *future*
ciphertext; what's already been pulled is still decryptable until the
underlying credential changes.
- Optional: `homelab nuke my-old-machine` SSHes in, shreds
`/etc/age/key.txt`, removes `/opt/homelab-context`, disables sync.
- **Removing a client.** No current equivalent confirmed for the old
`homelab client remove` (inventory removal + secret re-keying + key
revocation). Likely maps to an entity lifecycle transition
(`.agents/skills/lifecycle-deprecate-node/` or `lifecycle-destroy-node/`)
but those skills reference the same dead CLI and need their own check.
- **Granting a secret to an already-enrolled client.** The old flow
hand-edited `.sops.yaml` `creation_rules` + `sops updatekeys`. Given
Infisical is now the primary secrets backend (SOPS is the DR fallback),
the current mechanism is probably Infisical-side, not a `.sops.yaml` edit
— not confirmed.
## Troubleshooting
| Symptom | Cause | Fix |
| --- | --- | --- |
| `no hosts/<hostname>.yaml in the repo` | Hostname doesn't match inventory entry | Rename either side (see above) |
| `fatal: could not read Username for 'http://192.168.8.121:3000'` | bootstrap.sh's credentials file has wrong scheme | Fixed in commit `de6f8be`; pull latest `bootstrap.sh` |
| `gnutls_handshake() failed: TLS connection was non-properly terminated` cloning `git.hubris.network` | Client DNS resolves `*.hubris.network` to the public VPS IP | Configure split-DNS (LXC 180 / Netbird forwarder) or `/etc/hosts` override; or use `HOMELAB_REPO_URL=http://192.168.8.121:3000/dtoro/oikos.git` |
| `TLS/SSL connection has been closed (EOF)` connecting MCP | Same — `mcp.hubris.network` resolves to public VPS without this vhost | Same DNS fix |
| `Invalid Host header` from MCP server | FastMCP's DNS-rebinding protection (default whitelist is 127.0.0.1 only) | Fixed in commit `6848640`; pull latest `mcp/server.py` and redeploy |
| `python3-yaml` install fails on Fedora | Wrong package name | Use `python3-pyyaml` (Fedora) instead of `python3-yaml` (Debian) |
| `address already in use` for FastMCP | FastMCP defaults to 127.0.0.1:8000 | Fixed: server now sets `mcp.settings.host/port` from env (default `0.0.0.0:9810`) |
| `homelab: no age key at /etc/age/key.txt` even after bootstrap | `/etc/age` is 0700 root, so non-root users couldn't even stat the key file; existence check returned False under regular users | Fixed in commit `df6aca8`: the CLI re-execs `sops -d` via sudo when invoked as a non-root user. On older deployments, re-link the CLI with `sudo ln -sfn /opt/homelab-context/bin/homelab /usr/local/bin/homelab` after the 5-min sync. |
| `homelab` CLI doesn't pick up repo updates | Pre-`02db…` bootstrap copied the binary instead of symlinking | One-time migration: `sudo ln -sfn /opt/homelab-context/bin/homelab /usr/local/bin/homelab`. New bootstraps use the symlink, which auto-tracks the synced repo. |
| `homelab-context-sync.service` journal shows `fatal: could not read Username for 'https://git.hubris.network'` | Pre-fix bootstrap set the gitea credential helper via `git config --global`, which writes to `/root/.gitconfig` — invisible to the systemd timer's git process (no HOME set). | One-time migration: `sudo git config --system credential.helper "store --file=/etc/homelab-context/git-credentials"`. New bootstraps store the helper in `/etc/gitconfig` instead. |
| Enrollment 404s / entity not found | Hostname doesn't match a `planned`/`provisioning` inventory entry | See "Hostname mismatch" above |
| `gnutls_handshake() failed` / TLS errors reaching `*.hubris.network` | Client DNS resolves `*.hubris.network` to the public VPS instead of the LAN/mesh path | See the networking runbook (split-horizon DNS section) |
| Chat-mode `!` shell can't `sudo` (`a terminal is required to read the password`) | Claude Code's `!` invocation doesn't allocate a tty, and standard `sudo` won't read its password from stdin or a non-tty pipe. | Run the sudo'd command in a real terminal outside chat. For commands the agent issues repeatedly, configure passwordless sudo for the narrow set (e.g. `/etc/sudoers.d/homelab-self` with `<user> ALL=(ALL) NOPASSWD: /usr/bin/dnf upgrade -y, /usr/bin/apt-get *`). |
| `netbird status -d` reports `192.168.8.180:53 ... is Unavailable` but DNS actually works | netbird's UDP-53 probe times out over the relay latency (~90ms), but actual queries still flow through systemd-resolved. Cosmetic. | Ignore unless `dig @192.168.8.180 git.hubris.network` also fails — then check dnsmasq on [LXC 124](../../archive/knowledge/containers/106-auth-outpost.md). |
| `netbird ssh` rejected with `JWT authentication failed: validate token (expected issuer=https://netbird.hubris.network/oauth2 ...)` | Peer's SSH JWT validator cached the OLD embedded-Dex issuer from before the 2026-05-21 Authentik migration. `systemctl restart netbird` and `netbird down/up` don't clear it — `client/internal/engine_ssh.go` bails out of `updateSSH()` if the SSH server is already running. | Full daemon bounce: `sudo systemctl stop netbird; sleep 3; sudo systemctl start netbird`. Verify with `grep -iE "issuer\|audience" /var/log/netbird/client.log \| tail`. Apply once per peer post-migration. |
| `netbird ssh` JWT passes but session closes with `user privilege check failed: user dtoro not found: unknown user dtoro` | netbird-ssh defaults the remote username to the LOCAL one (operator's laptop user). Hubris and LXCs only have `root`. | Always use explicit `root@` prefix manually: `netbird ssh -p 22022 root@proxmox-server.netbird.selfhosted`. `homelab ssh <host>` does this automatically via `inventory.yaml`'s per-host `ssh.user` field (defaults to `root`). |
| `homelab ssh hubris` (or any host on the LAN) fails with `Connection refused` or hangs, despite mesh routing being up | Off-LAN networks (operator on a VPN / coffee shop / symmetric NAT) sometimes can't reach the LAN IP even with the netbird subnet route. | Newer homelab CLIs probe the LAN with a 1.5s TCP connect and transparently fall back to the netbird FQDN. If your `/usr/local/bin/homelab` is a symlink to `/opt/homelab-context/bin/homelab` it'll pick up the fix on the next 5-min context sync. Otherwise pull the latest from gitea. |
## Changelog
### 2026-06-02 — SSH reachability post-bootstrap steps
Added a new "Post-bootstrap: SSH reachability" section covering SSH key
generation, pubkey publication, deployment to hosts, SSH config generation,
and LAN IP registration. New workstations enrolled via this doc will
automatically join the universal SSH mesh.
### 2026-05-31 — cross-link to nomos-agent.md
### 2026-07-12 — trimmed to current architecture
Removed everything describing the retired `homelab` CLI-based two-step
enrollment ceremony (now: `CLIENTS.md`'s one-shot flow), the Nous-Hermes/
Goose cross-link (that whole flow was removed the same day), and CLI-syntax
troubleshooting rows with no current equivalent. Migrated the still-true
Netbird/DNS/SSH-distribution content to a knowledge-base runbook rather
than duplicating it here. What's left is genuinely current or explicitly
flagged as unverified. Original ~365-line version is in git history
(`git log -- .agents/operations/agent-enrollment.md`) if any of the removed
detail turns out to still be needed.
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([nomos-agent.md](nomos-agent.md)) and noted it at the top of this page. The Nomos flow extends `bootstrap.sh` with `--with-nomos` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.
### 2026-06-02 — SSH reachability post-bootstrap steps
Added a section covering SSH key generation, pubkey publication,
deployment to hosts, SSH config generation, and LAN IP registration. New
workstations enrolled via this doc automatically join the SSH mesh.
(Superseded 2026-07-12 — migrated to the networking runbook.)
### 2026-05-31 — cross-link to nomos-agent.md
Added a sibling page covering Nous-Hermes-on-Goose enrollment. (Removed
2026-07-12 along with the rest of that flow.)
### 2026-05-21 — netbird-ssh JWT issuer + username + LAN-fallback troubleshooting rows
Added three rows to the troubleshooting table covering issues surfaced during the netbird vanilla migration: (1) post-migration SSH JWT validator cache stuck on old Dex issuer (full `systemctl stop/start` required, not `restart`), (2) `user not found` from netbird-ssh's local-username default (use explicit `root@`), and (3) homelab CLI's LAN→netbird-FQDN fallback for off-LAN operators. Companion code change: per-host `ssh.user` field in `inventory.yaml` + `homelab` CLI's `ssh_target()` helper.
Added three rows to the troubleshooting table covering issues surfaced
during the netbird vanilla migration. (Migrated 2026-07-12 to the
networking runbook.)
### 2026-05-20 — initial page
Captures the enrollment flow validated during Phase 2 of the homelab

View File

@@ -53,34 +53,36 @@ Run from the [hubris host](../../archive/knowledge/hosts/hubris.md) as root. Whe
## Fleet apt operations
Two `homelab` subcommands wrap the common patterns; both fan out to hubris + every LXC.
| Command | What it does |
| --- | --- |
| `homelab apt-audit [--target HOST]` | Per-host table: dpkg-interrupted state, holds, upgradable count, non-apt binaries in system paths, DNS health. Exits nonzero if any host has dpkg-interrupted state. |
| `homelab apt-upgrade --target HOST` | Launch `apt update && apt upgrade` inside a transient `systemd-run --collect` unit on the target. Survives ssh teardown. Apt configured with `Acquire::Retries=3` + `ForceIPv4=true`. |
| `homelab apt-upgrade --all` | Same, fanned out across the standard targets. |
| `homelab apt-upgrade ... --status` | Show running unit + tail `/var/log/homelab-apt-upgrade.log` on each target. |
| `homelab apt-upgrade ... --safe` | Take a pre-upgrade snapshot per LXC first (`pct snapshot``vzdump` fallback for bind-mounted LXCs). Refuses if any snapshot fails unless `--force`. |
| `homelab apt-upgrade ... --force` | Skip both the dpkg-audit gate and snapshot-failure refusal. |
PVE/kernel deferral on hubris: `homelab apt-upgrade --target hubris` will try every upgrade, including kernel + `pve-*`. To skip those, `apt-mark hold` the relevant packages on hubris first; `homelab apt-audit` shows held packages so you can confirm.
**No current CLI equivalent.** `homelab apt-audit`/`apt-upgrade` (dpkg-state
audit, fanned-out apt upgrade with pre-upgrade snapshots) were part of the
retired Python `homelab` CLI and don't have a ported replacement — apt
patching today is ad hoc `run` MCP tool calls per host, without the
audit/snapshot/status wrapping this used to provide. If that wrapping is
still wanted, it needs to be rebuilt (e.g. as a runbook driving `run`, or a
new MCP tool) — see
[runbook-dpkg-interrupted](../skills/runbook-dpkg-interrupted/SKILL.md) for
the dpkg-interrupted recovery procedure specifically.
## Oikos (agent OS layer)
See [OIKOS.md](../OIKOS.md) for the operating model. Quick reference:
See [OIKOS.md](../OIKOS.md) for the operating model. The `homelab` CLI this
section used to document is retired; the actual current interface is the
33 MCP tools cataloged in [AGENTS.md](../../AGENTS.md#3-the-mcp-server) plus
the REST API. Closest current equivalents for what used to live here:
| Command | What it does |
| Old `homelab` command | Current equivalent |
| --- | --- |
| `homelab service <name> explain\|health\|docs\|log\|actions\|history` | Service Console v0 — context card, cached health (`--live` to force a probe), docs, logs, safe actions + risk class, ledger history |
| `homelab node <name> relations` | Ontology blast-radius query: what this host/service impacts, is affected by, and its full transitive blast radius |
| `homelab change preflight <service>` | Dry-run report before mutating: risk class, current health, config repo, verification command |
| `homelab decide <action> <entity>` | Decision classifier: risk × blast radius × confidence → auto-act or escalate |
| `homelab signal list\|raise\|ack\|resolve\|mute` | The attention layer — pending updates, thresholds, drift, anything needing attention |
| `homelab approval request\|list\|reply\|check` | Escalate-route grants (Matrix-delivered via Nomos, or the Oikos Console's `/approvals` page) |
| `homelab restart <service> [--approval-id <id>]` | `--approval-id` is required whenever the service's risk class needs approval (e.g. `caddy`, `dns`) — refuses mechanically without a valid grant |
| `homelab service <name> explain\|health\|docs\|log` | MCP `explain`, `get_service_status`, `tail_log`, `get_entity_knowledge` |
| `homelab node <name> relations` | MCP `get_blast_radius` |
| `homelab change preflight <service>` | MCP `preflight` |
| `homelab signal list\|ack\|resolve\|mute` | MCP `get_signal_history`, or REST `POST /api/v1/signals/{id}/ack\|resolve\|mute` (the control-room UI's Signals page wraps these) |
| `homelab approval request\|list\|reply\|check` | REST `GET/POST /api/v1/approvals*` (Matrix-delivered via the notifier, or the control-room UI's Operations page) |
| `homelab restart <service> --approval-id <id>` | MCP `run` (policy-gated — auto-executes if read-only/reversible_low, otherwise queues for the same Matrix/UI approval) |
| `homelab decide <action> <entity>` | No direct equivalent — classification now happens inline inside `run`/`request_execution`, not as a separate dry-run call |
Oikos Console (read-mostly dashboard): `oikos.hubris.network` once deployed — see [oikos/console/deploy/README.md](../../archive/oikos-cards/).
There is no separately-deployed "Oikos Console" anymore — the control-room
SPA (`web/`) is the operator dashboard, served standalone (see
[plans/2026-07-12-wails-desktop-app.md](../../plans/2026-07-12-wails-desktop-app.md)).
## Related
- [Hubris host](../../archive/knowledge/hosts/hubris.md)

View File

@@ -1,210 +0,0 @@
# Nomos agent — LLM-powered terminal sessions on a homelab client
Onboards [Nous Research's Hermes](https://nousresearch.com/) (a fine-tuned
Llama variant) as a working terminal agent on a homelab client. Builds on top
of standard client enrollment (see [agent-enrollment.md](agent-enrollment.md))
— this page covers only the Hermes-specific additions.
The agent runs as a [Goose](https://goose-docs.ai/) session. Goose provides:
- The chat loop, multi-turn history, and streaming
- The OpenRouter provider that routes to the configured LLM
- The built-in `developer` extension (shell + file editor — same surface Claude
Code has)
- A remote MCP extension pointed at `mcp.hubris.network` for read-only
homelab context (`list_lxcs`, `tail_log`, `search_docs`, etc.)
The persona is `/opt/homelab-context/NOMOS.md`, symlinked as Goose's global
`.goosehints` so it's injected into the system prompt on every session.
## Prerequisites
| Requirement | How |
| --- | --- |
| Standard enrollment complete (`homelab whoami` works) | [agent-enrollment.md](agent-enrollment.md) |
| `secrets/openrouter-api-key.yaml` exists with a real `sk-or-...` value | See "Seeding the OpenRouter key" below |
| The host's `age_pubkey` is on the openrouter-api-key.yaml sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-nomos` |
## Onboarding flow
```bash
# 1. On hubris (or any enrolled client): reserve the inventory entry.
homelab client add new-machine
# 2. Join new-machine to Netbird (setup-key or OIDC).
# 3. On new-machine: bootstrap with --with-nomos.
TOKEN=... # gitea PAT, read:repository
curl -fsSL -u "dtoro:$TOKEN" \
https://git.hubris.network/dtoro/oikos/raw/branch/main/bootstrap.sh \
-o /tmp/bootstrap.sh
sudo HOMELAB_GITEA_TOKEN=$TOKEN bash /tmp/bootstrap.sh --with-mcp --with-nomos
# 4. Back on hubris: finalize the age pubkey AND grant the Nomos secret.
homelab client add new-machine \
--finalize-pubkey age1... \
--with-nomos
# 5. Wait ≤5 min for sync, then on new-machine:
nomos "what LXCs are running?"
```
The bootstrap `--with-nomos` flag does five things, all idempotent:
1. Downloads the latest Goose binary into the operator's `~/.local/bin/goose`
(upstream installer) and symlinks `/usr/local/bin/goose` to it.
2. Symlinks `/opt/homelab-context/bin/nomos``/usr/local/bin/nomos`.
3. Symlinks `/opt/homelab-context/NOMOS.md``/root/NOMOS.md` (Linux) or
`/etc/NOMOS.md` (macOS) for `cat`-as-operator convenience.
4. Drops `~/.config/goose/config.yaml` pinning the provider, model, and
extensions (preserves any keys the operator added by hand).
5. Symlinks `~/.config/goose/.goosehints` → NOMOS.md, so the persona is
injected as the system prompt on every session.
## Seeding the OpenRouter key
The first time anyone enrolls with `--with-nomos`, the encrypted file
`secrets/openrouter-api-key.yaml` contains a placeholder. On hubris (or any
existing recipient):
```bash
sops secrets/openrouter-api-key.yaml
# editor opens; replace api_key value with the real sk-or-... key, save, close.
git -C /opt/homelab-context add secrets/openrouter-api-key.yaml
git -C /opt/homelab-context commit -m 'openrouter-api-key: seed real key'
git -C /opt/homelab-context push
```
Until this step happens, `nomos …` exits with `openrouter-api-key.yaml still
contains the placeholder`. Subsequent enrollees get the real key automatically
via `--with-nomos` (which adds them as a sops recipient on
`secrets/openrouter-api-key.yaml`).
## Granting the OpenRouter key to an already-enrolled host
If a host was enrolled without `--with-nomos` and you want to add it later:
```bash
# On hubris:
PUBKEY=$(homelab whoami --hostname <host> | grep age_pubkey | awk '{print $2}')
homelab client add <host> --finalize-pubkey "$PUBKEY" --with-nomos
```
`--finalize-pubkey` is required by the existing flow even when the pubkey is
unchanged — it's also the trigger that runs the sops grant.
After ≤5 min sync the host can decrypt the key. Bootstrap doesn't need to
re-run; only the secret recipient list changed.
## Verifying
```bash
homelab whoami # standard enrollment OK
homelab secret openrouter-api-key | head -c 8 # decrypts (prints `api_key:`)
which goose && which nomos # binaries present
goose info -v # provider/model wiring sane
nomos "what LXCs are running?" # interactive Goose session
# Non-interactive smoke test:
echo "List the homelab MCP tools you have available" | nomos
```
## Configuration
The bootstrap-managed keys in `~/.config/goose/config.yaml`:
```yaml
GOOSE_PROVIDER: openrouter
GOOSE_MODEL: deepseek/deepseek-v4-flash
GOOSE_MODE: smart_approve # asks before destructive tool calls
extensions:
developer:
type: builtin
bundled: true
enabled: true
name: developer
timeout: 300
homelab:
type: streamable_http
enabled: true
name: homelab
uri: https://mcp.hubris.network/mcp
timeout: 60
```
Override via env on a single bootstrap run:
```bash
HOMELAB_NOMOS_MODEL=nousresearch/hermes-3-llama-3.1-405b \
HOMELAB_NOMOS_MCP_URI=https://mcp.hubris.network/mcp \
sudo bash /tmp/bootstrap.sh --with-nomos
```
Any keys you add by hand (e.g. `GOOSE_TEMPERATURE`, extra `extensions.*`) are
preserved across re-bootstraps — the merge only overwrites the keys it manages.
## Tool permissions
`GOOSE_MODE: smart_approve` is the bootstrap default: Goose runs read-only
shell commands without prompting and asks for confirmation before destructive
ones. To make the agent fully unattended (e.g. for scheduled jobs), set
`GOOSE_MODE: auto` in `~/.config/goose/config.yaml`. To require confirmation on
every tool call, use `approve`. See
[goose-permissions](https://goose-docs.ai/docs/guides/managing-tools/goose-permissions/).
## Troubleshooting
| Symptom | Cause | Fix |
| --- | --- | --- |
| `nomos: could not decrypt secrets/openrouter-api-key.yaml` | Host isn't a recipient on the sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-nomos` from hubris |
| `nomos: openrouter-api-key.yaml still contains the placeholder` | No real key has been seeded yet | See "Seeding the OpenRouter key" above |
| Goose hangs on first `nomos` invocation with no output | Goose's interactive `configure` ran on first launch and is awaiting input | Re-run; the installer is supposed to skip it (CONFIGURE=false). If it persists, run `goose configure` once manually in a real terminal to commit the config. |
| `homelab` extension fails to connect / no MCP tools listed | MCP server upgraded in Go rewrite (`internal/mcp/server.go`, Streamable HTTP via official MCP SDK). Old FastMCP SSE transport is deprecated. | Run `docker compose --profile full up` on mac-mini, or wait for the production cutover from apps/105. |
| `goose: command not found` after bootstrap | Upstream installer dropped binary in `~/.local/bin/` but `/usr/local/bin/goose` symlink didn't land | Re-run bootstrap with `--with-nomos`; the symlink step is at the end of the install block. If still missing, `ln -sfn ~/.local/bin/goose /usr/local/bin/goose` manually. |
| Tool calls hit OpenRouter rate limits | One shared key across many hosts | Future: per-host keys; for now, see the rate-limits guide referenced in `goose info -v`. |
## Cross-references
- [agent-enrollment.md](agent-enrollment.md) — base client onboarding the
Nomos flow assumes is done.
- [`NOMOS.md`](../NOMOS.md) — the persona the Nomos agent reads on every
session start (via `~/.config/goose/.goosehints`).
- [`bin/nomos`](../../bin/nomos) — the wrapper that decrypts the OpenRouter key
and execs `goose session`.
- [`bootstrap.sh`](../../bootstrap.sh) — the `--with-nomos` flag's install block.
## Follow-ups
1. **Migrate the MCP server to streamable_http.** Goose 1.x deprecated SSE
(`"SSE transport is no longer supported - kept only for config file
compatibility"` in `crates/goose/src/agents/extension.rs`). Our FastMCP
server at `internal/mcp/server.go` uses Streamable HTTP (official MCP SDK). Until
that's changed, the `homelab` MCP extension in Goose will fail to connect.
The developer extension (shell + edit) covers most ops without it; this is
a polish item, not a blocker.
2. **Per-host OpenRouter keys** for billing attribution. Today all Nomos
hosts share one key.
3. **Pin the model version** rather than tracking `nousresearch/hermes-4-405b`
directly — OpenRouter periodically rotates the underlying weights.
4. **Local-inference fallback** (ollama / vllm) once the homelab has a GPU
node. The wrapper, persona, and MCP wiring stay unchanged; only
`GOOSE_PROVIDER`/`GOOSE_MODEL` change.
7. **Caveman auto-setup via post-pull hook.** The sync timer now calls
`tools/post-pull.sh`, which runs any `tools/*.setup.sh` after git pull.
Currently this auto-installs the Caveman npm package, wrapper scripts, and
compact output templates on all agent hosts (*token efficiency*).
## Changelog
### 2026-06-01 — caveman + post-pull auto-setup
Added `tools/post-pull.sh` sync hook that auto-runs `tools/*.setup.sh`
after every git pull. First user: `tools/setup-caveman.sh` installed Caveman
templating + `~/bin/caveman_wrapper.sh` + `~/templates/*.txt` for token-
efficient CLI output. Replaces raw `git pull` in launchd/systemd timers.
Also created `tools/caveman/` with the wrapper script, JS renderer, and
templates — the canonical source for all agent hosts.
Captures the Nomos-on-Goose onboarding flow added in the same commit as
`bootstrap.sh --with-nomos`, `bin/nomos`, the sops rule for
`secrets/openrouter-api-key.yaml`, and the `homelab client add --with-nomos`
extension. MCP streamable_http migration is queued as follow-up #1.

View File

@@ -2,39 +2,46 @@
name: client-enrollment
risk_class: config_mutation
inputs: [hostname, kind, role]
verification: "homelab doctor (on the new client)"
verification: "MCP whoami(hostname) shows the entity active"
docs_update_checklist: [hosts_narrative_page_if_lxc_or_vm]
---
# Client enrollment
Goal: bring a new host (workstation, LXC, VM) into inventory and the
secrets model, with mesh membership only where it's actually needed.
This wraps the existing `homelab client add` flow — see
[operations/agent-enrollment.md](../../operations/agent-enrollment.md) for
the full walkthrough; this runbook is the risk/lifecycle framing.
secrets model, with mesh membership only where it's actually needed. See
[CLIENTS.md](../../../CLIENTS.md#enrollment) for the actual current
flow and [operations/agent-enrollment.md](../../operations/agent-enrollment.md)
for operational notes; this runbook is the risk/lifecycle framing.
1. On any enrolled client: `homelab client add <hostname>` — appends a
`hosts.<name>:` block to `inventory.yaml` (lifecycle `state: planned`
`provisioning`, per [seeds/ontology.yaml](../../../seeds/ontology.yaml)),
commits + pushes.
1. The entity must exist in `planned`/`provisioning` state before the new
host can self-enroll — add a `hosts.<name>:` block to
`seeds/inventory.yaml` and `oikos seed` to ingest it (lifecycle
`planned``provisioning`, per
[seeds/ontology.yaml](../../../seeds/ontology.yaml)).
2. Netbird join is **optional, not a required step** — only needed for
hosts that must be reachable off-LAN (workstations that roam, e.g.
`republic-laptop`, `mac-mini`). A node reachable on the household LAN
(192.168.8.0/24 — most LXCs/VMs) doesn't need it: it's already
reachable directly, and off-LAN clients reach it too via hubris's
routed `192.168.8.0/24` Netbird network resource. Skip this step for
LAN-only nodes; do it (out-of-band, console or setup key) only for
hosts that need independent off-LAN reachability.
3. On the new host: run `bootstrap.sh` (add `--with-nomos` to also
enroll the Hermes agent). This provisions `/etc/age/key.txt`, the
sync timer, and prints an age pubkey.
4. Back on an enrolled client: `homelab client add <hostname>
--finalize-pubkey <age1...>` — sets `age_pubkey`, grants shared
secrets, re-keys SOPS, commits + pushes. This is the
`provisioning → active` transition.
5. Verify: `homelab doctor` on the new client should show all checks
green (clone, sync timer, age key, CLI symlink, MCP reachable).
`mac-mini`). A node reachable on the household LAN (192.168.8.0/24 —
most LXCs/VMs) doesn't need it. Skip for LAN-only nodes; do it
(out-of-band, console or setup key) only for hosts that need
independent off-LAN reachability.
3. On the new host: run `bootstrap.sh`. This calls
`POST /api/v1/clients/enroll`, which validates the entity exists and
the mesh IP is in an expected subnet, then returns an age keypair and
Infisical machine identity in one response — provisions
`/etc/age/key.txt`, `/etc/infisical/identity`, and the context poller.
4. **Known gap, confirmed 2026-07-12: `provisioning → active` has no
working path.** `EnrollClient` (`internal/httpapi/impl.go`) sets the
entity's state to `provisioning`, never `active`. `bootstrap.sh` prints
`POST /api/v1/clients/ws:$HNAME/activate` as the next step, but that
route doesn't exist — `api/openapi.yaml` only has `/clients/enroll`,
`/clients/{slug}/context`, `/clients/{slug}/secrets`. Until this is
fixed (add the route, or use the generic entity PATCH to flip `state`),
a freshly-enrolled client is stuck in `provisioning` — MCP `preflight`
and policy's `lifecycle_overrides` for `provisioning` still apply, but
nothing transitions it onward automatically.
5. Verify: MCP `whoami(hostname)` shows the entity in `active` state with
its peers and health.
Docs-update checklist: if the new host is an LXC/VM, add its narrative
page under `containers/` or `vms/` and set `doc_page` in its inventory

View File

@@ -2,7 +2,7 @@
name: config-change-deploy
risk_class: config_mutation
inputs: [service_name, change_description]
verification: "curl -sf <service_url> (or homelab service <name> health)"
verification: "curl -sf <service_url> (or MCP get_service_status)"
docs_update_checklist: [doc_page, changelog]
---
@@ -11,11 +11,10 @@ docs_update_checklist: [doc_page, changelog]
Goal: change a tracked config repo (Caddy, Gitea customizations, an app's
own repo) and get it live, safely.
1. `homelab change preflight <service>` — current health, the service's
`config_repo`, its risk class, and the verification command to run
after. If risk class requires approval (`config_mutation` or
`destructive`), stop and get operator sign-off before editing — see
`seeds/policy.yaml`.
1. MCP `preflight` — current health, the service's `config_repo`, its
risk class, and the verification command to run after. If risk class
requires approval (`config_mutation` or `destructive`), stop and get
operator sign-off before editing — see `seeds/policy.yaml`.
2. Clone/pull the `config_repo` (never edit the backend's working tree
directly — tracked configs change by commit + push, per
[OIKOS.md](../../OIKOS.md) conventions).
@@ -24,10 +23,10 @@ own repo) and get it live, safely.
[infrastructure/auto-deploy.md](../../../archive/knowledge/infrastructure/auto-deploy.md) for
the exact receiver/reload for this service).
5. Run the preflight's verification command. If it fails, check
`homelab service <name> log` for the reload/restart error.
6. Record the change: once `oikos/ledger.py` is wired into deploy tooling
(Week 3), this is automatic; until then, note the change and outcome
in the relevant investigation/plan doc.
MCP `tail_log` for the reload/restart error.
6. No manual record-keeping step needed — mutations made through the API
(e.g. via the `run` MCP tool) are recorded automatically in the
`audit_log` table.
Docs-update checklist: update the service's `doc_page` if the change
alters its behavior, ingress route, or ownership; add a changelog entry

View File

@@ -10,22 +10,23 @@ docs_update_checklist: [investigations_entry]
Goal: understand what broke and why, before touching anything.
1. `homelab service <name> explain` (or `homelab node <name> relations`
if the affected entity is a host) — get the blast radius and doc
pointer first. Don't start pulling logs blind.
2. `homelab service <name> health` + `homelab service <name> log` (or
MCP `get_service_status` / `tail_log`) for the affected service.
1. MCP `explain` (or `get_blast_radius` if the affected entity is a
host) — get the blast radius and doc pointer first. Don't start
pulling logs blind.
2. MCP `get_service_status` + `tail_log` for the affected service.
3. Walk the blast radius: is a shared dependency down (`caddy`, `dns`,
`authentik`, or the backend host itself)? `homelab node <name>
relations` shows "affected by" — check those first.
4. `homelab apt-audit` if the symptom looks like a dpkg/upgrade
interaction.
`authentik`, or the backend host itself)? MCP `get_blast_radius`
shows "affected by" — check those first.
4. If the symptom looks like a dpkg/upgrade interaction, see
[runbook-dpkg-interrupted](../runbook-dpkg-interrupted/SKILL.md) —
there's no fleet-wide apt-audit tool anymore, check the host directly.
5. Check the change ledger for recent mutations to the affected entity
or anything upstream of it: `homelab service <name> history` (once
populated) or grep `ledger/*.jsonl`.
6. Write findings to a new `knowledge/sources/investigations/<date>-<slug>.md` — symptom,
timeline, root cause, fix applied, prevention. This is the durable
record; don't rely on chat history.
or anything upstream of it: MCP `get_change_history` or `get_audit_trail`.
6. Write findings via MCP `upsert_knowledge` (`kind: investigation`) —
symptom, timeline, root cause, fix applied, prevention, `about` set to
the affected entity's slug. The DB is the durable record now, not a
markdown file — `search_knowledge`/`get_entity_knowledge` read it back;
a chat message alone is forgotten.
Docs-update checklist: always create the investigation entry. If the
root cause was stale/wrong inventory data (a `doc_page`, `config_repo`,

View File

@@ -2,7 +2,7 @@
name: lifecycle-activate-node
risk_class: config_mutation
inputs: [node_name]
verification: "homelab service <name> health (if it hosts a service); homelab doctor (if it's a client)"
verification: "MCP get_service_status (if it hosts a service); MCP whoami (if it's a client)"
docs_update_checklist: [doc_page_complete]
transition: "provisioning -> active"
---
@@ -14,23 +14,22 @@ enrolled if it needs secrets, mesh joined if it needs off-LAN reach,
ingress live if public, health check answering, doc page complete,
ledger entry.
1. If the node is a `homelab` client: finish enrollment per
[client-enrollment.md](../client-enrollment/SKILL.md) (`--finalize-pubkey`,
mesh join, `homelab doctor` green).
1. If the node self-enrolls as a client: finish enrollment per
[CLIENTS.md](../../../CLIENTS.md#enrollment) (`bootstrap.sh`
`/api/v1/clients/enroll`, mesh join, MCP `whoami` returns the entity).
2. If it hosts a public service: add the `services:` entry in
`inventory.yaml` (backend, url, doc_page, config_repo, risk_notes —
see the Week-1 service contract fields) and wire the Caddy route in
`dtoro/caddy-conf`.
3. Confirm the health check answers: `homelab service <name> health` or
a direct `curl`.
`seeds/inventory.yaml` (backend, url, doc_page, config_repo,
risk_notes) and wire the Caddy route in `dtoro/caddy-conf`.
3. Confirm the health check answers: MCP `get_service_status` or a
direct `curl`.
4. Flip `state: provisioning``state: active` (or delete the `state:`
field — `active` is the default) in `inventory.yaml`.
field — `active` is the default) in `seeds/inventory.yaml`, then
`oikos seed` to ingest.
5. Complete the doc page (stub → full narrative: role, specs, how it's
configured, dependencies).
6. Record the activation: `oikos/ledger.py append host:<name> activate
config_mutation --result ok` (or let the CLI wrapper do this once
Week 3's runbook automation lands).
6. No manual record-keeping step needed — the activation (via whatever
API call flipped the state) is recorded automatically in `audit_log`.
Regenerate derived data: `python3 mcp/build_host_files.py && python3
inventory.yaml` so `inventory.yaml`, the topology diagram, and
the context card all reflect the new state.
Regenerate: `oikos seed` re-ingests `seeds/inventory.yaml`; `oikos export`
writes DB state back out to the YAML if you mutated via the API/MCP
instead of editing the file directly.

View File

@@ -2,7 +2,7 @@
name: lifecycle-deprecate-node
risk_class: config_mutation
inputs: [node_name, replacement_node_or_reason]
verification: "homelab node <name> relations — 'affected by' must be empty before completing"
verification: "MCP get_blast_radius — 'affected by' must be empty before completing"
docs_update_checklist: [doc_page_deprecation_note]
transition: "active -> deprecated"
---
@@ -16,14 +16,14 @@ suggestion; `seeds/policy.yaml` `lifecycle_overrides.deprecated.refuse`
lists `new-inbound-edges` as refused going forward.
1. Set `state: deprecated` on the node.
2. `homelab node <name> relations` — read `affected_by`. Every entry
there is something still relying on this node.
2. MCP `get_blast_radius` — read `affected_by`. Every entry there is
something still relying on this node.
3. Migrate or retire each dependent one at a time (point its `backend`/
`config_repo`/ingress route elsewhere, or deprecate it too if it's
being retired alongside).
4. Re-run `homelab node <name> relations` after each dependent is moved.
The transition to `destroyed` is only safe once `affected_by` is
empty — check this every time, don't assume from memory.
4. Re-run MCP `get_blast_radius` after each dependent is moved. The
transition to `destroyed` is only safe once `affected_by` is empty —
check this every time, don't assume from memory.
5. Note the deprecation on the doc page: reason, replacement (if any),
date.

View File

@@ -2,7 +2,7 @@
name: lifecycle-destroy-node
risk_class: destructive
inputs: [node_name]
verification: "homelab node <name> relations returns unknown-entity; pct list on the backend no longer shows it"
verification: "MCP get_blast_radius returns unknown-entity; pct list on the backend no longer shows it"
docs_update_checklist: [archaeology_entry, containers_index_update]
transition: "deprecated -> destroyed"
---
@@ -15,28 +15,35 @@ recipients removed + re-keyed, ingress/DNS removed, archaeology entry,
ledger entry.
1. Confirm the node is `deprecated` with zero `affected_by` edges
(`homelab node <name> relations`) — do not skip this even if the
deprecation runbook was followed recently; state can drift.
2. If it's an enrolled client: `homelab client remove <name>` — revokes
the age key, re-keys SOPS, removes the inventory entry. This is
already destructive-class and confirmed in the CLI.
(MCP `get_blast_radius`) — do not skip this even if the deprecation
runbook was followed recently; state can drift.
2. **If it's an enrolled client: no current tool for revoking its age key /
removing its Infisical identity.** The old `homelab client remove`
(age key revocation + SOPS re-key + inventory removal, all one
destructive-class CLI call) is retired along with the rest of that CLI
and hasn't been re-verified against the current enrollment
architecture (`POST /api/v1/clients/enroll` + Infisical machine
identities) — see the "Open questions" section in
[agent-enrollment.md](../../operations/agent-enrollment.md). Until
that's confirmed, treat key/identity revocation as a manual step:
at minimum remove the client's `age_pubkey` from any SOPS recipient
lists and rotate credentials whose ciphertext it already decrypted.
3. Remove any ingress route (Caddy config repo) and DNS record still
pointing at it.
4. Verify backups of anything on it are retained per policy before the
disk goes away (see `backs-up-to`).
5. Destroy the LXC/VM (`pct destroy` / `qm destroy`).
6. Move the `hosts.<name>:` block (if any inventory remnant survives
`client remove`, e.g. infra-only LXCs with no age key) into
inventory.yaml's `archaeology:` section: `pve_id`, `destroyed` date,
`reason`. Add a row to `containers/index.md` "Recently destroyed"
table (kept for human-readable browsing alongside the structured
data).
7. `oikos/ledger.py append host:<name> destroy destructive --result ok`.
8. Regenerate: `python3 mcp/build_host_files.py && python3
inventory.yaml` — the node drops out of `inventory.yaml` and
appears in the topology doc's archaeology table.
6. Update the entity's `state` to `destroyed` in `seeds/inventory.yaml`
(or move it to an `archaeology:`-style section if the schema still has
one) — `pve_id`, `destroyed` date, `reason` — then `oikos seed` to
ingest. Add a row to `containers/index.md` "Recently destroyed" table
(kept for human-readable browsing alongside the structured data).
7. No manual ledger step — mutations through the API are recorded
automatically in the `audit_log` table (MCP `get_audit_trail`,
`get_change_history`). The old `oikos/ledger.py append` was retired
when this became automatic.
If the destroy fails partway (e.g. secrets revoked but pct destroy
errors), do not re-run step 2 — `client remove` is not idempotent
against a second revocation attempt on the issuance server. Finish the
remaining steps manually and note the partial state in an investigation.
If the destroy fails partway (e.g. secrets not fully revoked but pct
destroy errors), finish the remaining steps manually and note the
partial state in an investigation (MCP `upsert_knowledge`,
`kind: investigation`).

View File

@@ -2,7 +2,7 @@
name: lifecycle-migrate-node
risk_class: config_mutation
inputs: [node_name, source_host, target_host]
verification: "homelab node <name> relations (re-check blast radius); homelab service <svc> health for every hosted service"
verification: "MCP get_blast_radius (re-check blast radius); MCP get_service_status for every hosted service"
docs_update_checklist: [doc_page_migration_note, inventory_host_and_lan_ip]
transition: "active -> migrating -> active"
---
@@ -10,12 +10,12 @@ transition: "active -> migrating -> active"
# Lifecycle: migrate a node
Modeled on the strong Phase 1+2 migration
([plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)).
([archive/hermes-plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md](../../../archive/hermes-plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)).
Requires (ontology): preflight + backup-verified before migrating;
post-verify + Caddy backends checked + mounts checked + docs updated
before returning to `active`.
1. `homelab change preflight <every service the node hosts>` — capture
1. MCP `preflight` for every service the node hosts — capture
current health as a baseline.
2. Verify backups are current for anything with data at rest on the
node (see `backs-up-to` edges once populated).
@@ -29,11 +29,12 @@ before returning to `active`.
6. Post-verify: re-run the Week-1 drift check by hand — confirm Caddy's
backend IP for each affected service matches the new `lan_ip`
(automatic in Week 3's drift detector), confirm mounts still resolve.
7. `homelab service <name> health` for every service the node hosts.
7. MCP `get_service_status` for every service the node hosts.
8. Set `state: active`. Add a migration note to the node's doc page
(old host/IP → new, date, phase reference) — this repo's convention
for every past migration (see `archive/knowledge/containers/101-jellyfin.md`,
`containers/129-house.md`).
Regenerate: `python3 mcp/build_host_files.py && python3
inventory.yaml`.
Regenerate: `oikos seed` (re-ingests `seeds/inventory.yaml` into the DB —
the DB is authoritative at runtime, the YAML is the source of truth
on disk).

View File

@@ -21,10 +21,12 @@ chosen, doc page stub.
`qm create`), choosing the storage pool deliberately — record it as
the `storage:` field once populated (Week 1 schema; not yet backfilled
for existing nodes).
2. Add the inventory entry: `homelab client add <name>` for anything that
will run the `homelab` CLI, or a direct `hosts.<name>:` block with
`state: provisioning`, `kind`, `host`, `pve_id`, `lan_ip` for
infra-only LXCs that won't self-enroll.
2. Add the inventory entry: a `hosts.<name>:` block in
`seeds/inventory.yaml` with `state: provisioning`, `kind`, `host`,
`pve_id`, `lan_ip`, then `oikos seed` to ingest it. For anything that
will self-enroll as a client afterward (see
[CLIENTS.md](../../../CLIENTS.md#enrollment)), the entity must exist in
`planned`/`provisioning` state before `bootstrap.sh` runs there.
3. Stub the doc page (`containers/<pve_id>-<name>.md` or
`vms/<pve_id>-<name>.md`) — even a one-line "provisioning, see plan X"
is enough to satisfy the transition requirement.

View File

@@ -18,7 +18,7 @@ summarised into targets and fixed costs.
- `yuvomi-mcp` is running on LXC 129 and connected as an MCP server in Claude.
- The CSV is an N26 export (columns: Booking Date, Value Date, Partner Name,
Partner Iban, Type, Payment Reference, Account Name, Amount (EUR), …).
- API token: `homelab secret yuvomi-api-token` (decrypts on any enrolled client).
- API token: `yuvomi-api-token`, via Infisical (primary) or `oikos secret` (SOPS fallback).
- Direct API base: `https://house.hubris.network/api/v1`
---

View File

@@ -13,7 +13,8 @@ has packages that are **unpacked but not configured**. Symptoms:
manually run 'dpkg --configure -a' to correct the problem.`
- `dpkg --audit` lists packages with header
`The following packages have been unpacked but not yet configured.`
- `homelab apt-audit` shows `DPKG: DIRTY(N)` for the host.
- `dpkg --audit` on the host directly shows unpacked-not-configured packages
(there's no fleet-wide audit tool anymore — check per-host).
The system is still running the **old** binaries (still in memory), but the
**new** binaries are unpacked and waiting for their postinst to run. Two
@@ -33,19 +34,20 @@ config dirs, capabilities, etc.). The system might not come back up cleanly.
## Path A — target is still reachable over ssh (preferred)
```
homelab ssh <host> -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
ssh <host> -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
```
Or for an LXC by name:
Or for an LXC by name (via the MCP `run` tool, or directly on the Proxmox
host):
```
homelab pct <lxc> exec -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
pct exec <lxc> -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
```
When that returns, confirm:
```
homelab apt-audit --target <host>
ssh <host> -- dpkg --audit
```
Expect `DPKG: ok` and the remaining `UPGR` count to match what's intentionally
@@ -89,10 +91,11 @@ DEBIAN_FRONTEND=noninteractive dpkg --configure -a \
## Prevention
The `homelab apt-upgrade` wrapper launches apt inside a `systemd-run --collect`
unit on the target, so it survives ssh teardown — the failure mode that put
hubris into this state in the first place is no longer reachable through the
standard tool. If you absolutely need to run apt manually over ssh, wrap it:
The old `homelab apt-upgrade` wrapper (retired along with the rest of the
`homelab` CLI) used to launch apt inside a `systemd-run --collect` unit on
the target so it survived ssh teardown — that's the failure mode that put
hubris into this state in the first place. There's no fleet-wide wrapper
anymore; if you run apt manually over ssh, wrap it yourself the same way:
```
ssh <host> systemd-run --unit=apt-recovery --collect bash -c 'apt -y upgrade'

View File

@@ -2,7 +2,7 @@
name: service-health-check
risk_class: read_only
inputs: [service_name]
verification: "homelab service <name> health"
verification: "MCP get_service_status"
docs_update_checklist: []
---
@@ -10,15 +10,15 @@ docs_update_checklist: []
Goal: determine whether a service is actually healthy, without ad-hoc SSH.
1. `homelab service <name> explain` — read the context card: backend,
blast radius, doc pointer, risk notes.
2. `homelab service <name> health` — live health probe (HTTP code against
the service's `url`/`endpoint`). Once the Week-3 scheduler ships, this
reads a cached snapshot by default; pass `--live` to force a fresh probe.
3. If unhealthy, `homelab service <name> log` (or MCP `tail_log`) for the
last 200 lines.
4. Cross-check blast radius: `homelab node <name> relations` — is this
entity's own backend host healthy? A downstream failure (e.g. `strong`
1. MCP `explain` — read the context card: backend, blast radius, doc
pointer, risk notes.
2. MCP `get_service_status` — live health probe (HTTP code against the
service's `url`/`endpoint`); the scheduler also probes on its own
interval, so this may reflect a recent cached result, not necessarily
a fresh one.
3. If unhealthy, `tail_log` for the last 200 lines.
4. Cross-check blast radius: MCP `get_blast_radius` — is this entity's
own backend host healthy? A downstream failure (e.g. a Proxmox host
down) will show up here before the service's own logs explain anything.
5. If the fix is a restart: classify first (`seeds/policy.yaml`
`service-restart` is `reversible_low` unless the service has a

8
.gitignore vendored
View File

@@ -2,20 +2,12 @@
__pycache__/
*.pyc
# Regenerated every scheduler run; ephemeral health-probe cache.
oikos/state.json
# Compiled binaries (Go rewrite — bin/oikos, bin/nomos)
bin/oikos
bin/nomos
oikos/oikos
oikos
webhook
# Legacy Python oikos (superseded by cmd/oikos Go binary — Phase 1-6 rewrite).
# oikos/ kernel files are still imported by bin/homelab for operational CLI
# commands (ssh, pct, logs, restart, status, open, secret, client, sync, mcp).
# Remove oikos/* when bin/homelab is ported to Go.
backups/
.env
.infisical-credentials

104
AGENTS.md
View File

@@ -34,7 +34,8 @@ Run `hostname` (Linux) or `scutil --get LocalHostName` (macOS), then read:
That file tells you your role, your peers, what's mounted, and what services
you host. If it does not exist, this client was not enrolled — stop and tell
the operator to run `homelab client add <hostname>` from an existing client.
the operator; see [CLIENTS.md](CLIENTS.md#enrollment) for the enrollment flow
(the entity needs to exist in `planned`/`provisioning` state first).
## 2. The topology
@@ -49,17 +50,24 @@ the operator to run `homelab client add <hostname>` from an existing client.
## 3. The MCP server
The homelab exposes a Model Context Protocol server with structured tools.
Endpoint: `https://mcp.hubris.network/mcp`.
Endpoint: `https://mcp.hubris.network/mcp`. Every call needs
`Authorization: Bearer <token>` — the API has no unauthenticated path except
enrollment and `/healthz` (see "Authentication" below for where the token
comes from).
Available tools (21 total):
Available tools (33 total):
Context — observe + orient:
get_entity(slug), list_entities(type, limit, cursor),
get_relations(entity), get_blast_radius(entity),
search_knowledge(query) — ILIKE search over documents, investigations,
runbooks in the knowledge_entities table
get_entity_knowledge(entity_slug) — every document, investigation, and
runbook linked to one entity, in one call
get_patterns(status, entity_type, action) — learned action patterns
get_skills(status) — available automation skills
http_get(url) — fetch a public page/raw file (e.g. researching how to
deploy something before provisioning it); HTTP/HTTPS only, ~16KB cap
Management — live state:
get_service_status(service_slug) — systemctl is-active on target host
@@ -67,6 +75,8 @@ Available tools (21 total):
list_lxcs() — all LXC containers with ID, host, IP, health
get_lxc_state(lxc_slug) — pct status from Proxmox host
ping_service(service_slug) — HTTP reachability from entity_status
list_my_secrets(caller_pubkey) — secrets accessible to this client by
age public key
Oikos — decisions:
explain(service_slug) — compact context card (type, state, health, relations)
@@ -84,11 +94,27 @@ Available tools (21 total):
get_trend(entity_id, days=7) — metric slope over time
get_event_timeline(severity, entity_slug, limit) — recent events
Execution — the single mutation path:
request_execution(target, action, params) — policy-gated.
reversible_low (restart, reload, pct_exec, apt audit) runs immediately;
config_mutation (systemctl enable/disable, apt upgrade) queues for operator
approval via Matrix, then executes on ✅.
Knowledge — keep the graph current (none require approval; this updates
the knowledge graph, not live infrastructure):
upsert_knowledge(title, content) — record what you learned after solving
a non-obvious problem; the only way anything persists past a session
update_entity_attributes(slug, attributes) — merge a discovered fact
(IP, version, port, ...) into an entity so a future task doesn't
rediscover it from scratch
create_relationship(source, target, type) — record a discovered edge
(depends-on, hosts, routes-to, ...) between two entities
Execution — mutating the live infrastructure:
run(target, command) — the general execution primitive. Run any shell
command against a host or LXC; every command is auto-classified —
read-only inspection runs immediately, anything state-changing needs
operator approval, and destructive patterns (rm -rf, dd, mkfs,
pct/qm destroy, DROP TABLE, reboot, curl-pipe-to-shell, ...) always
need approval regardless of what you declare. Prefer this over
request_execution for anything not already covered by its fixed enum.
request_execution(target, action, params) — the older, fixed-enum path
(restart, systemctl, pct_exec, apt_upgrade, pct_create). Still the
route for those specific actions; policy-gated the same way `run` is.
get_execution_status(execution_id) — poll progress
**When to prefer MCP over grepping the clone:** always for knowledge queries.
@@ -97,7 +123,18 @@ the DB with entity links. `get_entity_knowledge("lxc:jellyfin")` returns documen
runbooks, and investigations in one call. Grep the clone only when MCP is
unreachable.
## 4. Knowledge conventions
## 4. Authentication
Every API/MCP route requires `Authorization: Bearer <token>` except
`POST /api/v1/clients/enroll` and `/healthz`. Enrollment (see
[CLIENTS.md](CLIENTS.md#enrollment)) does not currently issue a per-client
API/MCP bearer token — there is one shared
secret (`OIKOS_MCP_BEARER_TOKEN`, validated in `internal/httpapi/server.go`'s
`combinedAuth`); get it from the operator until per-client token issuance
exists. The SPA has its own flow instead: a first-launch Config screen that
stores a token in `localStorage` (see `web/src/pages/Config.svelte`).
## 5. Knowledge conventions
All narrative knowledge (documents, investigations, runbooks) lives in the DB
(`knowledge_entities` table) and is seeded from `seeds/knowledge.yaml`. Agents
@@ -122,47 +159,52 @@ per the DB-as-source-of-truth plan.
running state, update the DB *in the same session* via the API. The `oikos export`
command regenerates `seeds/knowledge.yaml` for version control.
## 5. Acting on the homelab
## 6. Acting on the homelab
- **Read state**: use MCP tools. Nomos (the AI agent) is the primary
operator interface — it has 21 MCP tools for observe/orient/decide/act.
- **Actions** (restart, logs, apt, pct exec): Nomos calls `request_execution`
via MCP. `reversible_low` actions execute immediately; `config_mutation`
and `destructive` actions are queued for operator approval via Matrix.
- **Secrets**: managed by Infisical (`oikos secret` subcommand for migration).
Never hardcode secrets — use env vars from `.env`.
operator interface — it has 33 MCP tools for observe/orient/decide/act
(§3).
- **Actions** (restart, logs, apt, pct exec, or anything else): Nomos calls
`run` (the general execution primitive) or `request_execution` (the older
fixed-enum path) via MCP. `reversible_low`/read-only actions execute
immediately; `config_mutation` and `destructive` actions are queued for
operator approval via Matrix or the control-room UI's Operations page.
- **Secrets**: managed by Infisical (`oikos secret` subcommand for
migration). Never hardcode secrets — use env vars from `.env`.
- **Mutations** (restart, edit configs, etc.): classified against
`seeds/policy.yaml`. `reversible_low` actions auto-execute;
`config_mutation`/`destructive` actions require approval.
a valid `--approval-id` from `homelab approval request` — see OIKOS.md.
`config_mutation`/`destructive` actions require approval — granted by
the operator via Matrix reply or the control-room UI, not a CLI flag.
See OIKOS.md.
## 6. Communication mode
## 7. Communication mode
Read and apply `/opt/homelab-context/.agents/shared/caveman.md` (if present). It defines the lab's
terse-communication standard — drop filler, keep substance, use fragments.
## 7. Auto-setup mechanism
## 8. Auto-setup mechanism
The homelab-context repo ships tooling that gets automatically installed
on every client after `git pull`. This is handled by `tools/post-pull.sh`
(replaces the raw git pull in the sync timer) which runs any script matching
`tools/*.setup.sh` after pull.
`tools/setup-*.sh` after pull.
Currently auto-setup:
- **Caveman + templates** (`tools/setup-caveman.sh`): Installs Caveman npm
package, wrapper scripts, and compact output templates for token-efficient
CLI output. Wrapper at `~/bin/caveman_wrapper.sh`.
- **Nomos agent persona** (`tools/setup-nomos-soul.sh`): Provisions
`~/.nomos/SOUL.md` from `NOMOS.md` on Nomos agents. This ensures every
Nomos agent follows the canonical homelab persona (token efficiency, source
of truth hierarchy). No-op on non-Nomos agents.
- **Host checks** (`tools/setup-checks.sh`): Deploys `checks/install.sh`'s
health-check scripts to `/opt/oikos/checks` on each host. The scheduler's
`ssh-script` check kind depends on these actually being there — 20 are
live in the DB as of 2026-07-12.
To add a new auto-setup, create `tools/<name>.setup.sh` in the repo,
To add a new auto-setup, create `tools/setup-<name>.sh` in the repo,
commit and push. All enrolled clients pick it up within 5 minutes.
To trigger sync manually: `sudo homelab sync` or wait for the 5-min timer.
To trigger sync manually: run `/opt/homelab/tools/context-poller.sh`, or
wait for the 5-min timer. (This mechanism — and the server-side
`tools_changed` detection behind it — only correctly recognized
`setup-*.sh` scripts as of 2026-07-12; before that it silently matched
nothing, so nothing auto-ran on any client via this path.)
## 8. When in doubt
## 9. When in doubt
Use MCP tools: `search_knowledge <query>` for narrative context,
`get_entity <slug>` for structured data, `get_entity_knowledge <slug>` for

View File

@@ -24,9 +24,22 @@ Docker stack on mac-mini and exposes an MCP server + REST API.
| State snapshot (health, disk, drift) | MCP `get_state_snapshot` |
| Secrets (Infisical) | REST API + `oikos secret` CLI |
| Approval tokens | Matrix via notifier |
| Run a command on a host/LXC (policy-gated) | MCP `run` |
| Record a discovered fact/relationship | MCP `update_entity_attributes`, `create_relationship`, `upsert_knowledge` |
All MCP tools are read-only. Mutations use the `homelab` CLI with operator
approval.
Most MCP tools are read-only; a few mutate the knowledge graph (recording
what you learned) or the live infrastructure (`run`, `request_execution`),
gated by risk classification and — for `config_mutation`/`destructive`
actions — operator approval. See [AGENTS.md](AGENTS.md#3-the-mcp-server) for
the full tool catalog.
## Authentication
Every API/MCP call needs `Authorization: Bearer <token>` — there is no
unauthenticated path except `POST /api/v1/clients/enroll` and `/healthz`.
Enrollment (below) does not currently hand out a per-client bearer token;
get the shared `OIKOS_MCP_BEARER_TOKEN` from the operator until per-client
token issuance exists.
## Enrollment
@@ -41,7 +54,6 @@ curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh | sudo b
# Or with optional tooling:
curl ... | sudo bash -s -- --with-mcp # wire Claude's MCP config
curl ... | sudo bash -s -- --with-nomos # install Goose + Nomos
```
This calls `POST /api/v1/clients/enroll` on the Oikos API, which:
@@ -56,7 +68,7 @@ This calls `POST /api/v1/clients/enroll` on the Oikos API, which:
### What changes on your machine
- `/opt/homelab/` — agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md)
- `/opt/homelab/tools/` — tooling scripts (caveman, nomos-soul)
- `/opt/homelab/tools/` — tooling scripts (checks)
- `/etc/age/key.txt` — age private key for SOPS decryption (fallback)
- `/etc/infisical/identity` — Infisical machine identity (primary secrets)
- Context poller — launchd/systemd timer hits `GET /api/v1/clients/{slug}/context` every 5 minutes for agent file updates

View File

@@ -35,6 +35,7 @@ cd web && OIKOS_API_TOKEN=dev-token npm run dev
```
cmd/oikos/ Single-binary entry point
cmd/nomos/ Nomos MCP client gateway
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
internal/ All Go packages
httpapi/ REST + MCP server (OpenAPI-generated)
mcp/ MCP tool implementations
@@ -55,10 +56,13 @@ migrations/ Forward-only SQL migrations (TimescaleDB)
seeds/ Bootstrap YAML: ontology, inventory, policy, knowledge
compose/ Dockerfiles + Caddy config
scripts/ Deploy, watchdog, rollback
checks/ Host health-check scripts run over SSH by the scheduler
tools/ Client auto-setup scripts (checks)
nomos/ Nomos config, persona, skills
.agents/ Agent instruction files + skills
plans/ Design documents
docs/adr/ Architecture decision records
docs/operations/ Runbooks (rollback, etc.)
```
## Commands
@@ -78,6 +82,8 @@ docs/adr/ Architecture decision records
| `make clean` | Remove binary + test cache |
| `make ui` | Build the SPA (`web/dist/`) |
| `make deploy-ui` | Build + deploy the SPA to the Caddy host |
| `make webhook` | Build `cmd/webhook` (deploy-webhook receiver) |
| `make tidy` | `go mod tidy` |
## Conventions

View File

@@ -65,7 +65,7 @@ cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
| 5 — Secrets | ✅ | Infisical backend + SOPS fallback, rotation runbooks |
| 6 — Deploy | ✅ | CI pipeline, cutover checklist, watchdog, rollback |
Full plan: [plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md).
Full plan: [plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md).
## Operations
@@ -120,6 +120,7 @@ output). A native desktop wrapper is planned — see
```
cmd/oikos/ Go entry point — single binary
cmd/nomos/ Nomos MCP client gateway
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
internal/ Go packages (httpapi, mcp, scheduler, actuator, learning,
notifier, policy, secrets, db, config, ontology, domain,
knowledge)
@@ -129,11 +130,16 @@ migrations/ Forward-only SQL migrations (TimescaleDB)
seeds/ Bootstrap YAML (ontology, inventory, policy, knowledge)
compose/ Dockerfiles + Caddy config
scripts/ Deploy, watchdog, verification, rollback
checks/ Host health-check scripts run over SSH by the scheduler
tools/ Client auto-setup scripts (checks)
ssh/ Deploy keys + authorized_keys management
vps/ Caddy/TURN config templates for the netbird VPS
nomos/ Nomos config, persona, skills
.agents/ Agent instruction files, shared conventions, skills
archive/ Historical reference (legacy wiki, plans, SOPS backups)
plans/ Design documents (active + done)
docs/adr/ Architecture decision records
docs/operations/ Runbooks (rollback, etc.)
```
## For agents

View File

@@ -3,15 +3,14 @@
#
# Thin client model (rev 2): no git clone, no sync timer. Fetches only the
# agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md) and tooling
# (caveman, nomos-soul) from the raw Gitea URL. Enrolls via the Oikos API
# to receive an age keypair and Infisical machine identity. A lightweight
# context poller replaces the old 5-minute git pull.
# (checks) from the raw Gitea URL. Enrolls via the Oikos API to receive an
# age keypair and Infisical machine identity. A lightweight context poller
# replaces the old 5-minute git pull.
#
# Usage:
# curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh \
# | sudo bash
# curl ... | sudo bash -s -- --with-mcp # wire Claude's .mcp.json
# curl ... | sudo bash -s -- --with-nomos # install Goose + Nomos
# curl ... | sudo bash -s -- --dry-run # show what would happen
#
# Prerequisites:
@@ -28,16 +27,10 @@ REPO_RAW_URL="${HOMELAB_RAW_URL:-https://git.hubris.network/dtoro/oikos/raw/main
OIKOS_API_URL="${HOMELAB_OIKOS_URL:-https://oikos.hubris.network/api/v1}"
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
MCP_URL="${HOMELAB_MCP_URL:-https://mcp.hubris.network/mcp}"
NOMOS_MCP_URI="${HOMELAB_NOMOS_MCP_URI:-https://mcp.hubris.network/mcp}"
NOMOS_MODEL="${HOMELAB_NOMOS_MODEL:-nousresearch/hermes-4-405b}"
WITH_MCP=0
WITH_NOMOS=0
DRY_RUN=0
GITEA_TOKEN="${HOMELAB_GITEA_TOKEN:-}"
GITEA_USER="${HOMELAB_GITEA_USER:-dtoro}"
# ── helpers ──────────────────────────────────────────────────────────
log() { echo "[oikos] $*"; }
@@ -79,10 +72,7 @@ detect_mesh_ip() {
while [ $# -gt 0 ]; do
case "$1" in
--with-mcp) WITH_MCP=1 ;;
--with-nomos) WITH_NOMOS=1 ;;
--dry-run) DRY_RUN=1 ;;
--gitea-token) GITEA_TOKEN="$2"; shift ;;
--gitea-user) GITEA_USER="$2"; shift ;;
*) die "unknown flag: $1" ;;
esac
shift
@@ -148,7 +138,7 @@ done
# ── fetch tools ──────────────────────────────────────────────────────
log "fetching tools..."
for tool in setup-caveman.sh setup-nomos-soul.sh caveman.js caveman_wrapper.sh post-pull.sh; do
for tool in setup-checks.sh post-pull.sh; do
url="$REPO_RAW_URL/tools/${tool}"
dest="$CLONE_DIR/tools/${tool}"
dry mkdir -p "$(dirname "$dest")"
@@ -161,16 +151,6 @@ for tool in setup-caveman.sh setup-nomos-soul.sh caveman.js caveman_wrapper.sh p
fi
done
# ── fetch caveman templates ──────────────────────────────────────────
for tmpl in git_log.txt git_status.txt test_results.txt; do
url="$REPO_RAW_URL/tools/caveman/templates/${tmpl}"
dest="$CLONE_DIR/tools/caveman/templates/${tmpl}"
dry mkdir -p "$(dirname "$dest")"
if curl -fsSL --connect-timeout 10 "$url" -o "$dest.tmp" 2>/dev/null; then
mv "$dest.tmp" "$dest"
fi
done
# ── detect control-plane (use localhost if API is reachable directly) ─
if [ -z "${HOMELAB_OIKOS_URL:-}" ]; then
if curl -s --connect-timeout 2 http://localhost:8090/api/v1/health >/dev/null 2>&1; then
@@ -300,7 +280,7 @@ case "$OS" in
esac
# ── run auto-setup scripts ───────────────────────────────────────────
for setup in "$CLONE_DIR"/tools/*.setup.sh; do
for setup in "$CLONE_DIR"/tools/setup-*.sh; do
[ -f "$setup" ] || continue
log "running setup: $(basename "$setup")"
dry bash "$setup"
@@ -319,17 +299,6 @@ if [ "$WITH_MCP" -eq 1 ]; then
log " + MCP wired to $MCP_URL"
fi
# ── --with-nomos: install Goose + Nomos wrapper ────────────────────
if [ "$WITH_NOMOS" -eq 1 ]; then
log "installing Nomos agent..."
GOOSE_URL="https://github.com/block/goose/releases/latest/download/goose-${OS}-${ARCH:-amd64}"
if [ "$OS" = Darwin ]; then GOOSE_URL="https://github.com/block/goose/releases/latest/download/goose-darwin-${ARCH:-arm64}"; fi
dry curl -fsSL "$GOOSE_URL" -o /usr/local/bin/goose 2>/dev/null && chmod +x /usr/local/bin/goose || warn "goose not installed"
# Drop Nomos persona
cp "$CLONE_DIR/NOMOS.md" "$CLONE_DIR/.agents/NOMOS.md" 2>/dev/null || true
log " + Nomos agent installed"
fi
# ── netbird SSH JWT cache ────────────────────────────────────────────
if command -v netbird >/dev/null 2>&1; then
dry netbird up --management-url https://netbird.hubris.network --ssh-jwt-cache-ttl 86400 2>/dev/null || true

View File

@@ -1,4 +1,7 @@
# Signal Trigger Architecture
# ADR 0013 — Signal trigger architecture
**Status:** Accepted
**Date:** 2026-07-08
## Overview

View File

@@ -1,6 +1,6 @@
# Oikos Entity Model — Types, Relationships & Interactions
# ADR 0014 — Entity model: types, relationships & interactions
**Status:** Adopted
**Status:** Accepted
**Date:** 2026-07-08
**Scope:** Full inventory of every entity type, relationship, state machine, and
cognition pipeline — with clear markers for what is **code-real** vs **schema-only**.

View File

@@ -0,0 +1,62 @@
# ADR 0015 — Bearer-token auth for every route + client/server split
Status: accepted (2026-07-12) · Plan: plans/2026-07-12-wails-desktop-app.md, Phase 0
## Context
The control-room SPA was embedded in the `oikos` binary (`go:embed`,
ADR 0001) and served at `/ui/*`. `combinedAuth` (`internal/httpapi/server.go`)
opened a dev-open bypass — no credential required at all — whenever
`OIKOS_ENV=dev` and no static token/OIDC issuer was configured. That was
true not just in local dev but in the actual mac-mini production deploy:
`docker-compose.yml`'s `api` service hardcoded `OIKOS_ENV: dev` with no
token set, so every route (`/api/v1/*`, `/mcp`, and an `/agent` reverse-proxy
mount to nomos that had never been wrapped in `combinedAuth` at all) was
reachable unauthenticated from anywhere on the mesh/LAN. A planned Wails
desktop client and any future non-browser client can't rely on same-origin
requests or a dev-open bypass; they need the SPA to be a standalone,
CORS-capable client that authenticates over HTTP like any other caller.
## Decision
- Delete the SPA embed (`web/embed.go`, the `/ui/*` routes). `web/` is a
standalone static build, deployed separately (`make ui` / `make
deploy-ui`), served at `/` by Caddy with SPA fallback.
- Remove the dev-open bypass entirely. Every route requires a valid
static bearer token (`OIKOS_API_TOKEN` / `OIKOS_MCP_BEARER_TOKEN`) or an
OIDC JWT, with two narrow exceptions: `/healthz` (liveness) and
`POST /api/v1/clients/enroll` (IP-gated in the handler instead).
`GET /api/v1/events/stream` additionally accepts the token as a
`?token=` query param, since `EventSource` can't set custom headers.
- Add CORS (`github.com/go-chi/cors`, `OIKOS_CORS_ORIGIN`, default `*`) so a
cross-origin SPA (Vite dev server, a future Wails webview) can reach the
API. No `AllowCredentials` — auth is a header, not a cookie, so
credentialed CORS mode isn't needed and the two don't combine safely with
a wildcard origin.
- Wrap the previously-unauthenticated `/agent` proxy mount in the same
`combinedAuth` middleware as every other route.
- `cmd/nomos` becomes an authenticated client of `api`: it now sends
`Authorization: Bearer $OIKOS_MCP_BEARER_TOKEN` on its own outbound calls
(MCP + the chat-assent approval-decision endpoint), which it never did
before — dev-open covered for it until now.
- The SPA gets a runtime config module (`web/src/lib/config.ts`) and a
first-launch `Config.svelte` screen: server URL + token, stored in
`localStorage`, injected into every `fetch()` via a shared
`fetchWithAuth` wrapper. Resolved fresh per request (not cached at
module-load time), so the same build works same-origin or cross-origin
without a rebuild.
## Consequences
- Closing dev-open was a live security fix, not just future-proofing —
verified post-deploy that unauthenticated requests to production now 401.
- Nomos's *own* HTTP gateway (`cmd/nomos`, port 8092) still has no auth of
its own — out of scope here, tracked separately
(plans/2026-07-11-nomos-agent-code-review.md, finding C1).
- Production Caddy (`dtoro/caddy-conf`, not this repo) does not yet expose
`oikos.hubris.network` at all, so the interaction between Authentik
forward-auth and bearer-token clients (a non-browser client can't
complete a browser SSO redirect) is unresolved — needs an `@enroll`-style
bypass for `/api/v1/*`/`/mcp`/`/agent/*` before public exposure. This
repo's `compose/caddy/Caddyfile.oikos` (a reference copy, not deployed
from here) has the bypass; the real config does not yet.
- There is one shared bearer secret for all agents/clients, not per-client
tokens — acceptable for the current fleet size, revisit if per-client
revocation becomes necessary.

View File

@@ -5,7 +5,7 @@ after acceptance — superseding decisions get a new ADR that links back.
Statuses: proposed | accepted | superseded-by-NNNN.
| ADR | Title |
|---|---|---|
|---|---|
| [0001](0001-go-single-binary.md) | Go with single-binary role packaging |
| [0002](0002-postgres-timescale-only-datastore.md) | PostgreSQL + TimescaleDB as the only datastore |
| [0003](0003-db-native-ontology-yaml-seeds.md) | DB-native ontology with YAML seed manifests |
@@ -16,7 +16,8 @@ Statuses: proposed | accepted | superseded-by-NNNN.
| [0008](0008-forward-only-migrations.md) | Forward-only migrations |
| [0009](0009-sse-over-websocket.md) | SSE over WebSocket for the event stream |
| [0010](0010-infisical-with-sops-fallback.md) | Infisical secrets with SOPS DR fallback |
| [0011](0011-client-lifecycle-flows.md) | Client lifecycle flows — enrollment, bootstrap, sync |
| [0012](0012-hermes-oikos-interactions.md) | HermesOikos interactions — agent/OS contract |
| [0013](0013-signal-triggers.md) | Signal triggers — host health checks via scheduler |
| [0011](0011-client-lifecycle-flows.md) | Client lifecycle sequence diagrams |
| [0012](0012-hermes-oikos-interactions.md) | Hermes/Oikos interaction architecture |
| [0013](0013-signal-triggers.md) | Signal trigger architecture |
| [0014](0014-entity-model.md) | Entity model — types, relationships, state machines, OODA loop |
| [0015](0015-api-bearer-auth-client-server-split.md) | Bearer-token auth for every route + client/server split |

View File

@@ -1290,7 +1290,10 @@ func (s *Server) GetClientContext(ctx context.Context, req gen.GetClientContextR
for rows.Next() {
var p string
if scanErr := rows.Scan(&p); scanErr == nil {
if strings.HasPrefix(p, "tools/") && strings.HasSuffix(p, ".setup.sh") {
// Matches tools/setup-*.sh (the auto-setup convention —
// see tools/post-pull.sh). Was tools/*.setup.sh until
// 2026-07-12, which never matched any real filename.
if strings.HasPrefix(p, "tools/setup-") && strings.HasSuffix(p, ".sh") {
toolsChanged = append(toolsChanged, p)
} else if p == ".sops.yaml" {
sopsChanged = true

View File

@@ -1,237 +0,0 @@
#!/usr/bin/env python3
"""
Generate knowledge/wiki/infrastructure/topology.md (Mermaid views) and per-entity context
cards from inventory.yaml.
Views:
1. Compute & ingress — hypervisors → guests → services → public URLs
2. Storage — mounts and pools per guest
Context cards (oikos/cards/<name>.md): one compact (~30-line) file per
host and service — identity, ontology edges, safe actions + risk class,
doc pointer, recent ledger history. This is the token-efficiency layer:
an agent orienting on an entity reads one card instead of several
search_docs/get_page round-trips.
Run from the repo root:
python3 oikos/gen-topology.py # writes topology.md + cards/
python3 oikos/gen-topology.py --check # exit 1 if output would change
Wired into the same regeneration path as mcp/build_host_files.py so the
diagrams and cards never drift from inventory. Edges follow
oikos/ontology.yaml (hosts, provides, routes-to, mounts, stores-on).
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
try:
import yaml
except ImportError: # pragma: no cover
print("PyYAML is required: pip install pyyaml", file=sys.stderr)
sys.exit(2)
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO))
from oikos import gen_topology_lib as lib # noqa: E402
from oikos import ledger as oikos_ledger # noqa: E402
from oikos import policy as oikos_policy # noqa: E402
from oikos import relations as oikos_relations # noqa: E402
INVENTORY = REPO / "inventory.yaml"
OUTPUT = REPO / "knowledge" / "wiki" / "infrastructure" / "topology.md"
CARDS_DIR = REPO / "oikos" / "cards"
BANNER = (
"<!-- Generated by oikos/gen-topology.py from inventory.yaml. -->\n"
"<!-- Do NOT edit by hand - your changes will be overwritten. -->\n"
)
# View/graph logic lives in oikos/gen_topology_lib.py (importable — this
# file's hyphenated name can't be). Re-exported here so existing call
# sites in this module don't need a rename.
node_id = lib.node_id
guest_label = lib.guest_label
compute_view = lib.compute_view
storage_view = lib.storage_view
archaeology_table = lib.archaeology_table
def _host_card(name: str, entry: dict, inv: dict) -> str:
lines = [f"# {name} (host:{name})\n"]
tag = "LXC" if entry.get("kind") == "lxc" else "VM" if entry.get("kind") == "vm" else entry.get("kind", "")
pve = entry.get("pve_id")
lines.append(f"- kind: {entry.get('kind', '?')}" + (f" ({tag} {pve})" if pve else ""))
lines.append(f"- state: {entry.get('state', 'active')}")
if entry.get("host"):
lines.append(f"- runs-on: host:{entry['host']}")
if entry.get("role"):
lines.append(f"- role: {entry['role']}")
addr = entry.get("lan_ip", "")
mesh = entry.get("mesh", {})
mesh_bits = []
for m, v in mesh.items():
if isinstance(v, dict) and (v.get("ip") or v.get("fqdn")):
mesh_bits.append(f"{m}:{v.get('fqdn') or v.get('ip')}")
if addr or mesh_bits:
lines.append(f"- address: {addr}" + (f" (mesh: {', '.join(mesh_bits)})" if mesh_bits else ""))
if entry.get("mounts"):
lines.append(f"- mounts: {', '.join(entry['mounts'])}")
doc = None
if entry.get("kind") == "lxc" and pve:
cand = REPO / "knowledge" / "wiki" / "containers" / f"{pve}-{name}.md"
if cand.exists():
doc = str(cand.relative_to(REPO))
elif entry.get("kind") == "vm" and pve:
cand = REPO / "knowledge" / "wiki" / "vms" / f"{pve}-{name}.md"
if cand.exists():
doc = str(cand.relative_to(REPO))
elif entry.get("kind") == "proxmox-host":
cand = REPO / "knowledge" / "wiki" / "hosts" / f"{name}.md"
if cand.exists():
doc = str(cand.relative_to(REPO))
if doc:
lines.append(f"- doc: {doc}")
if entry.get("age_pubkey"):
lines.append("- secrets: enrolled (age key present)")
rel = oikos_relations.relations(f"host:{name}", inv)
lines.append("\n## Blast radius")
lines.append(f"- impacts: {', '.join(rel['impacts']) or '(none)'}")
lines.append(f"- affected by: {', '.join(rel['affected_by']) or '(none)'}")
if rel["blast_radius"]:
lines.append(f"- full blast radius: {', '.join(rel['blast_radius'])}")
lines.append("\n## Safe actions")
lines.append("- see the services this host runs for action-level risk classes")
hist = oikos_ledger.history(f"host:{name}", limit=5)
lines.append("\n## Recent changes")
if hist:
for h in hist:
lines.append(f"- {h.get('ts', '?')} {h.get('action', '?')} ({h.get('risk', '?')}) — {h.get('result', '?')}")
else:
lines.append("- (none yet)")
return "\n".join(lines) + "\n"
def _service_card(name: str, entry: dict, inv: dict) -> str:
lines = [f"# {name} (service:{name})\n"]
if entry.get("backend"):
lines.append(f"- backend: host:{entry['backend']}")
url = entry.get("url") or entry.get("endpoint")
if url:
lines.append(f"- url: {url}")
if entry.get("doc_page"):
lines.append(f"- doc: {entry['doc_page']}")
if entry.get("config_repo"):
lines.append(f"- config repo: {entry['config_repo']}")
if entry.get("risk_notes"):
lines.append(f"- risk notes: {entry['risk_notes']}")
rel = oikos_relations.relations(f"service:{name}", inv)
lines.append("\n## Blast radius")
lines.append(f"- impacts: {', '.join(rel['impacts']) or '(none)'}")
lines.append(f"- affected by: {', '.join(rel['affected_by']) or '(none)'}")
lines.append("\n## Safe actions")
for a in oikos_policy.safe_actions_for_service(name, entry):
lines.append(f"- {a['action']}{a['risk']} (approval: {a['approval']})")
hist = oikos_ledger.history(f"service:{name}", limit=5)
lines.append("\n## Recent changes")
if hist:
for h in hist:
lines.append(f"- {h.get('ts', '?')} {h.get('action', '?')} ({h.get('risk', '?')}) — {h.get('result', '?')}")
else:
lines.append("- (none yet)")
return "\n".join(lines) + "\n"
def generate_cards(inv: dict) -> dict[Path, str]:
desired: dict[Path, str] = {}
for name, entry in inv.get("hosts", {}).items():
desired[CARDS_DIR / f"host-{name}.md"] = _host_card(name, entry, inv)
for name, entry in inv.get("services", {}).items():
if isinstance(entry, dict):
desired[CARDS_DIR / f"service-{name}.md"] = _service_card(name, entry, inv)
return desired
def write_cards(inv: dict, check: bool = False) -> int:
CARDS_DIR.mkdir(parents=True, exist_ok=True)
desired = generate_cards(inv)
diff_count = 0
for path, content in desired.items():
existing = path.read_text() if path.exists() else ""
if existing != content:
diff_count += 1
if not check:
path.write_text(content)
for existing_path in CARDS_DIR.glob("*.md"):
if existing_path not in desired:
diff_count += 1
if not check:
existing_path.unlink()
return diff_count
def render(inv: dict) -> str:
hosts = inv.get("hosts", {})
services = inv.get("services", {})
counts = (
f"{sum(1 for e in hosts.values() if e.get('kind') == 'proxmox-host')} hypervisors, "
f"{sum(1 for e in hosts.values() if e.get('kind') == 'lxc')} LXCs, "
f"{sum(1 for e in hosts.values() if e.get('kind') == 'vm')} VMs, "
f"{sum(1 for e in hosts.values() if e.get('kind') == 'workstation')} workstations, "
f"{len(services)} services"
)
parts = [
BANNER,
"# Topology (generated)\n",
f"Source: [inventory.yaml](../../../inventory.yaml) — {counts}.",
"Edge semantics: [oikos/ontology.yaml](../../../oikos/ontology.yaml). "
"Operating model: [OIKOS.md](../../../.agents/OIKOS.md).\n",
"## Compute & ingress\n",
"\n".join(compute_view(inv)) + "\n",
"## Storage (mounts)\n",
"\n".join(storage_view(inv)) + "\n",
]
arch = archaeology_table(inv)
if arch:
parts += ["## Archaeology (destroyed nodes)\n", "\n".join(arch) + "\n"]
return "\n".join(parts)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--check", action="store_true",
help="exit 1 if output would change (don't write)")
args = parser.parse_args()
inv = yaml.safe_load(INVENTORY.read_text())
content = render(inv)
existing = OUTPUT.read_text() if OUTPUT.exists() else ""
topology_changed = existing != content
card_diffs = write_cards(inv, check=args.check)
if args.check:
if topology_changed:
print(f"{OUTPUT.relative_to(REPO)} would change", file=sys.stderr)
if card_diffs:
print(f"{card_diffs} card(s) in oikos/cards/ would change", file=sys.stderr)
return 1 if (topology_changed or card_diffs) else 0
if topology_changed:
OUTPUT.write_text(content)
print(f"wrote {OUTPUT.relative_to(REPO)}")
if card_diffs:
print(f"wrote/updated {card_diffs} card(s) in oikos/cards/")
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -1,112 +0,0 @@
"""oikos/gen_topology_lib.py — shared Mermaid-view logic.
Split out of oikos/gen-topology.py so it's importable (a hyphenated
filename can't be `import`ed as a module). oikos/gen-topology.py is the
CLI entrypoint that writes knowledge/wiki/infrastructure/topology.md + oikos/cards/;
oikos/console/app.py imports this module directly to render the live
/graph page without shelling out.
"""
from __future__ import annotations
from pathlib import Path
import yaml
REPO = Path(__file__).resolve().parent.parent
INVENTORY = REPO / "inventory.yaml"
def load_inventory() -> dict:
return yaml.safe_load(INVENTORY.read_text())
def node_id(name: str) -> str:
"""Mermaid-safe node id."""
return name.replace("-", "_").replace(".", "_").replace("/", "_").strip("_")
def guest_label(name: str, entry: dict) -> str:
pve = entry.get("pve_id")
role = entry.get("role", "")
tag = f"LXC {pve}" if entry.get("kind") == "lxc" and pve else \
f"VM {pve}" if entry.get("kind") == "vm" and pve else entry.get("kind", "")
ip = entry.get("lan_ip", "")
parts = [name, tag, role, ip]
return "<br/>".join(str(p) for p in parts if p)
def compute_view(inv: dict) -> list[str]:
hosts = inv.get("hosts", {})
services = inv.get("services", {})
lines = ["```mermaid", "flowchart LR"]
hypervisors = {n: e for n, e in hosts.items() if e.get("kind") == "proxmox-host"}
guests = {n: e for n, e in hosts.items() if e.get("kind") in ("lxc", "vm")}
others = {n: e for n, e in hosts.items()
if e.get("kind") in ("workstation", "external")}
for hv in hypervisors:
lines.append(f' subgraph {node_id(hv)}_sub["{hv} (Proxmox)"]')
for g, e in guests.items():
if e.get("host") == hv:
lines.append(f' {node_id(g)}["{guest_label(g, e)}"]')
lines.append(" end")
# guests without a parent hypervisor recorded (e.g. rclone)
for g, e in guests.items():
if e.get("host") not in hypervisors:
lines.append(f' {node_id(g)}["{guest_label(g, e)}"]')
for n, e in others.items():
shape = "([{}])" if e.get("kind") == "workstation" else "[[{}]]"
lines.append(f' {node_id(n)}{shape.format(guest_label(n, e))}')
# ingress: public URL -> backend (routes-to)
for svc, e in sorted(services.items()):
if not isinstance(e, dict):
continue
backend = e.get("backend")
url = e.get("url") or (
f'https://{e["public_host"]}' if e.get("public_host") else None)
if backend and url and backend in hosts:
host = url.removeprefix("https://").removeprefix("http://")
# hypervisors are rendered as subgraphs; point edges at the subgraph id
target = node_id(backend) + ("_sub" if backend in hypervisors else "")
lines.append(
f' {node_id("url_" + svc)}(["{host}"]) -->|routes-to| {target}')
lines.append("```")
return lines
def storage_view(inv: dict) -> list[str]:
hosts = inv.get("hosts", {})
lines = ["```mermaid", "flowchart LR"]
pools: set[str] = set()
edges: list[str] = []
for name, e in hosts.items():
for mount in e.get("mounts", []):
pools.add(mount)
edges.append(f' {node_id(name)}["{name}"] -->|mounts| {node_id(mount)}')
for pool in sorted(pools):
lines.append(f' {node_id(pool)}[("{pool}")]')
lines.extend(sorted(set(edges)))
lines.append("```")
return lines
def archaeology_table(inv: dict) -> list[str]:
arch = inv.get("archaeology", {})
if not arch:
return []
lines = ["| Node | ID | Destroyed | Reason |", "|---|---|---|---|"]
entries = sorted(arch.items(), key=lambda kv: str(kv[1].get("destroyed", "")),
reverse=True)
for name, e in entries:
lines.append(
f'| {name} | {e.get("pve_id", "")} | {e.get("destroyed", "")} '
f'| {e.get("reason", "")} |')
return lines

View File

@@ -1,8 +1,11 @@
# 2026-07-08 — Liveness, drift, and UX cohesion
**Status:** In Progress — Phases 14 code complete; not yet deployed. Phase 5 deferred.
(Audited 2026-07-11 — still accurate; prompt caching within Phase 4 also
confirmed not implemented.)
**Status:** In Progress — Phases 14 code complete and now deployed
(re-verified 2026-07-12: mac-mini was redeployed from `main` that day for
unrelated auth work — plans/2026-07-12-wails-desktop-app.md — which carried
every commit up to that point, including this plan's, so "not yet deployed"
below is stale). Phase 5 deferred. (Audited 2026-07-11 — still accurate;
prompt caching within Phase 4 also confirmed not implemented.)
- **Phase 1 (drift/staleness):** done. Health/metrics/events misattribution
fix, staleness sweep, `/entities` health+freshness, dashboard/fleet-health
@@ -26,20 +29,19 @@ confirmed not implemented.)
- **Phase 4 (agent efficiency):** core piece done — prior turns' tool
calls/results are now replayed into the conversation (previously dropped
entirely), and a compact live fleet-health snapshot is injected into the
system prompt each turn so the agent starts oriented. Prompt caching and
reconsidering the default model are **not done** (lower priority, no
measured regression without them).
system prompt each turn so the agent starts oriented. Prompt caching is
**not done** (lower priority, no measured regression without it).
Reconsidering the default model — done, but not by this plan: switched to
`deepseek/deepseek-v4-pro` on 2026-07-10 (`cmd/nomos/agent.go:70`) for
reliability, per that commit's own comment ("the flash tier over-narrates,
occasionally emits canned refusals, and is unreliable at multi-step tool
use").
- **Phase 5 (CRUD):** `PatchEntity` and a full `/checks` CRUD API
(list/create/patch, including enable/disable) already existed server-side;
the new Monitoring card's toggle uses `PatchCheck`. **Not done**: a
"run check now" endpoint (no scheduler on-demand entrypoint exists yet),
relationship editing, and an entity attribute editor UI.
**Not yet deployed** — the live `oikos-api`/`oikos-scheduler`/nomos
containers still run the pre-fix binaries; rebuilding and restarting them
needs an explicit go-ahead since it touches the running homelab control
plane.
Addresses five felt problems with the current system: (1) the agent reports
stale machine state as if it were fresh, (2) sessions can't be opened and feel
disconnected from chat, (3) the Nomos agent re-derives state every turn and

View File

@@ -1,14 +1,24 @@
# 2026-07-08 — Oikos gaps, broken things, and improvements
**Status:** In Progress — audited 2026-07-11. Done: A1 (approval FK bug),
A3 (Hermes→Nomos help text), D1 (`upsert_knowledge`), D4-partial (general
`run` tool). Still open: A2 (notifier flooding/dedup), A4 (`resolveHost`
dead code), A5 (`queryRows` stringly-typed columns), A6 (stale
`get_state_snapshot` description), B1-B5 (enrollment auth, fake Infisical
creds, `/query` mesh-only auth unenforced, insecure host key checking,
optional `caller_pubkey`), D2/D3 (no `get_approval_status`/
`list_pending_approvals`/signal ack-resolve-mute tools), E (README tool
count, Caddyfile placeholders, NOMOS.md duplicate line).
**Status:** In Progress — audited 2026-07-11, re-audited 2026-07-12 for
drift from the `cmd/hermes``cmd/nomos` rename and later fixes. Done: A1
(approval FK bug), A3 (Hermes→Nomos help text), **Section C** (toy NLU /
silent-wrong-answer fallback — nomos now calls real `listTools()` and
routes unmatched queries to `/chat` instead of guessing, per
`cmd/nomos/main.go:444-465`), **D.5** (SOUL.md/actuator architecture
mismatch — `nomos/SOUL.md:21-22,40` now accurately documents SSH via the
`run` tool), D1 (`upsert_knowledge`), D4-partial (general `run` tool).
Still open: A2 (notifier flooding/dedup), A4 (`resolveHost` dead code), A5
(`queryRows` stringly-typed columns), A6 (stale `get_state_snapshot`
description), B1-B5 (enrollment auth, fake Infisical creds, `/query`
mesh-only auth unenforced, insecure host key checking, optional
`caller_pubkey`), D2/D3 (no `get_approval_status`/`list_pending_approvals`/
signal ack-resolve-mute tools), E-partial (Caddyfile placeholders still
present; tool count now 33, documented in AGENTS.md as of 2026-07-12).
2026-07-12 re-audit also refreshed every `cmd/hermes``cmd/nomos` and
`internal/mcp/server.go` line-number citation below (the file grew from 28
to 33 registered tools since 2026-07-11) — content/status of each finding
unchanged, only citations moved.
## Goal
@@ -78,22 +88,24 @@ named `tools/list`, which doesn't exist. The correct `listTools()` helper
### A4. `resolveHost` never returns a per-entity SSH user
`internal/mcp/server.go:943` — the named return `sshUser` is always `""`; the
per-entity user branch is dead and everything relies on `sshExec`'s global
default fallback. **Fix:** read the SSH user from entity attributes or delete
the dead return to make the behavior honest.
`internal/mcp/server.go:1222` (was :943 — line moved) — the named return
`sshUser` is always `""`; the per-entity user branch is dead and everything
relies on `sshExec`'s global default fallback. **Fix:** read the SSH user
from entity attributes or delete the dead return to make the behavior
honest.
### A5. `queryRows` stringifies every column
`internal/mcp/server.go:861` renders all values via `fmt.Sprintf("%v", ...)`,
so numbers, bools, timestamps, and JSON all reach agents as strings.
**Fix:** type-preserving serialization (pass through pgx-native values into
`json.Marshal`) — improves every read tool at once.
`internal/mcp/server.go:1090` (was :861 — line moved) renders all values via
`fmt.Sprintf("%v", ...)`, so numbers, bools, timestamps, and JSON all reach
agents as strings. **Fix:** type-preserving serialization (pass through
pgx-native values into `json.Marshal`) — improves every read tool at once.
### A6. `get_state_snapshot` description is stale
`internal/mcp/server.go:689` still advertises "disk, drift count" — columns
removed in commit 3ea43ad. **Fix:** update the description.
`internal/mcp/server.go:863` (was :689 — line moved) still advertises "disk,
drift count" — columns removed in commit 3ea43ad. **Fix:** update the
description.
---
@@ -101,12 +113,13 @@ removed in commit 3ea43ad. **Fix:** update the description.
### B1. Enrollment is unauthenticated, with a false comment
`internal/httpapi/server.go:97` says "unauthenticated (IP-gated in handler)"
but `EnrollClient` (`internal/httpapi/impl.go:1099`) performs no IP check at
all — the only gate is the target entity being in state
`planned`/`provisioning`. Caddy's `@enroll` matcher bypasses Authentik.
Anyone reaching `oikos.hubris.network` who knows (or guesses) a planned slug
receives that node's **age private key** in the HTTP response body.
`internal/httpapi/server.go:111` (was :97) says "unauthenticated (IP-gated
in handler)" but `EnrollClient` (`internal/httpapi/impl.go:1166`, was
:1099) performs no IP check at all — the only gate is the target entity
being in state `planned`/`provisioning`. Caddy's `@enroll` matcher bypasses
Authentik. Anyone reaching `oikos.hubris.network` who knows (or guesses) a
planned slug receives that node's **age private key** in the HTTP response
body. Still open — line numbers only, substance unchanged.
**Fix:** enforce a real gate (mesh-CIDR check, one-time enrollment token
minted when the entity is created, or both), and stop returning the age
@@ -114,34 +127,49 @@ private key in the response — have the client fetch it from the secret store.
### B2. Fake Infisical credentials returned to enrollees
`internal/httpapi/impl.go:1191-1192` returns `"inf_client_"+uuid` /
`"inf_secret_"+uuid` — random strings wired to nothing. Enrolled clients hold
credentials that authenticate against nothing.
`internal/httpapi/impl.go:1260-1261` (was :1191-1192) returns
`"inf_client_"+uuid` / `"inf_secret_"+uuid` — random strings wired to
nothing. Enrolled clients hold credentials that authenticate against
nothing. Still open — line numbers only, substance unchanged.
**Fix:** implement `CreateMachineIdentity` in `internal/secrets/infisical.go`,
or return no credentials and document the manual step.
### B3. Hermes `/query` has no auth
### B3. Nomos's `/query` has no auth
`hermes/config.yaml:9` sets `mesh_only: true` but `cmd/hermes/main.go` never
reads or enforces it — it serves any caller on :8092, who can invoke
`request_execution`. **Fix:** enforce mesh-CIDR (or bearer token) in the
handler; fail closed.
`nomos/config.yaml:9` (was `hermes/config.yaml:9`) sets `mesh_only: true`
but `cmd/nomos/main.go` (was `cmd/hermes/main.go`) never reads or enforces
it — it serves any caller on :8092, who can invoke `request_execution`.
Still open, now also tracked as C1 in
[2026-07-11-nomos-agent-code-review.md](2026-07-11-nomos-agent-code-review.md),
deferred by the operator. **Fix:** enforce mesh-CIDR (or bearer token) in
the handler; fail closed.
### B4. SSH host keys not verified
`ssh.InsecureIgnoreHostKey()` at `internal/mcp/server.go:920`.
`ssh.InsecureIgnoreHostKey()` at `internal/mcp/server.go:1155` (was :920).
Still open — line number only, substance unchanged.
**Fix:** known_hosts pinning (keys are already inventory-managed per node).
### B5. `list_my_secrets` enumerates all node pubkeys
Without `caller_pubkey`, `internal/mcp/server.go:709-720` returns every entity
that has an `age_pubkey`; nothing ties the caller to what it may list.
Without `caller_pubkey`, `internal/mcp/server.go:879-883` (was :709-720)
returns every entity that has an `age_pubkey`; nothing ties the caller to
what it may list. Still open — line numbers only, substance unchanged.
**Fix:** require `caller_pubkey` and scope results to the caller's
entitlements.
---
## C. User perspective (interacting via Hermes)
## C. User perspective (interacting via Hermes) — RESOLVED
**Resolved as of the Hermes→Nomos rewrite (verified 2026-07-12).** This
entire section described `cmd/hermes`, which no longer exists — Hermes was
renamed and rebuilt as `cmd/nomos`, a real LLM-backed agent loop, which is
exactly the recommendation below. `cmd/nomos/main.go:444-465` now calls the
real `listTools()` for "help"/"what can you do", and routes unmatched
queries to "natural language queries belong to `/chat`..." instead of
silently falling back to `get_health_summary`. Kept below for history —
original text unchanged.
- `routeQuery` NLU is hardcoded `strings.Contains`; `extractEntity`
(`cmd/hermes/main.go:173`) recognizes only 5 services (`authentik, caddy,
@@ -175,27 +203,41 @@ says 21 — both stale). Missing capabilities:
`pending_approval`, an agent has no way to check or reference the approval.
Add `get_approval_status` / `list_pending_approvals`.
4. Execution actions limited to `restart | systemctl | pct_exec |
apt_upgrade` — no deploy/rollback/config-edit path.
5. Architecture/doc mismatch: `hermes/SOUL.md` claims "no SSH access; all
mutations flow through the actuator", but the MCP server runs
`restart`/`pct_exec` synchronously over SSH from inside the api process
(`sshExec`, server.go:902). Align docs or move execution to the actuator.
apt_upgrade` — no deploy/rollback/config-edit path. Partially
superseded: the general `run` MCP tool (D4-partial, done) covers
arbitrary commands now; `request_execution`'s fixed enum is still there
for the specific actions it names (see
[2026-07-10-general-gated-execution.md](2026-07-10-general-gated-execution.md)).
5. **RESOLVED (verified 2026-07-12).** Architecture/doc mismatch:
`hermes/SOUL.md` claimed "no SSH access; all mutations flow through the
actuator", but the MCP server ran `restart`/`pct_exec` synchronously over
SSH from inside the api process. `nomos/SOUL.md:21-22,40` now accurately
documents SSH access via the policy-gated `run` tool — matches the
architecture the general-gated-execution plan built. No longer a
mismatch.
---
## E. Doc drift / housekeeping
- Tool counts: README 15 / AGENTS.md 21 / actual 28 — regenerate from
`internal/mcp/server.go` (consider a doc-gen make target).
- `compose/caddy/Caddyfile.oikos` retains literal `<mac-mini-mesh-ip>`
placeholders in all three vhosts.
- `.agents/HERMES.md` lists "`inventory.yaml`, `inventory.yaml`" (duplicate).
- `plans/index.md` drift: fix-MCP-tools row sat in Active with a broken link
after the file moved to `done/` (fixed alongside this plan); TRMNL listed
active though in `done/`; Grimmory header says `in-progress` though in
`done/`; `.hermes/plans/` (7 executed plans) missing from disk.
- `plans/2026-07-05-oikos-prometheus-lxc.md` (~0% done) references deleted
`oikos/scheduler.py` and `bin/homelab`; LXC 131 collision unresolved.
- **RESOLVED (verified 2026-07-12):** Tool counts. README 15 / AGENTS.md 21
/ actual 28 was already stale by 2026-07-11 (registered tools grew to
33) — AGENTS.md now documents all 33 with the full catalog (2026-07-12).
- **Still open:** `compose/caddy/Caddyfile.oikos` retains literal
`<mac-mini-mesh-ip>` placeholders (this repo's copy is a reference only —
see [2026-07-12-wails-desktop-app.md](2026-07-12-wails-desktop-app.md)'s
"Plan review" — the real config lives in `dtoro/caddy-conf`).
- **RESOLVED:** `.agents/HERMES.md` renamed to `.agents/NOMOS.md`; the
duplicate-line bug itself is still present at `.agents/NOMOS.md:11` —
only the file citation was stale, the underlying nit is still open.
- **RESOLVED (verified 2026-07-12):** `plans/index.md` drift — the broken
link, TRMNL/Grimmory Active/Done mismatch, and missing `.hermes/plans/`
entries described here are no longer present in the current
`plans/index.md`; already fixed sometime after this plan was written.
- **RESOLVED (verified 2026-07-12):** `plans/2026-07-05-oikos-prometheus-lxc.md`
already self-corrected both the deleted-file references and the LXC 131
collision in its own 2026-07-08 changelog — this bullet describes a
pre-fix state.
---

View File

@@ -182,16 +182,27 @@ board reflects reality instead of showing a task that looks perpetually
### C1. Nomos's own HTTP gateway has zero authentication
[docker-compose.yml:133](../docker-compose.yml) publishes port 8092 directly
[docker-compose.yml:144](../docker-compose.yml) publishes port 8092 directly
(`"8092:8092"`, comment: *"mesh-published"*) and
[Caddyfile.oikos](../compose/caddy/Caddyfile.oikos:19,34) reverse-proxies to
it from two routes. `grep -n "Authorization\|Bearer\|auth" cmd/nomos/main.go`
returns **nothing**`/chat`, `/sessions`, `/sessions/{id}` (including
`DELETE`), and `/query` have no credential check of any kind. Anyone who can
reach the LAN or mesh network can converse with Nomos directly: start tasks,
read/delete any session, answer pending questions, and — via chat-assent —
approve gated executions by typing "yes" or "I confirm" to whatever the agent
proposes, with no authentication at all. This is the same class of gap
[Caddyfile.oikos:52-54](../compose/caddy/Caddyfile.oikos) reverse-proxies to
it — as of the client/server split
([2026-07-12-wails-desktop-app.md](2026-07-12-wails-desktop-app.md)), only
from `nomos.hubris.network` now, not two routes: `/agent/*` on
`oikos.hubris.network` was repointed to go through `api`'s own authenticated
proxy mount instead of straight to nomos:8092, but that's `combinedAuth`
authenticating the *hop into api*, not anything nomos itself checks — this
finding is unaffected by that change, still fully open. `grep -n
"Authorization\|Bearer\|auth" cmd/nomos/main.go` still returns **nothing**
for nomos's inbound routes (nomos did gain outbound auth as *part of* the
client/server split — it now sends `Authorization: Bearer
$OIKOS_MCP_BEARER_TOKEN` on its own calls to `api` — but that's the opposite
direction from this finding) — `/chat`, `/sessions`, `/sessions/{id}`
(including `DELETE`), and `/query` have no credential check of any kind.
Anyone who can reach the LAN or mesh network can converse with Nomos
directly: start tasks, read/delete any session, answer pending questions,
and — via chat-assent — approve gated executions by typing "yes" or "I
confirm" to whatever the agent proposes, with no authentication at all. This
is the same class of gap
[oikos-gaps-and-improvements](2026-07-08-oikos-gaps-and-improvements.md)
flagged for the `api`/MCP surface (items B1-B5), but specifically for nomos's
*own* port, which doesn't sit behind `combinedAuth` the way `api`'s routes do.

View File

@@ -1,10 +1,33 @@
# 2026-07-12 — Wails desktop application
**Status:** In Progress — Phase 0 (0.1-0.4, 0.6) done and verified live
(browser: cross-origin static SPA + API on different ports, CORS, bearer
auth, SSE query-token auth, localStorage persistence across reload — see
"Plan review" for the gaps found and fixed along the way). Phase 1 (Wails
shell) not started.
**Status:** In Progress — Phase 0 (0.1-0.4, 0.6) done, verified live in a
local browser test, and **deployed to production** (mac-mini, commit
`0c0f35a`, 2026-07-12). Phase 1 (Wails shell) not started.
**Production deploy (2026-07-12):** merged to `main`, picked up by the
2-minute deploy poller (`scripts/deploy.sh`: pg_dump backup → rebuild →
rolling restart → health check), `healthy after 1s`. Verified post-deploy:
unauthenticated `/api/v1/*` now 401s (the dev-open bypass was live in
production before this — `OIKOS_ENV=dev` with no token set — so this closed
a real, currently-exploitable hole, not just future prep); `/healthz` stayed
open; nomos reconnected its MCP session with the new
`OIKOS_MCP_BEARER_TOKEN` and a real tool call round-tripped end to end
(`get_health_summary` via `/query`). A real random token was generated and
added to mac-mini's `.env` (not committed — gitignored) before deploy, so
the `${OIKOS_MCP_BEARER_TOKEN:-dev-token}` fallback in `docker-compose.yml`
never activated with the weak literal default.
**Deliberately not done as part of this deploy** (out of scope — a different
host/repo than "mac-mini", not touched): the Caddy LXC (121) and
`dtoro/caddy-conf`. Checked the real production Caddyfile directly — there is
**no `oikos.hubris.network` site block at all yet**, so the Authentik-bypass
risk (gap 1 below) doesn't apply yet; there's no public UI exposed to break.
`mcp.hubris.network` exists but still reverse-proxies to the old
pre-consolidation service on LXC 105 (`192.168.8.205:9810`), unrelated to
this stack — stale, but pre-existing and out of scope here. Exposing
`oikos.hubris.network` publicly (with the `@api` bypass this plan's
Caddyfile.oikos reference copy already has) is unstarted follow-up work, not
a regression from this deploy.
## Plan review — gaps found before starting Phase 0

View File

@@ -10,11 +10,11 @@ went sideways, open an investigation.
| ---- | ----- | ------ |
| 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned — not started |
| 2026-07-08 | [Oikos gaps, broken things, and improvements](2026-07-08-oikos-gaps-and-improvements.md) | In Progress — security items (B1-B5) and doc drift (E) still open |
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | In Progress |
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | In Progress — packaging/auth sections superseded by the Wails plan's Phase 0 (client/server split); M4 still open |
| 2026-07-08 | [Liveness, drift, and UX cohesion](2026-07-08-liveness-drift-and-ux-cohesion.md) | In Progress — Phase 5 deferred |
| 2026-07-10 | [General gated execution: unlimited actions, gated by risk](2026-07-10-general-gated-execution.md) | In Progress — enum retirement + auto-act revival still open |
| 2026-07-11 | [Nomos agent code review: gaps and improvement plan](2026-07-11-nomos-agent-code-review.md) | In Progress — only C1 (unauthenticated nomos gateway) still open, deferred |
| 2026-07-12 | [Wails desktop application](2026-07-12-wails-desktop-app.md) | In Progress — Phase 0 done, Phase 1 not started |
| 2026-07-12 | [Wails desktop application](2026-07-12-wails-desktop-app.md) | In Progress — Phase 0 done and deployed to production (2026-07-12), Phase 1 not started |
## Done

View File

@@ -1,23 +0,0 @@
#!/usr/bin/env node
// Caveman template renderer — reads template + data JSON files and renders output
// Installed automatically via homelab-context post-pull hook
const caveman = require("caveman");
const fs = require("fs");
const path = require("path");
const args = process.argv.slice(2);
if (args.length === 0) {
console.error("Usage: caveman <template> [data.json]");
process.exit(1);
}
const templatePath = args[0];
let data = {};
if (args.length > 1) {
data = JSON.parse(fs.readFileSync(args[1], "utf8"));
}
const template = fs.readFileSync(templatePath, "utf8");
const templateName = path.basename(templatePath, path.extname(templatePath));
caveman.register(templateName, template);
console.log(caveman.render(templateName, data).trim());

View File

@@ -1,76 +0,0 @@
#!/usr/bin/env bash
# Caveman + RTK Wrapper - Automated token-efficient output formatting
# Installed automatically via homelab-context post-pull hook
# Source: https://github.com/adityahimaone/hermes-agent-rtk-caveman
# Usage: caveman_wrapper.sh <workflow> [options]
set -e
WORKFLOW="$1"
shift
CAVEMAN=~/bin/caveman
TEMPLATES_DIR=~/templates
DATA_DIR=/tmp/caveman_data
mkdir -p "$DATA_DIR"
case "$WORKFLOW" in
git-status)
git status --porcelain | awk '
BEGIN { staged=0; modified=0; untracked=0; deleted=0 }
/^[MARC]./ { staged_arr[staged++] = substr($0, 4) }
/^.[MARC]/ { modified_arr[modified++] = substr($0, 4) }
/^\?\?/ { untracked_arr[untracked++] = substr($0, 4) }
/^D/ || /^.D/ { deleted_arr[deleted++] = substr($0, 4) }
END {
printf "{"
printf "\"staged\":["
for(i=0;i<staged;i++) printf "%s\"%s\"", (i>0?",":""), staged_arr[i]
printf "],\"modified\":["
for(i=0;i<modified;i++) printf "%s\"%s\"", (i>0?",":""), modified_arr[i]
printf "],\"untracked\":["
for(i=0;i<untracked;i++) printf "%s\"%s\"", (i>0?",":""), untracked_arr[i]
printf "],\"deleted\":["
for(i=0;i<deleted;i++) printf "%s\"%s\"", (i>0?",":""), deleted_arr[i]
printf "]}"
}' > "$DATA_DIR/git_status.json"
if command -v rtk &>/dev/null; then
rtk "$CAVEMAN" "$TEMPLATES_DIR/git_status.txt" "$DATA_DIR/git_status.json"
else
node "$CAVEMAN" "$TEMPLATES_DIR/git_status.txt" "$DATA_DIR/git_status.json"
fi
;;
git-log)
LIMIT="${1:-10}"
git log --oneline -"$LIMIT" --format='{"hash":"%h","author":"%an","date":"%ad","message":"%s"}' --date=short | \
jq -s '.' > "$DATA_DIR/git_log.json"
jq '{commits: .}' "$DATA_DIR/git_log.json" > "$DATA_DIR/git_log_final.json"
if command -v rtk &>/dev/null; then
rtk "$CAVEMAN" "$TEMPLATES_DIR/git_log.txt" "$DATA_DIR/git_log_final.json"
else
node "$CAVEMAN" "$TEMPLATES_DIR/git_log.txt" "$DATA_DIR/git_log_final.json"
fi
;;
test-results)
TEST_CMD="${1:-npx vitest run}"
$TEST_CMD --reporter json 2>/dev/null | \
jq '{total: .numTotalTests, passed: .numPassedTests, failed: .numFailedTests, suites: [.testResults[] | {name: .name, status: .status, duration: .duration}]}' > "$DATA_DIR/test_results.json" || true
if command -v rtk &>/dev/null; then
rtk "$CAVEMAN" "$TEMPLATES_DIR/test_results.txt" "$DATA_DIR/test_results.json"
else
node "$CAVEMAN" "$TEMPLATES_DIR/test_results.txt" "$DATA_DIR/test_results.json"
fi
;;
*)
echo "Usage: caveman_wrapper.sh <workflow> [options]"
echo " git-status - Compact git status"
echo " git-log [limit] - Recent git commits"
echo " test-results [cmd] - Compact test results"
exit 1
;;
esac

View File

@@ -1,3 +0,0 @@
Recent Commits:
{{- for d.commits as commit }} {{commit.hash}} {{commit.date}} {{commit.message}}
{{- end }}

View File

@@ -1,12 +0,0 @@
{{- if d.staged }}Staged:
{{- for d.staged as file }} + {{file}}
{{- end }}{{- end }}
{{- if d.modified }}Modified:
{{- for d.modified as file }} ~ {{file}}
{{- end }}{{- end }}
{{- if d.untracked }}Untracked:
{{- for d.untracked as file }} ? {{file}}
{{- end }}{{- end }}
{{- if d.deleted }}Deleted:
{{- for d.deleted as file }} - {{file}}
{{- end }}{{- end }}

View File

@@ -1,4 +0,0 @@
{{- if d.failed }}Tests: {{d.passed}}/{{d.total}} passed ({{d.failed}} failed)
{{- for d.suites as suite }}{{- if suite.status == "failed" }} {{suite.name}} ({{suite.duration}}ms)
{{- end }}{{- end }}{{- else }}All {{d.total}} tests passed
{{- end }}

View File

@@ -3,7 +3,7 @@
# Replaces raw `git pull` in the launchd/systemd timer.
# Runs after every git pull to auto-setup tools from the repo.
#
# Convention: any script at tools/*.setup.sh is sourced/exec'd after pull.
# Convention: any script at tools/setup-*.sh is sourced/exec'd after pull.
# This lets us ship new tooling to all agent hosts via a simple git push.
set -euo pipefail
@@ -31,7 +31,7 @@ else
fi
# 2. Run any auto-setup scripts
for setup_script in "$CONTEXT_DIR"/tools/*.setup.sh; do
for setup_script in "$CONTEXT_DIR"/tools/setup-*.sh; do
[ -f "$setup_script" ] || continue
echo "[post-pull] running $setup_script..."
bash "$setup_script" || echo "[post-pull] WARNING: $setup_script exited with code $?"

View File

@@ -1,34 +0,0 @@
#!/usr/bin/env bash
# setup-caveman.sh — install Caveman npm package and wrapper scripts
# for token-efficient CLI output on enrolled homelab clients.
set -euo pipefail
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
BIN_DIR="$HOME/bin"
TOOLS_DIR="$CLONE_DIR/tools"
mkdir -p "$BIN_DIR"
# Install the caveman npm package globally.
if ! command -v caveman >/dev/null 2>&1; then
if command -v npm >/dev/null 2>&1; then
npm install -g caveman 2>/dev/null || true
echo "[setup-caveman] caveman npm package installed"
fi
fi
# Copy wrapper to ~/bin.
if [ -f "$TOOLS_DIR/caveman_wrapper.sh" ]; then
cp "$TOOLS_DIR/caveman_wrapper.sh" "$BIN_DIR/caveman_wrapper.sh"
chmod +x "$BIN_DIR/caveman_wrapper.sh"
echo "[setup-caveman] wrapper installed to $BIN_DIR/caveman_wrapper.sh"
fi
# Copy templates.
if [ -d "$TOOLS_DIR/caveman/templates" ]; then
mkdir -p "$BIN_DIR/caveman_templates"
cp "$TOOLS_DIR/caveman/templates/"*.txt "$BIN_DIR/caveman_templates/" 2>/dev/null || true
echo "[setup-caveman] templates installed"
fi
echo "[setup-caveman] done"

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# setup-checks.sh — deploy check scripts to /opt/oikos/checks on each host.
# Auto-setup hook: tools/*.setup.sh runs after every git pull.
# Auto-setup hook: tools/setup-*.sh runs after every git pull.
set -euo pipefail
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"

View File

@@ -1,14 +0,0 @@
#!/usr/bin/env bash
# setup-nomos-soul.sh — provision Nomos agent persona.
# Copies ~/.nomos/SOUL.md from nomos/SOUL.md. No-op on non-Nomos agents.
set -euo pipefail
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
if [ -f "$CLONE_DIR/nomos/SOUL.md" ]; then
mkdir -p "$HOME/.nomos"
cp "$CLONE_DIR/nomos/SOUL.md" "$HOME/.nomos/SOUL.md"
echo "[setup-nomos-soul] SOUL.md provisioned"
else
echo "[setup-nomos-soul] no nomos/SOUL.md found; skipping"
fi