Documentation and repo-hygiene pass following the client/server split:
Plan drift (audited all other active plans against current code):
- oikos-gaps-and-improvements.md: mark Section C and D.5 resolved (both
described cmd/hermes, renamed to cmd/nomos with a real LLM loop since);
refresh ~10 stale file:line citations; fix tool-count (33, not 28).
- liveness-drift-and-ux-cohesion.md: fix stale default-model claim (now
deepseek-v4-pro since 2026-07-10) and "not yet deployed" status.
- nomos-agent-code-review.md: fix C1's citation (one unauthenticated route
to nomos now, not two, after the client/server split).
- wails-desktop-app.md: record the production deploy outcome.
Repo structure: added missing directories to README/CONTRIBUTING layout
tables (checks/, tools/, cmd/webhook/, docs/operations/), fixed a broken
link, added ADR 0015 documenting the auth/CORS/client-split model (there
wasn't one despite CONTRIBUTING's own process requiring it), normalized
ADR 0013/0014's format drift, added an Authentication section to
AGENTS.md/CLIENTS.md (every example call was missing the now-required
bearer header).
Retired the Goose+Nomos workstation flow (bootstrap.sh --with-nomos,
tools/setup-nomos-soul.sh, .agents/operations/nomos-agent.md) and the
Caveman auto-install tooling (tools/setup-caveman.sh, tools/caveman/) —
both superseded by the production containerized Nomos agent, which has
never used either. Kept .agents/shared/caveman.md itself (the terse
writing-style convention agents still follow by reading it).
Deleted the orphaned legacy Python oikos/ directory — nothing imports it,
and bin/homelab (the CLI it was kept for) no longer exists in the repo.
Rewrote .agents/operations/agent-enrollment.md (365 -> ~110 lines) and
commands.md to match the current architecture instead of the retired
`homelab` CLI; migrated the still-true networking prerequisites (Netbird,
split-horizon DNS, SSH key distribution) into the knowledge base as a
runbook via upsert_knowledge rather than duplicating them in markdown.
Updated all 10 .agents/skills/ runbooks referencing the dead CLI with
their real MCP tool / REST API equivalents, or flagged them as needing
verification where no equivalent is confirmed yet.
Two real bugs found and fixed, not just docs:
- The tools/setup-*.sh auto-setup glob was tools/*.setup.sh in THREE
places (tools/post-pull.sh, bootstrap.sh, and internal/httpapi/impl.go's
GetClientContext handler) since the mechanism's introduction on
2026-06-02 — never matched any real filename, so no client has ever
picked up an auto-setup script via git-pull or the context-poller sync.
Fixed all three; the Go server-side fix is the one that actually matters
since it's what the current context-poller mechanism depends on.
- bootstrap.sh removed dead vestigial --gitea-token/--gitea-user flags
(parsed, never consumed) left over from an earlier clone-based model.
Also flagged, not fixed (documented as an open gap in
client-enrollment/SKILL.md): bootstrap.sh tells a freshly-enrolled client
to call POST /api/v1/clients/{slug}/activate to finish enrollment, but
that route doesn't exist in api/openapi.yaml — EnrollClient sets entities
to provisioning and nothing currently transitions them to active.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Thin client model (rev 2): no git clone, no sync timer.
Changes:
- Fetches only CLIENTS.md, AGENTS.md, OIKOS.md, tools/ from raw Gitea URL
- Enrolls via POST /api/v1/clients/enroll (replaces dead Python
secrets-issuance service)
- Receives age keypair + Infisical identity from Oikos API
- Context poller replaces 5-minute git pull (launchd/systemd timer hits
GET /api/v1/clients/{slug}/context?since=)
- Removed --no-secrets, --no-mesh flags (degraded modes TBD)
- Removed dead bin/homelab symlink
- Removed Gitea credential configuration (no git clone = no git auth)
- Kept --with-mcp and --with-hermes flags for optional tooling
- auto-setup scripts run from fetched tools/ directory
Problem: after the wiki-hq reorg, agent-instruction and human-doc domains
were still scattered across the repo root, with three now-redundant stub
files cluttering it. The organizing principle wasn't visible in the layout.
Change — enforce three clear buckets:
- .agents/ = how agents operate: OIKOS.md, HERMES.md (moved from root),
shared/ conventions, domains/ schemas, skills/, and operations/ (operator
cheatsheet + enrollment + hermes-agent, moved from root).
- knowledge/ = what exists + evidence: wiki/, GLOSSARY.md, and sources/ now
including investigations/ (incident records are evidence/sources).
- root = substrate + two entry points (AGENTS.md, README.md), plus plans/
as its own design-intent domain.
Moves:
- investigations/ -> knowledge/sources/investigations/ (incl. archive/, index).
- operations/ -> .agents/operations/.
- HERMES.md -> .agents/HERMES.md.
- Deleted unreferenced root stubs CAVEMAN.md, CONTRIBUTING.md, and OIKOS.md
(its 7 remaining linkers repointed to .agents/OIKOS.md).
Consumers updated:
- inventory.yaml doc_page (agent-enrollment) + regenerated hosts/*.yaml + cards.
- tools/setup-hermes-soul.sh and bootstrap.sh (x2) -> .agents/HERMES.md.
- bin/homelab help string -> .agents/operations/hermes-agent.md.
- knowledge/operations schemas, llm-wiki, page-templates, incident-investigation
skill, AGENTS.md/README nav -> new investigations/operations paths.
- All markdown links rewritten via the path-resolving mapper.
Left in place (substrate/executable/separate-domain): hosts/, ledger/, tools/,
plans/, oikos/, mcp/, secrets/, bin/, inventory.yaml.
Verification: docs-lint at baseline (2 intentional cross-repo refs, no new
breakage); gen-topology.py --check exit 0; build_host_files.py idempotent; all
doc_page targets resolve; Hermes provisioning scripts point at the new path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Running bootstrap without --no-secrets always tried to install and
connect Netbird, even when the mesh-check right after it already knows
how to fall back to plain LAN reachability. On a host nobody's watching
interactively (e.g. driven over SSH), this hangs forever at the
device-code prompt — hit live on strong, had to kill the stuck
`netbird up` process manually. --no-mesh skips netbird install/up while
still allowing the existing LAN-fallback path to satisfy secrets
issuance.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Discovered live re-running bootstrap on strong for secrets issuance:
apt-get install sops fails outright (no such Debian package — matches
what agent-enrollment.md's manual-install recipe already does, fetching
the binary from GitHub releases instead of a package manager). dnf would
have the same problem. Added install_sops_binary(), used on both the
dnf and apt paths; Darwin still installs via brew, which does carry sops.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The pipx/mcp-CLI step and the Hermes goose installer both called
`sudo -u <user> ...` unconditionally. On minimal Linux images reached
via `ssh root@host` (no SUDO_USER, and often no `sudo` binary at all —
seen live on strong), this failed with "sudo: command not found" and
silently no-opped the mcp CLI install. Added a run_as() helper that
only shells out to sudo when there's a real invoking user distinct
from root.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`bootstrap.sh --with-hermes` installs the Goose CLI, drops a Goose
config pinning the OpenRouter provider + Nous Hermes model + the
homelab MCP extension, symlinks `bin/hermes` and HERMES.md, and links
HERMES.md as `.goosehints` so the persona is injected as the system
prompt every session.
`bin/hermes` decrypts `secrets/openrouter-api-key.yaml` via the existing
`homelab secret` flow and execs `goose session`.
`homelab client add --with-hermes` grants the new sops secret to the
host's age_pubkey at finalize time (parallel to the existing
shared-secrets grant). `client remove` revokes it.
`operations/hermes-agent.md` covers the end-to-end flow, verification,
troubleshooting, and queues one follow-up: the MCP server still runs
SSE-only but Goose 1.x deprecated SSE — the Goose config targets
`streamable_http` and the `homelab` extension won't connect until
`mcp/server.py` migrates. The `developer` extension (shell + edit +
`homelab` CLI) carries the agent in the meantime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes that together collapse new-workstation onboarding from ~7 steps
to ~2 commands:
* bootstrap.sh:
- Dep-check now AUTO-INSTALLS missing tools (apt/dnf/brew) instead of
printing instructions and exiting. Re-verifies after install.
- New pre-mesh-check block: if netbird isn't installed, installs it
from the netbird apt/dnf repo (or `brew install --cask netbird` on
Darwin), then if mgmt isn't connected, runs `netbird up
--management-url=https://netbird.hubris.network --ssh-jwt-cache-ttl 86400`.
Operator clicks the device-code URL inline. Waits up to ~30s for
Management: Connected before continuing. Skipped on --no-secrets +
--dry-run.
* containers/124-authentik.md: replaces the "KNOWN MISSING — Device Code
Stage" subsection with a working recipe — Authentik 2026.2 routes
/device via a BRAND-level "Device code flow" field, not a provider
field. Documented stage bindings for a `default-device-code-flow`
flow (identification → password → user-login → consent) and the
brand-level binding step.
* operations/agent-enrollment.md: Path B (interactive `netbird up`) is
now the default; Path A (setup-key) demoted to "headless/scripted"
alternative. "Install dependencies" section collapsed into a note
that bootstrap handles it, with the manual recipes kept in a
collapsible <details> block for air-gapped use.
The flow uniquely available to lab owners (single Authentik user today)
still relies on the first-time-owner sqlite promotion documented in
124-authentik.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
G1 from the apt-sweep backlog. After ssh ControlMaster setup, install
the `mcp[cli]` python package via pipx so `homelab mcp <tool>` works
out-of-the-box on new workstations. Skipped on LXCs / VMs.
Idempotent (`command -v mcp` guard), respects --dry-run, falls back
through brew (Darwin) → dnf → apt for pipx itself if not already
installed. Runs the install as $SUDO_USER (not root) so the binary
lands in the user's pipx env.
This closes one of the "discovered missing dep" gaps from the 2026-05-21
fleet sweep: republic-laptop had pipx-installed homelab CLI but no `mcp`
binary, so `homelab mcp <tool>` died with an instructional message
instead of just working.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After preflight detects MESH_CONNECTED=netbird and the host yaml says
kind != lxc (i.e. workstation, vm, proxmox-host), drop a Host block into
the enrolling user's ~/.ssh/config:
Host *.netbird.selfhosted
ControlMaster auto
ControlPath ~/.ssh/cm/%C
ControlPersist 2h
This is the real workaround for netbird's SSH JWT cache being flaky in
0.71.2 — we discovered that --ssh-jwt-cache-ttl can leave the daemon in
a state where stale cached tokens get sent and rejected with no fallback
to fresh SSO. ssh ControlMaster bypasses netbird-ssh-proxy entirely for
subsequent sessions: one SSO at the start of a working window covers all
back-to-back ssh / scp / `pct exec` ops until ControlPersist expires.
Validated 2026-05-21 on republic-laptop: ssh #1 prompted SSO once,
sshes #2 and #3 ran in ~1.1s each with no prompt.
Idempotent (sentinel comment check); writes ~/.ssh/cm/ with 700; uses
SUDO_USER's home when invoked via sudo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After preflight detects MESH_CONNECTED=netbird, run `netbird down && netbird up
--ssh-jwt-cache-ttl=86400` so ssh into mesh peers (e.g. `ssh -p 22022
root@proxmox-server.netbird.selfhosted ...`) stops triggering device-code SSO
on every connection.
Validated 2026-05-21 on republic-laptop: after one SSO, subsequent ssh
sessions within 24h skip the device-code flow and run instantly. Fleet
operations (e.g. pct exec through hubris into LXCs) reuse the cached JWT.
Notes:
- Flag is supported in netbird 0.71.x+ (netbirdio/netbird#4015). A version
probe (`netbird up --help | grep ssh-jwt-cache-ttl`) skips the section on
older clients.
- Flag belongs on `netbird up` (client config), NOT on the daemon's
ExecStart — putting it there crashes the daemon with "unknown flag".
- Runs LAST in bootstrap, after secrets issuance + MCP wiring, so the brief
mesh down/up doesn't disrupt earlier steps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End of enrollment, if secrets/gitea-pat.yaml is decryptable with the
just-issued age key (i.e. the operator has already run --finalize-pubkey
from another client), upgrade /etc/homelab-context/git-credentials from
the read-only bootstrap PAT to the write-scoped one. Best-effort: fails
silently if not yet a recipient, with a clear hint about what to do next.
Removes the post-bootstrap manual 'homelab refresh-creds' step from the
common flow; falls back to the documented one-liner for first-bootstrap
clients.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LXCs without a mesh CLI sit on 192.168.8.0/24 which is in the
issuance service's MESH_SUBNETS — they should be able to bootstrap
without netbird/tailscale installed. New third path probes the
issuance /health endpoint directly; if reachable, treat that as
satisfying the mesh precondition.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The systemd sync timer runs git without HOME set, so git config --global
(which writes /root/.gitconfig) is invisible to the timer's process — the
timer fails with 'could not read Username' silently. Switching to
--system writes to /etc/gitconfig which is HOME-agnostic.
Migration for already-bootstrapped hosts captured in agent-enrollment.md
troubleshooting.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 5-min sync pulls /opt/homelab-context but does not re-install the
CLI. A copy at /usr/local/bin/homelab therefore goes stale after every
CLI fix until someone re-runs bootstrap. Symlinking points
/usr/local/bin/homelab directly at the synced source, so updates land
on the next pull. Doc updated with the one-line migration for hosts
bootstrapped before this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caddy + split-horizon DNS now resolve these to LXC 105 (via 121).
Workstations off-LAN reach them via Netbird (192.168.8.0/24 is a
network resource routed through hubris).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Was apt-only on Linux; republic-laptop is Nobara so installs use dnf and
yaml is python3-pyyaml (not python3-yaml).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The homelab CLI imports yaml; missing on a fresh LXC. Preflight now
checks and emits the right install hint per OS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
git credential helper does exact prefix match including scheme. Hardcoding
https:// breaks for in-LAN clones using http://192.168.8.121:3000 (which
LXC 105 needs because its DNS doesn't have the split-horizon override).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each non-hubris client needs HTTPS auth against gitea for the initial
context clone (chicken-and-egg: a PAT stored in SOPS can't be fetched
until after the clone exists). Adds a --gitea-token flag that writes
credentials to /etc/homelab-context/git-credentials and points git's
credential.helper at it, so the clone and all future pulls succeed
without prompting.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the foundation for distributing homelab context to every client
(LXCs, VMs, workstations including republic-laptop, mac-mini, ludo-mini)
with a single source of truth, structured query layer (MCP), and per-client
age-key issuance for secrets:
- inventory.yaml — canonical topology (hosts, services, mesh addresses)
- hosts/*.yaml — per-host identity files generated from inventory by
mcp/build_host_files.py; do not edit by hand
- AGENTS.md — orientation doc symlinked to /root/AGENTS.md on every client
- bootstrap.sh — one-shot enroll (Linux + macOS), clones repo, fetches age
key from issuance, installs sync timer/launchd job, drops the homelab CLI
- bin/homelab — single-binary Python CLI: whoami, list, ssh, pct, logs,
restart, open, status, secret, sync, mcp, client add/remove, nuke
- mcp/server.py — FastMCP server: context tools + read-only management
tools (no mutations exposed); shell-outs use mcp-reader restricted ssh key
- mcp/deploy/ — claudio-monitor-style gitea webhook deploy scaffold for the
MCP service on LXC 105 (ports 9810 mcp, 9811 webhook)
- secrets-issuance/ — per-client age key auto-provisioning over the mesh;
source-IP gated against inventory, with denylist for revoked clients
(ports 9820 issue, 9821 webhook)
- secrets/, .sops.yaml — SOPS recipient scaffolding; the operator fills in
age public keys after Phase 3a generates them
- scripts/sync/ — systemd timer (Linux) + launchd plist (macOS) pulling
/opt/homelab-context every 5 min
Mesh: both Netbird (preferred, 100.122.0.0/16) and Tailscale accepted
during the in-flight migration; no client is gated on completing the move.
Plan reference: /root/.claude/plans/lets-make-a-plan-fluttering-trinket.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
