`bootstrap.sh --with-hermes` installs the Goose CLI, drops a Goose
config pinning the OpenRouter provider + Nous Hermes model + the
homelab MCP extension, symlinks `bin/hermes` and HERMES.md, and links
HERMES.md as `.goosehints` so the persona is injected as the system
prompt every session.
`bin/hermes` decrypts `secrets/openrouter-api-key.yaml` via the existing
`homelab secret` flow and execs `goose session`.
`homelab client add --with-hermes` grants the new sops secret to the
host's age_pubkey at finalize time (parallel to the existing
shared-secrets grant). `client remove` revokes it.
`operations/hermes-agent.md` covers the end-to-end flow, verification,
troubleshooting, and queues one follow-up: the MCP server still runs
SSE-only but Goose 1.x deprecated SSE — the Goose config targets
`streamable_http` and the `homelab` extension won't connect until
`mcp/server.py` migrates. The `developer` extension (shell + edit +
`homelab` CLI) carries the agent in the meantime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coordinated changes addressing follow-ups from the 2026-05-21 netbird vanilla
migration, plus a related off-LAN ergonomics fix:
bin/homelab:
- New ssh_target(name, force_mesh=False) helper resolves (addr, port, user)
from inventory, honoring ssh.netbird_port (forces mesh path) and ssh.user
(default "root"). Falls back to the netbird FQDN when LAN IP fails a
cached 1.5s TCP probe — helps off-LAN operators on VPN/symmetric-NAT
paths where the netbird subnet route doesn't reach 192.168.8.0/24.
- New ssh_base() builds the full `ssh ... user@addr` invocation; hubris_ssh()
is now a back-compat shim. cmd_ssh, cmd_logs, cmd_restart, cmd_nuke
refactored to use it — no more hardcoded "root@" anywhere.
inventory.yaml:
- New ssh.user convention (root by default, explicit per workstation).
- hubris.ssh.user=root (explicit, documents convention).
- republic-laptop, mac-mini: ssh.user=dtoro. ludo-mini left default (TODO).
- Comment block in the header explains the field + why it exists (netbird-ssh
defaults to LOCAL username; "user not found" on LXCs is the gotcha).
- hosts/*.yaml regenerated from build_host_files.py.
infrastructure/mesh.md:
- Migration changelog entry updated: Device Code Stage is now configured
(was "NOT yet" — landed in d41d73f); --setup-key no longer required.
- New subsection documenting the post-migration JWT-issuer cache bug:
client/internal/engine_ssh.go's updateSSH() bails out when sshServer is
already running, so systemctl restart and netbird down/up don't refresh
the SSH JWT validator. Full daemon stop/start is the fix.
- Companion username gotcha (`netbird ssh` defaulting to local username).
operations/agent-enrollment.md:
- Three new troubleshooting rows: JWT-issuer cache, user-not-found, and
LAN-unreachable-from-mesh-peer (the new homelab CLI behavior).
Verification: ssh_target resolution against the live inventory yields
- hubris → ssh -p 22022 root@proxmox-server.netbird.selfhosted (mesh-forced)
- jellyfin/gitea → ssh root@192.168.8.x (LAN reachable, probe passed)
- republic-laptop/mac-mini → ssh dtoro@<fqdn> (per ssh.user)
- ludo-mini → ssh root@<fqdn> (default)
Two changes that together collapse new-workstation onboarding from ~7 steps
to ~2 commands:
* bootstrap.sh:
- Dep-check now AUTO-INSTALLS missing tools (apt/dnf/brew) instead of
printing instructions and exiting. Re-verifies after install.
- New pre-mesh-check block: if netbird isn't installed, installs it
from the netbird apt/dnf repo (or `brew install --cask netbird` on
Darwin), then if mgmt isn't connected, runs `netbird up
--management-url=https://netbird.hubris.network --ssh-jwt-cache-ttl 86400`.
Operator clicks the device-code URL inline. Waits up to ~30s for
Management: Connected before continuing. Skipped on --no-secrets +
--dry-run.
* containers/124-authentik.md: replaces the "KNOWN MISSING — Device Code
Stage" subsection with a working recipe — Authentik 2026.2 routes
/device via a BRAND-level "Device code flow" field, not a provider
field. Documented stage bindings for a `default-device-code-flow`
flow (identification → password → user-login → consent) and the
brand-level binding step.
* operations/agent-enrollment.md: Path B (interactive `netbird up`) is
now the default; Path A (setup-key) demoted to "headless/scripted"
alternative. "Install dependencies" section collapsed into a note
that bootstrap handles it, with the manual recipes kept in a
collapsible <details> block for air-gapped use.
The flow uniquely available to lab owners (single Authentik user today)
still relies on the first-time-owner sqlite promotion documented in
124-authentik.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates to four pages reflecting the combined → vanilla mgmt+signal+relay+coturn
cutover and the IONOS-3478-firewall-exception discovery:
* infrastructure/mesh.md — rewrites the ICE/STUN section to cover the new
TURN endpoint, the IONOS upstream TCP-3478 filtering (load-bearing,
undocumented before today), and the verification probe. New changelog
entry covering the migration outcome + Device Code Stage gap.
* infrastructure/vps-hardening.md — "At a glance" lists the new 6-service
docker stack + host coturn. Firewall section notes the new
`iifname ens6 tcp dport 3478 accept` rule plus the IONOS upstream
exception. New changelog entry.
* containers/124-authentik.md — replaces the "Netbird IdP integration —
DEFERRED" section with the LANDED state: Provider details (Public
client type — Confidential breaks PKCE on the dashboard SPA), the
first-time owner-promotion sqlite recipe, the missing Device Code
Stage gap + workaround (setup-keys), and a note that the old 2026-04-22
pre-work Provider/App is now obsolete and safe to delete. Updated
changelog (Phase 6 landed).
* operations/agent-enrollment.md — new "Getting onto Netbird" subsection
explaining the setup-key path (currently the only working flow until
Device Code Stage lands) and why direct OIDC from the public internet
fails (auth.hubris.network is mesh-only-reachable). Prerequisites table
row updated to point at the new section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bundles the documentation slice of the apt-sweep backlog:
* operations/runbook-dpkg-interrupted.md (NEW) — Path A (ssh-reachable
recovery) + Path B (PVE web Shell when the netbird mesh broke
alongside the dpkg state, as happened during Wave 6 on hubris).
Closes B2.
* operations/commands.md — new "Fleet apt operations" section
documenting `homelab apt-audit` and `homelab apt-upgrade`
(--status / --safe / --force). Adds the dpkg-interrupted runbook to
Related.
* operations/agent-enrollment.md —
- new "Claude Code permissions for fleet ops" section with the
`permissions.allow` snippet (`Bash(ssh -p 22022 *)`,
`Bash(homelab *)`) for `~/.claude/settings.json`. Closes A3.
- two new Troubleshooting rows: chat-mode `!` sudo no-tty gotcha
(G2) and the cosmetic netbird DNS-probe warning.
* infrastructure/auto-deploy.md — new "Custom-built binaries that
overlap apt-managed paths" section describing the two acceptable
patterns (epoch-versioned .deb à la caddy 1:2.11.3-hubris1; or
apt-mark hold) and the discovery path via `homelab apt-audit`'s
NONAPT column. Closes D3.
Remaining backlog after this commit: A4 (upstream OpenSSH/netbird mux
bug), D1 (apt-mark hold caddy in caddy-conf bootstrap — superseded
in practice by the epoch .deb), E1/E2 (LXC DNS fallback for
tailscale-managed resolv.conf), F1/F2 (vzdump fallback doc; F3 already
shipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The systemd sync timer runs git without HOME set, so git config --global
(which writes /root/.gitconfig) is invisible to the timer's process — the
timer fails with 'could not read Username' silently. Switching to
--system writes to /etc/gitconfig which is HOME-agnostic.
Migration for already-bootstrapped hosts captured in agent-enrollment.md
troubleshooting.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 5-min sync pulls /opt/homelab-context but does not re-install the
CLI. A copy at /usr/local/bin/homelab therefore goes stale after every
CLI fix until someone re-runs bootstrap. Symlinking points
/usr/local/bin/homelab directly at the synced source, so updates land
on the next pull. Doc updated with the one-line migration for hosts
bootstrapped before this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the full enrollment flow validated during Phase 2 rollout: per-OS
dep install (dnf/apt/brew), Gitea PAT prerequisite, DNS gotchas, the
bootstrap command, post-bootstrap verification, the homelab client add
ceremony for new inventory entries, secret grant/revoke, and a
troubleshooting table mapping every failure mode we hit during validation
to the commit that fixed it.
Linked from README under Operations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
