Files
oikos/knowledge/wiki/infrastructure/ssh-access.md
dtoro 8a6422bd7d docs: move narrative wiki under knowledge/wiki/ (phase 3)
Problem: node and cross-cutting narratives lived at the repo root
(containers/, vms/, infrastructure/, host .md files), interleaved with the
machine-readable substrate.

Change:
- Move containers/ -> knowledge/wiki/containers/, vms/ -> knowledge/wiki/vms/,
  infrastructure/ -> knowledge/wiki/infrastructure/, hosts/{hubris,strong}.md ->
  knowledge/wiki/hosts/, infrastructure/references/ -> knowledge/sources/references/,
  GLOSSARY.md -> knowledge/GLOSSARY.md.
- Add knowledge/{index.md,log.md,sources/index.md} scaffolding.
- Rewrite all relative links repo-wide via a path-resolving mapper (inbound +
  outbound + between-moved-files), including .hermes/, runbooks, operations,
  investigations, plans, README, AGENTS.
- Repoint inventory.yaml doc_page fields and regenerate hosts/*.yaml (which
  embed doc_page); update oikos/gen-topology.py output path, candidate doc
  paths, and footer links; update code-comment doc paths.

Substrate untouched in place: inventory.yaml, hosts/*.yaml (regenerated,
idempotent), oikos/ code, mcp/, secrets/, bin/.

Verification:
- Logical broken-link set identical to pre-move baseline (net 128 -> 127; the
  topology regen fixed one, introduced none). Remaining are pre-existing refs
  to destroyed/archived nodes, out of scope for this move.
- gen-topology.py --check exit 0 (in sync); cards carry knowledge/wiki/ doc paths.
- build_host_files.py idempotent; all inventory doc_page targets resolve.
- MCP contract verified: get_page/search_docs/get_changelog resolve moved pages.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:35:23 +02:00

7.3 KiB

SSH access

How to reach every host in the fleet from any workstation, with LAN as the primary path and Netbird as the automatic backup.

Architecture

SSH access relies on three layers:

  1. Homelab inventory (inventory.yaml) — the single source of truth for every host's LAN IP, Netbird addresses, SSH user, and port.
  2. Key distribution (ssh/deploy-keys.sh) — deploys workstation SSH public keys to hubris and every running LXC, so any key-authorized workstation can log in anywhere.
  3. Config generation (homelab ssh-config --install) — generates ~/.ssh/config.d/homelab with short hostname aliases for every host, using LAN IPs (routed via Netbird's 192.168.8.0/24 subnet route when off-LAN) with Netbird FQDN fallbacks (<name>-mesh) for roaming workstations.

How it works

  • From on-LAN: ssh gitea resolves to 192.168.8.121 directly.
  • From off-LAN (Netbird): The same 192.168.8.121 works because hubris routes the 192.168.8.0/24 subnet through Netbird.
  • Roaming workstations: ssh mac-mini-mesh or ssh republic-laptop-mesh uses the Netbird FQDN as a fallback when the workstation is off its home subnet.

The homelab ssh <host> CLI command also has built-in LAN probing: it tries a 1.5s TCP connect to the LAN IP, and if that fails, falls back to the Netbird FQDN.

Key distribution

Each workstation's SSH public key lives in the repo at: ssh/authorized_keys/<hostname>.pub

To deploy or re-deploy all workstation keys to hubris + every running LXC:

# From hubris (or via homelab pct):
sudo bash /opt/homelab-context/ssh/deploy-keys.sh

# Or from any workstation:
ssh root@192.168.8.77 "bash /opt/homelab-context/ssh/deploy-keys.sh"

This script:

  • Reads all .pub files from ssh/authorized_keys/
  • Adds any missing keys to /etc/pve/priv/authorized_keys on hubris
  • For each running LXC, appends keys to /root/.ssh/authorized_keys
  • Is idempotent — skips keys already present

Config generation

To generate the SSH config on any workstation:

homelab ssh-config --install

This writes to ~/.ssh/config.d/homelab and ensures Include ~/.ssh/config.d/homelab is present in ~/.ssh/config.

The config is regenerated automatically on every homelab sync (which kicks the 5-minute context sync timer).

Adding a new workstation

When onboarding a new machine:

  1. Hostname must match an entry in inventory.yaml.
  2. If the workstation will be on the LAN, add its lan_ip to inventory.yaml and push. This gives it a primary LAN entry in the generated SSH config.
  3. Enable SSH Remote Login:
    • macOS: sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
    • Linux: sudo systemctl enable --now sshd
  4. Generate an SSH keypair if one doesn't exist:
    ssh-keygen -t ed25519 -a 100
    
  5. Publish the public key to the repo:
    cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/<hostname>.pub
    cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add <hostname> pubkey' && git push
    
  6. Deploy the key to all hosts:
    ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
    
  7. Generate the local SSH config:
    homelab ssh-config --install
    

Hosts

Hubris + strong (PVE cluster: Homelab)

Both nodes share /etc/pve/priv/authorized_keys — it's Proxmox cluster-synced, so a key added on either node is authorized on both.

Detail hubris strong
LAN IP 192.168.8.77 192.168.178.181
Cluster node name hubris strong (OS hostname kept as-is from install)
Netbird 100.122.38.109 (proxmox-server.netbird.selfhosted) not enrolled yet
Netbird SSH port 22022 (mesh-only, OIDC auth) n/a
SSH user root root

Authorized root keys currently deployed (cluster-wide):

  • root@hubris (self, RSA)
  • d.toro.v@pm.me (ed25519) — mac-mini
  • root@strong (RSA) — strong's own key, added 2026-07-01 for the cluster join

LXCs

Every LXC at 192.168.8.x accepts root SSH via authorized_keys. Keys are managed by ssh/deploy-keys.sh. SSH user is root.

LXC Name LAN IP Role
101 jellyfin 192.168.8.206 media-server
102 nfs-export 192.168.8.200 storage-export
103 paperless 192.168.8.130 document-archive
104 gitea 192.168.8.121 git-server
105 apps 192.168.8.205 docker-apps
106 auth-outpost 192.168.8.184 authentik-outpost
107 dns 192.168.8.185 dns-helper
114 nextcloud 192.168.8.224 file-sync
118 elementsynapse 192.168.8.239 matrix-server
119 sophia 192.168.8.157 workshop
120 mule-images 192.168.8.136 photo-management
121 caddy 192.168.8.175 reverse-proxy
122 arriman 192.168.8.132 arr-stack

Workstations

Name OS LAN IP Netbird FQDN SSH user
mac-mini macOS 192.168.8.174 mac-mini-234-17.netbird.selfhosted dtoro
republic-laptop Linux TBD republic-laptop.netbird.selfhosted dtoro

strong moved out of this table 2026-07-01 — it's a Proxmox host now, see the cluster table above.

VPS (external)

Detail Value
Public IP 82.165.190.79
Netbird 100.122.165.149 (FQDN: netbird-ionos.netbird.selfhosted)
SSH user root
Access Mesh-only — public port 22 is blocked by nftables. Key-only auth.

VPS

Access is mesh-only. From a mesh-connected peer:

ssh root@100.122.165.149
ssh root@netbird-ionos.netbird.selfhosted
# or via homelab:
homelab ssh netbird-vps

Verification

# From any workstation after running homelab ssh-config --install:
for name in hubris gitea apps sophia paperless caddy jellyfin nextcloud; do
  ssh -o BatchMode=yes "$name" "hostname" && echo "$name OK"
done

Changelog

2026-07-01 — strong reformatted to Proxmox, joined cluster; table corrected

strong moved from the Workstations table to the PVE-cluster table (was showing a stale 192.168.8.133, never actually reachable — the real LAN IP has always been 192.168.178.181, matching hosts/strong.yaml). Root key access bootstrapped via one-time console password, then key-only going forward. See hosts/hubris.md#cluster and hosts/strong.md.

2026-06-02 — universal SSH reachability

Replaced ad-hoc per-workstation SSH configs with inventory-generated configs (ssh/gen-config.py, homelab ssh-config). Added centralized key distribution (ssh/deploy-keys.sh, ssh/authorized_keys/). All LXCs now accept root SSH from any workstation whose pubkey is in the repo. mac-mini Remote Login enabled. Netbird subnet route (192.168.8.0/24 via hubris) provides off-LAN reachability for all LAN IPs.

2026-04-28 — wiki entry created

Initial documentation.

2026-04-23 — VPS SSH hardened to mesh-only

Public :22 blocked at nftables. Key-only sshd.

2026-04-22 — iMac key authorized on hubris

d.toro.v@pm.me added to /etc/pve/priv/authorized_keys.