oikos/infrastructure/ssh-access.md

# SSH access

How to reach every host in the fleet from any workstation, with LAN as
the primary path and Netbird as the automatic backup.

## Architecture

SSH access relies on three layers:

1. **Homelab inventory (`inventory.yaml`)** — the single source of truth
   for every host's LAN IP, Netbird addresses, SSH user, and port.
2. **Key distribution (`ssh/deploy-keys.sh`)** — deploys workstation SSH
   public keys to hubris and every running LXC, so any key-authorized
   workstation can log in anywhere.
3. **Config generation (`homelab ssh-config --install`)** — generates
   `~/.ssh/config.d/homelab` with short hostname aliases for every host,
   using LAN IPs (routed via Netbird's `192.168.8.0/24` subnet route when
   off-LAN) with Netbird FQDN fallbacks (`<name>-mesh`) for roaming
   workstations.

### How it works

- **From on-LAN:** `ssh gitea` resolves to `192.168.8.121` directly.
- **From off-LAN (Netbird):** The same `192.168.8.121` works because
  hubris routes the `192.168.8.0/24` subnet through Netbird.
- **Roaming workstations:** `ssh mac-mini-mesh` or `ssh republic-laptop-mesh`
  uses the Netbird FQDN as a fallback when the workstation is off its
  home subnet.

The `homelab ssh <host>` CLI command also has built-in LAN probing:
it tries a 1.5s TCP connect to the LAN IP, and if that fails, falls
back to the Netbird FQDN.

## Key distribution

Each workstation's SSH public key lives in the repo at:
`ssh/authorized_keys/<hostname>.pub`

To deploy or re-deploy all workstation keys to hubris + every running LXC:

```bash
# From hubris (or via homelab pct):
sudo bash /opt/homelab-context/ssh/deploy-keys.sh

# Or from any workstation:
ssh root@192.168.8.77 "bash /opt/homelab-context/ssh/deploy-keys.sh"
```

This script:
- Reads all `.pub` files from `ssh/authorized_keys/`
- Adds any missing keys to `/etc/pve/priv/authorized_keys` on hubris
- For each running LXC, appends keys to `/root/.ssh/authorized_keys`
- Is idempotent — skips keys already present

## Config generation

To generate the SSH config on any workstation:

```bash
homelab ssh-config --install
```

This writes to `~/.ssh/config.d/homelab` and ensures
`Include ~/.ssh/config.d/homelab` is present in `~/.ssh/config`.

The config is regenerated automatically on every `homelab sync` (which
kicks the 5-minute context sync timer).

## Adding a new workstation

When onboarding a new machine:

1. Hostname must match an entry in `inventory.yaml`.
2. If the workstation will be on the LAN, add its `lan_ip` to
   `inventory.yaml` and push. This gives it a primary LAN entry in the
   generated SSH config.
3. Enable SSH Remote Login:
   - **macOS:** `sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist`
   - **Linux:** `sudo systemctl enable --now sshd`
4. Generate an SSH keypair if one doesn't exist:
   ```bash
   ssh-keygen -t ed25519 -a 100
   ```
5. Publish the public key to the repo:
   ```bash
   cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/<hostname>.pub
   cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add <hostname> pubkey' && git push
   ```
6. Deploy the key to all hosts:
   ```bash
   ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
   ```
7. Generate the local SSH config:
   ```bash
   homelab ssh-config --install
   ```

## Hosts

### Hubris (PVE host)

| Detail | Value |
|--------|-------|
| LAN IP | `192.168.8.77` |
| Netbird | `100.122.38.109` (FQDN: `proxmox-server.netbird.selfhosted`) |
| Netbird SSH port | `22022` (mesh-only, OIDC auth) |
| SSH user | `root` |
| Authorized keys | `/etc/pve/priv/authorized_keys` (Proxmox cluster-synced) |

Authorized root keys currently deployed:
- `root@hubris` (self, RSA)
- `d.toro.v@pm.me` (ed25519) — mac-mini

### LXCs

Every LXC at `192.168.8.x` accepts root SSH via authorized_keys. Keys
are managed by `ssh/deploy-keys.sh`. SSH user is `root`.

| LXC | Name | LAN IP | Role |
|-----|------|--------|------|
| 101 | jellyfin | `192.168.8.206` | media-server |
| 102 | nfs-export | `192.168.8.200` | storage-export |
| 103 | paperless | `192.168.8.130` | document-archive |
| 104 | gitea | `192.168.8.121` | git-server |
| 105 | apps | `192.168.8.205` | docker-apps |
| 106 | auth-outpost | `192.168.8.184` | authentik-outpost |
| 107 | dns | `192.168.8.185` | dns-helper |
| 114 | nextcloud | `192.168.8.224` | file-sync |
| 118 | elementsynapse | `192.168.8.239` | matrix-server |
| 119 | sophia | `192.168.8.157` | workshop |
| 120 | mule-images | `192.168.8.136` | photo-management |
| 121 | caddy | `192.168.8.175` | reverse-proxy |
| 122 | arriman | `192.168.8.132` | arr-stack |
| 123 | claudio-bot | `192.168.8.230` | matrix-agent |
| 126 | plato | `192.168.8.190` | app |

### Workstations

| Name | OS | LAN IP | Netbird FQDN | SSH user |
|------|----|--------|--------------|----------|
| mac-mini | macOS | `192.168.8.174` | `mac-mini-234-17.netbird.selfhosted` | `dtoro` |
| republic-laptop | Linux | TBD | `republic-laptop.netbird.selfhosted` | `dtoro` |
| ludo-mini | Linux | `192.168.8.133` | `ludo-mini.netbird.selfhosted` | TBD |

### VPS (external)

| Detail | Value |
|--------|-------|
| Public IP | `82.165.190.79` |
| Netbird | `100.122.165.149` (FQDN: `netbird-ionos.netbird.selfhosted`) |
| SSH user | `root` |
| Access | Mesh-only — public port 22 is blocked by nftables. Key-only auth. |

## VPS

Access is mesh-only. From a mesh-connected peer:

```bash
ssh root@100.122.165.149
ssh root@netbird-ionos.netbird.selfhosted
# or via homelab:
homelab ssh netbird-vps
```

## Verification

```bash
# From any workstation after running homelab ssh-config --install:
for name in hubris gitea apps sophia paperless caddy jellyfin nextcloud; do
  ssh -o BatchMode=yes "$name" "hostname" && echo "$name OK"
done
```

## Related

- [Mesh migration](mesh.md)
- [VPS hardening](vps-hardening.md)
- [Agent enrollment](../operations/agent-enrollment.md)
- [Homelab CLI](../bin/homelab)

## Changelog

### 2026-06-02 — universal SSH reachability

Replaced ad-hoc per-workstation SSH configs with inventory-generated
configs (`ssh/gen-config.py`, `homelab ssh-config`). Added centralized
key distribution (`ssh/deploy-keys.sh`, `ssh/authorized_keys/`). All
LXCs now accept root SSH from any workstation whose pubkey is in the
repo. mac-mini Remote Login enabled. Netbird subnet route
(192.168.8.0/24 via hubris) provides off-LAN reachability for all LAN
IPs.

### 2026-04-28 — wiki entry created
Initial documentation.

### 2026-04-23 — VPS SSH hardened to mesh-only
Public `:22` blocked at nftables. Key-only sshd.

### 2026-04-22 — iMac key authorized on hubris
`d.toro.v@pm.me` added to `/etc/pve/priv/authorized_keys`.