dtoro/oikos
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

docs: update ssh-access.md + agent-enrollment.md with universal SSH setup

Browse Source
This commit is contained in:
authentik Default Admin
2026-06-02 00:20:28 +02:00
parent bebebf486a
commit c299409431
2 changed files with 254 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

199
infrastructure/ssh-access.md
View File

@@ -1,45 +1,200 @@
# SSH access
How to reach hubris and the VPS over SSH, and the dual-server gotcha.
How to reach every host in the fleet from any workstation, with LAN as
the primary path and Netbird as the automatic backup.
## Hubris
## Architecture
Two SSH endpoints — easy to hit the wrong one.
SSH access relies on three layers:
| Server | Listen | Auth | Notes |
| -------------- | ---------------------------- | --------------------------------- | ----- |
| OpenSSH | `0.0.0.0:22` | `authorized_keys` at `/etc/pve/priv/authorized_keys` (Proxmox cluster-synced; symlinked from `/root/.ssh/authorized_keys`) | Standard. |
| Netbird SSH | `100.122.38.109:22022` | OIDC / browser auth — bypasses `authorized_keys` | If a client lands here it'll open a browser tab to authenticate, then sometimes hang. Force port 22 or use the LAN IP. |
1. **Homelab inventory (`inventory.yaml`)** — the single source of truth
for every host's LAN IP, Netbird addresses, SSH user, and port.
2. **Key distribution (`ssh/deploy-keys.sh`)** — deploys workstation SSH
public keys to hubris and every running LXC, so any key-authorized
workstation can log in anywhere.
3. **Config generation (`homelab ssh-config --install`)** — generates
`~/.ssh/config.d/homelab` with short hostname aliases for every host,
using LAN IPs (routed via Netbird's `192.168.8.0/24` subnet route when
off-LAN) with Netbird FQDN fallbacks (`<name>-mesh`) for roaming
workstations.
### Authorized root keys
- `root@hubris` (self, RSA) — original.
- `d.toro.v@pm.me` (ed25519) — user's iMac (`mac-mini.netbird.selfhosted`, LAN `192.168.8.174`), added 2026-04-22.
### How it works
### Notes
- Password auth is enabled on hubris but the root password is **not** the one the user expects. Prefer key flows; don't try `ssh-copy-id` blind.
- Off-LAN access from the iMac uses the LAN path. As of 2026-04-22 the iMac's Netbird tunnel to hubris was P2P healthy but no packets were captured on `wt0`; needs revisit if remote access becomes critical.
- **From on-LAN:** `ssh gitea` resolves to `192.168.8.121` directly.
- **From off-LAN (Netbird):** The same `192.168.8.121` works because
hubris routes the `192.168.8.0/24` subnet through Netbird.
- **Roaming workstations:** `ssh mac-mini-mesh` or `ssh republic-laptop-mesh`
uses the Netbird FQDN as a fallback when the workstation is off its
home subnet.
## VPS (`82.165.190.79` / `100.122.165.149`)
The `homelab ssh <host>` CLI command also has built-in LAN probing:
it tries a 1.5s TCP connect to the LAN IP, and if that fails, falls
back to the Netbird FQDN.
- **Mesh-only.** Public `:22` is dropped by the nftables firewall. SSH reaches the VPS only over `wt0`.
- Key-only (`PasswordAuthentication no`, `PermitRootLogin prohibit-password`) via drop-in at `/etc/ssh/sshd_config.d/10-hubris-hardening.conf`. Original config backed up at `/etc/ssh/sshd_config.bak.<ts>`.
- `ListenAddress` is still 0.0.0.0; gating is firewall-layer.
- Authorized root keys: PVE (`root@hubris`), iMac (`d.toro.v@pm.me`). Add a new device with `ssh-copy-id root@100.122.165.149` from a mesh peer before disabling its access paths.
## Key distribution
See [VPS hardening](vps-hardening.md) for the firewall + fail2ban rules and recovery paths.
Each workstation's SSH public key lives in the repo at:
`ssh/authorized_keys/<hostname>.pub`
To deploy or re-deploy all workstation keys to hubris + every running LXC:
```bash
# From hubris (or via homelab pct):
sudo bash /opt/homelab-context/ssh/deploy-keys.sh
# Or from any workstation:
ssh root@192.168.8.77 "bash /opt/homelab-context/ssh/deploy-keys.sh"
```
This script:
- Reads all `.pub` files from `ssh/authorized_keys/`
- Adds any missing keys to `/etc/pve/priv/authorized_keys` on hubris
- For each running LXC, appends keys to `/root/.ssh/authorized_keys`
- Is idempotent — skips keys already present
## Config generation
To generate the SSH config on any workstation:
```bash
homelab ssh-config --install
```
This writes to `~/.ssh/config.d/homelab` and ensures
`Include ~/.ssh/config.d/homelab` is present in `~/.ssh/config`.
The config is regenerated automatically on every `homelab sync` (which
kicks the 5-minute context sync timer).
## Adding a new workstation
When onboarding a new machine:
1. Hostname must match an entry in `inventory.yaml`.
2. If the workstation will be on the LAN, add its `lan_ip` to
`inventory.yaml` and push. This gives it a primary LAN entry in the
generated SSH config.
3. Enable SSH Remote Login:
- **macOS:** `sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist`
- **Linux:** `sudo systemctl enable --now sshd`
4. Generate an SSH keypair if one doesn't exist:
```bash
ssh-keygen -t ed25519 -a 100
```
5. Publish the public key to the repo:
```bash
cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/<hostname>.pub
cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add <hostname> pubkey' && git push
```
6. Deploy the key to all hosts:
```bash
ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
```
7. Generate the local SSH config:
```bash
homelab ssh-config --install
```
## Hosts
### Hubris (PVE host)
| Detail | Value |
|--------|-------|
| LAN IP | `192.168.8.77` |
| Netbird | `100.122.38.109` (FQDN: `proxmox-server.netbird.selfhosted`) |
| Netbird SSH port | `22022` (mesh-only, OIDC auth) |
| SSH user | `root` |
| Authorized keys | `/etc/pve/priv/authorized_keys` (Proxmox cluster-synced) |
Authorized root keys currently deployed:
- `root@hubris` (self, RSA)
- `d.toro.v@pm.me` (ed25519) — mac-mini
### LXCs
Every LXC at `192.168.8.x` accepts root SSH via authorized_keys. Keys
are managed by `ssh/deploy-keys.sh`. SSH user is `root`.
| LXC | Name | LAN IP | Role |
|-----|------|--------|------|
| 101 | jellyfin | `192.168.8.206` | media-server |
| 102 | nfs-export | `192.168.8.200` | storage-export |
| 103 | paperless | `192.168.8.130` | document-archive |
| 104 | gitea | `192.168.8.121` | git-server |
| 105 | apps | `192.168.8.205` | docker-apps |
| 106 | auth-outpost | `192.168.8.184` | authentik-outpost |
| 107 | dns | `192.168.8.185` | dns-helper |
| 114 | nextcloud | `192.168.8.224` | file-sync |
| 118 | elementsynapse | `192.168.8.239` | matrix-server |
| 119 | sophia | `192.168.8.157` | workshop |
| 120 | mule-images | `192.168.8.136` | photo-management |
| 121 | caddy | `192.168.8.175` | reverse-proxy |
| 122 | arriman | `192.168.8.132` | arr-stack |
| 123 | claudio-bot | `192.168.8.230` | matrix-agent |
| 126 | plato | `192.168.8.190` | app |
### Workstations
| Name | OS | LAN IP | Netbird FQDN | SSH user |
|------|----|--------|--------------|----------|
| mac-mini | macOS | `192.168.8.174` | `mac-mini-234-17.netbird.selfhosted` | `dtoro` |
| republic-laptop | Linux | TBD | `republic-laptop.netbird.selfhosted` | `dtoro` |
| ludo-mini | Linux | `192.168.8.133` | `ludo-mini.netbird.selfhosted` | TBD |
### VPS (external)
| Detail | Value |
|--------|-------|
| Public IP | `82.165.190.79` |
| Netbird | `100.122.165.149` (FQDN: `netbird-ionos.netbird.selfhosted`) |
| SSH user | `root` |
| Access | Mesh-only — public port 22 is blocked by nftables. Key-only auth. |
## VPS
Access is mesh-only. From a mesh-connected peer:
```bash
ssh root@100.122.165.149
ssh root@netbird-ionos.netbird.selfhosted
# or via homelab:
homelab ssh netbird-vps
```
## Verification
```bash
# From any workstation after running homelab ssh-config --install:
for name in hubris gitea apps sophia paperless caddy jellyfin nextcloud; do
ssh -o BatchMode=yes "$name" "hostname" && echo "$name OK"
done
```
## Related
- [Hubris host](../hosts/hubris.md)
- [Mesh migration](mesh.md)
- [VPS hardening](vps-hardening.md)
- [Agent enrollment](../operations/agent-enrollment.md)
- [Homelab CLI](../bin/homelab)
## Changelog
### 2026-06-02 — universal SSH reachability
Replaced ad-hoc per-workstation SSH configs with inventory-generated
configs (`ssh/gen-config.py`, `homelab ssh-config`). Added centralized
key distribution (`ssh/deploy-keys.sh`, `ssh/authorized_keys/`). All
LXCs now accept root SSH from any workstation whose pubkey is in the
repo. mac-mini Remote Login enabled. Netbird subnet route
(192.168.8.0/24 via hubris) provides off-LAN reachability for all LAN
IPs.
### 2026-04-28 — wiki entry created
Initial documentation.
### 2026-04-23 — VPS SSH hardened to mesh-only
Public `:22` blocked at nftables. Key-only sshd. See [VPS hardening](vps-hardening.md).
Public `:22` blocked at nftables. Key-only sshd.
### 2026-04-22 — iMac key authorized on hubris
`d.toro.v@pm.me` added to `/etc/pve/priv/authorized_keys`.
`d.toro.v@pm.me` added to `/etc/pve/priv/authorized_keys`.

77
operations/agent-enrollment.md
View File

@@ -163,6 +163,77 @@ For Claude Code: start a new session — the `homelab` MCP server appears
in `~/.claude/.mcp.json` and registers 14 tools (8 context, 5 management,
1 secrets-metadata).
## Post-bootstrap: SSH reachability
A new workstation must be reachable from other workstations and must be
able to reach every host by short hostname. Run these steps after the
bootstrap verify passes:
### 1. Enable SSH server
```bash
# macOS:
sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
# Linux:
sudo systemctl enable --now sshd
```
### 2. Generate SSH key (if missing)
```bash
ls ~/.ssh/id_ed25519.pub 2>/dev/null || ssh-keygen -t ed25519 -a 100
```
### 3. Publish pubkey to the repo
```bash
cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/$(hostname -s).pub
cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add $(hostname -s) pubkey' && git push
```
### 4. Deploy keys to all hosts
From any existing enrolled machine (hubris or another workstation):
```bash
ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
```
This adds the new workstation's pubkey to hubris and every running LXC.
### 5. Generate SSH config
```bash
homelab ssh-config --install
```
Verify:
```bash
ssh hubris hostname # should return "hubris" without password
ssh gitea hostname # should return "gitea" without password
ssh mac-mini hostname # should return "mac-mini" without password (workstation-to-workstation)
```
### 6. Add LAN IP to inventory (if on LAN)
If the workstation has a static or reserved LAN IP, add it to
`inventory.yaml`:
```yaml
hosts:
your-hostname:
lan_ip: 192.168.8.xxx
```
This gives it a primary LAN entry in the generated SSH config (faster
than the Netbird fallback). Commit + push, then:
```bash
cd /opt/homelab-context && git pull --ff-only && homelab ssh-config --install
```
## Claude Code permissions for fleet ops
By default Claude Code's auto-mode classifier asks for confirmation on every
@@ -278,6 +349,12 @@ The CLI prints a follow-up checklist that the operator must do manually:
## Changelog
### 2026-06-02 — SSH reachability post-bootstrap steps
Added a new "Post-bootstrap: SSH reachability" section covering SSH key
generation, pubkey publication, deployment to hosts, SSH config generation,
and LAN IP registration. New workstations enrolled via this doc will
automatically join the universal SSH mesh.
### 2026-05-31 — cross-link to hermes-agent.md
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([hermes-agent.md](./hermes-agent.md)) and noted it at the top of this page. The Hermes flow extends `bootstrap.sh` with `--with-hermes` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.