Merge branch 'main' of git-ssh.hubris.network:dtoro/Homelab-Docs
# Conflicts: # .sops.yaml # containers/107-dns.md # infrastructure/dns.md
This commit is contained in:
@@ -91,6 +91,9 @@ The "delete NetBird managed zone → forward everything to Technitium" plan was
|
||||
|
||||
> Reference: [scripts/dns-sync.py](../scripts/dns-sync.py). The sync's source of truth is Technitium; it **deletes** NetBird records absent from Technitium (so obsolete names like `files`, `photos-new` get reaped).
|
||||
|
||||
### 2026-06-02 — 8 LXCs moved from DHCP to static IP
|
||||
All LXCs that Caddy reverse-proxies to by IP were on `ip=dhcp` and could float on reboot (arriman got a different lease mid-session and broke). Fixed via `pct set` + in-LXC `/etc/network/interfaces`. Affected: 101 jellyfin, 103 paperless, 104 gitea, 105 apps, 114 nextcloud, 118 elementsynapse, 120 mule-images, 121 caddy, 122 arriman. See [arriman changelog](../containers/122-arriman.md#changelog).
|
||||
|
||||
### 2026-06-01 — dnsmasq replaced by Technitium on [dns (107)](../containers/107-dns.md); LXC 124 retired
|
||||
Split-horizon DNS moved off [124](../containers/124-authentik.md) to a dedicated **Technitium** LXC at **`192.168.8.2`** (zone: specific A overrides + wildcard→VPS + replicated MX/SPF/CAA). NetBird `home-lab-dns` nameserver group cut over to `192.168.8.2` (with `.180` as a now-dead fallback). dnsmasq stopped, all names verified via Technitium, **LXC 124 shut down**. **Caveat:** the [NetBird managed DNS zone](../containers/124-authentik.md) still answers most app names *directly* (bypassing the nameserver group) — three overlapping DNS sources remain; see the single-source-of-truth decision (Phase 4). **Action needed:** update router DHCP DNS from the dead `.180` → `192.168.8.2` for any plain-LAN (non-mesh) clients.
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ page is the architecture reference.
|
||||
| Per-client clone | every enrolled client | `/opt/homelab-context/` | Read by `homelab` CLI, MCP server, claudio-bot |
|
||||
| `homelab` CLI | every enrolled client | `/usr/local/bin/homelab` → `/opt/homelab-context/bin/homelab` (symlink) | Operator surface for enroll/secret/ssh/pct |
|
||||
| Per-client age key | every enrolled client | `/etc/age/key.txt` (0600 root) | Decrypts SOPS-encrypted secrets the client is a recipient on |
|
||||
| MCP server | [apps (105)](../containers/105-apps.md) | `homelab-mcp.service` on port 9810 (https://mcp.hubris.network/sse) | 14 tools: 8 context (get_host, search_docs, …) + 5 read-only management (get_service_status, tail_log, …) + list_my_secrets |
|
||||
| MCP server | [apps (105)](../containers/105-apps.md) | `homelab-mcp.service` on port 9810 (https://mcp.hubris.network/mcp) | 14 tools: 8 context (get_host, search_docs, …) + 5 read-only management (get_service_status, tail_log, …) + list_my_secrets |
|
||||
| Secrets-issuance | [apps (105)](../containers/105-apps.md) | `secrets-issuance.service` on port 9820 (https://secrets.hubris.network/issue) | Generates per-client age keypair on first bootstrap; idempotent; admin-token-gated `/revoke` |
|
||||
| Sync timer | every enrolled client | `homelab-context-sync.timer` (Linux) / `network.hubris.homelab-context-sync.plist` (macOS) | `git pull --ff-only` every 5 min |
|
||||
| Encrypted secrets | `dtoro/Homelab-Docs` | `secrets/*.yaml` (SOPS+age) | Recipients declared in `.sops.yaml` |
|
||||
|
||||
84
infrastructure/network.md
Normal file
84
infrastructure/network.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# Network
|
||||
|
||||
Physical and logical network topology for the homelab.
|
||||
|
||||
## Why
|
||||
|
||||
The homelab runs on a dedicated internal subnet (`192.168.8.0/24`) isolated from the main household LAN (`192.168.178.0/24`). Isolation is enforced at Proxmox: LXC/VM traffic is bridged only on the internal `vmbr0` bridge; Proxmox routes packets out to Fritz!Box via `vmbr1`. The main LAN cannot reach homelab services directly without a Fritz!Box static route (which is configured to allow inbound).
|
||||
|
||||
Fritz!OS 8.x does not support second IP networks on LAN ports, so Proxmox (`hubris`) acts as the subnet router rather than the Fritz!Box.
|
||||
|
||||
## Hardware
|
||||
|
||||
| Device | Role |
|
||||
|---|---|
|
||||
| Fritz!Box 7590 | Main router / ISP gateway (`192.168.178.1`) |
|
||||
| SODOLA 5-Port 2.5Gbit Managed | Homelab switch — flat L2, all ports native |
|
||||
| hubris (Proxmox) | Subnet router — routes between `192.168.8.0/24` and `192.168.178.0/24` |
|
||||
|
||||
## Topology
|
||||
|
||||
```
|
||||
ISP
|
||||
└── Fritz!Box 7590 (192.168.178.1)
|
||||
│ static route: 192.168.8.0/24 → 192.168.178.10
|
||||
│
|
||||
└── SODOLA 5-Port 2.5Gbit
|
||||
├── Port 1 uplink → Fritz!Box LAN
|
||||
├── Port 2 hubris eno1 → vmbr1 (192.168.178.10)
|
||||
├── Port 3 [device]
|
||||
├── Port 4 [device]
|
||||
└── Port 5 spare
|
||||
|
||||
hubris internal bridges:
|
||||
vmbr1 192.168.178.10/24 eno1 (uplink, DHCP-reserved) gateway 192.168.178.1
|
||||
vmbr0 192.168.8.77/24 no physical port (internal)
|
||||
192.168.8.1/24 alias — LXC default gateway
|
||||
├── all 16 LXCs
|
||||
└── HAOS VM
|
||||
```
|
||||
|
||||
## Subnets
|
||||
|
||||
| Subnet | Gateway | Purpose |
|
||||
|---|---|---|
|
||||
| `192.168.178.0/24` | `192.168.178.1` | Household LAN — laptops, phones, Fritz!Box DHCP |
|
||||
| `192.168.8.0/24` | `192.168.8.1` (Proxmox `vmbr0` alias) | Homelab — all LXCs and VMs |
|
||||
|
||||
## DHCP
|
||||
|
||||
- **Household (`192.168.178.x`)**: Fritz!Box built-in DHCP. Proxmox `vmbr1` has a reservation: MAC `84:47:09:6b:e7:58` → `192.168.178.10`.
|
||||
- **Homelab (`192.168.8.x`)**: Technitium on [CT 107](../containers/107-dns.md) at `192.168.8.2`. Range `192.168.8.100–192.168.8.240`, gateway `192.168.8.1`, DNS `192.168.8.2`.
|
||||
|
||||
Most homelab LXCs use static IPs below `.100`. DHCP only covers new/transient containers.
|
||||
|
||||
## DNS
|
||||
|
||||
Split-horizon DNS for `*.hubris.network` served by Technitium on [CT 107](../containers/107-dns.md) at `192.168.8.2:53`. See [dns.md](dns.md) for full detail.
|
||||
|
||||
## Routing
|
||||
|
||||
Proxmox has `net.ipv4.ip_forward=1` (already enabled by PVE). Packets from LXCs on `vmbr0` destined for the internet exit via `vmbr1` → Fritz!Box. Fritz!Box masquerades all outbound WAN traffic. Fritz!Box has a static route (`192.168.8.0/24 → 192.168.178.10`) so return traffic reaches the LXCs.
|
||||
|
||||
No NAT on Proxmox — traffic flows without double-NAT.
|
||||
|
||||
## Remote access
|
||||
|
||||
- **NetBird mesh** — primary path for remote administration. Authenticated via [Authentik on the VPS](../vps/).
|
||||
- **Tailscale** — legacy, being phased out. See [mesh.md](mesh.md).
|
||||
|
||||
## Related
|
||||
|
||||
- [DNS](dns.md) — split-horizon config and entry list
|
||||
- [Ingress](ingress.md) — public entry points via VPS traefik
|
||||
- [Mesh](mesh.md) — NetBird / Tailscale VPN overlay
|
||||
- [hosts/hubris.md](../hosts/hubris.md) — Proxmox host (vmbr0/vmbr1 config)
|
||||
- [CT 107 — dns](../containers/107-dns.md) — Technitium DNS + DHCP server
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-06-02 — Executed migration; Proxmox as subnet router
|
||||
Fritz!OS 8.x does not support second IP networks on LAN ports, so the final design uses Proxmox as the router: `vmbr1` (eno1 → SODOLA → Fritz!Box) is the uplink at `192.168.178.10`; `vmbr0` is a portless internal bridge with `192.168.8.1` alias as the LXC gateway. Technitium DHCP enabled for `192.168.8.100–240`. Caddy service unit was missing and recreated. See [migration plan](../plans/2026-06-01-slate-ax-to-sodola-migration.md).
|
||||
|
||||
### 2026-06-01 — Initial network doc; Slate AX retired; SODOLA switch added
|
||||
Replaced the GL.iNet Slate AX sub-router with the SODOLA 5-Port 2.5Gbit managed switch. Eliminated double-NAT. See [migration plan](../plans/2026-06-01-slate-ax-to-sodola-migration.md).
|
||||
@@ -1,45 +1,200 @@
|
||||
# SSH access
|
||||
|
||||
How to reach hubris and the VPS over SSH, and the dual-server gotcha.
|
||||
How to reach every host in the fleet from any workstation, with LAN as
|
||||
the primary path and Netbird as the automatic backup.
|
||||
|
||||
## Hubris
|
||||
## Architecture
|
||||
|
||||
Two SSH endpoints — easy to hit the wrong one.
|
||||
SSH access relies on three layers:
|
||||
|
||||
| Server | Listen | Auth | Notes |
|
||||
| -------------- | ---------------------------- | --------------------------------- | ----- |
|
||||
| OpenSSH | `0.0.0.0:22` | `authorized_keys` at `/etc/pve/priv/authorized_keys` (Proxmox cluster-synced; symlinked from `/root/.ssh/authorized_keys`) | Standard. |
|
||||
| Netbird SSH | `100.122.38.109:22022` | OIDC / browser auth — bypasses `authorized_keys` | If a client lands here it'll open a browser tab to authenticate, then sometimes hang. Force port 22 or use the LAN IP. |
|
||||
1. **Homelab inventory (`inventory.yaml`)** — the single source of truth
|
||||
for every host's LAN IP, Netbird addresses, SSH user, and port.
|
||||
2. **Key distribution (`ssh/deploy-keys.sh`)** — deploys workstation SSH
|
||||
public keys to hubris and every running LXC, so any key-authorized
|
||||
workstation can log in anywhere.
|
||||
3. **Config generation (`homelab ssh-config --install`)** — generates
|
||||
`~/.ssh/config.d/homelab` with short hostname aliases for every host,
|
||||
using LAN IPs (routed via Netbird's `192.168.8.0/24` subnet route when
|
||||
off-LAN) with Netbird FQDN fallbacks (`<name>-mesh`) for roaming
|
||||
workstations.
|
||||
|
||||
### Authorized root keys
|
||||
- `root@hubris` (self, RSA) — original.
|
||||
- `d.toro.v@pm.me` (ed25519) — user's iMac (`mac-mini.netbird.selfhosted`, LAN `192.168.8.174`), added 2026-04-22.
|
||||
### How it works
|
||||
|
||||
### Notes
|
||||
- Password auth is enabled on hubris but the root password is **not** the one the user expects. Prefer key flows; don't try `ssh-copy-id` blind.
|
||||
- Off-LAN access from the iMac uses the LAN path. As of 2026-04-22 the iMac's Netbird tunnel to hubris was P2P healthy but no packets were captured on `wt0`; needs revisit if remote access becomes critical.
|
||||
- **From on-LAN:** `ssh gitea` resolves to `192.168.8.121` directly.
|
||||
- **From off-LAN (Netbird):** The same `192.168.8.121` works because
|
||||
hubris routes the `192.168.8.0/24` subnet through Netbird.
|
||||
- **Roaming workstations:** `ssh mac-mini-mesh` or `ssh republic-laptop-mesh`
|
||||
uses the Netbird FQDN as a fallback when the workstation is off its
|
||||
home subnet.
|
||||
|
||||
## VPS (`82.165.190.79` / `100.122.165.149`)
|
||||
The `homelab ssh <host>` CLI command also has built-in LAN probing:
|
||||
it tries a 1.5s TCP connect to the LAN IP, and if that fails, falls
|
||||
back to the Netbird FQDN.
|
||||
|
||||
- **Mesh-only.** Public `:22` is dropped by the nftables firewall. SSH reaches the VPS only over `wt0`.
|
||||
- Key-only (`PasswordAuthentication no`, `PermitRootLogin prohibit-password`) via drop-in at `/etc/ssh/sshd_config.d/10-hubris-hardening.conf`. Original config backed up at `/etc/ssh/sshd_config.bak.<ts>`.
|
||||
- `ListenAddress` is still 0.0.0.0; gating is firewall-layer.
|
||||
- Authorized root keys: PVE (`root@hubris`), iMac (`d.toro.v@pm.me`). Add a new device with `ssh-copy-id root@100.122.165.149` from a mesh peer before disabling its access paths.
|
||||
## Key distribution
|
||||
|
||||
See [VPS hardening](vps-hardening.md) for the firewall + fail2ban rules and recovery paths.
|
||||
Each workstation's SSH public key lives in the repo at:
|
||||
`ssh/authorized_keys/<hostname>.pub`
|
||||
|
||||
To deploy or re-deploy all workstation keys to hubris + every running LXC:
|
||||
|
||||
```bash
|
||||
# From hubris (or via homelab pct):
|
||||
sudo bash /opt/homelab-context/ssh/deploy-keys.sh
|
||||
|
||||
# Or from any workstation:
|
||||
ssh root@192.168.8.77 "bash /opt/homelab-context/ssh/deploy-keys.sh"
|
||||
```
|
||||
|
||||
This script:
|
||||
- Reads all `.pub` files from `ssh/authorized_keys/`
|
||||
- Adds any missing keys to `/etc/pve/priv/authorized_keys` on hubris
|
||||
- For each running LXC, appends keys to `/root/.ssh/authorized_keys`
|
||||
- Is idempotent — skips keys already present
|
||||
|
||||
## Config generation
|
||||
|
||||
To generate the SSH config on any workstation:
|
||||
|
||||
```bash
|
||||
homelab ssh-config --install
|
||||
```
|
||||
|
||||
This writes to `~/.ssh/config.d/homelab` and ensures
|
||||
`Include ~/.ssh/config.d/homelab` is present in `~/.ssh/config`.
|
||||
|
||||
The config is regenerated automatically on every `homelab sync` (which
|
||||
kicks the 5-minute context sync timer).
|
||||
|
||||
## Adding a new workstation
|
||||
|
||||
When onboarding a new machine:
|
||||
|
||||
1. Hostname must match an entry in `inventory.yaml`.
|
||||
2. If the workstation will be on the LAN, add its `lan_ip` to
|
||||
`inventory.yaml` and push. This gives it a primary LAN entry in the
|
||||
generated SSH config.
|
||||
3. Enable SSH Remote Login:
|
||||
- **macOS:** `sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist`
|
||||
- **Linux:** `sudo systemctl enable --now sshd`
|
||||
4. Generate an SSH keypair if one doesn't exist:
|
||||
```bash
|
||||
ssh-keygen -t ed25519 -a 100
|
||||
```
|
||||
5. Publish the public key to the repo:
|
||||
```bash
|
||||
cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/<hostname>.pub
|
||||
cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add <hostname> pubkey' && git push
|
||||
```
|
||||
6. Deploy the key to all hosts:
|
||||
```bash
|
||||
ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
|
||||
```
|
||||
7. Generate the local SSH config:
|
||||
```bash
|
||||
homelab ssh-config --install
|
||||
```
|
||||
|
||||
## Hosts
|
||||
|
||||
### Hubris (PVE host)
|
||||
|
||||
| Detail | Value |
|
||||
|--------|-------|
|
||||
| LAN IP | `192.168.8.77` |
|
||||
| Netbird | `100.122.38.109` (FQDN: `proxmox-server.netbird.selfhosted`) |
|
||||
| Netbird SSH port | `22022` (mesh-only, OIDC auth) |
|
||||
| SSH user | `root` |
|
||||
| Authorized keys | `/etc/pve/priv/authorized_keys` (Proxmox cluster-synced) |
|
||||
|
||||
Authorized root keys currently deployed:
|
||||
- `root@hubris` (self, RSA)
|
||||
- `d.toro.v@pm.me` (ed25519) — mac-mini
|
||||
|
||||
### LXCs
|
||||
|
||||
Every LXC at `192.168.8.x` accepts root SSH via authorized_keys. Keys
|
||||
are managed by `ssh/deploy-keys.sh`. SSH user is `root`.
|
||||
|
||||
| LXC | Name | LAN IP | Role |
|
||||
|-----|------|--------|------|
|
||||
| 101 | jellyfin | `192.168.8.206` | media-server |
|
||||
| 102 | nfs-export | `192.168.8.200` | storage-export |
|
||||
| 103 | paperless | `192.168.8.130` | document-archive |
|
||||
| 104 | gitea | `192.168.8.121` | git-server |
|
||||
| 105 | apps | `192.168.8.205` | docker-apps |
|
||||
| 106 | auth-outpost | `192.168.8.184` | authentik-outpost |
|
||||
| 107 | dns | `192.168.8.185` | dns-helper |
|
||||
| 114 | nextcloud | `192.168.8.224` | file-sync |
|
||||
| 118 | elementsynapse | `192.168.8.239` | matrix-server |
|
||||
| 119 | sophia | `192.168.8.157` | workshop |
|
||||
| 120 | mule-images | `192.168.8.136` | photo-management |
|
||||
| 121 | caddy | `192.168.8.175` | reverse-proxy |
|
||||
| 122 | arriman | `192.168.8.132` | arr-stack |
|
||||
| 123 | claudio-bot | `192.168.8.230` | matrix-agent |
|
||||
| 126 | plato | `192.168.8.190` | app |
|
||||
|
||||
### Workstations
|
||||
|
||||
| Name | OS | LAN IP | Netbird FQDN | SSH user |
|
||||
|------|----|--------|--------------|----------|
|
||||
| mac-mini | macOS | `192.168.8.174` | `mac-mini-234-17.netbird.selfhosted` | `dtoro` |
|
||||
| republic-laptop | Linux | TBD | `republic-laptop.netbird.selfhosted` | `dtoro` |
|
||||
| ludo-mini | Linux | `192.168.8.133` | `ludo-mini.netbird.selfhosted` | TBD |
|
||||
|
||||
### VPS (external)
|
||||
|
||||
| Detail | Value |
|
||||
|--------|-------|
|
||||
| Public IP | `82.165.190.79` |
|
||||
| Netbird | `100.122.165.149` (FQDN: `netbird-ionos.netbird.selfhosted`) |
|
||||
| SSH user | `root` |
|
||||
| Access | Mesh-only — public port 22 is blocked by nftables. Key-only auth. |
|
||||
|
||||
## VPS
|
||||
|
||||
Access is mesh-only. From a mesh-connected peer:
|
||||
|
||||
```bash
|
||||
ssh root@100.122.165.149
|
||||
ssh root@netbird-ionos.netbird.selfhosted
|
||||
# or via homelab:
|
||||
homelab ssh netbird-vps
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# From any workstation after running homelab ssh-config --install:
|
||||
for name in hubris gitea apps sophia paperless caddy jellyfin nextcloud; do
|
||||
ssh -o BatchMode=yes "$name" "hostname" && echo "$name OK"
|
||||
done
|
||||
```
|
||||
|
||||
## Related
|
||||
- [Hubris host](../hosts/hubris.md)
|
||||
|
||||
- [Mesh migration](mesh.md)
|
||||
- [VPS hardening](vps-hardening.md)
|
||||
- [Agent enrollment](../operations/agent-enrollment.md)
|
||||
- [Homelab CLI](../bin/homelab)
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-06-02 — universal SSH reachability
|
||||
|
||||
Replaced ad-hoc per-workstation SSH configs with inventory-generated
|
||||
configs (`ssh/gen-config.py`, `homelab ssh-config`). Added centralized
|
||||
key distribution (`ssh/deploy-keys.sh`, `ssh/authorized_keys/`). All
|
||||
LXCs now accept root SSH from any workstation whose pubkey is in the
|
||||
repo. mac-mini Remote Login enabled. Netbird subnet route
|
||||
(192.168.8.0/24 via hubris) provides off-LAN reachability for all LAN
|
||||
IPs.
|
||||
|
||||
### 2026-04-28 — wiki entry created
|
||||
Initial documentation.
|
||||
|
||||
### 2026-04-23 — VPS SSH hardened to mesh-only
|
||||
Public `:22` blocked at nftables. Key-only sshd. See [VPS hardening](vps-hardening.md).
|
||||
Public `:22` blocked at nftables. Key-only sshd.
|
||||
|
||||
### 2026-04-22 — iMac key authorized on hubris
|
||||
`d.toro.v@pm.me` added to `/etc/pve/priv/authorized_keys`.
|
||||
`d.toro.v@pm.me` added to `/etc/pve/priv/authorized_keys`.
|
||||
Reference in New Issue
Block a user