dtoro/oikos
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

hubris: document Phase 1 WiFi failover

Browse Source 
Host now dual-homed LAN + WiFi (192.168.8.141 on wlp3s0, metric 200).
wan-failover.service handles carrier-based default-route swap since the
bridge's carrier follows LXC veths and ignore_routes_with_linkdown can't
catch eno1 going down on its own.

LXC/VM guests still LAN-only; Phase 2 migrates them.
This commit is contained in:
Claudio
2026-04-28 23:27:21 +02:00
parent f5cd320433
commit 7454b5f110
1 changed files with 16 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
hosts/hubris.md
View File

@@ -6,7 +6,8 @@ Single-node Proxmox VE running 1 VM and 14 LXC containers (one of which is curre
- **Role:** Proxmox VE 9.1.2 hypervisor (kernel `6.14.11-4-pve`)
- **Hardware:** GMKtec NucBox M6 Ultra — AMD Ryzen 5 7640HS (Phoenix APU), 12 vCPU / ~28 GiB RAM, 2× Samsung 990 EVO Plus NVMe (one SSD primary, one for `library` LVM). 2× Realtek RTL8125 NICs (`r8169`).
- **BIOS:** 1.02 (2025-08-06) — vendor not on LVFS, no automated update path. See [investigations](../investigations/2026-04-21-hubris-crash-loop.md).
- **LAN:** `192.168.8.77/24` on bridge `vmbr0`, gateway `192.168.8.1`.
- **LAN (primary):** `192.168.8.77/24` on bridge `vmbr0` (slave: `eno1`), gateway `192.168.8.1`. Default route metric 0.
- **WiFi (failover):** `192.168.8.141/24` on `wlp3s0` (MediaTek MT7922, AX), DHCP from the same router. Default route metric 200. See [Phase 1 WiFi failover](#phase-1-wifi-failover) below.
- **Mesh:** Netbird `wt0` `100.122.38.109/16`. Resolver: `100.122.38.109` (the local netbird daemon, which forwards to LAN/upstream and learns `*.hubris.network` answers via that path). See [mesh](../infrastructure/mesh.md).
- **UI:** `https://proxmox.hubris.network` (via [caddy](../containers/121-caddy.md)) or `https://192.168.8.77:8006`.
@@ -34,6 +35,17 @@ See [containers/index](../containers/index.md). 14 active, 1 stopped.
- **Crash capture:** `/etc/sysctl.d/60-crash-capture.conf` (panic on oops/hardlockup/softlockup/rcu, auto-reboot 10 s), `/etc/modprobe.d/softdog.conf` (`soft_panic=1 soft_margin=60`), `/etc/systemd/system.conf.d/watchdog.conf` (`RuntimeWatchdogSec=15s`). Pstore traces collected to `/var/lib/systemd/pstore/` by `systemd-pstore.service`. **Caveat:** silent CPU lockups leave pstore empty.
- **`rasdaemon`** (Debian pkg) collects MCE / memory / PCIe AER / thermal events to `/var/lib/rasdaemon/ras-mc_event.db`. Query with `ras-mc-ctl --summary` / `--errors`. (mcelog is retired in Debian 13 — don't go looking for it.)
## Phase 1 WiFi failover
Host is dual-homed on LAN (`eno1`/`vmbr0`) and WiFi (`wlp3s0`) so management/SSH stay reachable when LAN drops. **Guests are not yet failed over** — the LXC fleet remains on `vmbr0`/`eno1`. Phase 2 will migrate guest networking off the bridge so the homelab survives full LAN loss.
- WiFi creds in `/etc/wpa_supplicant/wpa_supplicant-wlp3s0.conf` (hashed PSK, mode 600). SSID lives in `/etc/network/interfaces` as `wpa-conf`.
- Both interfaces sit on the same `192.168.8.0/24`; cross-talk avoided with `arp_ignore=1` + `arp_announce=2` on `eno1`/`vmbr0`/`wlp3s0` (set via `post-up` in `/etc/network/interfaces`).
- A second default route at metric 200 is added on `wlp3s0` (post-up). LAN wins while up.
- **Carrier-based failover:** `vmbr0`'s carrier follows the LXC veth members, so it stays `1` even when `eno1` loses link. `ignore_routes_with_linkdown` is therefore not enough on its own. `wan-failover.service` (`/usr/local/sbin/wan-failover.sh`) watches `/sys/class/net/eno1/carrier` via `ip monitor link` and removes/restores the `vmbr0` default route on transitions. Logs to `journalctl -t wan-failover`.
- Failover verified 2026-04-28: `ip link set eno1 down` → outbound HTTP keeps working via WiFi; `ip link set eno1 up` → vmbr0 default restored.
- Reachable on `192.168.8.77` (LAN) and `192.168.8.141` (WiFi); SSH works on either.
## Host services owned by external repos
| What | Repo | Path on host |
@@ -73,6 +85,9 @@ OpenSSH on `0.0.0.0:22`. Netbird's built-in SSH server is on `100.122.38.109:220
## Changelog
### 2026-04-28 — Phase 1 WiFi failover
Host now dual-homed: LAN `192.168.8.77` (primary) + WiFi `192.168.8.141` (failover, metric 200) on the GL-AXT1800-714-5G AP. Installed `wpasupplicant`+`iw`; added `wlp3s0` stanza to `/etc/network/interfaces` with `wpa-conf`; ARP isolation sysctls in `post-up`. Built `wan-failover.service` to remove the vmbr0 default route on `eno1` carrier loss, since the bridge's carrier doesn't follow `eno1` (the LXC veths keep it `1`). LXC/VM guests are still LAN-only — Phase 2 will migrate them.
### 2026-04-28 — wiki started
This wiki created. Live state at this date: 14 LXCs running (109 syncthing stopped), 1 VM, kernel `6.14.11-4-pve`, uptime 3 d 0 h post drive-removal A/B test. Compared to memory snapshot from a week ago, **destroyed**: LXC 100 (yunohost arr), 106 (flaresolverr), 107 (marimo), 110 (photoprism), 111 (karakeep), 112 (immich), 115 (reticulum). 100 + 106 destroyed per the planned 2026-04-21 \*arr migration retention; the others removed since.