mesh: document Netbird external-STUN fix (IONOS blocks UDP 3478)

infrastructure/mesh.md

@@ -8,6 +8,24 @@ The hubris fleet is migrating from Tailscale to Netbird. Netbird is the target e
 - **Netbird mgmt host** (`82.165.190.79`, FQDN `inspiring-ramanujan.netbird.selfhosted`, NB IP `100.122.165.149`) is now itself a peer on the mesh (joined 2026-04-22 via setup key, netbird 0.69.0). Routes the homelab network (`192.168.8.0/24`) via the PVE peer. This gives the mgmt host LAN access *and* split-horizon DNS for `*.hubris.network`. Useful independently of any Authentik integration.
 - **Most LXCs** still run Tailscale or use router DNS (`192.168.8.1`) / Tailscale MagicDNS (`100.100.100.100`), both of which return the *public* IONOS A record `*.hubris.network → 82.165.190.79`. The VPS only routes hostnames it actually publishes (today, `artifacto` + `blog`), so this path is a dead end for any LAN-only service.
 
+## ICE / STUN — must use external STUN, not embedded
+
+The bundled netbird-server image runs an embedded STUN listener on UDP 3478. **IONOS drops inbound UDP 3478 to the VPS upstream of the host firewall** (verified 2026-05-10 via `tcpdump -ni any udp port 3478`: 0 packets captured during external probes from hubris). Without a reachable STUN server, the management API hands peers a STUN URI nothing can talk to → no srflx candidates → ICE always fails → every peer falls back to the websocket relay (`rels://netbird.hubris.network:443`). All cross-NAT traffic is then bottlenecked by the relay/VPS bandwidth (observed ~366 kB/s for Nextcloud uploads).
+
+**Fix**: in `/opt/config.yaml` on the VPS, declare external STUN servers under `server:` — this disables the embedded STUN automatically:
+
+```yaml
+server:
+  stuns:
+    - uri: "stun:stun.l.google.com:19302"
+    - uri: "stun:stun1.l.google.com:19302"
+    - uri: "stun:stun.cloudflare.com:3478"
+```
+
+`docker restart netbird-server`, then `netbird down && netbird up` on each peer to force a resync. Verify with `netbird status -d` — `Connection type:` should flip from `Relayed` to `P2P` for peers that aren't behind double-NAT/CGNAT.
+
+If a peer is *still* relayed after this, it's a NAT-symmetry problem on its side, not a config bug — would need TURN to fix.
+
 ## Consequence — every LXC wired to Authentik needs an internal override
 
 Until each LXC is migrated to Netbird, anything that needs to reach `auth.hubris.network` (Authentik), `cloud.hubris.network` (Nextcloud), etc., must override the public answer with `192.168.8.175`.
@@ -81,6 +99,9 @@ Recipe for container-config changes (e.g. adding `extra_hosts`) on Portainer-man
 
 ## Changelog
 
+### 2026-05-10 — ICE direct p2p restored (external STUN swap)
+All peers were `Connection type: Relayed` because the embedded STUN on the VPS was unreachable from outside (IONOS drops UDP 3478 inbound). Swapped to Google + Cloudflare STUN in `/opt/config.yaml`. After `netbird down/up`, peers now report `P2P` with srflx/host candidates. Backup of pre-change config at `/opt/config.yaml.bak-20260510-185050`. Fixes a real-world Nextcloud upload bottleneck (~366 kB/s through the relay → LAN-direct on same-network peers).
+
 ### 2026-04-28 — wiki entry created
 Initial documentation.