cleanup: destroy LXC 124, remove stale docs, update DNS refs to Technitium
This commit is contained in:
@@ -47,7 +47,7 @@ The app repo at `/opt/<thing>` is the working tree, but the deploy tooling (`web
|
||||
> Each owns its own clone on LXC 105. They don't conflict because each
|
||||
> deploy.sh only touches its own service unit + venv.
|
||||
|
||||
> **Not yet wired:** `dtoro/claudio-monitor` (push, then `/opt/claudio-monitor/scripts/deploy.sh` manually). `dtoro/authentik-conf` is reserved but the LXC stack is not git-tracked yet. The dnsmasq config on [authentik (124)](../containers/124-authentik.md) is also not tracked — if it gets a `dtoro/dnsmasq-conf`, mirror the caddy-conf pattern.
|
||||
> **Not yet wired:** `dtoro/claudio-monitor` (push, then `/opt/claudio-monitor/scripts/deploy.sh` manually). The former authentik LXC (124) is destroyed — Authentik runs on the [VPS](../hosts/netbird-vps.md). DNS moved to [Technitium on dns (107)](../containers/107-dns.md).
|
||||
|
||||
## When you change a tracked config
|
||||
|
||||
|
||||
@@ -7,11 +7,11 @@ There is **no wildcard on the LAN side**. Every subdomain needs an explicit entr
|
||||
## Components
|
||||
|
||||
- **Authoritative public DNS:** IONOS. `*.hubris.network → 82.165.190.79` (was `74.118.126.4` until 2026-04-22).
|
||||
- **LAN authoritative for `hubris.network` records:** dnsmasq on [authentik (124)](../containers/124-authentik.md), `192.168.8.180:53` and `127.0.0.1:53`. Config at `/etc/dnsmasq.d/hubris-split.conf`. Forwards everything else to `1.1.1.1` and `8.8.8.8` (`no-resolv`, `server=...`).
|
||||
- **LAN authoritative for `hubris.network` records:** [Technitium DNS](https://technitium.com) on [dns (107)](../containers/107-dns.md) at `192.168.8.2:53`. Syncs A records to the NetBird managed DNS zone via cron (see [dns-sync.py](../scripts/dns-sync.py)). Formerly dnsmasq on [authentik (124)](../containers/124-authentik.md) (decommissioned 2026-06-04).
|
||||
- **PVE host** (`192.168.8.77`): resolver is the local Netbird daemon at `100.122.38.109:53`, which forwards to the LAN/upstream and learns hubris.network answers via that path. `netbird status` says "Nameservers: 0/0 Available" — confirming netbird does NOT manage a hubris.network zone; it just caches whatever the system resolver returns.
|
||||
- **Some LXCs** keep router DNS (`192.168.8.1`) or Tailscale MagicDNS (`100.100.100.100`), both of which return the public IONOS A record. Those LXCs need either a `/etc/hosts` override or local dnsmasq — see [mesh migration](mesh.md) for which technique applies where.
|
||||
|
||||
## Live entries (as of 2026-05-31)
|
||||
## Live entries (as of 2026-06-04)
|
||||
|
||||
```
|
||||
address=/auth.hubris.network/82.165.190.79 # → VPS, not Caddy (Authentik migrated 2026-05-31)
|
||||
@@ -41,24 +41,20 @@ Note: `nfs-export.hubris.network` is the only `.hubris.network` entry that point
|
||||
|
||||
## Why split-horizon
|
||||
|
||||
The IONOS wildcard points at the VPS for public ingress (per-host routers in [VPS traefik](ingress.md)). The VPS only routes hostnames it knows — anything else 404s. So LAN clients pointing at the public IP are a dead end for any service that isn't explicitly published. The dnsmasq override on LXC 124 keeps LAN traffic on the home Caddy.
|
||||
The IONOS wildcard points at the VPS for public ingress (per-host routers in [VPS traefik](ingress.md)). The VPS only routes hostnames it knows — anything else 404s. So LAN clients pointing at the public IP are a dead end for any service that isn't explicitly published. The [Technitium DNS](dns.md) override on `192.168.8.2` keeps LAN traffic on the home Caddy.
|
||||
|
||||
## The gotcha that cost a debug session (2026-04-22)
|
||||
|
||||
Creating a new Caddyfile site block is necessary but **not sufficient**. Without the LXC-124 dnsmasq entry, LAN queries fall through to upstream, get the public IONOS answer, and time out. Symptom: "subdomain doesn't load" even though Caddy config + cert are fine.
|
||||
Creating a new Caddyfile site block is necessary but **not sufficient**. Without the Technitium entry on [dns (107)](../containers/107-dns.md), LAN queries fall through to upstream, get the public IONOS answer, and time out. Symptom: "subdomain doesn't load" even though Caddy config + cert are fine.
|
||||
|
||||
## Recipe — adding a new subdomain
|
||||
|
||||
1. Edit `/etc/caddy/Caddyfile` on [caddy (121)](../containers/121-caddy.md), commit + push to `dtoro/caddy-conf`. Webhook reloads caddy. See [auto-deploy](auto-deploy.md).
|
||||
2. Append to `/etc/dnsmasq.d/hubris-split.conf` on [authentik (124)](../containers/124-authentik.md):
|
||||
```
|
||||
address=/<new>.hubris.network/192.168.8.175
|
||||
```
|
||||
3. `pct exec 124 -- systemctl restart dnsmasq` — reload/SIGHUP may not pick up the new `address=` line; hard restart is the reliable path.
|
||||
4. Verify: `dig @192.168.8.180 +short <new>.hubris.network` → `192.168.8.175`.
|
||||
5. On macOS clients, flush: `sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder`.
|
||||
2. Add the A record in the [Technitium UI](http://192.168.8.2) at `dns (107)` — the NetBird managed DNS zone sync picks it up within ~10 minutes via cron. Or add directly to the NetBird managed zone via API if you need it faster.
|
||||
3. Verify: `dig @192.168.8.2 +short <new>.hubris.network` → `192.168.8.175`.
|
||||
4. On macOS clients, flush: `sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder`.
|
||||
|
||||
> The dnsmasq config is **not** tracked in git. If you stand up a `dtoro/dnsmasq-conf` repo, wire it like `caddy-conf` (webhook on LXC 124).
|
||||
> The Technitium config on LXC 107 is the single source of truth. Never hand-edit the NetBird managed zone directly — the [`scripts/dns-sync.py`](../scripts/dns-sync.py) cron on 107 reconciles them and reaps stale records. See [dns.md changelog 2026-06-03](#2026-06-03--single-authoring-source-technitium--netbird-managed-zone-sync).
|
||||
|
||||
## Public path — what does and doesn't follow the LAN map
|
||||
|
||||
@@ -76,21 +72,17 @@ Either:
|
||||
- [Caddy (121)](../containers/121-caddy.md) — every LAN entry points here
|
||||
- [Ingress (VPS traefik)](ingress.md) — public-side counterpart
|
||||
- [Mesh migration](mesh.md) — per-LXC DNS workarounds during the transition
|
||||
- [Authentik (124)](../containers/124-authentik.md) — host of the split-horizon dnsmasq
|
||||
- [DNS server (107)](../containers/107-dns.md) — Technitium, current DNS authority
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-06-04 — Caddy: proxy auth.hubris.network → VPS; fix stale /etc/hosts across fleet
|
||||
Authentik had migrated to the VPS on 2026-05-31, but:
|
||||
- Caddy's `auth.hubris.network` block still proxied to dead `192.168.8.6:9000`
|
||||
- Caddy's `authentik` forward-auth snippet and `sso.hubris.network` block also referenced the stale IP
|
||||
- Gitea's `/etc/hosts` had both the VPS IP and the old Caddy IP, causing Gitea to fail on OAuth2 token exchange with `404` then `issuer mismatch`
|
||||
|
||||
**Fixed:**
|
||||
- Caddy Caddyfile: `auth.hubris.network`, `authentik` snippet, and `sso.hubris.network` all proxy to VPS (`82.165.190.79:443`) with proper Host header stripping
|
||||
- Gitea (LXC 104), Jellyfin (101), Paperless (103), Apps (105), Nextcloud (114), ElementSynapse (118), Mule-Images (120), ArrStack (122), Plato (126): /etc/hosts → `192.168.8.175` only (Caddy proxy)
|
||||
- Caddy LXC (121) /etc/hosts: self-reference removed, points to VPS
|
||||
- All stale `192.168.8.6` references eliminated from Caddyfile
|
||||
### 2026-06-04 — LXC 124 destroyed; Caddy proxy fixes; inventory/docs cleanup
|
||||
Authentik LXC 124 (192.168.8.180) destroyed — Authentik runs on VPS, DNS on Technitium (107).
|
||||
- Caddy: `auth.hubris.network`, `authentik` snippet, and `sso.hubris.network` all proxied to VPS
|
||||
- `header_up Host auth.hubris.network` added to strip `:443` from upstream Host header
|
||||
- All 9 LXCs' /etc/hosts updated: `auth.hubris.network → 192.168.8.175` (Caddy proxy)
|
||||
- Inventory: removed `hosts.authentik`, renamed `dnsmasq` service → `dns`
|
||||
- Docs: `124-authentik.md` deleted; dns.md references updated to Technitium (107)
|
||||
The "delete NetBird managed zone → forward everything to Technitium" plan was **abandoned** — NetBird's DNS defeats it: it **won't apply a nameserver group that contains the peer's own mesh IP** (the Mac's `100.122.234.17` → `Nameservers: 0/0 Available`), and nameserver-group forwarding to Technitium never actually took effect for mesh peers (the **managed zone was doing all the real work**; disabling it broke all mesh resolution). So the model is now:
|
||||
|
||||
- **Technitium (`192.168.8.2`) is the single place you author DNS** (UI/API, MX/SPF/CAA, full zone).
|
||||
|
||||
@@ -15,7 +15,7 @@ Public client
|
||||
[Home backend on 192.168.8.x]
|
||||
```
|
||||
|
||||
Mesh clients see [dnsmasq on LXC 124](dns.md) → `192.168.8.175` → home [Caddy (121)](../containers/121-caddy.md), unchanged. The two paths are independent.
|
||||
LAN clients resolve via the [Technitium DNS on dns (107)](dns.md) → `192.168.8.175` → home [Caddy (121)](../containers/121-caddy.md), unchanged. The two paths are independent.
|
||||
|
||||
## Why this shape
|
||||
|
||||
|
||||
Reference in New Issue
Block a user