dtoro
4efddb8bed
Problem: docs-lint (added in the wiki-hq reorg) surfaced 126 broken relative
links that predated this session — a container rename, incident/plan docs
that moved into archive/done subfolders without their inbound links being
updated, and a handful of relative-depth bugs in files nested under
containers/archive/ and plans/done/.
Fixes applied, by category:
- 124-authentik.md -> 106-auth-outpost.md (container was renamed; ~40 refs).
- investigations/{2026-04-21-hubris-crash-loop,2026-05-31-authentik-vps-migration}.md
-> archive/ prefix (both moved to investigations/archive/ previously).
- plans/{2026-06-01-slate-ax-to-sodola-migration,2026-06-04_130000-deprecate-claudio-bot,
2026-06-25-yuvomi-deployment}.md -> plans/done/ prefix.
- Depth bugs in files nested one level deeper than their siblings assumed
(investigations/archive/*, knowledge/wiki/containers/archive/*,
plans/done/*) — corrected relative-path depth.
- Destroyed containers with no surviving page (126-plato) delinked to the
containers/index.md archaeology row instead of a 404.
- ludo-mini.yaml -> strong.yaml (host was renamed, same physical machine).
- netbird-vps.md (no narrative page exists) -> netbird-vps.yaml (substrate
record, matching the existing convention for hosts without a wiki page).
- runbook-dpkg-interrupted.md refs -> .agents/skills/runbook-dpkg-interrupted/SKILL.md
(missed in the phase-4 runbook move because the referencing files used a
bare filename, not a runbooks/ prefix).
- One dangling forward-reference to a never-written investigation delinked
to the actual incident record it was describing.
Left alone: two links in knowledge/wiki/containers/101-jellyfin.md into
devops/homelab-authentik-admin/ — an intentional reference to a sibling repo,
not present in this checkout.
Verification: broken-link count 126 -> 2 (real remainder is the cross-repo
reference above); gen-topology.py --check still exit 0; build_host_files.py
still idempotent; all inventory.yaml doc_page targets still resolve.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
104 lines
8.3 KiB
Markdown
104 lines
8.3 KiB
Markdown
# Public ingress — VPS traefik + cert mirror
|
|
|
|
How home services reach the open internet without exposing the home network. Two-stage pattern: traefik on the IONOS VPS terminates TLS at the public edge, then reverse-proxies over the netbird mesh to home Caddy / direct backends.
|
|
|
|
## The shape
|
|
|
|
```
|
|
Public client
|
|
│ *.hubris.network → 82.165.190.79 (IONOS wildcard)
|
|
▼
|
|
[VPS traefik] priority 1: HostSNI(*) → netbird-proxy:8443 ← netbird control plane
|
|
priority 10: per-host HTTP routers ← home services
|
|
│ HTTP over netbird mesh
|
|
▼
|
|
[Home backend on 192.168.8.x]
|
|
```
|
|
|
|
LAN clients resolve via the [Technitium DNS on dns (107)](dns.md) → `192.168.8.175` → home [Caddy (121)](../containers/121-caddy.md), unchanged. The two paths are independent.
|
|
|
|
## Why this shape
|
|
|
|
- VPS traefik already has a HostSNI(*) TCP passthrough at priority 1 so netbird's own ingress (`netbird.hubris.network`, future `*.proxy.hubris.network`) is unaffected.
|
|
- Per-hostname HTTP file-provider routers at priority 10 win over the passthrough for the listed hosts and let traefik terminate TLS itself for those.
|
|
- Traefik's own ACME (`letsencrypt` resolver) fails on this box: HostSNI(*) grabs TLS-ALPN-01 challenges before traefik's `allowACMEByPass=true` can respond. Solution: home Caddy obtains certs via IONOS DNS-01 (no conflict) and the VPS *mirrors* the result over.
|
|
|
|
## Components
|
|
|
|
### On the VPS (`82.165.190.79`)
|
|
|
|
- `/opt/traefik-dynamic.yaml` — file-watched dynamic config. One `http.routers.<name>-public` + one `http.services.<name>-public` per exposed service, plus one entry in the top-level `tls.certificates` list per hostname.
|
|
- Let's-Encrypt-effective directory: `/letsencrypt/` inside the traefik container, backed by the host-side docker volume `opt_netbird_traefik_letsencrypt`.
|
|
- Backups before edits: `cp /opt/traefik-dynamic.yaml /opt/traefik-dynamic.yaml.bak.$(date +%s)` — several bak files live alongside.
|
|
|
|
### On the PVE host (`192.168.8.77`)
|
|
|
|
- `/usr/local/bin/hubris-public-cert-sync.sh` — runs daily via `hubris-public-cert-sync.timer`. Maps source hostname → VPS cert filenames in a bash assoc array. For each mapping: `pct pull` cert+key from [Caddy (121)](../containers/121-caddy.md)'s store, diff against the VPS copy, scp only on change.
|
|
- Filenames are stable per host so the dynamic.yaml never needs editing on renewal — traefik file-watches and hot-reloads the cert.
|
|
|
|
## Services currently exposed
|
|
|
|
| Hostname | Path scope | Backend | Middlewares | Cert files on VPS |
|
|
| ------------------------------ | -------------------------------- | -------------------------------- | -------------------------------------------- | ------------------------------------------ |
|
|
| `artifacto.hubris.network` | `/p/*`, `/static/*`, `/healthz` | `192.168.8.205:3100` | `artifacto-strip-sso` + `artifacto-ratelimit` (50 rps / 100 burst) | `fullchain.crt` / `privkey.key` |
|
|
| `blog.hubris.network` | whole host | `192.168.8.205:8080` | `blog-ratelimit` (100 rps / 200 burst) | `blog.fullchain.crt` / `blog.privkey.key` |
|
|
| `trmnl.hubris.network` | whole host | `192.168.8.211:9851` ([trmnl 128](../containers/128-trmnl.md)) | `trmnl-ratelimit` (20 rps / 40 burst) | `trmnl.fullchain.crt` / `trmnl.privkey.key` |
|
|
| `house.hubris.network` | whole host | `192.168.8.212:3000` ([house 129](../containers/129-house.md)) | `house-ratelimit` (30 rps / 60 burst) | `house.fullchain.crt` / `house.privkey.key` |
|
|
|
|
`artifacto-strip-sso` blanks inbound `X-Authentik-*` and `X-Artifacto-Gateway` so external clients can't spoof the SSO auto-login header contract. Path split is enforced at the VPS router rule, not by home Caddy. See [Artifacto on apps (105)](../containers/105-apps.md).
|
|
|
|
### `auth.hubris.network` — different pattern (local container, not cert-mirror)
|
|
|
|
Since 2026-05-31 [Authentik runs on the VPS itself](../../../investigations/archive/2026-05-31-authentik-vps-migration.md), so `auth.hubris.network` is served by a **local Docker container**, not proxied to a home backend. It therefore does **not** use the file-provider + cert-mirror pattern above:
|
|
|
|
- Routed via traefik **Docker provider labels** on the `authentik-server` service (`/opt/docker-compose.yml`), not `traefik-dynamic.yaml`.
|
|
- TLS via traefik's own `letsencrypt` resolver (works here because it's a normal HTTP router, not the HostSNI passthrough).
|
|
- Traefik reaches it over the `auth` Docker network (`172.30.1.0/24`); Postgres/Redis on that net are isolated from the netbird containers.
|
|
- Admin UI is IP-gated: an `admin-allowlist` ipAllowList middleware on `PathPrefix(/if/admin/)` (currently `5.61.168.0/24`). Login/flow endpoints stay public.
|
|
|
|
No cert-mirror entry and no `hubris-public-cert-sync.sh` mapping is needed for `auth`.
|
|
|
|
## Recipe — exposing another service
|
|
|
|
1. Ensure home Caddy on [LXC 121](../containers/121-caddy.md) already serves the hostname (cert exists at `/var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/<host>/`).
|
|
2. Add an entry to `HOSTS` in `/usr/local/bin/hubris-public-cert-sync.sh` mapping the hostname → VPS filenames. Run once: `systemctl start hubris-public-cert-sync.service`. Confirm the cert landed.
|
|
3. Edit `/opt/traefik-dynamic.yaml` on the VPS:
|
|
- Add to `tls.certificates`: paths `/letsencrypt/<crt>` and `/letsencrypt/<key>`.
|
|
- Add `http.routers.<host>-public`: `rule: 'Host(\`<host>\`)'` (or with path matchers if scope-gating), `entryPoints: [websecure]`, `priority: 10`, `tls: {}`, `service: <host>-public`, `middlewares: [...]`.
|
|
- Add a ratelimit middleware under `http.middlewares` if wanted.
|
|
- Add `http.services.<host>-public.loadBalancer.servers[0].url: 'http://<lan-ip>:<port>'`.
|
|
4. Verify:
|
|
```
|
|
ssh root@100.122.165.149 'curl -skI --resolve <host>:443:127.0.0.1 https://<host>/' # 2xx/3xx
|
|
curl -skI --resolve <host>:443:<vps-public-ip> https://<host>/ # same
|
|
```
|
|
5. **No DNS edit needed** — the IONOS wildcard already points at the VPS.
|
|
|
|
## What does NOT follow this pattern
|
|
|
|
- `netbird.hubris.network` (and any future `*.proxy.hubris.network`) uses the netbird-proxy / HostSNI passthrough path. Netbird handles its own cert via ACME cleanly because it *is* the passthrough target.
|
|
|
|
## Related
|
|
- [DNS split-horizon](dns.md)
|
|
- [Caddy (121)](../containers/121-caddy.md) — cert source, internal counterpart
|
|
- [Mesh migration](mesh.md) — netbird is the transport between VPS and home
|
|
- [VPS hardening](vps-hardening.md) — fail2ban / nftables that the access logs feed
|
|
- [Artifacto on apps (105)](../containers/105-apps.md) — first publicly-exposed service
|
|
|
|
## Changelog
|
|
|
|
### 2026-06-24 — `trmnl.hubris.network` exposed
|
|
TRMNL plugins middleware on [trmnl (128)](../containers/128-trmnl.md). File-provider router `trmnl-public` → `192.168.8.211:9851`, `trmnl-ratelimit` (20 rps / 40 burst), cert mirrored as `trmnl.fullchain.crt`/`trmnl.privkey.key`. Verified live from the internet (200 with token / 401 without). It was provisioned during a mesh outage — the `home-lab-network` (192.168.8.0/24) route had no active routing peer because the **mac-mini routing peer's netbird was down** (all home-backed public services 504'd). Bringing netbird up on mac-mini restored the route; no traefik change was needed.
|
|
|
|
### 2026-05-31 — `auth.hubris.network` now served locally on the VPS
|
|
Authentik migrated onto the VPS ([investigation](../../../investigations/archive/2026-05-31-authentik-vps-migration.md)). Unlike the home-backed services above, `auth` is a local container routed via traefik Docker-provider labels with traefik-managed Let's Encrypt — no cert-mirror, no `traefik-dynamic.yaml` router. Admin UI gated by an ipAllowList middleware. Traefik gained a second Docker network (`auth`, `172.30.1.0/24`) to reach it while keeping its DB/Redis isolated from the netbird stack.
|
|
|
|
### 2026-04-28 — wiki entry created
|
|
Initial documentation.
|
|
|
|
### 2026-04-23 — `blog.hubris.network` exposed
|
|
WriteFreely on [apps (105)](../containers/105-apps.md). Whole host is public.
|
|
|
|
### 2026-04-22 — pattern established with Artifacto
|
|
First service through the file-provider router. IONOS wildcard moved to the VPS this day. Cert mirror script + timer deployed on the PVE host.
|