Updates to four pages reflecting the combined → vanilla mgmt+signal+relay+coturn cutover and the IONOS-3478-firewall-exception discovery: * infrastructure/mesh.md — rewrites the ICE/STUN section to cover the new TURN endpoint, the IONOS upstream TCP-3478 filtering (load-bearing, undocumented before today), and the verification probe. New changelog entry covering the migration outcome + Device Code Stage gap. * infrastructure/vps-hardening.md — "At a glance" lists the new 6-service docker stack + host coturn. Firewall section notes the new `iifname ens6 tcp dport 3478 accept` rule plus the IONOS upstream exception. New changelog entry. * containers/124-authentik.md — replaces the "Netbird IdP integration — DEFERRED" section with the LANDED state: Provider details (Public client type — Confidential breaks PKCE on the dashboard SPA), the first-time owner-promotion sqlite recipe, the missing Device Code Stage gap + workaround (setup-keys), and a note that the old 2026-04-22 pre-work Provider/App is now obsolete and safe to delete. Updated changelog (Phase 6 landed). * operations/agent-enrollment.md — new "Getting onto Netbird" subsection explaining the setup-key path (currently the only working flow until Device Code Stage lands) and why direct OIDC from the public internet fails (auth.hubris.network is mesh-only-reachable). Prerequisites table row updated to point at the new section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
12 KiB
124 — authentik
Central Identity Provider for the lab. Also runs the split-horizon dnsmasq — ergo "the SSO and DNS box".
At a glance
- Hostname:
authentik - IP:
192.168.8.180(statically configured — the only LXC with a static IP) - Privilege: privileged
- Resources: 2 cores / 4 GiB RAM / 20 GiB rootfs
- Mounts: none from
/mnt/library - Public hostname:
auth.hubris.network→ caddy (121) →:9000 - Container DNS (in
/etc/pve/lxc/124.conf):192.168.8.1 1.1.1.1(router DNS plus a fallback added 2026-04-21 because router DNS flakes intermittently — Authentik is the resolver itself for the rest of the LAN, but its own LXC uses upstream).
Authentik stack (/opt/authentik/)
Upstream docker-compose.yml + .env. Services: postgresql (16-alpine), server, worker. Authentik 2026.x dropped the Redis dependency.
.envmode 600, untracked, holdsAUTHENTIK_SECRET_KEYandPG_PASS.AUTHENTIK_TAG=2026.2.2— pinned. Don't let it drift to:latest. Telemetry / update-check / error-reporting disabled.- Ports: 9000 (http), 9443 (https) on the LXC.
- Embedded outpost lives at
/outpost.goauthentik.io/*on the Authentik host — the forward-auth endpoint Caddy points at. - Stack is not git-tracked yet. If/when wiring auto-deploy: mirror the
mule-imagepattern (webhook receiver outside the app repo at/opt/authentik-deploy/). Repodtoro/authentik-confis reserved but not created.
Forward-auth pattern (every gated app)
- One Proxy Provider per app. Authentik enforces a UNIQUE constraint
application.provider_id, so one Provider = one Application. "Domain-level" only means they share the cookie domain. Each provider in "Forward auth (domain level)" mode, External hosthttps://auth.hubris.network, Cookie domainhubris.network. First one washubris-forward-auth(Paperless). - Authentication flow: MUST be
default-authentication-flow(NOTdefault-source-authentication— that's for IdP federation; givesFlowNonApplicableException+ 404 on the authorize endpoint). - Authorization flow:
default-provider-authorization-implicit-consent(or explicit). - Application Launch URL MUST be the full public URL
https://<sub>.hubris.network/— outpost matches incomingX-Forwarded-Hostagainst it. - Each Application MUST have at least one policy/group/user binding — zero bindings = outpost returns 404 on access.
- Restart Authentik after binding new apps to the outpost:
pct exec 124 -- docker compose -f /opt/authentik/docker-compose.yml restart server worker
Caddy snippet (authentik) lives at the top of /etc/caddy/Caddyfile. Points at http://192.168.8.180:9000 directly (NOT https://auth.hubris.network) to avoid hairpin TLS round-trip stripping X-Forwarded-Host. Must explicitly set header_up X-Forwarded-Host {host} in the forward-auth block. Used by gated sites with import authentik.
Per-app username override pattern (Authentik)
Used when the app's local user ID doesn't match the user's Authentik username (e.g., Nextcloud's admin ≠ Authentik's dtoro).
- On the Authentik user: add attribute
<app>_uid: <target_local_username>(YAML, Directory → Users → Edit → Attributes). - Customization → Property Mappings → Create → Scope Mapping (not SAML):
- Name:
<app>-uid-override, Scope name:<app>-uid, Expression:return {"nc_uid": user.attributes.get("<app>_uid", user.username)} - Use a custom claim key (e.g.
nc_uid), notpreferred_username— the defaultprofilescope mapping emitspreferred_usernameand will overwrite yours depending on ordering.
- Name:
- Attach the new scope to the provider (Providers → app → Scopes).
- On the app side, point its OIDC UID-mapping setting at the custom claim.
For Nextcloud:
occ user_oidc:provider <name> --mapping-uid=nc_uid
occ user_oidc:provider <name> --scope="openid profile email <app>-uid"
Bypass forward-auth for API paths (mobile apps)
If the app has its own token-based API auth and a mobile client, API paths must bypass forward-auth — mobile apps can't follow the browser login redirect. Pattern in the Caddyfile site block:
paperless.hubris.network {
tls { dns ionos {env.IONOS_AUTH_API_TOKEN} }
@api path /api/*
handle @api {
reverse_proxy 192.168.8.130:8000
}
handle {
import authentik
reverse_proxy 192.168.8.130:8000
}
}
API paths to bypass per app:
- Paperless:
/api/*(Bearer) - Sonarr / Radarr / Lidarr / etc.:
/api/*(X-Api-Key) - qBittorrent:
/api/*(session cookie from/api/v2/auth/login) - SABnzbd:
/api?*(apikey query param) — match/api*for query-string APIs - Homarr: no mobile client
- Portainer: mobile uses same session auth as web; no bypass typically needed
Backend trust of Authentik headers (skip the app's own login after SSO)
- Paperless:
PAPERLESS_ENABLE_HTTP_REMOTE_USER=trueandPAPERLESS_HTTP_REMOTE_USER_HEADER_NAME=HTTP_X_AUTHENTIK_USERNAMEin/opt/paperless/paperless.conf. Restartpaperless-webserver paperless-task-queue paperless-scheduler paperless-consumer. Django auto-creates matching users on first SSO login; promote to superuser via existing admin UI. - Apps without header-auth support: users log in twice (SSO + app login). Acceptable but degraded UX.
Per-app integration map
| App | Type | Notes |
|---|---|---|
| Paperless (103) | Forward-auth + REMOTE_USER | /api/* bypass |
| Nextcloud (114) | Native OIDC | nc_uid override; local dnsmasq required (Guzzle bypasses /etc/hosts) |
| mulita (120) | Native OIDC | extra_hosts override in compose |
| Booklore (105) | Native OIDC | Redirect URI /oauth2-callback; extra_hosts |
| Portainer (105) | Native OAuth2 | portainer_uid custom claim; --trusted-origins flag |
| WriteFreely (105) | Native OIDC | [oauth.generic] block; extra_hosts |
| qBittorrent (122) | Forward-auth via IP whitelist | Reverse-proxy support enabled in qBit |
| Artifacto (105) | Forward-auth + gateway-secret auto-login | Public /p/* paths bypass |
| Home Assistant VM (108) | HACS christiaangoossens/hass-oidc-auth |
automatic_user_linking: true, default_redirect: true. Supervisor DNS via ha dns options. |
Netbird IdP integration — LANDED 2026-05-21
The combined netbirdio/netbird-server image was replaced with the canonical vanilla stack (mgmt + signal + relay + dashboard 0.71.3) on the VPS so external OIDC actually works. Authentik is now the netbird dashboard's IdP. Full migration context in mesh.md changelog.
Active provider & app:
- Provider
NetBird(OAuth2/OpenID), Client type:Public(PKCE-only —Confidentialwould break the dashboard SPA's token exchange). - Client ID:
netbird-dashboard. Client Secret is in/opt/management.jsonPKCEAuthorizationFlow.ProviderConfig.ClientSecreton the VPS (TODO: sops-encrypt assecrets/netbird-authentik-oidc.yaml). - Application
NetBird, slugnetbird, launch URLhttps://netbird.hubris.network/. - Redirect URIs:
https://netbird.hubris.network/peers,/nb-auth,/nb-silent-auth, plushttps://netbird.hubris.network/for post-logout. - Scopes enabled on the provider:
openid,profile,email. - Discovery URL:
https://auth.hubris.network/application/o/netbird/.well-known/openid-configuration— netbird mgmt fetches this on startup; logsloaded OIDC configuration from the provided IDP configuration endpoint.
Login flow: netbird dashboard PKCE → Authentik authorize → redirect back to /nb-auth → JS token exchange at Authentik's /token endpoint → mgmt validates the bearer against Authentik's JWKS.
First-time owner promotion gotcha (write-down for future operators):
When a new Authentik user logs in for the first time against an account that already has peers, netbird mgmt adds them as role=user, blocked=1, pending_approval=1. The OLD account-owner (the one in store.db from before the IdP swap) can't be reached anymore, so there's no admin to approve. Recovery is a direct sqlite update on mgmt_data:
docker stop netbird-mgmt
sqlite3 /var/lib/docker/volumes/opt_mgmt_data/_data/store.db \
"UPDATE users SET role='owner', blocked=0, pending_approval=0 WHERE id='<new authentik sub>';"
docker start netbird-mgmt
The Authentik sub-claim is the value of the id column on the newly-created user row (look for role=user, blocked=1, pending_approval=1).
KNOWN MISSING — Device Code Stage
Authentik does NOT yet have an OAuth2DeviceCodeStage configured for this provider. Symptom: netbird up (without --setup-key) prints a device-code URL → user opens it → blank consent screen → device-code expires after 60s → CLI errors out. The token endpoint itself works (we verified with a direct device-code POST); only the consent UI is missing.
Workaround: every new peer must use netbird up --setup-key <KEY>. Setup keys are generated from the dashboard at https://netbird.hubris.network/setup-keys by an already-enrolled operator.
Proper fix (TODO): Authentik admin → Flows & Stages → Stages → Create → OAuth2 Device Code Stage + matching OAuth2 Device Code Finish Stage. Then either edit the provider's Authorization flow to include those stages, or create a dedicated device-code flow and bind the provider to it.
Self-service onboarding (not yet — future-session)
auth.hubris.network is only reachable from inside the netbird mesh (split-horizon DNS). A brand-new client that isn't on the mesh yet can't OIDC-login → setup-key is the only path. To enable self-service onboarding via Authentik from the public internet:
- Add a Traefik route on the VPS for
auth.hubris.networkthat forwards via the netbird-routed192.168.8.0/24to LXC 124. - DNS already points
auth.hubris.network → 82.165.190.79(IONOS wildcard).
Tracked in homelab memory as a queued follow-up.
Old pre-work to remove
The previous Provider for Netbird + netbird app from 2026-04-22 (client ID xZwVTFCsxWdBM3uIGS15wAAcVvsJiTtWdxVCEela, service account netbird-service) is now obsolete — replaced by netbird-dashboard above. Safe to delete from Authentik admin UI; nothing currently uses the old client ID. The service account + API token can also be removed unless we wire IdpManagerConfig in mgmt later (currently ManagerType: none).
DNS responsibility
dnsmasq runs alongside Authentik on this LXC, listening on 192.168.8.180:53 + 127.0.0.1:53, serving every *.hubris.network subdomain → 192.168.8.175. There is no wildcard — every site needs an explicit address= entry. See DNS split-horizon.
Related
- DNS split-horizon
- Caddy (121)
- Mesh migration
- Every gated app under containers/index
Changelog
2026-05-21 — Netbird IdP swap landed (Phase 6 done)
VPS migrated from combined netbird-server to vanilla mgmt+signal+relay+dashboard 0.71.3 (see mesh.md), enabling Authentik as the dashboard IdP via PKCE. New Provider/App = netbird-dashboard, replacing the deferred pre-work. Device Code Stage still missing — interactive netbird up fails consent; setup-keys are the workaround until that's added.
2026-04-28 — wiki entry created
Initial documentation.
2026-04-22 — Phase 6 (Netbird IdP swap) deferred
Combined netbird-server image couldn't take an external IdP. Pre-work in Authentik kept for later (now superseded by 2026-05-21 above). Netbird mgmt host instead joined its own mesh as a peer (100.122.165.149) for split-horizon DNS access.
2026-04-22 — Artifacto, mulita, WriteFreely, Portainer wired
Native OIDC for mulita / WriteFreely / Portainer; gateway-secret auto-login pattern for Artifacto.
2026-04-21 — deployed; Phases 1–5 complete
LXC 124 provisioned, stack at /opt/authentik, public URL via Caddy, Paperless + Booklore + Nextcloud + Home Assistant wired. dnsmasq for split-horizon DNS lives on the same LXC.