docs: move narrative wiki under knowledge/wiki/ (phase 3)

Problem: node and cross-cutting narratives lived at the repo root
(containers/, vms/, infrastructure/, host .md files), interleaved with the
machine-readable substrate.

Change:
- Move containers/ -> knowledge/wiki/containers/, vms/ -> knowledge/wiki/vms/,
  infrastructure/ -> knowledge/wiki/infrastructure/, hosts/{hubris,strong}.md ->
  knowledge/wiki/hosts/, infrastructure/references/ -> knowledge/sources/references/,
  GLOSSARY.md -> knowledge/GLOSSARY.md.
- Add knowledge/{index.md,log.md,sources/index.md} scaffolding.
- Rewrite all relative links repo-wide via a path-resolving mapper (inbound +
  outbound + between-moved-files), including .hermes/, runbooks, operations,
  investigations, plans, README, AGENTS.
- Repoint inventory.yaml doc_page fields and regenerate hosts/*.yaml (which
  embed doc_page); update oikos/gen-topology.py output path, candidate doc
  paths, and footer links; update code-comment doc paths.

Substrate untouched in place: inventory.yaml, hosts/*.yaml (regenerated,
idempotent), oikos/ code, mcp/, secrets/, bin/.

Verification:
- Logical broken-link set identical to pre-move baseline (net 128 -> 127; the
  topology regen fixed one, introduced none). Remaining are pre-existing refs
  to destroyed/archived nodes, out of scope for this move.
- gen-topology.py --check exit 0 (in sync); cards carry knowledge/wiki/ doc paths.
- build_host_files.py idempotent; all inventory doc_page targets resolve.
- MCP contract verified: get_page/search_docs/get_changelog resolve moved pages.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 14:35:23 +02:00
parent bb5c0717a1
commit 8a6422bd7d
129 changed files with 249 additions and 264 deletions

View File

@@ -80,7 +80,7 @@ stored as `state:` in inventory (absent = active). Destroyed nodes live in
the `archaeology:` section. Each transition is a runbook checklist; the `archaeology:` section. Each transition is a runbook checklist;
deprecation completes only when inbound edges reach zero. deprecation completes only when inbound edges reach zero.
Generated views: [infrastructure/topology.md](../infrastructure/topology.md) Generated views: [infrastructure/topology.md](../knowledge/wiki/infrastructure/topology.md)
(Mermaid, regenerated from inventory) and the live, clickable version at (Mermaid, regenerated from inventory) and the live, clickable version at
`oikos.hubris.network/graph` once the Console is deployed. `oikos.hubris.network/graph` once the Console is deployed.

View File

@@ -33,10 +33,10 @@ the operator to run `homelab client add <hostname>` from an existing client.
- `/opt/homelab-context/inventory.yaml` — every host, LXC, VM, and workstation - `/opt/homelab-context/inventory.yaml` — every host, LXC, VM, and workstation
with their mesh addresses, roles, and service mappings. Treat this file as with their mesh addresses, roles, and service mappings. Treat this file as
authoritative; anything you read in narrative pages should agree with it. authoritative; anything you read in narrative pages should agree with it.
- `/opt/homelab-context/infrastructure/mesh.md` — Tailscale → Netbird state. - `/opt/homelab-context/knowledge/wiki/infrastructure/mesh.md` — Tailscale → Netbird state.
Both meshes are accepted today; Netbird is preferred for new traffic. Both meshes are accepted today; Netbird is preferred for new traffic.
- `/opt/homelab-context/infrastructure/dns.md` — split-horizon DNS via - `/opt/homelab-context/knowledge/wiki/infrastructure/dns.md` — split-horizon DNS via
Technitium on [dns (107)](../containers/107-dns.md). `*.hubris.network` Technitium on [dns (107)](knowledge/wiki/containers/107-dns.md). `*.hubris.network`
resolves to 192.168.x.x on the LAN and to mesh addresses off-LAN. resolves to 192.168.x.x on the LAN and to mesh addresses off-LAN.
- `/opt/homelab-context/operations/commands.md` — the operator's cheatsheet - `/opt/homelab-context/operations/commands.md` — the operator's cheatsheet
for pct, caddy, DNS, and the Oikos command surface. Use these verbs when for pct, caddy, DNS, and the Oikos command surface. Use these verbs when
@@ -75,8 +75,9 @@ Grep is fine for browsing or when MCP is unreachable.
## 4. Wiki conventions ## 4. Wiki conventions
- Pages live under `containers/`, `hosts/`, `vms/`, `infrastructure/`, - Narrative pages live under `knowledge/wiki/{containers,hosts,vms,infrastructure}/`;
`investigations/`, `operations/`. Cross-link liberally; orphans are bugs. procedural pages under `investigations/`, `operations/`, `runbooks/`, `plans/`.
Cross-link liberally; orphans are bugs.
- Every page ends with a `## Changelog` section, entries in reverse-chrono - Every page ends with a `## Changelog` section, entries in reverse-chrono
order: order:
@@ -105,7 +106,7 @@ Grep is fine for browsing or when MCP is unreachable.
a valid `--approval-id` from `homelab approval request` — see OIKOS.md. a valid `--approval-id` from `homelab approval request` — see OIKOS.md.
For ad-hoc work, SSH and edit directly — but commit changes that touch For ad-hoc work, SSH and edit directly — but commit changes that touch
tracked configs (caddy, gitea custom, artifacto, mule-image, etc.; see tracked configs (caddy, gitea custom, artifacto, mule-image, etc.; see
`infrastructure/auto-deploy.md`). `knowledge/wiki/infrastructure/auto-deploy.md`).
- **Wiki updates**: same-session rule applies to any meaningful state change - **Wiki updates**: same-session rule applies to any meaningful state change
this client makes. this client makes.

View File

@@ -12,23 +12,23 @@ Living documentation for the **hubris** Proxmox homelab. Every node, every cross
- [Cross-cutting infrastructure](#cross-cutting-infrastructure) — DNS, ingress, mesh, storage, auth - [Cross-cutting infrastructure](#cross-cutting-infrastructure) — DNS, ingress, mesh, storage, auth
- [Investigations](#investigations) — incident timelines - [Investigations](#investigations) — incident timelines
- [Operations](#operations) — cheatsheet, enrollment, runbooks - [Operations](#operations) — cheatsheet, enrollment, runbooks
- [Glossary](GLOSSARY.md) — term definitions - [Glossary](knowledge/GLOSSARY.md) — term definitions
- [Conventions](#conventions) — wiki style, changelog hygiene, same-session update rule - [Conventions](#conventions) — wiki style, changelog hygiene, same-session update rule
## Map ## Map
### Proxmox hosts ### Proxmox hosts
- [`hubris`](hosts/hubris.md) — PVE node, GMKtec NucBox M6 Ultra, `192.168.8.77` — primary host, runs [8 LXCs](containers/index.md) + 2 VMs - [`hubris`](knowledge/wiki/hosts/hubris.md) — PVE node, GMKtec NucBox M6 Ultra, `192.168.8.77` — primary host, runs [8 LXCs](knowledge/wiki/containers/index.md) + 2 VMs
- [`strong`](hosts/strong.md) — PVE node (cluster hostname `strong`), `192.168.178.181` — 2nd member of `Homelab` cluster. Hosts [7 LXCs](containers/index.md) migrated from hubris (Phase 1+2, 2026-07-05) - [`strong`](knowledge/wiki/hosts/strong.md) — PVE node (cluster hostname `strong`), `192.168.178.181` — 2nd member of `Homelab` cluster. Hosts [7 LXCs](knowledge/wiki/containers/index.md) migrated from hubris (Phase 1+2, 2026-07-05)
### VMs ### VMs
- [100 — `zimaos`](vms/100-zimaos.md) — ZimaOS 1.6.1, NAS frontend (evaluation) - [100 — `zimaos`](knowledge/wiki/vms/100-zimaos.md) — ZimaOS 1.6.1, NAS frontend (evaluation)
- [108 — `haos-16.3`](vms/108-haos.md) — Home Assistant OS - [108 — `haos-16.3`](knowledge/wiki/vms/108-haos.md) — Home Assistant OS
- See [vms/index.md](vms/index.md) for the full table - See [vms/index.md](knowledge/wiki/vms/index.md) for the full table
### LXC containers ### LXC containers
See the full table with IPs, hosts, mounts, and status in See the full table with IPs, hosts, mounts, and status in
[`containers/index.md`](containers/index.md). Quick summary: [`containers/index.md`](knowledge/wiki/containers/index.md). Quick summary:
- **hubris** (10 active): 102 nfs-export, 103 paperless, 104 gitea, 105 apps, - **hubris** (10 active): 102 nfs-export, 103 paperless, 104 gitea, 105 apps,
114 nextcloud, 119 sophia, 120 mule-images, 121 caddy, 124 authentik (outpost), 114 nextcloud, 119 sophia, 120 mule-images, 121 caddy, 124 authentik (outpost),
@@ -38,21 +38,21 @@ See the full table with IPs, hosts, mounts, and status in
- **Destroyed (archaeology)**: 100 arr, 106 flaresolverr, 107 marimo, - **Destroyed (archaeology)**: 100 arr, 106 flaresolverr, 107 marimo,
109 syncthing, 110 photoprism, 111 karakeep, 112 immich, 115 reticulum, 109 syncthing, 110 photoprism, 111 karakeep, 112 immich, 115 reticulum,
123 claudio-bot, 125 seafile, 126 plato, 127 mule-photos-new — see 123 claudio-bot, 125 seafile, 126 plato, 127 mule-photos-new — see
[containers/index.md](containers/index.md#recently-destroyed-kept-for-archaeology) [containers/index.md](knowledge/wiki/containers/index.md#recently-destroyed-kept-for-archaeology)
### Cross-cutting infrastructure ### Cross-cutting infrastructure
- [Infrastructure index](infrastructure/index.md) — map of every cross-cutting system - [Infrastructure index](knowledge/wiki/infrastructure/index.md) — map of every cross-cutting system
- [Glossary](GLOSSARY.md) — term definitions - [Glossary](knowledge/GLOSSARY.md) — term definitions
- [DNS — split-horizon](infrastructure/dns.md) - [DNS — split-horizon](knowledge/wiki/infrastructure/dns.md)
- [Ingress — Caddy + VPS traefik](infrastructure/ingress.md) - [Ingress — Caddy + VPS traefik](knowledge/wiki/infrastructure/ingress.md)
- [Mesh — Tailscale → Netbird migration](infrastructure/mesh.md) - [Mesh — Tailscale → Netbird migration](knowledge/wiki/infrastructure/mesh.md)
- [Monitoring — Hermes health watchdog](infrastructure/monitoring.md) - [Monitoring — Hermes health watchdog](knowledge/wiki/infrastructure/monitoring.md)
- [Media permissions — `media` GID 10000](infrastructure/media-permissions.md) - [Media permissions — `media` GID 10000](knowledge/wiki/infrastructure/media-permissions.md)
- [SSH access](infrastructure/ssh-access.md) - [SSH access](knowledge/wiki/infrastructure/ssh-access.md)
- [Backups — rclone → Proton Drive (LXC 132); restic-on-USB deprecated](infrastructure/backups.md) - [Backups — rclone → Proton Drive (LXC 132); restic-on-USB deprecated](knowledge/wiki/infrastructure/backups.md)
- [Auto-deploy — gitea-webhook pipelines](infrastructure/auto-deploy.md) - [Auto-deploy — gitea-webhook pipelines](knowledge/wiki/infrastructure/auto-deploy.md)
- [VPS hardening — IONOS / netbird control plane](infrastructure/vps-hardening.md) - [VPS hardening — IONOS / netbird control plane](knowledge/wiki/infrastructure/vps-hardening.md)
- [Homelab context distribution](infrastructure/homelab-context.md) — cross-client `/opt/homelab-context` + MCP + secrets-issuance - [Homelab context distribution](knowledge/wiki/infrastructure/homelab-context.md) — cross-client `/opt/homelab-context` + MCP + secrets-issuance
### Investigations ### Investigations
Time-stamped incident notes / experiments in [`investigations/index.md`](investigations/index.md). Time-stamped incident notes / experiments in [`investigations/index.md`](investigations/index.md).
@@ -71,7 +71,7 @@ Resolved cases move to [`investigations/archive/`](investigations/archive/).
``` ```
- **Cross-linking is mandatory.** If a page references another node or system, link to it. Treat orphans as a bug. - **Cross-linking is mandatory.** If a page references another node or system, link to it. Treat orphans as a bug.
- **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running config, fix the wiki *and* note the change in the relevant changelog. - **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running config, fix the wiki *and* note the change in the relevant changelog.
- **Tracked configs.** A node whose config lives in a Gitea repo (Caddy, Gitea customizations, Artifacto, mule-image) is auto-deployed via webhook — see [auto-deploy](infrastructure/auto-deploy.md). Edits there must be pushed, not left local. - **Tracked configs.** A node whose config lives in a Gitea repo (Caddy, Gitea customizations, Artifacto, mule-image) is auto-deployed via webhook — see [auto-deploy](knowledge/wiki/infrastructure/auto-deploy.md). Edits there must be pushed, not left local.
- **No secrets.** This is a private repo on `git.hubris.network`, but still: paths to secret files are fine, secret values are not. - **No secrets.** This is a private repo on `git.hubris.network`, but still: paths to secret files are fine, secret values are not.
## Maintaining this wiki ## Maintaining this wiki

View File

@@ -34,7 +34,7 @@ services_hosted:
- name: artifacto - name: artifacto
backend: apps backend: apps
url: https://artifacto.hubris.network url: https://artifacto.hubris.network
doc_page: containers/105-apps.md doc_page: knowledge/wiki/containers/105-apps.md
config_repo: dtoro/Artifacto config_repo: dtoro/Artifacto
- name: homelab_mcp - name: homelab_mcp
backend: apps backend: apps
@@ -42,7 +42,7 @@ services_hosted:
systemd_unit: homelab-mcp systemd_unit: homelab-mcp
public_host: mcp.hubris.network public_host: mcp.hubris.network
endpoint: https://mcp.hubris.network/mcp endpoint: https://mcp.hubris.network/mcp
doc_page: infrastructure/homelab-context.md doc_page: knowledge/wiki/infrastructure/homelab-context.md
config_repo: dtoro/Homelab-Docs config_repo: dtoro/Homelab-Docs
note: MCP server. Read-only context + management. Reachable on the LAN via Caddy and from off-LAN via note: MCP server. Read-only context + management. Reachable on the LAN via Caddy and from off-LAN via
Netbird (192.168.8.0/24 is a network resource routed through hubris). Netbird (192.168.8.0/24 is a network resource routed through hubris).
@@ -58,7 +58,5 @@ services_hosted:
note: Issues per-client age private keys. Gated at source-IP layer (mesh + LAN subnets in MESH_SUBNETS). note: Issues per-client age private keys. Gated at source-IP layer (mesh + LAN subnets in MESH_SUBNETS).
risk_notes: "identity issuance \u2014 any change is security-sensitive; key operations are destructive-class" risk_notes: "identity issuance \u2014 any change is security-sensitive; key operations are destructive-class"
age_pubkey: age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0 age_pubkey: age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0
see_also:
- containers/105-apps.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -29,12 +29,10 @@ services_hosted:
- name: arr_stack - name: arr_stack
backend: arriman backend: arriman
note: jellyseerr / qbit / sab on docker compose note: jellyseerr / qbit / sab on docker compose
doc_page: containers/122-arriman.md doc_page: knowledge/wiki/containers/122-arriman.md
notes: notes:
- Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm. - Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm.
- Contains homarr, radarr, sonarr, lidarr, sabnzbd, qbittorrent, bazarr, flaresolverr, prowlarr, jellyseerr - Contains homarr, radarr, sonarr, lidarr, sabnzbd, qbittorrent, bazarr, flaresolverr, prowlarr, jellyseerr
- qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 (for seanime + Caddy access) - qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 (for seanime + Caddy access)
see_also:
- containers/122-arriman.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -16,7 +16,5 @@ mesh_globals:
- tailscale - tailscale
notes: notes:
- Runs Authentik outpost (reverse-proxy/SSO enforcement) for protected services - Runs Authentik outpost (reverse-proxy/SSO enforcement) for protected services
see_also:
- containers/106-auth-outpost.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -24,14 +24,12 @@ services_hosted:
backend: caddy backend: caddy
role: reverse-proxy role: reverse-proxy
note: terminates all *.hubris.network note: terminates all *.hubris.network
doc_page: containers/121-caddy.md doc_page: knowledge/wiki/containers/121-caddy.md
config_repo: dtoro/caddy-conf config_repo: dtoro/caddy-conf
risk_notes: "wide blast radius \u2014 every *.hubris.network route rides on it (see oikos/policy.yaml\ risk_notes: "wide blast radius \u2014 every *.hubris.network route rides on it (see oikos/policy.yaml\
\ service_overrides)" \ service_overrides)"
notes: notes:
- Terminates all *.hubris.network - Terminates all *.hubris.network
- /etc/caddy is a git checkout of dtoro/caddy-conf - /etc/caddy is a git checkout of dtoro/caddy-conf
see_also:
- containers/121-caddy.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -20,12 +20,10 @@ services_hosted:
- name: dns - name: dns
backend: dns backend: dns
note: Technitium DNS, split-horizon zone note: Technitium DNS, split-horizon zone
doc_page: containers/107-dns.md doc_page: knowledge/wiki/containers/107-dns.md
risk_notes: "LAN-wide resolver \u2014 misconfig breaks name resolution for every client" risk_notes: "LAN-wide resolver \u2014 misconfig breaks name resolution for every client"
notes: notes:
- Technitium DNS, split-horizon zone for *.hubris.network - Technitium DNS, split-horizon zone for *.hubris.network
- Primary DNS for 192.168.8.0/24 LAN (inventory.services.dns references this) - Primary DNS for 192.168.8.0/24 LAN (inventory.services.dns references this)
see_also:
- containers/107-dns.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -23,11 +23,9 @@ services_hosted:
- name: matrix - name: matrix
url: https://matrix.hubris.network url: https://matrix.hubris.network
backend: elementsynapse backend: elementsynapse
doc_page: containers/118-elementsynapse.md doc_page: knowledge/wiki/containers/118-elementsynapse.md
risk_notes: "alert/approval channel for Oikos \u2014 outage silences agent escalation" risk_notes: "alert/approval channel for Oikos \u2014 outage silences agent escalation"
notes: notes:
- Migrated from hubris to strong 2026-07-05 (Phase 1 of strong migration plan). - Migrated from hubris to strong 2026-07-05 (Phase 1 of strong migration plan).
see_also:
- containers/118-elementsynapse.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -27,12 +27,10 @@ services_hosted:
url: https://git.hubris.network url: https://git.hubris.network
backend: gitea backend: gitea
backend_url: http://192.168.8.121:3000 backend_url: http://192.168.8.121:3000
doc_page: containers/104-gitea.md doc_page: knowledge/wiki/containers/104-gitea.md
config_repo: dtoro/gitea-customizations config_repo: dtoro/gitea-customizations
risk_notes: hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync risk_notes: hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync
notes: notes:
- Bare repos live at /mnt/library/repos/dtoro/*.git - Bare repos live at /mnt/library/repos/dtoro/*.git
see_also:
- containers/104-gitea.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -21,7 +21,5 @@ notes:
- Docker host for Grimmory (community fork of Booklore). Created 2026-06-29. - Docker host for Grimmory (community fork of Booklore). Created 2026-06-29.
- Migrated from hubris to strong 2026-07-05 (Phase 2d). Books on ludo-lvm. - Migrated from hubris to strong 2026-07-05 (Phase 2d). Books on ludo-lvm.
age_pubkey: age1uellsemnjrzgfg9fxw4jefpy05laxzggwnwhh6ny3wl7alyp6v8q0muxet age_pubkey: age1uellsemnjrzgfg9fxw4jefpy05laxzggwnwhh6ny3wl7alyp6v8q0muxet
see_also:
- containers/130-grimmory.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -22,8 +22,6 @@ runs:
services_hosted: services_hosted:
- name: haos - name: haos
backend: haos backend: haos
doc_page: vms/108-haos.md doc_page: knowledge/wiki/vms/108-haos.md
see_also:
- vms/108-haos.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -22,7 +22,5 @@ notes:
- 192.168.8.212 was the hubris IP before migration (briefly picked up by teddycloud via DHCP; teddycloud - 192.168.8.212 was the hubris IP before migration (briefly picked up by teddycloud via DHCP; teddycloud
has since been given a static IP, see hosts.teddycloud) has since been given a static IP, see hosts.teddycloud)
age_pubkey: age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h age_pubkey: age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
see_also:
- containers/129-house.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -29,10 +29,8 @@ services_hosted:
url: https://proxmox.hubris.network url: https://proxmox.hubris.network
backend: hubris backend: hubris
port: 8006 port: 8006
doc_page: hosts/hubris.md doc_page: knowledge/wiki/hosts/hubris.md
risk_notes: "hypervisor UI \u2014 changes here affect every guest on the node" risk_notes: "hypervisor UI \u2014 changes here affect every guest on the node"
age_pubkey: age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6 age_pubkey: age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6
see_also:
- hosts/hubris.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -26,7 +26,7 @@ services_hosted:
- name: jellyfin - name: jellyfin
url: https://media.hubris.network url: https://media.hubris.network
backend: jellyfin backend: jellyfin
doc_page: containers/101-jellyfin.md doc_page: knowledge/wiki/containers/101-jellyfin.md
risk_notes: native Authentik OIDC via SSO-Auth plugin, no Caddy forward-auth gate; VAAPI transcode depends risk_notes: native Authentik OIDC via SSO-Auth plugin, no Caddy forward-auth gate; VAAPI transcode depends
on GPU passthrough on strong on GPU passthrough on strong
notes: notes:
@@ -35,7 +35,5 @@ notes:
- SSO-Auth plugin v4.0.0.4 with Authentik OIDC (no Caddy forward-auth gate) - SSO-Auth plugin v4.0.0.4 with Authentik OIDC (no Caddy forward-auth gate)
- GPU passed via dev0+dev1: /dev/dri/renderD128 + card0 - GPU passed via dev0+dev1: /dev/dri/renderD128 + card0
- Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm. - Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm.
see_also:
- containers/101-jellyfin.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -26,9 +26,7 @@ services_hosted:
- name: photos - name: photos
url: https://photos.hubris.network url: https://photos.hubris.network
backend: mule-images backend: mule-images
doc_page: containers/120-mule-images.md doc_page: knowledge/wiki/containers/120-mule-images.md
config_repo: dtoro/mule-image config_repo: dtoro/mule-image
see_also:
- containers/120-mule-images.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -23,7 +23,7 @@ services_hosted:
- name: authentik - name: authentik
url: https://auth.hubris.network url: https://auth.hubris.network
backend: netbird-vps backend: netbird-vps
doc_page: containers/106-auth-outpost.md doc_page: knowledge/wiki/containers/106-auth-outpost.md
note: core runs on the VPS since 2026-05-31; LAN forward-auth outpost is auth-outpost (LXC 106) at 192.168.8.6:9000. note: core runs on the VPS since 2026-05-31; LAN forward-auth outpost is auth-outpost (LXC 106) at 192.168.8.6:9000.
Previous backend value "authentik" referenced the retired embedded-outpost host (LXC 124). Previous backend value "authentik" referenced the retired embedded-outpost host (LXC 124).
risk_notes: "SSO provider \u2014 outage locks login to OIDC/forward-auth services" risk_notes: "SSO provider \u2014 outage locks login to OIDC/forward-auth services"

View File

@@ -26,8 +26,6 @@ services_hosted:
- name: nextcloud - name: nextcloud
url: https://cloud.hubris.network url: https://cloud.hubris.network
backend: nextcloud backend: nextcloud
doc_page: containers/114-nextcloud.md doc_page: knowledge/wiki/containers/114-nextcloud.md
see_also:
- containers/114-nextcloud.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -14,7 +14,5 @@ mesh_globals:
accepted: accepted:
- netbird - netbird
- tailscale - tailscale
see_also:
- containers/102-nfs-export.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -26,9 +26,7 @@ services_hosted:
- name: paperless - name: paperless
url: https://paperless.hubris.network url: https://paperless.hubris.network
backend: paperless backend: paperless
doc_page: containers/103-paperless.md doc_page: knowledge/wiki/containers/103-paperless.md
risk_notes: "document archive \u2014 treat data as irreplaceable; DB operations are destructive-class" risk_notes: "document archive \u2014 treat data as irreplaceable; DB operations are destructive-class"
see_also:
- containers/103-paperless.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -22,7 +22,5 @@ notes:
- MariaDB sidecar at /opt/romm/docker-compose.yml. - MariaDB sidecar at /opt/romm/docker-compose.yml.
- ROMs on ludo-lvm media volume at /mnt/media_local/roms. - ROMs on ludo-lvm media volume at /mnt/media_local/roms.
- 1 core / 2 GiB RAM / 16 GiB rootfs (ludo-lvm). - 1 core / 2 GiB RAM / 16 GiB rootfs (ludo-lvm).
see_also:
- containers/134-romm.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -25,7 +25,5 @@ notes:
- /anime mounted from strong ludo-lvm (/mnt/media_local/anime) - /anime mounted from strong ludo-lvm (/mnt/media_local/anime)
- Caddy: "https://seanime.hubris.network \u2192 192.168.8.248:43211" - Caddy: "https://seanime.hubris.network \u2192 192.168.8.248:43211"
- qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 for seanime access - qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 for seanime access
see_also:
- containers/133-seanime.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -19,7 +19,5 @@ mesh_globals:
- tailscale - tailscale
mounts: mounts:
- /mnt/library - /mnt/library
see_also:
- containers/119-sophia.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -28,7 +28,5 @@ notes:
\ (filename kept as-is, it's a historical planning doc). Only Phase 1 (Proxmox install + cluster join)\ \ (filename kept as-is, it's a historical planning doc). Only Phase 1 (Proxmox install + cluster join)\
\ is done; no physical drive move, service migration, or GPU passthrough has happened yet." \ is done; no physical drive move, service migration, or GPU passthrough has happened yet."
age_pubkey: age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4 age_pubkey: age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4
see_also:
- hosts/strong.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -23,7 +23,7 @@ services_hosted:
- name: teddycloud - name: teddycloud
url: https://teddy.hubris.network url: https://teddy.hubris.network
backend: teddycloud backend: teddycloud
doc_page: containers/131-teddycloud.md doc_page: knowledge/wiki/containers/131-teddycloud.md
note: self-hosted TeddyCloud (Toniebox cloud reimplementation), docker compose note: self-hosted TeddyCloud (Toniebox cloud reimplementation), docker compose
risk_notes: "no Caddy forward-auth gate (unlike sab.hubris.network on the same Caddyfile) \u2014 reachable\ risk_notes: "no Caddy forward-auth gate (unlike sab.hubris.network on the same Caddyfile) \u2014 reachable\
\ to anyone on the LAN/mesh who can resolve teddy.hubris.network; undocumented in inventory.yaml until\ \ to anyone on the LAN/mesh who can resolve teddy.hubris.network; undocumented in inventory.yaml until\
@@ -38,7 +38,5 @@ notes:
\ DHCP before that \u2014 see hosts/strong.md's 2026-07-05 changelog)." \ DHCP before that \u2014 see hosts/strong.md's 2026-07-05 changelog)."
- "No age_pubkey / homelab-context enrollment \u2014 not a homelab CLI client, just a docker-compose app\ - "No age_pubkey / homelab-context enrollment \u2014 not a homelab CLI client, just a docker-compose app\
\ container. Not a required follow-up unless it needs secrets." \ container. Not a required follow-up unless it needs secrets."
see_also:
- containers/131-teddycloud.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -22,9 +22,7 @@ services_hosted:
backend: trmnl backend: trmnl
url: https://trmnl.hubris.network url: https://trmnl.hubris.network
note: self-hosted middleware for TRMNL e-ink plugins (polled by TRMNL cloud) note: self-hosted middleware for TRMNL e-ink plugins (polled by TRMNL cloud)
doc_page: containers/128-trmnl.md doc_page: knowledge/wiki/containers/128-trmnl.md
config_repo: dtoro/terminalito config_repo: dtoro/terminalito
see_also:
- containers/128-trmnl.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -21,8 +21,6 @@ services_hosted:
- name: zimaos - name: zimaos
url: https://zimaos.hubris.network url: https://zimaos.hubris.network
backend: zimaos backend: zimaos
doc_page: vms/100-zimaos.md doc_page: knowledge/wiki/vms/100-zimaos.md
see_also:
- vms/100-zimaos.md
mcp_endpoint: https://mcp.hubris.network/mcp mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -48,26 +48,26 @@ services:
url: https://proxmox.hubris.network url: https://proxmox.hubris.network
backend: hubris backend: hubris
port: 8006 port: 8006
doc_page: hosts/hubris.md doc_page: knowledge/wiki/hosts/hubris.md
risk_notes: hypervisor UI — changes here affect every guest on the node risk_notes: hypervisor UI — changes here affect every guest on the node
gitea: gitea:
url: https://git.hubris.network url: https://git.hubris.network
backend: gitea backend: gitea
backend_url: http://192.168.8.121:3000 backend_url: http://192.168.8.121:3000
doc_page: containers/104-gitea.md doc_page: knowledge/wiki/containers/104-gitea.md
config_repo: dtoro/gitea-customizations config_repo: dtoro/gitea-customizations
risk_notes: hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync risk_notes: hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync
caddy: caddy:
backend: caddy backend: caddy
role: reverse-proxy role: reverse-proxy
note: terminates all *.hubris.network note: terminates all *.hubris.network
doc_page: containers/121-caddy.md doc_page: knowledge/wiki/containers/121-caddy.md
config_repo: dtoro/caddy-conf config_repo: dtoro/caddy-conf
risk_notes: wide blast radius — every *.hubris.network route rides on it (see oikos/policy.yaml service_overrides) risk_notes: wide blast radius — every *.hubris.network route rides on it (see oikos/policy.yaml service_overrides)
authentik: authentik:
url: https://auth.hubris.network url: https://auth.hubris.network
backend: netbird-vps backend: netbird-vps
doc_page: containers/106-auth-outpost.md doc_page: knowledge/wiki/containers/106-auth-outpost.md
note: >- note: >-
core runs on the VPS since 2026-05-31; LAN forward-auth outpost is core runs on the VPS since 2026-05-31; LAN forward-auth outpost is
auth-outpost (LXC 106) at 192.168.8.6:9000. Previous backend value auth-outpost (LXC 106) at 192.168.8.6:9000. Previous backend value
@@ -76,58 +76,58 @@ services:
dns: dns:
backend: dns backend: dns
note: Technitium DNS, split-horizon zone note: Technitium DNS, split-horizon zone
doc_page: containers/107-dns.md doc_page: knowledge/wiki/containers/107-dns.md
risk_notes: LAN-wide resolver — misconfig breaks name resolution for every client risk_notes: LAN-wide resolver — misconfig breaks name resolution for every client
jellyfin: jellyfin:
url: https://media.hubris.network url: https://media.hubris.network
backend: jellyfin backend: jellyfin
doc_page: containers/101-jellyfin.md doc_page: knowledge/wiki/containers/101-jellyfin.md
risk_notes: native Authentik OIDC via SSO-Auth plugin, no Caddy forward-auth gate; VAAPI transcode depends on GPU passthrough on strong risk_notes: native Authentik OIDC via SSO-Auth plugin, no Caddy forward-auth gate; VAAPI transcode depends on GPU passthrough on strong
nextcloud: nextcloud:
url: https://cloud.hubris.network url: https://cloud.hubris.network
backend: nextcloud backend: nextcloud
doc_page: containers/114-nextcloud.md doc_page: knowledge/wiki/containers/114-nextcloud.md
paperless: paperless:
url: https://paperless.hubris.network url: https://paperless.hubris.network
backend: paperless backend: paperless
doc_page: containers/103-paperless.md doc_page: knowledge/wiki/containers/103-paperless.md
risk_notes: document archive — treat data as irreplaceable; DB operations are destructive-class risk_notes: document archive — treat data as irreplaceable; DB operations are destructive-class
matrix: matrix:
url: https://matrix.hubris.network url: https://matrix.hubris.network
backend: elementsynapse backend: elementsynapse
doc_page: containers/118-elementsynapse.md doc_page: knowledge/wiki/containers/118-elementsynapse.md
risk_notes: alert/approval channel for Oikos — outage silences agent escalation risk_notes: alert/approval channel for Oikos — outage silences agent escalation
photos: photos:
url: https://photos.hubris.network url: https://photos.hubris.network
backend: mule-images backend: mule-images
doc_page: containers/120-mule-images.md doc_page: knowledge/wiki/containers/120-mule-images.md
config_repo: dtoro/mule-image config_repo: dtoro/mule-image
arr_stack: arr_stack:
backend: arriman backend: arriman
note: jellyseerr / qbit / sab on docker compose note: jellyseerr / qbit / sab on docker compose
doc_page: containers/122-arriman.md doc_page: knowledge/wiki/containers/122-arriman.md
artifacto: artifacto:
backend: apps backend: apps
url: https://artifacto.hubris.network url: https://artifacto.hubris.network
doc_page: containers/105-apps.md doc_page: knowledge/wiki/containers/105-apps.md
config_repo: dtoro/Artifacto config_repo: dtoro/Artifacto
trmnl: trmnl:
backend: trmnl backend: trmnl
url: https://trmnl.hubris.network url: https://trmnl.hubris.network
note: self-hosted middleware for TRMNL e-ink plugins (polled by TRMNL cloud) note: self-hosted middleware for TRMNL e-ink plugins (polled by TRMNL cloud)
doc_page: containers/128-trmnl.md doc_page: knowledge/wiki/containers/128-trmnl.md
config_repo: dtoro/terminalito config_repo: dtoro/terminalito
zimaos: zimaos:
url: https://zimaos.hubris.network url: https://zimaos.hubris.network
backend: zimaos backend: zimaos
doc_page: vms/100-zimaos.md doc_page: knowledge/wiki/vms/100-zimaos.md
haos: haos:
backend: haos backend: haos
doc_page: vms/108-haos.md doc_page: knowledge/wiki/vms/108-haos.md
teddycloud: teddycloud:
url: https://teddy.hubris.network url: https://teddy.hubris.network
backend: teddycloud backend: teddycloud
doc_page: containers/131-teddycloud.md doc_page: knowledge/wiki/containers/131-teddycloud.md
note: self-hosted TeddyCloud (Toniebox cloud reimplementation), docker compose note: self-hosted TeddyCloud (Toniebox cloud reimplementation), docker compose
risk_notes: no Caddy forward-auth gate (unlike sab.hubris.network on the same Caddyfile) — risk_notes: no Caddy forward-auth gate (unlike sab.hubris.network on the same Caddyfile) —
reachable to anyone on the LAN/mesh who can resolve teddy.hubris.network; undocumented reachable to anyone on the LAN/mesh who can resolve teddy.hubris.network; undocumented
@@ -138,7 +138,7 @@ services:
systemd_unit: homelab-mcp systemd_unit: homelab-mcp
public_host: mcp.hubris.network public_host: mcp.hubris.network
endpoint: https://mcp.hubris.network/mcp endpoint: https://mcp.hubris.network/mcp
doc_page: infrastructure/homelab-context.md doc_page: knowledge/wiki/infrastructure/homelab-context.md
config_repo: dtoro/Homelab-Docs config_repo: dtoro/Homelab-Docs
note: MCP server. Read-only context + management. Reachable on the LAN via Caddy note: MCP server. Read-only context + management. Reachable on the LAN via Caddy
and from off-LAN via Netbird (192.168.8.0/24 is a network resource routed through and from off-LAN via Netbird (192.168.8.0/24 is a network resource routed through

View File

@@ -90,9 +90,9 @@ print("session_duration:", stage.session_duration) # → "days=30"
## Related ## Related
- [Container 106 — auth-outpost](../containers/106-auth-outpost.md) - [Container 106 — auth-outpost](../knowledge/wiki/containers/106-auth-outpost.md)
- [Authentik VPS migration](2026-05-31-authentik-vps-migration.md) - [Authentik VPS migration](2026-05-31-authentik-vps-migration.md)
- [Ingress (VPS Traefik)](../infrastructure/ingress.md) - [Ingress (VPS Traefik)](../knowledge/wiki/infrastructure/ingress.md)
- `.hermes/plans/2026-06-06_232200-authentik-frequent-login-fix.md` — original plan - `.hermes/plans/2026-06-06_232200-authentik-frequent-login-fix.md` — original plan
## Changelog ## Changelog

View File

@@ -55,7 +55,7 @@ This is the same class of drift as the June 5th incidents (paperless, HAOS, apps
## Related ## Related
- [DHCP drift investigation (previous incident)](2026-06-05-homelab-dhcp-drift.md) - [DHCP drift investigation (previous incident)](2026-06-05-homelab-dhcp-drift.md)
- [Caddy (121)](../containers/121-caddy.md) - [Caddy (121)](../knowledge/wiki/containers/121-caddy.md)
- [elementsynapse (118)](../containers/118-elementsynapse.md) - [elementsynapse (118)](../knowledge/wiki/containers/118-elementsynapse.md)
- [dns-sync script](../scripts/dns-sync.py) - [dns-sync script](../scripts/dns-sync.py)
- [check-caddy-backends script](../scripts/check-caddy-backends.sh) - [check-caddy-backends script](../scripts/check-caddy-backends.sh)

View File

@@ -18,7 +18,7 @@ Terms and abbreviations used throughout the homelab wiki.
| **Mesh** | Overlay VPN for off-LAN connectivity. Netbird is current; Tailscale is legacy | | **Mesh** | Overlay VPN for off-LAN connectivity. Netbird is current; Tailscale is legacy |
| **Netbird** | Preferred mesh VPN. VPS hosts the management plane; all homelab nodes are members | | **Netbird** | Preferred mesh VPN. VPS hosts the management plane; all homelab nodes are members |
| **OIDC** | OpenID Connect. Protocol used by Authentik for SSO login flows | | **OIDC** | OpenID Connect. Protocol used by Authentik for SSO login flows |
| **Oikos** | Agent operating model ([.agents/OIKOS.md](.agents/OIKOS.md)). OODA loop, risk classes, policy, ontology | | **Oikos** | Agent operating model ([.agents/OIKOS.md](../.agents/OIKOS.md)). OODA loop, risk classes, policy, ontology |
| **PVE** | Proxmox Virtual Environment — the hypervisor on both hubris and strong | | **PVE** | Proxmox Virtual Environment — the hypervisor on both hubris and strong |
| **SOPS** | `sops` — Mozilla SOPS. Encrypts secrets with age keys so they live in the git repo | | **SOPS** | `sops` — Mozilla SOPS. Encrypts secrets with age keys so they live in the git repo |
| **Strong** | Secondary Proxmox VE node. Cluster member 2 (hostname `strong`, nickname ludo/ludo-mini) | | **Strong** | Secondary Proxmox VE node. Cluster member 2 (hostname `strong`, nickname ludo/ludo-mini) |
@@ -29,5 +29,5 @@ Terms and abbreviations used throughout the homelab wiki.
## See also ## See also
- [Infrastructure index](infrastructure/index.md) — cross-cutting systems each with their own doc page - [Infrastructure index](wiki/infrastructure/index.md) — cross-cutting systems each with their own doc page
- [OIKOS operating model](.agents/OIKOS.md) — agent policy, risk classes, lifecycle - [OIKOS operating model](../.agents/OIKOS.md) — agent policy, risk classes, lifecycle

14
knowledge/index.md Normal file
View File

@@ -0,0 +1,14 @@
# Knowledge
The durable, authoritative current-state documentation of the homelab: one page per node and per
cross-cutting system, synthesized from live state and evidence. Structure and rules are in
[the knowledge schema](../.agents/domains/knowledge/schema.md).
| Section | What it covers |
|---------|----------------|
| [wiki/hosts/](wiki/hosts/) | Proxmox host narratives — `hubris`, `strong`. |
| [wiki/containers/](wiki/containers/index.md) | LXC fleet — one page per container, plus the master table and archaeology. |
| [wiki/vms/](wiki/vms/index.md) | Virtual machines — ZimaOS, Home Assistant OS. |
| [wiki/infrastructure/](wiki/infrastructure/index.md) | Cross-cutting systems — DNS, ingress, mesh, storage, auth, monitoring, generated topology. |
| [sources/](sources/index.md) | External reference docs and the pointer to incident evidence. |
| [GLOSSARY.md](GLOSSARY.md) | Term definitions. |

7
knowledge/log.md Normal file
View File

@@ -0,0 +1,7 @@
# Knowledge — operations log
Append-only record of documentation-maintenance operations on the knowledge wiki (restructures,
source ingests, lint sweeps). One line per operation, newest last. Infrastructure changes belong in
each page's `## Changelog` and the Oikos change ledger, not here.
## [2026-07-06] restructure | moved node/infrastructure narratives under knowledge/wiki/; references under knowledge/sources/; repointed inventory doc_page fields and gen-topology.py output.

View File

@@ -0,0 +1,9 @@
# Sources
Immutable evidence the wiki synthesizes from. External reference docs live under `references/`;
incident evidence lives in [`investigations/`](../../investigations/index.md) (written once at
incident time, then linked from the changelogs of the nodes they implicate).
| Slug | Reference | Summary |
|------|-----------|---------|
| cert-sync-and-traefik-config | [references/cert-sync-and-traefik-config.md](references/cert-sync-and-traefik-config.md) | VPS traefik config and the LAN↔VPS certificate mirror. |

View File

@@ -78,7 +78,7 @@ User → media.hubris.network → Caddy (TLS, no forward-auth) → Jellyfin :809
`http://` redirect URIs that Authentik rejects) `http://` redirect URIs that Authentik rejects)
- `EnableAuthorization`: `false` (prevents plugin from overwriting admin - `EnableAuthorization`: `false` (prevents plugin from overwriting admin
permissions on each SSO login — see permissions on each SSO login — see
[jellyfin-sso-plugin](../devops/homelab-authentik-admin/references/jellyfin-sso-plugin.md)) [jellyfin-sso-plugin](../../../devops/homelab-authentik-admin/references/jellyfin-sso-plugin.md))
- `OidScopes`: `["email"]` (openid+profile added by default by the plugin; - `OidScopes`: `["email"]` (openid+profile added by default by the plugin;
must be non-null or `OidChallenge()` throws `ArgumentNullException`) must be non-null or `OidChallenge()` throws `ArgumentNullException`)
@@ -133,7 +133,7 @@ Member of the [media GID 10000](../infrastructure/media-permissions.md) standard
- [Media permissions](../infrastructure/media-permissions.md) - [Media permissions](../infrastructure/media-permissions.md)
- [arriman](122-arriman.md) — \*arr stack writes the libraries jellyfin reads - [arriman](122-arriman.md) — \*arr stack writes the libraries jellyfin reads
- [DNS split-horizon](../infrastructure/dns.md) - [DNS split-horizon](../infrastructure/dns.md)
- [Authentik admin](../devops/homelab-authentik-admin/SKILL.md) — OIDC provider creation, SSO plugin config - [Authentik admin](../../../devops/homelab-authentik-admin/SKILL.md) — OIDC provider creation, SSO plugin config
## Changelog ## Changelog

View File

@@ -54,7 +54,7 @@ We considered three options before building this:
| Option | Outcome | | Option | Outcome |
|---|---| |---|---|
| **NFS on hubris bare-metal host** | Best performance, but adds long-lived NFS/RPC daemons to a host with a recent crash episode ([hubris crash 2026-04-21/22](../investigations/index.md)). Rejected. | | **NFS on hubris bare-metal host** | Best performance, but adds long-lived NFS/RPC daemons to a host with a recent crash episode ([hubris crash 2026-04-21/22](../../../investigations/index.md)). Rejected. |
| **SMB on host** | Same host-blast-radius problem, plus 3050% lower throughput than NFS on Linux↔Linux. Rejected. | | **SMB on host** | Same host-blast-radius problem, plus 3050% lower throughput than NFS on Linux↔Linux. Rejected. |
| **NFS in a dedicated LXC** ← this | Within ~2% of host performance (LXC is namespace isolation; IO path is unchanged), zero new daemons on hubris, matches the existing fleet pattern. Selected. | | **NFS in a dedicated LXC** ← this | Within ~2% of host performance (LXC is namespace isolation; IO path is unchanged), zero new daemons on hubris, matches the existing fleet pattern. Selected. |

View File

@@ -1,6 +1,6 @@
# 106 — `auth-outpost` # 106 — `auth-outpost`
Authentik **forward-auth outpost** for LAN-gated apps. A stateless proxy that connects outbound to the [VPS Authentik core](../investigations/2026-05-31-authentik-vps-migration.md) and serves forward-auth locally, so [Caddy (121)](121-caddy.md) never hairpins auth through VPS Traefik. Authentik **forward-auth outpost** for LAN-gated apps. A stateless proxy that connects outbound to the [VPS Authentik core](../../../investigations/2026-05-31-authentik-vps-migration.md) and serves forward-auth locally, so [Caddy (121)](121-caddy.md) never hairpins auth through VPS Traefik.
## At a glance ## At a glance
- **Hostname:** `auth-outpost` - **Hostname:** `auth-outpost`
@@ -12,7 +12,7 @@ Authentik **forward-auth outpost** for LAN-gated apps. A stateless proxy that co
## Role ## Role
Runs one container — `ghcr.io/goauthentik/proxy` — that opens an outbound websocket to `https://auth.hubris.network` (the VPS core), pulls its proxy-provider config, and answers Caddy's `forward_auth` subrequests on `192.168.8.6:9000` (LAN-only bind). Because the call path is **Caddy → outpost (LAN)**, with no Traefik in between, `X-Forwarded-Host` is preserved — the failure that 404s when Caddy is pointed at `https://auth.hubris.network` directly (Traefik rewrites the header). See the [migration investigation](../investigations/2026-05-31-authentik-vps-migration.md). Runs one container — `ghcr.io/goauthentik/proxy` — that opens an outbound websocket to `https://auth.hubris.network` (the VPS core), pulls its proxy-provider config, and answers Caddy's `forward_auth` subrequests on `192.168.8.6:9000` (LAN-only bind). Because the call path is **Caddy → outpost (LAN)**, with no Traefik in between, `X-Forwarded-Host` is preserved — the failure that 404s when Caddy is pointed at `https://auth.hubris.network` directly (Traefik rewrites the header). See the [migration investigation](../../../investigations/2026-05-31-authentik-vps-migration.md).
## Service / port map ## Service / port map
| Service | Listen | Notes | | Service | Listen | Notes |
@@ -45,12 +45,12 @@ Fix: the LAN outpost gets its **own** domain.
- [124 — authentik](124-authentik.md) — old embedded-outpost host (now DNS-only) - [124 — authentik](124-authentik.md) — old embedded-outpost host (now DNS-only)
- [Caddy (121)](121-caddy.md) — forward-auth consumer - [Caddy (121)](121-caddy.md) — forward-auth consumer
- [Ingress (VPS traefik)](../infrastructure/ingress.md) - [Ingress (VPS traefik)](../infrastructure/ingress.md)
- [Authentik VPS migration](../investigations/2026-05-31-authentik-vps-migration.md) - [Authentik VPS migration](../../../investigations/2026-05-31-authentik-vps-migration.md)
## Changelog ## Changelog
### 2026-06-06 — Authentik session lifetime extended to 30 days ### 2026-06-06 — Authentik session lifetime extended to 30 days
VPS Authentik core `user_login` stage updated: `session_duration` changed from `seconds=0` (session cookie, cleared on browser close) to `days=30` (persistent 30-day cookie). Also set `AUTHENTIK_SESSIONS__UNAUTHENTICATED_AGE=days=30` in `/opt/authentik.env` on the VPS. See [investigation](../investigations/2026-06-06-authentik-session-lifetime.md). VPS Authentik core `user_login` stage updated: `session_duration` changed from `seconds=0` (session cookie, cleared on browser close) to `days=30` (persistent 30-day cookie). Also set `AUTHENTIK_SESSIONS__UNAUTHENTICATED_AGE=days=30` in `/opt/authentik.env` on the VPS. See [investigation](../../../investigations/2026-06-06-authentik-session-lifetime.md).
### 2026-06-01 — created; forward-auth cut over from LXC 124 ### 2026-06-01 — created; forward-auth cut over from LXC 124
New dedicated LXC for the LAN forward-auth outpost (Phase 1 of the [architecture migration](../investigations/2026-05-31-authentik-vps-migration.md)). Deployed `goauthentik/proxy:2026.5.2` pointed at the VPS core; repointed Caddy `(authentik)` from `192.168.8.180:9000``192.168.8.6:9000`. Verified Paperless/qBittorrent/Artifacto return the SSO redirect with **124-Authentik stopped**, confirming the frozen instance is out of the path. dnsmasq stays on 124 until [DNS is relocated](124-authentik.md). New dedicated LXC for the LAN forward-auth outpost (Phase 1 of the [architecture migration](../../../investigations/2026-05-31-authentik-vps-migration.md)). Deployed `goauthentik/proxy:2026.5.2` pointed at the VPS core; repointed Caddy `(authentik)` from `192.168.8.180:9000``192.168.8.6:9000`. Verified Paperless/qBittorrent/Artifacto return the SSO redirect with **124-Authentik stopped**, confirming the frozen instance is out of the path. dnsmasq stays on 124 until [DNS is relocated](124-authentik.md).

View File

@@ -29,7 +29,7 @@ Authoritative split-horizon DNS for `hubris.network` on the LAN/mesh, plus recur
- **Plain LAN clients (`192.168.178.x`):** Fritz!Box DHCP still hands out Fritz!Box itself (`192.168.178.1`) as DNS — no split-horizon for non-mesh clients. Changing this requires a secondary DNS fallback, which Fritz!OS 8.x doesn't expose in a single DHCP field. - **Plain LAN clients (`192.168.178.x`):** Fritz!Box DHCP still hands out Fritz!Box itself (`192.168.178.1`) as DNS — no split-horizon for non-mesh clients. Changing this requires a secondary DNS fallback, which Fritz!OS 8.x doesn't expose in a single DHCP field.
## dns-sync (Technitium = authoring source) ## dns-sync (Technitium = authoring source)
`/opt/dns-sync/sync.py` (cron `*/10`, logs `/var/log/dns-sync.log`) reconciles this zone's named A-records → the NetBird managed DNS zone via the NetBird API (`/api/dns/zones/{id}/records`). Token at `/opt/dns-sync/netbird-token` (mode 600; source of truth in sops `secrets/netbird-pat.yaml`). **Edit DNS only here**; the sync propagates to the mesh. It deletes NetBird records absent from Technitium. Tracked: [scripts/dns-sync.py](../scripts/dns-sync.py). *Why this exists:* NetBird won't forward to Technitium for mesh peers (self-IP / nameserver-group quirks), so we sync into the managed zone instead — see [dns.md](../infrastructure/dns.md). `/opt/dns-sync/sync.py` (cron `*/10`, logs `/var/log/dns-sync.log`) reconciles this zone's named A-records → the NetBird managed DNS zone via the NetBird API (`/api/dns/zones/{id}/records`). Token at `/opt/dns-sync/netbird-token` (mode 600; source of truth in sops `secrets/netbird-pat.yaml`). **Edit DNS only here**; the sync propagates to the mesh. It deletes NetBird records absent from Technitium. Tracked: [scripts/dns-sync.py](../../../scripts/dns-sync.py). *Why this exists:* NetBird won't forward to Technitium for mesh peers (self-IP / nameserver-group quirks), so we sync into the managed zone instead — see [dns.md](../infrastructure/dns.md).
## DHCP ## DHCP
@@ -55,7 +55,7 @@ Added for [trmnl (128)](128-trmnl.md) (LAN path via [Caddy (121)](121-caddy.md))
Although the 2026-06-03 changelog claimed "cron */10", **no crontab was actually configured** on the LXC. The sync was running only via ad-hoc manual invocations during incident debugging. Fixed by adding `/etc/cron.d/dns-sync`. Although the 2026-06-03 changelog claimed "cron */10", **no crontab was actually configured** on the LXC. The sync was running only via ad-hoc manual invocations during incident debugging. Fixed by adding `/etc/cron.d/dns-sync`.
### 2026-06-03 — DHCP pool narrowed to `.241.254` ### 2026-06-03 — DHCP pool narrowed to `.241.254`
Previous pool `.100.240` overlapped with all static LXCs/VMs (`.101.239`). Shrunk via API (`/api/dhcp/scopes/set`). 11 stale DHCP leases in `.101.110` remain until natural expiry (2026-06-04). See [plan](../plans/2026-06-03-dhcp-pool-exclude-static-ips.md). Previous pool `.100.240` overlapped with all static LXCs/VMs (`.101.239`). Shrunk via API (`/api/dhcp/scopes/set`). 11 stale DHCP leases in `.101.110` remain until natural expiry (2026-06-04). See [plan](../../../plans/2026-06-03-dhcp-pool-exclude-static-ips.md).
### 2026-06-03 — dns-sync added (Technitium → NetBird managed zone) ### 2026-06-03 — dns-sync added (Technitium → NetBird managed zone)
This Technitium became the single DNS authoring source; `/opt/dns-sync/sync.py` (cron */10) reconciles named A-records into the NetBird managed zone via the API. Fixed previously-broken mesh names (`sso`, `nfs-export`, `mcp`, `secrets`) by adding them to the managed zone; reaped obsolete `files`/`photos-new`. See [dns.md](../infrastructure/dns.md). This Technitium became the single DNS authoring source; `/opt/dns-sync/sync.py` (cron */10) reconciles named A-records into the NetBird managed zone via the API. Fixed previously-broken mesh names (`sso`, `nfs-export`, `mcp`, `secrets`) by adding them to the managed zone; reaped obsolete `files`/`photos-new`. See [dns.md](../infrastructure/dns.md).

View File

@@ -85,7 +85,7 @@ Gitea webhook id 2 on `dtoro/caddy-conf`. Receiver, deploy script, install scrip
- **Dirty-tree auto-stash:** stashes local changes before `git pull --ff-only` so the webhook doesn't fail on local edits - **Dirty-tree auto-stash:** stashes local changes before `git pull --ff-only` so the webhook doesn't fail on local edits
- **Auto-backup:** saves `Caddyfile.bak.<timestamp>` before any modifications, keeps last 5 - **Auto-backup:** saves `Caddyfile.bak.<timestamp>` before any modifications, keeps last 5
Also: [elementsynapse LXC 118](../containers/118-elementsynapse.md) found to have DHCP-overridden static IP (actual `.244` vs config `.239`) during incident investigation — fixed. Also: [elementsynapse LXC 118](118-elementsynapse.md) found to have DHCP-overridden static IP (actual `.244` vs config `.239`) during incident investigation — fixed.
### 2026-06-02 — caddy.service unit missing; recreated ### 2026-06-02 — caddy.service unit missing; recreated
After the Slate AX → SODOLA network migration, Caddy was not listening (ports 80/443 dead). Root cause: the custom hubris1 Debian package (`caddy_1:2.11.3-hubris1_amd64`) does not ship a systemd service unit file. The unit had previously existed but was lost (likely on a package reinstall). Recreated at `/lib/systemd/system/caddy.service` with standard Caddy service config + `EnvironmentFile=/etc/caddy/caddy.env` (already present in `caddy.service.d/override.conf`). **Risk:** the unit will be lost again if the package is reinstalled without the file being tracked. Fix: add the service unit to the `caddy-conf` repo or rebuild the hubris1 package to include it. After the Slate AX → SODOLA network migration, Caddy was not listening (ports 80/443 dead). Root cause: the custom hubris1 Debian package (`caddy_1:2.11.3-hubris1_amd64`) does not ship a systemd service unit file. The unit had previously existed but was lost (likely on a package reinstall). Recreated at `/lib/systemd/system/caddy.service` with standard Caddy service config + `EnvironmentFile=/etc/caddy/caddy.env` (already present in `caddy.service.d/override.conf`). **Risk:** the unit will be lost again if the package is reinstalled without the file being tracked. Fix: add the service unit to the `caddy-conf` repo or rebuild the hubris1 package to include it.

View File

@@ -35,7 +35,7 @@ Not yet SOPS-enrolled. The poll token is set directly in `/etc/trmnl-plugins/env
- [VPS ingress](../infrastructure/ingress.md) — public edge (cert mirror + traefik router) - [VPS ingress](../infrastructure/ingress.md) — public edge (cert mirror + traefik router)
- [DNS (107)](107-dns.md) — Technitium A record `trmnl → 192.168.8.175` (LAN path via Caddy) - [DNS (107)](107-dns.md) — Technitium A record `trmnl → 192.168.8.175` (LAN path via Caddy)
- [Gitea (104)](104-gitea.md) — source repo `dtoro/terminalito` - [Gitea (104)](104-gitea.md) — source repo `dtoro/terminalito`
- [Plan: 2026-06-24 TRMNL plugins LXC](../plans/2026-06-24-trmnl-plugins-lxc.md) - [Plan: 2026-06-24 TRMNL plugins LXC](../../../plans/2026-06-24-trmnl-plugins-lxc.md)
## Changelog ## Changelog
### 2026-06-24 — auto-deploy + LAN DNS wired ### 2026-06-24 — auto-deploy + LAN DNS wired

View File

@@ -40,7 +40,7 @@ Yuvomi family planner (formerly Oikos). Self-hosted family planner with 14 modul
- [DNS (107)](107-dns.md) — Technitium A record `house → 192.168.8.175` (LAN path via Caddy) - [DNS (107)](107-dns.md) — Technitium A record `house → 192.168.8.175` (LAN path via Caddy)
- [Paperless (103)](103-paperless.md) — native DMS connector (API at `:8000`) - [Paperless (103)](103-paperless.md) — native DMS connector (API at `:8000`)
- [TRMNL (128)](128-trmnl.md) — Google Calendar tokens source - [TRMNL (128)](128-trmnl.md) — Google Calendar tokens source
- [Deployment plan](../plans/2026-06-25-yuvomi-deployment.md) - [Deployment plan](../../../plans/2026-06-25-yuvomi-deployment.md)
## Changelog ## Changelog

View File

@@ -7,7 +7,7 @@ audio content against a local server instead of the official cloud.
Predates the client-enrollment convention entirely; nobody wrote it down. Found and Predates the client-enrollment convention entirely; nobody wrote it down. Found and
documented on 2026-07-06 after Oikos's drift detector (`oikos/drift.py`) flagged documented on 2026-07-06 after Oikos's drift detector (`oikos/drift.py`) flagged
`pve_id 131` as live on hubris (via `pct list`) with no `inventory.yaml` entry — see `pve_id 131` as live on hubris (via `pct list`) with no `inventory.yaml` entry — see
[OIKOS.md](../OIKOS.md)'s Week 3 build-status note. `containers/132-rclone.md` had already [OIKOS.md](../../../OIKOS.md)'s Week 3 build-status note. `containers/132-rclone.md` had already
mentioned it in passing ("LXC 131 was already taken by an undocumented `teddycloud` mentioned it in passing ("LXC 131 was already taken by an undocumented `teddycloud`
container"), and `hosts/strong.md`'s 2026-07-05 migration changelog fixed a DHCP conflict container"), and `hosts/strong.md`'s 2026-07-05 migration changelog fixed a DHCP conflict
for it — but it never got its own inventory entry or doc page until now. for it — but it never got its own inventory entry or doc page until now.
@@ -50,7 +50,7 @@ for the first time.
- [rclone (132)](132-rclone.md) — landed on pve_id 132 specifically because 131 was already - [rclone (132)](132-rclone.md) — landed on pve_id 132 specifically because 131 was already
taken by this container taken by this container
- [Containers index](index.md) - [Containers index](index.md)
- [OIKOS.md](../OIKOS.md) — drift detector that caught this - [OIKOS.md](../../../OIKOS.md) — drift detector that caught this
## Changelog ## Changelog

View File

@@ -95,7 +95,7 @@ after this set.
Fixed by symlinking `/usr/local/bin/{sops,homelab}` into `/usr/bin` (always on the minimal PATH), Fixed by symlinking `/usr/local/bin/{sops,homelab}` into `/usr/bin` (always on the minimal PATH),
rather than relying on `/etc/environment`. Same category as the documented [`pct exec` no-initgroups rather than relying on `/etc/environment`. Same category as the documented [`pct exec` no-initgroups
gotcha](../infrastructure/media-permissions.md#gotchas) — worth adding to gotcha](../infrastructure/media-permissions.md#gotchas) — worth adding to
[agent-enrollment.md troubleshooting](../operations/agent-enrollment.md#troubleshooting) if it recurs [agent-enrollment.md troubleshooting](../../../operations/agent-enrollment.md#troubleshooting) if it recurs
on future LXC bootstraps. on future LXC bootstraps.
## Known issue: `rclone-rcd.service` OOM-killed under 1 GiB RAM (root cause, resolved) ## Known issue: `rclone-rcd.service` OOM-killed under 1 GiB RAM (root cause, resolved)

View File

@@ -32,7 +32,7 @@ Most containers live on [`hubris`](../hosts/hubris.md). Some have been
| 106 | flaresolverr | ~2026-04-28 | Folded into the arriman docker compose | | 106 | flaresolverr | ~2026-04-28 | Folded into the arriman docker compose |
| 116 | heaper | 2026-05-14 | Decommissioned by user; data subtree at `/mnt/library/heaper` (224 MiB) retained | | 116 | heaper | 2026-05-14 | Decommissioned by user; data subtree at `/mnt/library/heaper` (224 MiB) retained |
| 126 | plato | 2026-06-28 | Notes/discovery workspace decommissioned; data at `/mnt/library/documents/plato` retained for archaeology | | 126 | plato | 2026-06-28 | Notes/discovery workspace decommissioned; data at `/mnt/library/documents/plato` retained for archaeology |
| 123 | claudio-bot (destroyed — see [archive](archive/123-claudio-bot.md)) | 2026-06-04 | Replaced by Hermes Agent on mac-mini; monitoring migrated to `homelab-health-watchdog` cron. See [deprecation plan](../plans/2026-06-04_130000-deprecate-claudio-bot.md) | | 123 | claudio-bot (destroyed — see [archive](archive/123-claudio-bot.md)) | 2026-06-04 | Replaced by Hermes Agent on mac-mini; monitoring migrated to `homelab-health-watchdog` cron. See [deprecation plan](../../../plans/2026-06-04_130000-deprecate-claudio-bot.md) |
| 109 | syncthing | 2026-05-14 | Decommissioned by user; `/mnt/library/syncthing` was already empty | | 109 | syncthing | 2026-05-14 | Decommissioned by user; `/mnt/library/syncthing` was already empty |
| 125 | seafile | 2026-05-13 | Seafile Pro evaluation, user disliked the product; teardown also removed `files.hubris.network` from caddy + dnsmasq | | 125 | seafile | 2026-05-13 | Seafile Pro evaluation, user disliked the product; teardown also removed `files.hubris.network` from caddy + dnsmasq |
| 107 | marimo | between 2026-04-21 and 2026-04-28 | Decommissioned | | 107 | marimo | between 2026-04-21 and 2026-04-28 | Decommissioned |

View File

@@ -8,7 +8,7 @@ workloads still live here. As of 2026-07-01, hubris is node 1 of the 2-node
## At a glance ## At a glance
- **Role:** Proxmox VE 9.1.2 hypervisor (kernel `6.14.11-4-pve`) - **Role:** Proxmox VE 9.1.2 hypervisor (kernel `6.14.11-4-pve`)
- **Hardware:** GMKtec NucBox M6 Ultra — AMD Ryzen 5 7640HS (Phoenix APU), 12 vCPU / ~28 GiB RAM, 2× Samsung 990 EVO Plus NVMe (one SSD primary, one for `library` LVM). 2× Realtek RTL8125 NICs (`r8169`). - **Hardware:** GMKtec NucBox M6 Ultra — AMD Ryzen 5 7640HS (Phoenix APU), 12 vCPU / ~28 GiB RAM, 2× Samsung 990 EVO Plus NVMe (one SSD primary, one for `library` LVM). 2× Realtek RTL8125 NICs (`r8169`).
- **BIOS:** 1.02 (2025-08-06) — vendor not on LVFS, no automated update path. See [investigations](../investigations/2026-04-21-hubris-crash-loop.md). - **BIOS:** 1.02 (2025-08-06) — vendor not on LVFS, no automated update path. See [investigations](../../../investigations/2026-04-21-hubris-crash-loop.md).
- **Uplink:** `vmbr1` (slave: `eno1`) → SODOLA switch → Fritz!Box 7590. DHCP-reserved `192.168.178.10/24`, gateway `192.168.178.1`. - **Uplink:** `vmbr1` (slave: `eno1`) → SODOLA switch → Fritz!Box 7590. DHCP-reserved `192.168.178.10/24`, gateway `192.168.178.1`.
- **Homelab bridge:** `vmbr0` — portless internal bridge, `192.168.8.77/24` + `192.168.8.1/24` alias (LXC default gateway). All 16 LXCs and the HAOS VM are on `vmbr0`. Proxmox routes between `vmbr0` and `vmbr1`; Fritz!Box has a static route `192.168.8.0/24 → 192.168.178.10`. - **Homelab bridge:** `vmbr0` — portless internal bridge, `192.168.8.77/24` + `192.168.8.1/24` alias (LXC default gateway). All 16 LXCs and the HAOS VM are on `vmbr0`. Proxmox routes between `vmbr0` and `vmbr1`; Fritz!Box has a static route `192.168.8.0/24 → 192.168.178.10`.
- **WiFi:** disabled 2026-06-02 — `wlp3s0` removed from `/etc/network/interfaces`, wpa config deleted. Was used as a failover to the now-retired Slate AX AP. - **WiFi:** disabled 2026-06-02 — `wlp3s0` removed from `/etc/network/interfaces`, wpa config deleted. Was used as a failover to the now-retired Slate AX AP.
@@ -46,7 +46,7 @@ Member of `Homelab`, a 2-node Proxmox cluster with [strong](strong.md)
it's a physical thinpool that only exists on this host's hardware. it's a physical thinpool that only exists on this host's hardware.
- strong currently hosts no LXCs/VMs — it exists solely as a cluster - strong currently hosts no LXCs/VMs — it exists solely as a cluster
member so far. See [strong.md](strong.md) and the [library-SSD member so far. See [strong.md](strong.md) and the [library-SSD
migration plan](../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md) migration plan](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)
for what comes next (physical drive move, service migration — not started). for what comes next (physical drive move, service migration — not started).
## Tenants ## Tenants
@@ -113,20 +113,20 @@ OpenSSH on `0.0.0.0:22`. Netbird's built-in SSH server is on `100.122.38.109:220
- [Media permissions](../infrastructure/media-permissions.md) - [Media permissions](../infrastructure/media-permissions.md)
- [Monitoring](../infrastructure/monitoring.md) - [Monitoring](../infrastructure/monitoring.md)
- [Backups (disabled)](../infrastructure/backups.md) - [Backups (disabled)](../infrastructure/backups.md)
- [Operations cheatsheet](../operations/commands.md) - [Operations cheatsheet](../../../operations/commands.md)
- [Investigation: 2026-04-21 crash loop](../investigations/2026-04-21-hubris-crash-loop.md) - [Investigation: 2026-04-21 crash loop](../../../investigations/2026-04-21-hubris-crash-loop.md)
- [strong — Proxmox host](strong.md) - [strong — Proxmox host](strong.md)
## Changelog ## Changelog
### 2026-07-01 — strong joined as a 2nd cluster node ("Homelab") ### 2026-07-01 — strong joined as a 2nd cluster node ("Homelab")
User reformatted `strong` (formerly a Linux dev workstation, `192.168.178.181`) to Proxmox VE 9.2.3. Cluster/OS hostname on that box is `strong` (left as-is from install). Bootstrapped root SSH on strong from a one-time console password (installed hubris's existing trusted key set: `root@hubris`, `d.toro.v@pm.me`), then generated a keypair on strong and pre-authorized it here (`root@strong`) so `pvecm add 192.168.8.77 --use_ssh 1` (run from strong) could join without an interactive password prompt. No cabling/routing changes needed — strong reaches hubris's corosync address (`192.168.8.77`) via the existing Fritz!Box static route. Cluster now 2 nodes, quorate, **no QDevice** (explicit choice — see [Cluster](#cluster) above for the quorum tradeoff this implies). strong hosts no guests yet; this is Phase 1 of the [library-SSD migration plan](../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md), nothing further from that plan has been executed. User reformatted `strong` (formerly a Linux dev workstation, `192.168.178.181`) to Proxmox VE 9.2.3. Cluster/OS hostname on that box is `strong` (left as-is from install). Bootstrapped root SSH on strong from a one-time console password (installed hubris's existing trusted key set: `root@hubris`, `d.toro.v@pm.me`), then generated a keypair on strong and pre-authorized it here (`root@strong`) so `pvecm add 192.168.8.77 --use_ssh 1` (run from strong) could join without an interactive password prompt. No cabling/routing changes needed — strong reaches hubris's corosync address (`192.168.8.77`) via the existing Fritz!Box static route. Cluster now 2 nodes, quorate, **no QDevice** (explicit choice — see [Cluster](#cluster) above for the quorum tradeoff this implies). strong hosts no guests yet; this is Phase 1 of the [library-SSD migration plan](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md), nothing further from that plan has been executed.
### 2026-06-02 — Slate AX retired; SODOLA switch added; network restructured ### 2026-06-02 — Slate AX retired; SODOLA switch added; network restructured
Replaced GL.iNet Slate AX sub-router with SODOLA 5-Port 2.5Gbit managed switch. Fritz!OS 8.x lacks second-IP-network support on LAN ports, so Proxmox now acts as the subnet router: `vmbr1` (eno1 → SODOLA → Fritz!Box) is the uplink at `192.168.178.10/24`; `vmbr0` is a portless internal bridge holding all LXCs/VMs with `192.168.8.1` as an alias (unchanged LXC gateway). Fritz!Box static route `192.168.8.0/24 → 192.168.178.10` enables inbound routing. No LXC configs changed. Eliminated double-NAT. WiFi (`wlp3s0`) also removed — was pointing at the Slate AX SSID, no longer useful. See [network](../infrastructure/network.md) and [migration plan](../plans/2026-06-01-slate-ax-to-sodola-migration.md). Replaced GL.iNet Slate AX sub-router with SODOLA 5-Port 2.5Gbit managed switch. Fritz!OS 8.x lacks second-IP-network support on LAN ports, so Proxmox now acts as the subnet router: `vmbr1` (eno1 → SODOLA → Fritz!Box) is the uplink at `192.168.178.10/24`; `vmbr0` is a portless internal bridge holding all LXCs/VMs with `192.168.8.1` as an alias (unchanged LXC gateway). Fritz!Box static route `192.168.8.0/24 → 192.168.178.10` enables inbound routing. No LXC configs changed. Eliminated double-NAT. WiFi (`wlp3s0`) also removed — was pointing at the Slate AX SSID, no longer useful. See [network](../infrastructure/network.md) and [migration plan](../../../plans/2026-06-01-slate-ax-to-sodola-migration.md).
### 2026-05-14 — LXC 109 (syncthing) decommissioned ### 2026-05-14 — LXC 109 (syncthing) decommissioned
User destroyed the syncthing LXC (had been stopped since 2026-04-21, never re-enabled). `pct destroy 109 --purge` cleaned `vm-109-disk-0` on `local-lvm` and the `/etc/pve/lxc/109.conf` entry. Data subtree `/mnt/library/syncthing` was already empty and retained as an empty dir. No DNS, Caddy, NFS-export, or claudio-monitor references to clean up. Entry moved to the "recently destroyed" table in [containers/index](../containers/index.md#recently-destroyed-kept-for-archaeology); references stripped from [README](../README.md), [media-permissions](../infrastructure/media-permissions.md), [vms/100-zimaos](../vms/100-zimaos.md), and [containers/102-nfs-export](../containers/102-nfs-export.md). User destroyed the syncthing LXC (had been stopped since 2026-04-21, never re-enabled). `pct destroy 109 --purge` cleaned `vm-109-disk-0` on `local-lvm` and the `/etc/pve/lxc/109.conf` entry. Data subtree `/mnt/library/syncthing` was already empty and retained as an empty dir. No DNS, Caddy, NFS-export, or claudio-monitor references to clean up. Entry moved to the "recently destroyed" table in [containers/index](../containers/index.md#recently-destroyed-kept-for-archaeology); references stripped from [README](../../../README.md), [media-permissions](../infrastructure/media-permissions.md), [vms/100-zimaos](../vms/100-zimaos.md), and [containers/102-nfs-export](../containers/102-nfs-export.md).
### 2026-05-14 — network performance baseline captured ### 2026-05-14 — network performance baseline captured
First explicit speed snapshot: WAN ↓113.5 / ↑19.9 Mbit (24.6 ms), `eno1` 1 Gb full-duplex negotiated, intra-host `vmbr0` ~34.7 Gbit/s host↔LXC and ~34.8 Gbit/s LXC↔LXC (single TCP stream, zero retransmits). `iperf3` + `speedtest-cli` installed on host. Noted `eno1` `rx_errors` at 1.62 M (~1.7 % of 96 M RX packets in 14 d uptime) plus 10.9 k `align_errors` — flagged for follow-up; expect to recheck the trend in ~1 week, suspect patch cable / switch port first if still climbing. See new "Network performance baseline" section above. First explicit speed snapshot: WAN ↓113.5 / ↑19.9 Mbit (24.6 ms), `eno1` 1 Gb full-duplex negotiated, intra-host `vmbr0` ~34.7 Gbit/s host↔LXC and ~34.8 Gbit/s LXC↔LXC (single TCP stream, zero retransmits). `iperf3` + `speedtest-cli` installed on host. Noted `eno1` `rx_errors` at 1.62 M (~1.7 % of 96 M RX packets in 14 d uptime) plus 10.9 k `align_errors` — flagged for follow-up; expect to recheck the trend in ~1 week, suspect patch cable / switch port first if still climbing. See new "Network performance baseline" section above.
@@ -138,7 +138,7 @@ User destroyed the heaper LXC. No `116.conf.bak` left behind in `/etc/pve/lxc/`.
`/etc/sysctl.d/99-bbr.conf` switches `net.ipv4.tcp_congestion_control` from `cubic` to `bbr` and `net.core.default_qdisc` from `fq_codel` to `fq`. Also bumps `rmem_max`/`wmem_max` to 64 MiB and widens `tcp_rmem`/`tcp_wmem`. `tcp_bbr` module pinned at boot via `/etc/modules-load.d/bbr.conf`. Triggered by Nextcloud client downloads from a WiFi laptop pulling ~2 MB/s despite a 152 Mbps link — server-side baseline through Caddy with BBR is ~400 MB/s single-stream loopback, so any client-perceived single-stream improvement is pure congestion-control win. Touches every LXC's outbound TCP since they all share this kernel. `/etc/sysctl.d/99-bbr.conf` switches `net.ipv4.tcp_congestion_control` from `cubic` to `bbr` and `net.core.default_qdisc` from `fq_codel` to `fq`. Also bumps `rmem_max`/`wmem_max` to 64 MiB and widens `tcp_rmem`/`tcp_wmem`. `tcp_bbr` module pinned at boot via `/etc/modules-load.d/bbr.conf`. Triggered by Nextcloud client downloads from a WiFi laptop pulling ~2 MB/s despite a 152 Mbps link — server-side baseline through Caddy with BBR is ~400 MB/s single-stream loopback, so any client-perceived single-stream improvement is pure congestion-control win. Touches every LXC's outbound TCP since they all share this kernel.
### 2026-04-29 — relocated to better-ventilated spot ### 2026-04-29 — relocated to better-ventilated spot
User physically moved the host to a new location with improved airflow. Post-move idle baseline (45 min uptime, light load): k10temp Tctl **47.2 °C**, amdgpu edge 42 °C, nvme0 composite 34.9 °C / sensor1 32.9 °C, nvme1 composite 38.9 °C / sensor1 52.9 °C, DRAM 3435.5 °C, ACPI zone 4749 °C. Compares well against the 2026-04-23 thermal-pad steady-state (nvme0 sensor1 6061 °C). Watch the lifetime NVMe warning-time counter over the coming days for confirmation. See [investigation](../investigations/2026-04-21-hubris-crash-loop.md#2026-04-29-physical-relocation). User physically moved the host to a new location with improved airflow. Post-move idle baseline (45 min uptime, light load): k10temp Tctl **47.2 °C**, amdgpu edge 42 °C, nvme0 composite 34.9 °C / sensor1 32.9 °C, nvme1 composite 38.9 °C / sensor1 52.9 °C, DRAM 3435.5 °C, ACPI zone 4749 °C. Compares well against the 2026-04-23 thermal-pad steady-state (nvme0 sensor1 6061 °C). Watch the lifetime NVMe warning-time counter over the coming days for confirmation. See [investigation](../../../investigations/2026-04-21-hubris-crash-loop.md#2026-04-29-physical-relocation).
### 2026-04-28 — Phase 1 WiFi failover ### 2026-04-28 — Phase 1 WiFi failover
Host now dual-homed: LAN `192.168.8.77` (primary) + WiFi `192.168.8.141` (failover, metric 200) on the GL-AXT1800-714-5G AP. Installed `wpasupplicant`+`iw`; added `wlp3s0` stanza to `/etc/network/interfaces` with `wpa-conf`; ARP isolation sysctls in `post-up`. Built `wan-failover.service` to remove the vmbr0 default route on `eno1` carrier loss, since the bridge's carrier doesn't follow `eno1` (the LXC veths keep it `1`). LXC/VM guests are still LAN-only — Phase 2 will migrate them. Host now dual-homed: LAN `192.168.8.77` (primary) + WiFi `192.168.8.141` (failover, metric 200) on the GL-AXT1800-714-5G AP. Installed `wpasupplicant`+`iw`; added `wlp3s0` stanza to `/etc/network/interfaces` with `wpa-conf`; ARP isolation sysctls in `post-up`. Built `wan-failover.service` to remove the vmbr0 default route on `eno1` carrier loss, since the bridge's carrier doesn't follow `eno1` (the LXC veths keep it `1`). LXC/VM guests are still LAN-only — Phase 2 will migrate them.
@@ -147,10 +147,10 @@ Host now dual-homed: LAN `192.168.8.77` (primary) + WiFi `192.168.8.141` (failov
This wiki created. Live state at this date: 14 LXCs running (109 syncthing stopped), 1 VM, kernel `6.14.11-4-pve`, uptime 3 d 0 h post drive-removal A/B test. Compared to memory snapshot from a week ago, **destroyed**: LXC 100 (yunohost arr), 106 (flaresolverr), 107 (marimo), 110 (photoprism), 111 (karakeep), 112 (immich), 115 (reticulum). 100 + 106 destroyed per the planned 2026-04-21 \*arr migration retention; the others removed since. This wiki created. Live state at this date: 14 LXCs running (109 syncthing stopped), 1 VM, kernel `6.14.11-4-pve`, uptime 3 d 0 h post drive-removal A/B test. Compared to memory snapshot from a week ago, **destroyed**: LXC 100 (yunohost arr), 106 (flaresolverr), 107 (marimo), 110 (photoprism), 111 (karakeep), 112 (immich), 115 (reticulum). 100 + 106 destroyed per the planned 2026-04-21 \*arr migration retention; the others removed since.
### 2026-04-23 — SSD cooling + thermal pads installed ### 2026-04-23 — SSD cooling + thermal pads installed
Thermal pads on both NVMe drives. Steady-state nvme0 composite 47 °C / sensor1 6061 °C, nvme1 3840 °C. Zero new warning-time minutes after install. Watch the lifetime warning-time counter going forward, not absolute sensor1. See [investigation](../investigations/2026-04-21-hubris-crash-loop.md#2026-04-23-thermal-pad-verdict). Thermal pads on both NVMe drives. Steady-state nvme0 composite 47 °C / sensor1 6061 °C, nvme1 3840 °C. Zero new warning-time minutes after install. Watch the lifetime warning-time counter going forward, not absolute sensor1. See [investigation](../../../investigations/2026-04-21-hubris-crash-loop.md#2026-04-23-thermal-pad-verdict).
### 2026-04-22 — drive removal A/B test ### 2026-04-22 — drive removal A/B test
Removed external USB backup drive (Silicon Motion `090c:2320`). Disabled the four `backup-library*.timer` units, commented the fstab entry. Goal: confirm whether the drive + UAS interaction on the AMD USB4 PCIe tunnel is the dominant root cause of the silent hard-locks. Pre-drive uptime was 33 days; with drive, repeated crashes despite UAS blacklist + mount-on-demand. **Result so far:** 3+ days uptime — the drive looks like the primary contributor; `cpu-epp` remains as belt-and-suspenders thermal protection. See [investigation](../investigations/2026-04-21-hubris-crash-loop.md). Removed external USB backup drive (Silicon Motion `090c:2320`). Disabled the four `backup-library*.timer` units, commented the fstab entry. Goal: confirm whether the drive + UAS interaction on the AMD USB4 PCIe tunnel is the dominant root cause of the silent hard-locks. Pre-drive uptime was 33 days; with drive, repeated crashes despite UAS blacklist + mount-on-demand. **Result so far:** 3+ days uptime — the drive looks like the primary contributor; `cpu-epp` remains as belt-and-suspenders thermal protection. See [investigation](../../../investigations/2026-04-21-hubris-crash-loop.md).
### 2026-04-22 — `cpu-epp.service` ordering bug fixed ### 2026-04-22 — `cpu-epp.service` ordering bug fixed
Was `After=multi-user.target` + `WantedBy=multi-user.target` — queued behind `pve-guests.service`, so the hottest boot window (20+ guests starting on `performance`) preceded EPP application. Now `After=sysinit.target` + `Before=pve-guests.service`. Was `After=multi-user.target` + `WantedBy=multi-user.target` — queued behind `pve-guests.service`, so the hottest boot window (20+ guests starting on `performance`) preceded EPP application. Now `After=sysinit.target` + `Before=pve-guests.service`.
@@ -159,4 +159,4 @@ Was `After=multi-user.target` + `WantedBy=multi-user.target` — queued behind `
`60-crash-capture.conf`, softdog `soft_panic=1`, RuntimeWatchdog 15 s. `rasdaemon` installed and enabled. Pure silicon hangs still leave no trace; this catches everything else. `60-crash-capture.conf`, softdog `soft_panic=1`, RuntimeWatchdog 15 s. `rasdaemon` installed and enabled. Pure silicon hangs still leave no trace; this catches everything else.
### 2026-04-21 — `cpu-epp.service` deployed ### 2026-04-21 — `cpu-epp.service` deployed
Pinned governor=`powersave`, EPP=`balance_power` at boot. Stopped the host idling at ~95 °C with everything pinned at 4.4 GHz. First fix in the [crash-loop incident](../investigations/2026-04-21-hubris-crash-loop.md). Pinned governor=`powersave`, EPP=`balance_power` at boot. Stopped the host idling at ~95 °C with everything pinned at 4.4 GHz. First fix in the [crash-loop incident](../../../investigations/2026-04-21-hubris-crash-loop.md).

View File

@@ -28,7 +28,7 @@ Proxmox VE on 2026-07-01. No LXCs/VMs deployed on it yet.
was actually created live; not worth renaming), restricted `nodes strong` was actually created live; not worth renaming), restricted `nodes strong`
in `/etc/pve/storage.cfg` — same pattern as hubris's `library` pool. in `/etc/pve/storage.cfg` — same pattern as hubris's `library` pool.
Empty so far; this is separate from the [library-SSD migration Empty so far; this is separate from the [library-SSD migration
plan](../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)'s plan](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)'s
planned drive move from hubris (that hasn't happened) — this is planned drive move from hubris (that hasn't happened) — this is
general-purpose VM/CT capacity. general-purpose VM/CT capacity.
- **Network:** `vmbr0` is bridged straight onto the household LAN — - **Network:** `vmbr0` is bridged straight onto the household LAN —
@@ -50,13 +50,13 @@ Proxmox VE on 2026-07-01. No LXCs/VMs deployed on it yet.
`bootstrap.sh --no-secrets` (reused the operator's existing Gitea PAT for `bootstrap.sh --no-secrets` (reused the operator's existing Gitea PAT for
the initial clone). `/opt/homelab-context`, the `homelab` CLI, and the the initial clone). `/opt/homelab-context`, the `homelab` CLI, and the
5-min sync timer are live; `homelab whoami` resolves correctly. See 5-min sync timer are live; `homelab whoami` resolves correctly. See
[agent-enrollment.md](../operations/agent-enrollment.md). [agent-enrollment.md](../../../operations/agent-enrollment.md).
- **Age key / secrets:** issued the same day over plain LAN (no Netbird - **Age key / secrets:** issued the same day over plain LAN (no Netbird
needed — see the `--no-mesh` bootstrap.sh fix below). Key lives at needed — see the `--no-mesh` bootstrap.sh fix below). Key lives at
`/etc/age/key.txt`; pubkey `age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4` `/etc/age/key.txt`; pubkey `age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4`
recorded in `inventory.yaml`. Not yet a recipient on any actual secret recorded in `inventory.yaml`. Not yet a recipient on any actual secret
(`hello.yaml`, `gitea-pat.yaml`, etc.) — that's a separate grant, see (`hello.yaml`, `gitea-pat.yaml`, etc.) — that's a separate grant, see
["Granting a secret to a new client"](../operations/agent-enrollment.md#granting-a-secret-to-a-new-client). ["Granting a secret to a new client"](../../../operations/agent-enrollment.md#granting-a-secret-to-a-new-client).
## Cluster membership ## Cluster membership
@@ -76,10 +76,10 @@ needed going forward.
## Related ## Related
- [hubris — Proxmox host](hubris.md) - [hubris — Proxmox host](hubris.md)
- [Library SSD migration plan](../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md) — the larger project this is Phase 1 of (filename kept as-is, historical) - [Library SSD migration plan](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md) — the larger project this is Phase 1 of (filename kept as-is, historical)
- [Network](../infrastructure/network.md) - [Network](../infrastructure/network.md)
- [SSH access](../infrastructure/ssh-access.md) - [SSH access](../infrastructure/ssh-access.md)
- [Agent enrollment](../operations/agent-enrollment.md) - [Agent enrollment](../../../operations/agent-enrollment.md)
## Changelog ## Changelog

View File

@@ -44,13 +44,13 @@ The app repo at `/opt/<thing>` is the working tree, but the deploy tooling (`web
| `dtoro/Homelab-Docs` → homelab-mcp | [apps (105)](../containers/105-apps.md) `/opt/homelab-mcp/` | B | `http://192.168.8.205:9811/deploy` | 10 | reinstalls `homelab-mcp.service` + restart | | `dtoro/Homelab-Docs` → homelab-mcp | [apps (105)](../containers/105-apps.md) `/opt/homelab-mcp/` | B | `http://192.168.8.205:9811/deploy` | 10 | reinstalls `homelab-mcp.service` + restart |
| `dtoro/Homelab-Docs` → secrets-issuance | [apps (105)](../containers/105-apps.md) `/opt/secrets-issuance/` | B | `http://192.168.8.205:9821/deploy` | 11 | reinstalls `secrets-issuance.service` + restart | | `dtoro/Homelab-Docs` → secrets-issuance | [apps (105)](../containers/105-apps.md) `/opt/secrets-issuance/` | B | `http://192.168.8.205:9821/deploy` | 11 | reinstalls `secrets-issuance.service` + restart |
| `dtoro/terminalito` | [trmnl (128)](../containers/128-trmnl.md) `/opt/terminalito/` | B | `http://192.168.8.211:9797/deploy` | 12 | reinstalls units + `systemctl restart trmnl-plugins` | | `dtoro/terminalito` | [trmnl (128)](../containers/128-trmnl.md) `/opt/terminalito/` | B | `http://192.168.8.211:9797/deploy` | 12 | reinstalls units + `systemctl restart trmnl-plugins` |
| `dtoro/Homelab-Docs` → oikos-console | [apps (105)](../containers/105-apps.md) `/opt/oikos-console/` | B | `http://192.168.8.205:9831/deploy` | 14 | reinstalls `oikos-console.service` + restart — see [oikos/console/deploy/README.md](../oikos/console/deploy/README.md) | | `dtoro/Homelab-Docs` → oikos-console | [apps (105)](../containers/105-apps.md) `/opt/oikos-console/` | B | `http://192.168.8.205:9831/deploy` | 14 | reinstalls `oikos-console.service` + restart — see [oikos/console/deploy/README.md](../../../oikos/console/deploy/README.md) |
> Note: `dtoro/Homelab-Docs` has **three webhooks** firing on the same push. > Note: `dtoro/Homelab-Docs` has **three webhooks** firing on the same push.
> Each owns its own clone on LXC 105. They don't conflict because each > Each owns its own clone on LXC 105. They don't conflict because each
> deploy.sh only touches its own service unit + venv. > deploy.sh only touches its own service unit + venv.
> **Not yet wired:** `dtoro/claudio-monitor` (push, then `/opt/claudio-monitor/scripts/deploy.sh` manually). The former authentik LXC (124) is destroyed — Authentik runs on the [VPS](../hosts/netbird-vps.md). DNS moved to [Technitium on dns (107)](../containers/107-dns.md). > **Not yet wired:** `dtoro/claudio-monitor` (push, then `/opt/claudio-monitor/scripts/deploy.sh` manually). The former authentik LXC (124) is destroyed — Authentik runs on the [VPS](../../../hosts/netbird-vps.md). DNS moved to [Technitium on dns (107)](../containers/107-dns.md).
## When you change a tracked config ## When you change a tracked config
@@ -117,7 +117,7 @@ If you're not sure what's already lurking, run `homelab apt-audit --fleet` and l
- [Gitea (104)](../containers/104-gitea.md) — webhook source for all of these - [Gitea (104)](../containers/104-gitea.md) — webhook source for all of these
- [Caddy (121)](../containers/121-caddy.md), [apps (105)](../containers/105-apps.md), [mule-images (120)](../containers/120-mule-images.md), [hubris host](../hosts/hubris.md) — webhook targets - [Caddy (121)](../containers/121-caddy.md), [apps (105)](../containers/105-apps.md), [mule-images (120)](../containers/120-mule-images.md), [hubris host](../hosts/hubris.md) — webhook targets
- [Backups (disabled)](backups.md) - [Backups (disabled)](backups.md)
- [Operations cheatsheet](../operations/commands.md) — `homelab apt-audit` / `homelab apt-upgrade` reference - [Operations cheatsheet](../../../operations/commands.md) — `homelab apt-audit` / `homelab apt-upgrade` reference
## Changelog ## Changelog

View File

@@ -24,7 +24,7 @@ See [132-rclone](../containers/132-rclone.md) for the full design.
## Legacy — restic on external drive (DISABLED 2026-04-22) ## Legacy — restic on external drive (DISABLED 2026-04-22)
Chunked monthly restic backup of `/mnt/library`'s irreplaceable subset. **Disabled 2026-04-22** as part of the [hubris crash-loop A/B test](../investigations/2026-04-21-hubris-crash-loop.md). Chunked monthly restic backup of `/mnt/library`'s irreplaceable subset. **Disabled 2026-04-22** as part of the [hubris crash-loop A/B test](../../../investigations/2026-04-21-hubris-crash-loop.md).
## Status ## Status
@@ -36,7 +36,7 @@ Chunked monthly restic backup of `/mnt/library`'s irreplaceable subset. **Disabl
Fstab entry commented out. USB drive de-authorized and physically removed. `backup-library-deploy.service` left enabled (harmless webhook receiver). Fstab entry commented out. USB drive de-authorized and physically removed. `backup-library-deploy.service` left enabled (harmless webhook receiver).
**Reason:** the host hang recurred 2026-04-22 18:42 after 30h despite the `cpu-epp` fix, the UAS blacklist, and mount-on-demand. User wants to confirm host stability without the drive at all (was stable 33 days before the drive arrived). See [investigation](../investigations/2026-04-21-hubris-crash-loop.md). **Reason:** the host hang recurred 2026-04-22 18:42 after 30h despite the `cpu-epp` fix, the UAS blacklist, and mount-on-demand. User wants to confirm host stability without the drive at all (was stable 33 days before the drive arrived). See [investigation](../../../investigations/2026-04-21-hubris-crash-loop.md).
**To re-enable:** uncomment fstab line, `systemctl enable --now` the four timers, re-attach drive. **To re-enable:** uncomment fstab line, `systemctl enable --now` the four timers, re-attach drive.
@@ -111,7 +111,7 @@ Single drive. RECOVERY.md flags the 3-2-1 gap. Mitigations (second drive, cloud
The `Silicon Motion Portable SSD` (vid:pid `090c:2320`) drops under sustained heavy writes through a hub chain. Bypass all hubs / use a rear motherboard USB 3 port if attaching it again. The `Silicon Motion Portable SSD` (vid:pid `090c:2320`) drops under sustained heavy writes through a hub chain. Bypass all hubs / use a rear motherboard USB 3 port if attaching it again.
After it was first attached on 2026-04-19, hubris crashed twice in 2.5 days (46h then 12h uptime). Kernel logs ended abruptly with routine apparmor entries — no panic, OOM, or MCE — the classic hard-lock signature. Preceded by `uas_eh_abort_handler` storms and xHCI resets on port 6-1. The UAS blacklist + mount-on-demand mitigations didn't fully eliminate it (recurrence 2026-04-22), prompting drive removal as the cleaner test. See [investigation](../investigations/2026-04-21-hubris-crash-loop.md). After it was first attached on 2026-04-19, hubris crashed twice in 2.5 days (46h then 12h uptime). Kernel logs ended abruptly with routine apparmor entries — no panic, OOM, or MCE — the classic hard-lock signature. Preceded by `uas_eh_abort_handler` storms and xHCI resets on port 6-1. The UAS blacklist + mount-on-demand mitigations didn't fully eliminate it (recurrence 2026-04-22), prompting drive removal as the cleaner test. See [investigation](../../../investigations/2026-04-21-hubris-crash-loop.md).
## Thermal monitoring ## Thermal monitoring
@@ -122,7 +122,7 @@ Moved out of this repo to `dtoro/claudio-monitor` on 2026-04-21 (commit `50dc213
- ~~[claudio-bot (123)](../containers/123-claudio-bot.md)~~ (destroyed 2026-06-04) - ~~[claudio-bot (123)](../containers/123-claudio-bot.md)~~ (destroyed 2026-06-04)
- [Monitoring](monitoring.md) - [Monitoring](monitoring.md)
- [Auto-deploy](auto-deploy.md) - [Auto-deploy](auto-deploy.md)
- [Investigation: 2026-04-21 crash loop](../investigations/2026-04-21-hubris-crash-loop.md) - [Investigation: 2026-04-21 crash loop](../../../investigations/2026-04-21-hubris-crash-loop.md)
## Changelog ## Changelog
@@ -133,7 +133,7 @@ Off-host backup moved to a plain `rclone sync` mirror on the new [LXC 132 `rclon
Initial documentation. Status remains DISABLED. Initial documentation. Status remains DISABLED.
### 2026-04-22 — DISABLED ### 2026-04-22 — DISABLED
Drive removed as the A/B test in the [crash investigation](../investigations/2026-04-21-hubris-crash-loop.md). Timers disabled, fstab commented, drive de-authorized. Drive removed as the A/B test in the [crash investigation](../../../investigations/2026-04-21-hubris-crash-loop.md). Timers disabled, fstab commented, drive de-authorized.
### 2026-04-21 — UAS blacklist + mount-on-demand shipped; root-caused host hangs to drive ### 2026-04-21 — UAS blacklist + mount-on-demand shipped; root-caused host hangs to drive
Drive identified as the source of the hangs after hubris crashed twice in 2.5 days. UAS blacklist forces BOT; helper script toggles `/sys/bus/usb/.../authorized` so the drive is de-authorized when not backing up. Recovery drill (restore 188KB PDF + hash compare) had passed earlier. Bug fixed in `backup-library.sh`: `python3 -c '…' KEY=VAL` does NOT pass env vars — env-var prefix must precede the command. Caused false-failure even after successful backups. Drive identified as the source of the hangs after hubris crashed twice in 2.5 days. UAS blacklist forces BOT; helper script toggles `/sys/bus/usb/.../authorized` so the drive is de-authorized when not backing up. Recovery drill (restore 188KB PDF + hash compare) had passed earlier. Bug fixed in `backup-library.sh`: `python3 -c '…' KEY=VAL` does NOT pass env vars — env-var prefix must precede the command. Caused false-failure even after successful backups.

View File

@@ -7,7 +7,7 @@ There is **no wildcard on the LAN side**. Every subdomain needs an explicit entr
## Components ## Components
- **Authoritative public DNS:** IONOS. `*.hubris.network → 82.165.190.79` (was `74.118.126.4` until 2026-04-22). - **Authoritative public DNS:** IONOS. `*.hubris.network → 82.165.190.79` (was `74.118.126.4` until 2026-04-22).
- **LAN authoritative for `hubris.network` records:** [Technitium DNS](https://technitium.com) on [dns (107)](../containers/107-dns.md) at `192.168.8.2:53`. Syncs A records to the NetBird managed DNS zone via cron (see [dns-sync.py](../scripts/dns-sync.py)). Formerly dnsmasq on [authentik (124)](../containers/124-authentik.md) (decommissioned 2026-06-04). - **LAN authoritative for `hubris.network` records:** [Technitium DNS](https://technitium.com) on [dns (107)](../containers/107-dns.md) at `192.168.8.2:53`. Syncs A records to the NetBird managed DNS zone via cron (see [dns-sync.py](../../../scripts/dns-sync.py)). Formerly dnsmasq on [authentik (124)](../containers/124-authentik.md) (decommissioned 2026-06-04).
- **PVE host** (`192.168.8.77`): resolver is the local Netbird daemon at `100.122.38.109:53`, which forwards to the LAN/upstream and learns hubris.network answers via that path. `netbird status` says "Nameservers: 0/0 Available" — confirming netbird does NOT manage a hubris.network zone; it just caches whatever the system resolver returns. - **PVE host** (`192.168.8.77`): resolver is the local Netbird daemon at `100.122.38.109:53`, which forwards to the LAN/upstream and learns hubris.network answers via that path. `netbird status` says "Nameservers: 0/0 Available" — confirming netbird does NOT manage a hubris.network zone; it just caches whatever the system resolver returns.
- **Some LXCs** keep router DNS (`192.168.8.1`) or Tailscale MagicDNS (`100.100.100.100`), both of which return the public IONOS A record. Those LXCs need either a `/etc/hosts` override or local dnsmasq — see [mesh migration](mesh.md) for which technique applies where. - **Some LXCs** keep router DNS (`192.168.8.1`) or Tailscale MagicDNS (`100.100.100.100`), both of which return the public IONOS A record. Those LXCs need either a `/etc/hosts` override or local dnsmasq — see [mesh migration](mesh.md) for which technique applies where.
@@ -53,7 +53,7 @@ Creating a new Caddyfile site block is necessary but **not sufficient**. Without
3. Verify: `dig @192.168.8.2 +short <new>.hubris.network``192.168.8.175`. 3. Verify: `dig @192.168.8.2 +short <new>.hubris.network``192.168.8.175`.
4. On macOS clients, flush: `sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder`. 4. On macOS clients, flush: `sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder`.
> The Technitium config on LXC 107 is the single source of truth. Never hand-edit the NetBird managed zone directly — the [`scripts/dns-sync.py`](../scripts/dns-sync.py) cron on 107 reconciles them and reaps stale records. See [dns.md changelog 2026-06-03](#2026-06-03--single-authoring-source-technitium--netbird-managed-zone-sync). > The Technitium config on LXC 107 is the single source of truth. Never hand-edit the NetBird managed zone directly — the [`scripts/dns-sync.py`](../../../scripts/dns-sync.py) cron on 107 reconciles them and reaps stale records. See [dns.md changelog 2026-06-03](#2026-06-03--single-authoring-source-technitium--netbird-managed-zone-sync).
## Public path — what does and doesn't follow the LAN map ## Public path — what does and doesn't follow the LAN map
@@ -94,7 +94,7 @@ The "delete NetBird managed zone → forward everything to Technitium" plan was
**Cleanup done same day:** removed the inert Mac-Mini Technitium secondary (mesh-only, served nobody); reverted the primary's `zoneTransfer=Allow`; fixed `home-lab-dns` group → `[192.168.8.2]` (dropped the self-referencing Mac IP → now `1/1 Available`); deleted the vestigial `Proxmox Names` group. **Cleanup done same day:** removed the inert Mac-Mini Technitium secondary (mesh-only, served nobody); reverted the primary's `zoneTransfer=Allow`; fixed `home-lab-dns` group → `[192.168.8.2]` (dropped the self-referencing Mac IP → now `1/1 Available`); deleted the vestigial `Proxmox Names` group.
> Reference: [scripts/dns-sync.py](../scripts/dns-sync.py). The sync's source of truth is Technitium; it **deletes** NetBird records absent from Technitium (so obsolete names like `files`, `photos-new` get reaped). > Reference: [scripts/dns-sync.py](../../../scripts/dns-sync.py). The sync's source of truth is Technitium; it **deletes** NetBird records absent from Technitium (so obsolete names like `files`, `photos-new` get reaped).
### 2026-06-06 — dns-sync cron finally installed (had been dormant since 2026-06-04 deployment) ### 2026-06-06 — dns-sync cron finally installed (had been dormant since 2026-06-04 deployment)
The `dns-sync.py` script on LXC 107 had been placed at `/opt/dns-sync/sync.py` on 2026-06-04 but **no crontab was configured** — the sync had never run automatically. The NetBird managed DNS zone was only in sync because manual runs happened during incident debugging. The `dns-sync.py` script on LXC 107 had been placed at `/opt/dns-sync/sync.py` on 2026-06-04 but **no crontab was configured** — the sync had never run automatically. The NetBird managed DNS zone was only in sync because manual runs happened during incident debugging.
@@ -110,7 +110,7 @@ All LXCs that Caddy reverse-proxies to by IP were on `ip=dhcp` and could float o
Split-horizon DNS moved off [124](../containers/124-authentik.md) to a dedicated **Technitium** LXC at **`192.168.8.2`** (zone: specific A overrides + wildcard→VPS + replicated MX/SPF/CAA). NetBird `home-lab-dns` nameserver group cut over to `192.168.8.2` (with `.180` as a now-dead fallback). dnsmasq stopped, all names verified via Technitium, **LXC 124 shut down**. **Caveat:** the [NetBird managed DNS zone](../containers/124-authentik.md) still answers most app names *directly* (bypassing the nameserver group) — three overlapping DNS sources remain; see the single-source-of-truth decision (Phase 4). **Action needed:** update router DHCP DNS from the dead `.180``192.168.8.2` for any plain-LAN (non-mesh) clients. Split-horizon DNS moved off [124](../containers/124-authentik.md) to a dedicated **Technitium** LXC at **`192.168.8.2`** (zone: specific A overrides + wildcard→VPS + replicated MX/SPF/CAA). NetBird `home-lab-dns` nameserver group cut over to `192.168.8.2` (with `.180` as a now-dead fallback). dnsmasq stopped, all names verified via Technitium, **LXC 124 shut down**. **Caveat:** the [NetBird managed DNS zone](../containers/124-authentik.md) still answers most app names *directly* (bypassing the nameserver group) — three overlapping DNS sources remain; see the single-source-of-truth decision (Phase 4). **Action needed:** update router DHCP DNS from the dead `.180``192.168.8.2` for any plain-LAN (non-mesh) clients.
### 2026-05-31 — `auth.hubris.network` re-pointed to the VPS (`82.165.190.79`) ### 2026-05-31 — `auth.hubris.network` re-pointed to the VPS (`82.165.190.79`)
Authentik migrated off LXC 124 onto the VPS (see [investigation](../investigations/2026-05-31-authentik-vps-migration.md)). The dnsmasq entry changed from `192.168.8.175` (home Caddy) to `82.165.190.79` (VPS traefik). This is the first LAN entry that intentionally points at the VPS rather than Caddy — `auth` is now a genuinely public service served directly from the VPS. **Gotcha logged:** the NetBird per-client resolver (`100.122.255.254`) caches dnsmasq answers and does **not** clear on `netbird down/up`; clients needed `/etc/hosts` overrides or `resolvectl flush-caches` to pick up the change. Since the service is now fully public, the long-term cleaner option is to drop the override entirely and let it fall through to the IONOS wildcard (which also points at the VPS). Authentik migrated off LXC 124 onto the VPS (see [investigation](../../../investigations/2026-05-31-authentik-vps-migration.md)). The dnsmasq entry changed from `192.168.8.175` (home Caddy) to `82.165.190.79` (VPS traefik). This is the first LAN entry that intentionally points at the VPS rather than Caddy — `auth` is now a genuinely public service served directly from the VPS. **Gotcha logged:** the NetBird per-client resolver (`100.122.255.254`) caches dnsmasq answers and does **not** clear on `netbird down/up`; clients needed `/etc/hosts` overrides or `resolvectl flush-caches` to pick up the change. Since the service is now fully public, the long-term cleaner option is to drop the override entirely and let it fall through to the IONOS wildcard (which also points at the VPS).
### 2026-05-14 — `nfs-export.hubris.network` added (direct, non-HTTP) ### 2026-05-14 — `nfs-export.hubris.network` added (direct, non-HTTP)
NFSv4 export server [nfs-export (102)](../containers/102-nfs-export.md) at `192.168.8.200`. Direct entry, not Caddy-fronted — NFS is L4, no HTTP reverse-proxy meaningful. NFSv4 export server [nfs-export (102)](../containers/102-nfs-export.md) at `192.168.8.200`. Direct entry, not Caddy-fronted — NFS is L4, no HTTP reverse-proxy meaningful.

View File

@@ -5,7 +5,7 @@ Code, Hermes Agent, future MCP-capable clients) on every machine in the lab
self-locating and able to read the same source of truth. self-locating and able to read the same source of truth.
Operational walkthrough for enrolling a new client lives in Operational walkthrough for enrolling a new client lives in
[operations/agent-enrollment.md](../operations/agent-enrollment.md); this [operations/agent-enrollment.md](../../../operations/agent-enrollment.md); this
page is the architecture reference. page is the architecture reference.
## What's where ## What's where
@@ -121,7 +121,7 @@ The MCP server and secrets-issuance each have their own clone
## Related ## Related
- [Operations: agent enrollment](../operations/agent-enrollment.md) — the - [Operations: agent enrollment](../../../operations/agent-enrollment.md) — the
step-by-step for adding a new client step-by-step for adding a new client
- [Auto-deploy](auto-deploy.md) — the `homelab-mcp` + `secrets-issuance` - [Auto-deploy](auto-deploy.md) — the `homelab-mcp` + `secrets-issuance`
pipelines (and the rest of the lab's webhook pipelines) pipelines (and the rest of the lab's webhook pipelines)
@@ -133,7 +133,7 @@ The MCP server and secrets-issuance each have their own clone
## Changelog ## Changelog
### 2026-05-20 — system live across hubris, apps, republic-laptop ### 2026-05-20 — system live across hubris, apps, republic-laptop
Phase 1 of the [cross-client context plan](../README.md) merged. Three Phase 1 of the [cross-client context plan](../../../README.md) merged. Three
clients enrolled end-to-end: PAT-based bootstrap, age-key issuance, SOPS clients enrolled end-to-end: PAT-based bootstrap, age-key issuance, SOPS
decrypt verified on each. Webhook auto-deploy for both LXC 105 services decrypt verified on each. Webhook auto-deploy for both LXC 105 services
wired (hook ids 10 + 11). `homelab refresh-creds` + atomic wired (hook ids 10 + 11). `homelab refresh-creds` + atomic

View File

@@ -60,7 +60,7 @@ are documented in their own pages. Each system below links to its full doc.
## Related ## Related
- [README](../README.md) — entry point - [README](../../../README.md) — entry point
- [Containers index](../containers/index.md) - [Containers index](../containers/index.md)
- [Operations cheatsheet](../operations/commands.md) - [Operations cheatsheet](../../../operations/commands.md)
- [OIKOS operating model](../OIKOS.md) - [OIKOS operating model](../../../OIKOS.md)

View File

@@ -49,7 +49,7 @@ LAN clients resolve via the [Technitium DNS on dns (107)](dns.md) → `192.168.8
### `auth.hubris.network` — different pattern (local container, not cert-mirror) ### `auth.hubris.network` — different pattern (local container, not cert-mirror)
Since 2026-05-31 [Authentik runs on the VPS itself](../investigations/2026-05-31-authentik-vps-migration.md), so `auth.hubris.network` is served by a **local Docker container**, not proxied to a home backend. It therefore does **not** use the file-provider + cert-mirror pattern above: Since 2026-05-31 [Authentik runs on the VPS itself](../../../investigations/2026-05-31-authentik-vps-migration.md), so `auth.hubris.network` is served by a **local Docker container**, not proxied to a home backend. It therefore does **not** use the file-provider + cert-mirror pattern above:
- Routed via traefik **Docker provider labels** on the `authentik-server` service (`/opt/docker-compose.yml`), not `traefik-dynamic.yaml`. - Routed via traefik **Docker provider labels** on the `authentik-server` service (`/opt/docker-compose.yml`), not `traefik-dynamic.yaml`.
- TLS via traefik's own `letsencrypt` resolver (works here because it's a normal HTTP router, not the HostSNI passthrough). - TLS via traefik's own `letsencrypt` resolver (works here because it's a normal HTTP router, not the HostSNI passthrough).
@@ -91,7 +91,7 @@ No cert-mirror entry and no `hubris-public-cert-sync.sh` mapping is needed for `
TRMNL plugins middleware on [trmnl (128)](../containers/128-trmnl.md). File-provider router `trmnl-public``192.168.8.211:9851`, `trmnl-ratelimit` (20 rps / 40 burst), cert mirrored as `trmnl.fullchain.crt`/`trmnl.privkey.key`. Verified live from the internet (200 with token / 401 without). It was provisioned during a mesh outage — the `home-lab-network` (192.168.8.0/24) route had no active routing peer because the **mac-mini routing peer's netbird was down** (all home-backed public services 504'd). Bringing netbird up on mac-mini restored the route; no traefik change was needed. TRMNL plugins middleware on [trmnl (128)](../containers/128-trmnl.md). File-provider router `trmnl-public``192.168.8.211:9851`, `trmnl-ratelimit` (20 rps / 40 burst), cert mirrored as `trmnl.fullchain.crt`/`trmnl.privkey.key`. Verified live from the internet (200 with token / 401 without). It was provisioned during a mesh outage — the `home-lab-network` (192.168.8.0/24) route had no active routing peer because the **mac-mini routing peer's netbird was down** (all home-backed public services 504'd). Bringing netbird up on mac-mini restored the route; no traefik change was needed.
### 2026-05-31 — `auth.hubris.network` now served locally on the VPS ### 2026-05-31 — `auth.hubris.network` now served locally on the VPS
Authentik migrated onto the VPS ([investigation](../investigations/2026-05-31-authentik-vps-migration.md)). Unlike the home-backed services above, `auth` is a local container routed via traefik Docker-provider labels with traefik-managed Let's Encrypt — no cert-mirror, no `traefik-dynamic.yaml` router. Admin UI gated by an ipAllowList middleware. Traefik gained a second Docker network (`auth`, `172.30.1.0/24`) to reach it while keeping its DB/Redis isolated from the netbird stack. Authentik migrated onto the VPS ([investigation](../../../investigations/2026-05-31-authentik-vps-migration.md)). Unlike the home-backed services above, `auth` is a local container routed via traefik Docker-provider labels with traefik-managed Let's Encrypt — no cert-mirror, no `traefik-dynamic.yaml` router. Admin UI gated by an ipAllowList middleware. Traefik gained a second Docker network (`auth`, `172.30.1.0/24`) to reach it while keeping its DB/Redis isolated from the netbird stack.
### 2026-04-28 — wiki entry created ### 2026-04-28 — wiki entry created
Initial documentation. Initial documentation.

View File

@@ -117,7 +117,7 @@ Recipe for container-config changes (e.g. adding `extra_hosts`) on Portainer-man
## Changelog ## Changelog
### 2026-05-31 (later) — Authentik moved to the VPS; mesh-dependency for auth eliminated (supersedes the band-aid below) ### 2026-05-31 (later) — Authentik moved to the VPS; mesh-dependency for auth eliminated (supersedes the band-aid below)
The earlier same-day fix routed `auth.hubris.network` through VPS Traefik → Caddy → LXC 124 **over the mesh**. That restored service but re-created the original fragility: if the mesh is dark when management restarts, the `192.168.8.175` backend is unreachable and management crash-loops again (the "Bootstrap note" in the entry below). That note is now **obsolete** — Authentik was migrated onto the VPS itself, so OIDC no longer touches the mesh. The `auth-authentik``192.168.8.175` route and its `skip-verify` transport were removed from `/opt/traefik-dynamic.yaml`; `auth.hubris.network` is now served by a local `authentik-server` container via Traefik Docker-provider labels, and netbird-mgmt has `depends_on: authentik-server: condition: service_healthy`. The socat / reverse-SSH bootstrap dance is no longer needed. Full detail: [2026-05-31 Authentik VPS migration](../investigations/2026-05-31-authentik-vps-migration.md). The earlier same-day fix routed `auth.hubris.network` through VPS Traefik → Caddy → LXC 124 **over the mesh**. That restored service but re-created the original fragility: if the mesh is dark when management restarts, the `192.168.8.175` backend is unreachable and management crash-loops again (the "Bootstrap note" in the entry below). That note is now **obsolete** — Authentik was migrated onto the VPS itself, so OIDC no longer touches the mesh. The `auth-authentik``192.168.8.175` route and its `skip-verify` transport were removed from `/opt/traefik-dynamic.yaml`; `auth.hubris.network` is now served by a local `authentik-server` container via Traefik Docker-provider labels, and netbird-mgmt has `depends_on: authentik-server: condition: service_healthy`. The socat / reverse-SSH bootstrap dance is no longer needed. Full detail: [2026-05-31 Authentik VPS migration](../../../investigations/2026-05-31-authentik-vps-migration.md).
### 2026-05-31 — Netbird mesh recovered; auth.hubris.network exposed via VPS Traefik ### 2026-05-31 — Netbird mesh recovered; auth.hubris.network exposed via VPS Traefik

View File

@@ -64,7 +64,7 @@ No NAT on Proxmox — traffic flows without double-NAT.
## Remote access ## Remote access
- **NetBird mesh** — primary path for remote administration. Authenticated via [Authentik on the VPS](../vps/). - **NetBird mesh** — primary path for remote administration. Authenticated via [Authentik on the VPS](../../../vps/).
- **Tailscale** — legacy, being phased out. See [mesh.md](mesh.md). - **Tailscale** — legacy, being phased out. See [mesh.md](mesh.md).
## Related ## Related
@@ -79,10 +79,10 @@ No NAT on Proxmox — traffic flows without double-NAT.
### 2026-06-17 — Fritz!Box DNSv4 server set to Technitium (192.168.8.2) ### 2026-06-17 — Fritz!Box DNSv4 server set to Technitium (192.168.8.2)
Household LAN clients (192.168.178.x) now resolve `*.hubris.network` to LAN IPs. Configured in Fritz!Box at Internet → Filter → DNS Server → DNSv4 Server → "Use other DNSv4 servers" → Preferred = `192.168.8.2`. No per-device or Netbird setup needed. Household LAN clients (192.168.178.x) now resolve `*.hubris.network` to LAN IPs. Configured in Fritz!Box at Internet → Filter → DNS Server → DNSv4 Server → "Use other DNSv4 servers" → Preferred = `192.168.8.2`. No per-device or Netbird setup needed.
Previous pool `.100.240` overlapped with all static LXCs/VMs (` .101.239`), creating IP conflict risk (DHCP could hand out an IP that a static service expects). Shrunk pool to `.241.254` via Technitium API. No services re-IP'd. 11 stale DHCP leases in `.101.110` will expire naturally. **Open:** ZimaOS (VM 100) holds DHCP lease `.103` but inventory expects `.195` — needs static IP set inside VM. See [plan](../plans/2026-06-03-dhcp-pool-exclude-static-ips.md). Previous pool `.100.240` overlapped with all static LXCs/VMs (` .101.239`), creating IP conflict risk (DHCP could hand out an IP that a static service expects). Shrunk pool to `.241.254` via Technitium API. No services re-IP'd. 11 stale DHCP leases in `.101.110` will expire naturally. **Open:** ZimaOS (VM 100) holds DHCP lease `.103` but inventory expects `.195` — needs static IP set inside VM. See [plan](../../../plans/2026-06-03-dhcp-pool-exclude-static-ips.md).
### 2026-06-02 — Executed migration; Proxmox as subnet router ### 2026-06-02 — Executed migration; Proxmox as subnet router
Fritz!OS 8.x does not support second IP networks on LAN ports, so the final design uses Proxmox as the router: `vmbr1` (eno1 → SODOLA → Fritz!Box) is the uplink at `192.168.178.10`; `vmbr0` is a portless internal bridge with `192.168.8.1` alias as the LXC gateway. Technitium DHCP enabled for `192.168.8.100240`. Caddy service unit was missing and recreated. See [migration plan](../plans/2026-06-01-slate-ax-to-sodola-migration.md). Fritz!OS 8.x does not support second IP networks on LAN ports, so the final design uses Proxmox as the router: `vmbr1` (eno1 → SODOLA → Fritz!Box) is the uplink at `192.168.178.10`; `vmbr0` is a portless internal bridge with `192.168.8.1` alias as the LXC gateway. Technitium DHCP enabled for `192.168.8.100240`. Caddy service unit was missing and recreated. See [migration plan](../../../plans/2026-06-01-slate-ax-to-sodola-migration.md).
### 2026-06-01 — Initial network doc; Slate AX retired; SODOLA switch added ### 2026-06-01 — Initial network doc; Slate AX retired; SODOLA switch added
Replaced the GL.iNet Slate AX sub-router with the SODOLA 5-Port 2.5Gbit managed switch. Eliminated double-NAT. See [migration plan](../plans/2026-06-01-slate-ax-to-sodola-migration.md). Replaced the GL.iNet Slate AX sub-router with the SODOLA 5-Port 2.5Gbit managed switch. Eliminated double-NAT. See [migration plan](../../../plans/2026-06-01-slate-ax-to-sodola-migration.md).

View File

@@ -178,8 +178,8 @@ done
- [Mesh migration](mesh.md) - [Mesh migration](mesh.md)
- [VPS hardening](vps-hardening.md) - [VPS hardening](vps-hardening.md)
- [Agent enrollment](../operations/agent-enrollment.md) - [Agent enrollment](../../../operations/agent-enrollment.md)
- [Homelab CLI](../bin/homelab) - [Homelab CLI](../../../bin/homelab)
## Changelog ## Changelog

View File

@@ -3,8 +3,8 @@
# Topology (generated) # Topology (generated)
Source: [inventory.yaml](../inventory.yaml) — 2 hypervisors, 20 LXCs, 2 VMs, 2 workstations, 18 services. Source: [inventory.yaml](../../../inventory.yaml) — 2 hypervisors, 20 LXCs, 2 VMs, 2 workstations, 18 services.
Edge semantics: [oikos/ontology.yaml](../oikos/ontology.yaml). Operating model: [OIKOS.md](../OIKOS.md). Edge semantics: [oikos/ontology.yaml](../../../oikos/ontology.yaml). Operating model: [OIKOS.md](../../../.agents/OIKOS.md).
## Compute & ingress ## Compute & ingress

View File

@@ -59,7 +59,7 @@ The alternative (dedicated virtual data disk on the `library` lvmthin pool, e.g.
## Changelog ## Changelog
### 2026-06-03 — Static IP set to `.195`; DHCP drift fixed ### 2026-06-03 — Static IP set to `.195`; DHCP drift fixed
ZimaOS had drifted from `.195` (Slate AX DHCP) → `.103` (Technitium DHCP), causing Caddy 502s. Injected `/etc/systemd/network/10-static.network` into overlay (match `en*/eth*`, address `192.168.8.195/24`, gateway `.1`, DNS `.2`). VM restarted; verified reachable at `.195`. Caddy (`zimaos.hubris.network`) now returns 200. See [plan](../plans/2026-06-03-dhcp-pool-exclude-static-ips.md). ZimaOS had drifted from `.195` (Slate AX DHCP) → `.103` (Technitium DHCP), causing Caddy 502s. Injected `/etc/systemd/network/10-static.network` into overlay (match `en*/eth*`, address `192.168.8.195/24`, gateway `.1`, DNS `.2`). VM restarted; verified reachable at `.195`. Caddy (`zimaos.hubris.network`) now returns 200. See [plan](../../../plans/2026-06-03-dhcp-pool-exclude-static-ips.md).
### 2026-05-15 — NFS mount relocated to `/media/library` (UI delete fix) ### 2026-05-15 — NFS mount relocated to `/media/library` (UI delete fix)

View File

@@ -11,4 +11,4 @@ Two QEMU VMs running on [hubris](../hosts/hubris.md):
- [Hubris host](../hosts/hubris.md) — both VMs run here - [Hubris host](../hosts/hubris.md) — both VMs run here
- [Containers index](../containers/index.md) — LXCs on both nodes - [Containers index](../containers/index.md) — LXCs on both nodes
- [README](../README.md) - [README](../../../README.md)

View File

@@ -5,7 +5,7 @@ Repo-side half of the Week-3 approval flow. This module owns the request/
grant lifecycle and the HMAC signing; it does NOT talk to Matrix directly. grant lifecycle and the HMAC signing; it does NOT talk to Matrix directly.
There is no dedicated Matrix bot in this homelab — alerts already go out There is no dedicated Matrix bot in this homelab — alerts already go out
as the operator's own Hermes agent posting to @dtoro:avispero (see as the operator's own Hermes agent posting to @dtoro:avispero (see
infrastructure/monitoring.md's homelab-health-watchdog). The integration knowledge/wiki/infrastructure/monitoring.md's homelab-health-watchdog). The integration
contract is: contract is:
1. An agent or the Week-3 scheduler calls `request()` (or the CLI 1. An agent or the Week-3 scheduler calls `request()` (or the CLI

View File

@@ -6,7 +6,7 @@
- role: docker-apps - role: docker-apps
- address: 192.168.8.205 (mesh: tailscale:apps) - address: 192.168.8.205 (mesh: tailscale:apps)
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/105-apps.md - doc: knowledge/wiki/containers/105-apps.md
- secrets: enrolled (age key present) - secrets: enrolled (age key present)
## Blast radius ## Blast radius
@@ -18,4 +18,4 @@
- see the services this host runs for action-level risk classes - see the services this host runs for action-level risk classes
## Recent changes ## Recent changes
- (none yet) - 2026-07-06T11:57:21+00:00 deploy-oikos-console (config_mutation) — ok

View File

@@ -6,7 +6,7 @@
- role: arr-stack - role: arr-stack
- address: 192.168.8.245 (mesh: tailscale:arr) - address: 192.168.8.245 (mesh: tailscale:arr)
- mounts: /mnt/media_local - mounts: /mnt/media_local
- doc: containers/122-arriman.md - doc: knowledge/wiki/containers/122-arriman.md
## Blast radius ## Blast radius
- impacts: service:arr_stack - impacts: service:arr_stack

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: authentik-gateway - role: authentik-gateway
- address: 192.168.8.6 - address: 192.168.8.6
- doc: containers/106-auth-outpost.md - doc: knowledge/wiki/containers/106-auth-outpost.md
## Blast radius ## Blast radius
- impacts: (none) - impacts: (none)

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: reverse-proxy - role: reverse-proxy
- address: 192.168.8.175 - address: 192.168.8.175
- doc: containers/121-caddy.md - doc: knowledge/wiki/containers/121-caddy.md
## Blast radius ## Blast radius
- impacts: service:caddy - impacts: service:caddy

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: dns-server - role: dns-server
- address: 192.168.8.2 - address: 192.168.8.2
- doc: containers/107-dns.md - doc: knowledge/wiki/containers/107-dns.md
## Blast radius ## Blast radius
- impacts: service:dns - impacts: service:dns
@@ -16,4 +16,4 @@
- see the services this host runs for action-level risk classes - see the services this host runs for action-level risk classes
## Recent changes ## Recent changes
- (none yet) - 2026-07-06T11:40:28+00:00 add-record (config_mutation) — ok

View File

@@ -5,7 +5,7 @@
- runs-on: host:strong - runs-on: host:strong
- role: matrix-server - role: matrix-server
- address: 192.168.8.242 - address: 192.168.8.242
- doc: containers/118-elementsynapse.md - doc: knowledge/wiki/containers/118-elementsynapse.md
## Blast radius ## Blast radius
- impacts: service:matrix - impacts: service:matrix

View File

@@ -6,7 +6,7 @@
- role: git-server - role: git-server
- address: 192.168.8.121 (mesh: tailscale:gitea) - address: 192.168.8.121 (mesh: tailscale:gitea)
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/104-gitea.md - doc: knowledge/wiki/containers/104-gitea.md
## Blast radius ## Blast radius
- impacts: service:gitea - impacts: service:gitea

View File

@@ -6,7 +6,7 @@
- role: book-library - role: book-library
- address: 192.168.8.247 - address: 192.168.8.247
- mounts: /mnt/media_local - mounts: /mnt/media_local
- doc: containers/130-grimmory.md - doc: knowledge/wiki/containers/130-grimmory.md
- secrets: enrolled (age key present) - secrets: enrolled (age key present)
## Blast radius ## Blast radius

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: home-automation - role: home-automation
- address: 192.168.8.101 (mesh: tailscale:homeassistant) - address: 192.168.8.101 (mesh: tailscale:homeassistant)
- doc: vms/108-haos.md - doc: knowledge/wiki/vms/108-haos.md
## Blast radius ## Blast radius
- impacts: service:haos - impacts: service:haos

View File

@@ -5,7 +5,7 @@
- runs-on: host:strong - runs-on: host:strong
- role: family-planner - role: family-planner
- address: 192.168.8.244 - address: 192.168.8.244
- doc: containers/129-house.md - doc: knowledge/wiki/containers/129-house.md
- secrets: enrolled (age key present) - secrets: enrolled (age key present)
## Blast radius ## Blast radius

View File

@@ -5,7 +5,7 @@
- role: hypervisor - role: hypervisor
- address: 192.168.8.77 (mesh: netbird:proxmox-server.netbird.selfhosted) - address: 192.168.8.77 (mesh: netbird:proxmox-server.netbird.selfhosted)
- mounts: /mnt/library - mounts: /mnt/library
- doc: hosts/hubris.md - doc: knowledge/wiki/hosts/hubris.md
- secrets: enrolled (age key present) - secrets: enrolled (age key present)
## Blast radius ## Blast radius

View File

@@ -6,7 +6,7 @@
- role: media-server - role: media-server
- address: 192.168.8.246 (mesh: tailscale:jellyfin) - address: 192.168.8.246 (mesh: tailscale:jellyfin)
- mounts: /mnt/media_local - mounts: /mnt/media_local
- doc: containers/101-jellyfin.md - doc: knowledge/wiki/containers/101-jellyfin.md
## Blast radius ## Blast radius
- impacts: service:jellyfin - impacts: service:jellyfin

View File

@@ -6,7 +6,7 @@
- role: photo-management - role: photo-management
- address: 192.168.8.136 (mesh: tailscale:muleimage) - address: 192.168.8.136 (mesh: tailscale:muleimage)
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/120-mule-images.md - doc: knowledge/wiki/containers/120-mule-images.md
## Blast radius ## Blast radius
- impacts: service:photos - impacts: service:photos

View File

@@ -6,7 +6,7 @@
- role: file-sync - role: file-sync
- address: 192.168.8.224 (mesh: tailscale:nextcloud) - address: 192.168.8.224 (mesh: tailscale:nextcloud)
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/114-nextcloud.md - doc: knowledge/wiki/containers/114-nextcloud.md
## Blast radius ## Blast radius
- impacts: service:nextcloud - impacts: service:nextcloud

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: storage-export - role: storage-export
- address: 192.168.8.200 - address: 192.168.8.200
- doc: containers/102-nfs-export.md - doc: knowledge/wiki/containers/102-nfs-export.md
## Blast radius ## Blast radius
- impacts: (none) - impacts: (none)

View File

@@ -6,7 +6,7 @@
- role: document-archive - role: document-archive
- address: 192.168.8.130 (mesh: tailscale:paperless) - address: 192.168.8.130 (mesh: tailscale:paperless)
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/103-paperless.md - doc: knowledge/wiki/containers/103-paperless.md
## Blast radius ## Blast radius
- impacts: service:paperless - impacts: service:paperless

View File

@@ -6,7 +6,7 @@
- role: rom-manager - role: rom-manager
- address: 192.168.8.249 - address: 192.168.8.249
- mounts: /mnt/media_local - mounts: /mnt/media_local
- doc: containers/134-romm.md - doc: knowledge/wiki/containers/134-romm.md
## Blast radius ## Blast radius
- impacts: (none) - impacts: (none)

View File

@@ -6,7 +6,7 @@
- role: anime-media-server - role: anime-media-server
- address: 192.168.8.248 - address: 192.168.8.248
- mounts: /mnt/media_local/anime - mounts: /mnt/media_local/anime
- doc: containers/133-seanime.md - doc: knowledge/wiki/containers/133-seanime.md
## Blast radius ## Blast radius
- impacts: (none) - impacts: (none)

View File

@@ -6,7 +6,7 @@
- role: workshop - role: workshop
- address: 192.168.8.109 (mesh: tailscale:sophia) - address: 192.168.8.109 (mesh: tailscale:sophia)
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/119-sophia.md - doc: knowledge/wiki/containers/119-sophia.md
## Blast radius ## Blast radius
- impacts: (none) - impacts: (none)

View File

@@ -4,7 +4,7 @@
- state: active - state: active
- role: hypervisor - role: hypervisor
- address: 192.168.178.181 - address: 192.168.178.181
- doc: hosts/strong.md - doc: knowledge/wiki/hosts/strong.md
- secrets: enrolled (age key present) - secrets: enrolled (age key present)
## Blast radius ## Blast radius

View File

@@ -6,7 +6,7 @@
- role: teddycloud - role: teddycloud
- address: 192.168.8.150 - address: 192.168.8.150
- mounts: /mnt/library - mounts: /mnt/library
- doc: containers/131-teddycloud.md - doc: knowledge/wiki/containers/131-teddycloud.md
## Blast radius ## Blast radius
- impacts: service:teddycloud - impacts: service:teddycloud
@@ -17,4 +17,4 @@
- see the services this host runs for action-level risk classes - see the services this host runs for action-level risk classes
## Recent changes ## Recent changes
- (none yet) - 2026-07-06T11:05:35+00:00 activate (config_mutation) — ok

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: trmnl-middleware - role: trmnl-middleware
- address: 192.168.8.211 - address: 192.168.8.211
- doc: containers/128-trmnl.md - doc: knowledge/wiki/containers/128-trmnl.md
## Blast radius ## Blast radius
- impacts: service:trmnl - impacts: service:trmnl

View File

@@ -5,7 +5,7 @@
- runs-on: host:hubris - runs-on: host:hubris
- role: nas-frontend-eval - role: nas-frontend-eval
- address: 192.168.8.195 - address: 192.168.8.195
- doc: vms/100-zimaos.md - doc: knowledge/wiki/vms/100-zimaos.md
## Blast radius ## Blast radius
- impacts: service:zimaos - impacts: service:zimaos

Some files were not shown because too many files have changed in this diff Show More