# 100 — `zimaos`

ZimaOS (IceWhale / CasaOS-family NAS distro), installed as a Proxmox VM to evaluate it as a potential primary NAS frontend in front of `/mnt/library` — alongside the existing fleet ([nextcloud (114)](../containers/114-nextcloud.md), [jellyfin (101)](../containers/101-jellyfin.md), [mule-images (120)](../containers/120-mule-images.md)).

## At a glance
- **Type:** QEMU VM
- **ZimaOS version:** `v1.6.1` (build 2026-04-21, kernel 6.12.25)
- **IP:** `192.168.8.195` (DHCP lease from the LAN router — see "Open items" below)
- **Hostname (LAN DNS):** [`zimaos.hubris.network`](../infrastructure/dns.md) → [caddy (121)](../containers/121-caddy.md) `192.168.8.175` → VM `192.168.8.195:80`. TLS terminates at Caddy (Let's Encrypt via IONOS DNS-01).
- **Resources:** 4 vCPU, 8 GiB RAM (balloon min 2 GiB), 64 GiB OS disk on `local-lvm`
- **BIOS / machine:** OVMF (UEFI), `q35`, `virtio-scsi-pci`
- **EFI disk:** none — ZimaOS installer writes a removable-media fallback (`/EFI/BOOT/BOOTX64.EFI`), so a persistent EFI vars disk isn't required. PVE warns "no efidisk configured! Using temporary efivars disk" on start; harmless.
- **ISO:** `/var/lib/vz/template/iso/zimaos-x86_64-1.6.1_installer.iso` (1.48 GB, from `IceWhaleTech/ZimaOS` GitHub releases; upstream publishes SHA256 only for the `.img`/`.raucb` variants, not the `.iso`)

## Web UI

- **LAN:** <https://zimaos.hubris.network> (via [caddy (121)](../containers/121-caddy.md))
- **Netbird mesh (off-LAN):** same URL — works because hubris advertises `192.168.8.0/24` into the mesh, and once a peer resolves `zimaos.hubris.network → 192.168.8.175`, the request tunnels home. DNS resolution from a mesh peer needs **either** a netbird Management nameserver group (`hubris.network` match domain → `192.168.8.180`), or an `/etc/hosts` override on the peer (`192.168.8.175 zimaos.hubris.network`). See [mesh](../infrastructure/mesh.md).
- **Direct fallback:** <http://192.168.8.195> — backend ZimaOS bundles its own Caddy (`Server: Caddy`, `Via: ZimaOS-Gateway`), which the front-end Caddy on LXC 121 reverse-proxies to. Two Caddys in series.

## SSH

Disabled by default. To enable:
1. Toggle SSH on in the ZimaOS web UI ("Settings → SSH"), OR
2. From the VM console, switch to TTY2 with **Alt+F2** and run `passwd` to set a root password first.

## Storage strategy

**Initial assumption (didn't work).** PVE 9's native virtiofs share with the host's existing `library` dir mapping (used by [108-haos](108-haos.md)) was the obvious path. Attaching it took one line — but ZimaOS's kernel ships *without* the virtiofs module (`modinfo virtiofs` → `Module virtiofs not found`). The kvm-side device was attached fine; the guest just can't mount it. Detached on 2026-05-14.

**What we ended up with.** A dedicated NFS re-export LXC [102-nfs-export](../containers/102-nfs-export.md) bind-mounts `/mnt/library` from the host and serves it over NFSv4 to ZimaOS. Performance is within ~2% of host-served NFS (LXC adds namespace isolation only, not an IO hop), with the upside that no NFS/RPC daemons run on the bare-metal host. Read-only during evaluation; promote to rw once the UI test confirms behavior.

**Filesystem layout inside ZimaOS.** `/etc/fstab` lives on a writable overlay so persistent mounts via fstab DO survive reboots, but the *standard* path to add a network share is the ZimaOS Files UI (Connect Network Storage → NFS). UID/GID mapping is squashed at the server to `www-data:media` (33:10000) — the same identity Nextcloud and mule-images use — so any rw activity from ZimaOS records as a normal `media`-group write.

**Caveats discovered during install.**
- Root filesystem is **squashfs / read-only**; only `/DATA` is writable.
- `dtoro` identity is `uid 999, gid 1000(samba), groups samba+wheel`. Primary group is **`samba`** (not `dtoro` — that group doesn't exist). SSH key needs `chown dtoro:samba` to satisfy sshd StrictModes.
- `dtoro`'s home directory is literally `/DATA` (not `/home/dtoro`). Authorized keys go at `/DATA/.ssh/authorized_keys`.
- `/etc` is a writable overlay rooted at `/mnt/overlay/upper_etc`. Mounts added to `/etc/fstab` persist across reboots.
- `mount.nfs4` is a stripped busybox variant — rejects `_netdev`, `nofail`, `actimeo=N`. The working minimal fstab line: `192.168.8.200:/mnt/library /media/library nfs rw,vers=4 0 0`. **Mount path matters** — see the 2026-05-15 Changelog entry on why the NFS export is mounted at `/media/library` and not `/DATA/library`.
- No `qemu-guest-agent` is bundled — `qm guest cmd` calls fail. IP discovery via `qm monitor` + `screendump` instead.
- No package manager at runtime — no `apt`, no `opkg`. Apps come via ZimaOS's CasaOS-style appstore only.

The alternative (dedicated virtual data disk on the `library` lvmthin pool, e.g. `qm set 100 --scsi1 library:1024`) was rejected because it would start empty and force content migration; the NFS path keeps `/mnt/library` as the single source of truth shared with the existing 8 LXCs.

## Open items

- **DHCP → static IP fixed (2026-06-03).** ZimaOS IP drifted from `.195` (Slate AX) → `.103` (Technitium) after the DHCP migration, causing Caddy 502s. Fixed by injecting a static systemd-networkd config and restarting the VM. IP now pinned at `192.168.8.195`. See [changelog](#2026-06-03--static-ip-set-to-195-dhcp-drift-fixed).
- **No Authentik wiring.** [authentik (124)](../containers/106-auth-outpost.md) isn't enforcing auth in front of ZimaOS yet — ZimaOS handles its own first-run wizard. The Caddyfile block uses bare `reverse_proxy` rather than the `import authentik` pattern used by e.g. artifacto; layer it in once the wizard is complete and a static admin user exists.
- **No PBS backup.** No Proxmox Backup Server configured on hubris today; this VM is not backed up.
- **qemu-guest-agent not installed.** ZimaOS's installer doesn't bundle it, so `qm guest cmd 100 ...` returns "QEMU guest agent is not running". IP discovery during this install was done via console screendump → `qm monitor` → `screendump`.

## Related
- [108 — `haos-16.3`](108-haos.md) — the existing VM, same q35/OVMF pattern; also a virtiofs `library` consumer
- [DNS](../infrastructure/dns.md) — split-horizon entry lives here
- [Caddy (121)](../containers/121-caddy.md) — would front this if/when promoted
- [Media permissions](../infrastructure/media-permissions.md) — relevant when wiring `/mnt/library`

## Changelog

### 2026-06-03 — Static IP set to `.195`; DHCP drift fixed
ZimaOS had drifted from `.195` (Slate AX DHCP) → `.103` (Technitium DHCP), causing Caddy 502s. Injected `/etc/systemd/network/10-static.network` into overlay (match `en*/eth*`, address `192.168.8.195/24`, gateway `.1`, DNS `.2`). VM restarted; verified reachable at `.195`. Caddy (`zimaos.hubris.network`) now returns 200. See [plan](../../../.hermes/plans/2026-06-03_223218-dhcp-pool-exclude-static-ips.md).

### 2026-05-15 — NFS mount relocated to `/media/library` (UI delete fix)

Symptom: deleting any file or folder inside the NFS-served `library` tree from the ZimaOS Files UI failed with `rename ... invalid cross-device link`. Root cause: ZimaOS's "Move to Trash" is implemented as `rename(2)` into a `.trash/` directory derived from the *drive root*, and icewhale-files identifies the drive root with the regex `^/media/([^/]+)`. With the NFS mounted at `/DATA/library`, the visible UI path was `/media/ZimaOS-HD/library/foo` → drive `ZimaOS-HD` → trash at `/media/ZimaOS-HD/.trash` (which lives on the local ext4 `/dev/sda8`). NFS → ext4 rename = `EXDEV`.

Fix: remount the NFS export at `/media/library` so it sits as its own `/media/<name>` segment. Now icewhale-files extracts drive=`library`, computes trash at `/media/library/.trash`, and the rename stays within the NFS — no cross-device. The local-storage daemon (LSBLK-based) doesn't surface the NFS in the Files-app *sidebar* (drives sidebar only lists block devices), but the icewhale-files daemon picks the mount up as its own "storage" tile in the Files view, and crucially exposes a direct **Delete** action there (no trash flow, since ZimaOS treats non-LSBLK mounts as foreign and skips trash for them). For a 4 TB NFS share, immediate delete is arguably the right behavior anyway.

Concrete change:
- `/etc/fstab` line changed from `… /DATA/library nfs rw,vers=4 0 0` to `… /media/library nfs rw,vers=4 0 0`.
- New mountpoint dir `/var/lib/casaos_data/.media/library` (= `/media/library` via the existing `/media` bind).
- Old `/DATA/library` mountpoint dir removed.

Side effect: navigating via `ZimaOS-HD → library` in the UI now shows an empty/missing folder. Users access the library via the separate `library` tile in the Files app instead.

### 2026-05-14 — Mesh-reachable from netbird peers
Verified `https://zimaos.hubris.network` reachable from `republic-laptop` (100.122.78.140) through the existing hubris→netbird subnet advertisement (`192.168.8.0/24`). No new infrastructure required — the path was already wired by [mesh](../infrastructure/mesh.md). Open: configure a netbird Management nameserver group for `hubris.network` so peers don't need per-device `/etc/hosts` overrides.

### 2026-05-14 — Phase 2 promoted to rw
NFS export on [nfs-export (102)](../containers/102-nfs-export.md) flipped from `ro` to `rw` after Files UI evaluation passed. Verified the squash: a write from ZimaOS appears on hubris's `/mnt/library` as `www-data:media` (uid 33, gid 10000), matching the existing tree convention.

### 2026-05-14 — Phase 2: NFS mount via new LXC 102 (read-only)
Virtiofs path abandoned — ZimaOS kernel 6.12.25 ships without the virtiofs module (verified by `modinfo virtiofs` returning "Module not found"). Pivoted to NFSv4: built [nfs-export (102)](../containers/102-nfs-export.md) as a dedicated 512-MiB privileged Debian LXC bind-mounting `/mnt/library` and serving it on `192.168.8.200:2049` with `ro,all_squash,anonuid=33,anongid=10000` to the LAN subnet. Detached the now-useless `virtiofs0` from VM 100. ZimaOS-side mount via Files UI (NFS protocol, server `192.168.8.200`, path `/mnt/library`). Promote export to `rw` after Files UI evaluation passes.

### 2026-05-14 — Wired through Caddy (HTTPS)
Added `zimaos.hubris.network` site block to `/etc/caddy/Caddyfile` on [caddy (121)](../containers/121-caddy.md): bare `reverse_proxy 192.168.8.195` + IONOS DNS-01 TLS, same pattern as plato/jellyfin. dnsmasq entry repointed from `192.168.8.195` to `192.168.8.175`. Let's Encrypt cert issued on first request. Caddy commit `a219176` pending push to `dtoro/caddy-conf`.

### 2026-05-14 — VM created, ZimaOS 1.6.1 installed (Phase 1)
`qm create 100` with q35/OVMF, no EFI disk, 4 vCPU / 8 GiB / 64 GiB on `local-lvm`. Installed via the official ISO (manual console install). Web UI verified at `http://192.168.8.195`. `onboot=1`, `startup order=20`. dnsmasq entry `zimaos.hubris.network → 192.168.8.195` initially added direct-to-VM on [authentik (124)](../containers/106-auth-outpost.md) (later repointed — see above). `/mnt/library` is **not** yet shared into the VM; Phase 2 (virtiofs) is gated on UI evaluation.