dtoro
6b75f7302d
- Migrations 010 (content_hash) + 011 (search tsvector column) - new: internal/knowledge/seed.go — knowledge seed ingest engine - new: internal/httpapi/knowledge.go — SearchKnowledge + GetEntityKnowledge - wire knowledge ingest into oikos seed pipeline - convert all 36 wiki docs + 6 investigations + 12 runbooks → seeds/knowledge.yaml - archive: knowledge/wiki/→archive/, oikos/cards/→archive/, .hermes/plans/→archive/ - delete: 9 superseded Python kernel files, ledger/, mcp/build_host_files.py - remove empty knowledge/ directory tree
277 lines
13 KiB
Markdown
277 lines
13 KiB
Markdown
# Plan: Migrate library SSD to ludo-mini + Proxmox gaming/media server
|
||
|
||
## Goal
|
||
|
||
Split the homelab into two Proxmox hosts:
|
||
|
||
| Host | Role | Storage |
|
||
|------|------|---------|
|
||
| **hubris** | Core services (reverse-proxy, SSO, Matrix, git, documents, HA) | SSD 1 — boot + LXC rootfs (unchanged) |
|
||
| **ludo-mini** | Gaming server + media/library services | SSD 2 — Samsung 990 EVO Plus 4 TB (moved from hubris) |
|
||
|
||
The library SSD physically moves from hubris to ludo-mini. hubris LXCs that still need `/mnt/library` access it over NFS from ludo-mini.
|
||
|
||
## Current state
|
||
|
||
### hubris hardware
|
||
- GMKtec NucBox M6 Ultra — AMD Ryzen 5 7640HS, 12 vCPU, ~28 GiB RAM
|
||
- 2× Samsung 990 EVO Plus NVMe:
|
||
- nvme0: `local` (95G) + `local-lvm` (856G) — boot, ISOs, LXC rootfs
|
||
- nvme1: `library` LVM (3.7T) — `/mnt/library` ext4 via `/dev/mapper/library-library`
|
||
|
||
### LXCs binding `/mnt/library` (host-level bind-mount)
|
||
| ID | Name | Role | I/O profile |
|
||
|----|------|------|-------------|
|
||
| 101 | jellyfin | Media streaming | Read-heavy, sequential |
|
||
| 103 | paperless | Document archive | Mixed, OCR writes |
|
||
| 104 | gitea | Git server | Mixed, lots of small files |
|
||
| 105 | apps | Docker (booklore, audiobookshelf, artifacto, MCP) | Mixed, depends on container |
|
||
| 114 | nextcloud | File sync | Mixed, WebDAV |
|
||
| 119 | sophia | Workshop | Low I/O |
|
||
| 120 | mule-images | Photo management | Write-heavy (processing), iGPU |
|
||
| 122 | arriman | *arr stack + downloads | Write-heavy (downloads) |
|
||
| 126 | plato | App (sub-mount: `/mnt/library/documents/plato`) | Light |
|
||
|
||
### NFS export chain (for VM 100 zimaos)
|
||
```
|
||
/mnt/library (ext4, host) → bind-mount → LXC 102 (nfs-export) → NFSv4 → VM 100 (zimaos)
|
||
```
|
||
|
||
### ludo-mini current
|
||
- Linux workstation, wired Ethernet 2.5 Gbps, `192.168.178.181` (household LAN)
|
||
- Runs Sunshine for game streaming
|
||
- No Proxmox, no LVM config
|
||
- Connected to SODOLA switch (same switch as hubris eno1)
|
||
|
||
### Network topology
|
||
```
|
||
Fritz!Box 7590 (192.168.178.1)
|
||
└── SODOLA 2.5G switch
|
||
├── hubris eno1 → vmbr1 (192.168.178.10)
|
||
│ └── routes to vmbr0 (192.168.8.0/24) — all LXCs
|
||
└── ludo-mini (192.168.178.181)
|
||
```
|
||
|
||
hubris routes between `192.168.8.0/24` (vmbr0) and `192.168.178.0/24` (vmbr1). So LXCs can reach ludo-mini via hubris as a router.
|
||
|
||
## Key decisions
|
||
|
||
### 1. Service split — what moves, what stays
|
||
|
||
**Move to ludo-mini** (high I/O, benefits from data locality + GPU):
|
||
- 101 jellyfin — media streaming, GPU transcoding
|
||
- 120 mule-images — photo processing, iGPU passthrough
|
||
- 122 arriman — *arr stack, downloads write to library
|
||
|
||
**Stay on hubris, NFS-mount library from ludo-mini:**
|
||
- 103 paperless — documents, moderate I/O
|
||
- 104 gitea — git repos (small files, some I/O sensitivity but acceptable over NFS)
|
||
- 105 apps — Docker apps, mixed workloads
|
||
- 114 nextcloud — file sync
|
||
- 119 sophia — workshop, light use
|
||
- 126 plato — app, light use
|
||
- 100 zimaos — NAS frontend, already NFS-mounted
|
||
|
||
### 2. NFS architecture
|
||
|
||
Instead of changing every LXC's mount config, keep the bind-mount pattern on hubris:
|
||
|
||
```
|
||
ludo-mini: /mnt/library (ext4, local NVMe)
|
||
│
|
||
└── NFSv4 export to 192.168.8.0/24
|
||
│
|
||
└── hubris host: NFS-mount at /mnt/library
|
||
│
|
||
└── LXCs: bind-mount /mnt/library (unchanged!)
|
||
```
|
||
|
||
This is transparent to all hubris LXCs — no container config changes needed. Only the hubris host changes from ext4 local mount to NFS mount. The LXC bind-mounts "just work" because `/mnt/library` is still at the same path on the host.
|
||
|
||
### 3. Network — ludo-mini reachability from hubris LXCs
|
||
|
||
LXCs on `192.168.8.0/24` reach ludo-mini (`192.168.178.181`) through hubris routing:
|
||
- `vmbr0` → hubris kernel routing → `vmbr1` → SODOLA → ludo-mini
|
||
- Already works (IP forwarding enabled on hubris)
|
||
|
||
**Alternative (cleaner):** Add a secondary IP `192.168.8.x` on ludo-mini's physical interface so it's directly on the homelab subnet. This avoids the router hop and keeps NFS traffic off kernel forwarding path. Worth considering but not required.
|
||
|
||
### 4. Gaming on ludo-mini with Proxmox
|
||
|
||
ludo-mini runs Sunshine (game streaming). Under Proxmox:
|
||
- **Option A:** Gaming VM with GPU passthrough — Sunshine + games in a VM, full GPU access
|
||
- **Option B:** LXC with GPU device passthrough (`/dev/dri`) — lighter, shares kernel
|
||
- **Option C:** Keep Sunshine on the Proxmox host itself (not recommended, but simplest)
|
||
|
||
Option A is the cleanest for isolation. Games need a full desktop environment and GPU drivers; a VM with GPU passthrough gives them that.
|
||
|
||
### 5. What about nfs-export (LXC 102)?
|
||
|
||
Currently exports `/mnt/library` to zimaos. After migration:
|
||
- If zimaos stays on hubris and accesses library via host NFS → bind-mount → LXC 102, that's triple-hop (ludo-mini → NFS → hubris → bind-mount → LXC 102 → NFS → zimaos). Terrible.
|
||
- Better: zimaos NFS-mounts directly from ludo-mini.
|
||
- So LXC 102 gets decommissioned (or repurposed).
|
||
- zimaos gets a new NFS mount pointing directly at ludo-mini.
|
||
|
||
## Migration phases
|
||
|
||
### Phase 1 — Preparation (no downtime)
|
||
|
||
1. **Document current state** on hubris:
|
||
- `pct list` — full container inventory
|
||
- `pct config <id>` for every library-mounting LXC
|
||
- `cat /etc/fstab` — capture the library mount line
|
||
- `df -h /mnt/library` — confirm space usage
|
||
- `lsblk -f` — UUID, filesystem
|
||
- Identify the exact NVMe device (`nvme1n1`)
|
||
|
||
2. **Pre-flight on ludo-mini:**
|
||
- Confirm hardware: CPU, RAM, available M.2 slots, GPU model
|
||
- Confirm it can take the Samsung 990 EVO Plus (M.2 NVMe, PCIe 4.0 x4)
|
||
- Verify BIOS supports virtualization (VT-d/AMD-Vi for PCIe passthrough)
|
||
- Check: does ludo-mini have a second drive for Proxmox OS? If not, we need to partition the library SSD for Proxmox boot + library LVM, which complicates things significantly
|
||
|
||
3. **Install Proxmox on ludo-mini:**
|
||
- Download Proxmox VE 9.x ISO
|
||
- Install to ludo-mini's system drive (NOT the library SSD)
|
||
- Configure networking: bridge for Proxmox, IP on 192.168.178.x
|
||
- Test: web UI accessible
|
||
|
||
4. **Prepare NFS server on ludo-mini Proxmox:**
|
||
- Create NFS-export LXC (or serve from host — simpler for now)
|
||
- Prepare `/etc/exports`: `192.168.8.0/24(rw,all_squash,anonuid=33,anongid=10000,no_subtree_check,sec=sys)`
|
||
- Same squash params as current nfs-export LXC 102
|
||
|
||
### Phase 2 — Physical SSD move (planned downtime)
|
||
|
||
1. **Graceful shutdown on hubris:**
|
||
- Stop all library-mounting LXCs (101, 103, 104, 105, 114, 119, 120, 122, 126)
|
||
- Unmount `/mnt/library` on hubris host
|
||
- Edit `/etc/fstab` to comment out the library mount line
|
||
- Power off hubris
|
||
|
||
2. **Physical drive swap:**
|
||
- Remove Samsung 990 EVO Plus (library SSD) from hubris
|
||
- Install into ludo-mini M.2 slot
|
||
- Power on ludo-mini
|
||
|
||
3. **Bring library online on ludo-mini:**
|
||
- Detect the new NVMe device
|
||
- If it's the whole device with LVM, activate the VG:
|
||
```
|
||
vgscan && vgchange -ay library
|
||
mount /dev/mapper/library-library /mnt/library
|
||
```
|
||
- Add to `/etc/fstab` for auto-mount
|
||
- Verify content: `ls /mnt/library` — same tree as before
|
||
|
||
4. **Start NFS export on ludo-mini:**
|
||
- `exportfs -ra`
|
||
- Verify: `showmount -e <ludo-mini-ip>`
|
||
|
||
### Phase 3 — Reconnect hubris LXCs
|
||
|
||
1. **Power on hubris** (without library SSD — it'll boot fine, just won't mount library)
|
||
|
||
2. **Mount NFS on hubris host:**
|
||
- Install `nfs-common` if not present
|
||
- Add to `/etc/fstab`:
|
||
```
|
||
192.168.178.181:/mnt/library /mnt/library nfs rw,vers=4,soft,timeo=30,retrans=3 0 0
|
||
```
|
||
Use `soft` to prevent hangs if ludo-mini is down; `hard` with `intr` is safer for data integrity but can block processes.
|
||
- `mount /mnt/library`
|
||
- **Verify permissions:** `ls -la /mnt/library` — should show `www-data:media` ownership for shared subtrees (same uid 33, gid 10000). The NFS all_squash guarantees this.
|
||
|
||
3. **Start LXCs:**
|
||
- Start the LXCs that stayed on hubris (103, 104, 105, 114, 119, 126)
|
||
- Their bind-mounts should work — `/mnt/library` is populated via NFS
|
||
- Verify each service: web UIs, git clone, document access
|
||
|
||
4. **Update DNS/Caddy** for services that moved:
|
||
- If jellyfin, arr services moved to ludo-mini, update Caddyfile to point to ludo-mini IPs
|
||
- Update DNS entries if needed
|
||
|
||
### Phase 4 — Migrate services to ludo-mini
|
||
|
||
1. **Create LXCs/VM on ludo-mini Proxmox:**
|
||
- 101 jellyfin — privileged LXC, mount `/mnt/library`, add `media` group
|
||
- 120 mule-images — privileged LXC, mount `/mnt/library` + `/dev/dri` passthrough
|
||
- 122 arriman — privileged LXC, mount `/mnt/library`
|
||
|
||
2. **Migrate configs:**
|
||
- Copy LXC configs from hubris (`/etc/pve/lxc/<id>.conf`) as templates
|
||
- Adjust network (IPs on household subnet or Proxmox bridge)
|
||
- Restore app data from backups or copy over NFS
|
||
|
||
3. **Gaming VM:**
|
||
- Create VM with GPU passthrough
|
||
- Pass through the dGPU for gaming performance
|
||
- Install Sunshine + game libraries
|
||
- Storage: VM disk on Proxmox storage, games on library SSD
|
||
|
||
4. **Update reverse proxy:**
|
||
- Caddy on hubris (121): update backend IPs for jellyfin, jellyseerr, qbit, sab, mule-images → point to ludo-mini
|
||
- Test: `media.hubris.network` serves from ludo-mini jellyfin
|
||
|
||
### Phase 5 — Cleanup
|
||
|
||
1. **Decommission nfs-export LXC 102** on hubris (no longer needed)
|
||
2. **Update zimaos (100)** — change NFS mount from `192.168.8.200` → `192.168.178.181`
|
||
3. **Remove old LXCs** from hubris (101, 120, 122) after confirming migration works
|
||
4. **Update inventory.yaml:**
|
||
- ludo-mini: `kind: proxmox-host`, add mounts/storage, add LXCs
|
||
- Move services from hubris to ludo-mini
|
||
- Remove nfs-export
|
||
5. **Update DNS:** `nfs-export.hubris.network` → ludo-mini IP (or remove)
|
||
6. **Run `homelab sync`** to propagate changes
|
||
|
||
## Open questions / unknowns
|
||
|
||
1. **Does ludo-mini have a second drive for Proxmox OS?** If not, we'd need to repartition the library SSD — carve out ~100 GB for Proxmox, then the rest for library LVM. This is risky (data loss if partitioning goes wrong) and requires a full backup first. **Alternative:** Buy a small SSD for ludo-mini's OS.
|
||
|
||
2. **What GPU does ludo-mini have?** Proxmox GPU passthrough requires IOMMU support and a GPU that doesn't have the reset bug. Need to check the exact GPU model.
|
||
|
||
3. **NFS performance for git (gitea)?** Git operations over NFS can be problematic (locking, stat() storms). Gitea bare repos at `/mnt/library/repos/*.git` might need testing. Worst case: move gitea's repo storage to local disk and keep `/mnt/library` for large file/LFS storage only.
|
||
|
||
4. **Media permission drift.** NFS `all_squash,anonuid=33,anongid=10000` ensures all writes from hubris LXCs (over NFS) and ludo-mini LXCs (local) land as `www-data:media`. This is the same squash currently used by nfs-export (102). Should be fine.
|
||
|
||
5. **ludo-mini network — add 192.168.8.x address?** Adding a secondary IP on ludo-mini's interface directly on the homelab subnet avoids routing through hubris for NFS traffic. Cleaner, but requires Proxmox bridge setup. Worth doing during Proxmox install.
|
||
|
||
6. **Sunshine migration.** Currently runs on ludo-mini bare metal. After Proxmox install, it needs to run in a VM. What happens to existing Sunshine configs, game libraries, save files? Need to preserve these during the Proxmox install.
|
||
|
||
7. **Backup before moving.** The library SSD holds 3.7 TB of irreplaceable data (documents, photos, repos). Restic backups are currently disabled. **Before physically moving the drive, verify the data is readable and consider doing one backup** — or at minimum, `rsync` critical directories.
|
||
|
||
## Files affected
|
||
|
||
| File | Change |
|
||
|------|--------|
|
||
| `/opt/homelab-context/inventory.yaml` | ludo-mini: workstation → proxmox-host; add LXCs, mounts; remove nfs-export; move service backends |
|
||
| `/opt/homelab-context/hosts/hubris.md` | Remove library storage, add NFS mount note |
|
||
| `/opt/homelab-context/hosts/ludo-mini.yaml` | Complete rewrite — Proxmox host, storage, tenants |
|
||
| `/opt/homelab-context/containers/102-nfs-export.md` | Mark decommissioned |
|
||
| `/opt/homelab-context/containers/index.md` | Move 101, 120, 122 to ludo-mini; remove 102 |
|
||
| `/opt/homelab-context/infrastructure/dns.md` | Update nfs-export entry |
|
||
| `/opt/homelab-context/infrastructure/media-permissions.md` | Note NFS squash from ludo-mini, not hubris |
|
||
| hubris `/etc/fstab` | Replace ext4 mount with NFS mount |
|
||
| ludo-mini `/etc/fstab` | Add library ext4 mount |
|
||
| ludo-mini `/etc/exports` | Add NFS export config |
|
||
| caddy (LXC 121) Caddyfile | Backend IPs for moved services |
|
||
| DNS (LXC 107 Technitium) | Update entries for moved services |
|
||
|
||
## Validation checklist
|
||
|
||
- [ ] ludo-mini Proxmox web UI accessible
|
||
- [ ] Library SSD detected and mountable on ludo-mini
|
||
- [ ] NFS export from ludo-mini: `showmount -e <ip>` shows `/mnt/library`
|
||
- [ ] hubris host NFS mount: `df -h /mnt/library` shows NFS, not ext4
|
||
- [ ] hubris LXCs start and bind-mount /mnt/library (content visible)
|
||
- [ ] gitea: `git clone` over SSH works, repos readable
|
||
- [ ] paperless: document ingestion works, OCR processing
|
||
- [ ] nextcloud: file sync, WebDAV
|
||
- [ ] jellyfin: media plays from ludo-mini, transcoding works
|
||
- [ ] arriman: downloads write to library, jellyfin picks up new media
|
||
- [ ] mule-images: photo import and processing
|
||
- [ ] zimaos: NFS mount from ludo-mini works, Files UI shows library
|
||
- [ ] Sunshine: game streaming from ludo-mini VM works
|
||
- [ ] All `*.hubris.network` services resolve and load through Caddy |