From b34d362c19676affdfa69efc534b26efa1fb7926 Mon Sep 17 00:00:00 2001 From: dtoro Date: Fri, 22 May 2026 00:53:34 +0200 Subject: [PATCH] photos: mule-image cutover to PhotoPrism stack on 120, retire 127 Mulimage 2.0 merge (dtoro/mule-image 70dc1b6) replaces the legacy FastAPI/Celery/Postgres stack with PhotoPrism + Go sidecar + MariaDB + SvelteKit. LXC 120 now runs that architecture in production; LXC 127 (the M0 evaluation host) was destroyed in the same session. - inventory.yaml: drop mule-photos-new host + photos_new service. - containers/index.md: bump 120's row (6 cores / 12 GiB / iGPU passthrough), move 127 to Recently-destroyed. - containers/120-mule-images.md: rewrite stack + library-access + auth sections; add 2026-05-22 cutover changelog entry. - containers/127-mule-photos-new.md: add closing changelog entry and decommission banner; rest left intact as archaeology. - hosts/mule-photos-new.yaml: removed (regenerated from inventory.yaml). Co-Authored-By: Claude Opus 4.7 (1M context) --- containers/120-mule-images.md | 120 ++++++++++++++++++------------ containers/127-mule-photos-new.md | 15 ++++ containers/index.md | 4 +- hosts/mule-photos-new.yaml | 26 ------- inventory.yaml | 11 --- 5 files changed, 89 insertions(+), 87 deletions(-) delete mode 100644 hosts/mule-photos-new.yaml diff --git a/containers/120-mule-images.md b/containers/120-mule-images.md index 3a005d6..69de592 100644 --- a/containers/120-mule-images.md +++ b/containers/120-mule-images.md @@ -1,68 +1,46 @@ # 120 — `mule-images` -Hosts `mule-image` / "mulita" — the photos app at `photos.hubris.network`. Auto-deploys from `dtoro/mule-image` on `git push origin main`. +Hosts `mule-image` — the photos app at `photos.hubris.network`. PhotoPrism + Go sidecar + SvelteKit, replacing the legacy FastAPI/Celery stack as of 2026-05-22 (see Changelog). Auto-deploys from `dtoro/mule-image` on `git push origin main`. ## At a glance - **Hostname:** `mule-images` - **IP:** `192.168.8.136` - **Privilege:** privileged -- **Resources:** 4 cores / 8 GiB RAM / 60 GiB rootfs -- **Mounts:** `/mnt/library` ↔ `/mnt/library` -- **Public hostname:** [`photos.hubris.network`](../infrastructure/dns.md) → [caddy](121-caddy.md) → `:3000` (frontend) +- **Resources:** 6 cores / 12 GiB RAM / 60 GiB rootfs / 2 GiB swap +- **Mounts:** `/mnt/library` ↔ `/mnt/library`; `/dev/dri/{card0,renderD128}` passed through for VA-API video accel on the AMD Phoenix1 iGPU. +- **Public hostname:** [`photos.hubris.network`](../infrastructure/dns.md) → [caddy](121-caddy.md) → path-routed to PhotoPrism `:2342` / sidecar `:8000` / nginx-static `:3000`. ## Stack (`/opt/mule-image`) -`/opt/mule-image` IS the working tree of `dtoro/mule-image`. Compose at `/opt/mule-image/docker-compose.yml`. Services: +`/opt/mule-image` IS the working tree of `dtoro/mule-image`. Compose stack: base `docker-compose.yml` + LAN-port-rebind `docker-compose.override.yml` (untracked) + VA-API `docker-compose.gpu.yml`. The SvelteKit frontend is built on the host and served as a static bundle by nginx — no `vite dev` in prod. -| Service | Port | Notes | -| ----------------- | ------ | ----- | -| frontend | 3000 | Reverse-proxied by Caddy | -| backend | 8001 | FastAPI | -| worker-light | — | Sole celery worker — runs default/high/low queues + beat scheduler | -| db | (pg) | postgres:16 (was pgvector/pg16 until 2026-05-14) | -| redis | (rd) | queue | +| Service / process | Port (LAN) | Notes | +| ------------------------ | --------------------- | ----- | +| pp-mariadb | (internal 3306) | MariaDB 11; holds PhotoPrism's `photoprism` DB + sidecar's `mule_sidecar.*` | +| pp-app | `0.0.0.0:2342` | PhotoPrism `:latest`; `PHOTOPRISM_FFMPEG_ENCODER=h264_vaapi` | +| pp-sidecar | `0.0.0.0:8000` | Go service (file rename / folder ops / heap convert / dup scan / per-photo marks); also reconciles `USER_BASEPATHS` into PhotoPrism's `auth_users` every 60 s | +| nginx (host process) | `0.0.0.0:3000` | Serves the SvelteKit static bundle from `web/build/` | -**No vision / ML services.** The OpenCLIP ONNX classifier and its dedicated `worker-vision` were ripped out on 2026-05-14 — see Changelog. +**Per-user scoping** is driven by `USER_BASEPATHS=admin:admin/files, muli:muli/files` in `.env`. Sidecar applies it to PhotoPrism's `auth_users` table on boot + every 60s, `mkdir -p`s each target so PhotoPrism's ACL filter has somewhere to point. -`worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below. +`.env` is **untracked** — `git reset --hard` won't touch it, but `git clean -fdx` would. Holds PP_*, MariaDB passwords, SIDECAR_DB_PASSWORD, OIDC_* (existing `mule-image` Authentik app, redirect URI now `/api/v1/oidc/redirect`), `USER_BASEPATHS`, and `PP_FFMPEG_ENCODER=vaapi`. -`.env` is **untracked** — `git checkout .env` will wipe it. Holds: -- `PHOTO_DIRS=/mnt/library/images/` -- `NEXTCLOUD_USERS_HOST_PATH=/mnt/library/homecloud` -- `NEXTCLOUD_BASE_URL=https://cloud.hubris.network` -- OIDC client secret + scopes -- `SECRET_KEY` (generated) -- `NEXTCLOUD_WEBHOOK_SECRET` (since 2026-05-11; bearer token NC presents on every webhook call to `/api/v1/internal/nc-webhook`) +`docker-compose.override.yml` is also untracked — it `!override`s the upstream `127.0.0.1:port` mappings to `0.0.0.0:port` so cross-host Caddy on [LXC 121](121-caddy.md) can reach pp-app + sidecar. -## Nextcloud integration (since 2026-05-11) +## Library access -Phase 1 + 2 of dropping the duplicated indexing work between mule and Nextcloud — see `~/.claude/projects/-root/memory/project_mule_nc_integration.md` for the full detail. +PhotoPrism reads the library **directly off the bind-mounted filesystem** — no Nextcloud webhook integration in the new stack. The base path is `/mnt/library/homecloud` and per-user scoping comes from `USER_BASEPATHS` (see Stack section above). -**Thumbnails (Phase 1, commit 576b0c2):** `GET /api/v1/photos/{id}/thumb/{size}` proxies `https://cloud.hubris.network/index.php/core/preview` keyed by `photos.nextcloud_fileid` (alembic 0018), authenticated with the photo owner's encrypted NC app password. NC handles JPEG/HEIC/TIFF/RAW/DNG/Movie previews natively. `X-Mule-Thumb-Source: nextcloud|disk` on every response. Worker now only writes the `medium` size to `/data/thumbs/{user}/{photo}/medium.webp` (the vision worker still loads it from disk); `small` + `large` are NC-proxied. Backfill script: `backend/scripts/backfill_nextcloud_fileid.py` — 17,591/17,591 photos filled. +- `dtoro` is mapped to NC user `admin` → `/mnt/library/homecloud/admin/files/` +- `muli` is mapped to NC user `muli` → `/mnt/library/homecloud/muli/files/` -**File events (Phase 2, commits 362fbc6 + f657e2c):** mule no longer runs `watchfiles`. NC's `webhook_listeners` app (already installed, registered via `backend/scripts/register_nc_webhooks.py`) POSTs every `NodeCreated|NodeWritten|NodeDeleted|NodeRenamed` event to `http://192.168.8.136:8001/api/v1/internal/nc-webhook` with `Authorization: Bearer $NEXTCLOUD_WEBHOOK_SECRET`. The endpoint dispatches the same `scan_folder` / `handle_file_deletion` machinery the watcher used. **NC dispatches webhooks asynchronously via its background-job queue** — `crontab -u www-data` on LXC 114 runs `cron.php` every minute (`*/1`), so end-to-end latency is ≲60 s. Re-running the registrar script is idempotent (it deletes existing webhooks pointing at the same URL first). +PhotoPrism's container user is `33:10000` (`www-data:media`), matching the host ownership of the NC tree. The legacy `oc_webhook_listeners` rows + `NEXTCLOUD_WEBHOOK_SECRET` are gone — when NC writes via WebDAV (still its own primary surface), PhotoPrism picks up the new file on its next index pass. -## Nextcloud-rooted libraries (since 2026-04-26) +## Authentication -Photo libraries live under each user's Nextcloud `files/` tree, NOT in `/mnt/library/images/*`. -- `/mnt/library/homecloud` is bind-mounted into `backend`, `worker-light`, `worker-watcher`, `worker-vision` as `/nextcloud-users`. Each NC user is `/nextcloud-users//files/`. -- Reads use that bind directly. -- Mutations (upload, delete, rename, move) dispatch through `services/nextcloud_dav.py` (HTTP Basic auth, per-user app password Fernet-encrypted in `users.nextcloud_app_password_enc`) so Nextcloud's `oc_filecache`, trashbin, comments, and desktop-sync clients stay coherent. -- Photo copy + cross-system moves return 501 with a "use Nextcloud's web UI" hint — defer until needed. -- `users.nextcloud_username` overrides the default OIDC `preferred_username`. **`dtoro` (mule-image) maps to `admin` (Nextcloud)** — don't assume username equality. -- Surviving SourceRoots in DB: `Photos` → `/nextcloud-users/admin/files/Photos`; `Memories` → `/nextcloud-users/admin/files/Memories` (both owned by `dtoro`). User `muli` has `nextcloud_username=muli` backfilled but no SourceRoot yet. -- Pre-migration DB dump: `/root/snapshots/mulita-pre-nc-migration-20260426-075132.dump` (11 MB) on the host. +OIDC via Authentik. App slug `mule-image`, redirect URI `https://photos.hubris.network/api/v1/oidc/redirect` (PhotoPrism's auto-derived path; PhotoPrism builds it from `PHOTOPRISM_SITE_URL`). `OIDC_REGISTER=true` auto-creates a PhotoPrism user at role `user` on first SSO login. The sidecar's basepath reconciler then assigns their scoped folder. -## Authentication (since 2026-04-22) - -Native OIDC via Authentik. Code in `backend/app/auth_oidc.py`, routes `/api/v1/auth/oidc/{login,callback}`. Authentik side: -- OAuth2/OIDC Provider, client ID `fCuHew48ONTskDjUKnMTZjFbVXuHwvQqTScQRNQ1` -- App slug `mule-image` -- Redirect URI: `https://photos.hubris.network/api/v1/auth/oidc/callback` - -Backend container needs `extra_hosts: auth.hubris.network:192.168.8.175` via `docker-compose.override.yml` (gitignored). Otherwise Authlib's metadata fetch fails with `SSL: CERTIFICATE_VERIFY_FAILED: self-signed certificate` (it ends up at a random public host because LXC DNS resolves the public IONOS A record). - -Caddyfile stays plain `reverse_proxy 192.168.8.136:3000` — no forward-auth, no `/api/*` bypass needed. +The `mule-image` Authentik app's redirect URI was migrated from the legacy FastAPI `/api/v1/auth/oidc/callback` path on 2026-05-22 — same client ID/secret were reused. The separate `mule-photos-new` Authentik app was deleted in the same operation. ## Auto-deploy @@ -70,11 +48,10 @@ Push to `dtoro/mule-image` `main` → gitea webhook → `http://192.168.8.136:97 - Validates HMAC against `/etc/mule-deploy/secret` - Filters to `refs/heads/main` - Runs `/opt/mule-deploy/deploy.sh` in a daemon thread (returns 202 immediately — docker builds exceed gitea's request timeout) -- `git pull --ff-only` + `docker compose up -d --build` + `docker image prune -f` +- `git fetch && git reset --hard origin/main`, force-recreates **only the sidecar** image (so PhotoPrism's `Session` HMAC key in `pp/storage/config/hub.yml` doesn't rotate and invalidate every in-flight OIDC state cookie), reconciles `pp-app` + `mariadb` in place, `cd web && npm ci && npm run build`, `systemctl reload nginx`. +- Conditionally layers `docker-compose.gpu.yml` when `/dev/dri/renderD128` exists, and always layers `docker-compose.override.yml` (the LAN-port rebind) when present. -Deploy tooling is **outside** the app repo: `/opt/mule-deploy/{deploy.sh,webhook.py}`, secret at `/etc/mule-deploy/secret`, unit at `/etc/systemd/system/mule-deploy-webhook.service`. Same shape as the Caddy + Artifacto pipelines. Gitea webhook id 6. - -`app.ini` `ALLOWED_HOST_LIST` on [gitea](104-gitea.md) includes `192.168.8.136`. +Deploy tooling is **outside** the app repo: `/opt/mule-deploy/{deploy.sh,webhook.py}`, secret at `/etc/mule-deploy/secret`. Gitea webhook id 6. `app.ini` `ALLOWED_HOST_LIST` on [gitea](104-gitea.md) includes `192.168.8.136`. Logs: `pct exec 120 -- journalctl -u mule-deploy-webhook -f`. Manual deploy: `pct exec 120 -- /opt/mule-deploy/deploy.sh`. @@ -91,6 +68,53 @@ For pushes from inside the LXC, gitea creds at `/etc/mule-deploy/git-credentials ## Changelog +### 2026-05-22 — Cutover to PhotoPrism stack (Mulimage 2.0) + +The `new` branch of `dtoro/mule-image` was merged into `main` as commit +`70dc1b6`. The merge replaces the legacy FastAPI + PostgreSQL + Celery ++ React stack with PhotoPrism + Go sidecar + MariaDB + SvelteKit, lifted +in from the M0 evaluation on (now-destroyed) [LXC 127](#). + +Cutover on 120 (this session): +- Bumped to 6 cores / 2 GB swap. `/dev/dri/{card0,renderD128}` already + passed through; added an opt-in `docker-compose.gpu.yml` overlay that + the deploy script layers in when the device is present. PhotoPrism now + uses `h264_vaapi` instead of `libx264`. +- Installed host nginx + a small `photos.conf` vhost serving the SvelteKit + static bundle from `/opt/mule-image/web/build/` on `:3000` — no Vite + dev server in prod. SvelteKit's `adapter-static` produces a real SPA + bundle. +- Replaced `/opt/mule-deploy/deploy.sh` with the 127-style multi-compose + deploy (force-recreates only sidecar to preserve PhotoPrism's Session + HMAC key; reconciles pp-app + mariadb in place; rebuilds web/ and + reloads nginx). +- `/opt/mule-image/.env` was rewritten to the PhotoPrism shape. Reused + passwords from `/root/mule-photos-new-secrets.txt`. `USER_BASEPATHS= + admin:admin/files, muli:muli/files` exposes both libraries. +- `docker-compose.override.yml` (untracked) `!override`s pp-app + + sidecar ports to `0.0.0.0` so cross-LXC caddy on [121](121-caddy.md) + can reach them. +- Authentik: `mule-image` app's redirect URI updated to `/api/v1/oidc/redirect`; + the separate `mule-photos-new` app deleted. +- Caddyfile (`dtoro/caddy-conf`): `photos.hubris.network` switched from + plain passthrough to path-matchers (PhotoPrism API + sidecar API + + nginx static catch-all), and `photos-new.hubris.network` was removed + entirely. +- Cleanups: 4 Nextcloud `oc_webhook_listeners` rows for the legacy + `/api/v1/internal/nc-webhook` endpoint deleted; gitea webhook id 9 + (the `refs/heads/new` deploy hook for 127) deleted; `192.168.8.181` + removed from gitea's `ALLOWED_HOST_LIST`; `photos-new.hubris.network` + removed from dnsmasq. +- LXC 127 destroyed via `pct destroy 127` after the new stack passed + curl verification end-to-end (PP `/api/v1/status` returns + `{"status":"operational"}` through caddy; sidecar `/api/sidecar/healthz` + returns `{"ok":true,...}`). +- Rollback safety net: legacy `mulita` postgres dump at + `/root/backups/mulita-legacy-final-20260521-212036.sql.gz` on hubris + (10 MB, 16,155 photo rows). The `mule-image_mule_db_data` docker + volume on 120 retains the on-disk postgres data for at least 24 h + before housekeeping removes it. + ### 2026-05-14 — Drop AI/vision pipeline, plain Postgres, DB↔FS refresh **AI removal (commits `a27267f` + `423a73a`).** The OpenCLIP-on-ONNX diff --git a/containers/127-mule-photos-new.md b/containers/127-mule-photos-new.md index 1af926a..bbf1574 100644 --- a/containers/127-mule-photos-new.md +++ b/containers/127-mule-photos-new.md @@ -144,8 +144,23 @@ curl -sk --resolve photos-new.hubris.network:443:192.168.8.175 \ https://photos-new.hubris.network/api/v1/oidc/login -i | head -2 # 302 → auth.hubris.network ``` +> **Decommissioned 2026-05-22.** The PhotoPrism + sidecar + SvelteKit stack +> validated here was promoted into production on [LXC 120](120-mule-images.md) +> via the `Mulimage 2.0` merge (`dtoro/mule-image` commit `70dc1b6`). This +> page is retained for archaeology; everything below is historic. See the +> 2026-05-22 entry in [120-mule-images.md](120-mule-images.md#changelog) for +> the cutover detail. + ## Changelog +### 2026-05-22 — Destroyed + +`pct destroy 127` after curl verification of the new 120 stack passed +end-to-end. dnsmasq `photos-new.hubris.network` line removed; gitea +webhook id 9 + `192.168.8.181` ALLOWED_HOST_LIST entry removed; caddy +`photos-new.hubris.network` site block dropped from `dtoro/caddy-conf`. +Authentik `mule-photos-new` app + provider deleted. + ### 2026-05-18 (pm) — OIDC state-cookie fix: deploy.sh no longer recreates pp-app User reported `failed to get state: securecookie: the value is not valid` after diff --git a/containers/index.md b/containers/index.md index 45e0179..2fafded 100644 --- a/containers/index.md +++ b/containers/index.md @@ -11,18 +11,18 @@ All containers live on [`hubris`](../hosts/hubris.md). Each row links to the per | 114 | [nextcloud](114-nextcloud.md) | 192.168.8.224 | priv | 4 | 6 GiB | 25 GiB | `/mnt/library` | `cloud.hubris.network` | running | | 118 | [elementsynapse](118-elementsynapse.md) | 192.168.8.239 | unpriv | 1 | 2 GiB | 8 GiB | — | `matrix.hubris.network` | running | | 119 | [sophia](119-sophia.md) | 192.168.8.157 | priv | 2 | 1 GiB | 10 GiB | `/mnt/library` | — | running | -| 120 | [mule-images](120-mule-images.md) | 192.168.8.136 | priv | 4 | 8 GiB | 60 GiB | `/mnt/library` | `photos.hubris.network` | running | +| 120 | [mule-images](120-mule-images.md) | 192.168.8.136 | priv | 6 | 12 GiB | 60 GiB | `/mnt/library` + `/dev/dri` (iGPU passthrough) | `photos.hubris.network` | running | | 121 | [caddy](121-caddy.md) | 192.168.8.175 | unpriv | 1 | 512 MiB | 6 GiB | — | (terminates all `*.hubris.network`) | running | | 122 | [arriman](122-arriman.md) | 192.168.8.132 | priv | 4 | 8 GiB | 24 GiB | `/mnt/library` | `jellyseerr` / `qbit` / `sab` | running | | 123 | [claudio-bot](123-claudio-bot.md) | 192.168.8.230 | unpriv | 1 | 512 MiB | 8 GiB | — | — | running | | 124 | [authentik](124-authentik.md) | 192.168.8.180 | priv | 2 | 4 GiB | 20 GiB | — | `auth.hubris.network` | running | | 126 | [plato](126-plato.md) | 192.168.8.190 | priv | 2 | 2 GiB | 8 GiB | `/mnt/library/documents/plato` | `plato.hubris.network` | running | -| 127 | [mule-photos-new](127-mule-photos-new.md) | 192.168.8.181 | unpriv | 6 | 8 GiB | 40 GiB | — (rsync scratch on rootfs) | `photos-new.hubris.network` | running | ## Recently destroyed (kept for archaeology) | ID | Name | Destroyed | Reason | | --- | ---------------- | --------------- | --------------------------------------------- | +| 127 | mule-photos-new | 2026-05-22 | PhotoPrism + sidecar + SvelteKit stack promoted to LXC 120 via Mulimage 2.0 merge (`70dc1b6`); M0 test LXC retired. Caddy + dnsmasq + gitea webhook + NC webhook listeners all cleaned up in the same cutover. | | 100 | arr (yunohost) | ~2026-04-28 | Migrated to docker stack on [arriman](122-arriman.md); planned retention window expired | | 106 | flaresolverr | ~2026-04-28 | Folded into the arriman docker compose | | 116 | heaper | 2026-05-14 | Decommissioned by user; data subtree at `/mnt/library/heaper` (224 MiB) retained | diff --git a/hosts/mule-photos-new.yaml b/hosts/mule-photos-new.yaml deleted file mode 100644 index bb890b0..0000000 --- a/hosts/mule-photos-new.yaml +++ /dev/null @@ -1,26 +0,0 @@ -# Generated by mcp/build_host_files.py from inventory.yaml. -# Do NOT edit by hand — your changes will be overwritten. -# Source of truth: ../inventory.yaml -name: mule-photos-new -kind: lxc -os: linux -role: photo-management -host: hubris -pve_id: 127 -lan_ip: 192.168.8.181 -mesh_globals: - primary: netbird - accepted: - - netbird - - tailscale -public_host: photos-new.hubris.network -runs: -- photos_new -services_hosted: -- name: photos_new - url: https://photos-new.hubris.network - backend: mule-photos-new -see_also: -- containers/127-mule-photos-new.md -mcp_endpoint: https://mcp.hubris.network/sse -secrets_issuance_endpoint: https://secrets.hubris.network/issue diff --git a/inventory.yaml b/inventory.yaml index c823ec9..c8ef328 100644 --- a/inventory.yaml +++ b/inventory.yaml @@ -69,9 +69,6 @@ services: photos: url: https://photos.hubris.network backend: mule-images - photos_new: - url: https://photos-new.hubris.network - backend: mule-photos-new plato: url: https://plato.hubris.network backend: plato @@ -306,14 +303,6 @@ hosts: mounts: - /mnt/library/documents/plato age_pubkey: '' - mule-photos-new: - kind: lxc - pve_id: 127 - host: hubris - os: linux - role: photo-management - lan_ip: 192.168.8.181 - public_host: photos-new.hubris.network zimaos: kind: vm pve_id: 100