diff --git a/containers/120-mule-images.md b/containers/120-mule-images.md index 82ef28a..e60f568 100644 --- a/containers/120-mule-images.md +++ b/containers/120-mule-images.md @@ -19,17 +19,27 @@ Hosts `mule-image` / "mulita" — the photos app at `photos.hubris.network`. Aut | frontend | 3000 | Reverse-proxied by Caddy | | backend | 8001 | FastAPI | | worker-vision | — | ML scan worker | -| worker-light | — | Lightweight worker | -| worker-watcher | — | FS watcher | +| worker-light | — | Lightweight worker (+ celery beat scheduler since 2026-05-11) | | db | (pg) | pgvector | | redis | (rd) | queue | +`worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below. + `.env` is **untracked** — `git checkout .env` will wipe it. Holds: - `PHOTO_DIRS=/mnt/library/images/` - `NEXTCLOUD_USERS_HOST_PATH=/mnt/library/homecloud` - `NEXTCLOUD_BASE_URL=https://cloud.hubris.network` - OIDC client secret + scopes - `SECRET_KEY` (generated) +- `NEXTCLOUD_WEBHOOK_SECRET` (since 2026-05-11; bearer token NC presents on every webhook call to `/api/v1/internal/nc-webhook`) + +## Nextcloud integration (since 2026-05-11) + +Phase 1 + 2 of dropping the duplicated indexing work between mule and Nextcloud — see `~/.claude/projects/-root/memory/project_mule_nc_integration.md` for the full detail. + +**Thumbnails (Phase 1, commit 576b0c2):** `GET /api/v1/photos/{id}/thumb/{size}` proxies `https://cloud.hubris.network/index.php/core/preview` keyed by `photos.nextcloud_fileid` (alembic 0018), authenticated with the photo owner's encrypted NC app password. NC handles JPEG/HEIC/TIFF/RAW/DNG/Movie previews natively. `X-Mule-Thumb-Source: nextcloud|disk` on every response. Worker now only writes the `medium` size to `/data/thumbs/{user}/{photo}/medium.webp` (the vision worker still loads it from disk); `small` + `large` are NC-proxied. Backfill script: `backend/scripts/backfill_nextcloud_fileid.py` — 17,591/17,591 photos filled. + +**File events (Phase 2, commits 362fbc6 + f657e2c):** mule no longer runs `watchfiles`. NC's `webhook_listeners` app (already installed, registered via `backend/scripts/register_nc_webhooks.py`) POSTs every `NodeCreated|NodeWritten|NodeDeleted|NodeRenamed` event to `http://192.168.8.136:8001/api/v1/internal/nc-webhook` with `Authorization: Bearer $NEXTCLOUD_WEBHOOK_SECRET`. The endpoint dispatches the same `scan_folder` / `handle_file_deletion` machinery the watcher used. **NC dispatches webhooks asynchronously via its background-job queue** — `crontab -u www-data` on LXC 114 runs `cron.php` every minute (`*/1`), so end-to-end latency is ≲60 s. Re-running the registrar script is idempotent (it deletes existing webhooks pointing at the same URL first). ## Nextcloud-rooted libraries (since 2026-04-26) @@ -80,6 +90,22 @@ For pushes from inside the LXC, gitea creds at `/etc/mule-deploy/git-credentials ## Changelog +### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2) + +Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files. + +**Phase 1 — thumbnail proxy (commits `576b0c2`, `28738ac`).** Photos table gains `nextcloud_fileid` (migration `0018`). `GET /api/v1/photos/{id}/thumb/{size}` proxies NC's `/index.php/core/preview` keyed by that fileid, authenticated with the owner's encrypted app password (small=240, medium=640, large=1280). Worker now only writes the `medium` size to `/data/thumbs` (the vision worker still loads it from disk); `small` + `large` no longer touch disk. Disk fallback stays wired for legacy rows and the rare 404 from NC (iPhone JPEGs mis-extensioned as `.DNG` — verified). Existing 17,591 photos backfilled via `backend/scripts/backfill_nextcloud_fileid.py`. Tested with real CR2, real DNG, fake DNG, JPEG, HEIC — all green. + +**Range support for `/original` (commit `18dce33`).** Pre-existing bug surfaced by Phase 1 testing: `