120 mule-images: Phase 4 — retire /data/thumbs; SECRET_KEY bug fix

- Vision fetches NC preview at 640px via new sync helper.
- WORKER_THUMB_SIZES = set(); generate_thumbnails still does pHash.
- All medium.webp purged after verification.
- Bug: SECRET_KEY was missing from workers in compose, so
  Fernet decrypt silently failed everywhere outside the backend
  container. Phase 3 had been falling back to ExifTool the whole
  time. Fix replicates SECRET_KEY to all worker services.
This commit is contained in:
claudio
2026-05-11 14:00:50 +02:00
parent 372b856b32
commit bb410eea49

View File

@@ -122,7 +122,14 @@ Verified end-to-end on `Photos/MuleTestFolder/{test1,test2,nested/test3}.jpg`: `
**Phase 3 — Memories-backed `extract_metadata` (commit `2a5759c`).** Memories app re-enabled on NC. `extract_metadata` now tries `GET /index.php/apps/memories/api/image/info/{fileid}` (auth: Basic + `OCS-APIRequest: true` header to bypass CSRF) before falling back to ExifTool. Replaces ~80 ms of subprocess with ~1-2 ms HTTP for ongoing imports. New helpers: `nextcloud_dav.fetch_memories_info_async()` + `metadata._apply_memories_metadata()`. We kept mule's full date-fallback chain (SubSec → DateTimeOriginal → CreateDate → MediaCreateDate → TrackCreateDate → filename heuristic → mtime) because 35% of the library (`taken_at_source='path'`) depends on the filename heuristic, and Memories alone would silently mis-date those photos to mtime. PhotoInfoPanel reads `exif.Make/Model/ISO/FNumber` from `photos.exif_json` — Memories' `exif` blob uses those exact plain key names, so no frontend adapter was needed. ExifTool subprocess is still in place as the fallback for brand-new photos racing the NC scan, non-NC photos, and any NC HTTP failure.
**Remaining open: vision worker still on disk.** It still loads `medium.webp` from `/data/thumbs/{user}/{photo_id}/`. Once vision moves to NC previews (or decodes `/proxy` in-memory), `WORKER_THUMB_SIZES` drops to empty and the local thumb pipeline retires entirely.
**Phase 4 — retire `/data/thumbs` (commits `5a67ed7` + `7a1c6b6`).** Vision worker now fetches NC's 640px preview via a new sync helper `nextcloud_dav.get_preview_bytes()` instead of reading `/data/thumbs/{id}/medium.webp`. `thumbs.WORKER_THUMB_SIZES = set()``generate_thumbnails` still computes pHash on the original-res pixels (perceptual dedup is mule-only) but stops writing files. All ~22k `medium.webp` purged after verification; `/data/thumbs` shrank from 4.1 GB → 94 MB residual.
**SECRET_KEY bug found mid-deploy (commit `7a1c6b6`).** `docker-compose.yml` only set `SECRET_KEY` on the `backend` service, not on the workers. Workers' Fernet-based `decrypt(nextcloud_app_password_enc)` silently returned empty and `_credentials_for()` raised `NextcloudCredentialsMissing`. This meant Phase 3's `extract_metadata` in worker-light had been silently falling back to ExifTool *the entire time*, and Phase 4's vision worker couldn't fetch NC previews at all. Fix replicates `SECRET_KEY=${SECRET_KEY:-...}` to all worker services. After any compose edit, run `docker exec mulita-<svc> env | grep SECRET_KEY` to confirm propagation.
**Caveats worth knowing for Phase 4:**
- Vision is `disabled` in production (`redis GET mulita:flags:vision.enabled == "false"`). The Phase 4 vision-from-NC path is correct but unexercised by live traffic; it'll matter whenever vision is re-enabled.
- NC's preview generator hasn't covered the whole library at the 640px+ tier. A sample of 6 photos: 4 with proper sizes, 1 stuck at 160px (`IMG_4954.DNG`), 1 unfetchable (`IMG_0193.DNG`). If vision re-enables and quality matters, run `pct exec 114 -- sudo -u www-data php occ preview:generate-all` to backfill.
- The HTTP `/thumb/{size}` endpoint's inline-regen disk fallback still writes one WebP per NC-404 event (mis-extensioned RAW etc). `/data/thumbs` will grow back very slowly from that path.
### 2026-05-11 — Duplicates view shows parent folder + full-path tooltip