iPhone .mov files are HEVC Main 10 with codec_tag hvc1. Safari decodes
that fine; Chrome and Firefox refuse 10-bit HEVC entirely, which the
browser surfaces as "playback is not supported" against the existing
/original endpoint. Confirmed against the user's
26-05-01 13-13-26 0525.mov: codec_name=hevc, profile=Main 10,
audio=aac/48kHz.
New endpoint /photos/{id}/playback handles this transparently:
- check the on-disk cache at /data/video-cache/{id}.mp4 first; serve
if newer than the source
- passthrough h264 in mp4/m4v/webm containers (ffprobe to confirm)
- otherwise transcode src -> H.264 8-bit MP4 with libx264 fast/CRF 23,
audio re-encoded to AAC because the iPhone 16 ships APAC audio that
no browser can decode; +faststart for progressive load
- atomic publish via tmp + os.replace so a failed run never leaves a
half-written cache entry
- HTTP Range support so <video> can seek the result
The .mov container is excluded from the passthrough fast path because
Chrome/Firefox refuse to play even h264-in-mov reliably, so .mov always
goes through the cache (transcode-or-remux). /original is refactored
to share the new _serve_file_with_range helper.
Frontend getVideoSrc swaps from /original to /playback. /original
stays for downloads and any non-<video> fetches.
First-hit cost is ~9s wall for a 13s 1080p HEVC clip on this box
(software libx264, 4 cores). Long videos are still sync-in-request
because the browser's <video> can't deal with a 202 response; if that
becomes painful, lift the transcode into a celery task with a polling
endpoint.
The scan_folder resurrect path was unflagging every discarded photo on
every backend boot. start_initial_scan fires scan_all_source_roots on
container start, which fans out scan_folder for every source root,
which walked every file and silently set is_discarded=False on rows
whose file was still on disk -- so every deploy wiped the user's
discard decisions. Today's series of resurrect log lines for
admin/Photos came from that path, not from any actual user re-upload.
Gate the resurrect on os.path.getmtime(file) > discarded_at so the
WebDAV-DELETE-then-re-upload and trashbin-restore-via-PUT-overwrite
flows still trigger (those rewrite the file and bump mtime), but
routine sweeps respect the user's intent. Rows with discarded_at NULL
(legacy) fall through to skipped -- preserve intent over cleanup.
While there: add a Saved toast to the single-photo updateMutation.
The previous patch made cache writes synchronous, which removed the
visible save delay but also removed any signal that the change was
actually persisted. Toast picks a per-field label from the patched
keys (Title updated / Date updated / etc.) and falls back to a count
for multi-field saves.
The Discarded sidebar entry navigates to /photos?is_discarded=true, which
already excludes is_hidden=true rows (the cross-cutting hidden-folder
filter). The /library/stats discarded_count did not, so the badge could
disagree with the actual list — e.g. dtoro saw 1,281 in the badge but
only 25 in the view because the hidden Memories/ source root holds 1,256
discarded rows. Aligning the count with the view, like every other
sidebar badge already does.
The 0525.mov-style export from Synology Photos uses 2-digit years, which
the existing patterns ignored (all required \d{4}). Result: filename
gave no signal, suggestion fell through to the YYYY/MM folder layout and
snapped to day 15. The explicit HH-MM-SS half rules out random digit
triples, so we trust YY → 2000+YY for this specific shape and surface
the actual capture time, not noon.
Last consumer of the on-disk thumbnail pipeline was the vision
worker reading /data/thumbs/{id}/medium.webp. Now it asks Nextcloud
for a 640px preview (the same edge size the old thumb used) and
decodes the bytes in-memory — no disk dependency.
- nextcloud_dav.get_preview_bytes: sync sibling of get_preview_async,
for the celery vision worker (which is sync).
- vision._load_thumb: tries NC preview first; transitional disk
fallback stays for rows still indexed during the rollout.
- thumbs.WORKER_THUMB_SIZES = set() — generate_thumbnails still runs
the decode + pHash side-effect (perceptual dedup is mule-only and
needs original-resolution pixels) but no longer writes thumbnail
files.
The HTTP thumbnail endpoint's disk fallback path stays in place
unchanged: for NC-404 cases (e.g. iPhone JPEGs mis-extensioned as
.DNG), inline Pillow regeneration still writes a tiny per-photo
file so subsequent requests are fast. That path is rare and the
files are small.
Disk impact: /data/thumbs currently has ~22k medium.webp totaling
~1 GB. They'll stop being read after the worker-vision container
restarts, but no automatic delete — purge with the same find
pattern used for small/large reclaim when ready:
find /data/thumbs -name "medium.webp" -delete
handle_directory_rename updated Folder.path but left Folder.name as
the old leaf basename. Path is load-bearing; name is purely display,
but a stale name shows wrong text in the sidebar tree until the next
manual refresh. Now sets folder.name = basename(new_prefix) on the
renamed folder itself; descendants keep their existing names because
the rename was on an ancestor (only their paths shift).
Same correctness as the existing PATCH /folders/{id} endpoint, which
also updates both name and path.
Phase 3 (fat refactor). extract_metadata now tries Memories'
HTTP API GET /index.php/apps/memories/api/image/info/{fileid}
before spawning ExifTool. Replaces ~80–100 ms of subprocess work
with a ~1–2 ms HTTP call for ongoing imports.
What we kept from the ExifTool path:
- Mule's date-fallback chain (SubSec → DateTimeOriginal → CreateDate
→ MediaCreateDate → TrackCreateDate → filename/folder guess → mtime).
Memories' single `datetaken` field falls back to mtime, which would
silently mis-date the 6k+ photos in our library that depend on
filename-encoded dates. _apply_memories_metadata re-applies the
same chain against Memories' `exif` dict.
- taken_at_source='manual' is still sacred — never overwritten.
- has_date_warning recomputed against the resolved taken_at.
Format compat: Memories' `exif` dict uses plain key names (Make,
Model, ISO, FNumber, DateTimeOriginal, GPSLatitude, ...) while the
old ExifTool path stored `EXIF:Make` etc. PhotoInfoPanel only reads
the four keys above and Memories has them in plain form, so the info
panel keeps working without an adapter. Full-text search (ILIKE on
exif_json) still hits camera names, lens names, dates etc. — value
content is identical, only the keys differ.
Fallback paths preserved:
- 404 from Memories (file not yet indexed by NC's scan, brand-new
upload): falls through to ExifTool.
- non-NC photos (no nextcloud_fileid or no app password): ExifTool.
- NC HTTP error or parse failure: ExifTool.
CSRF: Memories' /api/image/info/{id} is CSRF-checked. We send
`OCS-APIRequest: true` to bypass it, the same way the OCS clients
do. Auth is the user's existing Fernet-encrypted app password.
Verified end-to-end against:
- IMG_4954.DNG (real DNG with GPS): width/height/lat/lon/taken_at
match the previous ExifTool output exactly; exif_json switched
to Memories format (Make/Model/ISO/FNumber preserved).
- 20210817_000000_4A6737B6.jpg (path-dated archive photo): taken_at
remained 2021-08-17 from the filename heuristic, source='path'.
The `enabled` state of the Memories app is now required for new
imports to skip ExifTool — left enabled in commit 0a4c8d... (NC
admin action; not in this commit).
The raw-SQL prefix rewrite from f4a03b6 used
`SUBSTRING(filepath FROM LENGTH(:old_prefix) + 1)`. asyncpg's type
inference miscategorises the LENGTH() result and rejects the
parameter as "$2: int (expected str)" at execute time, so every
directory-rename webhook 500'd in production despite the surrounding
logic being correct.
Switch to the same per-row Python loop the existing PATCH
/api/v1/folders/{id} endpoint already uses. Folder renames are rare
and span ≤1k photos typically — the cost of N row UPDATEs is fine.
End-to-end verified:
RenameTestA -> RenameTestA-FromNC (WebDAV MOVE outside mule):
nc-webhook renamed (dir): {photos: 2, folders: 2, source_roots: 0}
DB rows now at -FromNC ✓
-FromNC -> -ViaMule (PATCH /folders/{id} inside mule):
mule rewrites synchronously
webhook fires back ~30s later
nc-webhook renamed (dir): {photos: 0, folders: 0, source_roots: 0}
idempotent no-op against an already-updated DB ✓
NC fires one NodeRenamedEvent on a directory rename — children don't
get their own events. The handler bailed on both paths having no
supported extension. Now:
- New `handle_directory_rename(old, new)` in scan.py does a single
transaction of prefix-rewrites against photos.filepath, folders.path,
and source_roots.path. Cross-source-root case (Photos/x → Memories/x)
is treated as discard-the-old-subtree; scan_folder dispatched by the
subsequent NodeWritten/NodeCreated picks up the new root.
- Webhook renamed branch checks "both source and target are
directories" and calls the helper. File renames keep the existing
delete-old + scan-new-parent path.
Idempotent: the SQL matches zero rows the second time around. That
makes the feedback loop safe — mule's existing PATCH /folders/{id}
endpoint already does a WebDAV MOVE + inline DB rewrite for NC paths,
and the resulting NodeRenamedEvent now flows back through this handler
without re-running the rewrite or leaving rows stale.
Trashbin restore (the documented "NC doesn't emit a subscribed event"
gap) is unchanged.
Two bugs surfaced by the Phase 2 deletion-roundtrip test:
A) Folder delete in NC only fires one NodeDeletedEvent (for the folder
itself, no .jpg suffix). The handler bailed with "unsupported
extension" and photos inside the folder kept is_discarded=false in
mule until the 30-min discard_missing_photos_beat caught up.
Fix: when the deleted path has no supported extension, call new
`handle_directory_deletion()` which UPDATEs every Photo whose
filepath starts with `dirpath + '/'`. Single SQL statement,
idempotent (excludes already-discarded rows so re-deliveries don't
re-stamp discarded_at).
C) PUT-overwrite of a previously-discarded file fired NodeWrittenEvent
→ scan_folder, but scan_folder's "Photo exists by filepath, skip"
branch left is_discarded=true. File was back on disk; mule still
treated it as gone.
Fix: in that branch, if the existing row is discarded, flip
is_discarded=false + clear discarded_at + re-queue extract_metadata
so EXIF / nextcloud_fileid pick up any changes to the bytes.
Together these close the gap for "delete then put back" round-trips
via the NC webhook path. Trashbin-restore (bug B in the test report)
remains an NC-side gap — NC doesn't emit any event mule subscribes to
for restore-from-trash. That stays a TODO.
End-to-end webhook flow is proven on this NC instance (NodeCreated +
NodeWritten both fired and dispatched scan_folder on a PUT test), so
the watchfiles-based polling layer is no longer needed.
- scanner.start_initial_scan no longer queues watch_folders on boot.
- scan.watch_folders kept as a one-line no-op shim so any leftover
apply_async in flight from the previous deploy doesn't crash a
worker. Will be deleted entirely after the queue drains.
- celery.py reroutes watch_folders to the `default` queue (worker-light)
so the no-op shim actually completes — the `watcher` queue is dead.
- docker-compose drops the mulita-worker-watcher service. Its celery
--beat responsibility (firing discard_missing_photos_beat every 30
min) moves to worker-light's command.
Latency note: NC dispatches webhook events through its background-job
queue, currently run by cron */5. After this commit lands you'll want
to tighten cron to */1 so new uploads land in mule within ~60s instead
of up to 5 min.
The watchfiles-based watcher works but duplicates Nextcloud's own
notion of "this file changed." NC has a webhook_listeners app that
can POST file events to an external URL. This adds the mule side of
that handshake.
- POST /api/v1/internal/nc-webhook authenticates a Bearer token
(NEXTCLOUD_WEBHOOK_SECRET, hmac.compare_digest) and dispatches the
same scan_folder / handle_file_deletion machinery the watcher used.
- Handles NodeCreated, NodeWritten, NodeDeleted, NodeRenamed.
Renamed is mapped to delete-old + scan-new-parent. Maps NC's
/admin/files/... path to the bind-mounted /nextcloud-users/admin/files/...
- backend/scripts/register_nc_webhooks.py is the idempotent
registrar: lists existing webhooks, deletes any pointing at the
target URL, then POSTs four fresh ones via OCS.
- Sets the env passthrough on backend + all workers in compose so
the same secret is available wherever the registrar might run.
watch_folders stays in place for now — webhooks become primary, the
watcher is a belt-and-suspenders fallback. Drop the watcher in a
follow-up once webhooks are proven reliable on this NC instance.
`_scan_all_source_roots_async` unconditionally dispatched backfill_gps
30s after each container boot. backfill_gps then queued one
extract_metadata task for every photo where latitude IS NULL — which is
most of the library (screenshots, indoor shots, scans, anything without
GPS in EXIF). The result was ~60k extract_metadata tasks piling onto
the default queue at every deploy, pinning worker-light at 180+% CPU
for ~30 min while it re-derived metadata that wasn't going to change.
The "scanned-before-the-GPS-fix" rationale in the original comment
hasn't applied for many releases. Manual trigger via
POST /api/v1/library/backfill-gps is preserved for the rare case where
the extractor really did change.
`GET /api/v1/photos/{id}/original` returned 200 with the full body for
every request, even ones with a Range header. Browsers refuse to play
<video> they can't seek and surface the failure as "format not
supported" — most visible on .mov / .mp4 over 5–10 MB.
Now parses `Range: bytes=START-END` (and bytes=-N for the tail), emits
206 with Content-Range, streams the slice in 1 MB chunks. Full body
responses advertise Accept-Ranges so the browser knows to retry with a
Range on the next request.
Single-range only — multipart/byteranges is rare in practice and not
worth the complexity.
The offset+limit loop walked the IS NULL set, but every batch's writes
shrank that set, so batch N+1 with offset=N*BATCH skipped over the rows
just filled. A 17k library backfilled only 9k before the loop walked
off the (now-shorter) NULL set.
Replace with a tail-recursive pattern: keep selecting LIMIT BATCH on
the NULL set, tracking rows that won't ever resolve in a `stuck` set so
the loop terminates instead of spinning on them.
mule-image was generating and storing three WebP sizes per photo in
/data/thumbs while Nextcloud already keeps its own previews for the
same source files. Frontend thumbnail requests now proxy NC's
/index.php/core/preview keyed by the photo's Nextcloud fileid,
authenticated with the owner's encrypted app password.
- new column photos.nextcloud_fileid (alembic 0018) plus an index
- get_preview_async + fetch_fileid helpers in nextcloud_dav.py
- thumb route proxies NC primary, falls back to /data/thumbs (legacy
rows / NC unreachable) so a single-file revert restores the old path
- extract_metadata caches the fileid on first run for new photos
- generate_thumbnails now writes only medium since the vision worker
still loads it from disk; small + large drop out of the worker path
- backend/scripts/backfill_nextcloud_fileid.py for one-shot population
of existing rows: docker exec mulita-backend python -m scripts.backfill_nextcloud_fileid
X-Mule-Thumb-Source response header marks each request 'nextcloud' or
'disk' for observability while the rollout settles.
Two copies of IMG_1234.jpg sitting in different folders looked
identical on the duplicates grid — same filename, same dimensions,
same Best heuristic. The user had no way to pick which copy to keep
without opening each in the preview overlay.
Backend: include filepath in the per-member payload from
GET /api/v1/library/duplicates/groups (was filename-only).
Frontend: a black 65% strip at the bottom of every duplicate
thumbnail showing the parent folder name (the actual discriminator
when filenames match), with the full filepath surfaced via the
native title tooltip on hover. The dimensions chip moves from
bottom-left to top-left so the bottom strip can run edge-to-edge.
memberToPhoto stops faking filepath=filename (a years-old workaround
that broke any code path needing the real path); the synthetic Photo
the grid hands to PhotoThumbnail now carries the real filepath.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Title (user_title) field hadn't earned its place in the sidebar
form — the underlying column stays on the model but the editable
row + its draft state + commit handler are gone.
Bulk Notes: a textarea in the multi-photo bulk panel that replaces
user_notes across the whole selection with one string. Apply commits;
Clear empties the draft without committing. New backend bulk action
'set_notes' validates the value is a string (or null/empty to clear)
and writes to every photo in the selection in one go. Wired through
the standard useBulkPhotoMutations optimistic-patch path, so the
photo cache flips immediately and rolls back on error.
user_notes added to the shared Photo type so patchPhotos accepts the
field; previously it was only on PhotoInfoPanel's local PhotoDetails.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /api/v1/photos rejected ?date_from=2026-04-10 with 422 because
pydantic v2's datetime parser doesn't accept date-only strings. The
frontend has been padding date_to with T23:59:59 forever to make the
upper bound inclusive, but date_from went out as a bare YYYY-MM-DD,
so every date-range filter request 422'd and the grid showed nothing.
Frontend: pad date_from with T00:00:00 the same way date_to gets
T23:59:59 — symmetry, and pydantic v2 accepts the full form.
Backend: change date_from/date_to to Optional[str] and parse with
datetime.fromisoformat in the handler. fromisoformat accepts both
bare dates ('2026-04-10' -> midnight) and full ISO strings, so any
older client that still sends a date-only value continues to work.
Tz-aware values get coerced to naive UTC before binding (matches the
taken_at column's shape and the same
fix applied to PATCH /photos/{id} earlier today). Bad input returns
400 with a clear message instead of pydantic's 422.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The taken_at extractor walked four EXIF fields in order: DateTimeOriginal,
CreateDate, MediaCreateDate, ModifyDate. The last one is set every time
a file is re-saved (Lightroom export, EXIF strip, batch resize), so any
photo whose original capture metadata was lost during editing ended up
labeled 'exif' with the *edit* date instead of the shoot date.
Changes:
- SubSecDateTimeOriginal at the top of the list (sub-second precision,
often carries OffsetTime).
- QuickTime:CreateDate added next to MediaCreateDate.
- ModifyDate dropped from the trusted list entirely.
- When no trusted EXIF date is present, fall back to guess_date_from_path
(already used for has_date_warning) and tag taken_at_source='path'.
Better than filesystem mtime, which on Nextcloud-mounted libraries
just reflects the upload time.
- Skip the date-write block entirely if photo.taken_at_source == 'manual'
so a rescan can't clobber a user correction.
- parse_exif_datetime: handle the all-zero placeholder some cameras
emit, accept tz-aware variants (%z), normalize to naive UTC.
Frontend: new 'PATH' badge in TakenAtEditor with a tooltip explaining
the date came from filename / folder rather than real EXIF.
Backfill: new backfill_taken_at celery task and
POST /api/v1/library/maintenance/backfill-taken-at endpoint that
re-enqueues extract_metadata for every non-manual photo. ~21k tasks
finish in ~15 min on the existing worker-light concurrency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix NULLed parent_id only for folders within the
SourceRoot being deleted, but folder rows under a *different*
SourceRoot whose path nests inside this one (e.g. 'Leóns 1st Year' at
.../Taco and Muli - 2024 onward/Leóns 1st Year) still pointed into
our delete set. folders_parent_id_fkey kept tripping. Widen the UPDATE
to NULL parent_id for any folder whose parent_id is in folder_ids,
regardless of source_root_id.
UI: trash button on a Nextcloud library now swaps to a spinning
Loader2 while the delete is in flight (only the row being deleted —
others stay as trash icons but disabled). Title updates to flag
that a cascade through every photo + folder can take a few seconds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PATCH /api/v1/photos/{id} returned 500 with
'can't subtract offset-naive and offset-aware datetimes' when the
frontend sent a tz-aware taken_at value (e.g. 2026-05-09T00:12+02:00).
The photos.taken_at column is timestamp without time zone, so asyncpg
refuses to bind a tz-aware datetime.
The frontend's datetime-local input is supposed to be naive but real-
world locales / browsers / paste flows occasionally include offsets.
Normalize on the server: if tzinfo is present, convert to UTC and drop
the tzinfo so both shapes round-trip cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folders have a self-referential parent_id FK with no ON DELETE rule.
A flat DELETE of the whole subtree trips folders_parent_id_fkey because
postgres checks the constraint per-row regardless of insertion / list
order. Hard-removing 'Taco and Muli - 2024 onward' (35-folder subtree)
returned 500 with ForeignKeyViolationError every attempt.
Fix: UPDATE folders SET parent_id = NULL WHERE id IN (folder_ids) before
the DELETE so the chain is broken cleanly. Same pattern used in
prune_missing_photos for the same constraint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes for the Nextcloud library lifecycle.
1. DELETE /api/v1/nextcloud/source-roots/{id} now actually deletes
the SourceRoot, every Folder under it, and every Photo in those
folders (Nextcloud files untouched). Was a soft-deactivate
(is_active=false) that left the rows around forever, so re-adding
the same path resurrected ghosts and prune-missing reported zero.
Returns {deleted_photos, deleted_folders}; the Settings UI toasts
the count and invalidates photos/folders/stats so cached lists
don't show ghosts. photo_tags and heap_photos already cascade via
ON DELETE CASCADE; FolderShare uses a stringly-typed folder_id
with no FK so cleaned up explicitly.
2. The watcher (watch_folders task) was getting killed every five
minutes by the global task_soft_time_limit=300 in app/tasks/celery.py
despite passing soft_time_limit=None on the decorator (None falls
back to the worker default in this Celery version). Override with
soft_time_limit=0, time_limit=0 (= unlimited) so the watch loop
actually stays alive. The 'Soft time limit (300s) exceeded' /
'Worker exited prematurely' lines should stop in worker-watcher
logs.
3. Added discard_missing_photos() in services/cleanup.py — a soft
variant of prune_missing_photos that walks every present source
root, checks os.path.exists for each non-discarded Photo, and
flips is_discarded=true on the missing ones (UPDATE not DELETE).
Wired as discard_missing_photos_beat in tasks/scan.py and
scheduled every 30 min via celery beat. Beat runs in-process on
worker-watcher (--beat flag in compose) — there's only ever one
watcher and we don't need a separate container.
Hard delete remains manual via prune-missing for users who want to
review before committing. The beat catch-up only soft-discards (file
gone -> mule-image trash, restorable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The default photos list (GET /api/v1/photos?per_page=N&sort=taken_at&order=desc)
filters NOT is_trashed AND NOT is_hidden and sorts by
(taken_at DESC NULLS LAST, id DESC). EXPLAIN on the 21k-row table
shows a seq-scan + top-N heapsort (~20ms standalone, multiplied under
concurrent fan-out on page load). The existing single-column
ix_photos_taken_at can't be used because the leading WHERE clause is
two booleans.
Partial index over the sort key, restricted to the visible subset.
Lets the planner index-scan in reverse and stop at LIMIT N.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
prune_missing_photos previously skipped every photo whose source root
path didn't resolve, on the assumption that a missing path meant the
underlying drive was unmounted (and silently deleting under those
conditions would be data loss). That conflated 'drive unmounted'
with 'user renamed the folder in their file manager'.
A library with 4,154 orphaned photo rows from a since-renamed Nextcloud
folder hit exactly this case: the /nextcloud-users mount was fine, but
the source root path 'Taco and Muli - 2024 onward' no longer existed
because the user had renamed it to 'Photo Archive 2004-2024'. Every
photo under it was reported as skipped_unmounted forever.
Classify source root state as present/renamed/unmounted by checking
whether the immediate parent is readable. 'renamed' is now treated as
prunable; 'unmounted' still skips. Warning messages differ so the user
knows which fix to apply.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LibRaw (rawpy 0.26.1, libraw 0.22.0) rejects Apple ProRAW Linear DNG with
'Unsupported file format or not RAW file'. These files aren't Bayer-pattern
RAW — they're TIFF containers holding an already-developed RGB image, so
PIL opens them directly. iPhone Linear DNG also has no embedded preview
exiftool can extract, so the existing fallback chain ran out of options.
Added PIL Image.open(src_path) as the last fallback in both code paths
(_generate_proxy_webp for /photos/{id}/proxy, and tasks.thumbs.process_raw_image
for thumbnail generation). Covers ~1,300 iPhone DNG files in the library
that were 415-ing on every detail view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without this, a non-admin hitting /api/v1/library/stats would see
every other user's active SourceRoot path in the response (e.g.
muli would see /nextcloud-users/admin/files/Photos). Cross-user
visibility into Nextcloud paths is a small info leak in a multi-user
setup. Admins still get the global list when they pass scope=global.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets each mule-image user (matched via OIDC preferred_username,
overridable in Settings) browse their Nextcloud files/ tree from the
mule-image UI and register subfolders as per-user SourceRoots. Reads
stay direct on the bind-mounted /nextcloud-users path; mutations
(upload, delete, rename, move within NC) dispatch through Nextcloud
WebDAV so oc_filecache, trashbin, comments, and desktop-sync clients
stay coherent.
Backend:
- users.nextcloud_username + nextcloud_app_password_enc (Fernet at rest,
key derived from SECRET_KEY) — alembic 0016
- services/nextcloud_dav.py: minimal WebDAV client (PUT, MKCOL, DELETE,
MOVE) with HTTP Basic auth via the per-user app password
- routers/nextcloud.py: GET /browse, /whoami, GET/POST/DELETE
/source-roots (path-scoped to current_user.nextcloud_username with
realpath traversal guard)
- PATCH /api/v1/auth/me to update nextcloud_username and app password
- OIDC callback defaults nextcloud_username from preferred_username on
first login; backfill on existing users; never overwrites a manual
override
- routers/upload.py: stream upload to NamedTemporaryFile, then PUT to
WebDAV (with MKCOL chain) when destination is NC-rooted; existing
Photo row creation runs unchanged
- routers/discard.py empty-trash: WebDAV DELETE for NC files
- routers/photos.py rename + move: WebDAV MOVE for NC paths;
cross-system move/copy returns a clean error
- routers/folders.py rename + create + permanent-delete: dispatch via
WebDAV when targeting NC-rooted paths
Frontend:
- AuthUser carries nextcloud_username + has_nextcloud_app_password
- services/api.ts: nextcloud + account namespaces
- components/dialogs/NextcloudFolderPicker.tsx: lazy tree browser, name
+ submit -> POST /source-roots
- SettingsDialog: new "Nextcloud library" card with username override +
validate, app-password input, list/remove of NC libraries, and the
picker entry point
docker-compose.yml: NEXTCLOUD_USERS_HOST_PATH bind to /nextcloud-users
on backend + 3 workers; NEXTCLOUD_USERS_ROOT + NEXTCLOUD_BASE_URL env.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds OIDC_LINK_BY_USERNAME as a last-resort linking step after
(issuer, sub) and email both miss. Matches IdP preferred_username
against users.username.
Why: local accounts created before OIDC never collected an email
(no UI for it), so the email fallback cannot relink them. A new
SSO login therefore falls into JIT and creates username-1. On a
single-tenant homelab where the IdP owns the namespace, matching
by username is safe and makes first-time SSO transparent for
pre-existing users. Gated behind a flag so multi-tenant deployments
keep the stricter default.
Adds optional SSO via Authentik (or any OIDC provider) alongside the
existing password flow, and pulls profile images from the provider's
`picture` claim or Gravatar so the sharing UI stops looking anonymous.
Password login stays available as a recovery path; JIT provisioning and
admin-group mapping are env-configurable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shares used to activate instantly on the owner's side with no notice to
the recipient. Introduce a pending/accepted lifecycle so a recipient
gets a bell notification on login and explicitly Accept or Decline
before the shared item lands in their sidebar.
Backend
- Migration 0014 adds `status` + `accepted_at` to heap_shares and
folder_shares; pre-existing rows are backfilled to 'accepted' so
nothing disappears from anyone's current sidebar. One-migration trick:
server_default 'accepted' during add_column, then strip so new inserts
fall through to the Python model default 'pending'.
- New recipient-only endpoints: POST /sharing/{heaps|folders}/{id}/accept
(idempotent) and /decline (hard delete, so re-invites are clean).
- New GET /sharing/pending returning {heaps, folders} of outstanding
invites with target_name + owner_username + permission.
- list_shared_{heaps,folders} now filter to status='accepted' and carry
share_id so the recipient can Leave without a second lookup.
- ShareResponse exposes status so the owner sees pending invites.
Frontend
- NotificationBell lives in the LeftSidebar user row: a Popover
triggered by Bell with a count badge. Each row shows owner avatar,
"{owner} shared {heap|folder} {name}" with a permission subtitle,
and Accept / Decline inline. Polls /sharing/pending every 60s.
- Shared Avatar helper extracted to sharing/Avatar.tsx — used by
ShareDialog, NotificationBell, and the sidebar shared rows so one
user's identity colour is stable everywhere.
- Sidebar shared-row polish: owner avatar bubble + Eye/Pencil
permission icon (was uppercase pill). Right-click opens a context
menu with Open / Leave; Leave calls the existing recipient-revoke
DELETE and invalidates the shared-{heaps,folders} query.
- ShareDialog shows an amber "Invited" pill next to pending recipients.
- New shadcn context-menu primitive (radix dep already installed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Memories view now supports the same keyboard shortcuts, heap membership,
and optimistic cache updates as the Timeline. Arrow/Ctrl+A/Escape nav is
extracted into a shared useGridKeyNav hook so both views stay in lockstep.
Duplicates view is virtualised with @tanstack/react-virtual and has
stabilised PhotoThumbnail props so React.memo actually elides work when
scrolling or toggling selection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the free-text username input with a Select populated from a new
/sharing/users endpoint. Users already on the target's share list are
filtered out, and the trigger surfaces loading / empty states. Matches
the existing permission model since sharing only ever required knowing
a username.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Drops face recognition, OCR, object detection, and semantic embeddings.
The sole remaining vision task is a CLIP-based binary classifier
(photography vs other); photos in "other" get needs_review=true so
screenshots, documents, memes and scans can be triaged from a new
filter pill in the UI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds Redis-backed feature flags for vision stages with admin UI toggles
and manual backfill trigger, photo upload and download routers with
frontend upload modal, and rawpy-based RAW decoding with JPEG fallback
for misnamed DNGs. Fixes pgvector serialization, is_trashed filter, and
naive-datetime bind in incremental duplicate regrouping; bumps Celery
time limits on regroup tasks beyond the 5-minute default.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
HEIC thumbnails:
- Switch fallback from ffmpeg to vips for HEIC files that pillow-heif
rejects. ffmpeg decoded gain map tiles instead of the primary image,
producing inverted/negative thumbnails. vips uses libheif's item
references correctly and extracts the full-resolution primary image.
Database pool exhaustion:
- Add idle_in_transaction_session_timeout=60s so Postgres auto-kills
leaked connections from disconnected thumbnail requests.
- Add pool_timeout=10 so new requests fail fast instead of hanging.
- Bump pool from 5+5 to 10+10 for thumbnail concurrency headroom.
- get_db rolls back on exception before closing.
Auth session persistence:
- Narrow 401 interceptor exclusion to only /auth/refresh and /auth/login
(was excluding all /auth/* including /auth/me, preventing token refresh
on boot).
- fetchMe only clears tokens on 401/403, not network errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
iPhone photos with depth maps or gain maps have too many auxiliary
image references for libheif 1.17, causing pillow-heif to throw
"Too many auxiliary image references". process_heic_image() now
falls back to ffmpeg when pillow-heif fails — ffmpeg's own HEIC
decoder handles these files without issue. Fixes 27/35 HEIC photos
that were stuck in failed state.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sharing:
- New HeapShare and FolderShare models with read/write permissions
- Sharing API router (CRUD for heap and folder shares)
- Heap endpoints accept shared access (photo_ids, add/remove with write)
- Photo list drops user_id filter in shared context, adds owner_username
- Media serving (thumb/original/proxy) falls back to share check on 404
- ShareDialog component for managing shares from kebab menus
- HeapsPanel shows "Shared with me" section for shared heaps
- LeftSidebar shows "Shared with me" section for shared folders
- Owner badge on PhotoThumbnail for photos from other users
Auth:
- Access token default bumped to 1 year, refresh to 10 years
- Refresh token persisted in localStorage (survives page reload)
- Timer-based refresh replaced with 401 axios interceptor
Vision pipeline fixes:
- Bootstrap sets Redis ready key even on partial export failure
- Export functions run conditionally (only for actually missing models)
- _load_thumb handles multi-user path (/data/thumbs/{user_id}/{photo_id}/)
- can_access_photo_via_share uses single subquery instead of N+1 loop
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Addresses 16 robustness, transparency, and performance issues across
the Celery media processing pipeline:
Critical:
- Singleton DB engine in vision tasks (was leaking one per task call)
- acks_late + task_reject_on_worker_lost so crashed workers don't lose tasks
- Global soft/hard time limits (5/10 min) to prevent hung worker slots
- Thumbnail copy-before-resize (in-place mutation degraded larger sizes)
- backfill_vision now checks each task type independently (OCR, faces, etc.)
- Parameterized LIMIT in backfill_vision (was f-string SQL injection)
High:
- try/except + retry(max=3) on all vision inference tasks
- extract_metadata writes processing_error on exiftool failure
- PIL Image handles closed in _load_thumb/_load_original
- Scan progress Redis keys auto-expire after 1 hour
- Watcher lock renewal is wall-clock based (30s) not event-count based
- worker_process_init signal warms up vision models on startup
Medium:
- Explicit task_routes for every task name (wildcards never matched)
- app.services.metadata added to Celery include list
- POST /maintenance/recover-stuck endpoint for photos stuck in processing
- Docker healthchecks for worker-light, worker-vision, and Redis
- Task ID in vision log lines for distributed tracing
- Bare except:pass narrowed to specific exceptions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add active source root directories to the library stats endpoint and
display them in the settings page. Hardcode container PHOTO_DIRS to
/photos since the volume mount handles host path mapping. Add .env to
.gitignore to prevent committing secrets.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Every user — including the initial admin — now gets their own
subdirectory under PHOTO_DIRS (e.g. /photos/admin, /photos/bob).
No one's source root points to the mount root itself, eliminating
cross-user photo overlap entirely.
- Setup endpoint: admin gets /photos/{username} like everyone else
- Migration: default admin media_path set to /photos/admin
- Remove scan directory pruning (no longer needed)
- Fix thumbnail retry URL: use & separator when token query param
already present (was producing ?token=...?retry=N)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the admin's source root is the mount root (/photos) and other
users have subdirectories (/photos/bob), the admin's scan now prunes
those directories from os.walk so photos aren't double-indexed under
the wrong user. The scanner queries all active source roots owned by
other users and excludes their paths during directory traversal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- detect_objects, classify_content, recluster_faces now look up the
photo's user_id and set it on created Tag rows — fixes tags being
invisible to the owning user due to NULL user_id
- Initial admin setup creates source root at the mount root (/photos)
instead of a subdirectory, since the admin owns the entire library
- Revert to OpenCLIP ViT-B/32 (512-d) as default embedder — SigLIP
requires transformers version alignment not yet available in the
Docker image. SigLIP2 code remains for future enablement.
- Add transformers to requirements for future SigLIP support
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move watch_folders to dedicated 'watcher' queue with its own
single-concurrency container so it never blocks scan/thumbnail slots
- Add get_current_user_media dependency that accepts ?token= query
param for <img src> / <video src> media endpoints (thumb, original,
proxy) — fixes 401 on thumbnails
- Append JWT token to all media URLs in the frontend
- Add missing 'memories' case in sidebar navigation switch
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Re-enable the watchfiles-based folder watcher with a Redis lock to
prevent multiple instances from stacking up across restarts. The
watcher is now automatically dispatched on startup when scanner.watch
is true (default), and only one instance runs at a time.
- Redis lock (SETNX + TTL renewal) ensures single-instance execution
- Graceful exit if another watcher holds the lock
- New POST /maintenance/start-watcher endpoint for manual control
- Fix: use settings.scanner/vision properties instead of mulita_config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a Memories view that surfaces photos taken on the current date in
previous years (like Google Photos / Immich). Only uses EXIF-sourced
dates to avoid false matches from filesystem timestamps.
- Backend: GET /api/v1/photos/memories returns groups by year, up to
12 photos each, filtered to non-discarded/non-hidden EXIF dates
- Frontend: MemoriesView with year-grouped thumbnail grid
- Sidebar: new "Memories" nav item with clock icon
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Centralize execution provider selection in providers.py with
auto-detection and graceful fallback. All ONNX sessions (embedder,
detector, face processor, recognizer) now use the configured providers.
- New VISION_EXECUTION_PROVIDERS env var: "auto" for GPU auto-detect,
or explicit "CUDAExecutionProvider,CPUExecutionProvider"
- Provider priority: CUDA > ROCm > OpenVINO > CPU (when set to "auto")
- docker-compose.yml includes commented-out NVIDIA GPU deploy section
- Supports onnxruntime-gpu as a drop-in replacement for onnxruntime
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace OpenCLIP ViT-B/32 (512-d, ~78% recall) with SigLIP2 ViT-B/16
(768-d, ~84% recall) as the default embedding model for significantly
better image-text retrieval quality.
- New SigLIP2Embedder class with 384px input and SigLIP normalization
- ONNX export pipeline for SigLIP2 visual + textual encoders
- Migration 0010: resize embeddings.vector from 512 to 768 dimensions
- Config-driven model selection: "siglip2_vitb16" (default) or
"openclip_vitb32" (legacy) — both models can coexist
- Content classifier follows the configured embedder family
- Existing embeddings cleared on migration; vision backfill regenerates
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace O(N²) pHash-only duplicate detection with a hybrid approach:
- pHash Hamming distance for exact/near-exact copies
- CLIP embedding cosine similarity via pgvector HNSW for visually
similar photos (crops, format changes, screenshots)
Post-scan now uses incremental mode: only newly added photos are
compared against the full library — O(new × log N) via HNSW index
instead of O(N²). Full regroup remains available from Settings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
