Switch the preview overlay from a still <img> to a <video> tag when the
focused photo's Type is "video". Uses PhotoPrism's /api/v1/videos/:hash
endpoint with the existing previewToken, falls back to a still thumb as
the poster, and autoplays muted so the controls reveal without
clobbering whatever else is on the page.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the legacy mule-image backend with PhotoPrism plus a thin
SvelteKit client and a Node sidecar for endpoints PhotoPrism doesn't
expose (file rename), and add a two-phase migrator (metadata via PUT,
heaps → albums) for the existing Postgres library.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tsc --noEmit caught seven TS6133 "declared but never used" + one
TS2614 "no exported member 'features'" left over from a27267f. Strip
Brain/RotateCcw icons, the unused Switch + Loader2 imports, the
adminApi + featuresApi + features module references, and the
SETTINGS_FEATURE_FLAGS_KEY constant. No runtime change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
- backend: app/services/vision/, app/tasks/vision.py,
app/services/feature_flags.py, app/routers/features.py — all
deleted; admin AI/feature-flag endpoints and the worker-vision
bootstrap call gone. Photo.needs_review and its index dropped.
- frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
types, "Needs Review" sidebar entry + filter, needs_review filter
URL param all gone.
- infra: worker-vision compose service + models_data volume deleted;
worker-light command no longer runs bootstrap_models; the db
image switches from pgvector/pgvector:pg16 to postgres:16; backend
Dockerfile drops the dedicated torch RUN layer; requirements.txt
drops torch/torchvision/open-clip-torch/onnxruntime.
Alembic 0019_drop_ai_remnants:
- drops photos.needs_review + ix_photos_needs_review
- DROP EXTENSION IF EXISTS vector (must run before the image swap;
the new postgres:16 doesn't ship pgvector)
New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits torch+torchvision into its own RUN layer so edits to
requirements.txt don't invalidate the ~200MB CPU-only torch download.
Adds buildkit cache mounts on both pip-install layers so even when a
layer is invalidated (or buildkit evicts it) the wheel is reused from
the on-disk pip cache instead of refetching from download.pytorch.org.
Triggered by two consecutive deploy failures where pytorch.org timed
out mid-download (2026-05-13 ~21:41 and possibly ~22:47).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- drop the redundant isLoadingMore boolean from the background-loading
store; derive from inFlight > 0 in the selector
- pull _active_source_root_paths out of get_library_stats and reuse it
from get_duplicate_groups (same is_active + admin-scope check, now
in one place)
- drop p.rstrip('/') in the duplicates folder-scope filter (SourceRoot
paths are never written with a trailing slash)
- match Timeline's initial-load affordance to the new bottom indicator
(Loader2 spinner + ellipsis instead of plain "Loading photos...")
- trim narrative comments that explained what the surrounding code does
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- duplicates: restrict the /library/duplicates/groups query to photos whose
folder path actually lives under an active SourceRoot in the user's
settings. Nextcloud's "move to trash" flow was leaving .delete/purge-1
Folder rows wired to the original source_root_id, leaking those entries
into the Duplicates view as ghost paths that the user never opted into.
- discard: add a spinner to the "Delete N" and "Empty discard pile"
buttons (and their confirm dialogs) while the destructive mutation is
in flight, so the user gets immediate feedback for a slow operation.
- timeline: render a bottom-of-grid "Loading more photos…" indicator
while usePhotosQuery's background cursor loop is still pulling pages.
Backed by a tiny Zustand store the loop drives via a balanced
start/stop (counter, not boolean, so rapid filter changes can't flip
the flag false while a fresh loop is alive).
- remove client-side upload UI + /upload endpoint. Nextcloud is the
authoritative ingress now; the duplicate path created confusion and
the backend route is gone too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The amber AlertTriangle in the bottom-right of every photo with
has_date_warning set was more visual noise than help — the filter pill's
"Date issues" option still surfaces the same photos when the user
actually wants to triage them. Keeps the BR corner uncluttered.
Backend field + filter pill option unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The search box on the right edge of the filter bar wasn't pulling its
weight — kills it entirely along with the supporting plumbing:
- FilterBar: remove input + Search icon import + local/debounced state
- filterStore: drop `q`, `setQ`, plus all references in INITIAL_FILTERS,
filtersToParams, hasActiveFilters, snapshotFilters
- usePhotosQuery: stop passing q through filtersToParams
- useFilterUrlSync: drop the `q` URL param read/write
- PhotoThumbnail + PreviewView: remove the search-match banner/chip and
findSearchMatch helper imports
- Timeline + MemoriesView: stop subscribing to / forwarding the prop
- useKeyboardShortcuts: drop the `/` and Cmd+F focus hotkeys
- KeyboardHints: drop the `/` hint and the now-stale `?` collision note
- delete hooks/useSearchQuery.ts (no callers) and lib/searchMatch.ts
Backend /photos/search endpoint left untouched — no UI reaches it now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a "Size" pill in the FilterBar with 5 presets (XS/S/M/L/XL, 96–272px,
default M=160) that drives the cell size in the Timeline, Memories, and
Duplicates grids. Preference persists in localStorage. Preview filmstrip
is intentionally untouched — it's a fixed-track nav rail, not a grid.
Centralised in a new viewSettingsStore so every grid reads from the same
source. Duplicates' virtualizer is poked on size change so row heights
and the keyboard nav's column count stay in sync.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five stacked optimisations for the thumbnail hot path so the timeline
grid lands in fewer round trips and fewer bytes.
1. PhotoThumbnail: switch from 'medium' (640px) to 'small' (240px) for
grid cells. 240px oversamples 150-200px logical cells on 2x retina
and drops payload 5-8x. Lightbox and preview filmstrip keep 'large'
and 'medium' respectively.
2. nextcloud_dav: pool the httpx client. A module-level AsyncClient
with HTTP/2 + keepalive (max_connections=64, keepalive_expiry=120s)
replaces the per-request constructor that paid a fresh TCP+TLS
handshake on every preview fetch. Auth is per-user so it stays at
the call site via auth=BasicAuth(...). Lifespan-managed: init in
main.py's lifespan startup, aclose on shutdown. requirements.txt
gains the http2 extra to pull in h2 (not currently installed).
Same change applies to fetch_memories_info_async since it hits the
same host.
3. PhotoThumbnail img: add decoding="async" so JPEG/WebP decode moves
off the main thread, plus fetchPriority="low" so grid backfill
doesn't fight UI fetches.
4. Eager-load Photo.user via joinedload from the thumb handler.
_get_photo_with_share_fallback gains an options parameter so other
callers stay zero-overhead; only the thumb handler asks for the
owner join. Eliminates the second SELECT users per request.
5. Disk-fallback path picks up Cache-Control: private, max-age=86400
in both the FileResponse and X-Accel branches so re-renders match
the NC primary path's caching behaviour.
Net: a warm grid page should drop from ~200-400 ms median per thumb to
well under 100 ms; payload drops ~5-8x; backend sustains higher
concurrency with fewer sockets to Nextcloud and one fewer Postgres
round-trip per request.
The /playback transcode cache lives at /data/video-cache. That
directory was created in container-local storage (the mkdir at
services/video.py import time), not on a shared volume — so the
worker pretranscode populated its own ephemeral copy while the
backend served from a different empty one, and the cache evaporated
on every container restart.
Add a video_cache_data named volume mounted on backend, worker-light,
and worker-vision so the pretranscoded files actually reach the
serving path and survive deploys.
Long videos blocked /playback for the entire encode duration. The fix
is to populate the cache before the user clicks, not when they click.
Changes:
- Extract ffprobe + ffmpeg helpers to services/video.py so the request
handler and the background task share one sync implementation. The
endpoint wraps calls in asyncio.to_thread; celery just calls them.
- New tasks/video.py with pretranscode_video. Idempotent: skips when
the cache is already current and skips passthrough-safe sources
(h264 in mp4/m4v/webm). 30-min task time limit so the long-tail
files (3GP archive, multi-minute 1080p clips) still complete.
- scan_folder now dispatches pretranscode_video alongside
generate_thumbnails / extract_metadata for any new video row.
- POST /library/maintenance/backfill-video-cache enqueues every
active video so the existing library catches up.
- libx264 preset bumped from fast to veryfast. ~2x throughput on this
CPU-only box, output a few % larger but well within disk budget.
- /playback simplifies to: cache check, passthrough if h264 in
web-safe container, else sync transcode (still there as fallback
for races against the queued task).
Once the backfill task drains, /playback should be near-instant for
every video. Any video added afterwards is pre-transcoded at scan
time, so the user keeps that property going forward.
Transcode wrote to {id}.mp4.tmp for atomic publish, but ffmpeg picks
the muxer from the output filename and rejected .tmp with Unable to
choose an output format. Add -f mp4 so the temp name is just storage
scratch, not a format hint.
iPhone .mov files are HEVC Main 10 with codec_tag hvc1. Safari decodes
that fine; Chrome and Firefox refuse 10-bit HEVC entirely, which the
browser surfaces as "playback is not supported" against the existing
/original endpoint. Confirmed against the user's
26-05-01 13-13-26 0525.mov: codec_name=hevc, profile=Main 10,
audio=aac/48kHz.
New endpoint /photos/{id}/playback handles this transparently:
- check the on-disk cache at /data/video-cache/{id}.mp4 first; serve
if newer than the source
- passthrough h264 in mp4/m4v/webm containers (ffprobe to confirm)
- otherwise transcode src -> H.264 8-bit MP4 with libx264 fast/CRF 23,
audio re-encoded to AAC because the iPhone 16 ships APAC audio that
no browser can decode; +faststart for progressive load
- atomic publish via tmp + os.replace so a failed run never leaves a
half-written cache entry
- HTTP Range support so <video> can seek the result
The .mov container is excluded from the passthrough fast path because
Chrome/Firefox refuse to play even h264-in-mov reliably, so .mov always
goes through the cache (transcode-or-remux). /original is refactored
to share the new _serve_file_with_range helper.
Frontend getVideoSrc swaps from /original to /playback. /original
stays for downloads and any non-<video> fetches.
First-hit cost is ~9s wall for a 13s 1080p HEVC clip on this box
(software libx264, 4 cores). Long videos are still sync-in-request
because the browser's <video> can't deal with a 202 response; if that
becomes painful, lift the transcode into a celery task with a polling
endpoint.
The scan_folder resurrect path was unflagging every discarded photo on
every backend boot. start_initial_scan fires scan_all_source_roots on
container start, which fans out scan_folder for every source root,
which walked every file and silently set is_discarded=False on rows
whose file was still on disk -- so every deploy wiped the user's
discard decisions. Today's series of resurrect log lines for
admin/Photos came from that path, not from any actual user re-upload.
Gate the resurrect on os.path.getmtime(file) > discarded_at so the
WebDAV-DELETE-then-re-upload and trashbin-restore-via-PUT-overwrite
flows still trigger (those rewrite the file and bump mtime), but
routine sweeps respect the user's intent. Rows with discarded_at NULL
(legacy) fall through to skipped -- preserve intent over cleanup.
While there: add a Saved toast to the single-photo updateMutation.
The previous patch made cache writes synchronous, which removed the
visible save delay but also removed any signal that the change was
actually persisted. Toast picks a per-field label from the patched
keys (Title updated / Date updated / etc.) and falls back to a count
for multi-field saves.
Single-photo updateMutation only invalidated, so the panel waited for
a follow-up GET /photos/{id} round-trip before showing the new value —
felt as a 200–500 ms lag after every taken_at / rating / notes edit.
Use the PATCH response (already the updated row) to merge into the
per-photo cache and patch every cached timeline list in place.
Bulk taken_at had the same shape: invalidate-only, no optimistic. When
the user dropped back from N selected to one of the modified photos
the panel briefly showed the pre-edit value. Move both bulkSetTakenAt
and bulkSetTakenAtMap into useBulkPhotoMutations alongside the rating/
color/notes pattern, with the same snapshot+patch+rollback primitives.
Tags + bulk tags still invalidate-only — separate change if needed.
The Discarded sidebar entry navigates to /photos?is_discarded=true, which
already excludes is_hidden=true rows (the cross-cutting hidden-folder
filter). The /library/stats discarded_count did not, so the badge could
disagree with the actual list — e.g. dtoro saw 1,281 in the badge but
only 25 in the view because the hidden Memories/ source root holds 1,256
discarded rows. Aligning the count with the view, like every other
sidebar badge already does.
The 0525.mov-style export from Synology Photos uses 2-digit years, which
the existing patterns ignored (all required \d{4}). Result: filename
gave no signal, suggestion fell through to the YYYY/MM folder layout and
snapped to day 15. The explicit HH-MM-SS half rules out random digit
triples, so we trust YY → 2000+YY for this specific shape and surface
the actual capture time, not noon.
The workers couldn't decrypt users.nextcloud_app_password_enc because
SECRET_KEY wasn't in their env. _credentials_for() then raised
NextcloudCredentialsMissing and our code swallowed it as "no NC
auth → fall back to local path."
Surfaced on the Phase 4 deploy when /data/thumbs/.../medium.webp
was purged and the vision worker had no disk fallback left. NC
preview fetch then returned None, the classifier got no image,
and the photo failed to classify.
Also masked Phase 3 silently — extract_metadata in worker-light
was falling back to ExifTool every time instead of hitting Memories
(which would have been fine because ExifTool produces the same
fields, but slower and unnecessary). With SECRET_KEY available, the
Memories primary path actually fires.
Last consumer of the on-disk thumbnail pipeline was the vision
worker reading /data/thumbs/{id}/medium.webp. Now it asks Nextcloud
for a 640px preview (the same edge size the old thumb used) and
decodes the bytes in-memory — no disk dependency.
- nextcloud_dav.get_preview_bytes: sync sibling of get_preview_async,
for the celery vision worker (which is sync).
- vision._load_thumb: tries NC preview first; transitional disk
fallback stays for rows still indexed during the rollout.
- thumbs.WORKER_THUMB_SIZES = set() — generate_thumbnails still runs
the decode + pHash side-effect (perceptual dedup is mule-only and
needs original-resolution pixels) but no longer writes thumbnail
files.
The HTTP thumbnail endpoint's disk fallback path stays in place
unchanged: for NC-404 cases (e.g. iPhone JPEGs mis-extensioned as
.DNG), inline Pillow regeneration still writes a tiny per-photo
file so subsequent requests are fast. That path is rare and the
files are small.
Disk impact: /data/thumbs currently has ~22k medium.webp totaling
~1 GB. They'll stop being read after the worker-vision container
restarts, but no automatic delete — purge with the same find
pattern used for small/large reclaim when ready:
find /data/thumbs -name "medium.webp" -delete
handle_directory_rename updated Folder.path but left Folder.name as
the old leaf basename. Path is load-bearing; name is purely display,
but a stale name shows wrong text in the sidebar tree until the next
manual refresh. Now sets folder.name = basename(new_prefix) on the
renamed folder itself; descendants keep their existing names because
the rename was on an ancestor (only their paths shift).
Same correctness as the existing PATCH /folders/{id} endpoint, which
also updates both name and path.
Phase 3 (fat refactor). extract_metadata now tries Memories'
HTTP API GET /index.php/apps/memories/api/image/info/{fileid}
before spawning ExifTool. Replaces ~80–100 ms of subprocess work
with a ~1–2 ms HTTP call for ongoing imports.
What we kept from the ExifTool path:
- Mule's date-fallback chain (SubSec → DateTimeOriginal → CreateDate
→ MediaCreateDate → TrackCreateDate → filename/folder guess → mtime).
Memories' single `datetaken` field falls back to mtime, which would
silently mis-date the 6k+ photos in our library that depend on
filename-encoded dates. _apply_memories_metadata re-applies the
same chain against Memories' `exif` dict.
- taken_at_source='manual' is still sacred — never overwritten.
- has_date_warning recomputed against the resolved taken_at.
Format compat: Memories' `exif` dict uses plain key names (Make,
Model, ISO, FNumber, DateTimeOriginal, GPSLatitude, ...) while the
old ExifTool path stored `EXIF:Make` etc. PhotoInfoPanel only reads
the four keys above and Memories has them in plain form, so the info
panel keeps working without an adapter. Full-text search (ILIKE on
exif_json) still hits camera names, lens names, dates etc. — value
content is identical, only the keys differ.
Fallback paths preserved:
- 404 from Memories (file not yet indexed by NC's scan, brand-new
upload): falls through to ExifTool.
- non-NC photos (no nextcloud_fileid or no app password): ExifTool.
- NC HTTP error or parse failure: ExifTool.
CSRF: Memories' /api/image/info/{id} is CSRF-checked. We send
`OCS-APIRequest: true` to bypass it, the same way the OCS clients
do. Auth is the user's existing Fernet-encrypted app password.
Verified end-to-end against:
- IMG_4954.DNG (real DNG with GPS): width/height/lat/lon/taken_at
match the previous ExifTool output exactly; exif_json switched
to Memories format (Make/Model/ISO/FNumber preserved).
- 20210817_000000_4A6737B6.jpg (path-dated archive photo): taken_at
remained 2021-08-17 from the filename heuristic, source='path'.
The `enabled` state of the Memories app is now required for new
imports to skip ExifTool — left enabled in commit 0a4c8d... (NC
admin action; not in this commit).
The raw-SQL prefix rewrite from f4a03b6 used
`SUBSTRING(filepath FROM LENGTH(:old_prefix) + 1)`. asyncpg's type
inference miscategorises the LENGTH() result and rejects the
parameter as "$2: int (expected str)" at execute time, so every
directory-rename webhook 500'd in production despite the surrounding
logic being correct.
Switch to the same per-row Python loop the existing PATCH
/api/v1/folders/{id} endpoint already uses. Folder renames are rare
and span ≤1k photos typically — the cost of N row UPDATEs is fine.
End-to-end verified:
RenameTestA -> RenameTestA-FromNC (WebDAV MOVE outside mule):
nc-webhook renamed (dir): {photos: 2, folders: 2, source_roots: 0}
DB rows now at -FromNC ✓
-FromNC -> -ViaMule (PATCH /folders/{id} inside mule):
mule rewrites synchronously
webhook fires back ~30s later
nc-webhook renamed (dir): {photos: 0, folders: 0, source_roots: 0}
idempotent no-op against an already-updated DB ✓
NC fires one NodeRenamedEvent on a directory rename — children don't
get their own events. The handler bailed on both paths having no
supported extension. Now:
- New `handle_directory_rename(old, new)` in scan.py does a single
transaction of prefix-rewrites against photos.filepath, folders.path,
and source_roots.path. Cross-source-root case (Photos/x → Memories/x)
is treated as discard-the-old-subtree; scan_folder dispatched by the
subsequent NodeWritten/NodeCreated picks up the new root.
- Webhook renamed branch checks "both source and target are
directories" and calls the helper. File renames keep the existing
delete-old + scan-new-parent path.
Idempotent: the SQL matches zero rows the second time around. That
makes the feedback loop safe — mule's existing PATCH /folders/{id}
endpoint already does a WebDAV MOVE + inline DB rewrite for NC paths,
and the resulting NodeRenamedEvent now flows back through this handler
without re-running the rewrite or leaving rows stale.
Trashbin restore (the documented "NC doesn't emit a subscribed event"
gap) is unchanged.
Two bugs surfaced by the Phase 2 deletion-roundtrip test:
A) Folder delete in NC only fires one NodeDeletedEvent (for the folder
itself, no .jpg suffix). The handler bailed with "unsupported
extension" and photos inside the folder kept is_discarded=false in
mule until the 30-min discard_missing_photos_beat caught up.
Fix: when the deleted path has no supported extension, call new
`handle_directory_deletion()` which UPDATEs every Photo whose
filepath starts with `dirpath + '/'`. Single SQL statement,
idempotent (excludes already-discarded rows so re-deliveries don't
re-stamp discarded_at).
C) PUT-overwrite of a previously-discarded file fired NodeWrittenEvent
→ scan_folder, but scan_folder's "Photo exists by filepath, skip"
branch left is_discarded=true. File was back on disk; mule still
treated it as gone.
Fix: in that branch, if the existing row is discarded, flip
is_discarded=false + clear discarded_at + re-queue extract_metadata
so EXIF / nextcloud_fileid pick up any changes to the bytes.
Together these close the gap for "delete then put back" round-trips
via the NC webhook path. Trashbin-restore (bug B in the test report)
remains an NC-side gap — NC doesn't emit any event mule subscribes to
for restore-from-trash. That stays a TODO.
End-to-end webhook flow is proven on this NC instance (NodeCreated +
NodeWritten both fired and dispatched scan_folder on a PUT test), so
the watchfiles-based polling layer is no longer needed.
- scanner.start_initial_scan no longer queues watch_folders on boot.
- scan.watch_folders kept as a one-line no-op shim so any leftover
apply_async in flight from the previous deploy doesn't crash a
worker. Will be deleted entirely after the queue drains.
- celery.py reroutes watch_folders to the `default` queue (worker-light)
so the no-op shim actually completes — the `watcher` queue is dead.
- docker-compose drops the mulita-worker-watcher service. Its celery
--beat responsibility (firing discard_missing_photos_beat every 30
min) moves to worker-light's command.
Latency note: NC dispatches webhook events through its background-job
queue, currently run by cron */5. After this commit lands you'll want
to tighten cron to */1 so new uploads land in mule within ~60s instead
of up to 5 min.
The watchfiles-based watcher works but duplicates Nextcloud's own
notion of "this file changed." NC has a webhook_listeners app that
can POST file events to an external URL. This adds the mule side of
that handshake.
- POST /api/v1/internal/nc-webhook authenticates a Bearer token
(NEXTCLOUD_WEBHOOK_SECRET, hmac.compare_digest) and dispatches the
same scan_folder / handle_file_deletion machinery the watcher used.
- Handles NodeCreated, NodeWritten, NodeDeleted, NodeRenamed.
Renamed is mapped to delete-old + scan-new-parent. Maps NC's
/admin/files/... path to the bind-mounted /nextcloud-users/admin/files/...
- backend/scripts/register_nc_webhooks.py is the idempotent
registrar: lists existing webhooks, deletes any pointing at the
target URL, then POSTs four fresh ones via OCS.
- Sets the env passthrough on backend + all workers in compose so
the same secret is available wherever the registrar might run.
watch_folders stays in place for now — webhooks become primary, the
watcher is a belt-and-suspenders fallback. Drop the watcher in a
follow-up once webhooks are proven reliable on this NC instance.
`_scan_all_source_roots_async` unconditionally dispatched backfill_gps
30s after each container boot. backfill_gps then queued one
extract_metadata task for every photo where latitude IS NULL — which is
most of the library (screenshots, indoor shots, scans, anything without
GPS in EXIF). The result was ~60k extract_metadata tasks piling onto
the default queue at every deploy, pinning worker-light at 180+% CPU
for ~30 min while it re-derived metadata that wasn't going to change.
The "scanned-before-the-GPS-fix" rationale in the original comment
hasn't applied for many releases. Manual trigger via
POST /api/v1/library/backfill-gps is preserved for the rare case where
the extractor really did change.
`GET /api/v1/photos/{id}/original` returned 200 with the full body for
every request, even ones with a Range header. Browsers refuse to play
<video> they can't seek and surface the failure as "format not
supported" — most visible on .mov / .mp4 over 5–10 MB.
Now parses `Range: bytes=START-END` (and bytes=-N for the tail), emits
206 with Content-Range, streams the slice in 1 MB chunks. Full body
responses advertise Accept-Ranges so the browser knows to retry with a
Range on the next request.
Single-range only — multipart/byteranges is rare in practice and not
worth the complexity.
The offset+limit loop walked the IS NULL set, but every batch's writes
shrank that set, so batch N+1 with offset=N*BATCH skipped over the rows
just filled. A 17k library backfilled only 9k before the loop walked
off the (now-shorter) NULL set.
Replace with a tail-recursive pattern: keep selecting LIMIT BATCH on
the NULL set, tracking rows that won't ever resolve in a `stuck` set so
the loop terminates instead of spinning on them.
mule-image was generating and storing three WebP sizes per photo in
/data/thumbs while Nextcloud already keeps its own previews for the
same source files. Frontend thumbnail requests now proxy NC's
/index.php/core/preview keyed by the photo's Nextcloud fileid,
authenticated with the owner's encrypted app password.
- new column photos.nextcloud_fileid (alembic 0018) plus an index
- get_preview_async + fetch_fileid helpers in nextcloud_dav.py
- thumb route proxies NC primary, falls back to /data/thumbs (legacy
rows / NC unreachable) so a single-file revert restores the old path
- extract_metadata caches the fileid on first run for new photos
- generate_thumbnails now writes only medium since the vision worker
still loads it from disk; small + large drop out of the worker path
- backend/scripts/backfill_nextcloud_fileid.py for one-shot population
of existing rows: docker exec mulita-backend python -m scripts.backfill_nextcloud_fileid
X-Mule-Thumb-Source response header marks each request 'nextcloud' or
'disk' for observability while the rollout settles.
Rapid arrow-nav was firing one /proxy fetch per photo with no way to
abort (new Image() has no abort). Holding the right arrow through ten
photos in two seconds left ten multi-MB transfers in flight competing
for bandwidth and the RAW/HEIC transcoder. Now the preload only kicks
in if the user lingers on a photo for 400ms; otherwise the timer is
cleared and no /proxy request is made.
Arrow nav inside preview only updates activePhotoId; selectedPhotos still
points at whatever was selected in the grid before opening preview. X and
S therefore fired against the wrong photo — the toast appeared but the
filmstrip tint for the currently-viewed photo never changed because that
photo was not the cull target.
cullTargets() (and togglePickOnSelection, now sharing it) now prefer
activePhotoId when viewMode === preview.
The /proxy endpoint is slow on first hit, especially for RAW/HEIC where it
transcodes synchronously. Preview now renders the pre-generated large thumb
immediately, then preloads /proxy via Image() and swaps src when ready, so
zoom (Z key / wheel) still reaches the original pixels.
When two duplicates live in folders with the same parent name (e.g.
matching '2023' subfolders under different archives), showing only
the parent gave both thumbnails the same label. Walk one level up:
the path strip now renders '…/<grandparent>/<parent>' so the user
can always tell two copies apart at a glance. Filename still
surfaces via the title tooltip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two copies of IMG_1234.jpg sitting in different folders looked
identical on the duplicates grid — same filename, same dimensions,
same Best heuristic. The user had no way to pick which copy to keep
without opening each in the preview overlay.
Backend: include filepath in the per-member payload from
GET /api/v1/library/duplicates/groups (was filename-only).
Frontend: a black 65% strip at the bottom of every duplicate
thumbnail showing the parent folder name (the actual discriminator
when filenames match), with the full filepath surfaced via the
native title tooltip on hover. The dimensions chip moves from
bottom-left to top-left so the bottom strip can run edge-to-edge.
memberToPhoto stops faking filepath=filename (a years-old workaround
that broke any code path needing the real path); the synthetic Photo
the grid hands to PhotoThumbnail now carries the real filepath.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Title (user_title) field hadn't earned its place in the sidebar
form — the underlying column stays on the model but the editable
row + its draft state + commit handler are gone.
Bulk Notes: a textarea in the multi-photo bulk panel that replaces
user_notes across the whole selection with one string. Apply commits;
Clear empties the draft without committing. New backend bulk action
'set_notes' validates the value is a string (or null/empty to clear)
and writes to every photo in the selection in one go. Wired through
the standard useBulkPhotoMutations optimistic-patch path, so the
photo cache flips immediately and rolls back on error.
user_notes added to the shared Photo type so patchPhotos accepts the
field; previously it was only on PhotoInfoPanel's local PhotoDetails.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The right sidebar had three top-level blocks (ActiveHeapCard + Header
title strip + scroll region with two parallel collapsibles 'Edit' and
'Metadata'). Three nested Section sub-collapsibles inside Metadata
added another row of chevrons per group. A lot of chrome for what is
fundamentally one form per photo.
Refactor:
- RightSidebar: remove ActiveHeapCard import + both usages
(empty-selection branch and single-photo branch). Single-photo
branch also drops the redundant Header strip; the new Metadata
collapsible's trigger IS the visible section title. Multi-photo
branch keeps Header (still needs 'N Photos Selected').
- PhotoInfoPanel: collapse the Edit and Metadata-with-sub-Sections
structure into two flat collapsibles. Metadata holds readonly facts
(Size / Dimensions grid, Path, GPS inlined) and the editable form
(Filename, Title, Date Taken, Notes, Tags, Rating + Color on one
row, Flag), separated by a thin horizontal rule. Camera lives in
its own collapsible at the bottom so a long EXIF block can't crowd
the form. Default expanded set narrows to ['metadata', 'camera'].
- Compact density: Notes rows=3 -> rows=2, rating + color share a
row, stars/swatches shrink h-5/w-5 -> h-4/w-4, space-y-2.5 -> 2,
Flag buttons text-sm -> text-xs, grid gaps tightened. The empty
'No GPS data' chip is hidden when there are no coordinates rather
than rendered as an empty row.
- Drop the unused local Section helper and the now-orphan
ActiveHeapCard.tsx file. Active-heap state stays in the store; the
Select / Discard buttons inside the form still consult activeHeap
on click.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Clicking a folder in the sidebar (or any section change) didn't
reset the timeline's scroll position. If the user was scrolled deep
into All Photos, the new folder loaded at the same y-offset, often
landing on empty space below the last row.
The section-change effect already cleared selection and reset the
auto-focus guard; just needed to also reset parentRef.current.scrollTop.
Synchronous so the first paint of the new section is anchored at
photo[0]; the auto-focus selectPhoto call still runs after render
to highlight the first photo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /api/v1/photos rejected ?date_from=2026-04-10 with 422 because
pydantic v2's datetime parser doesn't accept date-only strings. The
frontend has been padding date_to with T23:59:59 forever to make the
upper bound inclusive, but date_from went out as a bare YYYY-MM-DD,
so every date-range filter request 422'd and the grid showed nothing.
Frontend: pad date_from with T00:00:00 the same way date_to gets
T23:59:59 — symmetry, and pydantic v2 accepts the full form.
Backend: change date_from/date_to to Optional[str] and parse with
datetime.fromisoformat in the handler. fromisoformat accepts both
bare dates ('2026-04-10' -> midnight) and full ISO strings, so any
older client that still sends a date-only value continues to work.
Tz-aware values get coerced to naive UTC before binding (matches the
taken_at column's shape and the same
fix applied to PATCH /photos/{id} earlier today). Bad input returns
400 with a clear message instead of pydantic's 422.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The taken_at extractor walked four EXIF fields in order: DateTimeOriginal,
CreateDate, MediaCreateDate, ModifyDate. The last one is set every time
a file is re-saved (Lightroom export, EXIF strip, batch resize), so any
photo whose original capture metadata was lost during editing ended up
labeled 'exif' with the *edit* date instead of the shoot date.
Changes:
- SubSecDateTimeOriginal at the top of the list (sub-second precision,
often carries OffsetTime).
- QuickTime:CreateDate added next to MediaCreateDate.
- ModifyDate dropped from the trusted list entirely.
- When no trusted EXIF date is present, fall back to guess_date_from_path
(already used for has_date_warning) and tag taken_at_source='path'.
Better than filesystem mtime, which on Nextcloud-mounted libraries
just reflects the upload time.
- Skip the date-write block entirely if photo.taken_at_source == 'manual'
so a rescan can't clobber a user correction.
- parse_exif_datetime: handle the all-zero placeholder some cameras
emit, accept tz-aware variants (%z), normalize to naive UTC.
Frontend: new 'PATH' badge in TakenAtEditor with a tooltip explaining
the date came from filename / folder rather than real EXIF.
Backfill: new backfill_taken_at celery task and
POST /api/v1/library/maintenance/backfill-taken-at endpoint that
re-enqueues extract_metadata for every non-manual photo. ~21k tasks
finish in ~15 min on the existing worker-light concurrency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix NULLed parent_id only for folders within the
SourceRoot being deleted, but folder rows under a *different*
SourceRoot whose path nests inside this one (e.g. 'Leóns 1st Year' at
.../Taco and Muli - 2024 onward/Leóns 1st Year) still pointed into
our delete set. folders_parent_id_fkey kept tripping. Widen the UPDATE
to NULL parent_id for any folder whose parent_id is in folder_ids,
regardless of source_root_id.
UI: trash button on a Nextcloud library now swaps to a spinning
Loader2 while the delete is in flight (only the row being deleted —
others stay as trash icons but disabled). Title updates to flag
that a cascade through every photo + folder can take a few seconds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PATCH /api/v1/photos/{id} returned 500 with
'can't subtract offset-naive and offset-aware datetimes' when the
frontend sent a tz-aware taken_at value (e.g. 2026-05-09T00:12+02:00).
The photos.taken_at column is timestamp without time zone, so asyncpg
refuses to bind a tz-aware datetime.
The frontend's datetime-local input is supposed to be naive but real-
world locales / browsers / paste flows occasionally include offsets.
Normalize on the server: if tzinfo is present, convert to UTC and drop
the tzinfo so both shapes round-trip cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folders have a self-referential parent_id FK with no ON DELETE rule.
A flat DELETE of the whole subtree trips folders_parent_id_fkey because
postgres checks the constraint per-row regardless of insertion / list
order. Hard-removing 'Taco and Muli - 2024 onward' (35-folder subtree)
returned 500 with ForeignKeyViolationError every attempt.
Fix: UPDATE folders SET parent_id = NULL WHERE id IN (folder_ids) before
the DELETE so the chain is broken cleanly. Same pattern used in
prune_missing_photos for the same constraint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes for the Nextcloud library lifecycle.
1. DELETE /api/v1/nextcloud/source-roots/{id} now actually deletes
the SourceRoot, every Folder under it, and every Photo in those
folders (Nextcloud files untouched). Was a soft-deactivate
(is_active=false) that left the rows around forever, so re-adding
the same path resurrected ghosts and prune-missing reported zero.
Returns {deleted_photos, deleted_folders}; the Settings UI toasts
the count and invalidates photos/folders/stats so cached lists
don't show ghosts. photo_tags and heap_photos already cascade via
ON DELETE CASCADE; FolderShare uses a stringly-typed folder_id
with no FK so cleaned up explicitly.
2. The watcher (watch_folders task) was getting killed every five
minutes by the global task_soft_time_limit=300 in app/tasks/celery.py
despite passing soft_time_limit=None on the decorator (None falls
back to the worker default in this Celery version). Override with
soft_time_limit=0, time_limit=0 (= unlimited) so the watch loop
actually stays alive. The 'Soft time limit (300s) exceeded' /
'Worker exited prematurely' lines should stop in worker-watcher
logs.
3. Added discard_missing_photos() in services/cleanup.py — a soft
variant of prune_missing_photos that walks every present source
root, checks os.path.exists for each non-discarded Photo, and
flips is_discarded=true on the missing ones (UPDATE not DELETE).
Wired as discard_missing_photos_beat in tasks/scan.py and
scheduled every 30 min via celery beat. Beat runs in-process on
worker-watcher (--beat flag in compose) — there's only ever one
watcher and we don't need a separate container.
Hard delete remains manual via prune-missing for users who want to
review before committing. The beat catch-up only soft-discards (file
gone -> mule-image trash, restorable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Even when the user has a live Authentik session, hitting
photos.hubris.network used to drop them on the LoginPage with a 'Sign
in with Authentik' button they had to click manually. With OIDC set
up for a single trusted IdP that's friction with no upside.
LoginPage now reads /auth/config on mount and, if OIDC is enabled,
immediately navigates to the OIDC login URL. Authentik recognizes
the existing session and bounces the browser back through the
callback signed in — no clicks needed.
Two escape hatches so the user is never stuck:
- ?password=1 in the URL forces the password form
- sessionStorage 'skipAutoSso' flag, set by the logout flow and by
the OIDC callback's error branch, suppresses the next auto-redirect
so logouts actually log out and OIDC failures surface their error
instead of looping straight back to the IdP
While the redirect is in flight we show 'Signing in with Authentik...'
plus a small 'Use password instead' link, so users on a slow or
broken IdP connection aren't left staring at a spinner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Metadata collapsible was hosting two editable widgets (TagsEditor
and TakenAtEditor) buried inside the readonly sub-sections — Tags as
its own Section, taken-at wedged into Basic Info between size/dims
and the filepath. With both top-level collapsibles in place, the
clearer split is editable up top, readonly below.
Moved into the Edit collapsible (in identification → description →
categorization order):
Filename, Title, Date Taken, Notes, Tags, Rating, Color, Flag
Metadata now holds only readonly sub-sections:
Basic Info (size, dims, path), Camera, Location
Dropped the now-empty Tags Section from Metadata and the 'tags' key
from the default-expanded set.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
