Phase 3 (fat refactor). extract_metadata now tries Memories'
HTTP API GET /index.php/apps/memories/api/image/info/{fileid}
before spawning ExifTool. Replaces ~80–100 ms of subprocess work
with a ~1–2 ms HTTP call for ongoing imports.
What we kept from the ExifTool path:
- Mule's date-fallback chain (SubSec → DateTimeOriginal → CreateDate
→ MediaCreateDate → TrackCreateDate → filename/folder guess → mtime).
Memories' single `datetaken` field falls back to mtime, which would
silently mis-date the 6k+ photos in our library that depend on
filename-encoded dates. _apply_memories_metadata re-applies the
same chain against Memories' `exif` dict.
- taken_at_source='manual' is still sacred — never overwritten.
- has_date_warning recomputed against the resolved taken_at.
Format compat: Memories' `exif` dict uses plain key names (Make,
Model, ISO, FNumber, DateTimeOriginal, GPSLatitude, ...) while the
old ExifTool path stored `EXIF:Make` etc. PhotoInfoPanel only reads
the four keys above and Memories has them in plain form, so the info
panel keeps working without an adapter. Full-text search (ILIKE on
exif_json) still hits camera names, lens names, dates etc. — value
content is identical, only the keys differ.
Fallback paths preserved:
- 404 from Memories (file not yet indexed by NC's scan, brand-new
upload): falls through to ExifTool.
- non-NC photos (no nextcloud_fileid or no app password): ExifTool.
- NC HTTP error or parse failure: ExifTool.
CSRF: Memories' /api/image/info/{id} is CSRF-checked. We send
`OCS-APIRequest: true` to bypass it, the same way the OCS clients
do. Auth is the user's existing Fernet-encrypted app password.
Verified end-to-end against:
- IMG_4954.DNG (real DNG with GPS): width/height/lat/lon/taken_at
match the previous ExifTool output exactly; exif_json switched
to Memories format (Make/Model/ISO/FNumber preserved).
- 20210817_000000_4A6737B6.jpg (path-dated archive photo): taken_at
remained 2021-08-17 from the filename heuristic, source='path'.
The `enabled` state of the Memories app is now required for new
imports to skip ExifTool — left enabled in commit 0a4c8d... (NC
admin action; not in this commit).
The raw-SQL prefix rewrite from f4a03b6 used
`SUBSTRING(filepath FROM LENGTH(:old_prefix) + 1)`. asyncpg's type
inference miscategorises the LENGTH() result and rejects the
parameter as "$2: int (expected str)" at execute time, so every
directory-rename webhook 500'd in production despite the surrounding
logic being correct.
Switch to the same per-row Python loop the existing PATCH
/api/v1/folders/{id} endpoint already uses. Folder renames are rare
and span ≤1k photos typically — the cost of N row UPDATEs is fine.
End-to-end verified:
RenameTestA -> RenameTestA-FromNC (WebDAV MOVE outside mule):
nc-webhook renamed (dir): {photos: 2, folders: 2, source_roots: 0}
DB rows now at -FromNC ✓
-FromNC -> -ViaMule (PATCH /folders/{id} inside mule):
mule rewrites synchronously
webhook fires back ~30s later
nc-webhook renamed (dir): {photos: 0, folders: 0, source_roots: 0}
idempotent no-op against an already-updated DB ✓
NC fires one NodeRenamedEvent on a directory rename — children don't
get their own events. The handler bailed on both paths having no
supported extension. Now:
- New `handle_directory_rename(old, new)` in scan.py does a single
transaction of prefix-rewrites against photos.filepath, folders.path,
and source_roots.path. Cross-source-root case (Photos/x → Memories/x)
is treated as discard-the-old-subtree; scan_folder dispatched by the
subsequent NodeWritten/NodeCreated picks up the new root.
- Webhook renamed branch checks "both source and target are
directories" and calls the helper. File renames keep the existing
delete-old + scan-new-parent path.
Idempotent: the SQL matches zero rows the second time around. That
makes the feedback loop safe — mule's existing PATCH /folders/{id}
endpoint already does a WebDAV MOVE + inline DB rewrite for NC paths,
and the resulting NodeRenamedEvent now flows back through this handler
without re-running the rewrite or leaving rows stale.
Trashbin restore (the documented "NC doesn't emit a subscribed event"
gap) is unchanged.
Two bugs surfaced by the Phase 2 deletion-roundtrip test:
A) Folder delete in NC only fires one NodeDeletedEvent (for the folder
itself, no .jpg suffix). The handler bailed with "unsupported
extension" and photos inside the folder kept is_discarded=false in
mule until the 30-min discard_missing_photos_beat caught up.
Fix: when the deleted path has no supported extension, call new
`handle_directory_deletion()` which UPDATEs every Photo whose
filepath starts with `dirpath + '/'`. Single SQL statement,
idempotent (excludes already-discarded rows so re-deliveries don't
re-stamp discarded_at).
C) PUT-overwrite of a previously-discarded file fired NodeWrittenEvent
→ scan_folder, but scan_folder's "Photo exists by filepath, skip"
branch left is_discarded=true. File was back on disk; mule still
treated it as gone.
Fix: in that branch, if the existing row is discarded, flip
is_discarded=false + clear discarded_at + re-queue extract_metadata
so EXIF / nextcloud_fileid pick up any changes to the bytes.
Together these close the gap for "delete then put back" round-trips
via the NC webhook path. Trashbin-restore (bug B in the test report)
remains an NC-side gap — NC doesn't emit any event mule subscribes to
for restore-from-trash. That stays a TODO.
End-to-end webhook flow is proven on this NC instance (NodeCreated +
NodeWritten both fired and dispatched scan_folder on a PUT test), so
the watchfiles-based polling layer is no longer needed.
- scanner.start_initial_scan no longer queues watch_folders on boot.
- scan.watch_folders kept as a one-line no-op shim so any leftover
apply_async in flight from the previous deploy doesn't crash a
worker. Will be deleted entirely after the queue drains.
- celery.py reroutes watch_folders to the `default` queue (worker-light)
so the no-op shim actually completes — the `watcher` queue is dead.
- docker-compose drops the mulita-worker-watcher service. Its celery
--beat responsibility (firing discard_missing_photos_beat every 30
min) moves to worker-light's command.
Latency note: NC dispatches webhook events through its background-job
queue, currently run by cron */5. After this commit lands you'll want
to tighten cron to */1 so new uploads land in mule within ~60s instead
of up to 5 min.
The watchfiles-based watcher works but duplicates Nextcloud's own
notion of "this file changed." NC has a webhook_listeners app that
can POST file events to an external URL. This adds the mule side of
that handshake.
- POST /api/v1/internal/nc-webhook authenticates a Bearer token
(NEXTCLOUD_WEBHOOK_SECRET, hmac.compare_digest) and dispatches the
same scan_folder / handle_file_deletion machinery the watcher used.
- Handles NodeCreated, NodeWritten, NodeDeleted, NodeRenamed.
Renamed is mapped to delete-old + scan-new-parent. Maps NC's
/admin/files/... path to the bind-mounted /nextcloud-users/admin/files/...
- backend/scripts/register_nc_webhooks.py is the idempotent
registrar: lists existing webhooks, deletes any pointing at the
target URL, then POSTs four fresh ones via OCS.
- Sets the env passthrough on backend + all workers in compose so
the same secret is available wherever the registrar might run.
watch_folders stays in place for now — webhooks become primary, the
watcher is a belt-and-suspenders fallback. Drop the watcher in a
follow-up once webhooks are proven reliable on this NC instance.
`_scan_all_source_roots_async` unconditionally dispatched backfill_gps
30s after each container boot. backfill_gps then queued one
extract_metadata task for every photo where latitude IS NULL — which is
most of the library (screenshots, indoor shots, scans, anything without
GPS in EXIF). The result was ~60k extract_metadata tasks piling onto
the default queue at every deploy, pinning worker-light at 180+% CPU
for ~30 min while it re-derived metadata that wasn't going to change.
The "scanned-before-the-GPS-fix" rationale in the original comment
hasn't applied for many releases. Manual trigger via
POST /api/v1/library/backfill-gps is preserved for the rare case where
the extractor really did change.
`GET /api/v1/photos/{id}/original` returned 200 with the full body for
every request, even ones with a Range header. Browsers refuse to play
<video> they can't seek and surface the failure as "format not
supported" — most visible on .mov / .mp4 over 5–10 MB.
Now parses `Range: bytes=START-END` (and bytes=-N for the tail), emits
206 with Content-Range, streams the slice in 1 MB chunks. Full body
responses advertise Accept-Ranges so the browser knows to retry with a
Range on the next request.
Single-range only — multipart/byteranges is rare in practice and not
worth the complexity.
The offset+limit loop walked the IS NULL set, but every batch's writes
shrank that set, so batch N+1 with offset=N*BATCH skipped over the rows
just filled. A 17k library backfilled only 9k before the loop walked
off the (now-shorter) NULL set.
Replace with a tail-recursive pattern: keep selecting LIMIT BATCH on
the NULL set, tracking rows that won't ever resolve in a `stuck` set so
the loop terminates instead of spinning on them.
mule-image was generating and storing three WebP sizes per photo in
/data/thumbs while Nextcloud already keeps its own previews for the
same source files. Frontend thumbnail requests now proxy NC's
/index.php/core/preview keyed by the photo's Nextcloud fileid,
authenticated with the owner's encrypted app password.
- new column photos.nextcloud_fileid (alembic 0018) plus an index
- get_preview_async + fetch_fileid helpers in nextcloud_dav.py
- thumb route proxies NC primary, falls back to /data/thumbs (legacy
rows / NC unreachable) so a single-file revert restores the old path
- extract_metadata caches the fileid on first run for new photos
- generate_thumbnails now writes only medium since the vision worker
still loads it from disk; small + large drop out of the worker path
- backend/scripts/backfill_nextcloud_fileid.py for one-shot population
of existing rows: docker exec mulita-backend python -m scripts.backfill_nextcloud_fileid
X-Mule-Thumb-Source response header marks each request 'nextcloud' or
'disk' for observability while the rollout settles.
Rapid arrow-nav was firing one /proxy fetch per photo with no way to
abort (new Image() has no abort). Holding the right arrow through ten
photos in two seconds left ten multi-MB transfers in flight competing
for bandwidth and the RAW/HEIC transcoder. Now the preload only kicks
in if the user lingers on a photo for 400ms; otherwise the timer is
cleared and no /proxy request is made.
Arrow nav inside preview only updates activePhotoId; selectedPhotos still
points at whatever was selected in the grid before opening preview. X and
S therefore fired against the wrong photo — the toast appeared but the
filmstrip tint for the currently-viewed photo never changed because that
photo was not the cull target.
cullTargets() (and togglePickOnSelection, now sharing it) now prefer
activePhotoId when viewMode === preview.
The /proxy endpoint is slow on first hit, especially for RAW/HEIC where it
transcodes synchronously. Preview now renders the pre-generated large thumb
immediately, then preloads /proxy via Image() and swaps src when ready, so
zoom (Z key / wheel) still reaches the original pixels.
When two duplicates live in folders with the same parent name (e.g.
matching '2023' subfolders under different archives), showing only
the parent gave both thumbnails the same label. Walk one level up:
the path strip now renders '…/<grandparent>/<parent>' so the user
can always tell two copies apart at a glance. Filename still
surfaces via the title tooltip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two copies of IMG_1234.jpg sitting in different folders looked
identical on the duplicates grid — same filename, same dimensions,
same Best heuristic. The user had no way to pick which copy to keep
without opening each in the preview overlay.
Backend: include filepath in the per-member payload from
GET /api/v1/library/duplicates/groups (was filename-only).
Frontend: a black 65% strip at the bottom of every duplicate
thumbnail showing the parent folder name (the actual discriminator
when filenames match), with the full filepath surfaced via the
native title tooltip on hover. The dimensions chip moves from
bottom-left to top-left so the bottom strip can run edge-to-edge.
memberToPhoto stops faking filepath=filename (a years-old workaround
that broke any code path needing the real path); the synthetic Photo
the grid hands to PhotoThumbnail now carries the real filepath.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Title (user_title) field hadn't earned its place in the sidebar
form — the underlying column stays on the model but the editable
row + its draft state + commit handler are gone.
Bulk Notes: a textarea in the multi-photo bulk panel that replaces
user_notes across the whole selection with one string. Apply commits;
Clear empties the draft without committing. New backend bulk action
'set_notes' validates the value is a string (or null/empty to clear)
and writes to every photo in the selection in one go. Wired through
the standard useBulkPhotoMutations optimistic-patch path, so the
photo cache flips immediately and rolls back on error.
user_notes added to the shared Photo type so patchPhotos accepts the
field; previously it was only on PhotoInfoPanel's local PhotoDetails.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The right sidebar had three top-level blocks (ActiveHeapCard + Header
title strip + scroll region with two parallel collapsibles 'Edit' and
'Metadata'). Three nested Section sub-collapsibles inside Metadata
added another row of chevrons per group. A lot of chrome for what is
fundamentally one form per photo.
Refactor:
- RightSidebar: remove ActiveHeapCard import + both usages
(empty-selection branch and single-photo branch). Single-photo
branch also drops the redundant Header strip; the new Metadata
collapsible's trigger IS the visible section title. Multi-photo
branch keeps Header (still needs 'N Photos Selected').
- PhotoInfoPanel: collapse the Edit and Metadata-with-sub-Sections
structure into two flat collapsibles. Metadata holds readonly facts
(Size / Dimensions grid, Path, GPS inlined) and the editable form
(Filename, Title, Date Taken, Notes, Tags, Rating + Color on one
row, Flag), separated by a thin horizontal rule. Camera lives in
its own collapsible at the bottom so a long EXIF block can't crowd
the form. Default expanded set narrows to ['metadata', 'camera'].
- Compact density: Notes rows=3 -> rows=2, rating + color share a
row, stars/swatches shrink h-5/w-5 -> h-4/w-4, space-y-2.5 -> 2,
Flag buttons text-sm -> text-xs, grid gaps tightened. The empty
'No GPS data' chip is hidden when there are no coordinates rather
than rendered as an empty row.
- Drop the unused local Section helper and the now-orphan
ActiveHeapCard.tsx file. Active-heap state stays in the store; the
Select / Discard buttons inside the form still consult activeHeap
on click.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Clicking a folder in the sidebar (or any section change) didn't
reset the timeline's scroll position. If the user was scrolled deep
into All Photos, the new folder loaded at the same y-offset, often
landing on empty space below the last row.
The section-change effect already cleared selection and reset the
auto-focus guard; just needed to also reset parentRef.current.scrollTop.
Synchronous so the first paint of the new section is anchored at
photo[0]; the auto-focus selectPhoto call still runs after render
to highlight the first photo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /api/v1/photos rejected ?date_from=2026-04-10 with 422 because
pydantic v2's datetime parser doesn't accept date-only strings. The
frontend has been padding date_to with T23:59:59 forever to make the
upper bound inclusive, but date_from went out as a bare YYYY-MM-DD,
so every date-range filter request 422'd and the grid showed nothing.
Frontend: pad date_from with T00:00:00 the same way date_to gets
T23:59:59 — symmetry, and pydantic v2 accepts the full form.
Backend: change date_from/date_to to Optional[str] and parse with
datetime.fromisoformat in the handler. fromisoformat accepts both
bare dates ('2026-04-10' -> midnight) and full ISO strings, so any
older client that still sends a date-only value continues to work.
Tz-aware values get coerced to naive UTC before binding (matches the
taken_at column's shape and the same
fix applied to PATCH /photos/{id} earlier today). Bad input returns
400 with a clear message instead of pydantic's 422.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The taken_at extractor walked four EXIF fields in order: DateTimeOriginal,
CreateDate, MediaCreateDate, ModifyDate. The last one is set every time
a file is re-saved (Lightroom export, EXIF strip, batch resize), so any
photo whose original capture metadata was lost during editing ended up
labeled 'exif' with the *edit* date instead of the shoot date.
Changes:
- SubSecDateTimeOriginal at the top of the list (sub-second precision,
often carries OffsetTime).
- QuickTime:CreateDate added next to MediaCreateDate.
- ModifyDate dropped from the trusted list entirely.
- When no trusted EXIF date is present, fall back to guess_date_from_path
(already used for has_date_warning) and tag taken_at_source='path'.
Better than filesystem mtime, which on Nextcloud-mounted libraries
just reflects the upload time.
- Skip the date-write block entirely if photo.taken_at_source == 'manual'
so a rescan can't clobber a user correction.
- parse_exif_datetime: handle the all-zero placeholder some cameras
emit, accept tz-aware variants (%z), normalize to naive UTC.
Frontend: new 'PATH' badge in TakenAtEditor with a tooltip explaining
the date came from filename / folder rather than real EXIF.
Backfill: new backfill_taken_at celery task and
POST /api/v1/library/maintenance/backfill-taken-at endpoint that
re-enqueues extract_metadata for every non-manual photo. ~21k tasks
finish in ~15 min on the existing worker-light concurrency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix NULLed parent_id only for folders within the
SourceRoot being deleted, but folder rows under a *different*
SourceRoot whose path nests inside this one (e.g. 'Leóns 1st Year' at
.../Taco and Muli - 2024 onward/Leóns 1st Year) still pointed into
our delete set. folders_parent_id_fkey kept tripping. Widen the UPDATE
to NULL parent_id for any folder whose parent_id is in folder_ids,
regardless of source_root_id.
UI: trash button on a Nextcloud library now swaps to a spinning
Loader2 while the delete is in flight (only the row being deleted —
others stay as trash icons but disabled). Title updates to flag
that a cascade through every photo + folder can take a few seconds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PATCH /api/v1/photos/{id} returned 500 with
'can't subtract offset-naive and offset-aware datetimes' when the
frontend sent a tz-aware taken_at value (e.g. 2026-05-09T00:12+02:00).
The photos.taken_at column is timestamp without time zone, so asyncpg
refuses to bind a tz-aware datetime.
The frontend's datetime-local input is supposed to be naive but real-
world locales / browsers / paste flows occasionally include offsets.
Normalize on the server: if tzinfo is present, convert to UTC and drop
the tzinfo so both shapes round-trip cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folders have a self-referential parent_id FK with no ON DELETE rule.
A flat DELETE of the whole subtree trips folders_parent_id_fkey because
postgres checks the constraint per-row regardless of insertion / list
order. Hard-removing 'Taco and Muli - 2024 onward' (35-folder subtree)
returned 500 with ForeignKeyViolationError every attempt.
Fix: UPDATE folders SET parent_id = NULL WHERE id IN (folder_ids) before
the DELETE so the chain is broken cleanly. Same pattern used in
prune_missing_photos for the same constraint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes for the Nextcloud library lifecycle.
1. DELETE /api/v1/nextcloud/source-roots/{id} now actually deletes
the SourceRoot, every Folder under it, and every Photo in those
folders (Nextcloud files untouched). Was a soft-deactivate
(is_active=false) that left the rows around forever, so re-adding
the same path resurrected ghosts and prune-missing reported zero.
Returns {deleted_photos, deleted_folders}; the Settings UI toasts
the count and invalidates photos/folders/stats so cached lists
don't show ghosts. photo_tags and heap_photos already cascade via
ON DELETE CASCADE; FolderShare uses a stringly-typed folder_id
with no FK so cleaned up explicitly.
2. The watcher (watch_folders task) was getting killed every five
minutes by the global task_soft_time_limit=300 in app/tasks/celery.py
despite passing soft_time_limit=None on the decorator (None falls
back to the worker default in this Celery version). Override with
soft_time_limit=0, time_limit=0 (= unlimited) so the watch loop
actually stays alive. The 'Soft time limit (300s) exceeded' /
'Worker exited prematurely' lines should stop in worker-watcher
logs.
3. Added discard_missing_photos() in services/cleanup.py — a soft
variant of prune_missing_photos that walks every present source
root, checks os.path.exists for each non-discarded Photo, and
flips is_discarded=true on the missing ones (UPDATE not DELETE).
Wired as discard_missing_photos_beat in tasks/scan.py and
scheduled every 30 min via celery beat. Beat runs in-process on
worker-watcher (--beat flag in compose) — there's only ever one
watcher and we don't need a separate container.
Hard delete remains manual via prune-missing for users who want to
review before committing. The beat catch-up only soft-discards (file
gone -> mule-image trash, restorable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Even when the user has a live Authentik session, hitting
photos.hubris.network used to drop them on the LoginPage with a 'Sign
in with Authentik' button they had to click manually. With OIDC set
up for a single trusted IdP that's friction with no upside.
LoginPage now reads /auth/config on mount and, if OIDC is enabled,
immediately navigates to the OIDC login URL. Authentik recognizes
the existing session and bounces the browser back through the
callback signed in — no clicks needed.
Two escape hatches so the user is never stuck:
- ?password=1 in the URL forces the password form
- sessionStorage 'skipAutoSso' flag, set by the logout flow and by
the OIDC callback's error branch, suppresses the next auto-redirect
so logouts actually log out and OIDC failures surface their error
instead of looping straight back to the IdP
While the redirect is in flight we show 'Signing in with Authentik...'
plus a small 'Use password instead' link, so users on a slow or
broken IdP connection aren't left staring at a spinner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Metadata collapsible was hosting two editable widgets (TagsEditor
and TakenAtEditor) buried inside the readonly sub-sections — Tags as
its own Section, taken-at wedged into Basic Info between size/dims
and the filepath. With both top-level collapsibles in place, the
clearer split is editable up top, readonly below.
Moved into the Edit collapsible (in identification → description →
categorization order):
Filename, Title, Date Taken, Notes, Tags, Rating, Color, Flag
Metadata now holds only readonly sub-sections:
Basic Info (size, dims, path), Camera, Location
Dropped the now-empty Tags Section from Metadata and the 'tags' key
from the default-expanded set.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror the Metadata collapsible: an 'Edit' wrapper around filename,
title, notes, rating, color, and flag so the editable form is hidden
with one click. Default expanded.
Drop the clear-selection X from the panel Header — Esc still clears
selection and grid clicks do too. The X felt out of place once the
panel restructured around two equal collapsible groups (Edit /
Metadata) below a plain title bar.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The right sidepanel had three stacked flex regions (heap card +
header + PhotoInfoPanel) with PhotoInfoPanel maintaining its own
internal scroll. That made the editable fields (filename, title,
notes, rating, color, flag) stick at the top — separate from the
readonly metadata that scrolled below. Effectively two scroll
boundaries on one sidebar.
Move the scroll boundary up so only ActiveHeapCard + Header stay
pinned; editable fields and readonly metadata now scroll together.
Wrap the four readonly sections (Tags / Basic Info / Camera /
Location) in a single outer 'Metadata' collapsible so the user can
hide the whole block with one click. Sub-sections inside stay
individually collapsible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The default photos list (GET /api/v1/photos?per_page=N&sort=taken_at&order=desc)
filters NOT is_trashed AND NOT is_hidden and sorts by
(taken_at DESC NULLS LAST, id DESC). EXPLAIN on the 21k-row table
shows a seq-scan + top-N heapsort (~20ms standalone, multiplied under
concurrent fan-out on page load). The existing single-column
ix_photos_taken_at can't be used because the leading WHERE clause is
two booleans.
Partial index over the sort key, restricted to the visible subset.
Lets the planner index-scan in reverse and stop at LIMIT N.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Photos grid was fetching per_page=500 on the very first request, which
serialized hundreds of thumbnail requests behind a single sort+payload.
Split into PER_PAGE_INITIAL=100 (one viewport, fast paint) and
PER_PAGE_BACKGROUND=500 (subsequent prefetch pages, fewer round-trips).
Idle polling for scan-status and worker-status was set to 10s / 15s
respectively. With nothing queued the typical session was firing 4–6
status requests every minute through the single uvicorn event loop on
top of everything else. Bumped both to 30s. While actively scanning /
processing the 2s / 3s cadence is unchanged — that's where the user
actually wants live updates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
prune_missing_photos previously skipped every photo whose source root
path didn't resolve, on the assumption that a missing path meant the
underlying drive was unmounted (and silently deleting under those
conditions would be data loss). That conflated 'drive unmounted'
with 'user renamed the folder in their file manager'.
A library with 4,154 orphaned photo rows from a since-renamed Nextcloud
folder hit exactly this case: the /nextcloud-users mount was fine, but
the source root path 'Taco and Muli - 2024 onward' no longer existed
because the user had renamed it to 'Photo Archive 2004-2024'. Every
photo under it was reported as skipped_unmounted forever.
Classify source root state as present/renamed/unmounted by checking
whether the immediate parent is readable. 'renamed' is now treated as
prunable; 'unmounted' still skips. Warning messages differ so the user
knows which fix to apply.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LibRaw (rawpy 0.26.1, libraw 0.22.0) rejects Apple ProRAW Linear DNG with
'Unsupported file format or not RAW file'. These files aren't Bayer-pattern
RAW — they're TIFF containers holding an already-developed RGB image, so
PIL opens them directly. iPhone Linear DNG also has no embedded preview
exiftool can extract, so the existing fallback chain ran out of options.
Added PIL Image.open(src_path) as the last fallback in both code paths
(_generate_proxy_webp for /photos/{id}/proxy, and tasks.thumbs.process_raw_image
for thumbnail generation). Covers ~1,300 iPhone DNG files in the library
that were 415-ing on every detail view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Production runs were on the dev --reload single-worker config. The frontend
fans out ~15 parallel API calls on first paint (folders/tree, tags, heaps,
sharing/*, stats, photos, worker-status, scan/status); they all serialized
on one event loop and felt slow. Switch to 2 workers without --reload for
real concurrency. --proxy-headers preserved client IPs through nginx.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two response interceptors were stomping on each other:
1. api.ts (this file, registered at module import) — on 401, set
original._retry = true, removed access_token from localStorage,
and rejected. The comment claimed it relied on a "scheduled
refresh in AuthContext" that does not exist in the codebase.
2. AuthContext useEffect — proper refresh: POST /auth/refresh, swap
both tokens, retry the original request.
Axios runs response interceptors in registration order, so api.ts ran
first and pre-emptively burned the _retry flag + access_token before
AuthContext could see the 401. Result: every expired-token request
forced a re-login instead of a silent refresh.
Drop api.ts's response interceptor entirely. AuthContext owns the
refresh dance; the request interceptor here just attaches the bearer.
Companion bump in .env (gitignored): ACCESS_TOKEN_EXPIRE_MINUTES=10080
(7 days), REFRESH_TOKEN_EXPIRE_DAYS=365 — homelab posture, fewer
refresh round-trips per session even when the silent refresh works.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to 4c7e981 — the SettingsPage was opened to non-admins but
the LeftSidebar still gated the entry button on isAdmin, so non-admins
had no way to reach it. The page itself is the source of truth for
which tabs and controls are visible per role.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without this, a non-admin hitting /api/v1/library/stats would see
every other user's active SourceRoot path in the response (e.g.
muli would see /nextcloud-users/admin/files/Photos). Cross-user
visibility into Nextcloud paths is a small info leak in a multi-user
setup. Admins still get the global list when they pass scope=global.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Non-admins now see Library and AI Features tabs with data scoped to
themselves; only Users (admin management) stays admin-only.
Library tab: queries pass scope=global only when isAdmin, otherwise
omit scope so the backend _owner_filter falls back to current_user.
Stats, worker status, pipeline progress, duplicates, regenerate-thumbs
all respect this. Re-scan + maintenance buttons that hit user-scoped
endpoints continue to work for non-admins.
AI Features tab: feature flag state read via the public /features
endpoint for non-admins (just effective values, no override metadata),
admin-only flag toggle Switches show as disabled with an explanatory
tooltip, and the "Manual pipeline triggers" section (bulk classifier
backfill + rescan-all-source-roots) is hidden entirely for non-admins
since those are admin-bulk operations across every user.
Users tab: stays adminOnly as today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without this, the docker default resolver forwards the lookup to the
host gateway, which returns the public IONOS VPS IP. cloud.hubris is
not in the VPS traefik exposure list, so TLS handshakes during
WebDAV calls die with httpx.ConnectError: SSL UNEXPECTED_EOF.
extra_hosts pins it to caddy on 192.168.8.175, which holds the
cloud.hubris.network cert and proxies to the Nextcloud LXC. Applied
to every service for symmetry; only backend currently makes the
WebDAV calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets each mule-image user (matched via OIDC preferred_username,
overridable in Settings) browse their Nextcloud files/ tree from the
mule-image UI and register subfolders as per-user SourceRoots. Reads
stay direct on the bind-mounted /nextcloud-users path; mutations
(upload, delete, rename, move within NC) dispatch through Nextcloud
WebDAV so oc_filecache, trashbin, comments, and desktop-sync clients
stay coherent.
Backend:
- users.nextcloud_username + nextcloud_app_password_enc (Fernet at rest,
key derived from SECRET_KEY) — alembic 0016
- services/nextcloud_dav.py: minimal WebDAV client (PUT, MKCOL, DELETE,
MOVE) with HTTP Basic auth via the per-user app password
- routers/nextcloud.py: GET /browse, /whoami, GET/POST/DELETE
/source-roots (path-scoped to current_user.nextcloud_username with
realpath traversal guard)
- PATCH /api/v1/auth/me to update nextcloud_username and app password
- OIDC callback defaults nextcloud_username from preferred_username on
first login; backfill on existing users; never overwrites a manual
override
- routers/upload.py: stream upload to NamedTemporaryFile, then PUT to
WebDAV (with MKCOL chain) when destination is NC-rooted; existing
Photo row creation runs unchanged
- routers/discard.py empty-trash: WebDAV DELETE for NC files
- routers/photos.py rename + move: WebDAV MOVE for NC paths;
cross-system move/copy returns a clean error
- routers/folders.py rename + create + permanent-delete: dispatch via
WebDAV when targeting NC-rooted paths
Frontend:
- AuthUser carries nextcloud_username + has_nextcloud_app_password
- services/api.ts: nextcloud + account namespaces
- components/dialogs/NextcloudFolderPicker.tsx: lazy tree browser, name
+ submit -> POST /source-roots
- SettingsDialog: new "Nextcloud library" card with username override +
validate, app-password input, list/remove of NC libraries, and the
picker entry point
docker-compose.yml: NEXTCLOUD_USERS_HOST_PATH bind to /nextcloud-users
on backend + 3 workers; NEXTCLOUD_USERS_ROOT + NEXTCLOUD_BASE_URL env.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds OIDC_LINK_BY_USERNAME as a last-resort linking step after
(issuer, sub) and email both miss. Matches IdP preferred_username
against users.username.
Why: local accounts created before OIDC never collected an email
(no UI for it), so the email fallback cannot relink them. A new
SSO login therefore falls into JIT and creates username-1. On a
single-tenant homelab where the IdP owns the namespace, matching
by username is safe and makes first-time SSO transparent for
pre-existing users. Gated behind a flag so multi-tenant deployments
keep the stricter default.
Adds optional SSO via Authentik (or any OIDC provider) alongside the
existing password flow, and pulls profile images from the provider's
`picture` claim or Gravatar so the sharing UI stops looking anonymous.
Password login stays available as a recovery path; JIT provisioning and
admin-group mapping are env-configurable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shares used to activate instantly on the owner's side with no notice to
the recipient. Introduce a pending/accepted lifecycle so a recipient
gets a bell notification on login and explicitly Accept or Decline
before the shared item lands in their sidebar.
Backend
- Migration 0014 adds `status` + `accepted_at` to heap_shares and
folder_shares; pre-existing rows are backfilled to 'accepted' so
nothing disappears from anyone's current sidebar. One-migration trick:
server_default 'accepted' during add_column, then strip so new inserts
fall through to the Python model default 'pending'.
- New recipient-only endpoints: POST /sharing/{heaps|folders}/{id}/accept
(idempotent) and /decline (hard delete, so re-invites are clean).
- New GET /sharing/pending returning {heaps, folders} of outstanding
invites with target_name + owner_username + permission.
- list_shared_{heaps,folders} now filter to status='accepted' and carry
share_id so the recipient can Leave without a second lookup.
- ShareResponse exposes status so the owner sees pending invites.
Frontend
- NotificationBell lives in the LeftSidebar user row: a Popover
triggered by Bell with a count badge. Each row shows owner avatar,
"{owner} shared {heap|folder} {name}" with a permission subtitle,
and Accept / Decline inline. Polls /sharing/pending every 60s.
- Shared Avatar helper extracted to sharing/Avatar.tsx — used by
ShareDialog, NotificationBell, and the sidebar shared rows so one
user's identity colour is stable everywhere.
- Sidebar shared-row polish: owner avatar bubble + Eye/Pencil
permission icon (was uppercase pill). Right-click opens a context
menu with Open / Leave; Leave calls the existing recipient-revoke
DELETE and invalidates the shared-{heaps,folders} query.
- ShareDialog shows an amber "Invited" pill next to pending recipients.
- New shadcn context-menu primitive (radix dep already installed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous pass still read as two labeled sections with a target
"chip" that looked like an empty input and a dashed-border empty state
that looked like a drop zone. Rebuilt around the common share-modal
pattern: target name inlines into the title, a single compact invite
row (picker + Viewer/Editor dropdown + Share) sits at the top, and a
hoverable list below shows each person with an avatar, name,
permission subtitle, and an X that fades in on hover.
Also fixes the spacing: DialogContent was p-5 with non-flex children
so the gap utility silently did nothing — switching it to a flex
column puts every section on a 16px rhythm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous auto-focus guard was one-shot for the lifetime of the
component, so switching from All Photos → Discarded (or any other
filter-based section) carried over the old activePhotoId — and if it
wasn't in the new view, nothing was focused at all. A new effect
watches currentSection and, on any change (or fresh mount after a
Duplicates/Memories detour), resets the guard and clears the stale
selection so the existing auto-focus picks the first visible photo of
the new view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Target is now anchored in a chip at the top instead of a floating line.
Existing shares and the add-user form are split into labeled sections
with states for loading / empty. Each share row gets a hash-tinted
initial avatar and a semantic permission pill (primary = edit, muted =
view). The user picker is full-width with avatars in the dropdown, and
permission becomes a segmented "Can view / Can edit" control alongside
an icon-labeled Share button.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The group grid only pinned column width; rows defaulted to auto height,
so each cell collapsed to the size of its still-empty <img> and snapped
to 160px once the thumbnail arrived — visible layout jump, plus the
virtualizer re-measured every group on image load.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the grid is already filtered to discarded photos or to the active
heap, every cell would carry the same tint — the grayscale wash or the
green overlay stopped signalling anything and just made thumbnails
harder to read. Timeline now suppresses both when the corresponding
filter is active; the BR icon badges stay for colorblind readability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rules-of-hooks violation: useRef and three useCallbacks sat after the
isLoading/isError/empty early-return block, so first render (loading)
called N hooks and the post-data render called N+4, crashing the view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
