180 Commits

Author SHA1 Message Date
634abc2a95 feat(settings,index): per-user index sub-path, strip dead PP settings, scope duplicates
Lets a user pick a sub-folder under their library as a working index root,
stored server-side (new sidecar user_prefs table). The Library tree, reindex,
and both duplicate views (stacks + cross-folder scan) now re-root to it via a
single userLibraryBase() helper. Also fixes the cross-folder scan/archive
endpoints, which previously walked/touched the whole originals root instead
of being scoped per-user (archive now rejects out-of-scope paths, 403).

Removes PhotoPrism settings (Search/Maps/Server-UI/Features/Import) that only
steered PhotoPrism's own bundled SPA and were never read by mulimage's UI.

Also fixes the Library tree occasionally getting stuck on "Loading folders…"
by dropping gcTime:0 and gating the spinner on isLoading instead of isPending.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-06-30 22:41:33 +02:00
ba5684d120 feat(countries): replace Map view with browse-by-country (like Tags)
Removes the maplibre-gl Map view and adds "Countries" as a sixth
TagCategory, reusing the existing /tags/[category]/[[value]] browse
machinery instead of a bespoke map UI. Backed by a new self-contained
sidecar endpoint that aggregates photos.photo_country with BasePath
scoping, mirroring handleLabels/handleScopedCounts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-06-30 21:25:24 +02:00
74bae78270 fix(duplicates): use server-side path filter instead of client-side filtering
Move basePath filtering from client-side startsWith check to server-side
query filter (path:basePath*) for consistency with map implementation
and improved efficiency. Reduces amount of data fetched when user has
a basePath configured.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-30 21:08:33 +02:00
52ab3b6840 fix(map): remove coordinate swap, properly format basePath filter with quoting
Revert coordinate transformation (PhotoPrism already returns correct [lng,lat] format).
Fix basePath filter query string to properly quote paths with special characters
and add wildcard suffix using same quoteIfNeeded logic as filters store.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-30 21:08:04 +02:00
400b215036 fix(sidebar,map,duplicates): flatten sidebar hierarchy, filter by user basePath, fix map coordinates
- Flatten sidebar: remove collapsible Tags and Review sections, place all items at level-0
  Notes, tag categories (Labels/Keywords/People/Colors/Ratings) now appear directly in Views
  Review tabs (Causes/Stacks/Duplicates) and Hidden appear directly in Manage
- Filter duplicates by user base path to ensure multi-tenant isolation
  listDuplicateGroups now accepts optional basePath parameter
  update review page and sidebar to pass userBasePath() for proper per-user caching
- Filter map geo data by user base path using path: query filter
  map page now only shows geotagged photos from current user's library
- Fix map coordinate positioning: PhotoPrism /geo endpoint returns [lat,lng]
  but GeoJSON and MapLibre expect [lng,lat]. Transform coordinates and bbox
  on data receive to fix photo placement and zoom behavior

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-30 21:04:56 +02:00
e1e508671e fix(move): resolve full per-photo file list so videos actually move
The move resolved photos via the /photos search, whose merged Files array is
trimmed (often omitting a photo's video file) and which applies PhotoPrism's
quality/review/archive filters — so a video's .mov was never listed to move
and nothing happened. Resolve each UID via GET /photos/:uid instead (full file
list, no filters), shared by photos-move and heap-convert via resolvePhotosFull.
Unresolved UIDs are reported as skipped rather than aborting the batch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 22:52:56 +02:00
a52f171946 fix(folders): translate BasePath for create/rename/delete to fix "invalid path"
The sidebar shows user-relative paths (BasePath stripped) but the sidecar
operates on originals-relative paths. Folder create/rename/delete passed the
stripped path straight through, so a BasePath user's ops resolved to the wrong
directory and the sidecar returned "invalid path". Wrap outgoing paths with
toOriginalsPath and map returned paths back with toUserPath, matching the move
flow. Identity for admin accounts (empty BasePath).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 00:06:18 +02:00
e124809ad5 fix(move): move every originals file of a photo, not just the primary
Videos, Live Photos, and RAW+JPG pairs keep several files under Root "/". The
old movePhotoFiles moved only the primary (often the poster JPG), orphaning
the .mov: PhotoPrism then saw the photo as moved (dropped from the grid) while
the video stayed behind and broke. Move the whole originals group under one
shared stem (new uniqueStem helper) so siblings re-stack after reindex; fail
the photo and report it if any sibling can't move.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 00:06:18 +02:00
ad6e733622 fix(move): close the dialog when a move starts so header progress shows
The move dialog held its full-screen overlay open for the whole operation,
hiding exactly the header reindex/status pill the user waits on. Snapshot the
draft state, closeMove() up front, and run the move in the background with a
toast.loading→success/error — mirrors the archive flow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 00:06:18 +02:00
6d9b236ef6 feat(library): reindex button, live grid, archive-reappear fix, bigger carets
- Add one-click "reindex new files" button to the Library sidebar header
  (RefreshCw, calls startIndex rescan:false), spins + disables while active.
- Refresh the photos grid from the indexer WS stream (throttled during the
  scan + once on completion) so newly indexed files appear live.
- Fix archived photos flashing back into the grid when archiving others:
  drop the per-action settle-driven clearRemoved and reconcile removedIds
  against the actual cache instead (clears an id only once it's gone from
  the deduped pages). Covers archive, delete, and bulk-bar removals.
- Replace the tiny Unicode caret triangles with a 16px Lucide ChevronRight
  that rotates 90deg on expand, across folder tree rows, root, Tags, Review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 23:48:18 +02:00
669e5fde33 Merge pull request 'feat(move): "move to folder" for grid selections, folders, and m shortcut' (#3) from claude/infallible-sammet-3471fa into main
Reviewed-on: #3
2026-06-18 00:40:25 +02:00
b2b6060872 feat(move): "move to folder" for grid selections, folders, and m shortcut
Extends the heap-only "move to folder" action to grid single/bulk
selections, sidebar folders, and an `m` keyboard shortcut — all through
one shared dialog driven by a moveDialog store.

Backend (sidecar):
- Extract the heap move/copy + reindex loop into a reusable movePhotoFiles
  helper plus resolveMoveTarget
- POST /photos/move: move/copy an arbitrary UID list into a folder
- POST /folders/:rel/move: reparent a folder dir (whole subtree) under a
  new parent, guarding against moving into itself/a descendant

Frontend:
- moveDialog store + generalized MoveToFolderDialog (heap | photos | folder
  subjects); mounted once in +layout.svelte. Replaces HeapConvertDialog
- movePhotosToFolder / moveFolder service fns
- Entry points: BulkActionBar button, gridKeyNav `m`, FolderTree kebab,
  heap kebab — all call openMove()

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 00:10:19 +02:00
277fdc5a53 Merge pull request 'feat(archive): red-cross flash + instant tile removal on archive/delete' (#2) from claude/infallible-sammet-3471fa into main
Reviewed-on: #2
2026-06-17 23:58:21 +02:00
5be6fd9047 feat(archive): red-cross flash + instant tile removal on archive/delete
Archive/delete now flash a red cross then drop tiles from the grid
immediately, instead of a green check that lingered until the slow
server-reconcile refetch landed. Keyboard `x` archive previously never
called markRemoved, so tiles only vanished on refetch — that lag is gone.

- Add 'removed' bulk state + removedBulk() helper (red cross overlay)
- gridKeyNav archive/delete: removedBulk -> 500ms flash -> markRemoved,
  clearRemoved once refetch settles; restore stays green check
- BulkActionBar: BulkConfig.removing routes archive/delete through the
  red flash; approve/restore/label/note unchanged

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 23:45:38 +02:00
3e164c48d0 fix(notes): page PhotoPrism server-side so /notes shows every captioned photo
Client-side paging of listPhotosWithNotes stopped early for BasePath users:
the sidecar post-filters each page by BasePath, so a full upstream page can
arrive short, tripping the `length < PAGE` end condition before the library
is exhausted — hiding notes past the first slice.

Add GET /api/sidecar/notes: the sidecar pages /api/v1/photos to completion
(keying the loop off the raw upstream page length), filters to non-empty
Caption under the caller's BasePath, dedupes by UID, and returns the set.
listPhotosWithNotes now calls this single endpoint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 00:43:56 +02:00
259adb6a41 fix: tile overlay reactivity + facets showing all marked/noted photos
Problem 1 — per-photo progress overlay never rendered on the grid:
- bulkPhotoStates was `$state(new Map())`; a `.get(uid)` read in PhotoTile
  didn't reliably re-run when the entry flipped, so the spinner/check/X
  overlay never appeared. Switch to SvelteMap (svelte/reactivity).

Problem 2 — Notes / Colors / Ratings only showed the newest ~1000 photos:
- All three derived from `listPhotos({ count: 1000 })`, silently hiding
  older marked/noted photos.
- listPhotosWithNotes now pages the whole library.
- Add listPhotosByUids() and resolve the Colors/Ratings marks-pool from the
  complete marked-UID set (from getAllMarks) instead of the newest slice;
  wire it into the TagsBrowserSidebar panel and the tag drill page.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 00:37:30 +02:00
ccf2c6b7c7 fix: bulk label apply, instant archive removal, per-photo state, drop sidebar counts
Issue 1 — colors/labels not applying in bulk:
- sidecar validColors only accepted 4 of the 8 UI swatches, so teal/blue/
  purple/pink returned "invalid color" and rolled back the whole bulk txn.
  Add teal, blue, purple, pink to validColors.
- Add invalidateFacets() and call it on the success path of bulk marks,
  patchTargets, and single-photo edits so the Colors/Ratings/Notes facet
  sections refresh immediately instead of waiting out staleTime.

Issue 2 — archived photos linger in the grid:
- Add a UI-only removedIds set to the bulkAction store; archive/delete/
  restore/keep call markRemoved() on success so tiles vanish instantly,
  cleared once the server-reconcile refetch lands (no cache eviction).

Issue 3 — per-photo progress state:
- Wire startBulk/doneBulk/failBulk into all metadata applies, bulk
  (BulkMetadataSidebar) and single (RightSidebar), so colors/ratings/
  notes/dates/keywords show the spinner -> check -> X overlay.

Issue 4 — remove Left-sidebar count badges:
- Drop count badges from root folder, Archive, heaps, Notes, and the
  folder tree, plus the now-dead count queries and unused imports. Facet
  drill-panel counts are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 00:21:04 +02:00
a13e171295 bump post-action delay to 1s for PhotoPrism indexer to catch up
200ms was too short - PhotoPrism's batch archive marks photos in the DB
but the search index (used by /api/v1/photos) updates asynchronously.
1s gives the indexer time to reflect the change before we invalidate
and refetch the timeline.
2026-06-07 22:43:24 +02:00
73c36b4817 revert evictFromCache, add back delay + marks invalidation
Remove evictFromCache entirely - the cache-manipulation approach was
brittle and broke both archive (photos not removed) and tags
(colors/ratings showing empty). Replace with: 200ms delay before
invalidation to let PhotoPrism's indexer process the change, plus
invalidateQueries(['marks']) so tag caches (colors, ratings, notes)
refresh alongside the photo timeline.
2026-06-07 22:31:39 +02:00
82f2a40269 fix: folder loading gets error state + gcTime=0 for fresh refetch
Add a proper error branch to the folder tree so a failed sidecar request
shows an error state rather than a perpetual loading spinner. Also sets
gcTime: 0 so the query re-fetches fresh data when the sidebar remounts
instead of holding onto stale cache across navigations.
2026-06-07 22:10:44 +02:00
f6c0f7a507 fix: evictFromCache only targets infinite queries (not flat caches)
evictFromCache was removing archived UIDs from ALL ['photos']-prefixed
queries, including flat lists like marks-pool and with-notes. This
corrupted the tag drill pages — when navigating to Colors/Ratings after
setting marks, the pool was missing photos and the grid showed empty.
Now only infinite queries (those with a pages array) are filtered.
2026-06-07 22:05:41 +02:00
1df16a6142 perf: evict archived/deleted photos from cache immediately
Remove archived/restored/deleted/approved UIDs from all cached photo-list
pages right after the API confirms, so the grid updates on the same tick
instead of waiting for a network round-trip. Also removes the 400ms
doneBulk animation delay (now unnecessary since tiles vanish instantly).
2026-06-07 21:53:16 +02:00
5da1022ed1 feat: loading toasts for all photo actions
Add loading→success/error toast transition to every bulk operation
(archive, restore, delete, approve, add-to-heap, metadata patch).
Also wires gridKeyNav + CauseGroupCard into the bulkAction store so
keyboard-triggered actions show the same per-tile pending/done/error
feedback as BulkActionBar buttons.
2026-06-07 21:40:18 +02:00
dtoro
da63ad769a feat: bulk action progress — header pill + per-thumbnail states
- New bulkAction store: tracks active/label/detail state for the pill
  and a Map<uid, pending|done|error> for per-tile overlays
- Extract StatusPill.svelte from IndexerStatusPill (generic active/label/detail
  props); IndexerStatusPill becomes a one-line wrapper
- +layout.svelte: render a second StatusPill driven by bulkAction store,
  alongside the indexer pill in the AnimatedMule header
- BulkActionBar: extend withBusy with optional BulkConfig (ids/label/doneLabel);
  pending tiles dim + spinner on start, green checkmark flashes for 400ms before
  cache invalidation removes them; red overlay on error, auto-clears after 2s
- onApprove/batchEdit: wire onProgress callback to setDetail so the pill shows
  the filename currently being processed during fan-out keep operations
- batch.ts: add completedId as third arg to onProgress (backwards-compatible)
- PhotoTile: derive bulkState from store; pending/done/error overlays sit above
  the selection tint; hover-video guarded against pending tiles

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 09:43:12 +02:00
dtoro
86e38e152d perf: batch label stats query + fix scoped thumb 2026-06-07 00:33:23 +02:00
dtoro
3757eb0170 fix: use /usr/bin/mariadb-admin for mariadb healthcheck
MariaDB 11 renamed mysqladmin → mariadb-admin; neither the old
healthcheck.sh probe nor mysqladmin is reachable from the Docker
healthcheck exec context. Switch to the full-path binary that is
confirmed present in the container.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 00:23:28 +02:00
dtoro
cfd0c6aa81 fix: rename sidecar() helper → callSidecar() and fix mariadb healthcheck
The new `const sidecar` axios instance (added in 243e5d3) clashed with the
pre-existing `async function sidecar()` fetch helper, causing `npm run build`
to fail with a rolldown redeclaration error. Rename the fetch helper and all
its call sites to `callSidecar`.

Also replace the mariadb healthcheck command: `healthcheck.sh` calls the
`mariadb` CLI which isn't on PATH in the current image layer, causing the
container to stay permanently unhealthy and the deploy webhook to abort before
the `npm run build` step runs — leaving the old JS bundle serving from nginx.
Switch to `mysqladmin ping` which is available in all MariaDB 11 images.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 00:13:01 +02:00
243e5d3831 fix: sidecar response interceptor + folder proxy
- Added 401-handling response interceptor to sidecar axios instance
  (matches existing http instance) so expired/invalid tokens redirect
  to login instead of showing raw 404/401 errors
- Added GET /api/sidecar/folders — proxies PhotoPrism's
  /api/v1/folders/originals with BasePath post-filter
- Updated listFolders() frontend to call sidecar proxy
- Updated plan with remaining fixes
2026-06-06 23:01:38 +02:00
14a1b4e54e frontend: route all photo queries through sidecar timeline proxy
- listPhotos → /api/sidecar/timeline (was /photos)
- countPhotos → /api/sidecar/timeline (was /photos)
- hasPhotosMatching → /api/sidecar/timeline (was /photos)
- Sidecar handler forwards X-Count header for countPhotos()
- Sidecar adjusts X-Count to reflect post-filtered count
2026-06-06 20:17:13 +02:00
7df1c04c0f sidecar: scoped photos/timeline proxy (fixes review + archive leak)
- New GET /api/sidecar/timeline — proxies PP's /api/v1/photos and
  post-filters by FileName prefix matching the user's BasePath
- Also works for review/archive views (q=review:true, q=archived:true)
- Frontend route uses /timeline to avoid Gin route conflict with
  existing /photos/:uid/marks pattern
2026-06-06 20:11:07 +02:00
8f97590d9f sidecar: scoped labels + counts proxy (fixes cross-user label leak)
- New GET /api/sidecar/labels — proxies PP's labels, recalculates
  PhotoCount per user's BasePath via DB query
- New GET /api/sidecar/counts — returns user-scoped sidebar badges
  (all, review, archived, private, photos, videos, favorites)
- Fixed auth middleware to expose userUID and basePath on context
- Fixed ppClient.resolveSession — uses correct endpoint
  (GET /api/v1/session, not /api/v1/session/{token}) and correct
  JSON field names (UID, Name instead of UserUID, UserName)
- Frontend: listLabels now calls /api/sidecar/labels instead of /api/v1/labels
2026-06-06 19:23:22 +02:00
4c08eba27a fix: scope marks, labels, and subjects to the authenticated user
Marks (ratings/color labels) were stored without a user column — every
user saw every other user's marks. Labels and subjects from PhotoPrism's
global endpoints leaked across users because those endpoints ignore
BasePath ACL.

Sidecar:
- Add UserName as composite primary key on Mark (photo_uid, user_name)
- Replace validateSession with resolveSession that fetches the user
  identity from PhotoPrism's session endpoint
- Filter all mark queries by user_name

Frontend:
- Filter listLabels/listSubjects through a BasePath-aware existence
  check — each label/subject is kept only if the user has at least one
  matching photo (single count=1 probe per item, batched at concurrency 8)
- Skip filtering for admin users with empty BasePath (single-user compat)

Also documents USER_BASEPATHS in .env.example — the env var that drives
per-user library isolation via PhotoPrism's auth_users.base_path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-06 12:36:18 +02:00
6c96c22b33 feat(compose): opt-in GPU overlay for VA-API ffmpeg accel
Layer docker-compose.gpu.yml to mount /dev/dri/{card0,renderD128}
into pp-app, add it to render (992) + video (44) groups, and set
PHOTOPRISM_FFMPEG_ENCODER=vaapi. Hosts without a VA-API device just
skip the overlay (`-f docker-compose.yml -f docker-compose.gpu.yml`
becomes opt-in per deploy).

Drops video transcode + thumbnail generation from CPU to the iGPU
where present — large win for HEVC libraries. README documents the
flag; default behavior on the base compose is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 23:28:21 +02:00
70dc1b6bdf Merge pull request 'Mulimage 2.0' (#1) from new into main
Reviewed-on: #1
2026-05-21 22:48:54 +02:00
e3d4f6d92e web: SVG favicon (Greek lowercase mu) with light/dark adaptive fill
Use the Greek lowercase mu glyph as the app's favicon. The SVG
carries a `prefers-color-scheme` media query that flips the path
fill between near-black (light mode) and near-white (dark mode),
so it stays legible against any tab-bar background without an
extra browser hint.

Linked before the existing PNG so browsers that support SVG
favicons (Chrome 80+, Firefox 41+, Safari 9+) pick it up; the PNG
remains as a fallback. `apple-touch-icon` keeps the PNG since iOS
home-screen icons can't be SVG.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 00:30:23 +02:00
9fc650fb12 web(sidebar): auto-expand folder tree ancestors of the active folder
When the timeline navigates to a nested folder (RightSidebar's
open-folder icon, URL hydration, back/forward), the LeftSidebar
already highlighted the matching row via filters.folderPath — but
if the parent folder was collapsed in the persisted openSet, the
highlighted row wasn't visible at all.

Each FolderTree instance now runs an effect that adds every
ancestor of the active path to its openSet on filter change. The
root instance expands the top-level ancestor first, which mounts
the next-depth FolderTree instance — and the same effect runs
there, cascading down to the leaf. Persisted to localStorage so
the expansion sticks across reloads.

Skipped in `readonly` mode (heap-convert picker has its own
selectedPath and shouldn't drive the sidebar state).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 15:12:36 +02:00
29f7ad7073 web: deep-link from RightSidebar to folder/map + anchor-mode timeline + fix root count
RightSidebar:
- Folder + Location rows gain a small ArrowUpRight icon button that
  deep-links into the timeline / map view focused on the photo. OSM
  external link removed; the in-app map nav covers the same job.

Folder navigation:
- New navigateToFolder(path, { focusUid, focusTakenAt }) helper in the
  filters store; LeftSidebar's pickFolder collapses to a one-liner that
  reuses it.
- One-shot pending-focus stash carries both UID and TakenAt across the
  goto. URL-watch effect on the timeline consumes the stash so even
  same-folder navigations (where the filter doesn't change) get
  picked up.

Anchor-mode timeline query:
- listPhotosAround(q, takenAt, after, before) issues two parallel
  PhotoPrism calls (`after:<day-1>` oldest-first + `before:<day+1>`
  newest-first), merges + dedupes newest-first. Uses PhotoPrism's
  existing date-only DSL clauses — no server changes.
- When a deep-link stashes a TakenAt, page 0 of the photosQuery uses
  the merged window so the target photo is loaded even for photos
  buried past the standard newest-first cursor. Pages 1+ are disabled
  in anchor mode (PhotoPrism's day-precision cursor would infinite-loop
  on dense days; users see 120 around the target, refresh to drop the
  anchor).
- After page 0 lands, the existing scrollToIndex(targetIdx) expands the
  windowed render set + scrolls the tile into view.

Map view:
- /map honors `?lat=&lng=&zoom=&focus=` URL params, jumping to the
  photo's coordinates at zoom 17 instead of fitBounds-ing the full
  library. Params are stripped after first apply so a manual zoom-out
  + reload doesn't snap back.

LeftSidebar root count badge:
- Now matches what Cmd+A selects in the timeline. Old code used
  /config.count.all (library aggregate, includes archived/hidden/
  review). Switched to countPhotos('', { merged: true }) which counts
  the actual photo entries the timeline lists.
- countPhotos gains a `merged` option; with merged=true it returns the
  response body length instead of the X-Count header — PhotoPrism's
  X-Count is always the file-row count regardless of merged, so a
  HEIC + JPG companion pair inflated the badge to 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 15:04:23 +02:00
c134afe023 web(review): confidence-aware date guesser with combined filename + path signals
Refactor suggestDateFromPath to combine multiple signals instead of
trying patterns in priority order:

- Filename Y-M-D corroborated by path Y-M/Y-M-D → HIGH (filename-
  agrees-path). Fixes the case where a Samsung-style 20240226_xxx.jpg
  under 2024/02/ was returning the path-only 2024-02-01.
- Filename Y-M-D with no path signal → HIGH (filename-only).
- 10/13-digit Unix epoch in basename → HIGH (unix-timestamp) —
  covers WeChat (mmexport...) and FB saves.
- Path Y-M-D → HIGH (path-ymd).
- Path Y-M only → MEDIUM (path-ym-default-day, synthesised day=01).
  Sidebar row labels these "(estimated day)" so the user knows.

Filename parser now accepts `.` and space separators (covers macOS
screenshots, manual 2024.02.26 renames). Path parser accepts `.` too.
OriginalName participates as a secondary filename signal when present
and different from the on-disk basename.

Patterns we explicitly DO NOT parse, to avoid silent date flips:
DD-MM-YYYY / MM-DD-YYYY, 2-digit years, bare camera sequence numbers.

Add photoNameAndDir(p) helper next to primaryFile so RightSidebar,
BulkActionBar, photoActions, and gridKeyNav all derive {fileName,
path} the same way — fixes the bug where photo.FileName was
undefined on the single-photo detail endpoint and the basename branch
was being skipped entirely.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 12:04:48 +02:00
a54d90a2d9 web(review): YYYY/MM path fallback + suggestion row below date + tighten 'a' gate
- suggestDateFromPath: when only year+month appear in the path
  (e.g. 2024/01/), synthesize day=01 so date-only foldering yields
  a usable suggestion instead of null.
- RightSidebar: move the suggestion row below the Taken-at input.
- BulkActionBar + gridKeyNav: show the "Accept date & Keep" button
  and fire the bare 'a' shortcut only when EVERY targeted photo has
  a path-derivable date — no more silent approve-without-fix for
  mixed selections.
- gridKeyNav: drop local cachedPhoto duplicate, use the shared one.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 11:41:40 +02:00
97f51a05c4 web(review): scope date suggestion to EXIF Stripped tab + 'a' shortcut
- Gate the sidebar's "Suggested from path" row on /review?tab=stripped_exif
  instead of a per-photo TakenSrc heuristic — PhotoPrism stores a guessed
  TakenAt for stripped-EXIF photos too, so the heuristic was hiding the
  row even when a path-derived date was available.
- Same gate on the BulkActionBar's "Accept date & Keep" button.
- Extract acceptDateAndKeep() + cachedPhoto() into photoActions so the
  bar button and a new bare-'a' shortcut in gridKeyNav share one path.
- Show an 'A' kbd hint on the bar button.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 11:31:04 +02:00
d1ddc48f81 web(review): switch cause tabs to PhotoGrid, add date-from-path suggestion
- CauseGroupCard now wraps PhotoGrid so selection, keyboard nav, and
  preview flow through the standard timeline plumbing. Per-tile hover
  Approve/Archive and the group-wide Approve all are gone — the bottom
  BulkActionBar's review-section Keep/Archive handle single + bulk.
- Low Resolution tab opts into a new PhotoTile dimensionBadge prop so
  WxH stays visible on each tile.
- New suggestDateFromPath util parses YYYY-MM-DD from filename or
  folder path. RightSidebar surfaces it as an amber Apply row above
  the Taken-at input whenever the photo lacks a trusted TakenAt.
- BulkActionBar gains a "Accept date & Keep" button (review section
  only) that patches each selected photo's TakenAt from its path
  suggestion when available, then approves.
- Drop the Same folder / Same camera / Same year strips and the
  RelatedStrip component from the metadata sidebar.

Also bundles in-progress Notes route + tile components and small
tweaks to LeftSidebar, DuplicatesView, CrossFolderGroupCard, and
photoprism.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 11:20:02 +02:00
55c870c155 web(sidebar): nest Hidden under Review submenu
Hidden is the resting place for photos dismissed during review, so it
groups naturally with the Review subitems. Stays a section-nav button
(keeping its scoped count badge); only Archive remains as a flat Manage
entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:44:43 +02:00
981328faff web(review): expandable Review group in sidebar, tabs become subitems
Mirrors the Tags affordance: chevron-only toggle, no /review landing
entry, navigation only via subitems (cause buckets + Stacks +
Cross-folder linked as /review?tab=<id>). Cause list reuses the
review-groups query so empty buckets stay hidden. The /review toolbar
drops the pill row and shows the active tab as a breadcrumb segment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:42:43 +02:00
e1707c314d preview: keep metadata sidebar mounted across photo changes
Previously the modal's aside was gated on `focusedPhotoQuery.data`, so
each arrow-skim unmounted the sidebar until the next photo's metadata
arrived — which reflowed the preview pane sideways. Now the aside is
always mounted while the modal is open; its contents swap between the
metadata panel and a small InlineLoader the same way the timeline's
right-aside does.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:28:12 +02:00
64c0da794d web(sidebar): drop count badges from Map and Tags rows
Per-tag totals are already surfaced by the TagsBrowserSidebar, so the
main sidebar's Map/Tags rows stay as pure navigators. Also removes the
now-orphaned geo, marks, and keywords cache observers that only fed
those badges.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:24:53 +02:00
0f4e2e0b8f preview: LQIP + ±2 prefetch + bound action bar to its column
Preview pane paints instantly: a blurred copy of the same thumbnail the
grid loaded (cache hit) rides beneath the sharp fit_1280, which now
carries fetchpriority=high and decoding=async. A $effect prefetches
fit_1280 for the ±2 neighbours so arrow-skim hits the HTTP cache.
Carousel thumbs drop to fetchpriority=low so they yield to the main
image. Skeleton grid gains an mt-2 to breathe against the toolbar.
BulkActionBar moves inside the main column in both PreviewModal and the
/tags drill-in so it no longer stretches under the right sidebar.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:24:31 +02:00
ea1803ec2f web: 8-color label palette + VIDEO badge in carousel
Expand color labels from the 4-swatch Lightroom culling palette to 8
neutral colors (red/orange/yellow/green/teal/blue/purple/pink) with no
attached semantics, rendered as outlines that fill in when picked.
Carousel tiles now show a VIDEO badge, and the folder row in the right
sidebar always renders ("/" for root) instead of disappearing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 09:56:05 +02:00
fc5f30fad1 fix(sidebar): use count.labels for Tags badge, drop bogus label:* query
PhotoPrism's q-DSL has no "has any label" predicate: `label:*` matches
every photo regardless of label, `label:<slug>` only matches that one
slug, and `keywords:*` behaves the same way. The prior `all:true label:*`
returned a 400 (and the earlier "drop all:true" follow-up made it return
the unfiltered library size, which then fed into tagsTotal and inflated
the parent Tags badge to ~library_size on admin sessions).

Switch labelsBadge to the precomputed `configQuery.count.labels` — the
same source the Tags sub-row's `tagCategoryCount('labels')` already
uses. The parent Tags badge now sums the exact same numbers the sub-rows
display: labels, keywords, people (distinct slugs/keywords/subjects)
plus ratings/colors (photos carrying each mark). Drop the wantScoped
short-circuit since the values are all library-wide now anyway.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 06:41:07 +00:00
a7b8a60473 web: admin surfaces so the PhotoPrism UI is never needed
- Account tab in General settings — self-service password change.
- UsersDialog (admin-only footer entry) — full /api/v1/users CRUD with
  admin-issued password reset.
- People as a fifth tag category alongside Labels/Keywords/Colors/Ratings,
  backed by /api/v1/subjects and the `person:` DSL clause.
- About tab in Library settings — version, library counts, feature chips,
  and a collapsible env-config help panel for the bits PP has no runtime
  API for (OIDC, TF, WebDAV).
- Library tab expanded with Indexer-advanced, extra Downloads checksums,
  and a Features grid that only renders keys PhotoPrism actually returns.
- Fix the SettingsDialog null-draft race the same way GeneralSettingsDialog
  already had: normalize on open, never null on close.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 08:26:12 +02:00
24dfa996b3 web: de-brand PhotoPrism references in user-facing copy
Replaces "PhotoPrism" in UI strings (empty states, tooltips, toasts,
log header, login screen) with neutral terms like "the indexer", "the
library", "the server" — accurate regardless of backend. The login
header becomes "Mulimage" and drops the explicit PhotoPrism mention.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 08:26:12 +02:00
e364e4128f web: unify empty + loading states behind EmptyState/InlineLoader
Replaces ad-hoc "Loading…" text and bare empty messages with two
shared feedback primitives that carry subtle lucide icons, consistent
muted-foreground/destructive tones, and a11y signaling (role=status,
aria-busy, role=alert on destructive empties). Loading copy gains
context ("Loading photos/folders/heaps/metadata…") and the right-
sidebar idle state moves from a "ⓘ" glyph to a MousePointerClick
icon. SkeletonGrid stays as the initial-grid loader.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 08:26:12 +02:00
d2a76fa58c fix(sidebar): paginate folder counts; drop bogus all:true from labels query
Two distinct bugs were causing left-sidebar badges to under-report:

1. sidecar/folders/counts hard-capped each PP /photos call at count=1000
   and deduped UIDs from that single page. Any folder with >1000 file
   rows under it (typical for a multi-year root scan with HEIC sidecars)
   silently lost everything past row 1000. On this library the root
   badge reported 912 while the year subfolders summed to 1175. Loop
   offsets instead, breaking when PP returns a short page.

2. The Labels-badge query passed all:true label:* to PP, which 400s with
   "Unable to do that" - none of the other bucket queries prefix
   all:true. Drop it; the scoped() helper already injects the user's
   path clause when applicable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 22:44:43 +00:00
0d5f380948 preview: full-screen modal replaces inline split + tags route reorg
The old SplitGrid + InlinePreview pane is replaced by a full-screen
PreviewModal mounted once at the layout root. Open via Space on the
focused tile or double-click; close on Esc (or X / Space again).
Inside, PreviewPane renders the focused photo, RightSidebar carries
the metadata, BulkActionBar reuses the existing per-photo actions,
and PreviewCarousel windows ±50 thumbs around the focused index.

Selection contract matches the grid: plain click reduces, shift
extends the range, ⌘/Ctrl toggles, plain arrow drops the multi-
selection, shift-arrow extends. New clearBulkToFirst() helper makes
Esc / Clear collapse a bulk back to single-focus on its first member
before the next press fully dismisses (modal closes, grid clears
focus).

Tags route reorganised into /tags/[category]/[[value]] with its own
+layout and TagsBrowserSidebar; the old monolithic /tags/+page is
trimmed to a legacy redirect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 00:13:20 +02:00
680fa90cbe web: collapsible Tags section + file size on its own row
Groups score, color label, keywords, and auto-labels under one
collapsible "Tags" section on the metadata sidebar (open by default,
choice persists). Moves file size onto its own row with a HardDrive
icon so dimensions and weight read as independent facts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 23:34:01 +02:00
9bba097d91 repo cleanup: retire legacy mule-image stack, lock PhotoPrism UI to loopback
Delete the old Python+React mule-image stack (backend/, frontend/,
docker-compose.yml, mulita.yml, .env*) plus the one-shot migration and
sample dirs (migrate/, photos-sample/, photovault-app-prompt.md). Only
the PhotoPrism + Go sidecar + SvelteKit web stack remains, so drop the
".photoprism." qualifier from the compose+env filenames.

Bind PhotoPrism's port to 127.0.0.1 so the user-facing surface is just
the SvelteKit web/ app; admin reaches PP's UI via SSH tunnel. Flatten
PHOTOPRISM_INDEX_WORKERS' nested default (podman-compose's interpolator
doesn't expand ${A:-${B:-…}}). Rewrite README for the current stack.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 23:30:38 +02:00
155e9bb126 web: single-pointer selection + post-action focus, video debounce, tags metadata sidebar
Selection: plain arrow nav now clears prior multi-selection so exactly
one tile is ringed at a time; shift-extend still grows from the anchor.
onApprove / onRestore / onDelete advance focus via focusAfter(ids)
before clearing selection, matching onArchive.

Preview: defer mounting <VideoPlayer> by 250ms so arrow-skim across
video tiles doesn't open and immediately cancel range requests; hard-
abort the underlying <video> on unmount so the connection releases.

Tags drill view: right-sidebar metadata wired in (single-photo
RightSidebar, BulkMetadataSidebar for >=2 selected), resizable edge
mirrors the timeline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 23:52:16 +02:00
c783f129cc preview: serve fit_1280 + tighten timeline grid padding
InlinePreview was requesting fit_1920 for both the still image and
the video poster — roughly 3× the pixel count of what the pane
actually needs. Drop to fit_1280: still sharp inside the inline
pane (which the user resizes around 300–500px tall in practice)
while cutting payload by ~⅔.

Timeline: pull the grid wrapper padding in from `p-6 pb-24` to
`pr-2 pl-2 pb-2 overflow-x-hidden` now that the SplitGrid preview
pane sits above the grid — the old generous padding existed to
breathe under a full-screen modal that no longer renders inline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 23:25:10 +02:00
24c449f475 feat(preview): fanned deck of cards on multi-select
When `selection.ids.size >= 2`, the inline preview swaps the single-
photo view for a SelectionDeck: each selected thumbnail renders as
an absolutely-positioned card with a translate + rotate computed
from its index in the deck, so the spread reads as a fan. CSS
transition-transform handles the reflow as the deck grows or
shrinks; `in:fly` lands new cards from above, `out:scale` shrinks
removals into the stack. Hash resolution walks the existing
TanStack caches (per-photo + photos-infinite envelope) so the deck
is side-effect-free — no fetches just to render thumbs.

Drop the now-redundant Maximize hover affordance on PhotoTile and
the `onOpenPreview` plumbing through PhotoGrid / +page.svelte:
single-click already places a tile into the inline preview pane,
so the dedicated "open preview" button no longer has a job.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 22:50:40 +02:00
e36f1939c6 feat(preview): inline split-pane preview + sidebar metadata pass
Replace the fullscreen PreviewOverlay with an inline top pane above
each grid surface. SplitGrid + InlinePreview render the focused
photo/video inside the host page; a resizableVertical action drives
the divider and the height persists via the view store. Applied to
the timeline, /tags drill-in, /review cause tabs, /photo/[uid], and
/map. selection.focused is now the single source of truth for both
the inline pane and the right sidebar — preview.svelte store and
PreviewOverlay are removed.

Sidebar: drop the thumb; lead with icon-led filename and folder
rows that match the date/place rhythm. Move dims+size to the top
(below date) and camera/lens/exposure into the collapsible File
section. Read-only spans share the input padding so the text column
aligns across rows. Folder row sits between date and dims+size.

VideoPlayer: stop forcing width/height: 100% so videos honour their
intrinsic aspect ratio inside the pane. Key the player on file hash
in InlinePreview so navigating between videos remounts the element
and autoplay fires again.

Sidebar (LeftSidebar): switch the labels badge to a dedicated
countPhotos('label:*') query so it reports photos with a label
rather than PhotoPrism's category roll-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 22:46:06 +02:00
Claudio
2a75896274 sidecar: declarative USER_BASEPATHS reconciler
PhotoPrism's OSS edition has no way to map OIDC claims to BasePath, so
every freshly-registered OIDC user lands with BasePath="" and either
sees the whole library (admin) or nothing (guest) — never their own
subfolder.

Introduces a sidecar-driven reconciler with a single env knob the
admin sets in docker-compose / .env.photoprism:

  USER_BASEPATHS="test:test, alice:family/alice, bob:bob"

(`user:originals-relative-path` pairs, comma-separated.) On boot and
every 60s thereafter the sidecar:
  - mkdir -p's the target subdirectory under ORIGINALS_ROOT so
    PhotoPrism's path: ACL filter has somewhere real to point;
  - UPDATEs photoprism.auth_users.base_path for the matching row
    where it differs (idempotent, missing users skipped — they
    materialise on first OIDC login and the next pass catches them).

The reconciler uses a separate gorm connection scoped to the
`photoprism` schema with PhotoPrism's own DB user, since the existing
`sidecar` user only has grants on `mule_sidecar.*`. Connection stays
dormant when PP_DB_PASSWORD is empty — the feature is opt-in via env.

Compose changes: thread PP_DB_* + USER_BASEPATHS through to the
sidecar service. New users.go file isolates the reconciler logic;
main.go calls startUserBasepathReconciler() during boot.
2026-05-18 20:25:19 +00:00
Claudio
85847848c4 Sidebar: scope count badges to current user's library
PhotoPrism's /api/v1/config.count is library-wide and the same value
for every authenticated session. That made non-admins (and admins
with a non-empty BasePath) see badges that didn't match what the
timeline actually showed them.

Replaces the direct `configQuery.data?.count?.<bucket>` reads in
LeftSidebar with per-bucket queries against PhotoPrism's /photos
endpoint. The new `countPhotos(q)` helper sets `count=10000` and
reads the X-Count response header to get the true total in one round
-trip (PhotoPrism's ACL filter is what scopes the result, so the
header reflects "what this session can see").

Each bucket query appends `path:"<BasePath>*"` so admins-with-a-
BasePath stay scoped too; non-admins without a BasePath short-circuit
to `uid:none` (their effective visibility is zero, no point
querying). Admins without a BasePath skip the scoped queries
entirely and keep using the precomputed /config totals — same
network footprint as before for the common case.

Affected badges: Favorites, Hidden, Archive, Review, Tags (labels
component). Map already used `geoQuery` whose result is ACL-filtered
server-side, so its badge is per-user-correct without changes. The
`favorites` field was missing from PpClientConfig.count's TypeScript
type; added it.

Resolves the `test`-user complaint: sidebar showed the admin
library's totals next to Review / Hidden / Archive / Favorites
because those numbers came from /config, not from a user-scoped
query.
2026-05-18 20:11:52 +00:00
Claudio
b0c8c06b2b Sidebar: suppress global count badges for non-admin users
PhotoPrism's /api/v1/config.count returns library-wide aggregates to
any authenticated session, with no per-user scoping. The timeline
itself IS scoped (a guest sees zero photos), but the sidebar was
rendering admin-side totals next to Review / Hidden / Archive / Tags /
Map / root for non-admins — including a freshly-registered "test"
user with role=guest and BasePath="".

Until PhotoPrism gains per-user counters, the SPA now derives an
`isAdminUser` flag and gates every count that's drawn from
configQuery on it. Non-admin users see the labels without badges;
counts re-appear automatically when promoted. Per-folder counts from
the sidecar (which DO scope to BasePath) are unaffected.
2026-05-18 20:02:20 +00:00
Claudio
986dab7334 Clear TanStack Query cache on session change
Sidebar counts, marks, folder counts, etc. were keyed only on query
name, not on the authenticated user. Logging in as a non-admin kept
rendering the previous admin session's data because the cache was
never invalidated. clearSession and adoptSession now wipe the cache
so each identity starts fresh.

User-observed: the "test" user (role guest, BasePath="") saw the
admin library counts in the left sidebar after signing in.
2026-05-18 20:01:17 +00:00
829d7bed83 feat(sidebar,tags): per-tab counts and aggregated Tags badge
- /tags: each tab pill shows its own count (labels/keywords =
  distinct tags, ratings/colors = photos covered). Labels and marks
  queries become always-enabled on the route so every pill resolves
  immediately; keywords stays lazy.
- LeftSidebar: Map badge now reads from the shared `['geo']` cache so
  it matches /map's "N geotagged" footer instead of count.places
  (distinct locations). Tags badge sums the four inner counts;
  keywords contributes lazily once /tags?tab=keywords is visited.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 21:59:47 +02:00
d70244f17e feat(web): fold duplicates+inbox into /review; sidebar UX cleanup
- /duplicates and /inbox routes removed and folded into /review as
  additional tabs alongside cause tabs; /duplicates keeps a redirect
  for bookmarks.
- LeftSidebar: drop import/inbox tile and favorites; show per-user
  BasePath label at the folder root.
- RightSidebar: split file header into read-only path over editable
  basename (matches sidecar rename contract); date field switches to
  plain-text ISO YYYY-MM-DD (no native datetime picker) with strict
  validation and revert-on-invalid-blur; preserves original hour.
- BulkMetadataSidebar: same ISO-only date input with invalid-state
  styling and apply-button gating.
- BulkActionBar: drop redundant Restore and Undo buttons; ⌘Z still
  reachable via gridKeyNav.
- gridKeyNav: remove favorite toggle (F) alongside the favorites view
  retirement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 21:48:38 +02:00
a38c3c6e9b feat(duplicates): show count badges on Stacks/Cross-folder tabs
Stacks count appears immediately (cheap query); cross-folder count fills
in after its tab is visited (lazy disk scan). Page observes both queries
from cache so badges stay in sync with the panel content.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 21:14:56 +02:00
Claudio
8ac406ac1f Review view: reuse timeline action surface, simpler card UX
The /review cards now share the timeline's PhotoGrid, gridKeyNav,
BulkActionBar, and global selection store instead of carrying parallel
implementations. The card is reduced to a header (title + count +
Dismiss / Archive all), an advisory caption, and a PhotoGrid; keyboard
nav, hover affordances, multi-select, and bulk verbs come from the
shared machinery.

- New web/src/lib/services/photoActions.ts holds the canonical
  dismissPhotos / archivePhotos helpers (toast wording, focus advance,
  undo push, ['photos'] + ['review-groups'] cache invalidation).
  BulkActionBar.onApprove / onArchive and gridKeyNav.approveCullTargets
  / toggleArchive('archive') route through it. CauseGroupCard's
  Dismiss / Archive-all buttons call the same helpers - one code path
  from any surface.

- Approve verb renamed to "Dismiss" across BulkActionBar, gridKeyNav
  toasts ("Kept N" -> "Dismissed N"), and the new review card. The
  BulkActionBar Clear/Dismiss clear button is just "Clear" now so the
  verb only means the action.

- /review sets filters.section='review' on mount and restores on
  unmount, which is what swings the shared action surface into review
  semantics; an effect clears the selection on tab change so a
  previously-selected photo from another cause can't be hit by a new
  tab's bulk verb.

- The route mounts BulkActionBar at the bottom and swaps the right
  aside to BulkMetadataSidebar when selection.ids.size >= 2 - same as
  the timeline; gives the user a one-shot "apply this Date / Caption /
  Keyword to all selected" affordance for EXIF-stripped batches.

- CauseGroupCard drops its bespoke keyboard handler, ResizeObserver,
  focusedIdx state, per-tile hover Approve/Archive buttons, confirm()
  dialogs, and toast.loading worker loop. The unused CauseBadges
  component is removed.
2026-05-18 18:50:23 +00:00
70de4b65ec feat(duplicates): sidebar count + auto-run cross-folder scan
Sidebar Duplicates badge now sums stacks + cross-folder groups, with
cross-folder observed from cache (no eager disk scan from the sidebar).
Cross-folder tab auto-fires the scan on access; button becomes Rescan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 20:48:20 +02:00
79a9ef49d4 feat(web): scope library views to per-user BasePath
PhotoPrism's user entity carries a per-user BasePath; the web app now
mirrors that scope client-side so each user sees only their own subtree
in the sidebar, timeline, folder counts, and heap-convert target picker.
Admin without a BasePath is unchanged. Also removes the redundant
"✕ <folder>" pill below the folder tree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:18:39 +02:00
bd39d310ab feat(sidebar): path next to date + ISO date-only inputs
Moved the filepath display from the readonly facts row to immediately
above the Date Taken editor, and switched both the single-photo and
bulk Date Taken inputs from datetime-local to date (YYYY-MM-DD). The
date-only compare in commitTakenAt avoids clobbering the stored
time-of-day when the user blurs the field without editing it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:18:39 +02:00
Claudio
f9f276a986 Review view: tabs per cause (mirrors /tags pill row)
Each cause now becomes a URL-driven tab instead of stacking the cards
vertically. Empty buckets are filtered out by the adapter already, so
the tab row only shows causes with hits. Active tab persists via
?tab=<cause_key>; refresh / share / back land on the same panel.

The standalone ReviewView container is no longer used (the page
renders the active CauseGroupCard directly); deleted.
2026-05-18 15:54:25 +00:00
Claudio
ca9f6e6bd2 Review view: cause-grouped UI with actions + suggestions
Builds a dedicated /review route that mirrors /duplicates' chrome:
- /review/+page.svelte mounts Toolbar + ReviewView + RightSidebar
- CauseGroupCard.svelte renders one card per cause with Approve all
  and Archive all bulk actions plus a per-cause suggestion line
- CauseBadges.svelte shows every matching cause as chips on each tile
- services/adapters/review.ts fetches review:true and groups photos
  by primary cause; current taxonomy is low_resolution >
  stripped_exif > implausible_year > non_image_type > quality_other
  (low_resolution ranks first because it's the most actionable signal)

Sidebar gains an opt-in showRelated prop that adds three
RelatedStrip panels (same folder / camera / year) for the
'decide these together' workflow.

LeftSidebar's Review entry switches from a section filter to a
route link so /review picks up the click.

PpPhoto gains the missing Resolution field PhotoPrism actually
returns on list responses.
2026-05-18 09:03:30 +00:00
9d955d6b94 feat(timeline): muted hover-to-play preview on video tiles
PhotoPrism plays a silent preview of the actual video when you hover
its grid tile; this mirrors that. After a 250ms debounce the tile
mounts a muted, looping <video> over the thumbnail and cross-fades it
in on first decoded frame, so cursor-skimming doesn't fire N requests
and the tile never blanks mid-fetch. The byte-prefetch helper added
in af96922 is now redundant — the hover <video> warms the same caches
on its own.

Also tells Vidstack the playback URL is video/mp4 via a nested
<source>: our /api/v1/videos/.../avc URL has no extension, so
Vidstack's suffix sniff was failing, falling back to a HEAD probe,
and picking the wrong loader (which surfaced as
NS_ERROR_DOM_MEDIA_METADATA_ERR in Firefox).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 08:20:16 +02:00
af96922e89 feat(preview): inline video player via vidstack + hover-warm prefetch
Replace PreviewOverlay's bare <video controls> with a vidstack-driven
player wrapping the same source URL. Vidstack's default video layout
provides a polished chrome (gradient bottom bar, large play overlay,
hover-revealed scrubber) and registers <media-player>/<media-provider>
custom elements that vendor the browser quirks.

To keep first-frame latency low, PhotoTile starts a hover-warm fetch
of the playback URL after a short (120 ms) delay — a single Range
request of the first 512 KB pages the backend's pre-transcoded MP4
cache file into the OS page cache and lands in the browser's HTTP
cache, so when the player mounts and issues its own bytes=0- request
the response is satisfied from disk. Each hash is warmed at most once
per session; AbortController cancels hovers that don't commit.

The vidstack modules are dynamically imported on mount so they never
run during SvelteKit's static prerender — they side-effect
customElements.define() calls which would crash under SSR.
2026-05-18 08:13:59 +02:00
ccbc1050de fix(web): drop indexer debug log, harden settings dialog, escape search-placeholder quotes
Three small cleanups bundled:

- Remove the `console.debug('[indexer]', ...)` line in the indexer
  store. The PhotoPrism WS protocol is now verified; the log was a
  development aid that no longer earns its console noise.

- GeneralSettingsDialog: normalize cloned PpSettings so `ui` / `search`
  / `maps` are always real objects (some deployments return them
  unset), and re-clone the draft on each open instead of nulling it on
  close. The previous lifecycle let Dialog's exit animation keep the
  form mounted while `draft` was already null, which threw at runtime
  via the `bind:value={draft.ui!.theme}` getters.

- Search-input placeholder string: rewrite as a JS expression so the
  embedded `"vacation"` quotes inside the example don't terminate the
  HTML attribute early. The previous form was a Svelte parse error
  that stopped the dev-server module from loading.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 08:13:42 +02:00
Claudio
cfd85a1fe8 fix(sidebar): root badge shows the library total, not zero
Earlier we (a) wildcarded the per-folder count fan-out in the sidecar
so internal tree nodes (year folders, etc.) recurse, and (b) flipped
the timeline root view to mean "the whole library" instead of
"photos with no path component". The remaining piece — the badge on
the root row — still computed `total - Σ(folderCounts)`, which used
to give the count of root-direct photos. With recursive folder
counts that subtraction double-counts every nested photo (year +
month + …) and clamps the badge to 0.

Use PhotoPrism's authoritative `count.all` directly. That now matches
what the timeline shows under `/` (everything indexed) without an
extra round-trip.
2026-05-18 00:06:01 +02:00
aa63d4c11d feat(sidebar): persist metadata section collapse across photo switches
GPS, Credits & notes, and File sections in the right sidebar now read
and write their expanded state through the view store and persist it
to localStorage. Closed by default; the user's first toggle pins their
choice across subsequent photos and reloads.

Switched from the previous data-driven defaults ("open if this photo
has GPS / IPTC fields") to static defaults: a data-driven default would
change between photos, fire a programmatic `toggle` event on the
<details> element, and silently overwrite the user's persisted choice.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 00:03:38 +02:00
79ec511482 fix(timeline): clear selection after keyboard archive/delete/approve
X (archive/restore), Delete, and S (approve) keyboard handlers in
gridKeyNav advanced focus and invalidated the photos query but never
cleared the selection — so the archived/deleted/approved UIDs stayed in
the SvelteSet and kept their rings on tiles that hadn't unmounted yet.
A subsequent Ctrl-click would then pile new UIDs on top of the stale
set, leaving the user uncertain which photos a follow-up action would
actually target. The BulkActionBar button path already cleared selection
for the same reason; mirror that here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 00:03:38 +02:00
Claudio
505fef5dfc fix(folders): root shows whole library + sidecar counts recurse
Two paired fixes for the folder tree on the timeline:

* +page.svelte: drop the `Path === ''` post-filter for the root entry.
  PhotoPrism's indexer always nests photos under YYYY/MM, so "photos
  whose Path is empty" is always the empty set in practice — the root
  entry looked broken instead of "whole library". Treat `/` as the
  unscoped view and rely on subfolder selections (now wildcarded via
  filters.svelte.ts) for narrowing.

* sidecar/handlers_folders.go: the per-folder count fan-out used
  `q=path:<x>`, the same exact-match operator that just got fixed in
  the web filter. Result: every year-level folder reported count=0
  in the sidebar. Switch to `q=path:"<x>*"` so the count reflects
  the whole subtree (dedupe by UID still in place).
2026-05-17 23:25:43 +02:00
Claudio
8083328f2d fix(filters): make path:folder query recursive
filtersToQ emitted `path:<folder>` for any non-root folder, but
PhotoPrism's `path:` operator is exact-by-default — so picking the
"2024" node in the folder tree returned zero hits when all photos
lived in date-stamped sub-folders (`2024/01`, `2024/02`, …). PP's
indexer always nests photos under YYYY/MM, so every year-level
folder was empty in the timeline.

PhotoPrism supports a trailing `*` wildcard, so emit
`path:"<folder>*"` instead:

  path:"2024*"     →  matches `2024`, `2024/01`, `2024/02/...`, …
  path:"2024/01*"  →  matches `2024/01` plus descendants — still
                      correct for a leaf folder.

Confirmed against the M0 instance: picking 2024 now returns the full
year's photos; 2024/01 still returns its direct contents.
2026-05-17 23:22:29 +02:00
e669e80a91 feat(web): header pill showing PhotoPrism indexer status
Subscribes to PhotoPrism's /api/v1/ws channel on login and surfaces
index.indexing / index.updating / index.completed events as a small
status pill in the header (next to the AnimatedMule wordmark).

- Shows "Indexing" + the current filename (basename, monospace) during
  the scan pass, "Finalizing — <step>" during faces/counts/folders/
  purge/moments, and "Indexed in Ns" for ~4s after completion before
  fading.
- Per-file events arrive many per second on large libraries — throttled
  to 150 ms with a trailing-edge update so the pill stays calm and
  always lands on the most recent filename. Step and completion events
  bypass the throttle.
- Filename slot is fixed at 24ch so the pill width stays constant
  through a run (no horizontal jitter as filenames change length); the
  full path is exposed via the parent's `title` for hover.
- WS reconnect uses exponential backoff capped at 30 s, and the store
  tears down cleanly on logout so we don't leak sockets across
  identities.

Defensive parsing throughout: PhotoPrism's WS protocol isn't a stable
contract, so unknown event shapes are ignored rather than thrown —
worst-case the pill stays idle.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 23:13:57 +02:00
Claudio
3d8e050af4 fix(compose): cap indexer concurrency to keep shared hosts alive
Default PHOTOPRISM_INDEX_WORKERS is NumCPU/2 (3 on the M0 6-core test
LXC). Each worker forks TF + ffmpeg + libvips, so a fresh index of
~1.2k photos pushed the LXC's load avg above 50 and starved the
Proxmox host. Pin to PP_WORKERS / PP_INDEX_WORKERS (default 2) so
the indexer is calm by default; bump in .env.photoprism on dedicated
boxes.
2026-05-17 23:13:36 +02:00
Claudio
cce1d876c3 fix(compose): pass OIDC env vars under the names PhotoPrism actually reads
The compose file was using PHOTOPRISM_OIDC_ISSUER_URL / _CLIENT_ID /
_CLIENT_SECRET / _PROVIDER_NAME / _REDIRECT_URI, but PhotoPrism's CLI
flags are --oidc-uri / --oidc-client / --oidc-secret / --oidc-provider —
so the env vars it parses are PHOTOPRISM_OIDC_URI / _CLIENT / _SECRET /
_PROVIDER. With the old names PhotoPrism silently ignored them, OIDC
stayed dormant, and `photoprism show config` reported blank oidc-uri /
oidc-client even though everything else looked configured.

Confirmed on the M0 LXC: renaming the env vars makes the Authentik
"Sign in" button appear on /library/login, /api/v1/oidc/login emits a
proper 302 to the IdP authorize endpoint, and the callback creates the
OIDC user + session in the DB.

The user-facing `.env.photoprism` keys are unchanged (OIDC_PROVIDER_NAME,
OIDC_ISSUER_URL, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET); the compose file
just maps them to the correct PHOTOPRISM_* targets. OIDC_REDIRECT_URI
is removed because PhotoPrism derives the redirect from PHOTOPRISM_SITE_URL.
2026-05-17 22:56:07 +02:00
7e10f0b462 fix(web): align lazy-PreviewOverlay layout with renamed OIDC bootstrap
Stale import landed when 6b8c7ab rebased on top of the OIDC-rename
commit. The call site updated to bootstrapSessionFromPhotoPrism but
the import line kept the old bootstrapSessionFromCookies name —
svelte-check caught it on the next pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:50:27 +02:00
6b8c7abc20 perf(web): batch folder counts, bounded scroll scan, adaptive thumbs, lazy preview
Six-item frontend performance pass on the SvelteKit app.

P1 — Move per-folder photo counts to a new sidecar endpoint and defer
the fetch to requestIdleCallback. The old client-side path fired one
/photos?count=1000 per folder from the browser (≈1 MB JSON × N folders)
on every cold sidebar mount; the new POST /api/sidecar/folders/counts
fans out over loopback with bounded concurrency and returns a single
{path: count} payload of a few KB.

P2 — Bound the visibleRange scroll-scan around the previous visible
band instead of sweeping every shell from index 0 on each scroll-rAF.
Falls back to a full sweep on cache miss (filter reset, programmatic
jump) so behaviour is unchanged at the edges.

P3 — Adaptive thumbnail size + srcset. PhotoTile now picks the smallest
PhotoPrism tile_* variant (100/224/500) that covers the user's grid
preset at the current DPR. Adds decoding="async".

P4 — Lift the selection check above the {#each} loop. Mostly readability
— SvelteSet.has() is already per-key reactive — but keeps the hot loop
body terse.

P5 — Split dedupedAll / photos derivations so filter-store mutations
(search-as-you-type, section toggles) don't re-walk every loaded page;
only the cheap folder-scope filter re-runs.

P6 — Dynamic-import PreviewOverlay on first preview.uid !== null and
cache the loaded module; closing the overlay leaves the component
mounted with its internal {#if} collapsing the DOM.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:48:59 +02:00
Claudio
9a3ad3e579 fix(web): adopt OIDC session from PhotoPrism's localStorage (not cookies)
PhotoPrism's /api/v1/oidc/redirect handler doesn't actually set
auth_token/auth_session cookies — it returns an HTML page that does:

  setItem("pp:<storageNamespace>:session.id",       <session uid>)
  setItem("pp:<storageNamespace>:session.token",    <X-Auth-Token value>)
  setItem("pp:<storageNamespace>:session.user",     <user JSON>)
  setItem("pp:<storageNamespace>:session.provider", "oidc")
  window.location.href = "/library/login";

The deployment's reverse proxy is expected to bounce /library/login
(and /library/*) back to `/`; the SPA then reads PhotoPrism's
storageNamespace from /api/v1/config, looks up session.id and
session.token under that prefix, and adopts the session.

Confirmed via the M0 test instance: prior to this change, server-side
sessions were created on every OIDC return (DB row present) but the
browser had no way to claim them, so the user bounced back to /login.
2026-05-17 22:48:13 +02:00
Claudio
4abe6d758c feat(web): OIDC login button + cookie-based session bootstrap
The SvelteKit /login was username/password only; the legacy comment
even called out 'OIDC SSO ships in M4 when the IdP is wired up'.
Authentik is wired up now, so:

- /api/v1/config exposes ext.oidc when the IdP is configured. Fetch
  it on the login page and conditionally render "Sign in with
  {provider}", which kicks off /api/v1/oidc/login.
- After PhotoPrism completes the auth code exchange, it sets
  `auth_token` + `auth_session` cookies and redirects to siteUrl
  (/library/browse by default; the deployment's reverse proxy is
  expected to bounce that to /). bootstrapSessionFromCookies()
  reads those cookies, calls GET /api/v1/session/<id> with the
  cookie's token, and adopts the resulting session into the SPA
  store on mount.
- Root layout's auth guard now waits for the bootstrap pass before
  punting to /login, so a fresh OIDC return doesn't get redirected
  away before the session is read.
2026-05-17 22:06:33 +02:00
cb5bc120dc fix(web): restore button cursor: pointer affordance under Tailwind v4
Tailwind v4 dropped the default cursor: pointer on <button>, so most
interactive controls (bulk sidebar, star/color pickers, summary
disclosures) had no hover affordance. Add a global base rule covering
button / [role=button] / summary, plus cursor: not-allowed for disabled
states to mirror the existing opacity-50 styling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:00:34 +02:00
a72619e3d1 feat(timeline): focus follows archive, snaps to first on view load
- New selection.focusAfter(excluded) walks selection.order forward past
  the archived/restored set so X-ing through the timeline keeps the
  cursor on the next live photo instead of falling back to photo[0]
  via the auto-anchor effect. Wired into gridKeyNav.toggleArchive (X
  key) and BulkActionBar.onArchive.
- Auto-focus effect on the timeline always re-anchors to photos[0] on
  view load (pageCount → 1), instead of preserving a stale uid from
  the previous filter.
- PhotoGrid re-anchors focus when the previously focused uid isn't in
  the new photo set, so drilling into a /tags category drops the
  cursor on its first tile instead of carrying a stale selection from
  whatever view the user came from.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 21:56:54 +02:00
84e433ff63 fix(timeline): scroll-driven visibility scan, factor PhotoTile + SkeletonGrid
- visibleRange action rewritten to scan [data-uid-shell] divs on each
  rAF-throttled scroll instead of attaching an IntersectionObserver to
  sample tiles. The observer approach broke on return from /inbox: with
  cached photo data, shells mounted in the same Svelte pass as the
  scroll root and tileRegister fired before any __visibleRange stash
  was in place, so registrations dropped silently. Fast scrolling could
  also strand the observer in a dead zone when every sample tile left
  the viewport before the next was mounted. Shells are always rendered,
  so a DOM scan always finds a true first/last.
- Extract PhotoTile + SkeletonGrid so the timeline and the drill-in
  PhotoGrid share one tile chrome (selection animation, badges,
  hover-only "open preview" affordance).
- FolderTree count badge moves inside the row's button so the badge
  area becomes part of the click target instead of a dead zone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 21:33:03 +02:00
d35de8a2a9 feat(web): /tags tabs + keyword surfacing + dup tab restyle
- /tags hosts four tabs (Labels (auto) / Keywords / Ratings / Colors),
  URL-driven with pagination on the label + keyword grids; ratings and
  colors stay as fixed buckets.
- /duplicates tabs (Stacks / Cross-folder) restyled to pill row in the
  Toolbar to match /tags; tab state moved into the route and bound to
  ?tab=...
- New aggregateKeywords() service fans out per-photo getPhoto calls so
  user-typed Details.Keywords surface on /tags (PhotoPrism's /labels
  only returns classifier output).
- RightSidebar renders photo.Labels[] as dashed-border chips after the
  Keywords section, each linking to /?q=label:slug.
- /colors and /ratings routes redirect to /tags?tab=colors|ratings so
  old bookmarks still land somewhere useful; LeftSidebar drops their
  entries and the Tags badge now sums labels + ratings + colors.
- listFolderCounts dedupes by UID (merged=false returns one row per
  FILE, so HEIC+JPG / Live Photo / RAW+JPG pairs were inflating folder
  counts ~2x).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 21:19:44 +02:00
d5e4f23c0f feat(web): sidebar root folder + folder counts, drill-in colors/tags/ratings, click semantics rework
Sidebar
- New `/` root-folder entry at the top of the Folders group. Active
  when the timeline is scoped to root; the photo grid post-filters to
  `Path === ''` because PhotoPrism's `path:` operator can't express an
  exact-root match. Collapsible chevron, persisted to its own
  localStorage key, and a kebab carrying just "New subfolder".
- Per-folder count badges. `/api/v1/photos?q=path:X&count=1000` per
  folder in parallel via `listFolderCounts`; root count derived from
  `config.count.all − Σ subfolder counts`.
- Folder tree starts at depth=1 under the root so nested rows indent
  visually relative to `/`.
- Footer matches the Toolbar / action-bar h-9 height.

Timeline interaction
- Single click on a tile selects only that tile (clears others); the
  preview now lives on dblclick. Modifier clicks still go through
  `gridKeyNav`'s document handler (shift = range, cmd/ctrl = toggle).
- `x` (archive) now actually archives — PhotoPrism's photo PUT
  silently drops the Archived field, so we route through
  /batch/photos/{archive,restore} the same way the BulkActionBar
  already did. Mirror for `u`.
- Preview close restores the timeline focus + scrolls the last-shown
  photo into view via `forcedExpand`+`scrollTileIntoView` so it
  actually mounts (selection ring would otherwise stay invisible when
  the user navigated far in preview).
- `applyFolderScope` only narrows the timeline to root when the active
  view is a folder view (no heap / search / non-default section), so
  label clicks / heap views / favorites no longer drop subfolder
  photos.

Action bar
- Inline `h-9` row at the bottom of the main column (not `fixed`),
  matching the Toolbar's visual language. Right sidebar stays full
  height — the bar only spans the timeline width.
- Approve action wired for the review pile.

Colors / Tags / Ratings drill-ins
- New shared `PhotoGrid` component owning tile rendering, selection
  styling, single-click-selects + dblclick-previews, and `setOrder`
  for arrow-key nav.
- Each route's drill-in `<main>` carries `use:gridKeyNav` and a
  trailing `<BulkActionBar />` so shift/cmd/ctrl click, arrow keys,
  and the keyboard shortcuts work the same as the timeline.
- Tags switches from `goto('/?q=label:…')` to an in-place drill-in
  with a back button, mirroring `/colors`'s flow.
- Category cards + drill-in photo cards honour the global
  `view.thumbnailSize` (XS–XL) so the timeline's size selector now
  reaches into all four grids.

Settings
- General-settings dialog merges Appearance into UI and switches free
  text inputs to selects for the PhotoPrism theme / language / start
  page / map style (the value-from-server prepends if it's outside
  the curated list so we never silently rewrite a custom value). Time
  zone uses `<datalist>` with `Intl.supportedValuesOf('timeZone')`.

Sidecar
- Heap convert runs reindex synchronously per source path so the
  client's invalidate-and-refetch sees the moved files.

Inbox
- New /inbox route stub for the upcoming import workflow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 21:01:41 +02:00
5153aeebec fix(sidecar): podman keep-id mapping so mutations land as the host user
Without keep-id the container's UID:GID maps into the rootless podman
subuid range (524288+), so the sidecar couldn't create
`/photoprism/originals/.duplicates/` — the archive endpoint failed
with "mkdir: permission denied", and rename / folder ops would have
hit the same wall.

The PhotoPrism container already has this override for the same
reason; mirror it for the sidecar.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 18:23:37 +02:00
4362e475a7 build(sidecar): containerize + delete the Node prototype
- New sidecar/Dockerfile: multi-stage golang:1.25-alpine → distroless/
  static, ~12 MB final image, static CGO-free binary.
- Wire pp-sidecar into docker-compose.photoprism.yml so the whole
  stack (mariadb + photoprism + sidecar) starts with one
  `podman-compose up`. Container reaches mariadb + photoprism on the
  internal network; the host gets 127.0.0.1:8000 for Vite's proxy.
- New SIDECAR_LISTEN_ADDR env var (default 127.0.0.1 for the host-mode
  dev loop) so the container can bind 0.0.0.0:8000 and let the port
  mapping reach it. Without this the loopback bind was invisible to
  the host.
- Delete sidecar/legacy/server.mjs — the Node prototype's archival
  window is over; git history is its home now.
- Update sidecar/README with compose-first bringup; keep the host
  `go build` flow as the fast-iteration loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 17:23:36 +02:00
032dce6c85 feat(sidecar): port Node prototype to Go + Gin + GORM + MariaDB
Replace the Node prototype (server.mjs) with the stack the merge plan
calls for: Go 1.25, Gin for routing, GORM + MariaDB for persistence.
Same wire contract on /api/sidecar/* so the SvelteKit client doesn't
change.

- Marks move from a JSON file on disk to mule_sidecar.marks (auto-
  migrated by GORM on first boot). The Node prototype's marks.json
  was dev-only; not migrated.
- Folder/rename/heap-convert/duplicates handlers reproduce the
  prototype's behaviour, including the path-traversal defence
  (resolveUnderRoot + EvalSymlinks), the size-bucket prefilter for
  the duplicate hasher, and the background reindex fire-and-forget
  pattern.
- Auth model unchanged: requireSession middleware proxies the
  caller's X-Auth-Token to PhotoPrism's /api/v1/photos?count=1
  before any destructive op.
- Expose pp-mariadb on 127.0.0.1:3306 in docker-compose so the
  host Go process can reach mule_sidecar.* without joining the
  container network.
- Archive the Node prototype under sidecar/legacy/server.mjs for
  one cycle as reference.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 17:15:47 +02:00
0766b47bb2 feat(sidebar): library + general settings dialogs, sidebar footer, alignment polish
- Add a Settings cog to the Folders header that opens a tabbed library
  admin dialog (Library / Index / Import / Logs) wrapping PhotoPrism's
  /api/v1 settings, index, import and errors endpoints.
- Add a sticky footer to the left sidebar with the signed-in user's
  display name plus quick-toggle theme, general-settings cog (separate
  dialog for app prefs), and sign-out. Pull these out of the top
  Toolbar trailing slot.
- Align depth-0 folder rows with the rest of the sidebar entries (drop
  the leading chevron column when no children) and bring heap rows in
  line with folder rows so the kebab is part of the row's hover
  background instead of a detached chip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 16:51:56 +02:00
17df1ecd09 feat(preview): play videos inline in the lightbox overlay
Switch the preview overlay from a still <img> to a <video> tag when the
focused photo's Type is "video". Uses PhotoPrism's /api/v1/videos/:hash
endpoint with the existing previewToken, falls back to a still thumb as
the poster, and autoplays muted so the controls reveal without
clobbering whatever else is on the page.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 16:51:46 +02:00
8c2526d982 feat: PhotoPrism M0 bring-up — compose stack, web client, sidecar, migrate
Replace the legacy mule-image backend with PhotoPrism plus a thin
SvelteKit client and a Node sidecar for endpoints PhotoPrism doesn't
expose (file rename), and add a two-phase migrator (metadata via PUT,
heaps → albums) for the existing Postgres library.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 16:06:58 +02:00
claudio
423a73a8a6 fix(SettingsDialog): drop dead imports after AI tab removal
tsc --noEmit caught seven TS6133 "declared but never used" + one
TS2614 "no exported member 'features'" left over from a27267f. Strip
Brain/RotateCcw icons, the unused Switch + Loader2 imports, the
adminApi + featuresApi + features module references, and the
SETTINGS_FEATURE_FLAGS_KEY constant. No runtime change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 00:28:14 +02:00
claudio
a27267f7ad refactor: drop AI/vision pipeline + plain Postgres + full-refresh script
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
  - backend: app/services/vision/, app/tasks/vision.py,
    app/services/feature_flags.py, app/routers/features.py — all
    deleted; admin AI/feature-flag endpoints and the worker-vision
    bootstrap call gone. Photo.needs_review and its index dropped.
  - frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
    types, "Needs Review" sidebar entry + filter, needs_review filter
    URL param all gone.
  - infra: worker-vision compose service + models_data volume deleted;
    worker-light command no longer runs bootstrap_models; the db
    image switches from pgvector/pgvector:pg16 to postgres:16; backend
    Dockerfile drops the dedicated torch RUN layer; requirements.txt
    drops torch/torchvision/open-clip-torch/onnxruntime.

Alembic 0019_drop_ai_remnants:
  - drops photos.needs_review + ix_photos_needs_review
  - DROP EXTENSION IF EXISTS vector (must run before the image swap;
    the new postgres:16 doesn't ship pgvector)

New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 00:20:38 +02:00
claudio
6915c30911 build(backend): cache torch wheel + pip downloads across rebuilds
Splits torch+torchvision into its own RUN layer so edits to
requirements.txt don't invalidate the ~200MB CPU-only torch download.
Adds buildkit cache mounts on both pip-install layers so even when a
layer is invalidated (or buildkit evicts it) the wheel is reused from
the on-disk pip cache instead of refetching from download.pytorch.org.

Triggered by two consecutive deploy failures where pytorch.org timed
out mid-download (2026-05-13 ~21:41 and possibly ~22:47).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 23:08:10 +02:00
claudio
4e1af0f356 refactor: simplify after self-review
- drop the redundant isLoadingMore boolean from the background-loading
  store; derive from inFlight > 0 in the selector
- pull _active_source_root_paths out of get_library_stats and reuse it
  from get_duplicate_groups (same is_active + admin-scope check, now
  in one place)
- drop p.rstrip('/') in the duplicates folder-scope filter (SourceRoot
  paths are never written with a trailing slash)
- match Timeline's initial-load affordance to the new bottom indicator
  (Loader2 spinner + ellipsis instead of plain "Loading photos...")
- trim narrative comments that explained what the surrounding code does

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 20:50:17 +02:00
claudio
5e2823ae94 feat(library): tighten duplicates scope, drop in-app upload, polish loaders
- duplicates: restrict the /library/duplicates/groups query to photos whose
  folder path actually lives under an active SourceRoot in the user's
  settings. Nextcloud's "move to trash" flow was leaving .delete/purge-1
  Folder rows wired to the original source_root_id, leaking those entries
  into the Duplicates view as ghost paths that the user never opted into.
- discard: add a spinner to the "Delete N" and "Empty discard pile"
  buttons (and their confirm dialogs) while the destructive mutation is
  in flight, so the user gets immediate feedback for a slow operation.
- timeline: render a bottom-of-grid "Loading more photos…" indicator
  while usePhotosQuery's background cursor loop is still pulling pages.
  Backed by a tiny Zustand store the loop drives via a balanced
  start/stop (counter, not boolean, so rapid filter changes can't flip
  the flag false while a fresh loop is alive).
- remove client-side upload UI + /upload endpoint. Nextcloud is the
  authoritative ingress now; the duplicate path created confusion and
  the backend route is gone too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 20:44:36 +02:00
root
99edd7d395 ui(thumbs): remove the "date may be wrong" badge
The amber AlertTriangle in the bottom-right of every photo with
has_date_warning set was more visual noise than help — the filter pill's
"Date issues" option still surfaces the same photos when the user
actually wants to triage them. Keeps the BR corner uncluttered.

Backend field + filter pill option unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:55:32 +02:00
root
ab3c55dd96 feat(topbar): drop search box to reclaim filter-bar space
The search box on the right edge of the filter bar wasn't pulling its
weight — kills it entirely along with the supporting plumbing:

- FilterBar: remove input + Search icon import + local/debounced state
- filterStore: drop `q`, `setQ`, plus all references in INITIAL_FILTERS,
  filtersToParams, hasActiveFilters, snapshotFilters
- usePhotosQuery: stop passing q through filtersToParams
- useFilterUrlSync: drop the `q` URL param read/write
- PhotoThumbnail + PreviewView: remove the search-match banner/chip and
  findSearchMatch helper imports
- Timeline + MemoriesView: stop subscribing to / forwarding the prop
- useKeyboardShortcuts: drop the `/` and Cmd+F focus hotkeys
- KeyboardHints: drop the `/` hint and the now-stale `?` collision note
- delete hooks/useSearchQuery.ts (no callers) and lib/searchMatch.ts

Backend /photos/search endpoint left untouched — no UI reaches it now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:53:32 +02:00
root
c30b387dc3 feat(grid): user-configurable thumbnail size for grid views
Adds a "Size" pill in the FilterBar with 5 presets (XS/S/M/L/XL, 96–272px,
default M=160) that drives the cell size in the Timeline, Memories, and
Duplicates grids. Preference persists in localStorage. Preview filmstrip
is intentionally untouched — it's a fixed-track nav rail, not a grid.

Centralised in a new viewSettingsStore so every grid reads from the same
source. Duplicates' virtualizer is poked on size change so row heights
and the keyboard nav's column count stay in sync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:37:01 +02:00
Claudio
347f58b4f3 perf(thumbs): pool NC client, smaller grid thumbs, eager owner load
Five stacked optimisations for the thumbnail hot path so the timeline
grid lands in fewer round trips and fewer bytes.

1. PhotoThumbnail: switch from 'medium' (640px) to 'small' (240px) for
   grid cells. 240px oversamples 150-200px logical cells on 2x retina
   and drops payload 5-8x. Lightbox and preview filmstrip keep 'large'
   and 'medium' respectively.

2. nextcloud_dav: pool the httpx client. A module-level AsyncClient
   with HTTP/2 + keepalive (max_connections=64, keepalive_expiry=120s)
   replaces the per-request constructor that paid a fresh TCP+TLS
   handshake on every preview fetch. Auth is per-user so it stays at
   the call site via auth=BasicAuth(...). Lifespan-managed: init in
   main.py's lifespan startup, aclose on shutdown. requirements.txt
   gains the http2 extra to pull in h2 (not currently installed).
   Same change applies to fetch_memories_info_async since it hits the
   same host.

3. PhotoThumbnail img: add decoding="async" so JPEG/WebP decode moves
   off the main thread, plus fetchPriority="low" so grid backfill
   doesn't fight UI fetches.

4. Eager-load Photo.user via joinedload from the thumb handler.
   _get_photo_with_share_fallback gains an options parameter so other
   callers stay zero-overhead; only the thumb handler asks for the
   owner join. Eliminates the second SELECT users per request.

5. Disk-fallback path picks up Cache-Control: private, max-age=86400
   in both the FileResponse and X-Accel branches so re-renders match
   the NC primary path's caching behaviour.

Net: a warm grid page should drop from ~200-400 ms median per thumb to
well under 100 ms; payload drops ~5-8x; backend sustains higher
concurrency with fewer sockets to Nextcloud and one fewer Postgres
round-trip per request.
2026-05-12 00:30:43 +02:00
Claudio
68bbe6f024 fix(compose): mount video_cache_data volume on backend + workers
The /playback transcode cache lives at /data/video-cache. That
directory was created in container-local storage (the mkdir at
services/video.py import time), not on a shared volume — so the
worker pretranscode populated its own ephemeral copy while the
backend served from a different empty one, and the cache evaporated
on every container restart.

Add a video_cache_data named volume mounted on backend, worker-light,
and worker-vision so the pretranscoded files actually reach the
serving path and survive deploys.
2026-05-12 00:09:25 +02:00
Claudio
c4df92720b feat(playback): pre-transcode HEVC videos in background; veryfast preset
Long videos blocked /playback for the entire encode duration. The fix
is to populate the cache before the user clicks, not when they click.

Changes:
- Extract ffprobe + ffmpeg helpers to services/video.py so the request
  handler and the background task share one sync implementation. The
  endpoint wraps calls in asyncio.to_thread; celery just calls them.
- New tasks/video.py with pretranscode_video. Idempotent: skips when
  the cache is already current and skips passthrough-safe sources
  (h264 in mp4/m4v/webm). 30-min task time limit so the long-tail
  files (3GP archive, multi-minute 1080p clips) still complete.
- scan_folder now dispatches pretranscode_video alongside
  generate_thumbnails / extract_metadata for any new video row.
- POST /library/maintenance/backfill-video-cache enqueues every
  active video so the existing library catches up.
- libx264 preset bumped from fast to veryfast. ~2x throughput on this
  CPU-only box, output a few % larger but well within disk budget.
- /playback simplifies to: cache check, passthrough if h264 in
  web-safe container, else sync transcode (still there as fallback
  for races against the queued task).

Once the backfill task drains, /playback should be near-instant for
every video. Any video added afterwards is pre-transcoded at scan
time, so the user keeps that property going forward.
2026-05-12 00:03:15 +02:00
Claudio
2a5270d399 fix(playback): force mp4 muxer; ffmpeg cant infer format from .tmp extension
Transcode wrote to {id}.mp4.tmp for atomic publish, but ffmpeg picks
the muxer from the output filename and rejected .tmp with Unable to
choose an output format. Add -f mp4 so the temp name is just storage
scratch, not a format hint.
2026-05-11 22:52:36 +02:00
Claudio
1b6ff45726 feat(playback): transcode HEVC .mov to H.264 MP4 on first hit
iPhone .mov files are HEVC Main 10 with codec_tag hvc1. Safari decodes
that fine; Chrome and Firefox refuse 10-bit HEVC entirely, which the
browser surfaces as "playback is not supported" against the existing
/original endpoint. Confirmed against the user's
26-05-01 13-13-26 0525.mov: codec_name=hevc, profile=Main 10,
audio=aac/48kHz.

New endpoint /photos/{id}/playback handles this transparently:
- check the on-disk cache at /data/video-cache/{id}.mp4 first; serve
  if newer than the source
- passthrough h264 in mp4/m4v/webm containers (ffprobe to confirm)
- otherwise transcode src -> H.264 8-bit MP4 with libx264 fast/CRF 23,
  audio re-encoded to AAC because the iPhone 16 ships APAC audio that
  no browser can decode; +faststart for progressive load
- atomic publish via tmp + os.replace so a failed run never leaves a
  half-written cache entry
- HTTP Range support so <video> can seek the result

The .mov container is excluded from the passthrough fast path because
Chrome/Firefox refuse to play even h264-in-mov reliably, so .mov always
goes through the cache (transcode-or-remux). /original is refactored
to share the new _serve_file_with_range helper.

Frontend getVideoSrc swaps from /original to /playback. /original
stays for downloads and any non-<video> fetches.

First-hit cost is ~9s wall for a 13s 1080p HEVC clip on this box
(software libx264, 4 cores). Long videos are still sync-in-request
because the browser's <video> can't deal with a 202 response; if that
becomes painful, lift the transcode into a celery task with a polling
endpoint.
2026-05-11 22:40:05 +02:00
Claudio
09c12ea35b fix(scan): only resurrect discards when file changed; add Saved toast
The scan_folder resurrect path was unflagging every discarded photo on
every backend boot. start_initial_scan fires scan_all_source_roots on
container start, which fans out scan_folder for every source root,
which walked every file and silently set is_discarded=False on rows
whose file was still on disk -- so every deploy wiped the user's
discard decisions. Today's series of resurrect log lines for
admin/Photos came from that path, not from any actual user re-upload.

Gate the resurrect on os.path.getmtime(file) > discarded_at so the
WebDAV-DELETE-then-re-upload and trashbin-restore-via-PUT-overwrite
flows still trigger (those rewrite the file and bump mtime), but
routine sweeps respect the user's intent. Rows with discarded_at NULL
(legacy) fall through to skipped -- preserve intent over cleanup.

While there: add a Saved toast to the single-photo updateMutation.
The previous patch made cache writes synchronous, which removed the
visible save delay but also removed any signal that the change was
actually persisted. Toast picks a per-field label from the patched
keys (Title updated / Date updated / etc.) and falls back to a count
for multi-field saves.
2026-05-11 22:29:25 +02:00
Claudio
abe5c1ec6b fix(metadata-panel): apply mutation responses synchronously, no second GET
Single-photo updateMutation only invalidated, so the panel waited for
a follow-up GET /photos/{id} round-trip before showing the new value —
felt as a 200–500 ms lag after every taken_at / rating / notes edit.
Use the PATCH response (already the updated row) to merge into the
per-photo cache and patch every cached timeline list in place.

Bulk taken_at had the same shape: invalidate-only, no optimistic. When
the user dropped back from N selected to one of the modified photos
the panel briefly showed the pre-edit value. Move both bulkSetTakenAt
and bulkSetTakenAtMap into useBulkPhotoMutations alongside the rating/
color/notes pattern, with the same snapshot+patch+rollback primitives.

Tags + bulk tags still invalidate-only — separate change if needed.
2026-05-11 21:51:14 +02:00
Claudio
ea08d7e3e8 fix(library/stats): exclude hidden photos from discarded count
The Discarded sidebar entry navigates to /photos?is_discarded=true, which
already excludes is_hidden=true rows (the cross-cutting hidden-folder
filter). The /library/stats discarded_count did not, so the badge could
disagree with the actual list — e.g. dtoro saw 1,281 in the badge but
only 25 in the view because the hidden Memories/ source root holds 1,256
discarded rows. Aligning the count with the view, like every other
sidebar badge already does.
2026-05-11 21:13:26 +02:00
Claudio
356062ead3 feat(date-guess): recognise YY-MM-DD HH-MM-SS Synology export filenames
The 0525.mov-style export from Synology Photos uses 2-digit years, which
the existing patterns ignored (all required \d{4}). Result: filename
gave no signal, suggestion fell through to the YYYY/MM folder layout and
snapped to day 15. The explicit HH-MM-SS half rules out random digit
triples, so we trust YY → 2000+YY for this specific shape and surface
the actual capture time, not noon.
2026-05-11 20:56:47 +02:00
Claudio
7a1c6b618b fix(compose): propagate SECRET_KEY to all workers, not just backend
The workers couldn't decrypt users.nextcloud_app_password_enc because
SECRET_KEY wasn't in their env. _credentials_for() then raised
NextcloudCredentialsMissing and our code swallowed it as "no NC
auth → fall back to local path."

Surfaced on the Phase 4 deploy when /data/thumbs/.../medium.webp
was purged and the vision worker had no disk fallback left. NC
preview fetch then returned None, the classifier got no image,
and the photo failed to classify.

Also masked Phase 3 silently — extract_metadata in worker-light
was falling back to ExifTool every time instead of hitting Memories
(which would have been fine because ExifTool produces the same
fields, but slower and unnecessary). With SECRET_KEY available, the
Memories primary path actually fires.
2026-05-11 13:56:31 +02:00
Claudio
5a67ed7e7b feat(phase 4): vision fetches NC previews; stop writing /data/thumbs
Last consumer of the on-disk thumbnail pipeline was the vision
worker reading /data/thumbs/{id}/medium.webp. Now it asks Nextcloud
for a 640px preview (the same edge size the old thumb used) and
decodes the bytes in-memory — no disk dependency.

- nextcloud_dav.get_preview_bytes: sync sibling of get_preview_async,
  for the celery vision worker (which is sync).
- vision._load_thumb: tries NC preview first; transitional disk
  fallback stays for rows still indexed during the rollout.
- thumbs.WORKER_THUMB_SIZES = set() — generate_thumbnails still runs
  the decode + pHash side-effect (perceptual dedup is mule-only and
  needs original-resolution pixels) but no longer writes thumbnail
  files.

The HTTP thumbnail endpoint's disk fallback path stays in place
unchanged: for NC-404 cases (e.g. iPhone JPEGs mis-extensioned as
.DNG), inline Pillow regeneration still writes a tiny per-photo
file so subsequent requests are fast. That path is rare and the
files are small.

Disk impact: /data/thumbs currently has ~22k medium.webp totaling
~1 GB. They'll stop being read after the worker-vision container
restarts, but no automatic delete — purge with the same find
pattern used for small/large reclaim when ready:

    find /data/thumbs -name "medium.webp" -delete
2026-05-11 13:52:52 +02:00
Claudio
f4618ddf97 fix(rename): refresh Folder.name on webhook-driven directory rename
handle_directory_rename updated Folder.path but left Folder.name as
the old leaf basename. Path is load-bearing; name is purely display,
but a stale name shows wrong text in the sidebar tree until the next
manual refresh. Now sets folder.name = basename(new_prefix) on the
renamed folder itself; descendants keep their existing names because
the rename was on an ancestor (only their paths shift).

Same correctness as the existing PATCH /folders/{id} endpoint, which
also updates both name and path.
2026-05-11 13:46:06 +02:00
Claudio
2a5759cc8d feat(metadata): read from NC Memories first, ExifTool subprocess as fallback
Phase 3 (fat refactor). extract_metadata now tries Memories'
HTTP API GET /index.php/apps/memories/api/image/info/{fileid}
before spawning ExifTool. Replaces ~80–100 ms of subprocess work
with a ~1–2 ms HTTP call for ongoing imports.

What we kept from the ExifTool path:
- Mule's date-fallback chain (SubSec → DateTimeOriginal → CreateDate
  → MediaCreateDate → TrackCreateDate → filename/folder guess → mtime).
  Memories' single `datetaken` field falls back to mtime, which would
  silently mis-date the 6k+ photos in our library that depend on
  filename-encoded dates. _apply_memories_metadata re-applies the
  same chain against Memories' `exif` dict.
- taken_at_source='manual' is still sacred — never overwritten.
- has_date_warning recomputed against the resolved taken_at.

Format compat: Memories' `exif` dict uses plain key names (Make,
Model, ISO, FNumber, DateTimeOriginal, GPSLatitude, ...) while the
old ExifTool path stored `EXIF:Make` etc. PhotoInfoPanel only reads
the four keys above and Memories has them in plain form, so the info
panel keeps working without an adapter. Full-text search (ILIKE on
exif_json) still hits camera names, lens names, dates etc. — value
content is identical, only the keys differ.

Fallback paths preserved:
- 404 from Memories (file not yet indexed by NC's scan, brand-new
  upload): falls through to ExifTool.
- non-NC photos (no nextcloud_fileid or no app password): ExifTool.
- NC HTTP error or parse failure: ExifTool.

CSRF: Memories' /api/image/info/{id} is CSRF-checked. We send
`OCS-APIRequest: true` to bypass it, the same way the OCS clients
do. Auth is the user's existing Fernet-encrypted app password.

Verified end-to-end against:
- IMG_4954.DNG (real DNG with GPS): width/height/lat/lon/taken_at
  match the previous ExifTool output exactly; exif_json switched
  to Memories format (Make/Model/ISO/FNumber preserved).
- 20210817_000000_4A6737B6.jpg (path-dated archive photo): taken_at
  remained 2021-08-17 from the filename heuristic, source='path'.

The `enabled` state of the Memories app is now required for new
imports to skip ExifTool — left enabled in commit 0a4c8d... (NC
admin action; not in this commit).
2026-05-11 13:39:43 +02:00
Claudio
f27f3cb820 fix(handle_directory_rename): iterate in Python — asyncpg rejected SUBSTRING(... FROM LENGTH(...)+1)
The raw-SQL prefix rewrite from f4a03b6 used
`SUBSTRING(filepath FROM LENGTH(:old_prefix) + 1)`. asyncpg's type
inference miscategorises the LENGTH() result and rejects the
parameter as "$2: int (expected str)" at execute time, so every
directory-rename webhook 500'd in production despite the surrounding
logic being correct.

Switch to the same per-row Python loop the existing PATCH
/api/v1/folders/{id} endpoint already uses. Folder renames are rare
and span ≤1k photos typically — the cost of N row UPDATEs is fine.

End-to-end verified:

  RenameTestA -> RenameTestA-FromNC (WebDAV MOVE outside mule):
    nc-webhook renamed (dir): {photos: 2, folders: 2, source_roots: 0}
    DB rows now at -FromNC ✓

  -FromNC -> -ViaMule (PATCH /folders/{id} inside mule):
    mule rewrites synchronously
    webhook fires back ~30s later
    nc-webhook renamed (dir): {photos: 0, folders: 0, source_roots: 0}
    idempotent no-op against an already-updated DB ✓
2026-05-11 13:16:56 +02:00
Claudio
f4a03b63f4 feat(nc-webhook): handle folder rename via NodeRenamedEvent
NC fires one NodeRenamedEvent on a directory rename — children don't
get their own events. The handler bailed on both paths having no
supported extension. Now:

- New `handle_directory_rename(old, new)` in scan.py does a single
  transaction of prefix-rewrites against photos.filepath, folders.path,
  and source_roots.path. Cross-source-root case (Photos/x → Memories/x)
  is treated as discard-the-old-subtree; scan_folder dispatched by the
  subsequent NodeWritten/NodeCreated picks up the new root.

- Webhook renamed branch checks "both source and target are
  directories" and calls the helper. File renames keep the existing
  delete-old + scan-new-parent path.

Idempotent: the SQL matches zero rows the second time around. That
makes the feedback loop safe — mule's existing PATCH /folders/{id}
endpoint already does a WebDAV MOVE + inline DB rewrite for NC paths,
and the resulting NodeRenamedEvent now flows back through this handler
without re-running the rewrite or leaving rows stale.

Trashbin restore (the documented "NC doesn't emit a subscribed event"
gap) is unchanged.
2026-05-11 13:07:28 +02:00
Claudio
94088253f8 fix(nc-webhook): propagate folder deletes + resurrect un-discarded files
Two bugs surfaced by the Phase 2 deletion-roundtrip test:

A) Folder delete in NC only fires one NodeDeletedEvent (for the folder
   itself, no .jpg suffix). The handler bailed with "unsupported
   extension" and photos inside the folder kept is_discarded=false in
   mule until the 30-min discard_missing_photos_beat caught up.

   Fix: when the deleted path has no supported extension, call new
   `handle_directory_deletion()` which UPDATEs every Photo whose
   filepath starts with `dirpath + '/'`. Single SQL statement,
   idempotent (excludes already-discarded rows so re-deliveries don't
   re-stamp discarded_at).

C) PUT-overwrite of a previously-discarded file fired NodeWrittenEvent
   → scan_folder, but scan_folder's "Photo exists by filepath, skip"
   branch left is_discarded=true. File was back on disk; mule still
   treated it as gone.

   Fix: in that branch, if the existing row is discarded, flip
   is_discarded=false + clear discarded_at + re-queue extract_metadata
   so EXIF / nextcloud_fileid pick up any changes to the bytes.

Together these close the gap for "delete then put back" round-trips
via the NC webhook path. Trashbin-restore (bug B in the test report)
remains an NC-side gap — NC doesn't emit any event mule subscribes to
for restore-from-trash. That stays a TODO.
2026-05-11 12:50:52 +02:00
Claudio
f657e2c0ba feat: retire the watchfiles watcher in favour of NC webhooks
End-to-end webhook flow is proven on this NC instance (NodeCreated +
NodeWritten both fired and dispatched scan_folder on a PUT test), so
the watchfiles-based polling layer is no longer needed.

- scanner.start_initial_scan no longer queues watch_folders on boot.
- scan.watch_folders kept as a one-line no-op shim so any leftover
  apply_async in flight from the previous deploy doesn't crash a
  worker. Will be deleted entirely after the queue drains.
- celery.py reroutes watch_folders to the `default` queue (worker-light)
  so the no-op shim actually completes — the `watcher` queue is dead.
- docker-compose drops the mulita-worker-watcher service. Its celery
  --beat responsibility (firing discard_missing_photos_beat every 30
  min) moves to worker-light's command.

Latency note: NC dispatches webhook events through its background-job
queue, currently run by cron */5. After this commit lands you'll want
to tighten cron to */1 so new uploads land in mule within ~60s instead
of up to 5 min.
2026-05-11 12:28:36 +02:00
Claudio
362fbc6d83 feat(nc-webhook): receive Nextcloud file events instead of polling
The watchfiles-based watcher works but duplicates Nextcloud's own
notion of "this file changed." NC has a webhook_listeners app that
can POST file events to an external URL. This adds the mule side of
that handshake.

- POST /api/v1/internal/nc-webhook authenticates a Bearer token
  (NEXTCLOUD_WEBHOOK_SECRET, hmac.compare_digest) and dispatches the
  same scan_folder / handle_file_deletion machinery the watcher used.
- Handles NodeCreated, NodeWritten, NodeDeleted, NodeRenamed.
  Renamed is mapped to delete-old + scan-new-parent. Maps NC's
  /admin/files/... path to the bind-mounted /nextcloud-users/admin/files/...
- backend/scripts/register_nc_webhooks.py is the idempotent
  registrar: lists existing webhooks, deletes any pointing at the
  target URL, then POSTs four fresh ones via OCS.
- Sets the env passthrough on backend + all workers in compose so
  the same secret is available wherever the registrar might run.

watch_folders stays in place for now — webhooks become primary, the
watcher is a belt-and-suspenders fallback. Drop the watcher in a
follow-up once webhooks are proven reliable on this NC instance.
2026-05-11 12:19:59 +02:00
Claudio
d24c64e0a0 fix(scan): stop auto-queuing backfill_gps on every startup
`_scan_all_source_roots_async` unconditionally dispatched backfill_gps
30s after each container boot. backfill_gps then queued one
extract_metadata task for every photo where latitude IS NULL — which is
most of the library (screenshots, indoor shots, scans, anything without
GPS in EXIF). The result was ~60k extract_metadata tasks piling onto
the default queue at every deploy, pinning worker-light at 180+% CPU
for ~30 min while it re-derived metadata that wasn't going to change.

The "scanned-before-the-GPS-fix" rationale in the original comment
hasn't applied for many releases. Manual trigger via
POST /api/v1/library/backfill-gps is preserved for the rare case where
the extractor really did change.
2026-05-11 12:08:31 +02:00
Claudio
18dce33fa3 fix(original): support HTTP Range so <video> can play .mov etc
`GET /api/v1/photos/{id}/original` returned 200 with the full body for
every request, even ones with a Range header. Browsers refuse to play
<video> they can't seek and surface the failure as "format not
supported" — most visible on .mov / .mp4 over 5–10 MB.

Now parses `Range: bytes=START-END` (and bytes=-N for the tail), emits
206 with Content-Range, streams the slice in 1 MB chunks. Full body
responses advertise Accept-Ranges so the browser knows to retry with a
Range on the next request.

Single-range only — multipart/byteranges is rare in practice and not
worth the complexity.
2026-05-11 12:02:44 +02:00
Claudio
28738acb56 fix(backfill): drop offset-based pagination — it skipped filled rows
The offset+limit loop walked the IS NULL set, but every batch's writes
shrank that set, so batch N+1 with offset=N*BATCH skipped over the rows
just filled. A 17k library backfilled only 9k before the loop walked
off the (now-shorter) NULL set.

Replace with a tail-recursive pattern: keep selecting LIMIT BATCH on
the NULL set, tracking rows that won't ever resolve in a `stuck` set so
the loop terminates instead of spinning on them.
2026-05-11 11:43:03 +02:00
Claudio
576b0c236d feat(thumbs): proxy Nextcloud previews instead of duplicating the cache
mule-image was generating and storing three WebP sizes per photo in
/data/thumbs while Nextcloud already keeps its own previews for the
same source files. Frontend thumbnail requests now proxy NC's
/index.php/core/preview keyed by the photo's Nextcloud fileid,
authenticated with the owner's encrypted app password.

- new column photos.nextcloud_fileid (alembic 0018) plus an index
- get_preview_async + fetch_fileid helpers in nextcloud_dav.py
- thumb route proxies NC primary, falls back to /data/thumbs (legacy
  rows / NC unreachable) so a single-file revert restores the old path
- extract_metadata caches the fileid on first run for new photos
- generate_thumbnails now writes only medium since the vision worker
  still loads it from disk; small + large drop out of the worker path
- backend/scripts/backfill_nextcloud_fileid.py for one-shot population
  of existing rows: docker exec mulita-backend python -m scripts.backfill_nextcloud_fileid

X-Mule-Thumb-Source response header marks each request 'nextcloud' or
'disk' for observability while the rollout settles.
2026-05-11 11:34:58 +02:00
Claudio
9e9b1ba224 perf(preview): debounce full-res /proxy preload by 400ms
Rapid arrow-nav was firing one /proxy fetch per photo with no way to
abort (new Image() has no abort). Holding the right arrow through ten
photos in two seconds left ten multi-MB transfers in flight competing
for bandwidth and the RAW/HEIC transcoder. Now the preload only kicks
in if the user lingers on a photo for 400ms; otherwise the timer is
cleared and no /proxy request is made.
2026-05-11 11:06:36 +02:00
Claudio
11202a92e7 fix(preview): cull/pick shortcuts target the visible photo, not stale selection
Arrow nav inside preview only updates activePhotoId; selectedPhotos still
points at whatever was selected in the grid before opening preview. X and
S therefore fired against the wrong photo — the toast appeared but the
filmstrip tint for the currently-viewed photo never changed because that
photo was not the cull target.

cullTargets() (and togglePickOnSelection, now sharing it) now prefer
activePhotoId when viewMode === preview.
2026-05-11 10:47:02 +02:00
Claudio
a0d275490b ui(preview): preload thumbs for ±5 neighbors, not just adjacent 2026-05-11 10:32:14 +02:00
Claudio
6311412fc0 ui(preview): load 1280px thumb first, upgrade to full-res in bg
The /proxy endpoint is slow on first hit, especially for RAW/HEIC where it
transcodes synchronously. Preview now renders the pre-generated large thumb
immediately, then preloads /proxy via Image() and swaps src when ready, so
zoom (Z key / wheel) still reaches the original pixels.
2026-05-11 10:22:17 +02:00
claudio
611d445d92 ui(sidebar): ellipsize nextcloud-users/<user>/files prefix in path 2026-05-11 09:51:45 +02:00
Claudio
f14ea69223 ui(duplicates): show grandparent + parent in path strip
When two duplicates live in folders with the same parent name (e.g.
matching '2023' subfolders under different archives), showing only
the parent gave both thumbnails the same label. Walk one level up:
the path strip now renders '…/<grandparent>/<parent>' so the user
can always tell two copies apart at a glance. Filename still
surfaces via the title tooltip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 00:49:02 +02:00
Claudio
f290784bf3 ui(duplicates): show parent folder + full-path tooltip on each thumbnail
Two copies of IMG_1234.jpg sitting in different folders looked
identical on the duplicates grid — same filename, same dimensions,
same Best heuristic. The user had no way to pick which copy to keep
without opening each in the preview overlay.

Backend: include filepath in the per-member payload from
GET /api/v1/library/duplicates/groups (was filename-only).

Frontend: a black 65% strip at the bottom of every duplicate
thumbnail showing the parent folder name (the actual discriminator
when filenames match), with the full filepath surfaced via the
native title tooltip on hover. The dimensions chip moves from
bottom-left to top-left so the bottom strip can run edge-to-edge.

memberToPhoto stops faking filepath=filename (a years-old workaround
that broke any code path needing the real path); the synthetic Photo
the grid hands to PhotoThumbnail now carries the real filepath.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 00:16:48 +02:00
Claudio
f743733edd ui(sidebar): drop Title field, add bulk notes editor
The Title (user_title) field hadn't earned its place in the sidebar
form — the underlying column stays on the model but the editable
row + its draft state + commit handler are gone.

Bulk Notes: a textarea in the multi-photo bulk panel that replaces
user_notes across the whole selection with one string. Apply commits;
Clear empties the draft without committing. New backend bulk action
'set_notes' validates the value is a string (or null/empty to clear)
and writes to every photo in the selection in one go. Wired through
the standard useBulkPhotoMutations optimistic-patch path, so the
photo cache flips immediately and rolls back on error.

user_notes added to the shared Photo type so patchPhotos accepts the
field; previously it was only on PhotoInfoPanel's local PhotoDetails.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 00:07:02 +02:00
Claudio
c69322a89d ui(sidebar): compact two-section panel, drop active-heap card
The right sidebar had three top-level blocks (ActiveHeapCard + Header
title strip + scroll region with two parallel collapsibles 'Edit' and
'Metadata'). Three nested Section sub-collapsibles inside Metadata
added another row of chevrons per group. A lot of chrome for what is
fundamentally one form per photo.

Refactor:

- RightSidebar: remove ActiveHeapCard import + both usages
  (empty-selection branch and single-photo branch). Single-photo
  branch also drops the redundant Header strip; the new Metadata
  collapsible's trigger IS the visible section title. Multi-photo
  branch keeps Header (still needs 'N Photos Selected').

- PhotoInfoPanel: collapse the Edit and Metadata-with-sub-Sections
  structure into two flat collapsibles. Metadata holds readonly facts
  (Size / Dimensions grid, Path, GPS inlined) and the editable form
  (Filename, Title, Date Taken, Notes, Tags, Rating + Color on one
  row, Flag), separated by a thin horizontal rule. Camera lives in
  its own collapsible at the bottom so a long EXIF block can't crowd
  the form. Default expanded set narrows to ['metadata', 'camera'].

- Compact density: Notes rows=3 -> rows=2, rating + color share a
  row, stars/swatches shrink h-5/w-5 -> h-4/w-4, space-y-2.5 -> 2,
  Flag buttons text-sm -> text-xs, grid gaps tightened. The empty
  'No GPS data' chip is hidden when there are no coordinates rather
  than rendered as an empty row.

- Drop the unused local Section helper and the now-orphan
  ActiveHeapCard.tsx file. Active-heap state stays in the store; the
  Select / Discard buttons inside the form still consult activeHeap
  on click.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:43:33 +02:00
Claudio
f63daf16a8 fix(timeline): scroll to top on section / folder switch
Clicking a folder in the sidebar (or any section change) didn't
reset the timeline's scroll position. If the user was scrolled deep
into All Photos, the new folder loaded at the same y-offset, often
landing on empty space below the last row.

The section-change effect already cleared selection and reset the
auto-focus guard; just needed to also reset parentRef.current.scrollTop.
Synchronous so the first paint of the new section is anchored at
photo[0]; the auto-focus selectPhoto call still runs after render
to highlight the first photo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:25:48 +02:00
Claudio
d580796dc8 fix(photos): accept bare-date filter bounds, send T00:00:00 from UI
GET /api/v1/photos rejected ?date_from=2026-04-10 with 422 because
pydantic v2's datetime parser doesn't accept date-only strings. The
frontend has been padding date_to with T23:59:59 forever to make the
upper bound inclusive, but date_from went out as a bare YYYY-MM-DD,
so every date-range filter request 422'd and the grid showed nothing.

Frontend: pad date_from with T00:00:00 the same way date_to gets
T23:59:59 — symmetry, and pydantic v2 accepts the full form.

Backend: change date_from/date_to to Optional[str] and parse with
datetime.fromisoformat in the handler. fromisoformat accepts both
bare dates ('2026-04-10' -> midnight) and full ISO strings, so any
older client that still sends a date-only value continues to work.
Tz-aware values get coerced to naive UTC before binding (matches the
taken_at column's  shape and the same
fix applied to PATCH /photos/{id} earlier today). Bad input returns
400 with a clear message instead of pydantic's 422.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:16:09 +02:00
Claudio
7b153f0d28 fix(metadata): drop EXIF:ModifyDate fallback, prefer SubSec, fall back to path
The taken_at extractor walked four EXIF fields in order: DateTimeOriginal,
CreateDate, MediaCreateDate, ModifyDate. The last one is set every time
a file is re-saved (Lightroom export, EXIF strip, batch resize), so any
photo whose original capture metadata was lost during editing ended up
labeled 'exif' with the *edit* date instead of the shoot date.

Changes:
  - SubSecDateTimeOriginal at the top of the list (sub-second precision,
    often carries OffsetTime).
  - QuickTime:CreateDate added next to MediaCreateDate.
  - ModifyDate dropped from the trusted list entirely.
  - When no trusted EXIF date is present, fall back to guess_date_from_path
    (already used for has_date_warning) and tag taken_at_source='path'.
    Better than filesystem mtime, which on Nextcloud-mounted libraries
    just reflects the upload time.
  - Skip the date-write block entirely if photo.taken_at_source == 'manual'
    so a rescan can't clobber a user correction.
  - parse_exif_datetime: handle the all-zero placeholder some cameras
    emit, accept tz-aware variants (%z), normalize to naive UTC.

Frontend: new 'PATH' badge in TakenAtEditor with a tooltip explaining
the date came from filename / folder rather than real EXIF.

Backfill: new backfill_taken_at celery task and
POST /api/v1/library/maintenance/backfill-taken-at endpoint that
re-enqueues extract_metadata for every non-manual photo. ~21k tasks
finish in ~15 min on the existing worker-light concurrency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:02:32 +02:00
Claudio
76551d898b fix(nextcloud): NULL parent_id on cross-source-root child folders + spinner
The previous fix NULLed parent_id only for folders within the
SourceRoot being deleted, but folder rows under a *different*
SourceRoot whose path nests inside this one (e.g. 'Leóns 1st Year' at
.../Taco and Muli - 2024 onward/Leóns 1st Year) still pointed into
our delete set. folders_parent_id_fkey kept tripping. Widen the UPDATE
to NULL parent_id for any folder whose parent_id is in folder_ids,
regardless of source_root_id.

UI: trash button on a Nextcloud library now swaps to a spinning
Loader2 while the delete is in flight (only the row being deleted —
others stay as trash icons but disabled). Title updates to flag
that a cascade through every photo + folder can take a few seconds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:37:12 +02:00
Claudio
89f99d220a fix(photos): coerce tz-aware taken_at to naive UTC before DB write
PATCH /api/v1/photos/{id} returned 500 with
'can't subtract offset-naive and offset-aware datetimes' when the
frontend sent a tz-aware taken_at value (e.g. 2026-05-09T00:12+02:00).
The photos.taken_at column is timestamp without time zone, so asyncpg
refuses to bind a tz-aware datetime.

The frontend's datetime-local input is supposed to be naive but real-
world locales / browsers / paste flows occasionally include offsets.
Normalize on the server: if tzinfo is present, convert to UTC and drop
the tzinfo so both shapes round-trip cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:32:44 +02:00
Claudio
63dd39d172 fix(nextcloud): NULL parent_id before deleting Folder rows on SourceRoot remove
Folders have a self-referential parent_id FK with no ON DELETE rule.
A flat DELETE of the whole subtree trips folders_parent_id_fkey because
postgres checks the constraint per-row regardless of insertion / list
order. Hard-removing 'Taco and Muli - 2024 onward' (35-folder subtree)
returned 500 with ForeignKeyViolationError every attempt.

Fix: UPDATE folders SET parent_id = NULL WHERE id IN (folder_ids) before
the DELETE so the chain is broken cleanly. Same pattern used in
prune_missing_photos for the same constraint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:30:34 +02:00
Claudio
09a00f7419 feat(nextcloud): hard-delete SourceRoot + reliable delete sync
Two related fixes for the Nextcloud library lifecycle.

1. DELETE /api/v1/nextcloud/source-roots/{id} now actually deletes
   the SourceRoot, every Folder under it, and every Photo in those
   folders (Nextcloud files untouched). Was a soft-deactivate
   (is_active=false) that left the rows around forever, so re-adding
   the same path resurrected ghosts and prune-missing reported zero.
   Returns {deleted_photos, deleted_folders}; the Settings UI toasts
   the count and invalidates photos/folders/stats so cached lists
   don't show ghosts. photo_tags and heap_photos already cascade via
   ON DELETE CASCADE; FolderShare uses a stringly-typed folder_id
   with no FK so cleaned up explicitly.

2. The watcher (watch_folders task) was getting killed every five
   minutes by the global task_soft_time_limit=300 in app/tasks/celery.py
   despite passing soft_time_limit=None on the decorator (None falls
   back to the worker default in this Celery version). Override with
   soft_time_limit=0, time_limit=0 (= unlimited) so the watch loop
   actually stays alive. The 'Soft time limit (300s) exceeded' /
   'Worker exited prematurely' lines should stop in worker-watcher
   logs.

3. Added discard_missing_photos() in services/cleanup.py — a soft
   variant of prune_missing_photos that walks every present source
   root, checks os.path.exists for each non-discarded Photo, and
   flips is_discarded=true on the missing ones (UPDATE not DELETE).
   Wired as discard_missing_photos_beat in tasks/scan.py and
   scheduled every 30 min via celery beat. Beat runs in-process on
   worker-watcher (--beat flag in compose) — there's only ever one
   watcher and we don't need a separate container.

Hard delete remains manual via prune-missing for users who want to
review before committing. The beat catch-up only soft-discards (file
gone -> mule-image trash, restorable).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:19:58 +02:00
Claudio
99d504842e feat(auth): auto-redirect to Authentik when OIDC enabled
Even when the user has a live Authentik session, hitting
photos.hubris.network used to drop them on the LoginPage with a 'Sign
in with Authentik' button they had to click manually. With OIDC set
up for a single trusted IdP that's friction with no upside.

LoginPage now reads /auth/config on mount and, if OIDC is enabled,
immediately navigates to the OIDC login URL. Authentik recognizes
the existing session and bounces the browser back through the
callback signed in — no clicks needed.

Two escape hatches so the user is never stuck:
  - ?password=1 in the URL forces the password form
  - sessionStorage 'skipAutoSso' flag, set by the logout flow and by
    the OIDC callback's error branch, suppresses the next auto-redirect
    so logouts actually log out and OIDC failures surface their error
    instead of looping straight back to the IdP

While the redirect is in flight we show 'Signing in with Authentik...'
plus a small 'Use password instead' link, so users on a slow or
broken IdP connection aren't left staring at a spinner.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:59:15 +02:00
Claudio
eeeb16a0f1 ui(sidebar): split editable vs read-only between Edit and Metadata
The Metadata collapsible was hosting two editable widgets (TagsEditor
and TakenAtEditor) buried inside the readonly sub-sections — Tags as
its own Section, taken-at wedged into Basic Info between size/dims
and the filepath. With both top-level collapsibles in place, the
clearer split is editable up top, readonly below.

Moved into the Edit collapsible (in identification → description →
categorization order):
  Filename, Title, Date Taken, Notes, Tags, Rating, Color, Flag

Metadata now holds only readonly sub-sections:
  Basic Info (size, dims, path), Camera, Location

Dropped the now-empty Tags Section from Metadata and the 'tags' key
from the default-expanded set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:49:44 +02:00
Claudio
1695eae226 ui(sidebar): wrap edit form in collapsible, drop Header X button
Mirror the Metadata collapsible: an 'Edit' wrapper around filename,
title, notes, rating, color, and flag so the editable form is hidden
with one click. Default expanded.

Drop the clear-selection X from the panel Header — Esc still clears
selection and grid clicks do too. The X felt out of place once the
panel restructured around two equal collapsible groups (Edit /
Metadata) below a plain title bar.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:43:33 +02:00
Claudio
172f869e15 ui(sidebar): pin heap, single scroll area, collapsible metadata group
The right sidepanel had three stacked flex regions (heap card +
header + PhotoInfoPanel) with PhotoInfoPanel maintaining its own
internal scroll. That made the editable fields (filename, title,
notes, rating, color, flag) stick at the top — separate from the
readonly metadata that scrolled below. Effectively two scroll
boundaries on one sidebar.

Move the scroll boundary up so only ActiveHeapCard + Header stay
pinned; editable fields and readonly metadata now scroll together.
Wrap the four readonly sections (Tags / Basic Info / Camera /
Location) in a single outer 'Metadata' collapsible so the user can
hide the whole block with one click. Sub-sections inside stay
individually collapsible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:36:18 +02:00
Claudio
1408ec3fa3 perf(db): partial index for the photos list query
The default photos list (GET /api/v1/photos?per_page=N&sort=taken_at&order=desc)
filters NOT is_trashed AND NOT is_hidden and sorts by
(taken_at DESC NULLS LAST, id DESC). EXPLAIN on the 21k-row table
shows a seq-scan + top-N heapsort (~20ms standalone, multiplied under
concurrent fan-out on page load). The existing single-column
ix_photos_taken_at can't be used because the leading WHERE clause is
two booleans.

Partial index over the sort key, restricted to the visible subset.
Lets the planner index-scan in reverse and stop at LIMIT N.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:28:28 +02:00
Claudio
b1c3ee68dd perf(ui): smaller initial page, slower idle polling
Photos grid was fetching per_page=500 on the very first request, which
serialized hundreds of thumbnail requests behind a single sort+payload.
Split into PER_PAGE_INITIAL=100 (one viewport, fast paint) and
PER_PAGE_BACKGROUND=500 (subsequent prefetch pages, fewer round-trips).

Idle polling for scan-status and worker-status was set to 10s / 15s
respectively. With nothing queued the typical session was firing 4–6
status requests every minute through the single uvicorn event loop on
top of everything else. Bumped both to 30s. While actively scanning /
processing the 2s / 3s cadence is unchanged — that's where the user
actually wants live updates.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:28:16 +02:00
Claudio
4bb2c959a8 fix(cleanup): distinguish renamed source root from unmounted drive
prune_missing_photos previously skipped every photo whose source root
path didn't resolve, on the assumption that a missing path meant the
underlying drive was unmounted (and silently deleting under those
conditions would be data loss). That conflated 'drive unmounted'
with 'user renamed the folder in their file manager'.

A library with 4,154 orphaned photo rows from a since-renamed Nextcloud
folder hit exactly this case: the /nextcloud-users mount was fine, but
the source root path 'Taco and Muli - 2024 onward' no longer existed
because the user had renamed it to 'Photo Archive 2004-2024'. Every
photo under it was reported as skipped_unmounted forever.

Classify source root state as present/renamed/unmounted by checking
whether the immediate parent is readable. 'renamed' is now treated as
prunable; 'unmounted' still skips. Warning messages differ so the user
knows which fix to apply.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:24:28 +02:00
Claudio
758fda619e fix(raw): PIL fallback for iPhone Apple ProRAW / Linear DNG
LibRaw (rawpy 0.26.1, libraw 0.22.0) rejects Apple ProRAW Linear DNG with
'Unsupported file format or not RAW file'. These files aren't Bayer-pattern
RAW — they're TIFF containers holding an already-developed RGB image, so
PIL opens them directly. iPhone Linear DNG also has no embedded preview
exiftool can extract, so the existing fallback chain ran out of options.

Added PIL Image.open(src_path) as the last fallback in both code paths
(_generate_proxy_webp for /photos/{id}/proxy, and tasks.thumbs.process_raw_image
for thumbnail generation). Covers ~1,300 iPhone DNG files in the library
that were 415-ing on every detail view.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:20:53 +02:00
Claudio
0eee0cecde perf(backend): drop uvicorn --reload, run 2 workers in compose
Production runs were on the dev --reload single-worker config. The frontend
fans out ~15 parallel API calls on first paint (folders/tree, tags, heaps,
sharing/*, stats, photos, worker-status, scan/status); they all serialized
on one event loop and felt slow. Switch to 2 workers without --reload for
real concurrency. --proxy-headers preserved client IPs through nginx.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:09:51 +02:00
Claudio
4b137989c6 fix(auth): drop competing 401 interceptor in api.ts
Two response interceptors were stomping on each other:

  1. api.ts (this file, registered at module import) — on 401, set
     original._retry = true, removed access_token from localStorage,
     and rejected. The comment claimed it relied on a "scheduled
     refresh in AuthContext" that does not exist in the codebase.
  2. AuthContext useEffect — proper refresh: POST /auth/refresh, swap
     both tokens, retry the original request.

Axios runs response interceptors in registration order, so api.ts ran
first and pre-emptively burned the _retry flag + access_token before
AuthContext could see the 401. Result: every expired-token request
forced a re-login instead of a silent refresh.

Drop api.ts's response interceptor entirely. AuthContext owns the
refresh dance; the request interceptor here just attaches the bearer.

Companion bump in .env (gitignored): ACCESS_TOKEN_EXPIRE_MINUTES=10080
(7 days), REFRESH_TOKEN_EXPIRE_DAYS=365 — homelab posture, fewer
refresh round-trips per session even when the silent refresh works.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 11:14:45 +02:00
Claudio
f811aae641 fix(sidebar): show Settings entry to non-admin users
Companion to 4c7e981 — the SettingsPage was opened to non-admins but
the LeftSidebar still gated the entry button on isAdmin, so non-admins
had no way to reach it. The page itself is the source of truth for
which tabs and controls are visible per role.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 09:59:24 +02:00
Claudio
e4127f1e04 fix(library): scope source_dirs to current user in /stats
Without this, a non-admin hitting /api/v1/library/stats would see
every other user's active SourceRoot path in the response (e.g.
muli would see /nextcloud-users/admin/files/Photos). Cross-user
visibility into Nextcloud paths is a small info leak in a multi-user
setup. Admins still get the global list when they pass scope=global.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 09:46:54 +02:00
Claudio
4c7e981daf feat(settings): open Library + AI tabs to non-admin users
Non-admins now see Library and AI Features tabs with data scoped to
themselves; only Users (admin management) stays admin-only.

Library tab: queries pass scope=global only when isAdmin, otherwise
omit scope so the backend _owner_filter falls back to current_user.
Stats, worker status, pipeline progress, duplicates, regenerate-thumbs
all respect this. Re-scan + maintenance buttons that hit user-scoped
endpoints continue to work for non-admins.

AI Features tab: feature flag state read via the public /features
endpoint for non-admins (just effective values, no override metadata),
admin-only flag toggle Switches show as disabled with an explanatory
tooltip, and the "Manual pipeline triggers" section (bulk classifier
backfill + rescan-all-source-roots) is hidden entirely for non-admins
since those are admin-bulk operations across every user.

Users tab: stays adminOnly as today.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 09:45:21 +02:00
Claudio
65f6c14487 fix(nextcloud): pin cloud.hubris.network to LAN caddy IP in compose
Without this, the docker default resolver forwards the lookup to the
host gateway, which returns the public IONOS VPS IP. cloud.hubris is
not in the VPS traefik exposure list, so TLS handshakes during
WebDAV calls die with httpx.ConnectError: SSL UNEXPECTED_EOF.

extra_hosts pins it to caddy on 192.168.8.175, which holds the
cloud.hubris.network cert and proxies to the Nextcloud LXC. Applied
to every service for symmetry; only backend currently makes the
WebDAV calls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 08:17:52 +02:00
Claudio
bc0bb44c05 feat(nextcloud): per-user Nextcloud library integration
Lets each mule-image user (matched via OIDC preferred_username,
overridable in Settings) browse their Nextcloud files/ tree from the
mule-image UI and register subfolders as per-user SourceRoots. Reads
stay direct on the bind-mounted /nextcloud-users path; mutations
(upload, delete, rename, move within NC) dispatch through Nextcloud
WebDAV so oc_filecache, trashbin, comments, and desktop-sync clients
stay coherent.

Backend:
- users.nextcloud_username + nextcloud_app_password_enc (Fernet at rest,
  key derived from SECRET_KEY) — alembic 0016
- services/nextcloud_dav.py: minimal WebDAV client (PUT, MKCOL, DELETE,
  MOVE) with HTTP Basic auth via the per-user app password
- routers/nextcloud.py: GET /browse, /whoami, GET/POST/DELETE
  /source-roots (path-scoped to current_user.nextcloud_username with
  realpath traversal guard)
- PATCH /api/v1/auth/me to update nextcloud_username and app password
- OIDC callback defaults nextcloud_username from preferred_username on
  first login; backfill on existing users; never overwrites a manual
  override
- routers/upload.py: stream upload to NamedTemporaryFile, then PUT to
  WebDAV (with MKCOL chain) when destination is NC-rooted; existing
  Photo row creation runs unchanged
- routers/discard.py empty-trash: WebDAV DELETE for NC files
- routers/photos.py rename + move: WebDAV MOVE for NC paths;
  cross-system move/copy returns a clean error
- routers/folders.py rename + create + permanent-delete: dispatch via
  WebDAV when targeting NC-rooted paths

Frontend:
- AuthUser carries nextcloud_username + has_nextcloud_app_password
- services/api.ts: nextcloud + account namespaces
- components/dialogs/NextcloudFolderPicker.tsx: lazy tree browser, name
  + submit -> POST /source-roots
- SettingsDialog: new "Nextcloud library" card with username override +
  validate, app-password input, list/remove of NC libraries, and the
  picker entry point

docker-compose.yml: NEXTCLOUD_USERS_HOST_PATH bind to /nextcloud-users
on backend + 3 workers; NEXTCLOUD_USERS_ROOT + NEXTCLOUD_BASE_URL env.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 01:06:37 +02:00
root
80dd9d0a8b feat(auth): OIDC link by preferred_username (opt-in)
Adds OIDC_LINK_BY_USERNAME as a last-resort linking step after
(issuer, sub) and email both miss. Matches IdP preferred_username
against users.username.

Why: local accounts created before OIDC never collected an email
(no UI for it), so the email fallback cannot relink them. A new
SSO login therefore falls into JIT and creates username-1. On a
single-tenant homelab where the IdP owns the namespace, matching
by username is safe and makes first-time SSO transparent for
pre-existing users. Gated behind a flag so multi-tenant deployments
keep the stricter default.
2026-04-22 22:24:13 +02:00
e8e1adcf37 feat(auth): Authentik OIDC sign-in + Gravatar avatars
Adds optional SSO via Authentik (or any OIDC provider) alongside the
existing password flow, and pulls profile images from the provider's
`picture` claim or Gravatar so the sharing UI stops looking anonymous.
Password login stays available as a recovery path; JIT provisioning and
admin-group mapping are env-configurable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 21:06:32 +02:00
319be20389 feat(sharing): pending-state invites, notification bell, sidebar polish
Shares used to activate instantly on the owner's side with no notice to
the recipient. Introduce a pending/accepted lifecycle so a recipient
gets a bell notification on login and explicitly Accept or Decline
before the shared item lands in their sidebar.

Backend
- Migration 0014 adds `status` + `accepted_at` to heap_shares and
  folder_shares; pre-existing rows are backfilled to 'accepted' so
  nothing disappears from anyone's current sidebar. One-migration trick:
  server_default 'accepted' during add_column, then strip so new inserts
  fall through to the Python model default 'pending'.
- New recipient-only endpoints: POST /sharing/{heaps|folders}/{id}/accept
  (idempotent) and /decline (hard delete, so re-invites are clean).
- New GET /sharing/pending returning {heaps, folders} of outstanding
  invites with target_name + owner_username + permission.
- list_shared_{heaps,folders} now filter to status='accepted' and carry
  share_id so the recipient can Leave without a second lookup.
- ShareResponse exposes status so the owner sees pending invites.

Frontend
- NotificationBell lives in the LeftSidebar user row: a Popover
  triggered by Bell with a count badge. Each row shows owner avatar,
  "{owner} shared {heap|folder} {name}" with a permission subtitle,
  and Accept / Decline inline. Polls /sharing/pending every 60s.
- Shared Avatar helper extracted to sharing/Avatar.tsx — used by
  ShareDialog, NotificationBell, and the sidebar shared rows so one
  user's identity colour is stable everywhere.
- Sidebar shared-row polish: owner avatar bubble + Eye/Pencil
  permission icon (was uppercase pill). Right-click opens a context
  menu with Open / Leave; Leave calls the existing recipient-revoke
  DELETE and invalidates the shared-{heaps,folders} query.
- ShareDialog shows an amber "Invited" pill next to pending recipients.
- New shadcn context-menu primitive (radix dep already installed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 23:17:53 +02:00
11343c17dc ui(sharing): rework share dialog around the Drive/Notion pattern
The previous pass still read as two labeled sections with a target
"chip" that looked like an empty input and a dashed-border empty state
that looked like a drop zone. Rebuilt around the common share-modal
pattern: target name inlines into the title, a single compact invite
row (picker + Viewer/Editor dropdown + Share) sits at the top, and a
hoverable list below shows each person with an avatar, name,
permission subtitle, and an X that fades in on hover.

Also fixes the spacing: DialogContent was p-5 with non-flex children
so the gap utility silently did nothing — switching it to a flex
column puts every section on a 16px rhythm.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 22:34:29 +02:00
3c022cef68 ui(timeline): auto-focus first photo on every section switch
The previous auto-focus guard was one-shot for the lifetime of the
component, so switching from All Photos → Discarded (or any other
filter-based section) carried over the old activePhotoId — and if it
wasn't in the new view, nothing was focused at all. A new effect
watches currentSection and, on any change (or fresh mount after a
Duplicates/Memories detour), resets the guard and clears the stale
selection so the existing auto-focus picks the first visible photo of
the new view.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 22:21:29 +02:00
b6be24c357 ui(sharing): redesign share dialog with clearer structure + affordances
Target is now anchored in a chip at the top instead of a floating line.
Existing shares and the add-user form are split into labeled sections
with states for loading / empty. Each share row gets a hash-tinted
initial avatar and a semantic permission pill (primary = edit, muted =
view). The user picker is full-width with avatars in the dropdown, and
permission becomes a segmented "Can view / Can edit" control alongside
an icon-labeled Share button.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 22:15:44 +02:00
b518a293cd fix(duplicates): reserve 160px row height so thumbnails don't shift on load
The group grid only pinned column width; rows defaulted to auto height,
so each cell collapsed to the size of its still-empty <img> and snapped
to 160px once the thumbnail arrived — visible layout jump, plus the
virtualizer re-measured every group on image load.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 22:15:38 +02:00
b7f2eb7832 ui(timeline): drop redundant tints in discarded + active-heap views
When the grid is already filtered to discarded photos or to the active
heap, every cell would carry the same tint — the grayscale wash or the
green overlay stopped signalling anything and just made thumbnails
harder to read. Timeline now suppresses both when the corresponding
filter is active; the BR icon badges stay for colorblind readability.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 21:56:07 +02:00
96624bf853 fix(duplicates): hoist memo hooks above early returns
Rules-of-hooks violation: useRef and three useCallbacks sat after the
isLoading/isError/empty early-return block, so first render (loading)
called N hooks and the post-data render called N+4, crashing the view.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 21:49:32 +02:00
b9916866b3 ui(footer): move "Built with hubris" byline into LeftSidebar
Pulled the hubris/Roman-year line out of the TopBar and into a new
Footer component rendered below the Settings button in the left
sidebar bottom panel, where it reads as a quiet attribution rather
than competing with the title plate up top.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:11:00 +02:00
68d8a6d064 fix(sidebar): exclude watch_folders heartbeat from active task count
The watcher worker reports its periodic watch_folders task as
perpetually active, which kept the sidebar background-activity
spinner running even when no real work was in flight.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:31:06 +02:00
e6ca78881f ui(layout): move Date filter inputs to topbar, Active Heap to right sidebar
- FilterBar: new Date pill hosts from/to inputs; calendar stays in left
  sidebar (always visible, no collapse) with reduced padding and a
  taller MONTH_HEIGHT so 6-week months render fully.
- LeftSidebar: drop Library collapse; Heaps regains its chevron toggle
  to match Views/Folders.
- RightSidebar: render ActiveHeapCard above the Metadata header (with
  its own eyebrow); preview overlay reuses RightSidebar so the active
  heap stays visible there too.
- Toaster: top-right, more compact (smaller padding, font, gap).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:21:57 +02:00
root
c5582ffc65 ui(sidebar): collapsible Library section with Heaps nested inside
Wraps Views, Folders, Shared-with-me, and Heaps in a single
click-to-toggle Library section with a consistent h-9 eyebrow header
(matching the new Date header). Heaps keeps its own eyebrow
sub-section so it sits alongside Folders, and heap rows now reserve
the same chevron-slot spacer as leaf folder rows so indentation
lines up across hierarchies. ActiveHeapCard moves to the very top
of the sidebar so it stays visible under any panel state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:48:20 +02:00
root
eac005109c feat(filters): move date picker to sidebar, track visible photo order
Pulls the date range picker out of the filter-bar pill into a
dedicated always-visible section at the top of the left sidebar, and
teaches the timeline to publish its visible photo sequence so "open
first photo" shortcuts respect the on-screen order.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 17:12:34 +02:00
root
45f1649979 ui: uniform 160px thumbnail cells across all grid views
Timeline, Memories, and Duplicates now share a single fixed cell size
(THUMBNAIL_SIZE=160) with no 1fr stretching — cells stay exactly 160px
regardless of sidebar state, at the cost of a small right-edge strip
when the container width isn't a multiple of (160+gap). Width is
measured on the scroll container itself with padding subtracted so
sidebar expand/collapse reliably reflows the grid.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 15:41:40 +02:00
root
66b3bc5e1f perf+ux: cheaper sidebar toggle, virtualised filmstrip, compact toasts
Timeline's items array used to rebuild on every sub-pixel cellSize tick
during the sidebar CSS transition, causing visible jank with thousands
of photos. Row heights now resolve off a ref at virtualizer-measure
time, so items only rebuild when the column count actually changes.
PreviewFilmstrip is horizontally virtualised (~15 cells in the DOM
instead of N), cutting preview open latency on large libraries. Also
honor the user's explicit right-sidebar collapse (don't auto-reopen on
photo selection) and shrink the sonner toasts to a tighter form factor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 15:24:32 +02:00
root
d72a218b46 feat: full shortcut parity + perf fixes across memories and duplicates
Memories view now supports the same keyboard shortcuts, heap membership,
and optimistic cache updates as the Timeline. Arrow/Ctrl+A/Escape nav is
extracted into a shared useGridKeyNav hook so both views stay in lockstep.
Duplicates view is virtualised with @tanstack/react-virtual and has
stabilised PhotoThumbnail props so React.memo actually elides work when
scrolling or toggling selection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 11:51:13 +02:00
744a7fa0c3 feat(memories): use Timeline's PhotoThumbnail grid
MemoriesView now renders PhotoThumbnail cells wired up to the shared
photoStore so selection, heap membership, preview (double-click /
Enter), badges, drag-to-heap, and search-match highlighting all work
the same way they do in Timeline. Kept the per-year section grouping,
swapped the bespoke img tiles for the shared component.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:57:29 +02:00
967cf23b82 feat(sharing): user picker in share dialog
Replace the free-text username input with a Select populated from a new
/sharing/users endpoint. Users already on the target's share list are
filtered out, and the trigger surfaces loading / empty states. Matches
the existing permission model since sharing only ever required knowing
a username.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:34:09 +02:00
a073ee7fb9 perf+style: grid subscription hygiene, a11y, shadcn-style consistency
Perf / a11y (high-impact review items)
- Timeline arrow-key handler binds once per (viewMode, currentSection)
  and reads fresh state via navStateRef instead of an 8-element dep
  array of new-each-render values.
- usePhotosQuery collapses 14 individual Zustand selectors into one
  useShallow selector returning the params object.
- PhotoThumbnail no longer subscribes to the search query directly;
  Timeline subscribes once and passes it down as a prop.
- PhotoThumbnail gains role="button", tabIndex, aria-label, aria-pressed,
  Enter/Space key handlers and a focus-visible ring. Timeline marked
  role="grid"; RightSidebar marked role="region".

Style consistency
- Swap clsx for cn (tailwind-merge aware) across 17 files so
  conflicting utility classes collapse correctly.
- New Badge primitive (ui/badge.tsx) with default/neutral/overlay/
  outline variants; adopted in ColorsView, RatedView, TagsView for
  the repeated count overlay pill.
- Fix palette drift: text-amber-400 -> text-star, text-green-*
  -> text-pick, text-red-* -> text-reject (5 files).
- Button gains an xs size (h-6 px-1.5 text-[11px]) for the repeated
  compact-button pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:30:16 +02:00
e65e798021 perf+ux: cut grid re-renders, coalesce discard, dedup bulk mutations
Frontend cleanup pass driven by the post-shadcn review.

Performance
- Memoize PhotoThumbnail and route cell click/double-click through
  stable handlers so heap-membership invalidation no longer re-renders
  every visible thumbnail.
- Cap usePhotosQuery's eager background page-walk at 20 pages with a
  50ms inter-page yield — was unbounded (up to 100k photos cold).
- Drop the per-thumbnail loading spinner in favour of the existing
  pulse skeleton; only retry state still surfaces a spinner.

UX
- Coalesce rapid X/U presses into a single undo entry + one toast
  (1.2s window) so accidental bursts are easy to back out.
- Optimistic rating/color updates with per-id snapshot rollback on
  error, matching the existing discard pattern.
- Section-aware empty timeline state with a Clear-all-filters CTA.
- Carry the search-match chip from the grid into the preview header.
- Add a basket-icon badge for active heap membership so the green
  tint isn't the only signal (colorblind-safe).
- Standardise error toasts via formatApiError(): FastAPI detail,
  validation arrays, axios message, with a 'Network Error' filter.

Architecture
- Extract useBulkPhotoMutations and stop duplicating
  bulkRating/bulkColor across RightSidebar and useKeyboardShortcuts.
- Split RightSidebar (714 -> 448 LOC) and PhotoInfoPanel (952 -> 716)
  into co-located sub-components: BulkTakenAtEditor, BulkTagsEditor,
  TagsEditor, TakenAtEditor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:13:39 +02:00
288 changed files with 21675 additions and 34496 deletions

22
.env
View File

@@ -1,22 +0,0 @@
# Mulita / PhotoVault local environment.
# See .env.example for the full list of knobs and their docs.
# REQUIRED — host path to your photo library.
PHOTO_DIRS=/mnt/library/homecloud/admin/files/
# Ports — change if 3000 / 8001 collide with other services on the host.
FRONTEND_PORT=3000
BACKEND_PORT=8001
REDIS_PORT=6379
# CORS — wildcard for local dev. Lock down for real deployments.
ALLOWED_ORIGINS=*
# Logging + timezone.
LOG_LEVEL=INFO
TZ=UTC
# Celery worker pools — split worker-light (IO) and worker-vision (CPU).
# Defaults target a 6-core / 16 GB host.
CELERY_LIGHT_CONCURRENCY=2
CELERY_VISION_CONCURRENCY=5

View File

@@ -1,113 +1,83 @@
# ─────────────────────────────────────────────────────────────────────────────
# Mulita / PhotoVault — example environment file
# Example environment file. Copy to `.env` and adjust.
#
# Copy this file to `.env` and adjust the values for your setup. Every key
# below has a sensible default in docker-compose.yml, so you only need to
# uncomment the ones you actually want to change.
# ─────────────────────────────────────────────────────────────────────────────
# podman-compose --env-file .env \
# -f docker-compose.yml -f docker-compose.podman.yml up -d
# ── REQUIRED ─────────────────────────────────────────────────────────────────
# Host path to your photo library. The compose file mounts this at /photos
# inside the backend + worker containers. The backend creates a default
# source root pointing at /photos on first boot, so once this is set the
# library is scanned with zero further configuration.
# Host path to your photo library. PhotoPrism reads this in place and
# writes EXIF backwrites next to originals (when PP_ORIGINALS_MODE=rw).
PHOTO_DIRS=/mnt/library/homecloud/admin/files/
# Bootstrap admin password. The first PhotoPrism boot creates an `admin`
# account with this password. Rotate after first login from the UI.
PP_ADMIN_PASSWORD=please-change-me
# MariaDB passwords. Generate with `openssl rand -hex 24`.
PP_DB_PASSWORD=please-change-me
PP_DB_ROOT_PASSWORD=please-change-me
# ── OPTIONAL ─────────────────────────────────────────────────────────────────
# Loopback host port for PhotoPrism's API (and UI, if you tunnel to it).
# Vite proxies /api/v1/* here and the host-mode sidecar reaches it on
# localhost. Not published on the public interface.
PP_PORT=2342
# Site URL — used for share links, OIDC redirect URI, and reverse-proxy aware
# URL generation. Set to the public hostname once the proxy is in front.
PP_SITE_URL=http://localhost:2342/
# Auth mode — "password" for username/password (default), "public" for an
# unauthenticated kiosk mode (don't use this on a multi-user library).
PP_AUTH_MODE=password
# Library mount mode. "rw" allows rename / folder mutations / EXIF backwrite;
# "ro" is safe-for-archives but disables those sidecar endpoints. Set in
# lockstep with PP_READONLY below.
PP_ORIGINALS_MODE=rw
PP_READONLY=false
# UID/GID inside the PhotoPrism container. Set these to the host UID/GID that
# owns ${PHOTO_DIRS}. `id -u` and `id -g`.
PP_UID=1000
PP_GID=1000
# ── OIDC SSO (Authentik or equivalent) ───────────────────────────────────────
# Leave blank to keep OIDC dormant. Fill in to enable the "Sign in with OIDC"
# button on the login page; OIDC_REGISTER=true auto-creates accounts at role
# `user` (override to `admin` to grant full access on first SSO login).
#
# Examples:
# macOS / Linux: PHOTO_DIRS=/Users/you/Pictures
# Network share: PHOTO_DIRS=/mnt/nas/photos
# Windows (WSL): PHOTO_DIRS=/mnt/c/Users/you/Pictures
PHOTO_DIRS=./photos
# The compose file reads these and maps them to PhotoPrism's actual env-var
# names (PHOTOPRISM_OIDC_URI / _CLIENT / _SECRET / _PROVIDER) — see the
# comment in docker-compose.yml. The PhotoPrism callback URI is auto-derived
# from PP_SITE_URL; do not set it manually.
# OIDC_PROVIDER_NAME=Authentik
# OIDC_ISSUER_URL=https://auth.example.com/application/o/photoprism/
# OIDC_CLIENT_ID=...
# OIDC_CLIENT_SECRET=...
# OIDC_SCOPES=openid profile email
# OIDC_REGISTER=true
# OIDC_ROLE=user
# ── PORTS ────────────────────────────────────────────────────────────────────
# Host port the SPA is served on. Browse to http://<host>:<FRONTEND_PORT>/.
FRONTEND_PORT=3000
# Host port for the backend API. Almost never needed directly — the frontend
# nginx proxies /api/ to the backend over the internal compose network. Kept
# exposed for debugging / curl.
BACKEND_PORT=8001
# Redis host port. Internal services reach Redis on its container name; this
# is just for local debugging.
REDIS_PORT=6379
# ── AUTH ─────────────────────────────────────────────────────────────────────
# Secret key used to sign JWT tokens. Generate a strong random value for
# production (e.g. `openssl rand -base64 32`). The default is a deterministic
# placeholder acceptable only for local/homelab use.
# SECRET_KEY=change-me-to-a-random-string
# How long access and refresh tokens stay valid. Access tokens are short-lived
# and silently refreshed by the frontend; refresh tokens let a session survive
# across browser restarts.
# ACCESS_TOKEN_EXPIRE_MINUTES=60
# REFRESH_TOKEN_EXPIRE_DAYS=30
# ── CORS ─────────────────────────────────────────────────────────────────────
# Comma-separated list of allowed origins for direct browser access to the
# backend. Same-origin requests through the nginx / vite proxy never trip
# CORS, so this only matters when something hits the backend port directly
# from a different origin (e.g. another machine, dev tools, a reverse proxy
# under a different hostname).
# ── USER LIBRARY ISOLATION ───────────────────────────────────────────────────
# Maps PhotoPrism usernames to originals-relative subdirectories so each
# user only sees their own photos. Format: comma-separated user:path pairs.
# The sidecar reconciler applies this to auth_users.base_path on boot and
# every 60s. Leave empty for single-user deployments.
#
# Default "*" is permissive, fine for a single-user homelab. Lock it down in
# real deployments:
# ALLOWED_ORIGINS=https://photos.example.com
# ALLOWED_ORIGINS=https://photos.example.com,http://192.168.1.10:3000
ALLOWED_ORIGINS=*
# USER_BASEPATHS="alice:alice, bob:bob"
# Sidecar DB password — provisioned by mariadb/init/01-sidecar.sql on first
# boot. Rotate before any non-local deployment.
# SIDECAR_DB_PASSWORD=replace-at-m4-bringup
# ── LOGGING / TIMEZONE ───────────────────────────────────────────────────────
# ── LOGGING ──────────────────────────────────────────────────────────────────
# Python log level for the backend and Celery worker. Bump to DEBUG when
# chasing scan / thumbnail issues.
LOG_LEVEL=INFO
# Container timezone. Affects the timestamps in logs and the "added at"
# field on newly imported photos. Defaults to UTC.
# TZ=Europe/Berlin
# TZ=America/New_York
TZ=UTC
# ── WORKER CONCURRENCY ───────────────────────────────────────────────────────
#
# The ingestion pipeline runs on two Celery worker services with separate
# concurrency knobs so heavy vision tasks can't starve cheap IO tasks:
#
# worker-light (default / high / low queues)
# Runs: scan, thumbnails, EXIF, pHash, duplicate regrouping.
# Mostly IO-bound — 2 prefork children keep a library streaming in.
#
# worker-vision (vision queue)
# Runs: embeddings, object detection, OCR, face extraction, content
# classification. Each prefork child loads ~2 GB of ONNX model weights,
# so set this to roughly (physical_cores 1) and watch RAM.
#
# Defaults target a ~6 core / 16 GB host. Raise these, then
# docker compose up -d worker-light worker-vision
# to pick them up. Lower for a Pi; go higher on a workstation.
#
# The old `CELERYD_CONCURRENCY=N` single-worker variable is no longer
# read — delete it from your .env if it's set.
CELERY_LIGHT_CONCURRENCY=2
CELERY_VISION_CONCURRENCY=5
# ── INTERNAL (rarely overridden) ─────────────────────────────────────────────
# These point at the in-compose Redis and the bind-mounted SQLite db. Override
# only if you're running Mulita without docker-compose or against an external
# Redis.
# REDIS_URL=redis://redis:6379
# CELERY_BROKER_URL=redis://redis:6379
# CELERY_RESULT_BACKEND=redis://redis:6379
# DATABASE_URL=sqlite+aiosqlite:////data/db/mulita.db
PP_LOG_LEVEL=info

11
.gitignore vendored
View File

@@ -61,9 +61,18 @@ build/
# Docker
docker-compose.override.yml
# PhotoPrism state (sidecars, cache, thumbs, db backups) — regenerable.
/pp/storage/
/pp/import/
# Sidecar runtime state (per-user marks etc.) — generated, not seed data.
/sidecar/data/
# Sidecar Go build output.
/sidecar/mule-sidecar
# Photos (for development)
/photos/
# Thumbnails
/thumbs/
/trash/backend/yolov8n.pt

View File

@@ -0,0 +1,247 @@
# Plan: Populate `photos_users` to fix label isolation in PhotoPrism
**Date:** 2026-06-06
**Author:** Hermes Agent
**Status:** Draft
---
## 1. Goal
Fix the label isolation leak where a user with `base_path` set (e.g. `muli`) sees photos from other users' directories (e.g. `dtoro`) in PhotoPrism's labels view.
## 2. Current Context
### The problem
- PhotoPrism's `base_path` feature correctly scopes the main search (`/api/v1/photos`).
- Label views (`/api/v1/labels`) do **not** respect `base_path` — labels show photo counts and thumbnails from the entire library.
- User reports: "all photos on the main labels page are a mix of both" muli and dtoro.
### What we know
- **53 files changed** in the sidecar (Go + GORM, gorm.io/gorm v1.31.1).
- Sidecar has a working PhotoPrism DB connection via `PpDSN` (user: `photoprism`, schema: `photoprism.*`).
- The `photos_users` table exists in PhotoPrism's MariaDB schema but is **empty** (0 rows).
- Schema of `photos_users`:
```
photos_users:
uid varbinary(42) NOT NULL PRI (composite PK or single?)
user_uid varbinary(42) NOT NULL PRI
team_uid varbinary(42) YES MUL
perm int(10) unsigned YES
```
- Known user UIDs: `dtoro=utfetfdk0so2z9zl`, `muli=utg7jjbd8iwaghn6`
- Known base paths: `dtoro→dtoro`, `muli→muli`
- The `photo_path` column in `photos` stores paths like `muli/files/Photo Archive...` or `dtoro/Memories/...`
- Currently: ~88K photos, ~52K files indexed.
### The sidecar's current reconciler (`users.go`)
- Runs every 60s.
- Only calls `UPDATE auth_users SET base_path = ? WHERE user_name = ?`.
- Does **not** touch `photos_users`.
### Unknowns
1. **Does PhotoPrism use `photos_users` for general label filtering?** The table appears designed for explicit sharing (e.g. share a specific photo with another user), not for base_path ACL. PhotoPrism may ignore `photos_users` in label queries.
2. **Performance impact**: 88K photos × 2 users = up to 176K rows. Could slow label queries.
3. **Side effects**: If `photos_users` controls sharing, adding auto-entries might break explicit share workflow.
4. **`perm` values**: Unclear what `perm` value grants "view" access. Likely a bitmap (bit 0 = view).
## 3. Proposed Approach
### Phase 1: Investigate (prove the approach works before building)
**Step 1.1: Insert test rows into `photos_users` manually**
On the production DB, insert a few `photos_users` entries for muli mapping to some of muli's own photos, plus one entry mapping to a dtoro photo. Use a guessed `perm` value (e.g. `1` = view).
Then check:
- Does muli see fewer photos now? (If `photos_users` works as an exclusive ACL, yes.)
- Does the dtoro photo with a `photos_users` entry for muli show up for muli?
- Does the label view change?
**Step 1.2: Test with `perm` variations**
If `perm=1` does nothing, try `perm=2`, `perm=7`, or `perm=15` (common Unix-ish bitmap patterns).
**Step 1.3: Examine PhotoPrism source**
Check PhotoPrism's search/label code to confirm whether `photos_users` is joined in label queries. This tells us definitively whether the approach is viable.
### Phase 2: Build (if Phase 1 confirms the approach works)
**Step 2.1: Add `photos_users` GORM model**
New struct in `db.go` or a new file `perms.go`:
```go
type PhotoUser struct {
PhotoUID string `gorm:"primaryKey;size:42;column:uid"`
UserUID string `gorm:"primaryKey;size:42;column:user_uid"`
TeamUID string `gorm:"size:42;column:team_uid"`
Perm int `gorm:"column:perm"`
}
func (PhotoUser) TableName() string { return "photos_users" }
```
Note: GORM `AutoMigrate` is called on `mule_sidecar` schema, not `photoprism.*`. The `photos_users` table already exists in the `photoprism` schema — we only query/insert, never migrate.
**Step 2.2: Add `reconcilePhotoUsers` function**
New function in a new file `perms.go` alongside `users.go`. Signature:
```go
func reconcilePhotoUsers(ppDSN, originalsRoot string, mapping map[string]string) error
```
Logic:
1. For each `username:path` pair in `mapping`, look up the user's `user_uid` in `auth_users`.
2. Query `photos` for all `photo_uid` where `photo_path LIKE 'path/%'`.
3. Batch-insert entries into `photos_users` with a default `perm` value (to be determined in Phase 1).
4. Use `INSERT IGNORE` or `ON DUPLICATE KEY UPDATE` for idempotency.
5. Handle deletions: if a photo's path is changed (via rename), the old `photos_users` entry should be cleaned up.
**Step 2.3: Wire into reconciler loop**
Extend the existing `startUserBasepathReconciler` to call `reconcilePhotoUsers` after `reconcileUserBasepaths`.
```go
func apply() {
reconcileUserBasepaths(...)
reconcilePhotoUsers(...)
}
```
**Step 2.4: Handle re-index edge cases**
- When new photos are indexed, they won't have `photos_users` entries until the next 60s tick.
- Could add a webhook or a one-shot trigger after PhotoPrism's index completes.
- Alternative: accept the 60s lag as a design trade-off (current base_path reconciler already has this lag).
### Phase 3: Validate (if Phase 1 confirms)
1. Build the binary: `cd sidecar && CGO_ENABLED=0 go build -o mule-sidecar .`
2. Rebuild the Docker image and restart the sidecar.
3. Check `photos_users` has expected rows.
4. Log in as `muli` via Authentik SSO, browse labels — verify dtoro photos are gone.
5. Log in as `dtoro` — verify still sees own photos.
6. Verify no regression: search, album, folder views still work for both users.
## 4. Files Likely to Change
| File | Change |
|------|--------|
| `sidecar/perms.go` | **New file**`PhotoUser` model, `reconcilePhotoUsers` function |
| `sidecar/db.go` | Add `photos_users`-related constants/helpers (optional) |
| `sidecar/users.go` | Extend `reconcileUserBasepaths` or add a phase to the existing reconciler |
| `sidecar/main.go` | Wire the new reconciler phase (minor — call from existing ticker) |
| `sidecar/Dockerfile` | Unchanged (Go build picks up new `.go` files automatically) |
## 5. Tests & Validation
1. **Build check**: `go build ./...` from `sidecar/`
2. **Manual DB test** (Phase 1): Insert test `photos_users` rows via `docker exec pp-mariadb mysql ...`
3. **Integration test**: After deploy, check `photos_users` row count matches expected photo count per user.
4. **Label isolation check**: Browse labels as each user — confirm no cross-user leaks.
## 6. Source Code Analysis (Completed)
### How base_path scoping works in PhotoPrism
Found the critical function `ScopePhotosForSession` in `internal/entity/search/photos_scope.go`:
```go
func ScopePhotosForSession(stmt *gorm.DB, sess *entity.Session) *gorm.DB {
// Admin/library role → no scoping needed
if sess == nil || acl.Rules.AllowAny(acl.ResourcePhotos, sess.GetUserRole(), acl.Permissions{acl.AccessAll, acl.AccessLibrary}) {
return stmt
}
user := sess.GetUser()
if basePath := user.GetBasePath(); basePath == "" {
return stmt.Where(sharedAlbums + "photos.created_by = ? OR ...", ...)
} else {
return stmt.Where(sharedAlbums + "... OR photos.photo_path = ? OR photos.photo_path LIKE ?",
..., basePath, basePath + "/%")
}
}
```
Key: base_path filtering is done by adding `WHERE photos.photo_path LIKE 'muli/%'` to the SQL query. It is **NOT** done via `photos_users`.
### How endpoints use base_path
| Endpoint | Function | Applies base_path? |
|----------|----------|-------------------|
| `GET /api/v1/photos` | `SearchPhotos``UserPhotos``searchPhotos`**`ScopePhotosForSession`** | ✅ Yes |
| `GET /api/v1/labels` | `SearchLabels``search.Labels(frm)` — no session passed | ❌ **No** |
| Review tab | Uses `GET /api/v1/photos?quality=3` → goes through `ScopePhotosForSession` | ✅ Should scope |
| Archive tab | Uses `GET /api/v1/photos?archived=true` → goes through `ScopePhotosForSession` | ✅ Should scope |
| Albums | TBD — depends on whether they use `ScopePhotosForSession` | ⚠️ Unknown |
### The `photos_users` table
Found in `internal/entity/photo_user.go`:
```go
type PhotoUser struct {
UID string // photo_uid
UserUID string // user_uid
TeamUID string // team_uid
Perm uint // permission bitmap
}
```
This table is **not referenced** in `ScopePhotosForSession`, `searchPhotos`, or any label/album search function. It is only used for **explicit sharing** (via `FirstOrCreatePhotoUser` called when sharing a specific photo with another user).
**Conclusion: Populating `photos_users` will NOT fix the label, review, or archive tab isolation.** PhotoPrism does not consult this table for any of these queries.
### Why review/archive might show cross-user photos
Since review and archive use `GET /api/v1/photos` which goes through `ScopePhotosForSession`, they **should** be scoped. The issue might be:
1. **Pre-computed counts** in the sidebar tabs show total numbers across all users
2. **Label thumbnails** and category summaries are computed from the `labels` table which is global
3. The actual photo list in review/archive should be correctly scoped — the user may be seeing dtoro photos only in the summary/counts
### DB experiment results
Confirmed `photos_users` is empty (0 rows). Inserted 100 muli-photo entries + 1 dtoro-photo entry for muli with `perm=1`. Label API response unchanged — `photo_count` values remained the same (Dog: 733, Cat: 57), confirming labels ignore `photos_users`.
## 7. Updated Recommendation
**Abandon the `photos_users` approach.** It won't fix the problem because PhotoPrism never consults this table for labels, review, or archive queries.
### Real fix options
1. **Sidecar label filter** (recommended) — The sidecar already validates sessions via `resolveSession()` which returns the user's `BasePath`. Extend the sidecar to expose a **proxied `/api/v1/labels`** endpoint that:
- Accepts the caller's `X-Auth-Token` (already validated by `requireSession`)
- Forwards the request to PhotoPrism's `/api/v1/labels`
- **Filters the response** to remove labels whose `Thumb` belongs to a photo outside the user's `base_path`
- Recalculates `PhotoCount` for the user's scope (count photos under `base_path/%` for that label)
- Also filter `Count` values in the sidebar summary response
**Why this works:** The sidecar already has DB access to PhotoPrism's schema (`PpDSN`) and validates sessions. It can query `photos` to count label intersections per base_path.
2. **Same approach for review/archive sidebar counts** — Intercept the relevant metadata/summary endpoints to scope counts by base_path.
3. **Accept the limitation** — Labels show cross-user thumbnails/counts but the actual photo list is scoped.
### Implementation sketch for option 1
```
sidecar/
├── proxy.go # New file
│ ├── handleLabels(c) → GET /api/sidecar/labels → proxies to PP, filters by base_path
│ ├── handleReviewCount(c) → GET /api/sidecar/review → returns scoped count
│ └── handleArchiveCount(c) → GET /api/sidecar/archive → returns scoped count
```
The SvelteKit frontend would call `/api/sidecar/labels` instead of `/api/v1/labels`.
### Clean up: remove test rows from photos_users
Since the approach won't work, remove the test rows inserted during Phase 1:
```sql
DELETE FROM photoprism.photos_users WHERE user_uid = 'utg7jjbd8iwaghn6';
```

View File

@@ -0,0 +1,301 @@
# Plan: Fix user isolation in PhotoPrism — labels, review, and archive views
**Date:** 2026-06-06
**Author:** Hermes Agent
**Status:** Draft
---
## 1. Goal
Fix the three views where a user with `base_path` set (e.g. `muli`) sees photos from other users (e.g. `dtoro`):
1. **Labels** — labels list + label drill-down show all library photos
2. **Review** — photos needing review tab shows cross-user photos
3. **Archive** — archived photos tab shows cross-user photos
## 2. Current Context & Source Analysis
### 2.1 How base_path scoping works
PhotoPrism's `ScopePhotosForSession` (in `internal/entity/search/photos_scope.go`) is the only function that enforces user isolation. It adds `WHERE photos.photo_path = '<basePath>' OR photos.photo_path LIKE '<basePath>/%'` to the SQL query.
This is called by `searchPhotos()` — used by the **`GET /api/v1/photos`** endpoint (search, timeline, folders).
### 2.2 How endpoints use base_path
| View | Endpoint | Function chain | Applies base_path? |
|------|----------|----------------|-------------------|
| Main timeline | `GET /api/v1/photos` | `SearchPhotos``searchPhotos``ScopePhotosForSession` | ✅ Yes |
| Folders | `GET /api/v1/photos` with path filter | Same chain | ✅ Yes |
| **Labels** | `GET /api/v1/labels` | `SearchLabels``search.Labels(frm)`**no session** | ❌ **No** — queries `labels` table directly |
| Label drill-down | `GET /api/v1/photos?label=X` | Uses `searchPhotos``ScopePhotosForSession` | ✅ Should scope (if label= param doesn't bypass) |
| **Review tab** | `GET /api/v1/photos?q=review:true` | `searchPhotos``ScopePhotosForSession` | ✅ Should scope, BUT... |
| **Archive tab** | `GET /api/v1/photos?q=archived:true` | Same | ✅ Should scope, BUT... |
### 2.3 The review/archive problem: ACL overrides
In `searchPhotosForm()` (internal/api/photos_search.go):
```go
if acl.Rules.Deny(acl.ResourcePhotos, s.GetUserRole(), acl.ActionManage) {
frm.Quality = 3
}
```
For role=user, `Deny(ActionManage)` → true → sets `frm.Quality = 3` (minimum quality).
Then in `searchPhotos()` (internal/entity/search/photos.go):
```go
if acl.Rules.Deny(acl.ResourcePhotos, aclRole, acl.ActionDelete) {
frm.Archived = false
frm.Review = false
}
```
For role=user, `Deny(ActionDelete)` → true → **overrides `review:true` and `archived:true` to false**.
So the review and archive filters are **completely ignored** for the `user` role. The frontend sends `review:true` but the server discards it. The result: the review/archive tabs show ALL photos scoped by base_path (no quality/review/archive filter), which means basically the same as the main timeline.
### 2.4 Label problem: no session scoping at all
`search.Labels()` queries the `labels` table directly with a `WHERE photo_count > 0` clause. There is no session parameter, no `ScopePhotosForSession`, and no base_path or user filtering whatsoever. Labels are **library-wide** in PhotoPrism.
The label drill-down (click into a label) uses `GET /api/v1/photos?label=X` which DOES go through `ScopePhotosForSession`, so the photo list itself should be scoped — but the label thumbnails, counts, and covers are global.
### 2.5 ACL GrantDefaults — missing RoleUser entry
```go
var GrantDefaults = Roles{
RoleAdmin: GrantFullAccess, // FullAccess = AccessLibrary + everything
RoleGuest: GrantReactShared, // Only shared content
RoleVisitor: GrantViewShared, // Only shared content
RolePortal: GrantFullAccess,
RoleClient: GrantFullAccess,
// RoleUser and RoleViewer are NOT listed → fallback to RoleDefault (also missing) → denied
}
```
Because `RoleUser` is absent from `GrantDefaults`, the `Allow()` function falls back to `RoleDefault` which is also absent → returns `false` for all permissions. This means:
- `ScopePhotosForSession` correctly enters the `base_path` branch (good — user is isolated)
- BUT `ActionDelete` is denied → review/archive filters are forced off (bad — can't browse review/archive)
### 2.6 The `photos_users` table — ruled out
`internal/entity/photo_user.go` defines `PhotoUser` but it is **not referenced** in `ScopePhotosForSession`, `searchPhotos`, or any label/album search function. Populating it won't fix any of these issues.
## 3. Proposed Approach
### Phase 1: Sidecar proxy for labels (direct fix)
Extend the sidecar to expose a **scoped labels endpoint**:
```
GET /api/sidecar/labels → proxies to GET /api/v1/labels → filters by base_path
```
**How it works:**
1. Sidecar receives the caller's `X-Auth-Token`
2. `requireSession` middleware resolves the token → returns user's `BasePath`
3. Sidecar makes the same `/api/v1/labels` request to PhotoPrism (using the caller's token)
4. **Filter step**: for each label in the response, query the DB to count photos with that label AND `photo_path LIKE '<base_path>/%'`
5. Return filtered labels with corrected `PhotoCount` and `Thumb`
**Implementation:**
New file `sidecar/handlers_labels.go`:
```go
// handleLabels proxies to PP's /api/v1/labels, then post-filters
// counts and thumbnails by the caller's base_path.
func handleLabels(pp *ppClient, ppDSN string) gin.HandlerFunc {
return func(c *gin.Context) {
token := ctxToken(c)
user := ctxUser(c) // resolved from session, includes BasePath
// 1. Get raw labels from PhotoPrism
resp, _ := pp.call(c, "GET", "/api/v1/labels?"+c.Request.URL.RawQuery, token, nil)
// 2. Decode labels
var labels []PpLabel
json.Unmarshal(resp.Body, &labels)
// 3. For each label, recalculate count for this user's base_path
for i, l := range labels {
// Count photos with this label AND where photo_path matches base_path
var count int
db.Raw(`SELECT COUNT(*) FROM photos_labels pl
JOIN photos p ON pl.photo_uid = p.photo_uid
WHERE pl.label_uid = ? AND p.photo_path LIKE ?`,
l.UID, user.BasePath+"/%").Scan(&count)
labels[i].PhotoCount = count
// If count is 0, the thumb from the global label doesn't apply
// Could also update thumb to a user-scoped one
}
c.JSON(http.StatusOK, labels)
}
}
```
**Frontend change:** Update the label query in `web/src/routes/tags/[category]/[[value]]/+page.svelte` to call `/api/sidecar/labels` instead of `/api/v1/labels`.
### Phase 2: Fix review/archive — skip the ACL override
Two options:
**Option A (Recommended): Sidecar proxy for photos search**
Extend the sidecar with:
```
GET /api/sidecar/photos → proxies to GET /api/v1/photos → adds path filter
```
The sidecar intercepts the photos request and adds the `path:<basePath>` query parameter to PhotoPrism's API call. This forces PhotoPrism to add `WHERE photo_path LIKE '<base_path>/%'`.
For review/archive, the sidecar also adds `review:true` or `archived:true` BEFORE the ACL override happens (since the sidecar doesn't hit the ACL code).
**Option B: Custom frontend query**
The frontend explicitly adds `path:muli` to the query string for review/archive tabs:
```
GET /api/v1/photos?q=review:true path:muli&count=50
```
The `path` filter is a standard PhotoPrism search operator that adds `WHERE photos.photo_path = '<path>'`. But this only matches the exact path, not `path/%` (subdirectories). The `path:` operator does `photo_path = ?` (exact match) per the code at line 668.
**Option A is better** because:
- Works for all users without frontend changes
- Can add the proper `LIKE` prefix match
- Centralized logic in the sidecar
### Phase 3: Sidecar proxy for sidebar counts
The session response (or `GET /api/v1/config`) includes library-wide counts:
```json
"count": {
"review": 248,
"archived": 94,
"all": 88203,
"photos": 88000
}
```
These show the TOTAL across all users. The sidecar can proxy this and recalculate counts per base_path.
## 4. Step-by-step Plan
### Step 1: Sidecar — labels proxy
Files: `sidecar/handlers_labels.go` (new), `sidecar/main.go` (route wiring)
1. New types: `PpLabel` (mirrors PhotoPrism's label response shape)
2. Handler function `handleLabels()` that:
- Validates token via `requireSession`
- Gets `BasePath` from session
- Calls PhotoPrism's `/api/v1/labels`
- For each label, queries photos_labels + photos to count user-scoped photos
- Returns filtered labels
3. Wire route: `auth.GET("/labels", handleLabels(...))` in `main.go`
4. Frontend: change label fetch URL from `/api/v1/labels` to `/api/sidecar/labels`
### Step 2: Sidecar — photos proxy (review/archive fix)
Files: `sidecar/handlers_photos.go` (new), `sidecar/main.go` (route wiring)
1. Handler function `handlePhotos()` that:
- Validates token
- Gets `BasePath` from session
- Parses the query string to detect `review:true` or `archived:true`
- Forwards to PhotoPrism's `/api/v1/photos` with `path:<basePath>` added to query
- For review/archive, also ensures `review/archived` filter is NOT stripped
- Returns PhotoPrism's response
2. Two implementation variants:
**Variant A** (simpler): add `path:<basePath>` to the forwarded query. This only matches exact path, not subdirs (PhotoPrism's `path:` operator does exact match). Might miss photos in subdirectories.
**Variant B** (correct): Forward the query without path, then post-filter the response to remove photos whose `photo_path` doesn't match `basePath/%`. This is more robust.
### Step 3: Validation
1. Build sidecar: `cd sidecar && CGO_ENABLED=0 go build -o mule-sidecar .`
2. Rebuild Docker image: `docker compose build sidecar`
3. Restart sidecar: `docker compose up -d sidecar`
4. Test labels as muli — verify only muli's labels appear
5. Test review tab as muli — verify only muli's photos needing review appear
6. Test archive tab as muli — verify only muli's archived photos appear
7. Test same views as admin — verify dtoro still sees all
## 5. Files Likely to Change
| File | Change |
|------|--------|
| `sidecar/handlers_labels.go` | **New** — label proxy handler |
| `sidecar/handlers_photos.go` | **New** — photos proxy handler (or merged into one proxy.go) |
| `sidecar/handlers_folder.go` | Reference for existing handler patterns |
| `sidecar/main.go` | Wire new routes under `auth` group |
| `sidecar/pp.go` | May need new helper methods for label/photo API calls |
| `sidecar/users.go` | No change |
| `sidecar/db.go` | May add types for PpLabel, PpPhoto |
| `web/src/routes/tags/[category]/[[value]]/+page.svelte` | Change label fetch URL |
| `web/src/lib/stores/filters.svelte.ts` | Possibly change how review/archive queries are built |
## 6. Tests & Validation
**Build**: `cd sidecar && go build ./... && go vet ./...`
**Manual validation on LXC 120:**
```bash
# Test labels endpoint
curl -s "http://localhost:8000/api/sidecar/labels?count=5" \
-H "X-Auth-Token: <muli-token>" | python3 -c "import sys,json;d=json.load(sys.stdin);[print(l.get('Name','?'),l.get('PhotoCount')) for l in d[:5]]"
# Test photos endpoint with review
curl -s "http://localhost:8000/api/sidecar/photos?q=review:true&count=5" \
-H "X-Auth-Token: <muli-token>" | python3 -c "import sys,json;d=json.load(sys.stdin);print(f'{len(d)} photos')"
# Verify vs. admin token — counts should differ
```
**Cross-user check:** Log in as `muli` and `dtoro` in separate browser sessions. Verify:
- Labels show different counts per user
- Review photos are scoped per user
- Archive photos are scoped per user
## 7. Risks, Tradeoffs & Open Questions
### Risks
| Risk | Impact | Mitigation |
|------|--------|------------|
| Sidecar proxying adds latency | Slower page loads | Labels are small payloads; single DB query per label is fast |
| Frontend needs URL changes | Breaks if not updated | Do frontend change alongside sidecar deploy |
| Photo count queries on every label request | DB load | Cache results for 30s in the sidecar |
| PhotoPrism's label `PhotoCount` is stale | Mismatch with actual count | Acceptable — PhotoPrism's count is already cached |
| Review/archive fix depends on how PhotoPrism handles `path:` operator | Photos in subdirs missed | Use Variant B (post-filter by path prefix) |
### Open Questions
- **Q1**: For review/archive — is the user seeing dtoro's photos in the *grid* or only the *sidebar counts*? Need to verify actual API response vs what the frontend renders.
- **Q2**: What's the performance impact of running `SELECT COUNT(*) FROM photos_labels ... JOIN photos ...` for every label in the response? (Labels list is typically short, < 100)
- **Q3**: Does the frontend cache the label response aggressively? Need to invalidate cache on user switch.
- **Q4**: For the `path:` operator — does it do exact match or LIKE? From source: `WHERE photos.photo_path = ?` — exact match only.
### Tradeoffs
- **Sidecar proxy vs. frontend-only**: Proxy centralizes logic but adds network hop. Frontend-only is faster but more complex (every route needs path filtering).
- **Label count accuracy**: Recalculated per-user counts will differ from the library-wide counts. This is intentional — labels are scoped now.
- **Sidecar vs. patching PhotoPrism**: Sidecar approach is non-invasive (no fork/build of PP). PhotoPrism patch would be cleaner but requires maintaining a fork.
## 8. Recommendation
1. **Build the labels proxy** (Phase 1) — it directly solves the label isolation problem and can be done with existing sidecar infrastructure
2. **Investigate review/archive leak** first — run the actual API query as muli to confirm whether the photos search is actually scoped. The code analysis says it should be, but the user reports otherwise. If confirmed as a real leak, implement the sidecar photos proxy (Phase 2)
3. **Sidebar counts** (Phase 3) — lower priority, can be done after labels and review/archive are fixed
Before building, confirm with the user whether they see cross-user photos in the actual grid or only in the sidebar counts for review/archive.

View File

@@ -0,0 +1,227 @@
# Plan: Fix remaining user isolation issues — 404 errors and folder tree
**Date:** 2026-06-06
**Author:** Hermes Agent
**Status:** Draft
---
## 1. Goal
Fix the remaining issues after deploying the sidecar scoping proxy:
1. **404 on photo grid** — "Request failed with status code 404" in private window
2. **Folder tree shows other users** — on first load, the library tree lists other users' folders; a refresh fixes it
## 2. Current Context
### What's deployed
| Component | Status |
|-----------|--------|
| Sidecar labels proxy (`/api/sidecar/labels`) | ✅ Working |
| Sidecar counts proxy (`/api/sidecar/counts`) | ✅ Working |
| Sidecar timeline proxy (`/api/sidecar/timeline`) | ✅ Working through Caddy |
| Caddy fallback for `/api/v1/api/sidecar/*` | ✅ Working |
| Frontend rebuild with `sidecar` axios instance | ✅ Built and deployed |
### Verified working via Caddy
```bash
# Through public URL with valid admin token
curl https://photos.hubris.network/api/sidecar/timeline?count=1 → HTTP 200
curl https://photos.hubris.network/api/v1/photos?count=1 → HTTP 200
```
Both endpoints return 200 when tested directly through Caddy with a valid token.
### Reported issues
1. **404 on photo grid** — even in private window (no cache interference)
2. **Folder tree shows other users' folders** on first load, fixed by refresh
## 3. Root Cause Analysis
### Issue 1: 404 on photo grid
The `sidecar` axios instance (`baseURL: ''`) is missing the **response interceptor** that:
- Handles 401 → clears session → redirects to login
- Re-throws with meaningful error message
The `http` instance (for `/api/v1` endpoints) has this interceptor. Without it on `sidecar`:
- If the sidecar returns a non-2xx (401, 502 from upstream PP failure, etc.), axios throws a raw error
- The TanStack Query error boundary catches it and shows "Request failed with status code <status>"
- Very likely the sidecar is returning 401 on some calls (token expired / session not yet established) and the error message might show 404 because Caddy's catch-all returns 404 when a matcher doesn't find a route
**Hypothesis:** During OIDC login flow, the frontend may make some sidecar calls BEFORE the session is fully established (token loaded into `session.accessToken`). The `sidecar` interceptor checks `session.accessToken` but it might be null. Then the request to `/api/sidecar/timeline` has no auth header → sidecar returns 401 → no response interceptor → raw error.
**Fix:** Add the same 401 → login redirect interceptor to the `sidecar` instance.
### Issue 2: Folder tree shows other users
`listFolders()` calls `http.get('/folders/originals')` which hits PhotoPrism directly. PhotoPrism returns **all folders across the library** regardless of user. The frontend then filters by `userBasePath()` on the result:
```typescript
const bp = userBasePath();
if (bp === '') return folders; // On first load, bp might be empty!
return folders.filter((f) => f.Path === bp || f.Path.startsWith(bp + '/'))
```
On first load, `userBasePath()` returns `""` because:
1. The session data is loaded asynchronously
2. `session.user.BasePath` might not yet be populated when `listFolders` fires
3. The TanStack Query cache from a previous session might still have old data
After a refresh, the session is fully loaded, and `userBasePath()` returns the correct value.
A secondary issue: the `http` interceptor's 401 handler clears the session on 401. If the session expires during the app's lifetime, all subsequent requests fail with 401.
## 4. Proposed Approach
### Phase 1: Fix 404 — add response interceptor to sidecar
**File:** `web/src/lib/services/photoprism.ts`
Add the same 401 → login redirect interceptor to `sidecar` as already exists on `http`:
```typescript
sidecar.interceptors.response.use(
(r) => r,
(err: AxiosError) => {
if (err.response?.status === 401 && browser) {
clearSession();
const url = err.config?.url ?? '';
if (!url.endsWith('/session')) {
void goto('/login', { replaceState: true });
}
}
return Promise.reject(err);
}
);
```
### Phase 2: Fix folder tree — sidecar folder proxy
**File:** `sidecar/handlers_folders.go` (new)
Add a sidecar endpoint that proxies `/folders/originals` and post-filters by BasePath:
```
GET /api/sidecar/folders → proxies to GET /api/v1/folders/originals
→ removes folders not under user's base_path
→ returns filtered list
```
This avoids the timing issue entirely by filtering on the server side.
**Alternative (simpler):** Fix the frontend timing issue by ensuring `listFolders` doesn't fire until the session is ready.
### Phase 3: Change folder tree in frontend
**File:** `web/src/lib/services/photoprism.ts`
Change `listFolders()` to use `sidecar` instance and call `/api/sidecar/folders`:
```typescript
export async function listFolders(): Promise<PpFolder[]> {
const { data } = await sidecar.get<{ folders?: PpFolder[] }>(
'/api/sidecar/folders',
{ params: { recursive: true, uncached: true, files: false } }
);
const bp = userBasePath();
const folders = data.folders ?? [];
if (bp === '') return folders;
return folders
.filter((f) => f.Path === bp || f.Path.startsWith(bp + '/'))
.map((f) => ({ ...f, Path: toUserPath(f.Path) }));
}
```
## 5. Step-by-step Plan
### Step 1: Add sidecar response interceptor
1. Edit `web/src/lib/services/photoprism.ts`
2. Add the 401-handling response interceptor to the `sidecar` instance
3. The interceptor mirrors the existing `http` response interceptor exactly
### Step 2: Rebuild frontend
```bash
cd /opt/mule-image/web && npm run build
```
### Step 3: (Optional) Add sidecar folder proxy
1. New file `sidecar/handlers_folders_proxy.go`
2. Handler similar to `handlePhotos` — proxies to `/api/v1/folders/originals`, post-filters by `Path` prefix
3. Wire route in `main.go`: `auth.GET("/folders", handleFoldersProxy(pp))`
4. Build Docker image, restart sidecar
### Step 4: Update listFolders to use sidecar
1. Change `listFolders()` to use `sidecar` instance
2. Call `/api/sidecar/folders` instead of `/folders/originals`
### Step 5: Rebuild + validate
```bash
# Rebuild frontend
cd /opt/mule-image/web && npm run build
# Test through Caddy
curl -s "https://photos.hubris.network/api/sidecar/timeline?count=1" \
-H "X-Auth-Token: <token>" | head -c 200
# Verify folders
curl -s "https://photos.hubris.network/api/sidecar/folders" \
-H "X-Auth-Token: <token>" | python3 -c "import sys,json;d=json.load(sys.stdin);print(json.dumps(d[:3],indent=2))"
```
### Step 6: Commit
```bash
git add -A && git commit -m "fix: add sidecar response interceptor + folder proxy" && git push
```
## 6. Files Likely to Change
| File | Change |
|------|--------|
| `web/src/lib/services/photoprism.ts` | Add response interceptor to sidecar instance; change listFolders URL |
| `sidecar/handlers_folders_proxy.go` | **New** — folder proxy handler |
| `sidecar/main.go` | Wire folder proxy route |
## 7. Tests & Validation
**Manual:**
1. Open private window → navigate to photos.hubris.network
2. Log in as muli via Authentik OIDC
3. Verify photo grid loads without 404
4. Verify folder tree shows only muli's folders
5. Switch to dtoro account → verify folders/timeline scoped to dtoro
**API tests:**
```bash
# Sidecar timeline (no token → 401 redirect)
curl -s "https://photos.hubris.network/api/sidecar/timeline?count=1"
# Sidecar folders
curl -s "https://photos.hubris.network/api/sidecar/folders"
```
## 8. Risks & Open Questions
### Risks
| Risk | Impact | Mitigation |
|------|--------|------------|
| Sidecar returns 401 during OIDC login flow before session is ready | 404 showing instead of graceful redirect | Add response interceptor in Phase 1 |
| Folder proxy adds latency | Slower folder tree loading | Minimal — single proxy call, same as PP direct |
| `userBasePath()` timing issue in listFolders persists even with sidecar | Folder tree still shows wrong folders on first load | Sidecar filter is server-side → no timing dependency |
### Open Questions
- **Q1**: Are there other API calls that bypass the `sidecar` instance and might also be unscoped? (e.g., `listSubjects`, `listGeo`, etc.)
- **Q2**: Does the sidecar need a folder proxy, or is the timing fix sufficient? The timing fix (delaying `listFolders` until session is ready) is simpler but fragile.
- **Q3**: Could the 404 be from Caddy's catch-all returning 404 when the sidecar isn't reachable? The Caddy fallback timeout for the sidecar might need tuning.

330
README.md
View File

@@ -1,230 +1,140 @@
# Mulita - Self-Hosted Photo Management Application
# mule-image
A self-hosted, Docker-deployed photo management application inspired by Lightroom's workflow. Mulita provides a fast, keyboard-driven interface to browse, organize, tag, and manage your photo library.
## Features
- **Photo Organization**: Browse photos in a timeline view with virtual scrolling for performance
- **Thumbnail Generation**: Automatic thumbnail generation for all photo formats including RAW
- **Metadata Extraction**: Full EXIF/XMP metadata extraction and GPS mapping
- **Keyboard Shortcuts**: Lightroom-style keyboard navigation and actions
- **File Support**: JPEG, PNG, RAW formats (CR2, CR3, NEF, ARW, etc.), HEIC/HEIF, and videos
- **Heaps**: Temporary collections for organizing photos
- **Tags & Ratings**: Organize with tags, star ratings, and color labels — each with a card-grid browse view that drills into a full Timeline detail
- **Dark Mode**: Photography-optimized dark interface
- **Vision Pipeline**: YOLO object detection, OCR text extraction, CLIP embeddings for semantic search, InsightFace face detection and clustering
- **People View**: Browse identified people as cards, click to see all photos of a person
- **Map View**: Browse GPS-tagged photos on an interactive Leaflet map
- **Duplicate Detection**: Perceptual hash-based duplicate grouping with best-pick UI
- **Semantic Search**: Natural-language photo search powered by CLIP embeddings
## Tech Stack
### Backend
- Python 3.12 with FastAPI
- PostgreSQL + pgvector with SQLAlchemy (async) and Alembic migrations
- Celery + Redis for background tasks
- pyvips for fast thumbnail generation
- ExifTool for metadata extraction
- ONNX Runtime for vision models (YOLO, CLIP, InsightFace)
### Frontend
- React 18 with TypeScript
- Vite for fast development
- TanStack Query for data fetching
- TanStack Virtual for virtualized scrolling
- Tailwind CSS for styling
- Zustand for state management
## Quick Start
### Prerequisites
- Docker and Docker Compose
### Setup (one variable)
1. Clone the repo:
```bash
git clone <repository-url>
cd muleimage
```
2. Copy the example env file and set **one** variable — the **host**
directory that contains your photo library. Whatever you point at
will become your library inside Mulita.
```bash
cp .env.example .env
# then edit .env and set PHOTO_DIRS:
# macOS / Linux: PHOTO_DIRS=/Users/you/Pictures
# Network share: PHOTO_DIRS=/mnt/nas/photos
# Windows (WSL): PHOTO_DIRS=/mnt/c/Users/you/Pictures
```
3. Start the stack:
```bash
docker compose up -d
```
4. Open `http://localhost:3000`. On first boot Mulita will:
- Mount your `PHOTO_DIRS` at `/photos` inside the container
- Auto-create a source root called **Library** pointing at `/photos`
- Queue an initial scan, generate thumbnails, and start serving them
You don't need to touch `mulita.yml` or the API to get started.
### Configuration knobs
Everything is environment-driven. `PHOTO_DIRS` is the only required
value; the rest have sensible defaults documented in `.env.example`:
| Variable | Default | Notes |
|----------------------|---------|----------------------------------------------------|
| `PHOTO_DIRS` | — | **Required.** Host path mounted at `/photos`. |
| `FRONTEND_PORT` | `3000` | SPA host port. Bump if `3000` is taken. |
| `BACKEND_PORT` | `8001` | Direct backend port (debug only — frontend uses internal nginx proxy). |
| `REDIS_PORT` | `6379` | Redis host port (internal services don't need it). |
| `ALLOWED_ORIGINS` | `*` | Comma-separated CORS origins for direct backend access. Lock down for prod, e.g. `https://photos.example.com`. |
| `LOG_LEVEL` | `INFO` | Backend + worker log level. `DEBUG` for chasing scan issues. |
| `TZ` | `UTC` | Container timezone. Affects log timestamps and "added at". |
| `CELERYD_CONCURRENCY`| `4` | Parallel worker processes (scans, thumbs, metadata). Lower on a Pi, higher on a beefy host. |
### Accessing from another machine
The frontend talks to the backend through its bundled nginx, which
proxies `/api/` to the backend on the internal compose network. That
means requests are always **same-origin** as the page, so accessing
Mulita from another host works without any CORS dance:
```
http://<your-server-ip>:3000
```
If you want to put it behind a reverse proxy at e.g.
`https://photos.your.tld`, set `ALLOWED_ORIGINS` to that host so the
backend's direct port (`BACKEND_PORT`) also accepts cross-origin
requests if anything bypasses the proxy.
### How libraries are managed
Mulita is **config-driven**: the host directory you mount via
`PHOTO_DIRS` becomes your library, and the backend automatically
registers it as a source root on startup. There is no UI for adding
or removing source roots — to change what Mulita scans, edit `.env`
(or `docker-compose.yml` for multi-mount setups) and restart the
stack.
This keeps the model simple: **the docker mount IS the library**.
No two layers, no confusion about which view to use.
### Changing or adding libraries
To point at a different library:
1. Edit `PHOTO_DIRS` in `.env`
2. `docker compose down`
3. (Optional, for a clean slate) `docker volume rm muleimage_db_data muleimage_thumbs_data muleimage_proxies_data`
4. `docker compose up -d`
The new library shows up automatically. Without step 3 the old
library's metadata stays in the DB and you'll see a warning at
startup that the old source root's path is missing on disk —
that's a hint to clean up.
For multiple libraries, edit `docker-compose.yml` and add additional
mount lines:
```yaml
volumes:
- ${PHOTO_DIRS}:/photos:rw
- /Volumes/Archive:/archive:rw # additional library
```
Each mounted directory will need a corresponding source root row in
the DB; today that means `POST /api/v1/folders` via curl, or wait
for the multi-mount auto-registration that's on the roadmap.
### Read-only libraries
The default mount is `:rw` because file operations (rename, move,
empty discard pile) need to mutate the filesystem. If you want a
strict read-only library — pointing at a network share, an
authoritative archive, etc. — flip `:rw` to `:ro` in
`docker-compose.yml`. Mulita will keep working for browsing, rating,
color labels, picks, heaps, and the (soft) discard flag, but the
following will return an OS error:
- `PATCH /photos/{id}` with a new `filename` (rename)
- `POST /photos/move` (bulk move)
- `DELETE /discard/empty` (file unlinks)
**Heads up**: with `:rw`, Mulita has full write access to whatever
host directory you mount. Treat the same way you would Lightroom's
catalog folder.
Self-hosted photo management built on top of [PhotoPrism][pp]. A SvelteKit
frontend ([`web/`](web/)) plus a small Go service ([`sidecar/`](sidecar/))
fill in the keyboard-driven UI and the file/folder/mark endpoints
PhotoPrism's REST API does not expose. PhotoPrism itself handles
indexing, originals, thumbnails, and the database; we never re-implement
those.
## Architecture
The application consists of 5 Docker services:
- **frontend**: React SPA served by Nginx
- **backend**: FastAPI REST API
- **worker**: Celery workers for background tasks (thumbnails, metadata, vision pipeline)
- **redis**: Message broker for Celery
- **db**: PostgreSQL with pgvector extension (for CLIP/face embeddings)
## Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `` `` `` `` | Navigate photos |
| `Space` | Quick preview |
| `Enter` | Open loupe view |
| `T` | Add to active heap |
| `1-5` | Set star rating |
| `Tab` | Toggle left sidebar |
| `I` | Toggle metadata panel |
| `G` | Grid view |
| `E` | Loupe view |
| `Delete` | Move to trash |
## Development
### Backend Development
```bash
cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload
```text
┌──────────────────┐ /api/v1/* ┌──────────────┐
│ SvelteKit web/ │ ───────────────▶ │ photoprism │ ──▶ mariadb
│ (Vite : 5173) │ /api/sidecar/* │ :2342 │
│ │ ─────────┐ └──────────────┘
└──────────────────┘ ▼
┌──────────────┐
│ sidecar │ ──▶ mariadb (mule_sidecar.*)
│ :8000 │ ──▶ originals FS (rename / folders / dups)
└──────────────┘
```
### Frontend Development
Three compose services — `mariadb`, `photoprism`, `sidecar` — plus the
SvelteKit `web/` app served separately. PhotoPrism's port `2342` is
**bound to `127.0.0.1` only**; it isn't a user-facing surface. The
SvelteKit app is.
What the sidecar adds on top of PhotoPrism (full list in
[`sidecar/README.md`](sidecar/README.md)):
- Per-photo marks (rating + color) persisted to `mule_sidecar.marks`
- File rename + folder create/rename/delete with PhotoPrism reindex
- Heap (album) → folder conversion
- Perceptual-hash duplicate scan + archive
## Quick start
```bash
cd frontend
cp .env.example .env
# edit .env: set PHOTO_DIRS to the host path holding your library
# rotate PP_ADMIN_PASSWORD, PP_DB_PASSWORD, PP_DB_ROOT_PASSWORD
# before any non-local deployment.
podman-compose --env-file .env \
-f docker-compose.yml \
-f docker-compose.podman.yml \
up -d
```
Then serve the frontend. For local use the simplest path is the Vite
dev server:
```bash
cd web
npm install
npm run dev
# open http://localhost:5173
```
For a static deployment, `npm run build` produces a bundle under
`web/build/` that any static file host (nginx, Caddy, GitHub Pages-style)
can serve. Reverse-proxy `/api/v1/*` to `http://127.0.0.1:2342` and
`/api/sidecar/*` to `http://127.0.0.1:8000`.
PhotoPrism's own UI is still reachable from the host at
`http://127.0.0.1:2342` if you need admin features (user management,
settings) — set up an SSH tunnel from your laptop if the server is
remote.
## Configuration
Source roots are managed by the UI / API (the database owns them). Edit
`mulita.yml` to configure operational settings only:
All knobs live in [`.env.example`](.env.example). The required ones:
- Thumbnail sizes, quality, and format
- Scanner behaviour (watch, batch size, initial scan)
- Performance tuning (concurrency, cache TTLs, DB pool)
| Variable | Notes |
|----------------------|-----------------------------------------------------------------------------------------------|
| `PHOTO_DIRS` | Host path mounted at `/photoprism/originals`. The library. |
| `PP_ADMIN_PASSWORD` | First-boot admin password. Rotate. |
| `PP_DB_PASSWORD` | MariaDB password for the `photoprism` user. Rotate. |
| `PP_DB_ROOT_PASSWORD`| MariaDB root password. Rotate. |
| `PP_UID` / `PP_GID` | Host UID/GID that owns `PHOTO_DIRS`. PhotoPrism + sidecar drop to this user inside. |
| `PP_PORT` | Loopback host port for PhotoPrism (default `2342`). |
| `PP_ORIGINALS_MODE` | `rw` (default) or `ro` — see [Read-only libraries](#read-only-libraries). |
| `SIDECAR_PORT` | Loopback host port for the sidecar (default `8000`). |
## Performance
Sidecar-specific env (DB DSN, `USER_BASEPATHS`, etc.) is documented in
[`sidecar/README.md`](sidecar/README.md).
- Handles 100,000+ photos efficiently
- Virtual scrolling for smooth timeline navigation
- Thumbnail generation at 10+ photos/second
- PostgreSQL full-text search with tsvector indexing
- pgvector for fast nearest-neighbor embedding search
## Read-only libraries
## Future Features
The default originals mount is `:rw` because file operations (rename,
folder mutations, duplicate archive, heap convert) need to mutate the
filesystem. To run against a read-only archive, set
`PP_ORIGINALS_MODE=ro` in `.env`. Browsing, marks, ratings, and color
labels still work; the following sidecar endpoints return an OS error:
- Smart albums (auto-populated by saved filters)
- Export presets
- Multi-user support
- `POST /api/sidecar/files/:uid/rename`
- `POST /api/sidecar/folders` / `:rel/rename` / `DELETE /:rel`
- `POST /api/sidecar/albums/:uid/convert`
- `POST /api/sidecar/duplicates/archive`
## License
PhotoPrism's `PHOTOPRISM_READONLY` is controlled separately by
`PP_READONLY` and gates its own backwrite / import paths.
MIT
## Dev iteration loop
For fast iteration on the sidecar without rebuilding its image on every
change, run it as a host process — bring up just `mariadb` and
`photoprism` from compose, then build and run the Go binary locally.
Full instructions in [`sidecar/README.md`](sidecar/README.md#dev-iteration-loop-host-build).
## Layout
```text
.
├── docker-compose.yml base stack: mariadb + photoprism + sidecar
├── docker-compose.podman.yml rootless-podman overlay (keep-id mapping)
├── docker-compose.gpu.yml opt-in VA-API GPU passthrough overlay
├── .env.example required env vars (copy to .env)
├── mariadb/init/ first-boot SQL: creates mule_sidecar DB + user
├── pp/ PhotoPrism bind-mounted state (storage, import)
├── sidecar/ Go service — see sidecar/README.md
└── web/ SvelteKit frontend
```
## GPU video acceleration (optional)
Hosts with a VA-API-capable GPU (Intel iGPU, AMD APU, etc.) can layer
[`docker-compose.gpu.yml`](docker-compose.gpu.yml) to hand `/dev/dri/*`
to PhotoPrism and switch ffmpeg to hardware encode/decode — a large
perf win for video thumbnails and HEVC→H.264 transcodes:
```bash
docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d
```
Set `PP_FFMPEG_ENCODER=vaapi` in `.env` (default for the overlay). Verify
with `docker exec pp-app photoprism show config | grep -i ffmpeg`.
[pp]: https://photoprism.app/

View File

@@ -1,43 +0,0 @@
FROM python:3.12-slim
# Install system dependencies
RUN apt-get update && apt-get install -y \
# Build dependencies
gcc \
g++ \
make \
# Image processing libraries
libvips42 \
libvips-dev \
# ExifTool for metadata extraction
libimage-exiftool-perl \
# FFmpeg for video processing
ffmpeg \
# Git for some Python packages
git \
# PostgreSQL client (for potential future use)
postgresql-client \
# Clean up
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Copy requirements first for better caching
COPY requirements.txt .
# Install PyTorch CPU-only FIRST so open-clip-torch doesn't pull the full
# CUDA build (~7 GB). CPU inference is all we need — the heavy lifting
# happens through ONNX Runtime.
RUN pip install --no-cache-dir torch torchvision --index-url https://download.pytorch.org/whl/cpu \
&& pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY . .
# Create necessary directories
RUN mkdir -p /data/thumbs /data/db /data/proxies /data/models /app/config
# Expose port
EXPOSE 8000
# Run the application
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

View File

@@ -1,48 +0,0 @@
# Alembic configuration for PhotoVault.
#
# The actual database URL is loaded at runtime by alembic/env.py from the
# DATABASE_URL environment variable (with the async driver suffix stripped).
# The placeholder below is only used for `alembic revision --autogenerate`
# when no env var is set.
[alembic]
script_location = alembic
prepend_sys_path = .
version_path_separator = os
sqlalchemy.url = postgresql+psycopg2://mulita:mulita@localhost:5432/mulita
[post_write_hooks]
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

View File

@@ -1,95 +0,0 @@
"""
Alembic environment for PhotoVault.
Pulls DATABASE_URL from the environment so the same migrations work in
docker compose and locally. Strips the async driver suffix because Alembic
runs synchronously via psycopg2.
Future-migration note
---------------------
Fresh installs run `Base.metadata.create_all` in `app.database.init_db`
*before* migrations would normally apply, so any migration that adds a
column / index / table to an object the model already declares will see
that object already present. Write migrations defensively:
op.execute("ALTER TABLE photos ADD COLUMN IF NOT EXISTS new_col TEXT")
op.execute("CREATE INDEX IF NOT EXISTS ix_foo ON foo(bar)")
For brand-new tables that the model also declares, the same applies — use
`op.execute("CREATE TABLE IF NOT EXISTS ...")` or check first.
"""
from logging.config import fileConfig
import os
import sys
from pathlib import Path
from sqlalchemy import engine_from_config, pool
from alembic import context
# Make `app` importable from this script.
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from app.database import Base # noqa: E402
# Import all models so they're registered on Base.metadata for autogenerate.
from app.models import ( # noqa: E402, F401
Photo,
Folder,
SourceRoot,
Tag,
Heap,
HeapPhoto,
)
config = context.config
# Resolve DATABASE_URL from env. Strip async driver suffixes — Alembic
# uses sync drivers.
db_url = os.environ.get("DATABASE_URL") or config.get_main_option("sqlalchemy.url")
if db_url:
if "+asyncpg" in db_url:
db_url = db_url.replace("+asyncpg", "+psycopg2")
elif db_url.startswith("postgresql://"):
db_url = db_url.replace("postgresql://", "postgresql+psycopg2://", 1)
elif "+aiosqlite" in db_url:
db_url = db_url.replace("+aiosqlite", "")
config.set_main_option("sqlalchemy.url", db_url)
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode (emit SQL only)."""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Run migrations against a live database."""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

View File

@@ -1,26 +0,0 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}

View File

@@ -1,27 +0,0 @@
"""baseline (empty)
Revision ID: 0001_baseline
Revises:
Create Date: 2026-04-10
The current schema is created by SQLAlchemy `Base.metadata.create_all` in
`app.database.init_db()` on first boot. Alembic only owns deltas from
PR3 onward. This baseline is intentionally empty so `alembic upgrade head`
on a fresh DB simply creates the `alembic_version` table and stamps it.
"""
from typing import Sequence, Union
# revision identifiers, used by Alembic.
revision: str = "0001_baseline"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass

View File

@@ -1,85 +0,0 @@
"""extend tags for vision pipeline
Revision ID: 0002_extend_tags
Revises: 0001_baseline
Create Date: 2026-04-10
Add kind, source, representative_photo_id to tags table.
Add confidence, bbox, source to photo_tags association.
Switch uniqueness from (name) to (name, kind).
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB
revision: str = "0002_extend_tags"
down_revision: Union[str, None] = "0001_baseline"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ── tags table ────────────────────────────────────────────────────
op.execute("ALTER TABLE tags ADD COLUMN IF NOT EXISTS kind VARCHAR NOT NULL DEFAULT 'user'")
op.execute("ALTER TABLE tags ADD COLUMN IF NOT EXISTS source VARCHAR")
op.execute("ALTER TABLE tags ADD COLUMN IF NOT EXISTS representative_photo_id VARCHAR REFERENCES photos(id) ON DELETE SET NULL")
# Create index on kind for filtering
op.execute("CREATE INDEX IF NOT EXISTS ix_tags_kind ON tags(kind)")
# Drop old unique constraint on name (if it exists) and add (name, kind).
# SQLAlchemy create_all may have created either — handle both cases.
op.execute("""
DO $$
BEGIN
-- Drop the old single-column unique index/constraint if present.
IF EXISTS (
SELECT 1 FROM pg_indexes
WHERE tablename = 'tags' AND indexname = 'ix_tags_name'
) THEN
DROP INDEX ix_tags_name;
END IF;
-- Some SQLAlchemy versions create a unique constraint directly.
IF EXISTS (
SELECT 1 FROM information_schema.table_constraints
WHERE table_name = 'tags' AND constraint_name = 'tags_name_key'
) THEN
ALTER TABLE tags DROP CONSTRAINT tags_name_key;
END IF;
END $$;
""")
op.execute("""
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'uq_tags_name_kind'
) THEN
ALTER TABLE tags ADD CONSTRAINT uq_tags_name_kind UNIQUE (name, kind);
END IF;
END $$;
""")
# ── photo_tags table ──────────────────────────────────────────────
op.execute("ALTER TABLE photo_tags ADD COLUMN IF NOT EXISTS confidence FLOAT")
op.execute("ALTER TABLE photo_tags ADD COLUMN IF NOT EXISTS bbox JSONB")
op.execute("ALTER TABLE photo_tags ADD COLUMN IF NOT EXISTS source VARCHAR")
def downgrade() -> None:
# photo_tags columns
op.drop_column("photo_tags", "source")
op.drop_column("photo_tags", "bbox")
op.drop_column("photo_tags", "confidence")
# tags: restore old unique constraint
op.execute("ALTER TABLE tags DROP CONSTRAINT IF EXISTS uq_tags_name_kind")
op.execute("CREATE UNIQUE INDEX IF NOT EXISTS ix_tags_name ON tags(name)")
# tags columns
op.drop_column("tags", "representative_photo_id")
op.drop_column("tags", "source")
op.drop_column("tags", "kind")

View File

@@ -1,52 +0,0 @@
"""pgvector embeddings
Revision ID: 0003_pgvector_embeddings
Revises: 0002_extend_tags
Create Date: 2026-04-10
Rewrite the embeddings table to use pgvector Vector(512) instead of
LargeBinary. Add composite PK (photo_id, model), created_at, and
HNSW index on vector column.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0003_pgvector_embeddings"
down_revision: Union[str, None] = "0002_extend_tags"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Drop the old placeholder table and recreate with pgvector types.
# No data to preserve — it was never populated.
op.execute("DROP TABLE IF EXISTS embeddings")
op.execute("""
CREATE TABLE embeddings (
photo_id VARCHAR NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
model VARCHAR(64) NOT NULL,
vector vector(512),
created_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (photo_id, model)
)
""")
# HNSW index for cosine similarity search.
# Defer creation on large backfills — drop and recreate afterward.
op.execute("""
CREATE INDEX IF NOT EXISTS ix_embeddings_vector_hnsw
ON embeddings USING hnsw (vector vector_cosine_ops)
""")
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS embeddings")
op.execute("""
CREATE TABLE embeddings (
photo_id VARCHAR NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
model VARCHAR,
vector BYTEA,
PRIMARY KEY (photo_id)
)
""")

View File

@@ -1,82 +0,0 @@
"""ocr_text table and Postgres FTS
Revision ID: 0004_ocr_fts
Revises: 0003_pgvector_embeddings
Create Date: 2026-04-10
Create ocr_text table for storing OCR results. Add a tsvector column
to photos for unified full-text search (filename + user_title +
user_notes) with a GIN index. OCR text is rolled up into a materialized
view or joined at query time.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0004_ocr_fts"
down_revision: Union[str, None] = "0003_pgvector_embeddings"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ── ocr_text table ────────────────────────────────────────────────
op.execute("""
CREATE TABLE IF NOT EXISTS ocr_text (
id VARCHAR PRIMARY KEY,
photo_id VARCHAR NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
text TEXT NOT NULL,
language VARCHAR(8) DEFAULT '',
confidence FLOAT,
bbox JSONB,
created_at TIMESTAMPTZ DEFAULT now()
)
""")
op.execute("CREATE INDEX IF NOT EXISTS ix_ocr_text_photo_id ON ocr_text(photo_id)")
# ── tsvector column on photos ─────────────────────────────────────
op.execute("ALTER TABLE photos ADD COLUMN IF NOT EXISTS search_vector tsvector")
op.execute("CREATE INDEX IF NOT EXISTS ix_photos_search_vector ON photos USING GIN (search_vector)")
# Trigger to auto-update search_vector on INSERT/UPDATE
op.execute("""
CREATE OR REPLACE FUNCTION photos_search_vector_update() RETURNS trigger AS $$
BEGIN
NEW.search_vector :=
setweight(to_tsvector('english', coalesce(NEW.filename, '')), 'A') ||
setweight(to_tsvector('english', coalesce(NEW.user_title, '')), 'A') ||
setweight(to_tsvector('english', coalesce(NEW.user_notes, '')), 'B');
RETURN NEW;
END
$$ LANGUAGE plpgsql;
""")
op.execute("""
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_trigger WHERE tgname = 'photos_search_vector_trigger'
) THEN
CREATE TRIGGER photos_search_vector_trigger
BEFORE INSERT OR UPDATE OF filename, user_title, user_notes
ON photos
FOR EACH ROW
EXECUTE FUNCTION photos_search_vector_update();
END IF;
END $$;
""")
# Backfill existing rows
op.execute("""
UPDATE photos SET search_vector =
setweight(to_tsvector('english', coalesce(filename, '')), 'A') ||
setweight(to_tsvector('english', coalesce(user_title, '')), 'A') ||
setweight(to_tsvector('english', coalesce(user_notes, '')), 'B')
""")
def downgrade() -> None:
op.execute("DROP TRIGGER IF EXISTS photos_search_vector_trigger ON photos")
op.execute("DROP FUNCTION IF EXISTS photos_search_vector_update()")
op.execute("DROP INDEX IF EXISTS ix_photos_search_vector")
op.execute("ALTER TABLE photos DROP COLUMN IF EXISTS search_vector")
op.execute("DROP TABLE IF EXISTS ocr_text")

View File

@@ -1,41 +0,0 @@
"""face_embeddings table
Revision ID: 0005_face_embeddings
Revises: 0004_ocr_fts
Create Date: 2026-04-10
Create face_embeddings table with pgvector Vector(128) for SFace
recognition embeddings and HNSW index.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0005_face_embeddings"
down_revision: Union[str, None] = "0004_ocr_fts"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute("""
CREATE TABLE IF NOT EXISTS face_embeddings (
id VARCHAR PRIMARY KEY,
photo_id VARCHAR NOT NULL REFERENCES photos(id) ON DELETE CASCADE,
bbox JSONB,
vector vector(128),
cluster_id VARCHAR REFERENCES tags(id) ON DELETE SET NULL,
quality FLOAT,
created_at TIMESTAMPTZ DEFAULT now()
)
""")
op.execute("CREATE INDEX IF NOT EXISTS ix_face_embeddings_photo_id ON face_embeddings(photo_id)")
op.execute("CREATE INDEX IF NOT EXISTS ix_face_embeddings_cluster_id ON face_embeddings(cluster_id)")
op.execute("""
CREATE INDEX IF NOT EXISTS ix_face_embeddings_vector_hnsw
ON face_embeddings USING hnsw (vector vector_cosine_ops)
""")
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS face_embeddings")

View File

@@ -1,39 +0,0 @@
"""face_embeddings vector 128 -> 512
Revision ID: 0006_face_512d
Revises: 0005_face_embeddings
Create Date: 2026-04-10
Resize face_embeddings.vector from Vector(128) to Vector(512) for
ArcFace embeddings (InsightFace). Drops existing data and HNSW index,
recreates both.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0006_face_512d"
down_revision: Union[str, None] = "0005_face_embeddings"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Drop index, truncate (old 128-d vectors are incompatible), resize
op.execute("DROP INDEX IF EXISTS ix_face_embeddings_vector_hnsw")
op.execute("DELETE FROM face_embeddings")
op.execute("ALTER TABLE face_embeddings ALTER COLUMN vector TYPE vector(512)")
op.execute("""
CREATE INDEX IF NOT EXISTS ix_face_embeddings_vector_hnsw
ON face_embeddings USING hnsw (vector vector_cosine_ops)
""")
def downgrade() -> None:
op.execute("DROP INDEX IF EXISTS ix_face_embeddings_vector_hnsw")
op.execute("DELETE FROM face_embeddings")
op.execute("ALTER TABLE face_embeddings ALTER COLUMN vector TYPE vector(128)")
op.execute("""
CREATE INDEX IF NOT EXISTS ix_face_embeddings_vector_hnsw
ON face_embeddings USING hnsw (vector vector_cosine_ops)
""")

View File

@@ -1,67 +0,0 @@
"""folders + photos is_hidden flag
Revision ID: 0007_folder_hidden
Revises: 0006_face_512d
Create Date: 2026-04-11
Adds an "exclude from cross-cutting views" flag:
folders.is_hidden — user-toggled on a folder or source root. When
true, photos in that subtree are hidden from
library-wide views (All Photos, Map, Tags,
People, Search, Duplicates, sidebar counts) but
remain indexed and visible when the user
navigates into the folder directly.
photos.is_hidden — denormalized: true iff any ancestor folder in
the photo's folder chain has is_hidden=true.
Kept as a real column (rather than a recursive
query per read) because the filter runs on
essentially every photo query in the app, and
the toggle operation that recomputes it is
rare. Indexed so `WHERE NOT is_hidden` doesn't
fall off the rating/taken_at indexes.
Both columns default to false so existing rows need no backfill.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0007_folder_hidden"
down_revision: Union[str, None] = "0006_face_512d"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"folders",
sa.Column(
"is_hidden",
sa.Boolean(),
nullable=False,
server_default=sa.false(),
),
)
op.add_column(
"photos",
sa.Column(
"is_hidden",
sa.Boolean(),
nullable=False,
server_default=sa.false(),
),
)
op.create_index(
"ix_photos_is_hidden",
"photos",
["is_hidden"],
)
def downgrade() -> None:
op.drop_index("ix_photos_is_hidden", table_name="photos")
op.drop_column("photos", "is_hidden")
op.drop_column("folders", "is_hidden")

View File

@@ -1,49 +0,0 @@
"""photos has_date_warning flag
Revision ID: 0008_photos_date_warning
Revises: 0007_folder_hidden
Create Date: 2026-04-11
Adds `photos.has_date_warning` — a denormalized boolean that's true when
the scanner's folder/filename date guesser disagrees with the stored
taken_at by more than 24h (or taken_at is missing and the path would
provide a date). Surfacing this as a real column means the filter bar
can restrict the timeline to suspicious photos without the client
recomputing the heuristic for every row.
Indexed because the filter is meant to run on top of the existing
taken_at / folder queries that dominate the timeline, and we want the
partial `WHERE has_date_warning` scan to stay cheap as the library
grows.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0008_photos_date_warning"
down_revision: Union[str, None] = "0007_folder_hidden"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"photos",
sa.Column(
"has_date_warning",
sa.Boolean(),
nullable=False,
server_default=sa.false(),
),
)
op.create_index(
"ix_photos_has_date_warning",
"photos",
["has_date_warning"],
)
def downgrade() -> None:
op.drop_index("ix_photos_has_date_warning", table_name="photos")
op.drop_column("photos", "has_date_warning")

View File

@@ -1,144 +0,0 @@
"""users table and user_id foreign keys
Revision ID: 0009_users_and_auth
Revises: 0008_photos_date_warning
Create Date: 2026-04-12
Introduces multi-user support:
1. Creates the `users` table.
2. Adds `user_id` FK columns to photos, folders, source_roots, heaps, tags.
3. For existing installs: creates a default admin user and assigns all
existing rows to that user. The generated password is printed to the
backend logs — the admin should change it on first login.
4. Replaces the unique constraint on tags (name, kind) with
(name, kind, user_id) so each user can have their own tags.
"""
from typing import Sequence, Union
import uuid
import secrets
from alembic import op
import sqlalchemy as sa
revision: str = "0009_users_and_auth"
down_revision: Union[str, None] = "0008_photos_date_warning"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
# 1. Create users table (IF NOT EXISTS — safe on fresh installs where
# init_db's create_all has already laid down the schema).
conn.execute(sa.text("""
CREATE TABLE IF NOT EXISTS users (
id VARCHAR NOT NULL PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE,
email VARCHAR UNIQUE,
hashed_password VARCHAR NOT NULL,
role VARCHAR NOT NULL DEFAULT 'user',
is_active BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMP WITHOUT TIME ZONE DEFAULT now(),
media_path VARCHAR NOT NULL
)
"""))
conn.execute(sa.text(
"CREATE INDEX IF NOT EXISTS ix_users_username ON users (username)"
))
# 2. Add user_id columns (nullable initially for the data migration)
for table in ("photos", "folders", "source_roots", "heaps", "tags"):
conn.execute(sa.text(
f"ALTER TABLE {table} ADD COLUMN IF NOT EXISTS user_id VARCHAR"
))
conn.execute(sa.text(
f"CREATE INDEX IF NOT EXISTS ix_{table}_user_id ON {table} (user_id)"
))
# FK — check if it already exists before adding
fk_name = f"fk_{table}_user_id"
fk_exists = conn.execute(sa.text(
"SELECT 1 FROM information_schema.table_constraints "
"WHERE constraint_name = :name AND table_name = :tbl"
), {"name": fk_name, "tbl": table}).scalar()
if not fk_exists:
conn.execute(sa.text(
f"ALTER TABLE {table} ADD CONSTRAINT {fk_name} "
f"FOREIGN KEY (user_id) REFERENCES users(id)"
))
# 3. Data migration: if rows exist, create a default admin and assign
conn = op.get_bind()
photo_count = conn.execute(sa.text("SELECT COUNT(*) FROM photos")).scalar()
if photo_count > 0:
admin_id = str(uuid.uuid4())
generated_password = secrets.token_urlsafe(16)
# Hash the password using passlib at migration time
from passlib.context import CryptContext
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
hashed = pwd_context.hash(generated_password)
# Every user gets a subfolder — including the migrated admin.
conn.execute(
sa.text(
"INSERT INTO users (id, username, hashed_password, role, media_path) "
"VALUES (:id, :username, :hashed, :role, :media_path)"
),
{
"id": admin_id,
"username": "admin",
"hashed": hashed,
"role": "admin",
"media_path": "/photos/admin",
},
)
# Assign all existing rows to the default admin
for table in ("photos", "folders", "source_roots", "heaps", "tags"):
conn.execute(
sa.text(f"UPDATE {table} SET user_id = :uid WHERE user_id IS NULL"),
{"uid": admin_id},
)
import logging
logger = logging.getLogger("alembic.migration")
logger.warning(
f"=== MIGRATION 0009 === Default admin created. "
f"Username: admin | Password: {generated_password} | "
f"Change this password on first login!"
)
# 4. Replace tag unique constraint to include user_id
# Check whether the old constraint exists before trying to drop it
# (on fresh installs create_all creates the new constraint directly).
old_uq_exists = conn.execute(sa.text(
"SELECT 1 FROM information_schema.table_constraints "
"WHERE constraint_name = 'uq_tags_name_kind' AND table_name = 'tags'"
)).scalar()
if old_uq_exists:
op.drop_constraint("uq_tags_name_kind", "tags", type_="unique")
new_uq_exists = conn.execute(sa.text(
"SELECT 1 FROM information_schema.table_constraints "
"WHERE constraint_name = 'uq_tags_name_kind_user' AND table_name = 'tags'"
)).scalar()
if not new_uq_exists:
op.create_unique_constraint("uq_tags_name_kind_user", "tags", ["name", "kind", "user_id"])
def downgrade() -> None:
# Reverse the tag constraint
op.drop_constraint("uq_tags_name_kind_user", "tags", type_="unique")
op.create_unique_constraint("uq_tags_name_kind", "tags", ["name", "kind"])
# Drop user_id columns and FKs
for table in ("photos", "folders", "source_roots", "heaps", "tags"):
op.drop_constraint(f"fk_{table}_user_id", table, type_="foreignkey")
op.drop_index(f"ix_{table}_user_id", table_name=table)
op.drop_column(table, "user_id")
# Drop users table
op.drop_index("ix_users_username", table_name="users")
op.drop_table("users")

View File

@@ -1,39 +0,0 @@
"""embeddings vector 512 -> 768
Revision ID: 0010_embeddings_768d
Revises: 0009_users_and_auth
Create Date: 2026-04-12
Resize embeddings.vector from Vector(512) to Vector(768) for
SigLIP2 ViT-B/16 embeddings. Drops existing data and HNSW index,
recreates with the new dimension. Existing embeddings will be
regenerated by the vision backfill task.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0010_embeddings_768d"
down_revision: Union[str, None] = "0009_users_and_auth"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute("DROP INDEX IF EXISTS ix_embeddings_vector_hnsw")
op.execute("DELETE FROM embeddings")
op.execute("ALTER TABLE embeddings ALTER COLUMN vector TYPE vector(768)")
op.execute("""
CREATE INDEX IF NOT EXISTS ix_embeddings_vector_hnsw
ON embeddings USING hnsw (vector vector_cosine_ops)
""")
def downgrade() -> None:
op.execute("DROP INDEX IF EXISTS ix_embeddings_vector_hnsw")
op.execute("DELETE FROM embeddings")
op.execute("ALTER TABLE embeddings ALTER COLUMN vector TYPE vector(512)")
op.execute("""
CREATE INDEX IF NOT EXISTS ix_embeddings_vector_hnsw
ON embeddings USING hnsw (vector vector_cosine_ops)
""")

View File

@@ -1,54 +0,0 @@
"""Add sharing tables for heaps and folders
Revision ID: 0011_sharing
Revises: 0010_embeddings_768d
Create Date: 2026-04-13
Adds heap_shares and folder_shares tables so users can share
heaps and folders with other users (read or read+write).
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0011_sharing"
down_revision: Union[str, None] = "0010_embeddings_768d"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute("""
CREATE TABLE IF NOT EXISTS heap_shares (
id VARCHAR NOT NULL PRIMARY KEY,
heap_id VARCHAR NOT NULL REFERENCES heaps(id) ON DELETE CASCADE,
owner_id VARCHAR NOT NULL REFERENCES users(id),
shared_with_id VARCHAR NOT NULL REFERENCES users(id),
permission VARCHAR NOT NULL DEFAULT 'read',
created_at TIMESTAMP DEFAULT now(),
CONSTRAINT uq_heap_share UNIQUE (heap_id, shared_with_id)
)
""")
op.execute("CREATE INDEX IF NOT EXISTS ix_heap_shares_shared_with ON heap_shares(shared_with_id)")
op.execute("CREATE INDEX IF NOT EXISTS ix_heap_shares_heap_id ON heap_shares(heap_id)")
op.execute("""
CREATE TABLE IF NOT EXISTS folder_shares (
id VARCHAR NOT NULL PRIMARY KEY,
folder_id VARCHAR NOT NULL,
folder_type VARCHAR NOT NULL DEFAULT 'folder',
owner_id VARCHAR NOT NULL REFERENCES users(id),
shared_with_id VARCHAR NOT NULL REFERENCES users(id),
permission VARCHAR NOT NULL DEFAULT 'read',
created_at TIMESTAMP DEFAULT now(),
CONSTRAINT uq_folder_share UNIQUE (folder_id, shared_with_id)
)
""")
op.execute("CREATE INDEX IF NOT EXISTS ix_folder_shares_shared_with ON folder_shares(shared_with_id)")
op.execute("CREATE INDEX IF NOT EXISTS ix_folder_shares_folder_id ON folder_shares(folder_id)")
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS folder_shares")
op.execute("DROP TABLE IF EXISTS heap_shares")

View File

@@ -1,65 +0,0 @@
"""Strip AI pipeline to binary classifier only
Revision ID: 0012_strip_ai
Revises: 0011_sharing
Create Date: 2026-04-14
Removes face recognition, OCR, object detection, and semantic embeddings.
The remaining AI is a single binary 'photography' vs 'other' classifier
whose output feeds Tag(kind='content_type') and a new Photo.needs_review
flag.
Drops: embeddings, face_embeddings, ocr_text tables.
Drops: photo_tags rows produced by 'vision:yolov8n' and 'vision:sface'.
Drops: tags with kind IN ('object','scene','face_cluster').
Drops: tags.representative_photo_id column.
Adds: photos.needs_review (bool, default false) + partial index.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0012_strip_ai"
down_revision: Union[str, None] = "0011_sharing"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Drop dropped-AI tables. CASCADE clears any lingering FKs/indices.
op.execute("DROP TABLE IF EXISTS embeddings CASCADE")
op.execute("DROP TABLE IF EXISTS face_embeddings CASCADE")
op.execute("DROP TABLE IF EXISTS ocr_text CASCADE")
# Clear ML-produced photo_tags rows and their parent tags.
op.execute(
"DELETE FROM photo_tags WHERE source IN ('vision:yolov8n','vision:sface')"
)
op.execute(
"DELETE FROM tags WHERE kind IN ('object','scene','face_cluster')"
)
# Drop the face-cluster representative column.
op.execute("ALTER TABLE tags DROP COLUMN IF EXISTS representative_photo_id")
# Add the needs_review flag.
op.execute(
"ALTER TABLE photos ADD COLUMN IF NOT EXISTS needs_review "
"BOOLEAN NOT NULL DEFAULT false"
)
op.execute(
"CREATE INDEX IF NOT EXISTS ix_photos_needs_review "
"ON photos(needs_review) WHERE needs_review"
)
def downgrade() -> None:
# Data is not recoverable on downgrade — only the schema stubs are
# put back so a future reinstall of the old pipeline can re-populate.
op.execute("DROP INDEX IF EXISTS ix_photos_needs_review")
op.execute("ALTER TABLE photos DROP COLUMN IF EXISTS needs_review")
op.execute(
"ALTER TABLE tags ADD COLUMN IF NOT EXISTS representative_photo_id "
"VARCHAR REFERENCES photos(id) ON DELETE SET NULL"
)

View File

@@ -1,35 +0,0 @@
"""Drop legacy content_type tags from the 6-category classifier
Revision ID: 0013_drop_old_ct
Revises: 0012_strip_ai
Create Date: 2026-04-14
The previous classifier wrote Tag(kind='content_type', name IN
('photograph','screenshot','document','receipt','meme','artwork')).
The new binary classifier writes names ('photography','other'). Both
coexisted after the cutover so users saw duplicate groupings like
'photography' alongside 'photograph'. Drop the old names — photo_tags
rows cascade-delete via the FK.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0013_drop_old_ct"
down_revision: Union[str, None] = "0012_strip_ai"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
LEGACY_NAMES = ('photograph', 'screenshot', 'document', 'receipt', 'meme', 'artwork')
def upgrade() -> None:
op.execute(
"DELETE FROM tags WHERE kind = 'content_type' "
f"AND name IN {LEGACY_NAMES}"
)
def downgrade() -> None:
pass

View File

@@ -1,47 +0,0 @@
"""
Authentication utilities — password hashing and JWT token management.
"""
from datetime import datetime, timedelta, timezone
from jose import jwt, JWTError
from passlib.context import CryptContext
from app.config import settings
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
ALGORITHM = "HS256"
def hash_password(plain: str) -> str:
return pwd_context.hash(plain)
def verify_password(plain: str, hashed: str) -> bool:
return pwd_context.verify(plain, hashed)
def create_access_token(user_id: str, role: str) -> str:
expire = datetime.now(timezone.utc) + timedelta(minutes=settings.access_token_expire_minutes)
payload = {
"sub": user_id,
"role": role,
"exp": expire,
"type": "access",
}
return jwt.encode(payload, settings.secret_key, algorithm=ALGORITHM)
def create_refresh_token(user_id: str) -> str:
expire = datetime.now(timezone.utc) + timedelta(days=settings.refresh_token_expire_days)
payload = {
"sub": user_id,
"exp": expire,
"type": "refresh",
}
return jwt.encode(payload, settings.secret_key, algorithm=ALGORITHM)
def decode_token(token: str) -> dict:
"""Decode and validate a JWT. Raises JWTError on any problem."""
return jwt.decode(token, settings.secret_key, algorithms=[ALGORITHM])

View File

@@ -1,178 +0,0 @@
"""
Application configuration using Pydantic Settings
"""
from pydantic_settings import BaseSettings
from pydantic import BaseModel, Field
from typing import Optional
import yaml
from pathlib import Path
class ThumbnailSettings(BaseModel):
"""Thumbnail generation settings"""
small: int = 240
medium: int = 640
large: int = 1280
quality: int = 85
format: str = "webp"
class ScannerSettings(BaseModel):
"""File scanner settings"""
watch: bool = True
initial_scan_on_start: bool = True
batch_size: int = 100
concurrent_workers: int = 4
class PerformanceSettings(BaseModel):
"""Performance tuning settings"""
max_concurrent_thumbnails: int = 10
cache_ttl: int = 3600
db_pool_size: int = 10
db_pool_max_overflow: int = 10
db_pool_recycle: int = 3600
class ClassifierSettings(BaseModel):
"""Binary content classifier (photography vs other)."""
min_confidence: float = 0.3
class VisionSettings(BaseModel):
"""Vision pipeline — one binary classifier (photography vs other)."""
enabled: bool = True
backend: str = "onnx"
models_dir: str = "/data/models"
execution_providers: list[str] = ["CPUExecutionProvider"]
classifier: ClassifierSettings = ClassifierSettings()
worker_concurrency: int = 2
class MulitaConfig(BaseModel):
"""Main configuration from YAML file. Source roots and the discard
workflow are owned by the database now — only operational settings
live here."""
thumbnails: ThumbnailSettings = ThumbnailSettings()
scanner: ScannerSettings = ScannerSettings()
performance: PerformanceSettings = PerformanceSettings()
vision: VisionSettings = VisionSettings()
class Settings(BaseSettings):
"""Application settings"""
# Database — Postgres + pgvector by default. The SQLite escape hatch
# remains supported via the docker-compose.sqlite.yml override and by
# setting DATABASE_URL=sqlite+aiosqlite:///... in .env for local dev.
database_url: str = Field(
default="postgresql+asyncpg://mulita:mulita@db:5432/mulita",
env="DATABASE_URL"
)
# Redis
redis_url: str = Field(
default="redis://localhost:6379",
env="REDIS_URL"
)
# Celery
celery_broker_url: str = Field(
default="redis://localhost:6379",
env="CELERY_BROKER_URL"
)
celery_result_backend: str = Field(
default="redis://localhost:6379",
env="CELERY_RESULT_BACKEND"
)
# Photo directories
photo_dirs: str = Field(
default="/photos",
env="PHOTO_DIRS"
)
# API settings
api_host: str = Field(default="0.0.0.0", env="API_HOST")
api_port: int = Field(default=8000, env="API_PORT")
# CORS — comma-separated list of allowed origins, or "*" for any.
# Same-origin requests (the normal case behind nginx / vite proxy)
# never trip CORS, so this is only for direct browser access from
# other origins (LAN IP, reverse proxy, dev tools).
allowed_origins: str = Field(default="*", env="ALLOWED_ORIGINS")
# Logging — accepts standard python levels (DEBUG, INFO, WARNING,
# ERROR, CRITICAL). Bumped from INFO when chasing a problem.
log_level: str = Field(default="INFO", env="LOG_LEVEL")
# Auth — JWT signing key. Set SECRET_KEY in .env for production.
# If unset, a deterministic fallback is used (acceptable for
# single-machine homelab deploys, but set a real key if the instance
# is network-exposed).
secret_key: str = Field(
default="mulita-dev-secret-change-me",
env="SECRET_KEY",
)
access_token_expire_minutes: int = Field(default=525600, env="ACCESS_TOKEN_EXPIRE_MINUTES") # 1 year
refresh_token_expire_days: int = Field(default=3650, env="REFRESH_TOKEN_EXPIRE_DAYS") # 10 years
@property
def cors_origins(self) -> list[str]:
"""Parse the ALLOWED_ORIGINS env var into a list. Accepts:
- "*" → wildcard (single-element list ["*"])
- "http://a.com,http://b.com" → split + strip
Empty entries are dropped.
"""
raw = (self.allowed_origins or "").strip()
if not raw or raw == "*":
return ["*"]
return [o.strip() for o in raw.split(",") if o.strip()]
# App configuration from YAML
_config: Optional[MulitaConfig] = None
@property
def config(self) -> MulitaConfig:
"""Load configuration from YAML file"""
if self._config is None:
config_path = Path("/app/config/mulita.yml")
if not config_path.exists():
config_path = Path("mulita.yml")
if config_path.exists():
with open(config_path, "r") as f:
config_data = yaml.safe_load(f)
self._config = MulitaConfig(**config_data)
else:
self._config = MulitaConfig()
return self._config
@property
def thumbnails(self) -> ThumbnailSettings:
return self.config.thumbnails
@property
def scanner(self) -> ScannerSettings:
return self.config.scanner
@property
def performance(self) -> PerformanceSettings:
return self.config.performance
# ONNX Runtime execution providers, overridable via env var.
# Comma-separated: "CUDAExecutionProvider,CPUExecutionProvider"
# or "auto" for GPU auto-detection.
vision_execution_providers: str = Field(
default="CPUExecutionProvider",
env="VISION_EXECUTION_PROVIDERS",
)
@property
def vision(self) -> VisionSettings:
v = self.config.vision
# Override execution_providers from env if set.
providers = [p.strip() for p in self.vision_execution_providers.split(",") if p.strip()]
if providers:
v.execution_providers = providers
return v
class Config:
env_file = ".env"
case_sensitive = False
# Global settings instance
settings = Settings()

View File

@@ -1,204 +0,0 @@
"""
Database configuration and session management.
Schema management strategy
--------------------------
Postgres (default): Alembic owns schema deltas. `alembic upgrade head` is
run before the app starts (in the container CMD). `init_db()` calls
`create_all` afterward as the source of truth for fresh installs — it is
idempotent for existing tables and creates any tables defined on
`Base.metadata` that don't yet exist. Future Alembic migrations should be
written defensively (`IF NOT EXISTS` etc.) so they remain safe to run on a
fresh DB where `create_all` has already laid down the same objects.
SQLite (escape hatch via docker-compose.sqlite.yml): no Alembic. The
historical inline ALTER TABLE block stays in place so existing dev
installs keep upgrading.
"""
import os
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmaker
from sqlalchemy.orm import declarative_base
from sqlalchemy.pool import NullPool
from sqlalchemy import text
import logging
from pathlib import Path
from app.config import settings
logger = logging.getLogger(__name__)
_is_sqlite = settings.database_url.startswith("sqlite")
_is_postgres = settings.database_url.startswith("postgresql")
# When running inside a Celery worker we use NullPool rather than the
# default connection pool. The reasons stack up:
#
# 1. Celery's prefork model forks the master *after* imports, so every
# child inherits the same asyncpg Connection objects — they share
# a socket, and two children using one concurrently raises
# "another operation is in progress".
#
# 2. Task bodies run under `asyncio.run()`, which spins up a fresh
# event loop per invocation. A pooled asyncpg Connection created
# on loop A, returned to the pool, and checked out on loop B
# raises "Future attached to a different loop".
#
# NullPool dodges both: every session checkout opens a brand-new
# connection on the *current* loop and the connection is closed at
# session end. Connection setup is cheap compared to task cost, so this
# is the right default for the worker. The FastAPI backend keeps the
# normal pool because it serves many short requests on a single long-
# lived event loop, where pooling is a clear win.
_is_celery_worker = os.environ.get("MULITA_CELERY_WORKER") == "1"
if _is_sqlite:
db_path = Path(settings.database_url.replace("sqlite+aiosqlite:///", ""))
db_path.parent.mkdir(parents=True, exist_ok=True)
engine = create_async_engine(
settings.database_url,
echo=False,
connect_args={
"check_same_thread": False,
"timeout": 30,
},
)
elif _is_celery_worker:
engine = create_async_engine(
settings.database_url,
echo=False,
poolclass=NullPool,
)
else:
engine = create_async_engine(
settings.database_url,
echo=False,
pool_size=settings.performance.db_pool_size,
max_overflow=settings.performance.db_pool_max_overflow,
pool_recycle=settings.performance.db_pool_recycle,
pool_pre_ping=True,
pool_timeout=10, # fail fast if pool exhausted (default 30)
# Kill connections idle in a transaction for >60s. Prevents leaked
# sessions from thumbnail requests that disconnect mid-flight.
connect_args={"server_settings": {"idle_in_transaction_session_timeout": "60000"}},
)
# Create async session factory
AsyncSessionLocal = async_sessionmaker(
engine,
class_=AsyncSession,
expire_on_commit=False
)
# Base class for models
Base = declarative_base()
async def get_db() -> AsyncSession:
"""Dependency to get database session.
Rolls back any uncommitted transaction before closing so a client
disconnect doesn't leave idle-in-transaction connections in the pool.
"""
async with AsyncSessionLocal() as session:
try:
yield session
except Exception:
await session.rollback()
raise
finally:
await session.close()
async def init_db():
"""Initialize database, create tables if they don't exist"""
async with engine.begin() as conn:
# Import all models to register them with Base
from app.models import User, Photo, Folder, SourceRoot, Tag, PhotoTag, Heap, HeapPhoto # noqa: F401
# Create all tables. Note: create_all only creates *missing* tables —
# it does NOT add new columns to existing tables when the model gains
# them. On Postgres, Alembic handles deltas; on SQLite, the inline
# ALTER block below is the legacy fallback.
await conn.run_sync(Base.metadata.create_all)
gps_columns_added = False
if _is_sqlite:
# WAL mode for better concurrency.
await conn.execute(text("PRAGMA journal_mode=WAL"))
await conn.execute(text("PRAGMA synchronous=NORMAL"))
await conn.execute(text("PRAGMA cache_size=10000"))
await conn.execute(text("PRAGMA temp_store=MEMORY"))
# ── Idempotent column adds (SQLite only) ─────────────────────
# SQLite supports ADD COLUMN but not "IF NOT EXISTS" for
# columns, so introspect via PRAGMA first. Each entry is
# (column_name, ALTER statement). Add new columns at the
# bottom. On Postgres these live in Alembic migrations.
existing_cols = {
row[1]
for row in (
await conn.execute(text("PRAGMA table_info(photos)"))
).fetchall()
}
pending_alters: list[tuple[str, str]] = [
("phash", "ALTER TABLE photos ADD COLUMN phash VARCHAR(16)"),
(
"duplicate_group_id",
"ALTER TABLE photos ADD COLUMN duplicate_group_id VARCHAR",
),
("latitude", "ALTER TABLE photos ADD COLUMN latitude REAL"),
("longitude", "ALTER TABLE photos ADD COLUMN longitude REAL"),
]
for col_name, alter_sql in pending_alters:
if col_name not in existing_cols:
logger.info(f"Adding photos.{col_name} column")
await conn.execute(text(alter_sql))
if col_name in ("latitude", "longitude"):
gps_columns_added = True
await conn.execute(
text("CREATE INDEX IF NOT EXISTS ix_photos_phash ON photos(phash)")
)
await conn.execute(
text(
"CREATE INDEX IF NOT EXISTS ix_photos_duplicate_group_id "
"ON photos(duplicate_group_id)"
)
)
await conn.execute(
text(
"CREATE INDEX IF NOT EXISTS ix_photos_lat_lon "
"ON photos(latitude, longitude)"
)
)
logger.info("Database initialized successfully")
# If we just introduced the GPS columns on an existing SQLite
# install, kick off a one-shot backfill so the Map view is
# populated without a manual full re-scan. Postgres installs are
# always fresh (no SQLite→PG migration path), so this code path
# is SQLite-only.
if _is_sqlite and gps_columns_added:
try:
from app.tasks.scan import backfill_gps
backfill_gps.delay()
logger.info("Queued one-shot backfill_gps task after column add")
except Exception as e:
logger.warning(f"Could not queue backfill_gps task: {e}")
async def create_fts_table():
"""Create Full-Text Search table for SQLite. On Postgres this is
replaced by a tsvector column on the photos table (added in PR5)."""
if _is_sqlite:
async with engine.begin() as conn:
# Create FTS5 virtual table for full-text search
await conn.execute(text("""
CREATE VIRTUAL TABLE IF NOT EXISTS photos_fts USING fts5(
photo_id UNINDEXED,
filename,
user_title,
user_notes,
exif_text,
tokenize='unicode61'
)
"""))
logger.info("FTS5 table created successfully")

View File

@@ -1,355 +0,0 @@
"""
FastAPI dependencies for authentication and user-scoped data access.
"""
from typing import Optional
from fastapi import Depends, HTTPException, Query, Request, status
from fastapi.security import OAuth2PasswordBearer
from jose import JWTError
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.auth import decode_token
from app.database import get_db
from app.models.user import User
from app.models.photos import Photo
from app.models.folders import Folder, SourceRoot
from app.models.heaps import Heap, heap_photos
from app.models.tags import Tag
from app.models.sharing import HeapShare, FolderShare
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def get_current_user(
token: str = Depends(oauth2_scheme),
db: AsyncSession = Depends(get_db),
) -> User:
"""Decode JWT, look up user, raise 401 if invalid or inactive."""
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
try:
payload = decode_token(token)
user_id: str = payload.get("sub")
token_type: str = payload.get("type")
if user_id is None or token_type != "access":
raise credentials_exception
except JWTError:
raise credentials_exception
result = await db.execute(select(User).where(User.id == user_id))
user = result.scalar_one_or_none()
if user is None or not user.is_active:
raise credentials_exception
return user
async def get_current_user_media(
request: Request,
token: Optional[str] = Query(None, alias="token"),
db: AsyncSession = Depends(get_db),
) -> User:
"""Authenticate via Authorization header OR ?token= query parameter.
Used for media endpoints (thumbnails, originals, proxies) where the
URL is set as an <img src> or <video src> and the browser can't
attach an Authorization header. The frontend appends ?token=JWT to
media URLs so they pass auth without custom fetch logic.
"""
# Try Authorization header first.
auth_header = request.headers.get("Authorization", "")
jwt_token = None
if auth_header.startswith("Bearer "):
jwt_token = auth_header[7:]
elif token:
jwt_token = token
if not jwt_token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Missing token",
headers={"WWW-Authenticate": "Bearer"},
)
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
try:
payload = decode_token(jwt_token)
user_id: str = payload.get("sub")
token_type: str = payload.get("type")
if user_id is None or token_type != "access":
raise credentials_exception
except JWTError:
raise credentials_exception
result = await db.execute(select(User).where(User.id == user_id))
user = result.scalar_one_or_none()
if user is None or not user.is_active:
raise credentials_exception
return user
async def require_admin(
user: User = Depends(get_current_user),
) -> User:
"""Raise 403 if user is not an admin."""
if user.role != "admin":
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin privileges required",
)
return user
# ---------------------------------------------------------------------------
# User-scoped query helpers
# ---------------------------------------------------------------------------
def user_photos_query(user: User):
"""Base select for photos owned by user, with tags eager-loaded."""
return (
select(Photo)
.options(selectinload(Photo.tags))
.where(Photo.user_id == user.id)
)
async def get_user_photo(
photo_id: str,
user: User,
db: AsyncSession,
) -> Photo:
"""Fetch a single photo by ID, scoped to the user. Raises 404."""
result = await db.execute(
select(Photo)
.options(selectinload(Photo.tags))
.where(Photo.id == photo_id, Photo.user_id == user.id)
)
photo = result.scalar_one_or_none()
if photo is None:
raise HTTPException(status_code=404, detail="Photo not found")
return photo
async def get_user_folder(
folder_id: str,
user: User,
db: AsyncSession,
) -> Folder:
"""Fetch a single folder by ID, scoped to the user. Raises 404."""
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)
folder = result.scalar_one_or_none()
if folder is None:
raise HTTPException(status_code=404, detail="Folder not found")
return folder
async def get_user_heap(
heap_id: str,
user: User,
db: AsyncSession,
) -> Heap:
"""Fetch a single heap by ID, scoped to the user. Raises 404."""
result = await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == user.id)
)
heap = result.scalar_one_or_none()
if heap is None:
raise HTTPException(status_code=404, detail="Heap not found")
return heap
async def get_user_tag(
tag_id: str,
user: User,
db: AsyncSession,
) -> Tag:
"""Fetch a single tag by ID, scoped to the user. Raises 404."""
result = await db.execute(
select(Tag).where(Tag.id == tag_id, Tag.user_id == user.id)
)
tag = result.scalar_one_or_none()
if tag is None:
raise HTTPException(status_code=404, detail="Tag not found")
return tag
# ---------------------------------------------------------------------------
# Sharing helpers
# ---------------------------------------------------------------------------
async def get_user_or_shared_heap(
heap_id: str,
user: User,
db: AsyncSession,
) -> tuple:
"""Fetch a heap the user owns OR has a share for.
Returns ``(heap, permission)`` where *permission* is
``'owner'``, ``'read'``, or ``'write'``. Raises 404 if no access.
"""
# Fast path: owned by current user.
result = await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == user.id)
)
heap = result.scalar_one_or_none()
if heap:
return heap, "owner"
# Shared path.
result = await db.execute(
select(HeapShare).where(
HeapShare.heap_id == heap_id,
HeapShare.shared_with_id == user.id,
)
)
share = result.scalar_one_or_none()
if share:
result = await db.execute(select(Heap).where(Heap.id == heap_id))
heap = result.scalar_one_or_none()
if heap:
return heap, share.permission
raise HTTPException(status_code=404, detail="Heap not found")
async def get_user_or_shared_folder(
folder_id: str,
user: User,
db: AsyncSession,
) -> tuple:
"""Fetch a folder (or source root) the user owns OR has a share for.
Returns ``(entity, permission)`` where *entity* is a Folder or
SourceRoot and *permission* is ``'owner'``, ``'read'``, or ``'write'``.
"""
# Try owned folder first.
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)
folder = result.scalar_one_or_none()
if folder:
return folder, "owner"
# Try owned source root.
result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == user.id)
)
sr = result.scalar_one_or_none()
if sr:
return sr, "owner"
# Shared path.
result = await db.execute(
select(FolderShare).where(
FolderShare.folder_id == folder_id,
FolderShare.shared_with_id == user.id,
)
)
share = result.scalar_one_or_none()
if share:
if share.folder_type == "source_root":
result = await db.execute(select(SourceRoot).where(SourceRoot.id == folder_id))
else:
result = await db.execute(select(Folder).where(Folder.id == folder_id))
entity = result.scalar_one_or_none()
if entity:
return entity, share.permission
raise HTTPException(status_code=404, detail="Folder not found")
async def resolve_username(
username: str,
db: AsyncSession,
) -> User:
"""Look up an active user by username. Raises 404 if not found."""
result = await db.execute(
select(User).where(User.username == username, User.is_active.is_(True))
)
user = result.scalar_one_or_none()
if user is None:
raise HTTPException(status_code=404, detail="User not found")
return user
async def can_access_photo_via_share(
photo_id: str,
user: User,
db: AsyncSession,
) -> bool:
"""Check whether *user* can access *photo_id* through any share.
Returns True if the photo belongs to a heap or folder that has been
shared with the user. Used as a fallback in media-serving endpoints
after the direct ownership check fails.
"""
import os
# Check heap shares: photo in any heap shared with user?
result = await db.execute(
select(heap_photos.c.photo_id).where(
heap_photos.c.photo_id == photo_id,
heap_photos.c.heap_id.in_(
select(HeapShare.heap_id).where(HeapShare.shared_with_id == user.id)
),
).limit(1)
)
if result.scalar_one_or_none() is not None:
return True
# Check folder shares: photo in any folder (or descendant) shared with user?
result = await db.execute(
select(Photo.folder_id).where(Photo.id == photo_id)
)
photo_folder_id = result.scalar_one_or_none()
if photo_folder_id is None:
return False
# Get the photo's folder path for prefix matching.
result = await db.execute(
select(Folder.path, Folder.source_root_id).where(Folder.id == photo_folder_id)
)
row = result.one_or_none()
if row is None:
return False
photo_path, photo_sr_id = row
# Check source root shares — photo's source root matches a shared root?
result = await db.execute(
select(FolderShare.folder_id).where(
FolderShare.shared_with_id == user.id,
FolderShare.folder_type == "source_root",
FolderShare.folder_id == photo_sr_id,
).limit(1)
)
if result.scalar_one_or_none() is not None:
return True
# Check folder shares — photo's folder is at or below a shared folder?
# Single query: join folder_shares → folders to get shared paths, then
# check if the photo's path starts with any of them.
result = await db.execute(
select(Folder.path).where(
Folder.id.in_(
select(FolderShare.folder_id).where(
FolderShare.shared_with_id == user.id,
FolderShare.folder_type == "folder",
)
)
)
)
for (shared_path,) in result.all():
if photo_path == shared_path or photo_path.startswith(shared_path + os.sep):
return True
return False

View File

@@ -1,114 +0,0 @@
"""
Mulita - Photo Management Application
Main FastAPI application entry point
"""
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
import logging
import os
from app.config import settings
from app.database import init_db
from app.routers import photos, folders, heaps, tags, discard, library, search, auth, admin, sharing, upload, download, features
from app.services.scanner import start_initial_scan, bootstrap_default_source_root
from app.services.cleanup import cleanup_data_integrity
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Manage application lifecycle"""
logger.info("Starting Mulita application...")
# Initialize database
await init_db()
# First-boot convenience: if there are no source roots in the DB yet,
# create one for the default /photos mount so the user sees their
# library immediately without configuring anything in the UI.
try:
await bootstrap_default_source_root()
except Exception as e:
logger.error(f"Bootstrap source root failed (continuing): {e}")
# One-shot cleanup of duplicate source_roots / folders left over from
# earlier scanner versions that didn't normalize paths. Idempotent.
try:
await cleanup_data_integrity()
except Exception as e:
logger.error(f"Startup cleanup failed (continuing): {e}")
# Start initial scan if configured
if settings.scanner.initial_scan_on_start:
logger.info("Starting initial library scan...")
await start_initial_scan()
yield
logger.info("Shutting down Mulita application...")
# Create FastAPI app
app = FastAPI(
title="Mulita Photo Management API",
description="Self-hosted photo management application inspired by Lightroom",
version="1.0.0",
lifespan=lifespan
)
# Configure CORS. The frontend normally talks to the backend through the
# nginx (prod) or vite (dev) proxy, so requests are same-origin and never
# trip CORS. ALLOWED_ORIGINS in .env controls the fallback for direct
# browser access from other origins (LAN IP, reverse proxy under a
# different host). Defaults to "*" since this is a single-user homelab
# tool; lock it down by setting e.g. ALLOWED_ORIGINS=https://photos.your.tld
# in production deployments.
_origins = settings.cors_origins
app.add_middleware(
CORSMiddleware,
allow_origins=_origins,
# Wildcard origins can't be combined with credentials per the CORS
# spec, so credentials get auto-disabled in that case.
allow_credentials=_origins != ["*"],
allow_methods=["*"],
allow_headers=["*"],
)
# Mount static files for serving thumbnails (with X-Accel-Redirect support)
if os.path.exists("/data/thumbs"):
app.mount("/thumbs", StaticFiles(directory="/data/thumbs"), name="thumbs")
# Include routers
app.include_router(auth.router, prefix="/api/v1/auth", tags=["auth"])
app.include_router(admin.router, prefix="/api/v1/admin", tags=["admin"])
app.include_router(sharing.router, prefix="/api/v1", tags=["sharing"])
app.include_router(photos.router, prefix="/api/v1/photos", tags=["photos"])
app.include_router(folders.router, prefix="/api/v1/folders", tags=["folders"])
app.include_router(heaps.router, prefix="/api/v1/heaps", tags=["heaps"])
app.include_router(tags.router, prefix="/api/v1/tags", tags=["tags"])
app.include_router(discard.router, prefix="/api/v1/discard", tags=["discard"])
app.include_router(library.router, prefix="/api/v1/library", tags=["library"])
app.include_router(search.router, prefix="/api/v1/photos/search", tags=["search"])
app.include_router(upload.router, prefix="/api/v1/upload", tags=["upload"])
app.include_router(download.router, prefix="/api/v1/download", tags=["download"])
app.include_router(features.router, prefix="/api/v1/features", tags=["features"])
@app.get("/")
async def root():
"""Root endpoint"""
return {
"name": "Mulita Photo Management API",
"version": "1.0.0",
"status": "running"
}
@app.get("/health")
async def health_check():
"""Health check endpoint for Docker"""
return {"status": "healthy"}

View File

@@ -1,22 +0,0 @@
"""
Database models for Mulita
"""
from app.models.user import User
from app.models.photos import Photo
from app.models.folders import Folder, SourceRoot
from app.models.tags import Tag, PhotoTag
from app.models.heaps import Heap, HeapPhoto
from app.models.sharing import HeapShare, FolderShare
__all__ = [
'User',
'Photo',
'Folder',
'SourceRoot',
'Tag',
'PhotoTag',
'Heap',
'HeapPhoto',
'HeapShare',
'FolderShare',
]

View File

@@ -1,58 +0,0 @@
"""
Folder and SourceRoot model definitions
"""
from sqlalchemy import Column, String, Integer, Boolean, DateTime, ForeignKey, Index
from sqlalchemy.sql import func
from sqlalchemy.orm import relationship
import uuid
from app.database import Base
class SourceRoot(Base):
__tablename__ = 'source_roots'
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
path = Column(String, unique=True, nullable=False)
is_active = Column(Boolean, default=True)
added_at = Column(DateTime, server_default=func.now())
# Owner
user_id = Column(String, ForeignKey('users.id'), nullable=True, index=True)
# Relationships
folders = relationship("Folder", back_populates="source_root")
class Folder(Base):
__tablename__ = 'folders'
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
path = Column(String, unique=True, nullable=False)
parent_id = Column(String, ForeignKey('folders.id'))
source_root_id = Column(String, ForeignKey('source_roots.id'))
# Owner
user_id = Column(String, ForeignKey('users.id'), nullable=True, index=True)
photo_count = Column(Integer, default=0)
last_scanned = Column(DateTime)
# "Hide from views" — when true, photos in this folder (and every
# descendant folder) are excluded from cross-cutting views like
# All Photos, Map, Tags, People, Search and the sidebar counts.
# Photos are still scanned, thumbnailed and indexed — they just
# stop showing up unless the user navigates directly to a folder
# inside the hidden subtree. The effective flag is materialized
# onto Photo.is_hidden so queries don't have to walk parent_id.
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false')
# Relationships
source_root = relationship("SourceRoot", back_populates="folders")
photos = relationship("Photo", backref="folder")
# Indexes
__table_args__ = (
Index('ix_folders_path', 'path'),
Index('ix_folders_parent_id', 'parent_id'),
Index('ix_folders_source_root_id', 'source_root_id'),
)

View File

@@ -1,40 +0,0 @@
"""
Heap model definitions
"""
from sqlalchemy import Column, String, Integer, Boolean, DateTime, ForeignKey, Table, Index
from sqlalchemy.sql import func
from sqlalchemy.orm import relationship
import uuid
from app.database import Base
# Association table for many-to-many relationship with additional fields
heap_photos = Table(
'heap_photos',
Base.metadata,
Column('heap_id', String, ForeignKey('heaps.id', ondelete='CASCADE'), primary_key=True),
Column('photo_id', String, ForeignKey('photos.id', ondelete='CASCADE'), primary_key=True),
Column('added_at', DateTime, server_default=func.now()),
Column('sort_order', Integer, default=0),
Index('ix_heap_photos_heap_id', 'heap_id'),
Index('ix_heap_photos_photo_id', 'photo_id'),
)
class Heap(Base):
__tablename__ = 'heaps'
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
created_at = Column(DateTime, server_default=func.now())
updated_at = Column(DateTime, onupdate=func.now())
is_active = Column(Boolean, default=False) # For active heap feature
# Owner
user_id = Column(String, ForeignKey('users.id'), nullable=True, index=True)
# Relationships
photos = relationship("Photo", secondary=heap_photos, backref="heaps")
class HeapPhoto:
"""Helper class for heap-photo associations (not a table model)"""
pass

View File

@@ -1,127 +0,0 @@
"""
Photo model definition
"""
from sqlalchemy import Column, String, Integer, Float, Boolean, DateTime, ForeignKey, Text, Index
from sqlalchemy.sql import func
from datetime import datetime
import uuid
from app.database import Base
class Photo(Base):
__tablename__ = 'photos'
# Primary key
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
# Owner
user_id = Column(String, ForeignKey('users.id'), nullable=True, index=True)
# File information
filepath = Column(String, unique=True, nullable=False)
filename = Column(String, nullable=False)
folder_id = Column(String, ForeignKey('folders.id'))
file_hash = Column(String, index=True) # SHA-256 hash for duplicate detection
# Media information
media_type = Column(String, nullable=False) # 'photo' | 'video' | 'raw' | 'heic'
original_format = Column(String) # 'CR3', 'NEF', 'HEIC', 'MP4', 'JPEG', etc.
width = Column(Integer)
height = Column(Integer)
file_size = Column(Integer)
# Timestamps
taken_at = Column(DateTime) # from EXIF DateTimeOriginal, fallback to file mtime
taken_at_source = Column(String) # 'exif' | 'filesystem' | 'manual'
added_at = Column(DateTime, server_default=func.now())
updated_at = Column(DateTime, onupdate=func.now())
# Discard status. The DB column names stay is_trashed/trashed_at to avoid
# a migration; only the Python attribute name reflects the rename.
is_discarded = Column('is_trashed', Boolean, default=False)
discarded_at = Column('trashed_at', DateTime)
# "Hidden from views" — materialized from Folder.is_hidden walking
# the ancestry chain. True iff any ancestor folder (including the
# photo's direct folder) is hidden. Cross-cutting queries filter
# `AND NOT is_hidden`; per-folder browses ignore the flag so the
# user can still open a hidden folder and see its contents. The
# column is maintained by two places: the scanner sets it on new
# rows, and POST /folders/{id}/hide recomputes it on toggle.
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# "Needs review" — set by the content classifier when a photo is
# classified as 'other' (screenshot, document, meme, scan, etc.) so
# the user can page through non-photographs in the UI and triage them.
needs_review = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# "Capture date is probably wrong" — denormalized from the folder/filename
# date-guesser. Set at scan time and recomputed on every taken_at edit so
# the filter bar can query it directly. See services/date_guess.py for
# the heuristic; kept as a stored column because recomputing on every
# list query would mean running the regex stack across thousands of rows.
has_date_warning = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# Thumbnail paths
thumb_small = Column(String) # path to 240px thumb
thumb_medium = Column(String) # path to 640px thumb
thumb_large = Column(String) # path to 1280px thumb
# Processing status
processing_status = Column(String, default='pending') # 'pending' | 'processing' | 'completed' | 'failed'
processing_error = Column(Text)
# Metadata
exif_json = Column(Text) # full EXIF/XMP blob as JSON
# GPS coordinates extracted from EXIF, in signed decimal degrees
# (S latitude / W longitude are negative). Stored as first-class columns
# so the Map view and any future location filters can query/index them
# without parsing exif_json on every request.
latitude = Column(Float)
longitude = Column(Float)
# User-editable fields
user_title = Column(String)
user_notes = Column(Text)
rating = Column(Integer, default=0) # 0-5 stars
color_label = Column(String) # 'red'|'orange'|'yellow'|'green'|'blue'|'purple'|NULL
# Note: is_rejected was merged into is_discarded (a single soft "discarded"
# concept). is_picked was unified with active-heap membership — picking a
# photo just means adding it to the active heap. Both DB columns may still
# exist on legacy installs but are no longer read or written.
# Duplicate detection.
#
# - file_hash (above): SHA-256 of the raw bytes. Catches byte-identical
# copies but not visually-identical re-encodes / resizes / screenshots.
# - phash: 16-char hex of a 64-bit perceptual hash, computed by the
# thumbs worker from the decoded original frame. Robust to resize and
# re-compression — this is what actually identifies "the same photo
# saved twice with different JPEG quality".
# - duplicate_group_id: shared by every photo in the same duplicate
# cluster. Maintained by app.services.duplicates.regroup_duplicates,
# not on individual writes — recomputed in batches after scans / on
# demand from the Settings panel.
# - is_duplicate: derived boolean (group_id IS NOT NULL). Kept as a real
# column so the existing PhotoThumbnail badge and /library/stats
# duplicates count don't have to change.
is_duplicate = Column(Boolean, default=False)
phash = Column(String(16), index=True)
duplicate_group_id = Column(String, index=True)
# Live photo support
live_photo_video_id = Column(String, ForeignKey('photos.id'))
# Indexes for performance
__table_args__ = (
Index('ix_photos_taken_at', 'taken_at'),
Index('ix_photos_folder_id', 'folder_id'),
Index('ix_photos_is_trashed', 'is_trashed'),
Index('ix_photos_rating', 'rating'),
Index('ix_photos_color_label', 'color_label'),
Index('ix_photos_media_type', 'media_type'),
Index('ix_photos_processing_status', 'processing_status'),
Index('ix_photos_lat_lon', 'latitude', 'longitude'),
Index('ix_photos_needs_review', 'needs_review'),
)

View File

@@ -1,52 +0,0 @@
"""
Sharing models — cross-user access to heaps and folders.
HeapShare grants another user read or read+write access to a heap.
FolderShare does the same for a folder (or source root).
"""
import uuid
from sqlalchemy import (
Column, DateTime, ForeignKey, Index, String, UniqueConstraint, func,
)
from app.database import Base
class HeapShare(Base):
__tablename__ = "heap_shares"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
heap_id = Column(
String, ForeignKey("heaps.id", ondelete="CASCADE"), nullable=False,
)
# Denormalized from heap.user_id for fast "shares I own" lookups.
owner_id = Column(String, ForeignKey("users.id"), nullable=False)
shared_with_id = Column(String, ForeignKey("users.id"), nullable=False)
permission = Column(String, nullable=False, default="read") # 'read' | 'write'
created_at = Column(DateTime, server_default=func.now())
__table_args__ = (
UniqueConstraint("heap_id", "shared_with_id", name="uq_heap_share"),
Index("ix_heap_shares_shared_with", "shared_with_id"),
Index("ix_heap_shares_heap_id", "heap_id"),
)
class FolderShare(Base):
__tablename__ = "folder_shares"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
# Can reference either a Folder.id or a SourceRoot.id.
folder_id = Column(String, nullable=False)
folder_type = Column(String, nullable=False, default="folder") # 'folder' | 'source_root'
owner_id = Column(String, ForeignKey("users.id"), nullable=False)
shared_with_id = Column(String, ForeignKey("users.id"), nullable=False)
permission = Column(String, nullable=False, default="read") # 'read' | 'write'
created_at = Column(DateTime, server_default=func.now())
__table_args__ = (
UniqueConstraint("folder_id", "shared_with_id", name="uq_folder_share"),
Index("ix_folder_shares_shared_with", "shared_with_id"),
Index("ix_folder_shares_folder_id", "folder_id"),
)

View File

@@ -1,57 +0,0 @@
"""
Tag model definitions.
Tags are unified across user-created tags, ML-detected objects, scene
labels, and face clusters via the `kind` column. The `photo_tags`
association carries per-photo ML metadata (confidence, bounding box,
source model).
"""
from sqlalchemy import Column, String, Float, ForeignKey, Table, Index, UniqueConstraint # noqa: F401
from sqlalchemy.orm import relationship
from sqlalchemy.dialects.postgresql import JSONB
import uuid
from app.database import Base
# Association table for many-to-many relationship
photo_tags = Table(
'photo_tags',
Base.metadata,
Column('photo_id', String, ForeignKey('photos.id', ondelete='CASCADE'), primary_key=True),
Column('tag_id', String, ForeignKey('tags.id', ondelete='CASCADE'), primary_key=True),
# ML metadata — null for user-applied tags
Column('confidence', Float, nullable=True),
Column('bbox', JSONB, nullable=True), # [x1, y1, x2, y2] normalized 0-1
Column('source', String, nullable=True), # e.g. "vision:clip_classifier"
Index('ix_photo_tags_photo_id', 'photo_id'),
Index('ix_photo_tags_tag_id', 'tag_id'),
)
class Tag(Base):
__tablename__ = 'tags'
__table_args__ = (
UniqueConstraint('name', 'kind', 'user_id', name='uq_tags_name_kind_user'),
)
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False, index=True)
color = Column(String) # Hex color code for UI display
# Owner
user_id = Column(String, ForeignKey('users.id'), nullable=True, index=True)
# Tag classification
kind = Column(String, nullable=False, default='user', index=True)
# kind values: 'user' | 'content_type'
# Which model produced this tag (null for user-created)
source = Column(String, nullable=True)
# e.g. "vision:clip_classifier", null
# Relationships
photos = relationship("Photo", secondary=photo_tags, backref="tags")
class PhotoTag:
"""Helper class for photo-tag associations (not a table model)"""
pass

View File

@@ -1,23 +0,0 @@
"""
User model definition
"""
from sqlalchemy import Column, String, Boolean, DateTime
from sqlalchemy.sql import func
import uuid
from app.database import Base
class User(Base):
__tablename__ = 'users'
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
username = Column(String(50), unique=True, nullable=False, index=True)
email = Column(String, unique=True, nullable=True)
hashed_password = Column(String, nullable=False)
role = Column(String, nullable=False, default='user') # 'admin' | 'user'
is_active = Column(Boolean, default=True)
created_at = Column(DateTime, server_default=func.now())
# Absolute path to this user's photo directory (e.g., "/photos/daniel")
media_path = Column(String, nullable=False)

View File

@@ -1,369 +0,0 @@
"""
Admin router — user management and app configuration.
All endpoints require admin role.
"""
import os
import logging
from typing import Optional, List
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel
from sqlalchemy import select, func as sa_func
from sqlalchemy.ext.asyncio import AsyncSession
from app.auth import hash_password
from app.database import get_db
from app.dependencies import require_admin
from app.models.user import User
from app.models.photos import Photo
from app.models.folders import SourceRoot
from app.config import settings
from app.services.feature_flags import (
ALL_FLAGS,
snapshot as flags_snapshot,
set_flag,
reset_flag,
is_enabled,
FLAG_VISION_ENABLED,
)
logger = logging.getLogger(__name__)
router = APIRouter()
# ---------------------------------------------------------------------------
# Schemas
# ---------------------------------------------------------------------------
class CreateUserRequest(BaseModel):
username: str
password: str
role: str = "user" # 'admin' | 'user'
class UpdateUserRequest(BaseModel):
role: Optional[str] = None
is_active: Optional[bool] = None
new_password: Optional[str] = None
class UserDetailResponse(BaseModel):
id: str
username: str
email: Optional[str]
role: str
is_active: bool
media_path: str
created_at: Optional[str]
photo_count: int = 0
class UserListResponse(BaseModel):
users: List[UserDetailResponse]
total: int
# ---------------------------------------------------------------------------
# User CRUD
# ---------------------------------------------------------------------------
@router.get("/users", response_model=UserListResponse)
async def list_users(
admin: User = Depends(require_admin),
db: AsyncSession = Depends(get_db),
):
"""List all users with their photo counts."""
result = await db.execute(select(User).order_by(User.created_at))
users = result.scalars().all()
user_list = []
for u in users:
count_result = await db.execute(
select(sa_func.count(Photo.id)).where(Photo.user_id == u.id)
)
photo_count = count_result.scalar() or 0
user_list.append(UserDetailResponse(
id=u.id,
username=u.username,
email=u.email,
role=u.role,
is_active=u.is_active,
media_path=u.media_path,
created_at=u.created_at.isoformat() if u.created_at else None,
photo_count=photo_count,
))
return UserListResponse(users=user_list, total=len(user_list))
@router.post("/users", status_code=201, response_model=UserDetailResponse)
async def create_user(
body: CreateUserRequest,
admin: User = Depends(require_admin),
db: AsyncSession = Depends(get_db),
):
"""Create a new user. Creates their media directory and source root."""
if body.role not in ("admin", "user"):
raise HTTPException(status_code=400, detail="Role must be 'admin' or 'user'")
if len(body.username.strip()) < 2:
raise HTTPException(status_code=400, detail="Username must be at least 2 characters")
if len(body.password) < 6:
raise HTTPException(status_code=400, detail="Password must be at least 6 characters")
# Check for duplicate username
existing = await db.execute(
select(User).where(User.username == body.username.strip())
)
if existing.scalar_one_or_none() is not None:
raise HTTPException(status_code=409, detail="Username already taken")
media_path = os.path.join(settings.photo_dirs, body.username.strip())
os.makedirs(media_path, exist_ok=True)
user = User(
username=body.username.strip(),
hashed_password=hash_password(body.password),
role=body.role,
media_path=media_path,
)
db.add(user)
await db.flush() # get user.id before creating source root
source_root = SourceRoot(
name=f"{user.username}'s Library",
path=media_path,
user_id=user.id,
)
db.add(source_root)
await db.commit()
logger.info(f"Admin '{admin.username}' created user '{user.username}' (role={user.role})")
return UserDetailResponse(
id=user.id,
username=user.username,
email=user.email,
role=user.role,
is_active=user.is_active,
media_path=user.media_path,
created_at=user.created_at.isoformat() if user.created_at else None,
photo_count=0,
)
@router.get("/users/{user_id}", response_model=UserDetailResponse)
async def get_user(
user_id: str,
admin: User = Depends(require_admin),
db: AsyncSession = Depends(get_db),
):
"""Get a single user's details."""
result = await db.execute(select(User).where(User.id == user_id))
user = result.scalar_one_or_none()
if user is None:
raise HTTPException(status_code=404, detail="User not found")
count_result = await db.execute(
select(sa_func.count(Photo.id)).where(Photo.user_id == user.id)
)
photo_count = count_result.scalar() or 0
return UserDetailResponse(
id=user.id,
username=user.username,
email=user.email,
role=user.role,
is_active=user.is_active,
media_path=user.media_path,
created_at=user.created_at.isoformat() if user.created_at else None,
photo_count=photo_count,
)
@router.patch("/users/{user_id}", response_model=UserDetailResponse)
async def update_user(
user_id: str,
body: UpdateUserRequest,
admin: User = Depends(require_admin),
db: AsyncSession = Depends(get_db),
):
"""Update a user's role, active status, or password."""
result = await db.execute(select(User).where(User.id == user_id))
user = result.scalar_one_or_none()
if user is None:
raise HTTPException(status_code=404, detail="User not found")
if body.role is not None:
if body.role not in ("admin", "user"):
raise HTTPException(status_code=400, detail="Role must be 'admin' or 'user'")
# Prevent demoting the last admin
if user.role == "admin" and body.role == "user":
admin_count = (await db.execute(
select(sa_func.count(User.id)).where(User.role == "admin", User.is_active == True)
)).scalar()
if admin_count <= 1:
raise HTTPException(status_code=400, detail="Cannot demote the last admin")
user.role = body.role
if body.is_active is not None:
# Prevent deactivating the last admin
if user.role == "admin" and not body.is_active:
admin_count = (await db.execute(
select(sa_func.count(User.id)).where(User.role == "admin", User.is_active == True)
)).scalar()
if admin_count <= 1:
raise HTTPException(status_code=400, detail="Cannot deactivate the last admin")
user.is_active = body.is_active
if body.new_password is not None:
if len(body.new_password) < 6:
raise HTTPException(status_code=400, detail="Password must be at least 6 characters")
user.hashed_password = hash_password(body.new_password)
await db.commit()
count_result = await db.execute(
select(sa_func.count(Photo.id)).where(Photo.user_id == user.id)
)
photo_count = count_result.scalar() or 0
return UserDetailResponse(
id=user.id,
username=user.username,
email=user.email,
role=user.role,
is_active=user.is_active,
media_path=user.media_path,
created_at=user.created_at.isoformat() if user.created_at else None,
photo_count=photo_count,
)
@router.delete("/users/{user_id}")
async def delete_user(
user_id: str,
admin: User = Depends(require_admin),
db: AsyncSession = Depends(get_db),
):
"""Soft-delete a user by deactivating them. Media is preserved."""
result = await db.execute(select(User).where(User.id == user_id))
user = result.scalar_one_or_none()
if user is None:
raise HTTPException(status_code=404, detail="User not found")
if user.id == admin.id:
raise HTTPException(status_code=400, detail="Cannot delete yourself")
# Prevent deleting the last admin
if user.role == "admin":
admin_count = (await db.execute(
select(sa_func.count(User.id)).where(User.role == "admin", User.is_active == True)
)).scalar()
if admin_count <= 1:
raise HTTPException(status_code=400, detail="Cannot delete the last admin")
user.is_active = False
await db.commit()
logger.info(f"Admin '{admin.username}' deactivated user '{user.username}'")
return {"status": "ok", "detail": f"User '{user.username}' deactivated"}
# ---------------------------------------------------------------------------
# AI / vision feature flags + manual triggers
# ---------------------------------------------------------------------------
class FeatureFlagUpdate(BaseModel):
"""PATCH body for toggling a feature flag.
``value`` sets an explicit override (true/false); omitting it clears
the override and reverts the flag to its YAML default.
"""
value: Optional[bool] = None
@router.get("/feature-flags")
async def get_feature_flags(admin: User = Depends(require_admin)):
"""Return every tunable feature flag with its current effective
value, YAML default, and whether an admin override is in effect."""
return {"flags": flags_snapshot()}
@router.patch("/feature-flags/{flag_name}")
async def update_feature_flag(
flag_name: str,
body: FeatureFlagUpdate,
admin: User = Depends(require_admin),
):
"""Set or clear an override for one flag. With ``value`` set, the
flag is pinned to that boolean; without it, the override is deleted
and the YAML default takes over again.
New value is observed by vision tasks on their next invocation —
there's no worker restart required.
"""
if flag_name not in ALL_FLAGS:
raise HTTPException(status_code=404, detail=f"Unknown flag: {flag_name}")
try:
if body.value is None:
reset_flag(flag_name)
action = "cleared override"
else:
set_flag(flag_name, bool(body.value))
action = f"set to {body.value}"
except RuntimeError as e:
# Redis unreachable — surface as 503 so the UI doesn't think it
# succeeded silently.
raise HTTPException(status_code=503, detail=str(e))
logger.info(f"Admin '{admin.username}' {action} for flag '{flag_name}'")
return {"flags": flags_snapshot()}
class BackfillVisionBody(BaseModel):
"""POST body for triggering a classifier backfill. ``limit`` caps how
many photos are queued."""
limit: Optional[int] = None
@router.post("/ai/backfill")
async def trigger_ai_backfill(
body: BackfillVisionBody,
admin: User = Depends(require_admin),
):
"""Queue a classifier backfill pass."""
if not is_enabled(FLAG_VISION_ENABLED):
raise HTTPException(
status_code=400,
detail="Vision is currently disabled; enable it before running a backfill.",
)
if body.limit is not None and body.limit <= 0:
raise HTTPException(status_code=400, detail="limit must be positive")
from app.tasks.vision import backfill_vision
result = backfill_vision.apply_async(kwargs={'limit': body.limit})
logger.info(
f"Admin '{admin.username}' queued vision backfill "
f"(limit={body.limit}, celery_id={result.id})"
)
return {
"status": "queued",
"task_id": result.id,
"limit": body.limit,
}
@router.post("/ai/rescan")
async def trigger_full_rescan(admin: User = Depends(require_admin)):
"""Dispatch the same scan_all_source_roots job the backend runs at
startup. Picks up any new files on disk and, through the
post-scan hook, queues a vision backfill for whatever still lacks
embeddings / OCR / etc.
"""
from app.tasks.scan import scan_all_source_roots
result = scan_all_source_roots.apply_async()
logger.info(
f"Admin '{admin.username}' queued full rescan (celery_id={result.id})"
)
return {"status": "queued", "task_id": result.id}

View File

@@ -1,202 +0,0 @@
"""
Authentication router — login, token refresh, profile, first-run setup.
"""
import os
import logging
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel
from sqlalchemy import select, func as sa_func
from sqlalchemy.ext.asyncio import AsyncSession
from app.auth import hash_password, verify_password, create_access_token, create_refresh_token, decode_token
from app.database import get_db
from app.dependencies import get_current_user
from app.models.user import User
from app.models.folders import SourceRoot
from app.config import settings
logger = logging.getLogger(__name__)
router = APIRouter()
# ---------------------------------------------------------------------------
# Request / response schemas
# ---------------------------------------------------------------------------
class LoginRequest(BaseModel):
username: str
password: str
class TokenResponse(BaseModel):
access_token: str
refresh_token: str
token_type: str = "bearer"
class RefreshRequest(BaseModel):
refresh_token: str
class UserResponse(BaseModel):
id: str
username: str
email: Optional[str]
role: str
is_active: bool
created_at: Optional[str]
class SetupRequest(BaseModel):
username: str
password: str
class ChangePasswordRequest(BaseModel):
current_password: str
new_password: str
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.post("/login", response_model=TokenResponse)
async def login(body: LoginRequest, db: AsyncSession = Depends(get_db)):
"""Authenticate with username + password, receive JWT tokens."""
result = await db.execute(
select(User).where(User.username == body.username)
)
user = result.scalar_one_or_none()
if user is None or not verify_password(body.password, user.hashed_password):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid username or password",
)
if not user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Account is deactivated",
)
return TokenResponse(
access_token=create_access_token(user.id, user.role),
refresh_token=create_refresh_token(user.id),
)
@router.post("/refresh", response_model=TokenResponse)
async def refresh_token(body: RefreshRequest, db: AsyncSession = Depends(get_db)):
"""Exchange a valid refresh token for a new access + refresh pair."""
try:
payload = decode_token(body.refresh_token)
if payload.get("type") != "refresh":
raise ValueError("not a refresh token")
user_id = payload["sub"]
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired refresh token",
)
result = await db.execute(select(User).where(User.id == user_id))
user = result.scalar_one_or_none()
if user is None or not user.is_active:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="User not found or deactivated",
)
return TokenResponse(
access_token=create_access_token(user.id, user.role),
refresh_token=create_refresh_token(user.id),
)
@router.get("/me", response_model=UserResponse)
async def get_me(current_user: User = Depends(get_current_user)):
"""Return the authenticated user's profile."""
return UserResponse(
id=current_user.id,
username=current_user.username,
email=current_user.email,
role=current_user.role,
is_active=current_user.is_active,
created_at=current_user.created_at.isoformat() if current_user.created_at else None,
)
@router.post("/change-password")
async def change_password(
body: ChangePasswordRequest,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Change the authenticated user's password."""
if not verify_password(body.current_password, current_user.hashed_password):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Current password is incorrect",
)
current_user.hashed_password = hash_password(body.new_password)
await db.commit()
return {"status": "ok"}
@router.post("/setup", response_model=TokenResponse, status_code=201)
async def setup(body: SetupRequest, db: AsyncSession = Depends(get_db)):
"""First-run only: create the initial admin account.
Returns 409 if any user already exists. This endpoint is
unauthenticated by design — it can only run once.
"""
count = (await db.execute(select(sa_func.count(User.id)))).scalar()
if count > 0:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Setup already completed — users exist",
)
if len(body.username.strip()) < 2:
raise HTTPException(status_code=400, detail="Username must be at least 2 characters")
if len(body.password) < 6:
raise HTTPException(status_code=400, detail="Password must be at least 6 characters")
# Every user — including the initial admin — gets their own subfolder
# under the photo mount root. Nobody owns the root directory itself.
media_path = os.path.join(settings.photo_dirs, body.username.strip())
os.makedirs(media_path, exist_ok=True)
user = User(
username=body.username.strip(),
hashed_password=hash_password(body.password),
role="admin",
media_path=media_path,
)
db.add(user)
await db.flush() # get user.id before creating source root
source_root = SourceRoot(
name=f"{user.username}'s Library",
path=media_path,
user_id=user.id,
)
db.add(source_root)
await db.commit()
logger.info(f"Initial admin account created: {user.username}")
return TokenResponse(
access_token=create_access_token(user.id, user.role),
refresh_token=create_refresh_token(user.id),
)
@router.get("/status")
async def auth_status(db: AsyncSession = Depends(get_db)):
"""Public endpoint: returns whether setup has been completed.
The frontend calls this to decide whether to show the setup page
or the login page.
"""
count = (await db.execute(select(sa_func.count(User.id)))).scalar()
return {"setup_completed": count > 0}

View File

@@ -1,97 +0,0 @@
"""
Discard API router
"""
import os
import logging
from fastapi import APIRouter, Depends, HTTPException, Body
from sqlalchemy import select, and_
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Photo
from app.models.user import User
from app.dependencies import get_current_user
logger = logging.getLogger(__name__)
router = APIRouter()
@router.get("")
async def list_discarded(db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""List discarded photos"""
result = await db.execute(
select(Photo).where(Photo.is_discarded == True, Photo.user_id == current_user.id)
)
photos = result.scalars().all()
return photos
@router.post("/restore")
async def restore_photos(photo_ids: list[str], db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Restore photos from the discard pile"""
result = await db.execute(
select(Photo).where(and_(Photo.id.in_(photo_ids), Photo.is_discarded == True, Photo.user_id == current_user.id))
)
photos = result.scalars().all()
for photo in photos:
photo.is_discarded = False
photo.discarded_at = None
await db.commit()
return {"status": "success", "restored": len(photos)}
@router.delete("/empty")
async def empty_discard(db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Permanently delete all discarded photos and unlink their files from
disk. Failures on individual files are logged but don't abort the batch.
"""
result = await db.execute(
select(Photo).where(Photo.is_discarded == True, Photo.user_id == current_user.id)
)
photos = result.scalars().all()
return await _permanently_delete(db, photos)
@router.delete("")
async def delete_discarded(
photo_ids: list[str] = Body(..., embed=True),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Permanently delete a specific subset of discarded photos. The photos
must already be in the discard pile — non-discarded ids are skipped so
this can never bypass the soft-delete safety net.
"""
if not photo_ids:
return {"status": "success", "deleted": 0, "file_errors": 0}
result = await db.execute(
select(Photo).where(
and_(Photo.id.in_(photo_ids), Photo.is_discarded == True, Photo.user_id == current_user.id)
)
)
photos = result.scalars().all()
return await _permanently_delete(db, photos)
async def _permanently_delete(db: AsyncSession, photos: list[Photo]) -> dict:
"""Shared helper: unlink files for the given photos and delete their
rows. Per-file errors are counted but don't abort the batch.
"""
deleted = 0
file_errors = 0
for photo in photos:
try:
if photo.filepath and os.path.exists(photo.filepath):
os.unlink(photo.filepath)
except OSError as e:
file_errors += 1
logger.error(f"Failed to unlink {photo.filepath}: {e}")
await db.delete(photo)
deleted += 1
await db.commit()
return {
"status": "success",
"deleted": deleted,
"file_errors": file_errors,
}

View File

@@ -1,247 +0,0 @@
"""
Download router — streams a .zip of every photo in a folder (recursively)
or a heap back to the browser.
Auth: both endpoints accept the regular Authorization header *or* a
``?token=JWT`` query string, mirroring the media endpoints. That lets the
frontend trigger a download with a plain ``<a href>`` (which can't set a
header), keeping the client side a one-liner.
Implementation: we build the zip into a ``NamedTemporaryFile`` and then
stream its bytes back, deleting the temp file on the way out. Stored
(uncompressed) mode because photos and videos are already compressed —
deflating them again just burns CPU for a fraction of a percent. For
very large libraries the temp-file route is mildly wasteful vs. a true
streaming zip (zipstream-ng etc), but it avoids a new dependency and
handles arbitrary folder sizes without blowing out RAM.
"""
import logging
import os
import re
import tempfile
import zipfile
from typing import List
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.dependencies import get_current_user_media
from app.models import Folder, Heap, Photo, SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
logger = logging.getLogger(__name__)
router = APIRouter()
def _safe_filename(name: str) -> str:
"""Strip characters that Content-Disposition or Windows filesystems
would choke on. Keeps the download's filename readable without
needing any escaping on the client side."""
cleaned = re.sub(r'[\\/:*?"<>|\r\n\t]', '_', name).strip().strip('.')
return cleaned or 'download'
async def _collect_folder_photos(
folder_id: str,
user: User,
db: AsyncSession,
) -> tuple[str, str, List[Photo]]:
"""Resolve a folder or source-root id → (base_path, display_name,
photos). ``base_path`` is the prefix we strip off each photo's
filepath when naming zip entries, so the archive mirrors the user's
on-disk structure under that folder.
"""
folder = (await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)).scalar_one_or_none()
base_path: str
display_name: str
if folder is not None:
base_path = os.path.normpath(folder.path)
display_name = folder.name or os.path.basename(base_path)
else:
sr = (await db.execute(
select(SourceRoot).where(
SourceRoot.id == folder_id,
SourceRoot.user_id == user.id,
)
)).scalar_one_or_none()
if sr is None:
raise HTTPException(status_code=404, detail="Folder not found")
base_path = os.path.normpath(sr.path)
display_name = sr.name or os.path.basename(base_path)
# Every photo whose filepath is at or below the base path — matches
# the same prefix convention folders.py uses for recursive deletes.
descendant_prefix = base_path.rstrip(os.sep) + os.sep
result = await db.execute(
select(Photo).where(
Photo.user_id == user.id,
Photo.is_discarded == False, # noqa: E712
(Photo.filepath == base_path) | (Photo.filepath.like(descendant_prefix + '%')),
)
)
photos = list(result.scalars().all())
return base_path, display_name, photos
def _build_zip(
photos: List[Photo],
arcname_fn,
) -> tempfile.NamedTemporaryFile:
"""Write ``photos`` into a fresh ZIP_STORED temp file.
``arcname_fn(photo, used_names)`` returns the entry name to use for
the given photo; the caller supplies it because folder downloads
want path-preserving names while heap downloads flatten to bare
filenames (with a collision suffix).
"""
tmp = tempfile.NamedTemporaryFile(delete=False, suffix='.zip')
try:
used: set[str] = set()
with zipfile.ZipFile(tmp, 'w', zipfile.ZIP_STORED, allowZip64=True) as zf:
for p in photos:
if not p.filepath or not os.path.exists(p.filepath):
# Silent skip: the scanner may have indexed files
# that have since been moved / unlinked by a shell.
continue
name = arcname_fn(p, used)
used.add(name)
try:
zf.write(p.filepath, name)
except OSError as e:
logger.warning(f"Skipping {p.filepath} in zip: {e}")
tmp.close()
return tmp
except Exception:
tmp.close()
try:
os.unlink(tmp.name)
except OSError:
pass
raise
def _stream_and_cleanup(path: str):
"""Yield the temp zip in 1 MiB chunks and unlink it when the
iterator is exhausted (or GC'd, if the client disconnects early)."""
try:
with open(path, 'rb') as f:
while True:
chunk = f.read(1024 * 1024)
if not chunk:
break
yield chunk
finally:
try:
os.unlink(path)
except OSError as e:
logger.debug(f"Temp zip cleanup failed for {path}: {e}")
def _dedupe(name: str, used: set[str]) -> str:
"""Return ``name`` (or ``name (2)``, ``name (3)`` ...) such that the
result doesn't collide with anything in ``used``. Needed for heap
downloads where two members can have identical filenames from
different folders."""
if name not in used:
return name
stem, ext = os.path.splitext(name)
n = 2
while True:
cand = f"{stem} ({n}){ext}"
if cand not in used:
return cand
n += 1
@router.get("/folders/{folder_id}")
async def download_folder(
folder_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user_media),
):
"""Zip every (non-discarded) photo under a folder/source-root and
stream it back. Entries preserve the folder structure relative to
the downloaded root so the resulting archive is a faithful snapshot.
"""
base_path, display_name, photos = await _collect_folder_photos(
folder_id, current_user, db
)
if not photos:
raise HTTPException(status_code=404, detail="No photos to download")
def arcname(p: Photo, _used: set[str]) -> str:
# Relative path from the download root, falling back to the
# bare filename if the photo somehow lives outside base_path.
abs_path = os.path.normpath(p.filepath)
if abs_path.startswith(base_path + os.sep):
rel = abs_path[len(base_path) + 1:]
elif abs_path == base_path:
rel = os.path.basename(abs_path)
else:
rel = p.filename or os.path.basename(abs_path)
# Nest everything under display_name so users see one top-level
# folder inside the zip rather than loose files.
return os.path.join(_safe_filename(display_name), rel)
tmp = _build_zip(photos, arcname)
filename = _safe_filename(display_name) + '.zip'
return StreamingResponse(
_stream_and_cleanup(tmp.name),
media_type='application/zip',
headers={
'Content-Disposition': f'attachment; filename="{filename}"',
'Content-Length': str(os.path.getsize(tmp.name)),
},
)
@router.get("/heaps/{heap_id}")
async def download_heap(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user_media),
):
"""Zip every photo in a heap. Heaps are flat collections, so entries
use the original filename (with a ``(2)`` collision suffix when
two members share a name)."""
heap = (await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == current_user.id)
)).scalar_one_or_none()
if heap is None:
raise HTTPException(status_code=404, detail="Heap not found")
result = await db.execute(
select(Photo)
.join(heap_photos, heap_photos.c.photo_id == Photo.id)
.where(
heap_photos.c.heap_id == heap_id,
Photo.is_discarded == False, # noqa: E712
)
)
photos = list(result.scalars().all())
if not photos:
raise HTTPException(status_code=404, detail="Heap is empty")
def arcname(p: Photo, used: set[str]) -> str:
bare = p.filename or os.path.basename(p.filepath or 'photo')
entry = os.path.join(_safe_filename(heap.name), _dedupe(bare, used))
return entry
tmp = _build_zip(photos, arcname)
filename = _safe_filename(heap.name) + '.zip'
return StreamingResponse(
_stream_and_cleanup(tmp.name),
media_type='application/zip',
headers={
'Content-Disposition': f'attachment; filename="{filename}"',
'Content-Length': str(os.path.getsize(tmp.name)),
},
)

View File

@@ -1,23 +0,0 @@
"""
Public feature-flag read API — lets the authenticated frontend know
which AI-powered sections to render.
This is NOT the admin mutation endpoint (that's in ``admin.py`` and
gated by ``require_admin``). Here we only expose the effective boolean
state so the UI can hide things like the People view, Tags view, or
text-search affordances when the underlying pipeline stage is off.
"""
from fastapi import APIRouter, Depends
from app.dependencies import get_current_user
from app.models.user import User
from app.services.feature_flags import ALL_FLAGS, is_enabled
router = APIRouter()
@router.get("")
async def get_enabled_features(_: User = Depends(get_current_user)):
"""Return ``{flag_name: bool}`` for every known flag, reflecting
the currently effective value (admin override or YAML default)."""
return {name: is_enabled(name) for name in ALL_FLAGS}

View File

@@ -1,553 +0,0 @@
"""
Folders API router. Source roots themselves are config-driven (PHOTO_DIRS
in .env → backend bootstrap on startup) — adding or removing one is a
docker-compose change. Sub-folders inside a source root can be created,
renamed, and deleted from the UI; those changes are mirrored to disk.
"""
import logging
import os
import shutil
from typing import Literal, Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import select, func, update as sql_update, delete as sql_delete
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Folder, SourceRoot, Photo
from app.models.user import User
from app.dependencies import get_current_user, get_user_folder
logger = logging.getLogger(__name__)
router = APIRouter()
class FolderRename(BaseModel):
name: str
class FolderCreate(BaseModel):
name: str
parent_id: str # Folder.id (NOT a SourceRoot id)
class FolderHide(BaseModel):
hidden: bool
def _validate_folder_name(name: str) -> str:
"""Trim + sanity-check a folder name. Rejects names that contain a
path separator or that resolve to a parent traversal — those would
let the user escape the parent directory through this endpoint.
"""
name = (name or '').strip()
if not name:
raise HTTPException(status_code=400, detail="Name cannot be empty")
if '/' in name or '\\' in name or name in ('.', '..'):
raise HTTPException(status_code=400, detail="Invalid folder name")
return name
@router.get("")
async def get_folders(db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Get all source folders"""
# Get source roots instead of regular folders
result = await db.execute(select(SourceRoot).where(SourceRoot.is_active == True, SourceRoot.user_id == current_user.id))
source_roots = result.scalars().all()
folders_list = []
for root in source_roots:
# Get photo count for this source root
folder_result = await db.execute(
select(Folder).where(Folder.source_root_id == root.id)
)
folders = folder_result.scalars().all()
photo_count = sum(f.photo_count for f in folders)
folders_list.append({
"id": root.id,
"name": root.name or os.path.basename(root.path),
"path": root.path,
"photo_count": photo_count
})
return {"folders": folders_list}
@router.get("/tree")
async def get_folder_tree(db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Recursive folder tree, one root per active SourceRoot. The tree
starts at the Folder row matching the SourceRoot.path (the scanner
creates one for every walked directory), with the SourceRoot's
display name overlaid so the top-level entry reads as "Library"
instead of "/photos".
Returns a list of root nodes; each node has:
{ id, name, path, photo_count, children: [...] }
photo_count is **recursive** — every node reports the total non-
discarded photos in its own subtree, so the badge matches what the
user sees when they click the row (which also filters recursively).
The stored Folder.photo_count column is intentionally NOT trusted;
the scanner's bookkeeping for that field has historically been
wrong (it leaks the global total into whichever folder os.walk
visited last). We compute counts here from the photos table.
Sub-folders that physically belong to the same source root but
weren't created on disk (e.g. the / row the scanner sometimes
creates as a parent walk) are skipped via path-prefix filtering.
"""
sr_result = await db.execute(
select(SourceRoot).where(SourceRoot.is_active == True, SourceRoot.user_id == current_user.id) # noqa: E712
)
source_roots = sr_result.scalars().all()
out = []
for sr in source_roots:
# Folders physically inside this source root, by path prefix.
prefix = os.path.normpath(sr.path).rstrip(os.sep)
f_result = await db.execute(
select(Folder).where(
Folder.source_root_id == sr.id,
# Either the folder IS the source root, or it sits beneath it.
(Folder.path == prefix) | (Folder.path.like(prefix + os.sep + '%'))
)
)
folders = f_result.scalars().all()
if not folders:
continue
# Direct (non-recursive) photo counts per folder, computed from
# the photos table. Excludes discarded AND hidden photos so the
# sidebar badge matches the "All Photos"-style cross-cutting
# views. Users can still click into a hidden folder and see its
# contents; the badge count simply won't reflect those photos.
folder_ids = [f.id for f in folders]
direct_counts: dict[str, int] = {}
if folder_ids:
count_result = await db.execute(
select(Photo.folder_id, func.count(Photo.id))
.where(
Photo.is_discarded == False, # noqa: E712
Photo.is_hidden == False, # noqa: E712
Photo.folder_id.in_(folder_ids),
)
.group_by(Photo.folder_id)
)
direct_counts = {row[0]: int(row[1]) for row in count_result.all()}
# Build a path → node map so we can attach children regardless of
# parent_id consistency. We populate photo_count with the direct
# count first, then accumulate descendants in a post-order pass.
# `is_hidden` on each node carries the user-set folder flag (NOT
# the effective ancestry flag) so the frontend can render the
# hidden icon on the exact folder the user toggled.
nodes = {
f.path: {
"id": f.id,
"name": f.name or os.path.basename(f.path),
"path": f.path,
"photo_count": direct_counts.get(f.id, 0),
"is_hidden": bool(f.is_hidden),
"children": [],
}
for f in folders
}
root_node = None
for f in folders:
node = nodes[f.path]
if f.path == prefix:
root_node = node
# Override the display name with the source root's label.
node["name"] = sr.name or node["name"]
continue
parent_path = os.path.normpath(os.path.dirname(f.path))
parent = nodes.get(parent_path)
if parent is not None:
parent["children"].append(node)
# If parent isn't in the set (orphan from a partial scan), drop
# the node — it can't be rendered consistently.
if root_node is not None:
# Sort children alphabetically at every level.
def sort_recursive(n):
n["children"].sort(key=lambda c: c["name"].lower())
for c in n["children"]:
sort_recursive(c)
sort_recursive(root_node)
# Post-order: each node's recursive count is its own direct
# count plus the sum of every descendant's recursive count.
def accumulate(n) -> int:
total = n["photo_count"]
for c in n["children"]:
total += accumulate(c)
n["photo_count"] = total
return total
accumulate(root_node)
out.append(root_node)
return out
@router.patch("/{folder_id}")
async def rename_folder(
folder_id: str,
body: FolderRename,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Rename a folder. Two cases:
- SourceRoot id → just change the display label. The on-disk path
is owned by the docker mount and never moves.
- Folder id → rename the directory on disk AND update every
descendant Folder.path + Photo.filepath that
lived under the old prefix. Refuses to rename
the source-root folder itself (= the row that
matches the SourceRoot.path) because that would
require renaming the docker mount.
"""
name = _validate_folder_name(body.name)
# Try SourceRoot first (display-only rename).
sr_result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id)
)
source_root = sr_result.scalar_one_or_none()
if source_root:
source_root.name = name
await db.commit()
return {
"id": source_root.id,
"name": source_root.name,
"path": source_root.path,
}
# Otherwise it's a Folder row.
folder = await get_user_folder(folder_id, current_user, db)
# Refuse to rename the bare source root mount through here.
sr_check = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder.source_root_id, SourceRoot.user_id == current_user.id)
)
sr = sr_check.scalar_one_or_none()
if sr and os.path.normpath(folder.path) == os.path.normpath(sr.path):
raise HTTPException(
status_code=400,
detail="Cannot rename the source root mount; rename the docker mount instead.",
)
old_path = os.path.normpath(folder.path).rstrip(os.sep)
parent_dir = os.path.dirname(old_path)
new_path = os.path.join(parent_dir, name)
if os.path.exists(new_path):
raise HTTPException(
status_code=400,
detail=f"A folder named '{name}' already exists here",
)
try:
shutil.move(old_path, new_path)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Rename failed: {e}")
# Update folder paths: this row + every descendant. SQLite REPLACE
# rewrites the prefix; we use the trailing separator on the LIKE
# pattern so a folder named "foo" doesn't accidentally match "foobar".
await db.execute(
sql_update(Folder)
.where(Folder.id == folder.id)
.values(path=new_path, name=name)
)
descendant_prefix = old_path + os.sep
descendants = await db.execute(
select(Folder).where(Folder.path.like(descendant_prefix + '%'))
)
for d in descendants.scalars().all():
d.path = new_path + d.path[len(old_path):]
# Update every photo whose filepath lives under the old prefix.
photos_result = await db.execute(
select(Photo).where(Photo.filepath.like(descendant_prefix + '%'))
)
for p in photos_result.scalars().all():
p.filepath = new_path + p.filepath[len(old_path):]
# Photos directly inside this folder (not in a subdir) won't match
# the descendant_prefix LIKE if their old path was old_path + '/file'
# — actually they DO match, since 'oldpath/file' starts with
# 'oldpath/'. So the loop above already covers them.
await db.commit()
return {
"id": folder.id,
"name": folder.name,
"path": folder.path,
}
@router.post("", status_code=201)
async def create_folder(body: FolderCreate, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Create a new sub-folder under an existing Folder. Mirrors the
create to disk so the next scan sees it. Body: { name, parent_id }.
parent_id MUST be an existing Folder row id (any descendant of a
source root); creating a brand-new top-level mount is a docker
operation, not a UI one.
"""
name = _validate_folder_name(body.name)
parent = await get_user_folder(body.parent_id, current_user, db)
new_path = os.path.join(parent.path, name)
if os.path.exists(new_path):
raise HTTPException(
status_code=400,
detail=f"A folder named '{name}' already exists here",
)
try:
os.makedirs(new_path, exist_ok=False)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Create failed: {e}")
new_folder = Folder(
name=name,
path=new_path,
source_root_id=parent.source_root_id,
user_id=current_user.id,
photo_count=0,
)
db.add(new_folder)
await db.commit()
await db.refresh(new_folder)
return {
"id": new_folder.id,
"name": new_folder.name,
"path": new_folder.path,
"parent_id": parent.id,
}
@router.delete("/{folder_id}")
async def delete_folder(
folder_id: str,
mode: Literal['discard', 'permanent'] = Query('discard'),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Delete a folder. Behavior depends on mode:
- mode=discard (default): mark every photo whose filepath lives
under this folder as is_discarded=true. The folder row, its
descendant rows, and the on-disk directory are LEFT INTACT —
the user can still recover photos from the discard pile, and
a re-scan won't double-import them.
- mode=permanent: unlink every photo file under this folder,
remove the photo + folder rows from the DB, and rmtree the
on-disk directory. Irreversible.
Refuses to delete the bare source-root mount in either mode (deleting
the docker mount through the UI would be a footgun).
"""
folder = await get_user_folder(folder_id, current_user, db)
sr_check = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder.source_root_id, SourceRoot.user_id == current_user.id)
)
sr = sr_check.scalar_one_or_none()
if sr and os.path.normpath(folder.path) == os.path.normpath(sr.path):
raise HTTPException(
status_code=400,
detail="Cannot delete the source root mount through the UI",
)
folder_path = os.path.normpath(folder.path).rstrip(os.sep)
descendant_prefix = folder_path + os.sep
# Collect every photo under this folder OR any descendant. We match
# by filepath prefix instead of folder_id because that catches photos
# in nested subfolders without a recursive folder walk.
photos_result = await db.execute(
select(Photo).where(
(Photo.filepath == folder_path)
| (Photo.filepath.like(descendant_prefix + '%'))
)
)
photos = photos_result.scalars().all()
if mode == 'discard':
from datetime import datetime
now = datetime.utcnow()
for p in photos:
p.is_discarded = True
p.discarded_at = now
await db.commit()
return {
"status": "success",
"mode": "discard",
"discarded": len(photos),
}
# mode == 'permanent'
file_errors = 0
for p in photos:
try:
if p.filepath and os.path.exists(p.filepath):
os.unlink(p.filepath)
except OSError as e:
file_errors += 1
logger.error(f"Failed to unlink {p.filepath}: {e}")
await db.delete(p)
# Delete this folder + every descendant Folder row.
await db.execute(
sql_delete(Folder).where(
(Folder.id == folder.id)
| (Folder.path.like(descendant_prefix + '%'))
)
)
try:
if os.path.isdir(folder_path):
shutil.rmtree(folder_path)
except OSError as e:
logger.error(f"Failed to rmtree {folder_path}: {e}")
# Don't raise — DB rows are already gone, leaving an orphan
# directory is the lesser evil.
await db.commit()
return {
"status": "success",
"mode": "permanent",
"deleted_photos": len(photos),
"file_errors": file_errors,
}
async def _recompute_photo_hidden_flags(db: AsyncSession) -> None:
"""Rematerialize photos.is_hidden from the full folder ancestry.
`photos.is_hidden` is true iff any ancestor folder in the photo's
folder chain (including the folder the photo is directly in) has
`folders.is_hidden = true`. Rather than do a recursive walk in
Python, we lean on Postgres's WITH RECURSIVE to compute each
folder's effective hidden state in a single query, then join on
photos to bulk-update the flag.
Called after any folders.is_hidden toggle AND after moving photos
between folders, since the photo's effective-hidden state can
change even when no folder flag changes. Cheap — one O(folders)
CTE + one O(photos) UPDATE. On a 13k-photo library this runs in
under 50ms.
"""
from sqlalchemy import text as _text
await db.execute(
_text("""
WITH RECURSIVE folder_chain AS (
-- Base: source-root folders (no parent_id). Their own
-- is_hidden is the starting effective value.
SELECT id, is_hidden AS effective_hidden
FROM folders
WHERE parent_id IS NULL
UNION ALL
-- Step: a child folder inherits from its parent. The
-- effective flag is true if the parent's effective flag
-- is true OR the child's own flag is true. Short-circuit
-- would be nice but a plain OR does the job.
SELECT f.id, (f.is_hidden OR fc.effective_hidden) AS effective_hidden
FROM folders f
JOIN folder_chain fc ON f.parent_id = fc.id
)
UPDATE photos p
SET is_hidden = fc.effective_hidden
FROM folder_chain fc
WHERE p.folder_id = fc.id
AND p.is_hidden IS DISTINCT FROM fc.effective_hidden
""")
)
@router.post("/{folder_id}/hide")
async def set_folder_hidden(
folder_id: str,
body: FolderHide,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Toggle the "hide from views" flag on a folder or source root.
A hidden folder's photos are excluded from every cross-cutting view
(All Photos, Map, Tags, People, Search, sidebar counts, duplicates)
but remain fully indexed and visible when the user navigates
directly into the folder. The flag cascades to every descendant
folder via the photos.is_hidden recompute — the child folder's own
`is_hidden` column stays where the user set it, but a photo under a
hidden ancestor will still be marked hidden.
Accepts both Folder ids and SourceRoot ids. For a SourceRoot, we
look up the root Folder row (the one matching source_root.path) and
flip that — source roots themselves don't carry the column because
the whole subtree lives on a single Folder row anyway.
"""
# SourceRoot path — resolve to the Folder row at the mount point.
sr_result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id)
)
source_root = sr_result.scalar_one_or_none()
folder: Optional[Folder]
if source_root:
root_folder_result = await db.execute(
select(Folder).where(
Folder.source_root_id == source_root.id,
Folder.user_id == current_user.id,
Folder.path == os.path.normpath(source_root.path),
)
)
folder = root_folder_result.scalar_one_or_none()
if folder is None:
raise HTTPException(
status_code=404,
detail="Source root has no indexed Folder row yet; scan first.",
)
else:
folder = await get_user_folder(folder_id, current_user, db)
folder.is_hidden = bool(body.hidden)
await db.flush()
# Rematerialize photos.is_hidden across the whole tree. Cheap
# enough (tens of ms on a typical library) that we don't need to
# scope the update to just this folder's subtree — doing it
# globally also fixes any drift introduced by earlier moves.
await _recompute_photo_hidden_flags(db)
await db.commit()
return {
"id": folder.id,
"name": folder.name,
"path": folder.path,
"is_hidden": folder.is_hidden,
}
@router.post("/{folder_id}/scan")
async def scan_folder(folder_id: str, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Trigger manual re-scan of source root folder"""
from app.tasks.celery import celery_app
result = await db.execute(select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id))
source_root = result.scalar_one_or_none()
if not source_root:
raise HTTPException(status_code=404, detail="Source folder not found")
# Queue scan task using the task name defined in the decorator
task = celery_app.send_task('scan_folder', args=[source_root.path, source_root.id])
return {"status": "success", "message": f"Scan queued for {source_root.path}", "task_id": task.id}

View File

@@ -1,448 +0,0 @@
"""
Heaps API router
"""
import os
import shutil
import logging
from typing import Optional, Literal
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import select, func, update, insert, delete
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Heap, Photo, Folder
from app.models.folders import SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
from app.dependencies import get_current_user, get_user_heap, get_user_or_shared_heap
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ───────────────────────────────────────────────────────────────
class HeapCreate(BaseModel):
name: str
class HeapUpdate(BaseModel):
name: Optional[str] = None
is_active: Optional[bool] = None
class HeapPhotosBody(BaseModel):
photo_ids: list[str]
class HeapConvertBody(BaseModel):
target_id: str # folder id OR source root id
mode: Literal['move', 'copy'] = 'move'
delete_heap: bool = False
# Optional subfolder name to create inside the target. If provided, the
# actual destination is target_dir/subfolder_name (created if missing).
# Path separators and dot-segments are rejected.
subfolder_name: Optional[str] = None
# ── Endpoints ─────────────────────────────────────────────────────────────
@router.get("")
async def list_heaps(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all heaps with photo counts."""
# LEFT JOIN heap_photos and group so we can return counts in one query.
count_subq = (
select(
heap_photos.c.heap_id,
func.count(heap_photos.c.photo_id).label("photo_count"),
)
.group_by(heap_photos.c.heap_id)
.subquery()
)
stmt = (
select(Heap, count_subq.c.photo_count)
.outerjoin(count_subq, Heap.id == count_subq.c.heap_id)
.where(Heap.user_id == current_user.id)
.order_by(Heap.created_at.asc())
)
result = await db.execute(stmt)
rows = result.all()
return [
{
"id": h.id,
"name": h.name,
"is_active": bool(h.is_active),
"created_at": h.created_at,
"updated_at": h.updated_at,
"photo_count": int(count or 0),
}
for h, count in rows
]
@router.post("", status_code=201)
async def create_heap(
body: HeapCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Create a new heap."""
name = (body.name or "").strip()
if not name:
raise HTTPException(status_code=400, detail="Heap name is required")
heap = Heap(name=name, user_id=current_user.id)
db.add(heap)
await db.commit()
await db.refresh(heap)
return {
"id": heap.id,
"name": heap.name,
"is_active": bool(heap.is_active),
"created_at": heap.created_at,
"updated_at": heap.updated_at,
"photo_count": 0,
}
@router.patch("/{heap_id}")
async def update_heap(
heap_id: str,
body: HeapUpdate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Rename a heap and/or toggle active state. Setting is_active=true on
one heap deactivates all others (single-active invariant)."""
heap = await get_user_heap(heap_id, current_user, db)
if body.name is not None:
name = body.name.strip()
if not name:
raise HTTPException(status_code=400, detail="Heap name is required")
heap.name = name
if body.is_active is not None:
if body.is_active:
# Clear active flag on all other heaps for this user
await db.execute(
update(Heap)
.where(Heap.user_id == current_user.id)
.values(is_active=False)
)
heap.is_active = True
else:
heap.is_active = False
await db.commit()
await db.refresh(heap)
return {
"id": heap.id,
"name": heap.name,
"is_active": bool(heap.is_active),
"created_at": heap.created_at,
"updated_at": heap.updated_at,
}
@router.post("/{heap_id}/duplicate", status_code=201)
async def duplicate_heap(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Create a new heap with the same membership as an existing one. The
new heap is named "{original} (copy)" and is never the active target —
duplicating shouldn't quietly steal the user's T-key destination.
"""
source = await get_user_heap(heap_id, current_user, db)
new_heap = Heap(name=f"{source.name} (copy)", is_active=False, user_id=current_user.id)
db.add(new_heap)
await db.flush() # populate new_heap.id without committing yet
# Bulk-copy the membership rows. SELECT photo_id FROM heap_photos WHERE
# heap_id = :src — INSERT each into the new heap. Done as a single
# INSERT...SELECT to avoid round-tripping ids through Python.
member_rows = await db.execute(
select(heap_photos.c.photo_id).where(heap_photos.c.heap_id == heap_id)
)
photo_ids = [row[0] for row in member_rows.all()]
if photo_ids:
await db.execute(
insert(heap_photos),
[{"heap_id": new_heap.id, "photo_id": pid} for pid in photo_ids],
)
await db.commit()
await db.refresh(new_heap)
return {
"id": new_heap.id,
"name": new_heap.name,
"is_active": False,
"photo_count": len(photo_ids),
"created_at": new_heap.created_at,
"updated_at": new_heap.updated_at,
}
@router.delete("/{heap_id}", status_code=204)
async def delete_heap(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Delete a heap. Photos themselves are unaffected — only the membership
rows in heap_photos cascade-delete."""
heap = await get_user_heap(heap_id, current_user, db)
await db.delete(heap)
await db.commit()
return None
@router.get("/{heap_id}/photo_ids")
async def get_heap_photo_ids(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Return just the photo ids belonging to a heap. Used by the frontend
to maintain a fast client-side membership lookup for the active heap
(for the basket affordance on thumbnails) without fetching full photo
records."""
await get_user_or_shared_heap(heap_id, current_user, db)
result = await db.execute(
select(heap_photos.c.photo_id).where(heap_photos.c.heap_id == heap_id)
)
return [row[0] for row in result.all()]
@router.post("/{heap_id}/photos")
async def add_photos_to_heap(
heap_id: str,
body: HeapPhotosBody,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Add photos to a heap. Idempotent: re-adding existing members is a
no-op (handled by an INSERT OR IGNORE-style filter on duplicates).
Shared users with write permission can add their own photos."""
_heap, permission = await get_user_or_shared_heap(heap_id, current_user, db)
if permission == "read":
raise HTTPException(status_code=403, detail="Read-only access to this heap")
if not body.photo_ids:
return {"status": "success", "added": 0}
# Find which ids are already members so we don't violate the PK.
existing = await db.execute(
select(heap_photos.c.photo_id).where(
heap_photos.c.heap_id == heap_id,
heap_photos.c.photo_id.in_(body.photo_ids),
)
)
existing_ids = {row[0] for row in existing.all()}
new_ids = [pid for pid in body.photo_ids if pid not in existing_ids]
if new_ids:
await db.execute(
insert(heap_photos),
[{"heap_id": heap_id, "photo_id": pid} for pid in new_ids],
)
await db.commit()
return {"status": "success", "added": len(new_ids), "already_present": len(existing_ids)}
@router.post("/{heap_id}/convert")
async def convert_heap_to_folder(
heap_id: str,
body: HeapConvertBody,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Convert a heap into a folder by moving (or copying) every member
photo into the target directory. Optionally deletes the heap row at
the end.
target_id may be a Folder id or a SourceRoot id (matches the
/photos/move convention so the same dropdown can populate it).
"""
heap = await get_user_heap(heap_id, current_user, db)
# Resolve target_id → (target_dir, target_folder)
sr_check = await db.execute(
select(SourceRoot).where(SourceRoot.id == body.target_id)
)
source_root = sr_check.scalar_one_or_none()
if source_root is not None:
parent_dir = source_root.path
parent_source_root_id = source_root.id
else:
folder_check = await db.execute(
select(Folder).where(Folder.id == body.target_id)
)
parent_folder = folder_check.scalar_one_or_none()
if parent_folder is None:
raise HTTPException(status_code=404, detail="Target folder not found")
parent_dir = parent_folder.path
parent_source_root_id = parent_folder.source_root_id
if not os.path.isdir(parent_dir):
raise HTTPException(
status_code=400,
detail=f"Target parent does not exist: {parent_dir}",
)
# Resolve target_dir, creating an optional subfolder if requested.
if body.subfolder_name is not None:
sub = body.subfolder_name.strip()
if not sub:
raise HTTPException(status_code=400, detail="Subfolder name cannot be empty")
if '/' in sub or '\\' in sub or sub in ('.', '..'):
raise HTTPException(status_code=400, detail="Invalid subfolder name")
target_dir = os.path.join(parent_dir, sub)
if not os.path.exists(target_dir):
try:
os.makedirs(target_dir)
except OSError as e:
raise HTTPException(
status_code=500,
detail=f"Failed to create subfolder: {e}",
)
elif not os.path.isdir(target_dir):
raise HTTPException(
status_code=400,
detail=f"{target_dir} exists but is not a directory",
)
else:
target_dir = parent_dir
# Ensure a Folder row for the target, reusing the scanner helper so
# path normalization + dedupe stay consistent.
from app.tasks.scan import get_or_create_folder
target_folder = await get_or_create_folder(db, target_dir, parent_source_root_id)
# Fetch the heap's photos via the join table.
photo_result = await db.execute(
select(Photo)
.join(heap_photos, Photo.id == heap_photos.c.photo_id)
.where(heap_photos.c.heap_id == heap_id)
)
photos = photo_result.scalars().all()
moved = 0
copied = 0
errors: list[dict] = []
def _unique_target_name(directory: str, filename: str) -> Optional[str]:
if not os.path.exists(os.path.join(directory, filename)):
return filename
stem, ext = os.path.splitext(filename)
for i in range(1, 100):
suffix = '' if i == 1 else f' {i}'
candidate = f"{stem} (copy{suffix}){ext}"
if not os.path.exists(os.path.join(directory, candidate)):
return candidate
return None
for photo in photos:
if not os.path.exists(photo.filepath):
errors.append({"id": photo.id, "error": "source file missing"})
continue
if body.mode == 'move':
if photo.folder_id == target_folder.id:
continue # already there
new_path = os.path.join(target_dir, photo.filename)
if os.path.exists(new_path):
errors.append({"id": photo.id, "error": f"name collision: {photo.filename}"})
continue
try:
shutil.move(photo.filepath, new_path)
except OSError as e:
errors.append({"id": photo.id, "error": str(e)})
continue
photo.filepath = new_path
photo.folder_id = target_folder.id
moved += 1
else: # copy
new_name = _unique_target_name(target_dir, photo.filename)
if new_name is None:
errors.append({"id": photo.id, "error": "too many name collisions"})
continue
new_path = os.path.join(target_dir, new_name)
try:
shutil.copy2(photo.filepath, new_path)
except OSError as e:
errors.append({"id": photo.id, "error": str(e)})
continue
new_photo = Photo(
filepath=new_path,
filename=new_name,
folder_id=target_folder.id,
file_hash=photo.file_hash,
media_type=photo.media_type,
original_format=photo.original_format,
width=photo.width,
height=photo.height,
file_size=photo.file_size,
taken_at=photo.taken_at,
taken_at_source=photo.taken_at_source,
user_title=photo.user_title,
user_notes=photo.user_notes,
rating=photo.rating,
color_label=photo.color_label,
exif_json=photo.exif_json,
is_duplicate=True,
processing_status='pending',
)
db.add(new_photo)
copied += 1
if body.delete_heap:
await db.delete(heap)
await db.commit()
return {
"status": "success",
"mode": body.mode,
"moved": moved,
"copied": copied,
"errors": errors,
"heap_deleted": body.delete_heap,
}
@router.delete("/{heap_id}/photos")
async def remove_photos_from_heap(
heap_id: str,
body: HeapPhotosBody,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Remove photos from a heap. Removing a non-member is a no-op.
Shared users with write permission can remove photos."""
_heap, permission = await get_user_or_shared_heap(heap_id, current_user, db)
if permission == "read":
raise HTTPException(status_code=403, detail="Read-only access to this heap")
if not body.photo_ids:
return {"status": "success", "removed": 0}
res = await db.execute(
delete(heap_photos).where(
heap_photos.c.heap_id == heap_id,
heap_photos.c.photo_id.in_(body.photo_ids),
)
)
await db.commit()
return {"status": "success", "removed": res.rowcount or 0}

View File

@@ -1,833 +0,0 @@
"""
Library API router for stats, scanning, and maintenance.
The /maintenance/* endpoints are surfaced through the frontend Settings
panel. They're intentionally idempotent and operate by re-queueing the
existing Celery tasks rather than doing any heavy lifting in the
request thread.
"""
import logging
import os
import shutil
from typing import List, Optional
from fastapi import APIRouter, Depends, Query
from pydantic import BaseModel, Field
from sqlalchemy import select, func, update, true as sa_true
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Photo
from app.models.folders import SourceRoot
from app.models.user import User
from app.dependencies import get_current_user
logger = logging.getLogger(__name__)
router = APIRouter()
def _owner_filter(user: User, scope: str | None):
"""Return a column expression scoping photos to the current user,
or a pass-through true() when an admin requests global scope."""
if scope == "global" and user.role == "admin":
return sa_true()
return Photo.user_id == user.id
# Media types we accept in the regenerate-thumbnails request body. Mirrors
# the values produced by `app.tasks.scan.get_media_type`.
_VALID_MEDIA_TYPES = {'photo', 'raw', 'heic', 'video'}
@router.get("/stats")
async def get_library_stats(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
scope: str | None = Query(None),
):
"""Get library statistics. Pass ?scope=global (admin only) for
cross-user totals (used by the Settings page)."""
owner = _owner_filter(current_user, scope)
visible = owner & (Photo.is_discarded.is_(False)) & (Photo.is_hidden.is_(False))
all_photos_count = (
await db.execute(select(func.count(Photo.id)).where(visible))
).scalar() or 0
rated_count = (
await db.execute(
select(func.count(Photo.id)).where(visible, Photo.rating >= 1)
)
).scalar() or 0
colored_count = (
await db.execute(
select(func.count(Photo.id)).where(
visible, Photo.color_label.is_not(None)
)
)
).scalar() or 0
with_gps_count = (
await db.execute(
select(func.count(Photo.id)).where(
visible, Photo.latitude.is_not(None)
)
)
).scalar() or 0
duplicates_count = (
await db.execute(
select(func.count(Photo.id)).where(
visible, Photo.is_duplicate.is_(True)
)
)
).scalar() or 0
discarded_count = (
await db.execute(
select(func.count(Photo.id)).where(owner, Photo.is_discarded.is_(True))
)
).scalar() or 0
needs_review_count = (
await db.execute(
select(func.count(Photo.id)).where(visible, Photo.needs_review.is_(True))
)
).scalar() or 0
# Legacy split (kept for the existing /stats consumers).
photo_count = (
await db.execute(
select(func.count(Photo.id)).where(
owner,
Photo.media_type.in_(['photo', 'heic', 'raw'])
)
)
).scalar() or 0
video_count = (
await db.execute(
select(func.count(Photo.id)).where(owner, Photo.media_type == 'video')
)
).scalar() or 0
size = (await db.execute(select(func.sum(Photo.file_size)).where(owner))).scalar() or 0
# Source root directories (active ones only).
roots = (
await db.execute(
select(SourceRoot.path).where(SourceRoot.is_active.is_(True)).order_by(SourceRoot.path)
)
).scalars().all()
return {
"all_photos": all_photos_count,
"rated": rated_count,
"colored": colored_count,
"with_gps": with_gps_count,
"duplicates": duplicates_count,
"discarded": discarded_count,
"needs_review": needs_review_count,
"total_photos": photo_count,
"total_videos": video_count,
"total_size": size,
"total_size_gb": round(size / (1024**3), 2) if size else 0,
"source_dirs": roots,
}
@router.post("/scan")
async def trigger_scan(current_user: User = Depends(get_current_user)):
"""Trigger full library re-scan"""
from app.tasks.scan import scan_all_source_roots
scan_all_source_roots.delay()
return {"status": "success", "message": "Library scan started"}
@router.post("/maintenance/recover-stuck")
async def recover_stuck_photos(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Reset photos stuck in 'processing' for more than 30 minutes back to
'pending' so the pipeline can retry them. Returns the count of recovered
photos."""
from datetime import datetime, timedelta, timezone
cutoff = datetime.now(timezone.utc) - timedelta(minutes=30)
result = await db.execute(
update(Photo)
.where(
Photo.processing_status == 'processing',
Photo.updated_at < cutoff,
)
.values(
processing_status='pending',
processing_error='Auto-recovered from stuck processing state',
)
)
await db.commit()
count = result.rowcount
if count:
logger.info("Recovered %d stuck photos back to pending", count)
return {"status": "success", "recovered": count}
@router.post("/backfill-gps")
async def trigger_backfill_gps(current_user: User = Depends(get_current_user)):
"""Re-run EXIF metadata extraction on every photo that's still missing
GPS coordinates. Useful after fixing the EXIF parser, or any time the
Map view looks emptier than expected. Returns immediately — work runs
on the Celery worker."""
from app.tasks.scan import backfill_gps
backfill_gps.delay()
return {"status": "success", "message": "GPS backfill queued"}
@router.get("/scan/status")
async def get_scan_status(db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Get current scan status"""
import redis
from app.config import settings
# Connect to Redis to get scan status
r = redis.Redis.from_url(settings.redis_url)
# Get scan status from Redis (set by worker tasks)
is_scanning = r.get('scan:active') == b'true'
current_folder = r.get('scan:current_folder')
processed_files = int(r.get('scan:processed_files') or 0)
total_files = int(r.get('scan:total_files') or 0)
errors = r.lrange('scan:errors', 0, -1)
return {
"is_scanning": is_scanning,
"current_folder": current_folder.decode() if current_folder else None,
"processed_files": processed_files,
"total_files": total_files,
"errors": [e.decode() for e in errors] if errors else []
}
# ---------------------------------------------------------------------------
# Maintenance endpoints — surfaced via the Settings panel.
# ---------------------------------------------------------------------------
class RegenerateThumbnailsRequest(BaseModel):
"""Optional filters narrowing which photos get re-queued. With both
fields omitted the request resets every photo in the library."""
media_types: Optional[List[str]] = Field(
default=None,
description="Restrict to these media_type values (photo/raw/heic/video).",
)
only_failed: bool = Field(
default=False,
description="If true, only re-queue photos whose processing_status is 'failed'.",
)
only_pending: bool = Field(
default=False,
description="If true, only (re-)queue photos whose processing_status is 'pending'. "
"Useful for kicking rows that were created by a scan but never had "
"their thumbnail task picked up.",
)
@router.get("/maintenance/thumbnail-stats")
async def get_thumbnail_stats(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
scope: str | None = Query(None),
):
"""Counts of photos by processing_status, plus a media-type breakdown
so the Settings panel can show the user what's outstanding."""
owner = _owner_filter(current_user, scope)
status_rows = (
await db.execute(
select(Photo.processing_status, func.count(Photo.id))
.where(owner)
.group_by(Photo.processing_status)
)
).all()
media_rows = (
await db.execute(
select(Photo.media_type, func.count(Photo.id))
.where(owner)
.group_by(Photo.media_type)
)
).all()
by_status = {status or 'unknown': count for status, count in status_rows}
by_media_type = {media or 'unknown': count for media, count in media_rows}
total = sum(by_status.values())
return {
"total": total,
"pending": by_status.get('pending', 0),
"processing": by_status.get('processing', 0),
"completed": by_status.get('completed', 0),
"failed": by_status.get('failed', 0),
"by_media_type": by_media_type,
}
@router.post("/maintenance/regenerate-thumbnails")
async def regenerate_thumbnails(
body: RegenerateThumbnailsRequest,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
scope: str | None = Query(None),
):
"""Reset matching photos' on-disk thumbnail directories and re-queue
Celery thumbnail generation. Used by the Settings panel for the
'regenerate video thumbnails' / 'regenerate failed' buttons.
Files on disk are removed under /data/thumbs/<photo_id>/ so the next
request to /photos/{id}/thumb/{size} actually re-generates instead of
serving the stale placeholder.
"""
from app.tasks.thumbs import generate_thumbnails
owner = _owner_filter(current_user, scope)
# Validate media_types early so a typo can't silently match nothing.
media_types = body.media_types
if media_types is not None:
invalid = [m for m in media_types if m not in _VALID_MEDIA_TYPES]
if invalid:
return {
"status": "error",
"message": f"Invalid media_types: {invalid}. "
f"Allowed: {sorted(_VALID_MEDIA_TYPES)}",
}
query = select(Photo).where(owner)
if media_types:
query = query.where(Photo.media_type.in_(media_types))
if body.only_failed:
query = query.where(Photo.processing_status == 'failed')
if body.only_pending:
query = query.where(Photo.processing_status == 'pending')
photos = (await db.execute(query)).scalars().all()
cleared_dirs = 0
file_errors = 0
for photo in photos:
thumb_dir = f"/data/thumbs/{photo.id}"
if os.path.isdir(thumb_dir):
try:
shutil.rmtree(thumb_dir)
cleared_dirs += 1
except OSError as e:
file_errors += 1
logger.warning(f"Could not clear thumb dir {thumb_dir}: {e}")
photo.processing_status = 'pending'
photo.processing_error = None
photo.thumb_small = None
photo.thumb_medium = None
photo.thumb_large = None
await db.commit()
# Queue celery tasks AFTER the commit so the worker sees the reset
# state when it picks the job up.
queued = 0
for photo in photos:
try:
generate_thumbnails.delay(photo.id)
queued += 1
except Exception as e:
logger.warning(f"Could not queue thumbnail job for {photo.id}: {e}")
return {
"status": "success",
"matched": len(photos),
"queued": queued,
"cleared_dirs": cleared_dirs,
"file_errors": file_errors,
"filters": {
"media_types": media_types,
"only_failed": body.only_failed,
},
}
@router.get("/maintenance/worker-status")
async def get_worker_status(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
scope: str | None = Query(None),
):
"""Diagnostics for the Celery worker fleet + recent task failures.
Surfaced in the Settings panel so the user can spot a stuck queue or
a worker that's gone away without tailing container logs. Returns:
- workers: list of {name, status, active, concurrency, queues}
derived from celery_app.control.inspect(). `status` is 'online'
when ping succeeds, 'unreachable' otherwise. Empty list means no
workers are responding at all (broker down, container crashed,
wrong queue routing, etc.).
- queues: per-queue depth read from Redis (LLEN of each queue key
used by celery.kombu). Mirrors what tasks are waiting to be
picked up.
- failures: aggregate count of photos with processing_status='failed'
plus the most recent N error messages so the user can see *why*
things failed without opening the DB.
- broker_ok: bool — could we even reach Redis?
"""
owner = _owner_filter(current_user, scope)
from app.tasks.celery import celery_app
from app.config import settings
import redis as _redis
# ----- Celery inspect (workers + active tasks) -------------------------
# Each inspect.* call is a separate broadcast-and-wait with its own
# timeout, so running them serially multiplies the wait. Fan them out
# to threads and gather, collapsing 6 × timeout into ~1 × timeout.
# Timeout dropped to 0.5s — a responsive worker answers within a few
# ms; anything past that is effectively "not responding" for the
# purposes of a settings dashboard.
import asyncio
workers: list[dict] = []
inspect_error: Optional[str] = None
try:
inspect = celery_app.control.inspect(timeout=0.5)
ping, active, reserved, scheduled, stats, active_queues = await asyncio.gather(
asyncio.to_thread(inspect.ping),
asyncio.to_thread(inspect.active),
asyncio.to_thread(inspect.reserved),
asyncio.to_thread(inspect.scheduled),
asyncio.to_thread(inspect.stats),
asyncio.to_thread(inspect.active_queues),
)
ping = ping or {}
active = active or {}
reserved = reserved or {}
scheduled = scheduled or {}
stats = stats or {}
active_queues = active_queues or {}
worker_names = set(ping) | set(active) | set(stats)
for name in sorted(worker_names):
wstats = stats.get(name) or {}
pool = wstats.get('pool') or {}
workers.append({
"name": name,
"status": "online" if name in ping else "unreachable",
"active": len(active.get(name, []) or []),
"reserved": len(reserved.get(name, []) or []),
"scheduled": len(scheduled.get(name, []) or []),
"concurrency": pool.get('max-concurrency'),
"processed": (wstats.get('total') or {}),
"queues": [q.get('name') for q in (active_queues.get(name) or [])],
"active_tasks": [
{
"id": t.get('id'),
"name": t.get('name'),
"args": t.get('args'),
"time_start": t.get('time_start'),
}
for t in (active.get(name) or [])[:10]
],
})
except Exception as e:
inspect_error = str(e)
logger.warning(f"Celery inspect failed: {e}")
# ----- Broker / queue depth --------------------------------------------
broker_ok = False
queue_depths: dict[str, int] = {}
broker_error: Optional[str] = None
try:
r = _redis.Redis.from_url(settings.redis_url, socket_timeout=1.0)
r.ping()
broker_ok = True
# `vision` runs the content classifier — the only heavy queue.
for q in ('default', 'high', 'low', 'vision'):
try:
queue_depths[q] = int(r.llen(q) or 0)
except Exception:
queue_depths[q] = 0
except Exception as e:
broker_error = str(e)
logger.warning(f"Redis broker unreachable: {e}")
# ----- Recent task failures from the photos table ----------------------
failed_total = (
await db.execute(
select(func.count(Photo.id)).where(owner, Photo.processing_status == 'failed')
)
).scalar() or 0
recent_failed_rows = (
await db.execute(
select(
Photo.id,
Photo.filename,
Photo.media_type,
Photo.processing_error,
Photo.updated_at,
)
.where(owner, Photo.processing_status == 'failed')
.order_by(Photo.updated_at.desc().nullslast())
.limit(20)
)
).all()
recent_failures = [
{
"photo_id": row[0],
"filename": row[1],
"media_type": row[2],
"error": (row[3] or '')[:500],
"updated_at": row[4].isoformat() if row[4] else None,
}
for row in recent_failed_rows
]
# ----- Most recent scan errors (Redis list) ----------------------------
scan_errors: list[str] = []
try:
if broker_ok:
r = _redis.Redis.from_url(settings.redis_url, socket_timeout=1.0)
raw = r.lrange('scan:errors', 0, 19) or []
scan_errors = [e.decode(errors='replace') for e in raw]
except Exception as e:
logger.debug(f"Could not read scan:errors: {e}")
return {
"broker_ok": broker_ok,
"broker_error": broker_error,
"inspect_error": inspect_error,
"workers": workers,
"worker_count": len(workers),
"queues": queue_depths,
"failures": {
"total": failed_total,
"recent": recent_failures,
},
"scan_errors": scan_errors,
}
@router.get("/maintenance/pipeline-stats")
async def get_pipeline_stats(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
scope: str | None = Query(None),
):
"""Per-stage progress across the ingestion pipeline.
Returns a `{stage_key: {done, total, label}}` map so the Settings
panel can render one progress bar per stage. `total` is the number
of non-discarded photos the stage is *expected* to run on — which is
every non-discarded photo for most stages, or a narrower subset when
a stage is image-only (e.g. embeddings don't run on videos).
Keep the shape flat + serialisable; the frontend turns it straight
into a list of rows without needing to know about the models.
"""
from app.models.tags import photo_tags # association Table, not a model
owner = _owner_filter(current_user, scope)
not_discarded = owner & Photo.is_discarded.is_(False)
async def scalar_count(query):
return (await db.execute(query)).scalar() or 0
# Total non-discarded photos — the denominator for most stages.
total_photos = await scalar_count(
select(func.count(Photo.id)).where(not_discarded)
)
# Image-only denominator (embeddings, tags, faces, OCR, phash). We
# exclude videos because those stages either don't apply or run off
# the extracted video frame which is treated separately.
total_images = await scalar_count(
select(func.count(Photo.id)).where(
not_discarded, Photo.media_type != 'video'
)
)
completed = await scalar_count(
select(func.count(Photo.id)).where(
not_discarded, Photo.processing_status == 'completed'
)
)
with_exif = await scalar_count(
select(func.count(Photo.id)).where(
not_discarded, Photo.exif_json.is_not(None)
)
)
with_gps = await scalar_count(
select(func.count(Photo.id)).where(
not_discarded,
Photo.latitude.is_not(None),
Photo.longitude.is_not(None),
)
)
with_phash = await scalar_count(
select(func.count(Photo.id)).where(
not_discarded, Photo.phash.is_not(None)
)
)
# Classified: distinct photos with a content_type tag.
classified_done = await scalar_count(
select(func.count(func.distinct(photo_tags.c.photo_id)))
.select_from(photo_tags)
.join(Photo, Photo.id == photo_tags.c.photo_id)
.where(not_discarded, photo_tags.c.source == 'vision:clip_classifier')
)
needs_review_count = await scalar_count(
select(func.count(Photo.id)).where(not_discarded, Photo.needs_review.is_(True))
)
duplicate_groups = await scalar_count(
select(func.count(func.distinct(Photo.duplicate_group_id))).where(
not_discarded, Photo.duplicate_group_id.is_not(None)
)
)
duplicate_members = await scalar_count(
select(func.count(Photo.id)).where(
not_discarded, Photo.duplicate_group_id.is_not(None)
)
)
# Ordered list so the frontend renders stages in pipeline order
# without needing to know the sequence itself.
stages = [
{
"key": "thumbnails",
"label": "Thumbnails & pHash",
"done": completed,
"total": total_photos,
"hint": "Generated on scan. Unlocks every downstream stage.",
},
{
"key": "exif",
"label": "EXIF metadata",
"done": with_exif,
"total": total_photos,
"hint": "Camera, lens, capture time. Required for GPS + taken_at.",
},
{
"key": "gps",
"label": "GPS coordinates",
"done": with_gps,
"total": total_photos,
"hint": "Subset of EXIF. Drives the map view; many photos legitimately have none.",
"partial": True, # not every photo is expected to have GPS
},
{
"key": "phash",
"label": "Perceptual hashes",
"done": with_phash,
"total": total_images,
"hint": "Feeds duplicate detection.",
},
{
"key": "classification",
"label": "Content classification (photo vs other)",
"done": classified_done,
"total": total_images,
"hint": f"{needs_review_count} photos flagged for review.",
},
{
"key": "duplicates",
"label": "Duplicate groups",
"done": duplicate_groups,
"total": duplicate_groups, # same — current count, not a progress ratio
"hint": f"{duplicate_members} photos in {duplicate_groups} groups. Run regroup_duplicates after new imports.",
"standalone": True,
},
]
return {
"total_photos": total_photos,
"total_images": total_images,
"stages": stages,
}
@router.get("/maintenance/missing-stats")
async def get_missing_stats(current_user: User = Depends(get_current_user)):
"""Count photos whose files no longer exist on disk under a mounted
source root. Surfaced in Settings so the user can see a number before
pulling the trigger on prune-missing. Cheap enough to call freely."""
from app.services.cleanup import prune_missing_photos
return await prune_missing_photos(dry_run=True)
@router.post("/maintenance/prune-missing")
async def run_prune_missing(current_user: User = Depends(get_current_user)):
"""Actually delete the orphaned photo rows reported by /missing-stats.
Common cause: PHOTO_DIRS in .env was repointed at a different library
leaving every old row dangling. Skips any photo whose source root
isn't currently mounted (almost always means an unmounted drive)."""
from app.services.cleanup import prune_missing_photos
try:
return {"status": "success", **(await prune_missing_photos(dry_run=False))}
except Exception as e:
logger.error(f"Prune missing failed: {e}")
return {"status": "error", "message": str(e)}
@router.post("/maintenance/cleanup")
async def run_data_integrity_cleanup(current_user: User = Depends(get_current_user)):
"""Re-run the source-roots / folders / photos data-integrity cleanup
that normally only runs on backend startup. Idempotent."""
from app.services.cleanup import cleanup_data_integrity
try:
await cleanup_data_integrity()
return {"status": "success"}
except Exception as e:
logger.error(f"Manual cleanup failed: {e}")
return {"status": "error", "message": str(e)}
# ─────────────────────────────────────────────────────────────────────────
# Duplicate detection
# ─────────────────────────────────────────────────────────────────────────
@router.get("/duplicates/groups")
async def get_duplicate_groups(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
scope: str | None = Query(None),
):
"""Return every duplicate group with its members.
Drives the frontend grouped grid view in the Duplicates section. One
SQL query, bucketed in Python — no N+1, no per-member fetch. Groups
are sorted by member_count DESC then earliest taken_at DESC so the
biggest / most recent clusters bubble to the top.
Each group also carries a `reason` field:
* "exact" — every member shares the same SHA-256 (true byte
duplicates that the perceptual hash trivially caught)
* "similar" — members differ at the byte level but match perceptually
"""
owner = _owner_filter(current_user, scope)
rows = (
await db.execute(
select(
Photo.id,
Photo.filename,
Photo.taken_at,
Photo.file_size,
Photo.width,
Photo.height,
Photo.thumb_small,
Photo.file_hash,
Photo.folder_id,
Photo.media_type,
Photo.duplicate_group_id,
)
.where(owner)
.where(Photo.duplicate_group_id.is_not(None))
.where(Photo.is_discarded.is_(False))
.where(Photo.is_hidden.is_(False))
.order_by(Photo.duplicate_group_id)
)
).all()
# Bucket members by group_id.
groups: dict[str, list[dict]] = {}
for row in rows:
member = {
"id": row[0],
"filename": row[1],
"taken_at": row[2].isoformat() if row[2] else None,
"file_size": row[3],
"width": row[4],
"height": row[5],
"thumb_small": row[6],
"file_hash": row[7],
"folder_id": row[8],
"media_type": row[9],
}
groups.setdefault(row[10], []).append(member)
def earliest(g: list[dict]) -> str:
# Used as a secondary sort key. Photos with no taken_at sort last
# by returning a far-future sentinel.
taken = [m["taken_at"] for m in g if m["taken_at"]]
return min(taken) if taken else "9999"
out = []
for group_id, members in groups.items():
if len(members) < 2:
# Defensive: a regroup race could leave a singleton briefly.
# Skip it so the UI never shows a "group of 1".
continue
# exact iff every member shares the same non-null file_hash
# (true byte-identical copies that pHash also caught). Anything
# else — different hashes, missing hashes — counts as "similar".
all_hashes = [m["file_hash"] for m in members]
reason = (
"exact"
if len(set(all_hashes)) == 1 and all_hashes[0] is not None
else "similar"
)
out.append({
"group_id": group_id,
"member_count": len(members),
"reason": reason,
"members": members,
})
out.sort(key=lambda g: (-g["member_count"], earliest(g["members"])))
return {
"groups": out,
"total_groups": len(out),
"total_members": sum(g["member_count"] for g in out),
}
@router.post("/maintenance/regroup-duplicates")
async def trigger_regroup_duplicates(current_user: User = Depends(get_current_user)):
"""Recompute duplicate groups from current perceptual hashes.
Fires the celery `regroup_duplicates` task which walks every photo's
phash, clusters by Hamming distance, and rewrites duplicate_group_id /
is_duplicate columns. Idempotent."""
from app.tasks.thumbs import regroup_duplicates_task
try:
regroup_duplicates_task.delay()
return {"status": "queued"}
except Exception as e:
logger.error(f"Regroup queue failed: {e}")
return {"status": "error", "message": str(e)}
@router.post("/maintenance/backfill-phashes")
async def trigger_backfill_phashes(current_user: User = Depends(get_current_user)):
"""Compute perceptual hashes for every photo currently missing one.
One-shot recovery path for libraries that existed before the phash
column was added — the thumbs worker computes phash for everything
new, but old rows need a backfill pass."""
from app.tasks.thumbs import backfill_phashes
try:
backfill_phashes.delay()
return {"status": "queued"}
except Exception as e:
logger.error(f"Backfill queue failed: {e}")
return {"status": "error", "message": str(e)}
@router.post("/maintenance/start-watcher")
async def start_file_watcher(current_user: User = Depends(get_current_user)):
"""Start the filesystem watcher. Uses a Redis lock so only one
instance runs at a time — safe to call repeatedly."""
from app.tasks.scan import watch_folders
try:
watch_folders.apply_async(countdown=2)
return {"status": "queued"}
except Exception as e:
logger.error(f"Watcher queue failed: {e}")
return {"status": "error", "message": str(e)}

File diff suppressed because it is too large Load Diff

View File

@@ -1,70 +0,0 @@
"""
Search API router — unified hybrid search endpoint.
"""
from typing import Optional
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import select
from app.database import get_db
from app.models import Photo
from app.services.search import hybrid_search
from app.models.user import User
from app.dependencies import get_current_user
router = APIRouter()
class SearchRequest(BaseModel):
q: Optional[str] = None
filters: Optional[dict] = None
limit: int = 50
offset: int = 0
@router.post("")
async def search_photos(body: SearchRequest, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""FTS search over photo metadata with optional tag and date filters."""
filters = body.filters or {}
results = await hybrid_search(
db=db,
q=body.q,
tag_ids=filters.get("tag_ids"),
date_from=filters.get("date_from"),
date_to=filters.get("date_to"),
limit=body.limit,
offset=body.offset,
)
if not results:
return {"results": [], "total": 0}
# Hydrate with photo data
photo_ids = [r["photo_id"] for r in results]
stmt = select(Photo).where(Photo.id.in_(photo_ids), Photo.user_id == current_user.id)
rows = (await db.execute(stmt)).scalars().all()
photo_map = {p.id: p for p in rows}
hydrated = []
for r in results:
photo = photo_map.get(r["photo_id"])
if not photo:
continue
hydrated.append({
"id": photo.id,
"filename": photo.filename,
"filepath": photo.filepath,
"media_type": photo.media_type,
"width": photo.width,
"height": photo.height,
"taken_at": photo.taken_at.isoformat() if photo.taken_at else None,
"rating": photo.rating,
"color_label": photo.color_label,
"thumb_small": photo.thumb_small,
"thumb_medium": photo.thumb_medium,
"score": r["score"],
})
return {"results": hydrated, "total": len(hydrated)}

View File

@@ -1,364 +0,0 @@
"""
Sharing API router — manage cross-user access to heaps and folders.
"""
import logging
from typing import Literal, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import select, func
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.heaps import Heap, heap_photos
from app.models.folders import Folder, SourceRoot
from app.models.photos import Photo
from app.models.sharing import HeapShare, FolderShare
from app.models.user import User
from app.dependencies import (
get_current_user,
get_user_heap,
get_user_folder,
resolve_username,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/sharing", tags=["sharing"])
# ── Schemas ──────────────────────────────────────────────────────────────
class ShareCreate(BaseModel):
username: str
permission: Literal["read", "write"] = "read"
class ShareResponse(BaseModel):
id: str
shared_with_id: str
shared_with_username: str
permission: str
created_at: str
class SharedHeapResponse(BaseModel):
id: str
name: str
owner_username: str
permission: str
photo_count: int
class SharedFolderResponse(BaseModel):
id: str
name: str
folder_type: str
owner_username: str
permission: str
photo_count: int
# ── Heap sharing ─────────────────────────────────────────────────────────
@router.get("/heaps/shared-with-me")
async def list_shared_heaps(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all heaps that have been shared with the current user."""
result = await db.execute(
select(HeapShare, Heap, User)
.join(Heap, HeapShare.heap_id == Heap.id)
.join(User, HeapShare.owner_id == User.id)
.where(HeapShare.shared_with_id == current_user.id)
)
rows = result.all()
items = []
for share, heap, owner in rows:
# Count photos in this heap.
count_result = await db.execute(
select(func.count()).select_from(heap_photos).where(
heap_photos.c.heap_id == heap.id
)
)
count = count_result.scalar() or 0
items.append(SharedHeapResponse(
id=heap.id,
name=heap.name,
owner_username=owner.username,
permission=share.permission,
photo_count=count,
))
return items
@router.get("/heaps/{heap_id}")
async def list_heap_shares(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all shares for a heap (owner only)."""
heap = await get_user_heap(heap_id, current_user, db)
result = await db.execute(
select(HeapShare, User)
.join(User, HeapShare.shared_with_id == User.id)
.where(HeapShare.heap_id == heap.id)
)
return [
ShareResponse(
id=share.id,
shared_with_id=user.id,
shared_with_username=user.username,
permission=share.permission,
created_at=share.created_at.isoformat() if share.created_at else "",
)
for share, user in result.all()
]
@router.post("/heaps/{heap_id}", status_code=201)
async def share_heap(
heap_id: str,
body: ShareCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Share a heap with another user (owner only)."""
heap = await get_user_heap(heap_id, current_user, db)
target_user = await resolve_username(body.username, db)
if target_user.id == current_user.id:
raise HTTPException(status_code=400, detail="Cannot share with yourself")
# Check for existing share.
existing = await db.execute(
select(HeapShare).where(
HeapShare.heap_id == heap.id,
HeapShare.shared_with_id == target_user.id,
)
)
if existing.scalar_one_or_none():
raise HTTPException(status_code=409, detail="Already shared with this user")
share = HeapShare(
heap_id=heap.id,
owner_id=current_user.id,
shared_with_id=target_user.id,
permission=body.permission,
)
db.add(share)
await db.commit()
logger.info("Heap %s shared with %s (%s)", heap.name, target_user.username, body.permission)
return {"status": "shared", "share_id": share.id}
@router.delete("/heaps/{heap_id}/{share_id}", status_code=204)
async def revoke_heap_share(
heap_id: str,
share_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Revoke a heap share. The owner can revoke any share; a recipient
can revoke their own share (i.e. leave)."""
result = await db.execute(
select(HeapShare).where(HeapShare.id == share_id, HeapShare.heap_id == heap_id)
)
share = result.scalar_one_or_none()
if share is None:
raise HTTPException(status_code=404, detail="Share not found")
# Must be the owner or the recipient themselves.
if share.owner_id != current_user.id and share.shared_with_id != current_user.id:
raise HTTPException(status_code=403, detail="Not authorized")
await db.delete(share)
await db.commit()
# ── Folder sharing ───────────────────────────────────────────────────────
@router.get("/folders/shared-with-me")
async def list_shared_folders(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all folders/source roots shared with the current user."""
result = await db.execute(
select(FolderShare, User)
.join(User, FolderShare.owner_id == User.id)
.where(FolderShare.shared_with_id == current_user.id)
)
rows = result.all()
items = []
for share, owner in rows:
# Resolve the folder/source root name and photo count.
if share.folder_type == "source_root":
sr_result = await db.execute(
select(SourceRoot).where(SourceRoot.id == share.folder_id)
)
entity = sr_result.scalar_one_or_none()
if not entity:
continue
name = entity.name
# Count all photos under this source root's folders.
count_result = await db.execute(
select(func.count()).select_from(Photo).where(
Photo.folder_id.in_(
select(Folder.id).where(Folder.source_root_id == entity.id)
),
Photo.is_discarded.is_(False),
)
)
else:
folder_result = await db.execute(
select(Folder).where(Folder.id == share.folder_id)
)
entity = folder_result.scalar_one_or_none()
if not entity:
continue
name = entity.name
import os
target_path = os.path.normpath(entity.path).rstrip(os.sep)
count_result = await db.execute(
select(func.count()).select_from(Photo).where(
Photo.folder_id.in_(
select(Folder.id).where(
(Folder.path == target_path)
| (Folder.path.like(target_path + os.sep + "%"))
)
),
Photo.is_discarded.is_(False),
)
)
count = count_result.scalar() or 0
items.append(SharedFolderResponse(
id=share.folder_id,
name=name,
folder_type=share.folder_type,
owner_username=owner.username,
permission=share.permission,
photo_count=count,
))
return items
@router.get("/folders/{folder_id}")
async def list_folder_shares(
folder_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all shares for a folder (owner only)."""
# Verify ownership — try folder then source root.
owned = False
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == current_user.id)
)
if result.scalar_one_or_none():
owned = True
else:
result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id)
)
if result.scalar_one_or_none():
owned = True
if not owned:
raise HTTPException(status_code=404, detail="Folder not found")
result = await db.execute(
select(FolderShare, User)
.join(User, FolderShare.shared_with_id == User.id)
.where(FolderShare.folder_id == folder_id)
)
return [
ShareResponse(
id=share.id,
shared_with_id=user.id,
shared_with_username=user.username,
permission=share.permission,
created_at=share.created_at.isoformat() if share.created_at else "",
)
for share, user in result.all()
]
@router.post("/folders/{folder_id}", status_code=201)
async def share_folder(
folder_id: str,
body: ShareCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Share a folder or source root with another user (owner only)."""
# Determine folder_type and verify ownership.
folder_type = "folder"
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == current_user.id)
)
entity = result.scalar_one_or_none()
if entity is None:
result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id)
)
entity = result.scalar_one_or_none()
if entity is None:
raise HTTPException(status_code=404, detail="Folder not found")
folder_type = "source_root"
target_user = await resolve_username(body.username, db)
if target_user.id == current_user.id:
raise HTTPException(status_code=400, detail="Cannot share with yourself")
existing = await db.execute(
select(FolderShare).where(
FolderShare.folder_id == folder_id,
FolderShare.shared_with_id == target_user.id,
)
)
if existing.scalar_one_or_none():
raise HTTPException(status_code=409, detail="Already shared with this user")
share = FolderShare(
folder_id=folder_id,
folder_type=folder_type,
owner_id=current_user.id,
shared_with_id=target_user.id,
permission=body.permission,
)
db.add(share)
await db.commit()
logger.info("Folder %s shared with %s (%s)", entity.name, target_user.username, body.permission)
return {"status": "shared", "share_id": share.id}
@router.delete("/folders/{folder_id}/{share_id}", status_code=204)
async def revoke_folder_share(
folder_id: str,
share_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Revoke a folder share (owner or self-remove)."""
result = await db.execute(
select(FolderShare).where(FolderShare.id == share_id, FolderShare.folder_id == folder_id)
)
share = result.scalar_one_or_none()
if share is None:
raise HTTPException(status_code=404, detail="Share not found")
if share.owner_id != current_user.id and share.shared_with_id != current_user.id:
raise HTTPException(status_code=403, detail="Not authorized")
await db.delete(share)
await db.commit()

View File

@@ -1,155 +0,0 @@
"""
Tags API router.
Unified across user tags and the binary content-type classifier
('photography' | 'other') via the `kind` column.
"""
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import select, func, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Photo, Tag
from app.models.tags import photo_tags
from app.models.user import User
from app.dependencies import get_current_user
router = APIRouter()
# ── Schemas ───────────────────────────────────────────────────────────────
class TagCreate(BaseModel):
name: str
color: Optional[str] = None
kind: str = "user"
class TagUpdate(BaseModel):
name: Optional[str] = None
color: Optional[str] = None
# ── Endpoints ─────────────────────────────────────────────────────────────
@router.get("")
async def list_tags(
kind: Optional[str] = Query(None, description="Filter by kind: user, content_type"),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all tags with their photo counts, optionally filtered by kind.
Photo counts here drive the Tags / People sidebar badges, so they
exclude discarded + hidden-folder photos to match the rest of the
cross-cutting views. A tag that only appears on hidden-folder
photos will still show up with count=0 — we don't drop empty tags
because the user may want to see them in the management UI.
"""
count_subq = (
select(
photo_tags.c.tag_id,
func.count(photo_tags.c.photo_id).label("photo_count"),
func.min(photo_tags.c.photo_id).label("first_photo_id"),
)
.select_from(
photo_tags.join(Photo, Photo.id == photo_tags.c.photo_id)
)
.where(
Photo.user_id == current_user.id,
Photo.is_discarded.is_(False),
Photo.is_hidden.is_(False),
)
.group_by(photo_tags.c.tag_id)
.subquery()
)
stmt = (
select(Tag, count_subq.c.photo_count, count_subq.c.first_photo_id)
.outerjoin(count_subq, Tag.id == count_subq.c.tag_id)
.where(Tag.user_id == current_user.id)
)
if kind:
stmt = stmt.where(Tag.kind == kind)
stmt = stmt.order_by(Tag.name.asc())
result = await db.execute(stmt)
rows = result.all()
return [
{
"id": tag.id,
"name": tag.name,
"color": tag.color,
"kind": tag.kind,
"source": tag.source,
"representative_photo_id": first_photo_id,
"photo_count": int(count or 0),
}
for tag, count, first_photo_id in rows
]
@router.post("", status_code=201)
async def create_tag(body: TagCreate, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Create a new tag. The (name, kind) pair is unique — re-creating an
existing pair returns the existing row (idempotent for autocomplete)."""
name = (body.name or "").strip()
if not name:
raise HTTPException(status_code=400, detail="Tag name is required")
existing = await db.execute(
select(Tag).where(Tag.name == name, Tag.kind == body.kind, Tag.user_id == current_user.id)
)
found = existing.scalar_one_or_none()
if found:
return {
"id": found.id, "name": found.name, "color": found.color,
"kind": found.kind, "photo_count": 0,
}
tag = Tag(name=name, color=body.color, kind=body.kind, user_id=current_user.id)
db.add(tag)
await db.commit()
await db.refresh(tag)
return {
"id": tag.id, "name": tag.name, "color": tag.color,
"kind": tag.kind, "photo_count": 0,
}
@router.patch("/{tag_id}")
async def update_tag(
tag_id: str, body: TagUpdate, db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Rename or recolor a tag."""
result = await db.execute(select(Tag).where(Tag.id == tag_id, Tag.user_id == current_user.id))
tag = result.scalar_one_or_none()
if not tag:
raise HTTPException(status_code=404, detail="Tag not found")
if body.name is not None:
name = body.name.strip()
if not name:
raise HTTPException(status_code=400, detail="Tag name is required")
tag.name = name
if body.color is not None:
tag.color = body.color or None
await db.commit()
await db.refresh(tag)
return {"id": tag.id, "name": tag.name, "color": tag.color, "kind": tag.kind}
@router.delete("/{tag_id}", status_code=204)
async def delete_tag(tag_id: str, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user)):
"""Delete a tag. Photo associations cascade-delete via the FK."""
result = await db.execute(select(Tag).where(Tag.id == tag_id, Tag.user_id == current_user.id))
tag = result.scalar_one_or_none()
if not tag:
raise HTTPException(status_code=404, detail="Tag not found")
await db.delete(tag)
await db.commit()
return None

View File

@@ -1,303 +0,0 @@
"""
Upload router — lets users drop files (or whole folders) from their
desktop into a destination Folder, preserving any sub-folder structure
they bring with them.
Each POST handles one file. The frontend fans out many parallel requests
per drop, giving it per-file progress without the server having to
invent a chunking protocol. For folder uploads, the browser passes
`webkitRelativePath` under the `relative_path` field; any leading
sub-directories there are materialised on disk (and as Folder rows)
under the destination.
Uploaded files are placed under the destination folder on the owner's
media mount, indexed immediately (Photo row created), and queued for
the same thumb + metadata pipeline that the scanner uses. An optional
`heap_id` also drops them into a heap in the same request.
"""
import hashlib
import logging
import os
from pathlib import Path
from datetime import datetime
from typing import Optional
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
from sqlalchemy import insert, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.dependencies import get_current_user
from app.models import Folder, Heap, Photo, SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
from app.services.date_guess import has_date_warning
from app.tasks.scan import SUPPORTED_EXTENSIONS, get_media_type
from app.tasks.thumbs import generate_thumbnails
from app.services.metadata import extract_metadata
logger = logging.getLogger(__name__)
router = APIRouter()
MAX_UPLOAD_BYTES = 500 * 1024 * 1024 # 500 MB per file cap.
def _validate_segment(segment: str) -> str:
"""Reject path segments that would escape the destination directory."""
segment = segment.strip()
if not segment or segment in ('.', '..') or '/' in segment or '\\' in segment:
raise HTTPException(status_code=400, detail=f"Invalid path segment: {segment!r}")
return segment
def _sanitize_relative_path(rel: Optional[str]) -> list[str]:
"""Split `relative_path` into safe segments (dirs + filename).
Empty or missing → []. Any absolute path, backslash, or `..` segment
raises 400 — we never want an upload to escape the destination.
"""
if not rel:
return []
# Normalise backslashes to forward slashes; browsers on Windows send
# webkitRelativePath with forward slashes anyway, but defend in depth.
rel = rel.replace('\\', '/').strip('/')
if not rel:
return []
segs = [_validate_segment(s) for s in rel.split('/') if s]
return segs
async def _resolve_destination(
folder_id: str,
user: User,
db: AsyncSession,
) -> Folder:
"""Resolve `folder_id` to a concrete Folder row the user owns.
Accepts both Folder ids and SourceRoot ids (for source roots, we
return the Folder row at the mount path — the scanner creates one
for every source root it walks). Raises 404 if neither matches.
"""
folder = (await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)).scalar_one_or_none()
if folder is not None:
return folder
sr = (await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == user.id)
)).scalar_one_or_none()
if sr is None:
raise HTTPException(status_code=404, detail="Destination folder not found")
root_folder = (await db.execute(
select(Folder).where(
Folder.source_root_id == sr.id,
Folder.user_id == user.id,
Folder.path == os.path.normpath(sr.path),
)
)).scalar_one_or_none()
if root_folder is None:
# First-time source root with no walk yet — create the row now so
# uploads work even before the initial scan has run.
root_folder = Folder(
name=sr.name or os.path.basename(sr.path),
path=os.path.normpath(sr.path),
source_root_id=sr.id,
user_id=user.id,
)
os.makedirs(root_folder.path, exist_ok=True)
db.add(root_folder)
await db.flush()
return root_folder
async def _ensure_subfolder(
parent: Folder,
name: str,
user: User,
db: AsyncSession,
) -> Folder:
"""Return (or create) a Folder row named `name` under `parent`.
Also mkdirs the directory on disk. Idempotent — safe to call for a
path segment that already exists as a Folder row or directory.
"""
child_path = os.path.normpath(os.path.join(parent.path, name))
existing = (await db.execute(
select(Folder).where(
Folder.path == child_path,
Folder.user_id == user.id,
)
)).scalar_one_or_none()
if existing is not None:
os.makedirs(child_path, exist_ok=True)
return existing
os.makedirs(child_path, exist_ok=True)
child = Folder(
name=name,
path=child_path,
parent_id=parent.id,
source_root_id=parent.source_root_id,
user_id=user.id,
is_hidden=parent.is_hidden,
)
db.add(child)
await db.flush()
return child
def _unique_path(target_dir: str, filename: str) -> tuple[str, str]:
"""Return a (filepath, filename) that doesn't collide with an
existing file on disk. Suffixes " (2)", " (3)", ... until a free
slot is found. Prevents upload-over-existing and keeps the user's
original file intact.
"""
base, ext = os.path.splitext(filename)
candidate = os.path.join(target_dir, filename)
n = 2
while os.path.exists(candidate):
new_name = f"{base} ({n}){ext}"
candidate = os.path.join(target_dir, new_name)
n += 1
return candidate, os.path.basename(candidate)
@router.post("")
async def upload_file(
file: UploadFile = File(...),
destination_folder_id: str = Form(...),
relative_path: Optional[str] = Form(None),
heap_id: Optional[str] = Form(None),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Upload a single file into a destination folder (and optionally a
heap). For folder uploads, `relative_path` carries the sub-folder
chain from the browser's `webkitRelativePath`, and we materialise
it under the destination on disk + as Folder rows.
Returns the created photo's id on success. 4xx on unsupported file
type, bad path, missing destination, or too-large file.
"""
# --- validate inputs -------------------------------------------------
raw_name = file.filename or ''
if not raw_name:
raise HTTPException(status_code=400, detail="Missing filename")
# Prefer the leaf of relative_path when present (it contains the
# original filename as the browser saw it inside the picked folder).
segs = _sanitize_relative_path(relative_path)
if segs:
leaf = segs[-1]
subdirs = segs[:-1]
else:
leaf = _validate_segment(os.path.basename(raw_name))
subdirs = []
ext = Path(leaf).suffix.lower()
if ext not in SUPPORTED_EXTENSIONS:
raise HTTPException(
status_code=400,
detail=f"Unsupported file type: {ext or '(none)'}",
)
dest_folder = await _resolve_destination(destination_folder_id, current_user, db)
target_folder = dest_folder
for seg in subdirs:
target_folder = await _ensure_subfolder(target_folder, seg, current_user, db)
target_dir = target_folder.path
os.makedirs(target_dir, exist_ok=True)
filepath, final_name = _unique_path(target_dir, leaf)
# --- stream to disk, hash as we go ----------------------------------
hasher = hashlib.sha256()
total = 0
try:
with open(filepath, 'wb') as out:
while True:
chunk = await file.read(1024 * 1024)
if not chunk:
break
total += len(chunk)
if total > MAX_UPLOAD_BYTES:
out.close()
os.unlink(filepath)
raise HTTPException(
status_code=413,
detail=f"File exceeds {MAX_UPLOAD_BYTES // (1024*1024)}MB limit",
)
hasher.update(chunk)
out.write(chunk)
except HTTPException:
raise
except Exception as e:
logger.error(f"Upload write failed for {filepath}: {e}")
if os.path.exists(filepath):
try:
os.unlink(filepath)
except OSError:
pass
raise HTTPException(status_code=500, detail=f"Upload failed: {e}")
file_hash = hasher.hexdigest()
# --- validate heap before committing the DB row ---------------------
if heap_id:
heap = (await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == current_user.id)
)).scalar_one_or_none()
if heap is None:
# Destination heap vanished — still keep the file + photo row,
# but tell the caller so the UI can surface the mismatch.
heap_id = None
# --- create Photo row ------------------------------------------------
mtime_dt = datetime.fromtimestamp(os.stat(filepath).st_mtime)
photo = Photo(
filepath=filepath,
filename=final_name,
folder_id=target_folder.id,
user_id=current_user.id,
file_hash=file_hash,
media_type=get_media_type(filepath),
original_format=Path(filepath).suffix.upper()[1:],
file_size=total,
taken_at=mtime_dt,
taken_at_source='filesystem',
has_date_warning=has_date_warning(filepath, mtime_dt),
is_hidden=bool(target_folder.is_hidden),
processing_status='pending',
)
db.add(photo)
await db.flush()
if heap_id:
await db.execute(
insert(heap_photos),
[{"heap_id": heap_id, "photo_id": photo.id}],
)
await db.commit()
# Queue the same background work the scanner does so thumbnails +
# EXIF show up without the user having to trigger a rescan.
try:
generate_thumbnails.delay(photo.id)
extract_metadata.delay(photo.id)
except Exception as e:
logger.warning(f"Failed to queue post-upload tasks for {photo.id}: {e}")
return {
"photo_id": photo.id,
"filename": final_name,
"folder_id": target_folder.id,
"folder_path": target_folder.path,
"heap_id": heap_id,
}

View File

@@ -1,74 +0,0 @@
"""
Pydantic schemas for photos
"""
from pydantic import BaseModel, Field
from typing import Optional, List, Dict, Any
from datetime import datetime
class PhotoBase(BaseModel):
"""Base photo schema"""
filename: str
media_type: str
original_format: Optional[str] = None
width: Optional[int] = None
height: Optional[int] = None
file_size: Optional[int] = None
taken_at: Optional[datetime] = None
taken_at_source: Optional[str] = None
user_title: Optional[str] = None
user_notes: Optional[str] = None
rating: int = 0
color_label: Optional[str] = None
class PhotoResponse(PhotoBase):
"""Photo response schema"""
id: str
filepath: str
folder_id: Optional[str] = None
file_hash: Optional[str] = None
added_at: datetime
updated_at: Optional[datetime] = None
is_discarded: bool = False
discarded_at: Optional[datetime] = None
thumb_small: Optional[str] = None
thumb_medium: Optional[str] = None
thumb_large: Optional[str] = None
processing_status: str = 'pending'
processing_error: Optional[str] = None
exif_json: Optional[str] = None
latitude: Optional[float] = None
longitude: Optional[float] = None
is_duplicate: bool = False
needs_review: bool = False
has_date_warning: bool = False
live_photo_video_id: Optional[str] = None
owner_username: Optional[str] = None
# tags: List[Dict[str, Any]] = [] # TODO: Enable when using eager loading
class Config:
orm_mode = True
from_attributes = True
class PhotoUpdate(BaseModel):
"""Photo update schema"""
filename: Optional[str] = None
user_title: Optional[str] = None
user_notes: Optional[str] = None
rating: Optional[int] = Field(None, ge=0, le=5)
color_label: Optional[str] = None
is_discarded: Optional[bool] = None
taken_at: Optional[datetime] = None
class PhotoListResponse(BaseModel):
"""Photo list response with pagination"""
photos: List[PhotoResponse]
total: int
page: int
per_page: int
pages: int
class BulkAction(BaseModel):
"""Bulk action on photos"""
ids: List[str]
action: str # 'discard', 'restore', 'delete_permanent', 'move', 'copy', 'add_tag', 'remove_tag', 'set_rating', 'set_color'
value: Optional[Any] = None # For actions that need a value (rating, color, tag_id, folder_id)

View File

@@ -1,275 +0,0 @@
"""
One-shot data integrity cleanup for source_roots / folders / photos.
Earlier versions of the scanner stored paths verbatim, so trailing slashes
and redundant separators produced duplicate SourceRoot and Folder rows for
the same physical directory. The watcher also auto-created source roots
when fired with a parent dir. This module merges the duplicates and
re-points photos to the canonical folder so the data lines up with the
post-fix scanner.
Idempotent: safe to run on every backend startup.
"""
import os
import logging
from datetime import datetime
from sqlalchemy import select, update, func
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import AsyncSessionLocal
from app.models import Photo, Folder, SourceRoot
logger = logging.getLogger(__name__)
def _normalize_path(path: str) -> str:
return os.path.normpath(path)
async def _dedupe_source_roots(session: AsyncSession) -> int:
"""Group source roots by normalized path and merge duplicates. Returns
the number of rows deleted."""
result = await session.execute(select(SourceRoot))
rows = result.scalars().all()
groups: dict[str, list[SourceRoot]] = {}
for sr in rows:
norm = _normalize_path(sr.path)
groups.setdefault(norm, []).append(sr)
deleted = 0
for norm, srs in groups.items():
if len(srs) == 1:
# Make sure the canonical row's path is normalized too.
if srs[0].path != norm:
srs[0].path = norm
continue
# Pick the canonical row: prefer one with a non-empty name and the
# earliest added_at (most likely the original).
canonical = sorted(
srs,
key=lambda s: (not bool(s.name), s.added_at or datetime.max),
)[0]
canonical.path = norm
for sr in srs:
if sr.id == canonical.id:
continue
# Re-point folders that referenced the duplicate root.
await session.execute(
update(Folder)
.where(Folder.source_root_id == sr.id)
.values(source_root_id=canonical.id)
)
await session.delete(sr)
deleted += 1
return deleted
async def _dedupe_folders(session: AsyncSession) -> int:
"""Group folders by normalized path and merge duplicates. Returns the
number of rows deleted."""
result = await session.execute(select(Folder))
rows = result.scalars().all()
groups: dict[str, list[Folder]] = {}
for f in rows:
norm = _normalize_path(f.path)
groups.setdefault(norm, []).append(f)
deleted = 0
for norm, folders in groups.items():
if len(folders) == 1:
if folders[0].path != norm:
folders[0].path = norm
continue
# Canonical = the one with the most photos already attached, then
# the lowest-id (deterministic tiebreaker).
canonical = sorted(
folders,
key=lambda f: (-(f.photo_count or 0), f.id),
)[0]
canonical.path = norm
for f in folders:
if f.id == canonical.id:
continue
# Re-point photos to the canonical folder.
await session.execute(
update(Photo)
.where(Photo.folder_id == f.id)
.values(folder_id=canonical.id)
)
await session.delete(f)
deleted += 1
return deleted
async def _recompute_folder_counts(session: AsyncSession) -> None:
"""Set folder.photo_count to the actual non-discarded photo count."""
result = await session.execute(select(Folder))
folders = result.scalars().all()
for f in folders:
count_result = await session.execute(
select(func.count(Photo.id)).where(
Photo.folder_id == f.id,
Photo.is_discarded == False, # noqa: E712
)
)
f.photo_count = int(count_result.scalar() or 0)
async def _warn_stale_source_roots(session: AsyncSession) -> int:
"""Log a warning for any active source root whose path no longer exists
on disk. Doesn't delete — a missing path could be a temporarily
unmounted drive, and silently dropping user data is worse than
surfacing a noisy log line.
"""
result = await session.execute(select(SourceRoot))
rows = result.scalars().all()
stale = 0
for sr in rows:
if not os.path.isdir(sr.path):
stale += 1
logger.warning(
f"Source root '{sr.name}' path is missing on disk: {sr.path} "
f"— is the docker mount still in place? "
f"(Edit docker-compose.yml or PHOTO_DIRS in .env to fix.)"
)
return stale
async def find_missing(
session: AsyncSession,
) -> tuple[list[str], list[str], list[str]]:
"""Walk every non-discarded photo + every folder and check whether
they still resolve on disk. Returns
(deletable_photo_ids, deletable_folder_ids, skipped_photo_ids).
Skipped rows are photos/folders whose owning source_root path itself
doesn't resolve — that's almost always an unmounted drive, and
silently deleting those rows would be data loss. The caller can
surface the skip count separately so the user knows the cleanup
wasn't a no-op by accident.
"""
sr_rows = (await session.execute(select(SourceRoot))).scalars().all()
sr_mounted: dict[str, bool] = {sr.id: os.path.isdir(sr.path) for sr in sr_rows}
photos = (await session.execute(
select(Photo.id, Photo.filepath, Photo.folder_id)
.where(Photo.is_discarded.is_(False))
)).all()
folders = (await session.execute(
select(Folder.id, Folder.path, Folder.source_root_id)
)).all()
folder_to_sr = {fid: srid for fid, _path, srid in folders}
deletable_photos: list[str] = []
skipped: list[str] = []
for pid, fp, folder_id in photos:
sr_id = folder_to_sr.get(folder_id)
if sr_id is None or not sr_mounted.get(sr_id, False):
skipped.append(pid)
continue
if not os.path.exists(fp):
deletable_photos.append(pid)
deletable_folders: list[str] = []
for fid, fpath, sr_id in folders:
if sr_id is None or not sr_mounted.get(sr_id, False):
continue
if not os.path.isdir(fpath):
deletable_folders.append(fid)
return deletable_photos, deletable_folders, skipped
async def prune_missing_photos(dry_run: bool = True) -> dict:
"""Delete photo + folder rows whose paths are no longer on disk *and*
whose source root is currently mounted. Common cause: PHOTO_DIRS in
.env was repointed at a different library, leaving every old row
orphaned.
Set dry_run=False to actually delete. The default is intentionally
safe so the matching count can be surfaced in the UI before the
user commits to it.
Function name kept for backwards compatibility — it now also prunes
folders, not just photos.
"""
from sqlalchemy import delete
async with AsyncSessionLocal() as session:
try:
deletable_photos, deletable_folders, skipped = await find_missing(session)
if not dry_run:
CHUNK = 500
# Photos first (folders may FK from them via folder_id).
for i in range(0, len(deletable_photos), CHUNK):
await session.execute(
delete(Photo).where(
Photo.id.in_(deletable_photos[i:i + CHUNK])
)
)
# Then drop folders that ALSO no longer have any photos
# pointing at them. We re-check after the photo delete so
# we don't strand a folder that legitimately exists on
# disk but happened to match the orphan list.
if deletable_folders:
for i in range(0, len(deletable_folders), CHUNK):
chunk = deletable_folders[i:i + CHUNK]
# Only delete folders that now have zero photos
# left attached (defensive — should always be 0
# if the path is gone, but a concurrent scan
# could re-create rows).
still_used = (await session.execute(
select(Photo.folder_id)
.where(Photo.folder_id.in_(chunk))
.distinct()
)).scalars().all()
safe = [f for f in chunk if f not in set(still_used)]
if safe:
await session.execute(
delete(Folder).where(Folder.id.in_(safe))
)
await session.commit()
logger.info(
f"Pruned {len(deletable_photos)} photo rows + "
f"{len(deletable_folders)} folder rows"
)
key_p = "would_delete" if dry_run else "deleted"
key_f = "would_delete_folders" if dry_run else "deleted_folders"
return {
key_p: len(deletable_photos),
key_f: len(deletable_folders),
"skipped_unmounted": len(skipped),
"dry_run": dry_run,
}
except Exception as e:
logger.error(f"prune_missing_photos failed: {e}")
await session.rollback()
raise
async def cleanup_data_integrity() -> dict:
"""Top-level entry point. Runs the dedupe + count refresh in a single
transaction. Returns a small summary dict for logging."""
async with AsyncSessionLocal() as session:
try:
sr_deleted = await _dedupe_source_roots(session)
f_deleted = await _dedupe_folders(session)
await _recompute_folder_counts(session)
stale = await _warn_stale_source_roots(session)
await session.commit()
summary = {
"source_roots_merged": sr_deleted,
"folders_merged": f_deleted,
"source_roots_stale": stale,
}
if sr_deleted or f_deleted:
logger.info(f"Cleanup merged duplicates: {summary}")
return summary
except Exception as e:
logger.error(f"Cleanup failed: {e}")
await session.rollback()
raise

View File

@@ -1,214 +0,0 @@
"""
Folder/filename-based date guessing and "taken_at looks wrong" detection.
Direct Python port of `frontend/src/lib/guessDateFromPath.ts` — the logic
must stay in sync because the frontend renders the suggestion hint in the
info panel while the backend owns the `has_date_warning` flag that the
filter bar queries. Any heuristic change has to be applied to both files.
The guesser walks a filepath, tries the filename first as the source of
truth, then falls back to folder segments (deepest first) and multi-
segment layouts. Returns ``None`` when no recognisable date can be
extracted. `has_date_warning()` compares the guess to a stored `taken_at`
and reports whether the difference is large enough to flag.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from datetime import datetime
from typing import Literal, Optional
Confidence = Literal["high", "medium", "low"]
Source = Literal["folder", "filename"]
@dataclass(frozen=True)
class DateGuess:
date: datetime
confidence: Confidence
matched: str
source: Source
_MIN_YEAR = 1970
# Bump the ceiling annually via `datetime.now()` rather than a literal so
# we don't ship a time bomb. `+1` allows near-future timestamps (cameras
# with a slightly advanced clock at year end) without opening the door to
# 4-digit serial numbers that happen to start with "30xx".
def _max_year() -> int:
return datetime.now().year + 1
def _valid_year(y: int) -> bool:
return _MIN_YEAR <= y <= _max_year()
def _make_date(y: int, m: int, d: int) -> Optional[datetime]:
if not _valid_year(y):
return None
if not (1 <= m <= 12):
return None
if not (1 <= d <= 31):
return None
try:
# Noon local so downstream day-bucketing is stable across timezone
# rounding. The frontend mirrors this.
return datetime(y, m, d, 12, 0, 0)
except ValueError:
return None
def _segments(filepath: str) -> list[str]:
return [s for s in re.split(r"[\\/]+", filepath) if s]
_COMPACT_RE = re.compile(r"(?<!\d)(\d{4})(\d{2})(\d{2})(?!\d)")
_DASHED_RE = re.compile(r"(?<!\d)(\d{4})[-_.](\d{1,2})[-_.](\d{1,2})(?!\d)")
_MONTH_RE = re.compile(r"(?<!\d)(\d{4})[-_.](\d{1,2})(?!\d)")
_YEAR_RE = re.compile(r"(?<!\d)(\d{4})(?!\d)")
_FOUR_DIGITS = re.compile(r"^\d{4}$")
_ONE_OR_TWO = re.compile(r"^\d{1,2}$")
def _guess_from_string(
input: str,
source: Source,
allow_year_only: bool,
) -> Optional[DateGuess]:
if not input:
return None
m = _COMPACT_RE.search(input)
if m:
d = _make_date(int(m.group(1)), int(m.group(2)), int(m.group(3)))
if d:
return DateGuess(
date=d,
confidence="high",
matched=f"{m.group(1)}-{m.group(2)}-{m.group(3)}",
source=source,
)
m = _DASHED_RE.search(input)
if m:
d = _make_date(int(m.group(1)), int(m.group(2)), int(m.group(3)))
if d:
return DateGuess(
date=d,
confidence="high",
matched=f"{m.group(1)}-{m.group(2)}-{m.group(3)}",
source=source,
)
m = _MONTH_RE.search(input)
if m:
d = _make_date(int(m.group(1)), int(m.group(2)), 15)
if d:
return DateGuess(
date=d,
confidence="medium",
matched=f"{m.group(1)}-{m.group(2)}",
source=source,
)
if allow_year_only:
m = _YEAR_RE.search(input)
if m:
d = _make_date(int(m.group(1)), 7, 1)
if d:
return DateGuess(
date=d,
confidence="low",
matched=m.group(1),
source=source,
)
return None
def _guess_from_folder_layout(folders: list[str]) -> Optional[DateGuess]:
# YYYY / MM / DD
for i in range(len(folders) - 2):
a, b, c = folders[i], folders[i + 1], folders[i + 2]
if _FOUR_DIGITS.match(a) and _ONE_OR_TWO.match(b) and _ONE_OR_TWO.match(c):
d = _make_date(int(a), int(b), int(c))
if d:
return DateGuess(
date=d,
confidence="high",
matched=f"{a}/{b}/{c}",
source="folder",
)
# YYYY / MM
for i in range(len(folders) - 1):
a, b = folders[i], folders[i + 1]
if _FOUR_DIGITS.match(a) and _ONE_OR_TWO.match(b):
d = _make_date(int(a), int(b), 15)
if d:
return DateGuess(
date=d,
confidence="medium",
matched=f"{a}/{b}",
source="folder",
)
return None
_CONFIDENCE_RANK: dict[Confidence, int] = {"high": 3, "medium": 2, "low": 1}
def guess_date_from_path(filepath: str) -> Optional[DateGuess]:
"""Filename wins when it has any viable match; otherwise walk folder
segments deepest-first and pick the strongest hit."""
if not filepath:
return None
segs = _segments(filepath)
if not segs:
return None
filename = segs[-1]
folders = segs[:-1]
from_filename = _guess_from_string(filename, "filename", allow_year_only=False)
if from_filename:
return from_filename
best: Optional[DateGuess] = None
for seg in reversed(folders):
hit = _guess_from_string(seg, "folder", allow_year_only=True)
if not hit:
continue
if not best or _CONFIDENCE_RANK[hit.confidence] > _CONFIDENCE_RANK[best.confidence]:
best = hit
if hit.confidence == "high":
break
from_layout = _guess_from_folder_layout(folders)
if from_layout and (
not best or _CONFIDENCE_RANK[from_layout.confidence] > _CONFIDENCE_RANK[best.confidence]
):
best = from_layout
return best
_ONE_DAY = 24 * 60 * 60
def has_date_warning(filepath: str, taken_at: Optional[datetime]) -> bool:
"""True when the path-based guess disagrees with ``taken_at`` by more
than 24h, or when ``taken_at`` is missing and the path would supply
one. This is the authoritative flag stored on `photos.has_date_warning`
and queried by the timeline filter."""
guess = guess_date_from_path(filepath)
if not guess:
return False
if taken_at is None:
return True
try:
diff = abs((taken_at - guess.date).total_seconds())
except (TypeError, ValueError):
return False
return diff > _ONE_DAY

View File

@@ -1,298 +0,0 @@
"""
Duplicate detection: group photos by perceptual-hash + CLIP similarity.
Strategy
--------
Two complementary signals are fused into a single grouping:
1. **Perceptual hash (pHash)** — 16-char hex hash from the thumbnail
worker. Catches byte-identical copies and mild re-encodes via
Hamming distance (threshold ≤ 6 bits out of 64).
2. **CLIP embedding similarity** — cosine distance over 512-d vectors
stored in the `embeddings` table with an HNSW index. Catches
visually similar photos even when pHash diverges (e.g. crops,
different formats, screenshots of the same content).
Both signals feed a union-find structure that merges overlapping matches
into connected components.
Incremental mode (default post-scan)
-------------------------------------
`incremental_regroup` only compares *newly added* photos (those whose
`added_at` > watermark) against the entire library. Each new photo does:
- An HNSW vector similarity query: O(log N) via the index.
- A pHash comparison against a small candidate set (same group members
or nearby CLIP results) rather than the full N² sweep.
This makes the post-scan cost O(new × log N) instead of O(N²).
Full regroup
------------
`regroup_duplicates` still performs the full pairwise pHash pass +
CLIP sweep, used for initial setup and manual re-detection.
"""
from __future__ import annotations
import logging
import uuid
from datetime import datetime, timezone
from typing import Optional
from sqlalchemy import select, update
from app.database import AsyncSessionLocal
from app.models.photos import Photo
logger = logging.getLogger(__name__)
# pHash Hamming distance threshold (6 out of 64 bits).
DEFAULT_PHASH_THRESHOLD = 6
def _hex_to_int(h: str) -> int:
"""Parse a 16-char hex pHash to a Python int. Returns -1 on bad input
so the pairwise loop can skip the row without raising."""
try:
return int(h, 16)
except (TypeError, ValueError):
return -1
def _hamming(a: int, b: int) -> int:
"""Population count of XOR — the canonical hash distance metric."""
x = a ^ b
try:
return x.bit_count() # type: ignore[attr-defined]
except AttributeError:
return bin(x).count('1')
class _UnionFind:
"""Tiny union-find / disjoint-set used to merge similar photos into
connected components."""
def __init__(self, keys: list[str]) -> None:
self._index = {k: i for i, k in enumerate(keys)}
n = len(keys)
self.parent = list(range(n))
self.rank = [0] * n
def find(self, x: int) -> int:
while self.parent[x] != x:
self.parent[x] = self.parent[self.parent[x]]
x = self.parent[x]
return x
def union_by_key(self, key_a: str, key_b: str) -> None:
ia, ib = self._index.get(key_a), self._index.get(key_b)
if ia is None or ib is None:
return
ra, rb = self.find(ia), self.find(ib)
if ra == rb:
return
if self.rank[ra] < self.rank[rb]:
ra, rb = rb, ra
self.parent[rb] = ra
if self.rank[ra] == self.rank[rb]:
self.rank[ra] += 1
def components(self, keys: list[str]) -> dict[int, list[str]]:
"""Return {root_idx: [photo_ids...]} for groups of size >= 2."""
groups: dict[int, list[str]] = {}
for key in keys:
idx = self._index[key]
root = self.find(idx)
groups.setdefault(root, []).append(key)
return {r: members for r, members in groups.items() if len(members) >= 2}
async def regroup_duplicates(
phash_threshold: int = DEFAULT_PHASH_THRESHOLD,
**_ignored,
) -> dict:
"""Full recompute of duplicate groups using pHash similarity.
Idempotent — safe to call as often as you like. Returns a summary dict.
"""
async with AsyncSessionLocal() as session:
# Pull all visible photos with a phash or embedding.
rows = (
await session.execute(
select(Photo.id, Photo.phash)
.where(Photo.is_discarded.is_(False))
.where(Photo.is_hidden.is_(False))
)
).all()
if not rows:
await _clear_all_groups(session)
await session.commit()
return {'photos_considered': 0, 'groups': 0, 'members': 0}
ids = [row[0] for row in rows]
phash_map = {row[0]: _hex_to_int(row[1]) for row in rows if row[1]}
uf = _UnionFind(ids)
# ── Phase 1: pHash pairwise (O(N²) on photos with phash) ──
phash_ids = [pid for pid in ids if pid in phash_map]
phash_vals = [phash_map[pid] for pid in phash_ids]
n = len(phash_ids)
for i in range(n):
hi = phash_vals[i]
if hi < 0:
continue
for j in range(i + 1, n):
hj = phash_vals[j]
if hj < 0:
continue
if _hamming(hi, hj) <= phash_threshold:
uf.union_by_key(phash_ids[i], phash_ids[j])
# ── Write results ──
await _clear_all_groups(session)
groups = uf.components(ids)
groups_created = 0
members_total = 0
for member_ids in groups.values():
group_id = str(uuid.uuid4())
await session.execute(
update(Photo)
.where(Photo.id.in_(member_ids))
.values(duplicate_group_id=group_id, is_duplicate=True)
)
groups_created += 1
members_total += len(member_ids)
await session.commit()
logger.info(
f"regroup_duplicates: {len(ids)} photos, "
f"{groups_created} group(s), {members_total} member(s)"
)
return {
'photos_considered': len(ids),
'groups': groups_created,
'members': members_total,
}
async def incremental_regroup(
since: Optional[datetime] = None,
phash_threshold: int = DEFAULT_PHASH_THRESHOLD,
**_ignored,
) -> dict:
"""Incremental duplicate detection for newly added photos using pHash."""
async with AsyncSessionLocal() as session:
# If no watermark, fall back to full regroup.
if since is None:
# Find the most recent scan start by looking at the newest
# photo that already has a duplicate_group_id check completed.
# As a simple heuristic, use photos added in the last hour.
from datetime import timedelta
since = datetime.now(timezone.utc) - timedelta(hours=1)
# Photo.added_at is stored as TIMESTAMP WITHOUT TIME ZONE, so
# asyncpg rejects aware datetimes with "can't subtract offset-naive
# and offset-aware". Normalise: if `since` has a tzinfo, convert
# it to UTC and drop the tzinfo so the bind parameter is naive.
if since.tzinfo is not None:
since = since.astimezone(timezone.utc).replace(tzinfo=None)
# Get newly added photos (the "new" set).
new_rows = (
await session.execute(
select(Photo.id, Photo.phash)
.where(Photo.added_at >= since)
.where(Photo.is_discarded.is_(False))
.where(Photo.is_hidden.is_(False))
)
).all()
if not new_rows:
return {'photos_considered': 0, 'new_photos': 0, 'groups_updated': 0, 'members_added': 0}
new_ids = [r[0] for r in new_rows]
new_phash = {r[0]: _hex_to_int(r[1]) for r in new_rows if r[1]}
# Get ALL existing photos for union-find (we need to merge into
# existing groups).
all_rows = (
await session.execute(
select(Photo.id, Photo.phash, Photo.duplicate_group_id)
.where(Photo.is_discarded.is_(False))
.where(Photo.is_hidden.is_(False))
)
).all()
all_ids = [r[0] for r in all_rows]
all_phash = {r[0]: _hex_to_int(r[1]) for r in all_rows if r[1]}
existing_groups: dict[str, str] = {
r[0]: r[2] for r in all_rows if r[2]
}
uf = _UnionFind(all_ids)
# Pre-seed existing groups into the union-find so we merge into
# them rather than creating parallel groups.
group_to_members: dict[str, list[str]] = {}
for pid, gid in existing_groups.items():
group_to_members.setdefault(gid, []).append(pid)
for members in group_to_members.values():
for i in range(1, len(members)):
uf.union_by_key(members[0], members[i])
# ── Phase 1: pHash — compare each new photo against ALL photos ──
for new_id in new_ids:
nh = new_phash.get(new_id, -1)
if nh < 0:
continue
for existing_id, eh in all_phash.items():
if existing_id == new_id or eh < 0:
continue
if _hamming(nh, eh) <= phash_threshold:
uf.union_by_key(new_id, existing_id)
# ── Write results ──
# Only update groups that contain at least one new photo.
# Clear all groups first, then rewrite.
await _clear_all_groups(session)
groups = uf.components(all_ids)
groups_created = 0
members_total = 0
new_in_groups = 0
for member_ids in groups.values():
group_id = str(uuid.uuid4())
await session.execute(
update(Photo)
.where(Photo.id.in_(member_ids))
.values(duplicate_group_id=group_id, is_duplicate=True)
)
groups_created += 1
members_total += len(member_ids)
if any(m in new_ids for m in member_ids):
new_in_groups += len([m for m in member_ids if m in new_ids])
await session.commit()
logger.info(
f"incremental_regroup: {len(new_ids)} new photos, "
f"{groups_created} group(s), {new_in_groups} new member(s) grouped"
)
return {
'photos_considered': len(all_ids),
'new_photos': len(new_ids),
'groups_updated': groups_created,
'members_added': new_in_groups,
}
async def _clear_all_groups(session) -> None:
"""Reset duplicate_group_id / is_duplicate on every photo."""
await session.execute(
update(Photo).values(duplicate_group_id=None, is_duplicate=False)
)

View File

@@ -1,72 +0,0 @@
"""
EXIF write-back helpers.
The rest of the app reads EXIF at scan time and stashes the result in Postgres
(see `services/metadata.py`). This module handles the reverse direction: when
the user corrects a date in the UI we also rewrite the relevant EXIF tags on
disk so a later rescan won't clobber the fix and external tools see the same
truth the DB does.
"""
import asyncio
import logging
import subprocess
from datetime import datetime
from pathlib import Path
logger = logging.getLogger(__name__)
EXIFTOOL_TIMEOUT_SECONDS = 30
class ExifWriteError(RuntimeError):
"""Raised when exiftool fails to write tags to a file."""
def _format_exif_dt(dt: datetime) -> str:
return dt.strftime("%Y:%m:%d %H:%M:%S")
async def write_taken_at(filepath: str, dt: datetime) -> None:
"""Rewrite DateTimeOriginal / CreateDate / ModifyDate on the file.
- ``-overwrite_original`` so we don't litter the library with
``<name>_original`` sidecars.
- ``-P`` preserves the file's mtime so the scanner's mtime-based
change detection stays quiet.
- We set all three common date tags together because different viewers
read different ones; keeping them in lockstep avoids confusing
downstream tools and our own re-extraction pass.
"""
if not Path(filepath).exists():
raise ExifWriteError(f"File not found: {filepath}")
stamp = _format_exif_dt(dt)
cmd = [
"exiftool",
"-overwrite_original",
"-P",
f"-DateTimeOriginal={stamp}",
f"-CreateDate={stamp}",
f"-ModifyDate={stamp}",
filepath,
]
def _run() -> subprocess.CompletedProcess:
return subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=EXIFTOOL_TIMEOUT_SECONDS,
)
try:
result = await asyncio.to_thread(_run)
except subprocess.TimeoutExpired as exc:
raise ExifWriteError(f"exiftool timed out writing {filepath}") from exc
except FileNotFoundError as exc:
raise ExifWriteError("exiftool binary not available") from exc
if result.returncode != 0:
msg = (result.stderr or result.stdout or "unknown error").strip()
logger.warning("exiftool write failed for %s: %s", filepath, msg)
raise ExifWriteError(msg)

View File

@@ -1,137 +0,0 @@
"""
Runtime feature flags for the vision pipeline.
Only one flag now — the master vision switch. Runtime overrides live in
Redis under ``mulita:flags:<name>``; an unset key falls back to the
YAML default.
"""
from __future__ import annotations
import logging
from typing import Optional
import redis
from app.config import settings
logger = logging.getLogger(__name__)
FLAG_VISION_ENABLED = 'vision.enabled'
ALL_FLAGS = (FLAG_VISION_ENABLED,)
_VISION_QUEUE = 'vision'
_REDIS: Optional[redis.Redis] = None
def _redis() -> Optional[redis.Redis]:
global _REDIS
if _REDIS is None:
try:
_REDIS = redis.Redis.from_url(
settings.celery_broker_url, decode_responses=True
)
_REDIS.ping()
except Exception as e:
logger.warning(f"feature_flags: Redis unavailable, using YAML defaults ({e})")
_REDIS = None
return _REDIS
def _yaml_default(name: str) -> bool:
if name == FLAG_VISION_ENABLED:
return bool(settings.vision.enabled)
raise ValueError(f"Unknown feature flag: {name!r}")
def _redis_key(name: str) -> str:
return f"mulita:flags:{name}"
def is_enabled(name: str) -> bool:
r = _redis()
if r is not None:
try:
raw = r.get(_redis_key(name))
if raw is not None:
return raw.lower() == 'true'
except Exception as e:
logger.warning(f"feature_flags: Redis read failed for {name} ({e})")
return _yaml_default(name)
def set_flag(name: str, value: bool) -> None:
if name not in ALL_FLAGS:
raise ValueError(f"Unknown feature flag: {name!r}")
r = _redis()
if r is None:
raise RuntimeError("Redis unavailable; cannot update feature flags")
r.set(_redis_key(name), 'true' if value else 'false')
_apply_worker_side_effects(name)
def reset_flag(name: str) -> None:
if name not in ALL_FLAGS:
raise ValueError(f"Unknown feature flag: {name!r}")
r = _redis()
if r is None:
raise RuntimeError("Redis unavailable; cannot reset feature flags")
r.delete(_redis_key(name))
_apply_worker_side_effects(name)
def _apply_worker_side_effects(name: str) -> None:
"""Attach or detach the vision consumer and purge queued work when
the master flag flips. Best-effort — state is already persisted."""
if name != FLAG_VISION_ENABLED:
return
try:
from app.tasks.celery import celery_app
except Exception as e:
logger.warning(f"feature_flags: celery app unavailable for side effects ({e})")
return
try:
if is_enabled(FLAG_VISION_ENABLED):
celery_app.control.add_consumer(_VISION_QUEUE, reply=False)
logger.info("feature_flags: vision re-enabled; consumer added")
else:
celery_app.control.cancel_consumer(_VISION_QUEUE, reply=False)
_purge_queue(_VISION_QUEUE)
logger.info("feature_flags: vision disabled; consumer cancelled and queue purged")
except Exception as e:
logger.warning(f"feature_flags: worker side effects failed: {e}")
def _purge_queue(queue: str) -> int:
r = _redis()
if r is None:
return 0
try:
return int(r.delete(queue) or 0)
except Exception as e:
logger.warning(f"feature_flags: purge {queue} failed: {e}")
return 0
def snapshot() -> dict[str, dict[str, object]]:
r = _redis()
out: dict[str, dict[str, object]] = {}
for name in ALL_FLAGS:
default = _yaml_default(name)
override = None
if r is not None:
try:
raw = r.get(_redis_key(name))
if raw is not None:
override = raw.lower() == 'true'
except Exception:
pass
out[name] = {
'effective': override if override is not None else default,
'default': default,
'overridden': override is not None,
}
return out

View File

@@ -1,291 +0,0 @@
"""
Metadata extraction service using ExifTool
"""
import json
import logging
import re
import asyncio
from datetime import datetime
from typing import Dict, Optional
import subprocess
from pathlib import Path
from celery import shared_task
from sqlalchemy import select
from app.database import AsyncSessionLocal
from app.models import Photo
from app.services.date_guess import has_date_warning
logger = logging.getLogger(__name__)
def parse_exif_datetime(date_str: str) -> Optional[datetime]:
"""Parse EXIF datetime string to Python datetime"""
if not date_str:
return None
# Common EXIF datetime formats
formats = [
"%Y:%m:%d %H:%M:%S",
"%Y-%m-%d %H:%M:%S",
"%Y:%m:%d %H:%M:%S.%f",
"%Y-%m-%dT%H:%M:%S",
"%Y-%m-%dT%H:%M:%S.%f",
"%Y-%m-%dT%H:%M:%S%z"
]
for fmt in formats:
try:
return datetime.strptime(date_str, fmt)
except ValueError:
continue
return None
_DMS_RE = re.compile(
r"""\s*
(?P<deg>-?\d+(?:\.\d+)?)\s*(?:deg|°|d)?\s*
(?:(?P<min>\d+(?:\.\d+)?)\s*[\'m]?\s*)?
(?:(?P<sec>\d+(?:\.\d+)?)\s*[\"”s]?\s*)?
(?P<ref>[NSEW])?\s*$""",
re.IGNORECASE | re.VERBOSE,
)
def _parse_coord(value, ref: str | None) -> float | None:
"""Coerce a single GPS coordinate from any form ExifTool may emit.
ExifTool's ``-j`` JSON output applies print conversion by default, so
coordinates can come back as:
* a number (``48.1278``) — happens for some sources / when ``-n`` is set
* a plain DMS string (``"48 deg 7' 39.96\\""``) — bare ``EXIF:GPSLatitude``
* a DMS-with-ref string (``"48 deg 7' 39.96\\" N"``) — ``Composite:GPSLatitude``
The optional ``ref`` argument lets the caller pass an explicit
``GPSLatitudeRef`` / ``GPSLongitudeRef`` ('N'/'S'/'E'/'W') when the
string itself doesn't carry one. Returns signed decimal degrees, or
``None`` if the value is unparseable.
"""
if value is None:
return None
# Numeric path — already decimal degrees, possibly already signed.
if isinstance(value, (int, float)):
out = float(value)
else:
m = _DMS_RE.match(str(value))
if not m:
return None
deg = float(m.group('deg'))
minutes = float(m.group('min') or 0)
seconds = float(m.group('sec') or 0)
out = abs(deg) + minutes / 60.0 + seconds / 3600.0
if deg < 0:
out = -out
embedded_ref = m.group('ref')
if embedded_ref:
ref = embedded_ref
if ref:
r = ref[0].upper()
if r in ('S', 'W'):
out = -abs(out)
elif r in ('N', 'E'):
out = abs(out)
return out
def extract_gps(exif_data: Dict) -> tuple:
"""Return (lat, lon) in signed decimal degrees, or (None, None).
With ``exiftool -G -j`` GPS values are keyed under their group.
``Composite:GPSLatitude`` / ``Composite:GPSLongitude`` carry the
hemisphere reference inline (``"48 deg 7' 39.96\\" N"``) while the bare
``EXIF:GPSLatitude`` / ``EXIF:GPSLongitude`` need the separate
``EXIF:GPSLatitudeRef`` / ``EXIF:GPSLongitudeRef`` to know the sign.
Pre-fix this function read the *unprefixed* keys ``GPSLatitude`` /
``GPSLongitude`` (which never exist in ``-G`` output) AND assumed
they were already floats — so it silently dropped every photo's GPS.
"""
lat = _parse_coord(exif_data.get('Composite:GPSLatitude'), None)
lon = _parse_coord(exif_data.get('Composite:GPSLongitude'), None)
if lat is None or lon is None:
lat = _parse_coord(
exif_data.get('EXIF:GPSLatitude'),
exif_data.get('EXIF:GPSLatitudeRef'),
)
lon = _parse_coord(
exif_data.get('EXIF:GPSLongitude'),
exif_data.get('EXIF:GPSLongitudeRef'),
)
if lat is None or lon is None:
return None, None
if not (-90 <= lat <= 90 and -180 <= lon <= 180):
return None, None
# Some cameras emit (0, 0) when they have no GPS lock — treat as missing
if lat == 0 and lon == 0:
return None, None
return lat, lon
def extract_key_metadata(exif_data: Dict) -> Dict:
"""Extract key metadata fields for FTS indexing"""
key_fields = []
# Camera information
if 'EXIF:Make' in exif_data:
key_fields.append(exif_data['EXIF:Make'])
if 'EXIF:Model' in exif_data:
key_fields.append(exif_data['EXIF:Model'])
if 'EXIF:LensModel' in exif_data:
key_fields.append(exif_data['EXIF:LensModel'])
# Location information
lat, lon = extract_gps(exif_data)
if lat is not None and lon is not None:
key_fields.append(f"GPS: {lat}, {lon}")
# IPTC/XMP keywords
keywords = exif_data.get('IPTC:Keywords') or exif_data.get('XMP:Subject')
if keywords:
if isinstance(keywords, list):
key_fields.extend(keywords)
else:
key_fields.append(keywords)
# Copyright and creator
if 'EXIF:Copyright' in exif_data:
key_fields.append(exif_data['EXIF:Copyright'])
if 'XMP:Creator' in exif_data:
key_fields.append(exif_data['XMP:Creator'])
if 'EXIF:Artist' in exif_data:
key_fields.append(exif_data['EXIF:Artist'])
return {
'exif_text': ' '.join(str(f) for f in key_fields),
'camera_make': exif_data.get('EXIF:Make'),
'camera_model': exif_data.get('EXIF:Model'),
'lens_model': exif_data.get('EXIF:LensModel'),
'gps_latitude': lat,
'gps_longitude': lon,
}
@shared_task(name='extract_metadata')
def extract_metadata(photo_id: str):
"""Extract metadata from a photo using ExifTool"""
return asyncio.run(_extract_metadata_async(photo_id))
async def _extract_metadata_async(photo_id: str):
"""Async implementation of metadata extraction"""
async with AsyncSessionLocal() as session:
try:
# Get photo from database
result = await session.execute(
select(Photo).where(Photo.id == photo_id)
)
photo = result.scalar_one_or_none()
if not photo:
logger.error(f"Photo not found: {photo_id}")
return {'status': 'error', 'message': 'Photo not found'}
# Check if file exists
if not Path(photo.filepath).exists():
logger.error(f"File not found: {photo.filepath}")
return {'status': 'error', 'message': 'File not found'}
# Run ExifTool to extract metadata
cmd = [
'exiftool',
'-j', # JSON output
'-G', # Group names
'-s', # Short output format
'-All', # All metadata
photo.filepath
]
try:
result = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=30,
stdin=subprocess.DEVNULL,
)
if result.returncode != 0:
logger.error(f"ExifTool error: {result.stderr}")
photo.processing_error = f"ExifTool: {result.stderr[:500]}"
await session.commit()
return {'status': 'error', 'message': result.stderr}
# Parse JSON output
metadata = json.loads(result.stdout)
if metadata and len(metadata) > 0:
exif_data = metadata[0]
# Store full metadata as JSON
photo.exif_json = json.dumps(exif_data)
# Extract taken_at date
date_fields = [
'EXIF:DateTimeOriginal',
'EXIF:CreateDate',
'QuickTime:MediaCreateDate',
'EXIF:ModifyDate'
]
for field in date_fields:
if field in exif_data:
taken_at = parse_exif_datetime(exif_data[field])
if taken_at:
photo.taken_at = taken_at
photo.taken_at_source = 'exif'
break
# Re-run the path-vs-date heuristic now that we know
# whether EXIF provided a real capture date. A true EXIF
# date that matches the folder clears the warning the
# scanner set during the filesystem-mtime pass.
photo.has_date_warning = has_date_warning(
photo.filepath, photo.taken_at
)
# Extract dimensions if not already set
if not photo.width:
photo.width = exif_data.get('EXIF:ImageWidth') or exif_data.get('File:ImageWidth')
if not photo.height:
photo.height = exif_data.get('EXIF:ImageHeight') or exif_data.get('File:ImageHeight')
# Extract GPS coordinates into first-class columns so the
# Map view can query them without parsing exif_json.
lat, lon = extract_gps(exif_data)
photo.latitude = lat
photo.longitude = lon
# Extract and store key metadata for search
key_metadata = extract_key_metadata(exif_data)
await session.commit()
logger.info(f"Metadata extracted for photo {photo_id}")
return {
'status': 'success',
'photo_id': photo_id,
'taken_at': photo.taken_at.isoformat() if photo.taken_at else None
}
except subprocess.TimeoutExpired:
logger.error(f"ExifTool timeout for {photo.filepath}")
photo.processing_error = 'ExifTool timeout'
await session.commit()
return {'status': 'error', 'message': 'ExifTool timeout'}
except json.JSONDecodeError as e:
logger.error(f"Failed to parse ExifTool output: {e}")
photo.processing_error = f"Invalid ExifTool output: {e}"
await session.commit()
return {'status': 'error', 'message': 'Invalid ExifTool output'}
except Exception as e:
logger.error(f"Error extracting metadata for {photo_id}: {e}")
return {'status': 'error', 'message': str(e)}

View File

@@ -1,103 +0,0 @@
"""
Scanner service for initial library scan and per-user source root bootstrap.
"""
import os
import logging
from sqlalchemy import select
from app.database import AsyncSessionLocal
from app.models import SourceRoot
from app.models.user import User
from app.tasks.scan import scan_all_source_roots
from app.config import settings
logger = logging.getLogger(__name__)
async def bootstrap_user_source_root(user: User, session=None) -> None:
"""Create the media directory and a source root for a user.
Called when a new user is created (by the admin or the setup endpoint).
If the user already has a source root, this is a no-op.
"""
own_session = session is None
if own_session:
session = AsyncSessionLocal()
try:
# Check if user already has a source root
result = await session.execute(
select(SourceRoot).where(SourceRoot.user_id == user.id)
)
if result.scalar_one_or_none() is not None:
return
os.makedirs(user.media_path, exist_ok=True)
source_root = SourceRoot(
name=f"{user.username}'s Library",
path=user.media_path,
user_id=user.id,
)
session.add(source_root)
if own_session:
await session.commit()
else:
await session.flush()
logger.info(
f"Bootstrapped source root for user '{user.username}': "
f"{user.media_path}"
)
finally:
if own_session:
await session.close()
async def bootstrap_default_source_root() -> None:
"""Legacy bootstrap — for existing installs that have source roots
without user_id (pre-auth migration). On fresh installs, source roots
are created per-user via bootstrap_user_source_root. If there are
already source roots in the DB, this is a no-op.
"""
async with AsyncSessionLocal() as session:
result = await session.execute(select(SourceRoot))
if result.scalars().first() is not None:
return # Already have source roots.
# No source roots and no users means fresh install — the setup
# endpoint will create the first user + source root.
user_count = (await session.execute(
select(User)
)).scalars().first()
if user_count is None:
logger.info(
"No users or source roots — waiting for first-run setup."
)
return
async def start_initial_scan():
"""Start the initial library scan and optionally the file watcher.
The file watcher uses a Redis lock to ensure only one instance runs
across all workers, so it's safe to dispatch on every startup — only
the first one will actually watch, the rest exit immediately.
"""
try:
scan_all_source_roots.delay()
logger.info("Initial scan queued successfully")
except Exception as e:
logger.error(f"Failed to start initial scan: {e}")
# Start the file watcher if enabled in config.
from app.config import settings
if settings.scanner.watch:
try:
from app.tasks.scan import watch_folders
# Countdown gives the initial scan time to register source roots
# before the watcher tries to load them.
watch_folders.apply_async(countdown=10)
logger.info("File watcher queued (Redis-locked, single instance)")
except Exception as e:
logger.warning(f"Could not queue file watcher: {e}")

View File

@@ -1,77 +0,0 @@
"""
FTS search over photos.search_vector with optional tag/date filters.
"""
import logging
from typing import Optional
from sqlalchemy import select, text
from sqlalchemy.ext.asyncio import AsyncSession
from app.models import Photo
logger = logging.getLogger(__name__)
async def hybrid_search(
db: AsyncSession,
q: Optional[str] = None,
tag_ids: Optional[list[str]] = None,
date_from: Optional[str] = None,
date_to: Optional[str] = None,
limit: int = 50,
offset: int = 0,
) -> list[dict]:
"""Full-text search using photos.search_vector. No embeddings, no OCR."""
if q:
try:
fts_stmt = text("""
SELECT id, ts_rank(search_vector, plainto_tsquery('english', :q)) AS rank
FROM photos
WHERE search_vector @@ plainto_tsquery('english', :q)
AND is_trashed = false
AND is_hidden = false
ORDER BY rank DESC
LIMIT 500
""")
rows = (await db.execute(fts_stmt, {"q": q})).fetchall()
except Exception as e:
logger.warning("FTS search failed: %s", e)
rows = []
scored = [(pid, float(rank)) for pid, rank in rows]
if tag_ids:
from app.models.tags import photo_tags
photo_ids = [pid for pid, _ in scored]
if not photo_ids:
return []
stmt = select(photo_tags.c.photo_id).where(
photo_tags.c.photo_id.in_(photo_ids),
photo_tags.c.tag_id.in_(tag_ids),
).distinct()
valid = {row[0] for row in (await db.execute(stmt)).fetchall()}
scored = [(pid, s) for pid, s in scored if pid in valid]
page = scored[offset : offset + limit]
return [{"photo_id": pid, "score": s} for pid, s in page]
# No text query — recent photos with tag/date filters.
if tag_ids:
from app.models.tags import photo_tags
sub = select(photo_tags.c.photo_id).where(
photo_tags.c.tag_id.in_(tag_ids)
).distinct().subquery()
stmt = select(Photo.id).join(sub, Photo.id == sub.c.photo_id)
else:
stmt = select(Photo.id)
stmt = stmt.where(
Photo.is_discarded.is_(False),
Photo.is_hidden.is_(False),
)
if date_from:
stmt = stmt.where(Photo.taken_at >= date_from)
if date_to:
stmt = stmt.where(Photo.taken_at <= date_to)
stmt = stmt.order_by(Photo.added_at.desc()).offset(offset).limit(limit)
rows = (await db.execute(stmt)).fetchall()
return [{"photo_id": row[0], "score": 0.0} for row in rows]

View File

@@ -1,7 +0,0 @@
"""
Vision pipeline services — embedding, OCR, object detection, face recognition.
All inference is done through the ModelRegistry singleton, which lazy-loads
ONNX Runtime sessions on first use and caches them for the lifetime of the
worker process.
"""

View File

@@ -1,25 +0,0 @@
"""
Abstract base classes for the vision backend.
The pipeline is now a single binary classifier: photography vs other.
Feature extraction is an internal detail of the classifier and is not
exposed as a separate service.
"""
from abc import ABC, abstractmethod
from dataclasses import dataclass
import numpy as np
@dataclass
class ClassificationResult:
label: str
confidence: float
class ContentClassifier(ABC):
"""Classifies an image into 'photography' or 'other'."""
@abstractmethod
def classify(self, image: np.ndarray) -> ClassificationResult:
...

View File

@@ -1,51 +0,0 @@
"""
Ensure the OpenCLIP ViT-B/32 visual encoder is present on worker boot.
Exported via export_models.py if missing.
"""
import logging
from pathlib import Path
from app.config import settings
logger = logging.getLogger(__name__)
REQUIRED = [
("embed/visual.onnx", "OpenCLIP ViT-B/32 visual encoder"),
]
def bootstrap(models_dir: str | None = None):
base = Path(models_dir or settings.vision.models_dir)
base.mkdir(parents=True, exist_ok=True)
missing = [(rel, desc) for rel, desc in REQUIRED if not (base / rel).exists()]
if missing:
logger.warning("Missing %d model file(s); attempting automatic export", len(missing))
try:
from app.services.vision import export_models
export_models.export_openclip_visual(base)
except Exception as e:
logger.error(
"Export failed: %s. Run `python -m app.services.vision.export_models "
"--models-dir %s` manually to retry.",
e, base,
)
still_missing = [(r, d) for r, d in REQUIRED if not (base / r).exists()]
if still_missing:
for rel, desc in still_missing:
logger.error(" still missing: %s%s", base / rel, desc)
else:
logger.info("All model files present in %s", base)
try:
import redis as _redis
_redis.from_url(settings.redis_url).set("mulita:vision:ready", "1")
logger.info("Set mulita:vision:ready in Redis")
except Exception as e:
logger.warning("Could not set vision readiness flag in Redis: %s", e)
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO)
bootstrap()

View File

@@ -1,106 +0,0 @@
"""
Binary content classifier: 'photography' vs 'other'.
Uses OpenCLIP ViT-B/32 image features (ONNX) and two pre-computed text
prompt centroids. Text centroids are computed once with the native
open_clip text encoder and cached to {models_dir}/classifier/vectors.npz
so steady-state worker startup doesn't pay the PyTorch cost.
"""
from __future__ import annotations
import logging
from pathlib import Path
import numpy as np
from app.config import VisionSettings
from app.services.vision.base import ClassificationResult, ContentClassifier
from app.services.vision.embed import CLIPVisualEncoder
logger = logging.getLogger(__name__)
PROMPTS = {
"photography": [
"a photograph taken with a camera",
"a real photo of a real scene or person",
"a candid photograph",
"a portrait photograph",
"a landscape photograph",
],
"other": [
"a screenshot of a computer screen",
"a screenshot of a phone screen",
"a screen capture of a user interface",
"a scanned document",
"a photo of a document with printed text",
"a photo of a receipt",
"a photo of a bill or invoice",
"an internet meme with text overlay",
"a funny image with caption text",
"a digital illustration or graphic design",
],
}
def _compute_text_centroids() -> dict[str, np.ndarray]:
"""Compute the 'photography' and 'other' centroid vectors using the
open_clip text encoder. Only called on the cache-miss path."""
import open_clip
import torch
logger.info("Computing CLIP text centroids for binary classifier")
model, _, _ = open_clip.create_model_and_transforms(
"ViT-B-32", pretrained="laion2b_s34b_b79k"
)
model.eval()
tokenizer = open_clip.get_tokenizer("ViT-B-32")
centroids: dict[str, np.ndarray] = {}
for label, prompts in PROMPTS.items():
tokens = tokenizer(prompts)
with torch.no_grad():
feats = model.encode_text(tokens)
feats = feats / feats.norm(dim=-1, keepdim=True)
avg = feats.mean(dim=0)
avg = avg / avg.norm()
centroids[label] = avg.numpy().astype(np.float32)
return centroids
class CLIPContentClassifier(ContentClassifier):
def __init__(self, settings: VisionSettings):
self._min_confidence = settings.classifier.min_confidence
self._encoder = CLIPVisualEncoder(settings)
cache_dir = Path(settings.models_dir) / "classifier"
cache_dir.mkdir(parents=True, exist_ok=True)
cache_path = cache_dir / "vectors.npz"
if cache_path.exists():
logger.info("Loading cached text centroids from %s", cache_path)
data = np.load(cache_path)
self._photo = data["photography"].astype(np.float32)
self._other = data["other"].astype(np.float32)
else:
centroids = _compute_text_centroids()
self._photo = centroids["photography"]
self._other = centroids["other"]
np.savez(cache_path, photography=self._photo, other=self._other)
logger.info("Cached text centroids to %s", cache_path)
def classify(self, image: np.ndarray) -> ClassificationResult:
vec = self._encoder.encode(image)
s_photo = float(np.dot(vec, self._photo))
s_other = float(np.dot(vec, self._other))
if s_photo >= s_other:
label = "photography"
margin = s_photo - s_other
else:
label = "other"
margin = s_other - s_photo
# 0.01 margin → ~0.3 conf, 0.03+ → ~1.0
confidence = min(1.0, margin * 30)
return ClassificationResult(label=label, confidence=confidence)

View File

@@ -1,58 +0,0 @@
"""
OpenCLIP ViT-B/32 visual encoder (ONNX). Produces 512-d image features
consumed by the content classifier. Not exposed as a standalone service;
the classifier owns the lifecycle.
"""
import logging
from pathlib import Path
import numpy as np
import onnxruntime as ort # noqa: F401 (provider plumbing relies on this)
from app.config import VisionSettings
logger = logging.getLogger(__name__)
_MEAN = np.array([0.48145466, 0.4578275, 0.40821073], dtype=np.float32)
_STD = np.array([0.26862954, 0.26130258, 0.27577711], dtype=np.float32)
_SIZE = 224
def _preprocess(image: np.ndarray) -> np.ndarray:
from PIL import Image
img = Image.fromarray(image).convert("RGB")
w, h = img.size
scale = _SIZE / min(w, h)
img = img.resize((int(w * scale), int(h * scale)), Image.BICUBIC)
w, h = img.size
left = (w - _SIZE) // 2
top = (h - _SIZE) // 2
img = img.crop((left, top, left + _SIZE, top + _SIZE))
arr = np.array(img, dtype=np.float32) / 255.0
arr = (arr - _MEAN) / _STD
arr = arr.transpose(2, 0, 1)
return arr[np.newaxis]
class CLIPVisualEncoder:
"""OpenCLIP ViT-B/32 image encoder, 512-d normalized output."""
def __init__(self, settings: VisionSettings):
model_path = Path(settings.models_dir) / "embed" / "visual.onnx"
from app.services.vision.providers import create_session
from app.config import settings as app_settings
logger.info("Loading CLIP visual encoder from %s", model_path)
self._session = create_session(
str(model_path),
configured_providers=app_settings.vision.execution_providers,
)
def encode(self, image: np.ndarray) -> np.ndarray:
inp = _preprocess(image)
name = self._session.get_inputs()[0].name
out = self._session.run(None, {name: inp})[0][0]
out = out / np.linalg.norm(out)
return out.astype(np.float32)

View File

@@ -1,62 +0,0 @@
"""
Export the OpenCLIP ViT-B/32 visual encoder to ONNX.
Run once on any machine with Python + pip (no GPU needed):
pip install open-clip-torch onnx
python -m app.services.vision.export_models [--models-dir /data/models]
Produces:
embed/visual.onnx (~350 MB)
"""
import argparse
import logging
from pathlib import Path
logger = logging.getLogger(__name__)
def export_openclip_visual(models_dir: Path):
import torch
import open_clip
out_dir = models_dir / "embed"
out_dir.mkdir(parents=True, exist_ok=True)
visual_path = out_dir / "visual.onnx"
if visual_path.exists():
logger.info("OpenCLIP visual.onnx already exists, skipping export")
return
logger.info("Loading OpenCLIP ViT-B-32 laion2b_s34b_b79k...")
model, _, _ = open_clip.create_model_and_transforms(
"ViT-B-32", pretrained="laion2b_s34b_b79k"
)
model.eval()
logger.info("Exporting visual encoder → %s", visual_path)
dummy = torch.randn(1, 3, 224, 224)
torch.onnx.export(
model.visual,
dummy,
str(visual_path),
input_names=["image"],
output_names=["embedding"],
dynamic_axes={"image": {0: "batch"}},
opset_version=14,
dynamo=False,
)
logger.info("Visual encoder exported (%.1f MB)", visual_path.stat().st_size / 1e6)
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--models-dir", type=Path, default=Path("/data/models"))
args = parser.parse_args()
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
args.models_dir.mkdir(parents=True, exist_ok=True)
export_openclip_visual(args.models_dir)
if __name__ == "__main__":
main()

View File

@@ -1,86 +0,0 @@
"""
ONNX Runtime execution provider resolution with GPU auto-detection.
Resolves configured execution providers against what's actually available
in the current ONNX Runtime build. Falls back to CPU if no GPU provider
is available. Logs the selected provider so users can confirm GPU is active.
"""
import logging
import onnxruntime as ort
logger = logging.getLogger(__name__)
_resolved: list[str] | None = None
def get_providers(configured: list[str] | None = None) -> list[str]:
"""Return the best available execution providers.
1. If `configured` is provided, filter to only those that are
actually available in the current ORT build.
2. If none of the configured providers are available, fall back
to CPUExecutionProvider.
3. Auto-detect: if configured is ["auto"], probe for GPU providers.
Results are cached after first call.
"""
global _resolved
if _resolved is not None:
return _resolved
available = set(ort.get_available_providers())
logger.info("ONNX Runtime available providers: %s", sorted(available))
if configured is None or configured == ["CPUExecutionProvider"]:
_resolved = ["CPUExecutionProvider"]
return _resolved
if configured == ["auto"]:
# Auto-detect: prefer CUDA > ROCm > OpenVINO > CPU
priority = [
"CUDAExecutionProvider",
"ROCMExecutionProvider",
"OpenVINOExecutionProvider",
]
for p in priority:
if p in available:
_resolved = [p, "CPUExecutionProvider"]
logger.info("Auto-detected GPU provider: %s", p)
return _resolved
_resolved = ["CPUExecutionProvider"]
logger.info("No GPU provider detected, using CPU")
return _resolved
# Filter configured list to available providers.
resolved = [p for p in configured if p in available]
if not resolved:
logger.warning(
"None of the configured providers %s are available. "
"Falling back to CPU. Available: %s",
configured,
sorted(available),
)
resolved = ["CPUExecutionProvider"]
else:
# Always include CPU as fallback.
if "CPUExecutionProvider" not in resolved:
resolved.append("CPUExecutionProvider")
_resolved = resolved
logger.info("Using ONNX Runtime providers: %s", _resolved)
return _resolved
def create_session(
model_path: str,
opts: ort.SessionOptions | None = None,
configured_providers: list[str] | None = None,
) -> ort.InferenceSession:
"""Create an ONNX InferenceSession with the best available providers."""
providers = get_providers(configured_providers)
if opts is None:
opts = ort.SessionOptions()
opts.inter_op_num_threads = 2
opts.intra_op_num_threads = 2
return ort.InferenceSession(model_path, opts, providers=providers)

View File

@@ -1,29 +0,0 @@
"""
ModelRegistry — lazy-loads the single content classifier per worker.
"""
import logging
from functools import lru_cache
from app.config import settings
from app.services.vision.base import ContentClassifier
logger = logging.getLogger(__name__)
class ModelRegistry:
def __init__(self):
self._vision = settings.vision
@lru_cache(maxsize=1)
def get_classifier(self) -> ContentClassifier:
logger.info("Loading content classifier (backend=%s)", self._vision.backend)
from app.services.vision.classify import CLIPContentClassifier
return CLIPContentClassifier(self._vision)
def warmup(self):
logger.info("Warming up vision classifier...")
self.get_classifier()
logger.info("Vision warmup complete")
registry = ModelRegistry()

View File

@@ -1,15 +0,0 @@
"""
Celery tasks module
"""
from app.tasks.celery import celery_app
from app.tasks.scan import scan_folder, scan_all_source_roots, watch_folders
from app.tasks.thumbs import generate_thumbnails, regenerate_all_thumbnails
__all__ = [
'celery_app',
'scan_folder',
'scan_all_source_roots',
'watch_folders',
'generate_thumbnails',
'regenerate_all_thumbnails'
]

View File

@@ -1,81 +0,0 @@
"""
Celery configuration and app initialization
"""
import logging
import os
from celery import Celery
from celery.signals import worker_process_init
from app.config import settings
logger = logging.getLogger(__name__)
# Create Celery app
celery_app = Celery(
'mulita',
broker=settings.celery_broker_url,
backend=settings.celery_result_backend,
include=[
'app.tasks.scan',
'app.tasks.thumbs',
'app.tasks.vision',
'app.services.metadata', # extract_metadata lives here
]
)
# Configure Celery
celery_app.conf.update(
task_serializer='json',
accept_content=['json'],
result_serializer='json',
timezone='UTC',
enable_utc=True,
# Robust acknowledgment: keep message in broker until task succeeds.
task_acks_late=True,
task_reject_on_worker_lost=True,
# Global time limits — individual tasks can override via decorator.
task_soft_time_limit=300, # 5 min — raises SoftTimeLimitExceeded
task_time_limit=600, # 10 min — SIGKILL
# Explicit routes for every task name. Wildcard patterns don't match
# short names produced by @shared_task(name='...').
task_routes={
# Vision queue — CPU-bound binary classification
'classify_content': {'queue': 'vision'},
'vision_fanout': {'queue': 'vision'},
# High-priority queue — thumbnails & duplicates
'generate_thumbnails': {'queue': 'high'},
'regenerate_all_thumbnails': {'queue': 'high'},
'backfill_phashes': {'queue': 'high'},
'regroup_duplicates': {'queue': 'high'},
'incremental_regroup_duplicates': {'queue': 'high'},
# Low-priority queue — scans
'scan_folder': {'queue': 'low'},
'scan_all_source_roots': {'queue': 'low'},
'backfill_gps': {'queue': 'low'},
# Dedicated watcher queue
'watch_folders': {'queue': 'watcher'},
},
task_default_queue='default',
task_default_exchange='default',
task_default_exchange_type='direct',
task_default_routing_key='default',
broker_connection_retry_on_startup=True,
)
@worker_process_init.connect
def _warmup_vision_models(**kwargs):
"""Pre-load vision models in the worker process so the first task
doesn't pay cold-start latency. Only runs on the vision queue."""
# The worker name contains the queue — only warm up vision workers.
worker_queues = os.environ.get("CELERY_QUEUES", "")
if "vision" not in worker_queues:
# Heuristic: check the celery command line for -Q vision
import sys
if "vision" not in " ".join(sys.argv):
return
try:
from app.services.vision.registry import registry
registry.warmup()
except Exception:
logger.exception("Vision model warmup failed")

View File

@@ -1,618 +0,0 @@
"""
Celery tasks for scanning folders and indexing photos
"""
import os
import hashlib
import asyncio
from pathlib import Path
from datetime import datetime, timezone
import logging
import json
from typing import List, Dict, Optional
from celery import shared_task
from sqlalchemy import select, func
from sqlalchemy.ext.asyncio import AsyncSession
import aiofiles
import redis
from app.database import AsyncSessionLocal
from app.models import Photo, Folder, SourceRoot
from app.config import settings
from app.tasks.thumbs import generate_thumbnails
from app.services.metadata import extract_metadata
from app.services.date_guess import has_date_warning
logger = logging.getLogger(__name__)
# Redis keys read by GET /api/v1/library/scan/status. The frontend
# ScanProgress widget polls that endpoint, so anything we want to surface
# in the UI lives here.
REDIS_KEY_ACTIVE = 'scan:active'
REDIS_KEY_CURRENT_FOLDER = 'scan:current_folder'
REDIS_KEY_PROCESSED = 'scan:processed_files'
REDIS_KEY_TOTAL = 'scan:total_files'
REDIS_KEY_ERRORS = 'scan:errors'
MAX_ERROR_ENTRIES = 50 # cap the errors list so a noisy scan doesn't blow Redis
def _get_redis():
"""Connect to the broker for progress writes. Returns None on failure
so a Redis outage doesn't prevent the scan itself from running."""
try:
return redis.Redis.from_url(settings.celery_broker_url)
except Exception as e:
logger.warning(f"Could not reach Redis for scan progress: {e}")
return None
# Supported file extensions
PHOTO_EXTENSIONS = {'.jpg', '.jpeg', '.png', '.tiff', '.tif', '.webp', '.bmp'}
RAW_EXTENSIONS = {'.cr2', '.cr3', '.nef', '.arw', '.raf', '.dng', '.orf', '.rw2', '.pef', '.srw'}
HEIC_EXTENSIONS = {'.heic', '.heif'}
VIDEO_EXTENSIONS = {'.mp4', '.mov', '.avi', '.mkv', '.mts', '.m2ts', '.3gp', '.wmv', '.flv'}
SUPPORTED_EXTENSIONS = PHOTO_EXTENSIONS | RAW_EXTENSIONS | HEIC_EXTENSIONS | VIDEO_EXTENSIONS
def get_media_type(filepath: str) -> str:
"""Determine media type from file extension"""
ext = Path(filepath).suffix.lower()
if ext in PHOTO_EXTENSIONS:
return 'photo'
elif ext in RAW_EXTENSIONS:
return 'raw'
elif ext in HEIC_EXTENSIONS:
return 'heic'
elif ext in VIDEO_EXTENSIONS:
return 'video'
return 'unknown'
async def calculate_file_hash(filepath: str) -> str:
"""Calculate SHA-256 hash of a file"""
hash_sha256 = hashlib.sha256()
try:
async with aiofiles.open(filepath, 'rb') as f:
while chunk := await f.read(8192):
hash_sha256.update(chunk)
return hash_sha256.hexdigest()
except Exception as e:
logger.error(f"Error calculating hash for {filepath}: {e}")
return ""
@shared_task(bind=True, name='scan_folder')
def scan_folder(self, folder_path: str, source_root_id: Optional[str] = None):
"""
Scan a folder and index all photos/videos
"""
# Run async function in sync context
return asyncio.run(_scan_folder_async(folder_path, source_root_id, self))
async def _scan_folder_async(folder_path: str, source_root_id: Optional[str], task):
"""Async implementation of folder scanning. Writes progress to Redis so
GET /api/v1/library/scan/status can surface it to the frontend
ScanProgress widget."""
logger.info(f"Starting scan of folder: {folder_path}")
r = _get_redis()
PROGRESS_TTL = 3600 # 1 hour — auto-expire if scan crashes
def progress_set(key: str, value) -> None:
if r is None:
return
try:
r.set(key, str(value), ex=PROGRESS_TTL)
except Exception as e:
logger.debug(f"scan progress set failed: {e}")
def progress_push_error(message: str) -> None:
if r is None:
return
try:
r.lpush(REDIS_KEY_ERRORS, message)
r.ltrim(REDIS_KEY_ERRORS, 0, MAX_ERROR_ENTRIES - 1)
except Exception as e:
logger.debug(f"scan progress push_error failed: {e}")
# Mark scan active immediately so the UI starts polling fast.
progress_set(REDIS_KEY_ACTIVE, 'true')
progress_set(REDIS_KEY_CURRENT_FOLDER, folder_path)
async with AsyncSessionLocal() as session:
try:
# Get or create source root
if not source_root_id:
source_root = await get_or_create_source_root(session, folder_path)
source_root_id = source_root.id
else:
source_root = (await session.execute(
select(SourceRoot).where(SourceRoot.id == source_root_id)
)).scalar_one_or_none()
# Inherit user_id from the source root's owner
owner_user_id = source_root.user_id if source_root else None
# Per-scan memoization cache for "is this folder's effective
# is_hidden true?" Populated on first lookup by walking the
# parent_id chain up to the source root. Keyed by folder_id
# so repeated photos in the same folder pay only one lookup.
hidden_folder_cache: dict[str, bool] = {}
async def is_folder_effectively_hidden(folder_row: Folder) -> bool:
if folder_row.id in hidden_folder_cache:
return hidden_folder_cache[folder_row.id]
# Walk parents. If the current folder is hidden, short-
# circuit. Otherwise climb until we hit a root (no
# parent_id) or a cached ancestor.
if folder_row.is_hidden:
hidden_folder_cache[folder_row.id] = True
return True
parent_id = folder_row.parent_id
while parent_id is not None:
if parent_id in hidden_folder_cache:
hidden_folder_cache[folder_row.id] = hidden_folder_cache[parent_id]
return hidden_folder_cache[folder_row.id]
parent = (
await session.execute(
select(Folder).where(Folder.id == parent_id)
)
).scalar_one_or_none()
if parent is None:
break
if parent.is_hidden:
hidden_folder_cache[folder_row.id] = True
return True
parent_id = parent.parent_id
hidden_folder_cache[folder_row.id] = False
return False
# Pre-walk to compute the total file count upfront. Without this
# the progress bar would jump every time a new subfolder is
# encountered because the running total kept growing.
total_files = 0
for _root, _dirs, files in os.walk(folder_path):
total_files += sum(
1 for f in files if Path(f).suffix.lower() in SUPPORTED_EXTENSIONS
)
progress_set(REDIS_KEY_TOTAL, total_files)
progress_set(REDIS_KEY_PROCESSED, 0)
processed_files = 0
errors = []
for root, dirs, files in os.walk(folder_path):
# Get or create folder entry
folder = await get_or_create_folder(session, root, source_root_id, owner_user_id)
progress_set(REDIS_KEY_CURRENT_FOLDER, root)
# Filter supported files
supported_files = [f for f in files if Path(f).suffix.lower() in SUPPORTED_EXTENSIONS]
# Process files in batches
batch_size = settings.scanner.batch_size
for i in range(0, len(supported_files), batch_size):
batch = supported_files[i:i + batch_size]
# Defer task dispatch until AFTER commit so workers don't
# query for rows that aren't visible to other sessions yet.
pending_dispatch: list[str] = []
for filename in batch:
filepath = os.path.join(root, filename)
try:
# Check if file already exists in database
existing = await session.execute(
select(Photo).where(Photo.filepath == filepath)
)
if existing.scalar_one_or_none():
logger.debug(f"File already indexed: {filepath}")
processed_files += 1
progress_set(REDIS_KEY_PROCESSED, processed_files)
continue
# Get file stats
stat = os.stat(filepath)
# Calculate file hash for duplicate detection
file_hash = await calculate_file_hash(filepath)
# Check for duplicate by hash. We only care
# whether *any* other photo shares this hash, so
# use a count rather than scalar_one_or_none()
# which raises "Multiple rows were found" the
# moment the library has 2+ copies of the same
# file (i.e. exactly the case we're trying to
# flag).
is_dup = False
if file_hash:
dup_count = (await session.execute(
select(func.count(Photo.id)).where(
Photo.file_hash == file_hash
)
)).scalar() or 0
is_dup = dup_count > 0
# Inherit the effective-hidden flag from the
# folder's ancestry. If any ancestor folder
# has is_hidden=true, the new photo is
# immediately marked hidden so it never
# briefly appears in cross-cutting views
# between scan and the next manual recompute.
effective_hidden = await is_folder_effectively_hidden(folder)
# Create photo entry
mtime_dt = datetime.fromtimestamp(stat.st_mtime)
photo = Photo(
filepath=filepath,
filename=filename,
folder_id=folder.id,
user_id=owner_user_id,
file_hash=file_hash,
media_type=get_media_type(filepath),
original_format=Path(filepath).suffix.upper()[1:],
file_size=stat.st_size,
taken_at=mtime_dt,
taken_at_source='filesystem',
# First-pass flag based on the filesystem mtime;
# metadata.extract_metadata re-runs this once
# EXIF has been parsed so a real DateTimeOriginal
# can clear the warning.
has_date_warning=has_date_warning(filepath, mtime_dt),
is_duplicate=is_dup,
is_hidden=effective_hidden,
processing_status='pending'
)
session.add(photo)
await session.flush() # Assign defaults / FK ids
# Queue dispatch happens after the batch commit
# below; otherwise the worker can race the writer
# and see "Photo not found".
pending_dispatch.append(photo.id)
processed_files += 1
progress_set(REDIS_KEY_PROCESSED, processed_files)
# Celery internal progress (used by celery tooling)
if processed_files % 10 == 0:
task.update_state(
state='PROGRESS',
meta={
'current': processed_files,
'total': total_files,
'folder': root,
}
)
except Exception as e:
logger.error(f"Error processing file {filepath}: {e}")
errors.append({'file': filepath, 'error': str(e)})
progress_push_error(f"{filepath}: {e}")
continue
# Commit batch, then queue worker tasks. Dispatch order
# matters: commit first so workers can find the rows.
await session.commit()
for photo_id in pending_dispatch:
generate_thumbnails.delay(photo_id)
extract_metadata.delay(photo_id)
# Update folder scan timestamp
folder.last_scanned = datetime.utcnow()
folder.photo_count = processed_files
await session.commit()
logger.info(f"Scan complete. Processed {processed_files}/{total_files} files. Errors: {len(errors)}")
return {
'status': 'completed',
'processed': processed_files,
'total': total_files,
'errors': errors,
}
except Exception as e:
logger.error(f"Scan failed: {e}")
progress_push_error(f"scan failed: {e}")
await session.rollback()
raise
finally:
# Always mark inactive on the way out so a crashed scan doesn't
# leave the UI thinking we're still scanning.
progress_set(REDIS_KEY_ACTIVE, 'false')
def _normalize_path(path: str) -> str:
"""Canonicalise a filesystem path so we don't get duplicate DB rows for
the same physical directory due to trailing slashes, redundant separators,
or `.` segments. Symlinks are NOT resolved (we want to keep mount paths
intact for cross-machine portability)."""
return os.path.normpath(path)
async def get_or_create_source_root(session: AsyncSession, path: str) -> SourceRoot:
"""Get or create a source root entry, matching by normalized path."""
from sqlalchemy import select
norm = _normalize_path(path)
result = await session.execute(
select(SourceRoot).where(SourceRoot.path == norm)
)
source_root = result.scalar_one_or_none()
if not source_root:
source_root = SourceRoot(
name=Path(norm).name,
path=norm,
)
session.add(source_root)
await session.flush()
return source_root
async def get_or_create_folder(
session: AsyncSession, path: str, source_root_id: str, user_id: str = None
) -> Folder:
"""Get or create a folder entry, matching by normalized path."""
from sqlalchemy import select
norm = _normalize_path(path)
result = await session.execute(
select(Folder).where(Folder.path == norm)
)
folder = result.scalar_one_or_none()
if not folder:
parent_path = _normalize_path(str(Path(norm).parent))
if parent_path != norm: # Not the filesystem root
parent_result = await session.execute(
select(Folder).where(Folder.path == parent_path)
)
parent = parent_result.scalar_one_or_none()
if parent:
parent_id = parent.id
else:
# Recursively create parent
parent = await get_or_create_folder(session, parent_path, source_root_id, user_id)
parent_id = parent.id
else:
parent_id = None
folder = Folder(
name=Path(norm).name,
path=norm,
parent_id=parent_id,
source_root_id=source_root_id,
user_id=user_id,
)
session.add(folder)
await session.flush()
return folder
@shared_task(name='scan_all_source_roots')
def scan_all_source_roots():
"""Scan every active source root currently registered in the DB."""
# Clear stale per-scan progress before queuing new work so the UI sees
# a clean slate even if a previous run crashed mid-flight.
r = _get_redis()
if r is not None:
try:
r.delete(REDIS_KEY_ERRORS)
r.set(REDIS_KEY_PROCESSED, 0)
r.set(REDIS_KEY_TOTAL, 0)
except Exception as e:
logger.debug(f"scan_all_source_roots redis reset failed: {e}")
return asyncio.run(_scan_all_source_roots_async())
async def _scan_all_source_roots_async():
"""Read every active SourceRoot from the DB and queue a scan_folder task
for each. Source roots whose path no longer exists on disk are skipped
with a warning (the cleanup service surfaces those at startup too).
After dispatching the scans, queue a delayed `regroup_duplicates`
pass so duplicate clusters are recomputed once the new photos have
finished thumbnailing (and therefore picked up phashes). The
countdown is a best-effort hint — on a big library the user can
still hit Settings → Re-detect duplicates to force a fresh pass.
"""
from app.tasks.thumbs import incremental_regroup_duplicates_task
from app.tasks.vision import backfill_vision
async with AsyncSessionLocal() as session:
result = await session.execute(
select(SourceRoot).where(SourceRoot.is_active == True) # noqa: E712
)
source_roots = result.scalars().all()
dispatched = 0
for sr in source_roots:
if os.path.exists(sr.path):
scan_folder.delay(sr.path, sr.id)
dispatched += 1
else:
logger.warning(f"Source root path does not exist: {sr.path}")
if dispatched > 0:
# 60s gives the thumbs worker a window to compute phashes for
# the new photos before regrouping. The task is idempotent, so
# firing too early just means the next manual run picks up the
# late arrivals — no corrupted state.
try:
# Use incremental mode: only compare newly added photos
# against the full library via CLIP HNSW + pHash.
# O(new × log N) instead of O(N²).
scan_start = datetime.now(timezone.utc).isoformat()
incremental_regroup_duplicates_task.apply_async(
kwargs={'since_iso': scan_start},
countdown=60,
)
except Exception as e:
logger.warning(f"Could not queue post-scan regroup: {e}")
# 90s lets thumbnails finish so photos reach processing_status
# 'completed', which backfill_vision uses as its filter.
try:
backfill_vision.apply_async(countdown=90)
except Exception as e:
logger.warning(f"Could not queue post-scan vision backfill: {e}")
# Re-extract metadata for photos missing GPS coordinates.
# Runs on every startup so photos scanned before the GPS fix
# eventually get their coordinates populated.
try:
backfill_gps.apply_async(countdown=30)
except Exception as e:
logger.warning(f"Could not queue post-scan GPS backfill: {e}")
WATCHER_LOCK_KEY = "mulita:watch_folders:lock"
WATCHER_LOCK_TTL = 60 # 1 min — renewed every event batch via wall-clock check
@shared_task(name='watch_folders', bind=True, soft_time_limit=None, time_limit=None)
def watch_folders(self):
"""
Watch folders for changes using watchfiles. Long-running task that
monitors filesystem events under every active source root.
Uses a Redis lock to ensure only one instance runs across all
workers. The lock is renewed periodically so it survives restarts
without leaving orphan watchers.
"""
import redis as redis_lib
from watchfiles import watch
r = redis_lib.from_url(settings.redis_url)
# Acquire exclusive lock — if another watcher is already running,
# this instance exits immediately instead of stacking up.
lock = r.lock(WATCHER_LOCK_KEY, timeout=WATCHER_LOCK_TTL)
if not lock.acquire(blocking=False):
logger.info("watch_folders: another instance is already running, exiting")
return {'status': 'skipped', 'reason': 'another instance is running'}
try:
roots: list[tuple[str, str]] = []
try:
async def _load_roots():
async with AsyncSessionLocal() as session:
result = await session.execute(
select(SourceRoot).where(SourceRoot.is_active == True) # noqa: E712
)
return [
(os.path.normpath(sr.path), sr.id)
for sr in result.scalars().all()
if os.path.exists(sr.path)
]
roots = asyncio.run(_load_roots())
except Exception as e:
logger.error(f"watch_folders could not load source roots: {e}")
return
if not roots:
logger.warning("No valid source roots to watch")
return
paths = [p for p, _ in roots]
logger.info(f"Starting folder watcher for: {paths}")
def find_source_root_for(path: str) -> Optional[str]:
"""Return the source_root id whose path contains `path`, or None."""
normalized = os.path.normpath(path)
for root_path, root_id in roots:
if normalized == root_path or normalized.startswith(root_path + os.sep):
return root_id
return None
import time
last_renew = time.monotonic()
for changes in watch(*paths, rust_timeout=30_000):
# Renew the Redis lock on a wall-clock schedule (every 30s)
# instead of every N events, so quiet directories don't let
# the lock expire. watchfiles' rust_timeout ensures we wake
# at least every 30s even with no FS events.
now = time.monotonic()
if now - last_renew >= 30:
try:
lock.extend(WATCHER_LOCK_TTL)
last_renew = now
except Exception:
logger.warning("watch_folders: failed to renew Redis lock")
for change_type, filepath in changes:
filepath = str(filepath)
if Path(filepath).suffix.lower() not in SUPPORTED_EXTENSIONS:
continue
if change_type == 'added' or change_type == 'modified':
parent_dir = str(Path(filepath).parent)
source_root_id = find_source_root_for(parent_dir)
if source_root_id is None:
continue
scan_folder.delay(parent_dir, source_root_id)
logger.info(f"File {change_type}: {filepath}, queued scan for {parent_dir}")
elif change_type == 'deleted':
asyncio.run(handle_file_deletion(filepath))
finally:
try:
lock.release()
except Exception:
logger.warning("watch_folders: could not release Redis lock (may have expired)")
async def handle_file_deletion(filepath: str):
"""Handle deletion of a file from the filesystem"""
from sqlalchemy import select
async with AsyncSessionLocal() as session:
result = await session.execute(
select(Photo).where(Photo.filepath == filepath)
)
photo = result.scalar_one_or_none()
if photo:
# Mark as missing or delete from database
photo.is_discarded = True
photo.discarded_at = datetime.utcnow()
await session.commit()
logger.info(f"Marked photo as discarded: {filepath}")
@shared_task(name='backfill_gps')
def backfill_gps():
"""Re-run metadata extraction on every non-discarded photo that is
missing latitude/longitude. Used both as a one-shot kick-off after the
GPS columns are added on an existing install (see app/database.py) and
as a manual trigger from POST /api/v1/library/backfill-gps. Each
extract_metadata call is itself a Celery task, so this just enqueues —
it does not block on extraction completing."""
return asyncio.run(_backfill_gps_async())
async def _backfill_gps_async():
async with AsyncSessionLocal() as session:
# Newest-first so the most recent photos get their GPS + EXIF
# written before the worker climbs back through the archive.
result = await session.execute(
select(Photo.id)
.where(
Photo.latitude.is_(None),
Photo.is_discarded.is_(False),
)
.order_by(
Photo.taken_at.desc().nullslast(),
Photo.added_at.desc().nullslast(),
)
)
photo_ids = [row[0] for row in result.all()]
for pid in photo_ids:
extract_metadata.delay(pid)
logger.info(f"backfill_gps: queued extract_metadata for {len(photo_ids)} photos")
return {'queued': len(photo_ids)}

View File

@@ -1,552 +0,0 @@
"""
Celery tasks for thumbnail generation
"""
import os
import asyncio
from pathlib import Path
import logging
from typing import Tuple, Optional
import json
from celery import shared_task
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from PIL import Image
import imageio
from pillow_heif import register_heif_opener
import ffmpeg
# Try to import optional libraries
try:
import pyvips
PYVIPS_AVAILABLE = True
except ImportError:
PYVIPS_AVAILABLE = False
print("pyvips not available, using Pillow for image processing")
try:
import rawpy
RAWPY_AVAILABLE = True
except ImportError:
RAWPY_AVAILABLE = False
print("rawpy not available, using exiftool for RAW preview extraction")
from app.database import AsyncSessionLocal
from app.models import Photo
from app.config import settings
# Register HEIF opener with Pillow
register_heif_opener()
logger = logging.getLogger(__name__)
# Thumbnail sizes configuration
THUMB_SIZES = {
'small': settings.thumbnails.small,
'medium': settings.thumbnails.medium,
'large': settings.thumbnails.large
}
def get_thumb_path(photo_id: str, size: str, user_id: str = None) -> str:
"""Get the path for a thumbnail file.
When user_id is provided, thumbnails are stored under a user-specific
subdirectory to enforce isolation between users.
"""
if user_id:
thumb_dir = f"/data/thumbs/{user_id}/{photo_id}"
else:
thumb_dir = f"/data/thumbs/{photo_id}"
os.makedirs(thumb_dir, exist_ok=True)
return f"{thumb_dir}/{size}.{settings.thumbnails.format}"
def process_standard_image(filepath: str) -> Image.Image:
"""Process standard image formats (JPEG, PNG, etc.)"""
return Image.open(filepath)
def process_raw_image(filepath: str) -> Image.Image:
"""Process RAW image formats"""
if RAWPY_AVAILABLE:
try:
with rawpy.imread(filepath) as raw:
# Use half_size for faster processing
rgb = raw.postprocess(use_camera_wb=True, half_size=True)
# Convert numpy array to PIL Image
return Image.fromarray(rgb, 'RGB')
except Exception as e:
logger.error(f"Error processing RAW file {filepath}: {e}")
# Try to extract embedded JPEG preview
return extract_raw_preview(filepath)
else:
# Use exiftool to extract embedded preview
return extract_raw_preview(filepath)
def extract_raw_preview(filepath: str) -> Optional[Image.Image]:
"""Extract embedded JPEG preview from RAW file"""
try:
# Use exiftool to extract preview
import subprocess
import tempfile
with tempfile.NamedTemporaryFile(suffix='.jpg', delete=False) as tmp:
cmd = ['exiftool', '-b', '-PreviewImage', filepath]
result = subprocess.run(cmd, capture_output=True)
if result.returncode == 0 and result.stdout:
tmp.write(result.stdout)
tmp.flush()
return Image.open(tmp.name)
except Exception as e:
logger.error(f"Error extracting RAW preview from {filepath}: {e}")
return None
def process_heic_image(filepath: str) -> Image.Image:
"""Process HEIC/HEIF image formats.
Tries pillow-heif first (fast, native). Falls back to ffmpeg for
files that libheif rejects — e.g. iPhone photos with too many
auxiliary image references (depth maps, gain maps).
"""
try:
img = Image.open(filepath)
if img.mode != 'RGB':
img = img.convert('RGB')
return img
except Exception as e:
logger.warning(f"pillow-heif failed for {filepath}: {e} — trying vips")
# vips fallback: handles tiled Apple HEIC files (bursts, HDR gain
# maps, depth maps) that pillow-heif/libheif rejects due to too many
# auxiliary image references.
import subprocess, tempfile
try:
with tempfile.NamedTemporaryFile(suffix='.png', delete=False) as tmp:
tmp_path = tmp.name
result = subprocess.run(
['vips', 'heifload', filepath, tmp_path],
capture_output=True, timeout=60, stdin=subprocess.DEVNULL,
)
if result.returncode == 0:
img = Image.open(tmp_path).convert('RGB')
os.unlink(tmp_path)
return img
logger.error(f"vips HEIC decode failed for {filepath}: {result.stderr.decode()[-200:]}")
os.unlink(tmp_path)
except Exception as e2:
logger.error(f"vips fallback failed for {filepath}: {e2}")
raise RuntimeError(f"Cannot decode HEIC: {filepath}")
def process_video_thumbnail(filepath: str) -> Image.Image:
"""Extract a still frame from a video file as a PIL Image."""
import tempfile
from io import BytesIO
tmp_path: Optional[str] = None
try:
# Find a usable seek timestamp. Some camera MOVs only expose
# duration at the format level, and stream 0 isn't always the
# video stream — search explicitly and fall back to the format
# duration, then to t=0 if neither is available.
probe = ffmpeg.probe(filepath)
duration: Optional[float] = None
for stream_info in probe.get('streams', []):
if stream_info.get('codec_type') != 'video':
continue
raw_duration = stream_info.get('duration')
if raw_duration is not None:
try:
duration = float(raw_duration)
break
except (TypeError, ValueError):
pass
if duration is None:
raw_duration = probe.get('format', {}).get('duration')
if raw_duration is not None:
try:
duration = float(raw_duration)
except (TypeError, ValueError):
duration = None
# Seek to 10% in for a representative frame; clamp very short
# clips to t=0 so we don't seek past the end.
timestamp = max(0.0, (duration or 0.0) * 0.1)
# NamedTemporaryFile creates the file on disk, so we MUST tell
# ffmpeg to overwrite it (otherwise it prompts on stdin and the
# call hangs/fails — which is why videos were getting the gray
# placeholder). We close the handle immediately and clean up
# in `finally` ourselves.
with tempfile.NamedTemporaryFile(suffix='.jpg', delete=False) as tmp:
tmp_path = tmp.name
stream = ffmpeg.input(filepath, ss=timestamp)
stream = ffmpeg.output(
stream,
tmp_path,
vframes=1,
format='image2',
vcodec='mjpeg',
)
ffmpeg.run(
stream,
capture_stdout=True,
capture_stderr=True,
overwrite_output=True,
)
# Load the frame fully into memory so we can delete the temp
# file immediately. Pillow's `Image.open` is lazy, which would
# otherwise leave the file dangling.
with open(tmp_path, 'rb') as fh:
data = fh.read()
if not data:
raise RuntimeError("ffmpeg produced an empty frame")
return Image.open(BytesIO(data)).copy()
except ffmpeg.Error as e:
stderr = (e.stderr or b'').decode('utf-8', errors='replace')
logger.error(
f"ffmpeg failed extracting video thumbnail from {filepath}: {stderr}"
)
return create_placeholder_thumbnail('video')
except Exception as e:
logger.error(f"Error extracting video thumbnail from {filepath}: {e}")
return create_placeholder_thumbnail('video')
finally:
if tmp_path and os.path.exists(tmp_path):
try:
os.unlink(tmp_path)
except OSError:
pass
def create_placeholder_thumbnail(media_type: str) -> Image.Image:
"""Create a placeholder thumbnail for failed processing"""
# Create a simple gray placeholder
img = Image.new('RGB', (640, 480), color=(128, 128, 128))
return img
def auto_rotate_image(image: Image.Image) -> Image.Image:
"""Auto-rotate image based on EXIF orientation"""
try:
# Get EXIF data
exif = image._getexif()
if exif:
orientation = exif.get(274) # Orientation tag
rotation_map = {
3: 180,
6: 270, # Note: PIL uses different rotation values than vips
8: 90
}
if orientation in rotation_map:
image = image.rotate(rotation_map[orientation], expand=True)
except (AttributeError, KeyError, TypeError):
pass # No orientation data available
return image
def generate_thumbnail(image: Image.Image, size: int, output_path: str):
"""Generate a thumbnail of the specified size.
Works on a copy so the caller's image is never mutated — this is
critical because the thumbnail loop iterates multiple sizes and
in-place shrinking would degrade later (larger) sizes.
"""
img = image.copy()
img.thumbnail((size, size), Image.Resampling.LANCZOS)
img.save(
output_path,
'WEBP',
quality=settings.thumbnails.quality,
method=4 # Balance between speed and compression
)
img.close()
@shared_task(bind=True, name='generate_thumbnails')
def generate_thumbnails(self, photo_id: str):
"""Generate thumbnails for a photo"""
return asyncio.run(_generate_thumbnails_async(photo_id, self))
async def _generate_thumbnails_async(photo_id: str, task):
"""Async implementation of thumbnail generation"""
async with AsyncSessionLocal() as session:
# Declared up front so the except block below can safely check it
# even if the initial SELECT raises (e.g. asyncpg transport error).
photo: Optional[Photo] = None
try:
# Get photo from database
result = await session.execute(
select(Photo).where(Photo.id == photo_id)
)
photo = result.scalar_one_or_none()
if not photo:
logger.error(f"Photo not found: {photo_id}")
return {'status': 'error', 'message': 'Photo not found'}
# Check if file exists
if not os.path.exists(photo.filepath):
logger.error(f"File not found: {photo.filepath}")
photo.processing_status = 'failed'
photo.processing_error = 'File not found'
await session.commit()
return {'status': 'error', 'message': 'File not found'}
# Update processing status
photo.processing_status = 'processing'
await session.commit()
# Load and process the image based on type
image = None
if photo.media_type == 'photo':
image = process_standard_image(photo.filepath)
elif photo.media_type == 'raw':
image = process_raw_image(photo.filepath)
elif photo.media_type == 'heic':
image = process_heic_image(photo.filepath)
elif photo.media_type == 'video':
image = process_video_thumbnail(photo.filepath)
else:
logger.error(f"Unsupported media type: {photo.media_type}")
image = create_placeholder_thumbnail(photo.media_type)
# Fallback: some files wear a RAW/HEIC extension but are actually
# plain JPEGs — e.g. iPhones that write ProRAW-style .DNG for
# images where no RAW sensor data was captured, or re-exports
# that kept the original suffix. Pillow can open them directly,
# so before giving up, try reading the file as a standard image.
if not image and photo.media_type in ('raw', 'heic'):
try:
image = process_standard_image(photo.filepath)
if image is not None:
logger.info(
f"{photo.filepath}: {photo.media_type} decode failed "
f"but file opens as a standard image — using fallback"
)
except Exception as e:
logger.debug(
f"Standard-image fallback failed for {photo.filepath}: {e}"
)
if not image:
raise Exception("Failed to process image")
# Auto-rotate based on EXIF
image = auto_rotate_image(image)
# Store original dimensions
photo.width = image.width
photo.height = image.height
# Perceptual hash from the original-resolution decoded frame.
# pHash is robust to resize/recompression but the thumbnail
# loop below mutates `image` in place, so this MUST run before
# the loop sees it. Failures are non-fatal — phash is a
# nice-to-have, not a blocker for thumbnail generation.
try:
import imagehash
photo.phash = str(imagehash.phash(image)) # 16-char hex
except Exception as e:
logger.warning(f"phash failed for {photo_id}: {e}")
photo.phash = None
# Generate thumbnails for each size
for size_name, size_value in THUMB_SIZES.items():
thumb_path = get_thumb_path(photo_id, size_name, photo.user_id)
generate_thumbnail(image, size_value, thumb_path)
# Update database with thumbnail path
setattr(photo, f'thumb_{size_name}', thumb_path)
# Update progress
task.update_state(
state='PROGRESS',
meta={'current_size': size_name, 'photo_id': photo_id}
)
# Update processing status
photo.processing_status = 'completed'
photo.processing_error = None
await session.commit()
logger.info(f"Thumbnails generated for photo {photo_id}")
# Dispatch vision pipeline only after thumbnails succeeded —
# vision tasks need the generated thumbnails to run inference.
try:
from app.tasks.vision import vision_fanout
vision_fanout.delay(photo_id)
except Exception as e:
logger.warning(f"Could not dispatch vision_fanout for {photo_id}: {e}")
return {'status': 'success', 'photo_id': photo_id}
except Exception as e:
logger.error(f"Error generating thumbnails for {photo_id}: {e}")
# Update error status. If the session is in a bad state (e.g.
# the original failure was a transport error) rollback first so
# the status write has a clean transaction to commit into.
try:
await session.rollback()
except Exception:
pass
if photo is not None:
try:
photo.processing_status = 'failed'
photo.processing_error = str(e)
await session.commit()
except Exception:
logger.exception(
f"Could not mark photo {photo_id} as failed"
)
return {'status': 'error', 'message': str(e)}
@shared_task(name='regenerate_all_thumbnails')
def regenerate_all_thumbnails():
"""Regenerate thumbnails for all photos"""
return asyncio.run(_regenerate_all_thumbnails_async())
async def _regenerate_all_thumbnails_async():
"""Async implementation of regenerating all thumbnails.
Queue order matters on first-boot and recovery runs: we dispatch
newest-first (by EXIF taken_at, fallback added_at) so the user's
most recent photos become fully-indexed before the 2012 archive even
starts. Picking up the library in pipeline order means the grid,
timeline and All Photos view populate top-down instead of the worker
chewing through random insertion-order rows while the UI still
shows grey placeholders.
"""
async with AsyncSessionLocal() as session:
# Get all photos that need thumbnails, newest first.
result = await session.execute(
select(Photo)
.where(Photo.processing_status.in_(['pending', 'failed']))
.order_by(
Photo.taken_at.desc().nullslast(),
Photo.added_at.desc().nullslast(),
)
)
photos = result.scalars().all()
logger.info(f"Regenerating thumbnails for {len(photos)} photos")
for photo in photos:
generate_thumbnails.delay(photo.id)
return {'status': 'queued', 'count': len(photos)}
# ── Perceptual hash backfill ────────────────────────────────────────────
#
# When phash was added post-launch, every existing photo has phash=NULL.
# This task fills them in by reading the existing thumb_large (the cheap
# option — pHash is robust to scale, and the thumb is already on local
# disk so we avoid re-decoding the original RAW/HEIC). Falls back to the
# original filepath if the thumb isn't available for some reason. Runs
# in batches to keep memory bounded and to give the user incremental
# progress visible in the worker logs.
@shared_task(name='backfill_phashes')
def backfill_phashes():
"""Compute and persist phash for every photo currently missing one."""
return asyncio.run(_backfill_phashes_async())
async def _backfill_phashes_async():
import imagehash
from PIL import Image as _PILImage
BATCH = 100
total_done = 0
total_failed = 0
async with AsyncSessionLocal() as session:
while True:
# Newest-first so the recent end of the library gets phashes
# (and therefore duplicate detection) ahead of the archive.
result = await session.execute(
select(Photo)
.where(Photo.phash.is_(None))
.where(Photo.processing_status == 'completed')
.order_by(
Photo.taken_at.desc().nullslast(),
Photo.added_at.desc().nullslast(),
)
.limit(BATCH)
)
batch = result.scalars().all()
if not batch:
break
for photo in batch:
source = photo.thumb_large or photo.filepath
try:
if not source or not os.path.exists(source):
photo.phash = None
total_failed += 1
continue
with _PILImage.open(source) as im:
photo.phash = str(imagehash.phash(im))
total_done += 1
except Exception as e:
logger.warning(f"phash backfill failed for {photo.id}: {e}")
total_failed += 1
await session.commit()
logger.info(
f"Backfilled phashes: {total_done} done, {total_failed} failed"
)
return {
'status': 'success',
'computed': total_done,
'failed': total_failed,
}
@shared_task(
name='regroup_duplicates',
# Full regroup scales with O(N²) on phash plus one pgvector query per
# embedded photo. On a 16k-photo library that's comfortably past the
# default 5-minute soft limit — bump to 2h / 2h30m. (Passing None here
# does NOT disable limits; Celery falls back to the worker default
# of 300s/600s. An explicit number overrides.)
soft_time_limit=7200,
time_limit=9000,
)
def regroup_duplicates_task():
"""Full recompute of duplicate groups (pHash + CLIP similarity).
Used by the Settings → Re-detect duplicates button."""
from app.services.duplicates import regroup_duplicates
return asyncio.run(regroup_duplicates())
@shared_task(
name='incremental_regroup_duplicates',
# O(new × N); still cheaper than a full regroup but can easily exceed
# the 5-minute default after a big batch import. Same caveat as
# regroup_duplicates above — None would just re-inherit the worker
# default, so we pass explicit values.
soft_time_limit=3600,
time_limit=4200,
)
def incremental_regroup_duplicates_task(since_iso: str | None = None):
"""Incremental duplicate detection for newly added photos.
Compares only photos added after `since_iso` against the full library
using CLIP vector similarity (O(new × log N) via HNSW) plus pHash.
Default post-scan path — much faster than a full regroup."""
from app.services.duplicates import incremental_regroup
from datetime import datetime, timezone
since = None
if since_iso:
since = datetime.fromisoformat(since_iso)
return asyncio.run(incremental_regroup(since=since))

View File

@@ -1,194 +0,0 @@
"""
Celery tasks for the vision pipeline.
A single binary classifier decides whether a photo is 'photography' or
'other'. Photos classified as 'other' get needs_review=true so the user
can triage screenshots / documents / memes in the UI.
"""
import logging
from pathlib import Path
import numpy as np
from celery import shared_task
from sqlalchemy import create_engine, text as sa_text, select, delete, update
from sqlalchemy.orm import Session, sessionmaker
from PIL import Image
from app.config import settings
from app.services.feature_flags import is_enabled, FLAG_VISION_ENABLED
logger = logging.getLogger(__name__)
VISION_READY_KEY = "mulita:vision:ready"
def _vision_worker_ready() -> bool:
try:
import redis as _redis
return bool(_redis.from_url(settings.redis_url).exists(VISION_READY_KEY))
except Exception:
return False
_sync_engine = None
def _get_sync_engine():
global _sync_engine
if _sync_engine is None:
sync_url = settings.database_url.replace("+asyncpg", "+psycopg2").replace("+aiosqlite", "")
_sync_engine = create_engine(sync_url, pool_pre_ping=True, pool_size=3, max_overflow=5)
return _sync_engine
def _get_sync_session() -> Session:
return sessionmaker(bind=_get_sync_engine())()
def _load_thumb(photo_id: str, size: str = "medium") -> np.ndarray | None:
thumb_base = Path("/data/thumbs")
thumb_path = thumb_base / photo_id / f"{size}.webp"
if not thumb_path.exists():
matches = list(thumb_base.glob(f"*/{photo_id}/{size}.webp"))
if matches:
thumb_path = matches[0]
else:
logger.warning("Thumbnail not found: %s", thumb_path)
return None
try:
img = Image.open(thumb_path).convert("RGB")
img.load()
arr = np.array(img)
img.close()
return arr
except Exception as e:
logger.warning("Corrupt or unreadable thumbnail for %s: %s", photo_id, e)
return None
@shared_task(name='vision_fanout', queue='vision')
def vision_fanout(photo_id: str):
"""Dispatch vision work for a photo. Today this is just the binary
classifier; the indirection stays so scanner/upload code keeps one
entrypoint."""
if not is_enabled(FLAG_VISION_ENABLED):
return {'status': 'skipped', 'reason': 'vision disabled'}
classify_content.delay(photo_id)
return {'status': 'dispatched', 'photo_id': photo_id}
@shared_task(name='classify_content', queue='vision', bind=True, max_retries=3)
def classify_content(self, photo_id: str):
"""Run the binary classifier and write:
- a Tag(kind='content_type', name IN ('photography','other'))
- Photo.needs_review = (label == 'other')
"""
if not is_enabled(FLAG_VISION_ENABLED):
return {'status': 'skipped', 'reason': 'vision disabled'}
image = _load_thumb(photo_id, "medium")
if image is None:
return {'status': 'error', 'message': 'thumbnail not found'}
try:
from app.services.vision.registry import registry
classifier = registry.get_classifier()
result = classifier.classify(image)
except Exception as exc:
logger.exception("classify_content failed for %s", photo_id)
raise self.retry(exc=exc, countdown=60)
from app.models import Photo
from app.models.tags import Tag, photo_tags
source_name = "vision:clip_classifier"
label = result.label
confidence = result.confidence
session = _get_sync_session()
try:
photo = session.execute(
select(Photo).where(Photo.id == photo_id)
).scalar_one_or_none()
if photo is None:
return {'status': 'error', 'message': 'photo not found'}
owner_id = photo.user_id
# Drop any previous classification for this photo.
session.execute(
delete(photo_tags).where(
photo_tags.c.photo_id == photo_id,
photo_tags.c.source == source_name,
)
)
tag = session.execute(
select(Tag).where(
Tag.name == label, Tag.kind == 'content_type', Tag.user_id == owner_id
)
).scalar_one_or_none()
if not tag:
tag = Tag(name=label, kind='content_type', source=source_name, user_id=owner_id)
session.add(tag)
session.flush()
session.execute(
photo_tags.insert().values(
photo_id=photo_id,
tag_id=tag.id,
confidence=confidence,
source=source_name,
)
)
session.execute(
update(Photo)
.where(Photo.id == photo_id)
.values(needs_review=(label == 'other'))
)
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
logger.info("[%s] Classified %s as %s (%.2f)", self.request.id, photo_id, label, confidence)
return {'status': 'success', 'photo_id': photo_id, 'label': label}
@shared_task(name='backfill_vision', bind=True, max_retries=10)
def backfill_vision(self, limit: int | None = None, **_ignored):
"""Queue classify_content for photos without a content_type tag."""
if not _vision_worker_ready():
logger.info("Vision worker not ready yet — retrying in 30s")
raise self.retry(countdown=30)
ordering = "ORDER BY p.taken_at DESC NULLS LAST, p.added_at DESC NULLS LAST"
limit_clause = " LIMIT :lim" if limit else ""
params: dict = {}
if limit:
params["lim"] = int(limit)
session = _get_sync_session()
try:
sql = f"""
SELECT p.id FROM photos p
WHERE p.processing_status = 'completed'
AND NOT EXISTS (
SELECT 1 FROM photo_tags pt
WHERE pt.photo_id = p.id
AND pt.source = 'vision:clip_classifier'
)
{ordering}{limit_clause}
"""
ids = [r[0] for r in session.execute(sa_text(sql), params).fetchall()]
finally:
session.close()
for pid in ids:
classify_content.delay(pid)
logger.info("Backfill queued %d photos for classification", len(ids))
return {'status': 'queued', 'count': len(ids)}

View File

@@ -1,54 +0,0 @@
"""Post-init_db bootstrap: run or stamp Alembic migrations.
On a FRESH Postgres install, init_db's create_all has already built the
full schema from the current models. Running `alembic upgrade head` would
fail because the older migrations try ADD COLUMN on columns that already
exist. So we detect the fresh-install case (alembic_version table is
missing or empty) and `stamp head` instead.
On an EXISTING install, the alembic_version table has a revision and
`upgrade head` applies only the new deltas.
"""
import subprocess
import sys
from sqlalchemy import create_engine, text, inspect
from app.config import settings
def run():
# Use a sync engine for this one-shot script.
sync_url = settings.database_url.replace("+asyncpg", "").replace("+aiosqlite", "")
engine = create_engine(sync_url)
with engine.connect() as conn:
inspector = inspect(engine)
tables = inspector.get_table_names()
if "alembic_version" not in tables:
# Fresh install — create_all built everything. Stamp head.
print("Fresh install detected — stamping alembic head")
subprocess.run(
[sys.executable, "-m", "alembic", "stamp", "head"],
check=True,
)
else:
row = conn.execute(text("SELECT version_num FROM alembic_version")).first()
if row is None:
print("Empty alembic_version — stamping head")
subprocess.run(
[sys.executable, "-m", "alembic", "stamp", "head"],
check=True,
)
else:
print(f"Existing install at revision {row[0]} — running alembic upgrade head")
subprocess.run(
[sys.executable, "-m", "alembic", "upgrade", "head"],
check=True,
)
engine.dispose()
if __name__ == "__main__":
run()

View File

@@ -1,63 +0,0 @@
# Core dependencies
fastapi==0.109.0
uvicorn[standard]==0.27.0
python-multipart==0.0.6
# Database
sqlalchemy[asyncio]==2.0.25
aiosqlite==0.19.0 # SQLite escape hatch (docker-compose.sqlite.yml override)
asyncpg==0.29.0 # async Postgres driver (default)
psycopg2-binary==2.9.9 # sync Postgres driver, used by Alembic CLI
alembic==1.13.1
# Redis and Celery
redis==5.0.1
celery==5.3.6
flower==2.0.1
# Image processing
# pyvips==2.2.1 # Optional - having compatibility issues, using Pillow as fallback
rawpy==0.26.1 # RAW decoder (CR2/NEF/ARW/DNG/…). cp312 wheels
# ship with libraw bundled; the older 0.19 pin
# had numpy 2.x incompatibilities — 0.26 is fine
# with our numpy 1.26. iPhone ProRAW-style DNGs
# that aren't real RAW still fail here; thumbs.py
# falls back to opening them as JPEG in that case.
pillow==10.2.0
pillow-heif==0.15.0
imagehash==4.3.1 # perceptual hash for duplicate detection
imageio==2.33.1
imageio-ffmpeg==0.4.9
# Video processing
ffmpeg-python==0.2.0
# Metadata extraction
pyexiftool==0.5.6
# File watching
watchfiles==0.21.0
# Vision pipeline (ONNX Runtime CPU inference)
onnxruntime==1.18.1
open-clip-torch==2.24.0 # tokenizer + export helper; inference via ONNX
numpy>=1.26.0,<2.0
# Utilities
pyyaml==6.0.1
pydantic==2.5.3
pydantic-settings==2.1.0
python-dotenv==1.0.0
httpx==0.26.0
aiofiles==23.2.1
# Security and authentication
python-jose[cryptography]==3.3.0
passlib[bcrypt]==1.7.4
bcrypt==4.0.1
# Development
pytest==7.4.4
pytest-asyncio==0.23.3
black==23.12.1
ruff==0.1.11

24
docker-compose.gpu.yml Normal file
View File

@@ -0,0 +1,24 @@
# Overlay for hosts with a VA-API-capable GPU passed through (Intel
# QSV, AMD VCN/VCE, any VA-API driver). PhotoPrism's :latest image
# ships VA-API-enabled ffmpeg; this file just wires the device + group
# membership + encoder selection. Layered in by the deploy script on
# hosts where /dev/dri/renderD128 exists.
#
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d
services:
photoprism:
devices:
- /dev/dri/renderD128:/dev/dri/renderD128
- /dev/dri/card0:/dev/dri/card0
# Match host GIDs (render=992, video=44 on Debian). PhotoPrism's
# container user (PP_UID:PP_GID, typically 33:10000) is not in
# these groups by default; group_add grants access to the device
# nodes without changing the primary user.
group_add:
- "992"
- "44"
environment:
PHOTOPRISM_FFMPEG_ENCODER: ${PP_FFMPEG_ENCODER:-vaapi}
PHOTOPRISM_FFMPEG_BITRATE: ${PP_FFMPEG_BITRATE:-32}

34
docker-compose.podman.yml Normal file
View File

@@ -0,0 +1,34 @@
# Podman-rootless overlay for the PhotoPrism stack.
#
# Apply alongside the base compose file:
# podman-compose --env-file .env \
# -f docker-compose.yml \
# -f docker-compose.podman.yml \
# up -d
#
# Adds the podman-specific bits that would break a vanilla docker compose run:
# - userns_mode: keep-id maps container UID to the invoking host UID, so
# PhotoPrism (running as PP_UID:PP_GID inside) can actually read the
# bind-mounted originals volume on the host (which is owned by the host
# user, not by uid 1000-in-the-container-namespace).
# - the explicit security_opt entries on the base file work in podman as-is.
services:
# MariaDB writes to a named volume managed by podman; its in-container
# `mysql` user expects to own that volume. keep-id breaks this by mapping
# in-container UID 999 to a podman-subuid that doesn't own the volume,
# so let mariadb use the default userns mapping (root-in-namespace).
mariadb:
# No userns_mode override — use podman defaults.
init: true
# PhotoPrism does need keep-id, so its container UID maps back to the
# host UID that owns the bind-mounted originals/.
photoprism:
userns_mode: keep-id
# Sidecar mutates the originals tree (rename / folder ops / heap
# convert / .duplicates archive) — same keep-id mapping so its writes
# land as the host user, not as a podman-subuid the host doesn't own.
sidecar:
userns_mode: keep-id

View File

@@ -1,50 +0,0 @@
# SQLite escape hatch override.
#
# Usage (omit the `db` service from the up command):
#
# docker compose -f docker-compose.yml -f docker-compose.sqlite.yml \
# up frontend backend worker redis
#
# This pins the backend and worker to the legacy SQLite database file at
# /data/db/mulita.db (in the existing db_data volume), drops the dependency
# on Postgres, and skips Alembic — the SQLite schema is still managed by
# the inline ALTERs in app/database.py:init_db.
#
# Vision features that depend on pgvector (PR4 onward) will refuse to enable
# in this mode; the search/embedding endpoints will return 503 with a clear
# error pointing back at the default Postgres setup.
services:
backend:
command: sh -c "uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload"
environment:
- DATABASE_URL=sqlite+aiosqlite:////data/db/mulita.db
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-*}
- SECRET_KEY=${SECRET_KEY:-mulita-dev-secret-change-me}
- ACCESS_TOKEN_EXPIRE_MINUTES=${ACCESS_TOKEN_EXPIRE_MINUTES:-60}
- REFRESH_TOKEN_EXPIRE_DAYS=${REFRESH_TOKEN_EXPIRE_DAYS:-30}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
depends_on:
redis:
condition: service_started
worker:
environment:
- DATABASE_URL=sqlite+aiosqlite:////data/db/mulita.db
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- CELERYD_CONCURRENCY=${CELERYD_CONCURRENCY:-4}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
depends_on:
redis:
condition: service_started
backend:
condition: service_started

View File

@@ -1,270 +1,209 @@
# Compose stack for the PhotoPrism-backed photo app: mariadb + photoprism +
# Go sidecar. The SvelteKit web/ frontend runs separately (Vite in dev,
# static build in prod) and proxies /api/v1/* to photoprism and
# /api/sidecar/* to the sidecar.
#
# podman-compose --env-file .env \
# -f docker-compose.yml -f docker-compose.podman.yml up -d
services:
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
container_name: mulita-frontend
mariadb:
# Fully-qualified for podman (which refuses short names by default).
# Docker resolves the same digest.
image: docker.io/library/mariadb:11
container_name: pp-mariadb
restart: unless-stopped
command:
- --innodb-buffer-pool-size=512M
- --transaction-isolation=READ-COMMITTED
- --character-set-server=utf8mb4
- --collation-server=utf8mb4_unicode_ci
- --max-connections=512
- --innodb-rollback-on-timeout=OFF
- --innodb-lock-wait-timeout=120
environment:
MARIADB_AUTO_UPGRADE: "1"
MARIADB_INITDB_SKIP_TZINFO: "1"
MARIADB_DATABASE: ${PP_DB_NAME:-photoprism}
MARIADB_USER: ${PP_DB_USER:-photoprism}
MARIADB_PASSWORD: ${PP_DB_PASSWORD:?set PP_DB_PASSWORD in .env}
MARIADB_ROOT_PASSWORD: ${PP_DB_ROOT_PASSWORD:?set PP_DB_ROOT_PASSWORD in .env}
# Loopback-only host port so the mule-sidecar (running as a host process
# in M4) can reach `mule_sidecar.*` over TCP. Not exposed beyond
# 127.0.0.1; the photoprism container still resolves mariadb by service
# name on the photoprism-network bridge.
ports:
# Host port is configurable via FRONTEND_PORT in .env so multiple
# instances / other services on the same host don't collide.
- "${FRONTEND_PORT:-3000}:80"
depends_on:
- backend
networks:
- mulita-network
restart: unless-stopped
backend:
build:
context: ./backend
dockerfile: Dockerfile
container_name: mulita-backend
ports:
# Direct backend access on the host is rarely needed (the frontend
# talks to it through the nginx /api proxy on the same network),
# but it's exposed for debugging / curl. Override with BACKEND_PORT.
- "${BACKEND_PORT:-8001}:8000"
- "127.0.0.1:${PP_DB_PORT:-3306}:3306"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
# The single host → container mount for your photo library. Set
# PHOTO_DIRS in .env to your library root. Mounted :rw because file
# operations (rename, move, empty discard pile) need to mutate the
# filesystem; flip to :ro for a strict read-only library and the
# write endpoints will return EROFS.
- ${PHOTO_DIRS:-./photos}:/photos:rw
- thumbs_data:/data/thumbs
- proxies_data:/data/proxies
- db_data:/data/db # retained so the docker-compose.sqlite.yml override has somewhere to put mulita.db
# Run Alembic migrations before starting uvicorn. On a fresh Postgres
# the empty 0001 baseline is a no-op stamp; create_all in init_db then
# builds the schema.
# init_db creates all tables from models (idempotent create_all),
# then Alembic runs migrations for existing installs. On fresh DBs
# create_all already built the full schema, so bootstrap.py stamps
# alembic head to skip redundant ALTER statements.
command: sh -c "python -c 'import asyncio; from app.database import init_db; asyncio.run(init_db())' && python bootstrap.py && uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload"
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=/photos
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-*}
- SECRET_KEY=${SECRET_KEY:-mulita-dev-secret-change-me}
- ACCESS_TOKEN_EXPIRE_MINUTES=${ACCESS_TOKEN_EXPIRE_MINUTES:-60}
- REFRESH_TOKEN_EXPIRE_DAYS=${REFRESH_TOKEN_EXPIRE_DAYS:-30}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
depends_on:
redis:
condition: service_started
db:
condition: service_healthy
networks:
- mulita-network
restart: unless-stopped
# ── Celery workers ─────────────────────────────────────────────────────
#
# The ingestion pipeline is split across two worker services so CPU-heavy
# vision tasks (embed / detect / OCR / faces / classify) cannot starve
# the fast IO-bound tasks (scan / thumbnails / EXIF / phash / duplicates).
#
# worker-light listens on default,high,low — IO-bound, cheap
# worker-vision listens on vision — CPU-bound, loads ONNX
#
# Both share the same image, photo volume, and model cache, so there's
# no disk duplication and model weights are loaded lazily only by
# worker-vision. Each service has its own concurrency knob; both
# workers ship their heartbeat to the same Redis broker so the
# Settings > Workers panel lists them side-by-side.
#
# Sizing defaults target a 6-core / 16 GB host:
# CELERY_LIGHT_CONCURRENCY=2 (enough for parallel thumbnail + EXIF)
# CELERY_VISION_CONCURRENCY=5 (5 × ~2GB ONNX = ~10GB RAM, 5/6 cores)
# Raise these in .env and run `docker compose up -d worker-light worker-vision`
# to scale. Keep light under ~4 and vision under your physical core
# count; more just thrashes.
worker-light:
build:
context: ./backend
dockerfile: Dockerfile
image: mule-image-worker
container_name: mulita-worker-light
command: sh -c "python -m app.services.vision.bootstrap_models && celery -A app.tasks.celery worker --loglevel=${LOG_LEVEL:-info} --concurrency=${CELERY_LIGHT_CONCURRENCY:-2} -Q default,high,low -n light@%h"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
- ${PHOTO_DIRS:-./photos}:/photos:rw
- thumbs_data:/data/thumbs
- proxies_data:/data/proxies
- db_data:/data/db
- models_data:/data/models
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=/photos
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
# NullPool — see app/database.py for rationale.
- MULITA_CELERY_WORKER=1
depends_on:
redis:
condition: service_started
backend:
condition: service_started
db:
condition: service_healthy
- pp_mariadb_data:/var/lib/mysql
# The init script creates the mule_sidecar database + user that the Go
# sidecar service will use in M4. Idempotent; no-op on subsequent boots.
# ":Z" is the SELinux private-relabel flag — needed on Fedora/RHEL hosts,
# silently no-op on Debian/Ubuntu and macOS Docker Desktop.
- ./mariadb/init:/docker-entrypoint-initdb.d:ro,Z
healthcheck:
test: ["CMD-SHELL", "celery -A app.tasks.celery inspect ping -d light@$$HOSTNAME 2>/dev/null | grep -q OK"]
interval: 30s
timeout: 10s
retries: 3
start_period: 120s
networks:
- mulita-network
restart: unless-stopped
# Dedicated watcher worker — runs the long-lived watch_folders task
# on its own queue so it never blocks scan/thumbnail workers.
worker-watcher:
build:
context: ./backend
dockerfile: Dockerfile
image: mule-image-worker
container_name: mulita-worker-watcher
command: sh -c "celery -A app.tasks.celery worker --loglevel=${LOG_LEVEL:-info} --concurrency=1 -Q watcher -n watcher@%h"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
- ${PHOTO_DIRS:-./photos}:/photos:rw
- db_data:/data/db
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=/photos
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
- MULITA_CELERY_WORKER=1
depends_on:
redis:
condition: service_started
db:
condition: service_healthy
networks:
- mulita-network
restart: unless-stopped
worker-vision:
build:
context: ./backend
dockerfile: Dockerfile
image: mule-image-worker
container_name: mulita-worker-vision
command: sh -c "python -m app.services.vision.bootstrap_models && celery -A app.tasks.celery worker --loglevel=${LOG_LEVEL:-info} --concurrency=${CELERY_VISION_CONCURRENCY:-5} -Q vision -n vision@%h"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
- ${PHOTO_DIRS:-./photos}:/photos:rw
- thumbs_data:/data/thumbs
- proxies_data:/data/proxies
- db_data:/data/db
- models_data:/data/models
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=/photos
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
- MULITA_CELERY_WORKER=1
# ONNX Runtime execution providers. Set to "auto" to auto-detect
# GPU (CUDA > ROCm > OpenVINO > CPU), or explicitly:
# "CUDAExecutionProvider,CPUExecutionProvider"
# "ROCMExecutionProvider,CPUExecutionProvider"
# Default: CPU only. To enable GPU, also uncomment the deploy
# section below and install nvidia-container-toolkit on the host.
- VISION_EXECUTION_PROVIDERS=${VISION_EXECUTION_PROVIDERS:-CPUExecutionProvider}
# Pin each ONNX session to one intra-op thread so N prefork children
# × default-all-cores doesn't oversubscribe the box. With
# concurrency=5 and OMP=1, vision peaks at 5 busy cores, leaving
# one for worker-light + system. These env vars cover the three
# threading runtimes ONNX Runtime might pick up on first use.
- OMP_NUM_THREADS=1
- OPENBLAS_NUM_THREADS=1
- MKL_NUM_THREADS=1
# Uncomment for NVIDIA GPU passthrough:
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: all
# capabilities: [gpu]
healthcheck:
test: ["CMD-SHELL", "celery -A app.tasks.celery inspect ping -d vision@$$HOSTNAME 2>/dev/null | grep -q OK"]
interval: 30s
timeout: 10s
retries: 3
start_period: 300s
depends_on:
redis:
condition: service_started
backend:
condition: service_started
db:
condition: service_healthy
networks:
- mulita-network
restart: unless-stopped
db:
image: pgvector/pgvector:pg16
container_name: mulita-db
environment:
POSTGRES_USER: mulita
POSTGRES_PASSWORD: mulita
POSTGRES_DB: mulita
volumes:
- pg_data:/var/lib/postgresql/data
networks:
- mulita-network
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -U mulita -d mulita"]
interval: 5s
timeout: 5s
retries: 10
redis:
image: redis:7-alpine
container_name: mulita-redis
# Host port exposed only for local debugging; the backend / worker
# reach Redis via the internal mulita-network on its container name.
ports:
- "${REDIS_PORT:-6379}:6379"
volumes:
- redis_data:/data
networks:
- mulita-network
restart: unless-stopped
command: redis-server --appendonly yes
healthcheck:
test: ["CMD", "redis-cli", "ping"]
test: ["CMD", "/usr/bin/mariadb-admin", "ping", "-h", "127.0.0.1", "--silent"]
interval: 10s
timeout: 5s
retries: 5
retries: 12
start_period: 60s
networks: [photoprism-network]
photoprism:
image: docker.io/photoprism/photoprism:latest
container_name: pp-app
restart: unless-stopped
depends_on:
mariadb:
condition: service_healthy
# PhotoPrism's container drops to a non-root user via PHOTOPRISM_UID /
# PHOTOPRISM_GID. Match the host user that owns ${PHOTO_DIRS} so the
# process can read originals (and later write sidecars).
user: "${PP_UID:-1000}:${PP_GID:-1000}"
security_opt:
- seccomp:unconfined
- apparmor:unconfined
ports:
# Loopback only — the SvelteKit web/ app (Vite dev or built bundle)
# is the user-facing surface; PhotoPrism's own UI stays off the
# public interface. Vite proxies /api/v1/* here, and the host-mode
# sidecar reaches PHOTOPRISM_BASE_URL=http://localhost:2342. Admin
# access to PP's UI is via SSH tunnel only.
- "127.0.0.1:${PP_PORT:-2342}:2342"
environment:
PHOTOPRISM_ADMIN_USER: ${PP_ADMIN_USER:-admin}
PHOTOPRISM_ADMIN_PASSWORD: ${PP_ADMIN_PASSWORD:?set PP_ADMIN_PASSWORD in .env}
PHOTOPRISM_AUTH_MODE: ${PP_AUTH_MODE:-password}
PHOTOPRISM_SITE_URL: ${PP_SITE_URL:-http://localhost:2342/}
PHOTOPRISM_ORIGINALS_LIMIT: ${PP_ORIGINALS_LIMIT:-50000}
PHOTOPRISM_HTTP_COMPRESSION: gzip
PHOTOPRISM_LOG_LEVEL: ${PP_LOG_LEVEL:-info}
# Indexer concurrency. Defaults to NumCPU/2 (= 3 on a 6-core LXC),
# but each worker forks TF + ffmpeg + libvips so effective load is
# much higher — a fresh index of 1.2k photos on M0 pushed the LXC
# load to 50+ and starved sibling containers. Pin to a low value
# for shared hosts; raise on dedicated machines.
PHOTOPRISM_WORKERS: ${PP_WORKERS:-2}
# podman-compose doesn't expand nested ${A:-${B:-…}}, so keep this
# one-level. Override both PP_WORKERS and PP_INDEX_WORKERS if you
# want them to differ.
PHOTOPRISM_INDEX_WORKERS: ${PP_INDEX_WORKERS:-2}
# M0 safety: keep originals read-only. Flip to "false" in M2 when the
# right-sidebar enables metadata edits and we want EXIF backwrite.
PHOTOPRISM_READONLY: ${PP_READONLY:-true}
PHOTOPRISM_EXPERIMENTAL: "false"
PHOTOPRISM_DISABLE_CHOWN: "true"
PHOTOPRISM_DISABLE_WEBDAV: ${PP_DISABLE_WEBDAV:-false}
PHOTOPRISM_DISABLE_SETTINGS: "false"
PHOTOPRISM_DISABLE_TLS: "true"
PHOTOPRISM_DEFAULT_TLS: "false"
# AI/vision pipeline back on — per plan we re-introduce TF labels + faces.
PHOTOPRISM_TENSORFLOW_OFF: "false"
PHOTOPRISM_DETECT_NSFW: "true"
PHOTOPRISM_UPLOAD_NSFW: "true"
# Database
PHOTOPRISM_DATABASE_DRIVER: mysql
PHOTOPRISM_DATABASE_SERVER: mariadb:3306
PHOTOPRISM_DATABASE_NAME: ${PP_DB_NAME:-photoprism}
PHOTOPRISM_DATABASE_USER: ${PP_DB_USER:-photoprism}
PHOTOPRISM_DATABASE_PASSWORD: ${PP_DB_PASSWORD}
# Sidecars next to originals — read by the migrator at M5.
PHOTOPRISM_SIDECAR_PATH: ""
PHOTOPRISM_SIDECAR_YAML: "true"
# EXIF backwrite — disabled in M0 (READONLY blocks writes anyway).
# Override in .env: PP_BACKUP_DATABASE=true.
PHOTOPRISM_DISABLE_BACKUPS: "false"
PHOTOPRISM_BACKUP_DATABASE: ${PP_BACKUP_DATABASE:-true}
PHOTOPRISM_DISABLE_EXIFTOOL: "false"
# OIDC — set in .env when the IdP (Authentik) is wired up.
# Empty values keep OIDC dormant; the username/password login still works.
# PhotoPrism's CLI flags are --oidc-uri / --oidc-client / --oidc-secret
# / --oidc-provider, so the env-var names it actually reads are
# PHOTOPRISM_OIDC_URI / _CLIENT / _SECRET / _PROVIDER (NOT _ISSUER_URL
# / _CLIENT_ID / _CLIENT_SECRET / _PROVIDER_NAME — those are silently
# ignored, OIDC stays dormant, and `photoprism show config` reports
# blank oidc-uri / oidc-client). PHOTOPRISM_OIDC_REDIRECT is a bool
# (auto-redirect-from-/library/login), not a URL — PhotoPrism builds
# the callback from PHOTOPRISM_SITE_URL.
PHOTOPRISM_OIDC_PROVIDER: ${OIDC_PROVIDER_NAME:-${OIDC_PROVIDER:-}}
PHOTOPRISM_OIDC_URI: ${OIDC_ISSUER_URL:-${OIDC_URI:-}}
PHOTOPRISM_OIDC_CLIENT: ${OIDC_CLIENT_ID:-${OIDC_CLIENT:-}}
PHOTOPRISM_OIDC_SECRET: ${OIDC_CLIENT_SECRET:-${OIDC_SECRET:-}}
PHOTOPRISM_OIDC_SCOPES: ${OIDC_SCOPES:-openid profile email}
PHOTOPRISM_OIDC_REGISTER: ${OIDC_REGISTER:-true}
PHOTOPRISM_OIDC_ROLE: ${OIDC_ROLE:-user}
PHOTOPRISM_OIDC_REDIRECT: ${OIDC_REDIRECT:-false}
working_dir: /photoprism
volumes:
# Existing photo library — mounted read-only in M0; flip to :rw in M2
# when the right-sidebar starts saving edits. ",Z" relabels for SELinux
# on Fedora/RHEL; silent no-op elsewhere.
- "${PHOTO_DIRS:?set PHOTO_DIRS in .env}:/photoprism/originals:${PP_ORIGINALS_MODE:-ro},Z"
- "./pp/storage:/photoprism/storage:Z"
- "./pp/import:/photoprism/import:Z"
networks: [photoprism-network]
# mule-sidecar — Go + Gin + GORM service for endpoints PhotoPrism's API
# does not expose (file rename, folder mutations, heap convert, duplicate
# scan, per-photo marks). Same wire contract as the M3 Node prototype;
# the SvelteKit dev server proxies /api/sidecar/* here.
sidecar:
build:
context: ./sidecar
container_name: pp-sidecar
restart: unless-stopped
depends_on:
mariadb:
condition: service_healthy
photoprism:
condition: service_started
# Match PhotoPrism's UID/GID so renames/folder mutations preserve the
# ownership the indexer expects on the bind-mounted originals.
user: "${PP_UID:-1000}:${PP_GID:-1000}"
ports:
# Loopback only — Vite (host) proxies /api/sidecar/* to this port.
# Behind a reverse proxy in production; never published beyond the
# host.
- "127.0.0.1:${SIDECAR_PORT:-8000}:8000"
environment:
ORIGINALS_ROOT: /photoprism/originals
PHOTOPRISM_BASE_URL: http://photoprism:2342
# Bind on all interfaces inside the container so the host-side
# 127.0.0.1:8000 port mapping can reach the listener. The Go
# binary defaults to 127.0.0.1 for the host-mode dev loop.
SIDECAR_LISTEN_ADDR: 0.0.0.0
SIDECAR_PORT: "8000"
SIDECAR_DB_HOST: mariadb
SIDECAR_DB_PORT: "3306"
SIDECAR_DB_USER: sidecar
# Rotate before any non-local deployment. Provisioned by
# mariadb/init/01-sidecar.sql on first boot of the mariadb volume.
SIDECAR_DB_PASSWORD: ${SIDECAR_DB_PASSWORD:-replace-at-m4-bringup}
SIDECAR_DB_NAME: mule_sidecar
# Second DB connection for poking PhotoPrism's own schema (only
# used by the user-basepath reconciler today). Stays inert if
# PP_DB_PASSWORD is empty — the reconciler then silently no-ops.
PP_DB_HOST: mariadb
PP_DB_PORT: "3306"
PP_DB_USER: ${PP_DB_USER:-photoprism}
PP_DB_PASSWORD: ${PP_DB_PASSWORD:-}
PP_DB_NAME: ${PP_DB_NAME:-photoprism}
# Declarative username → originals-relative BasePath mapping.
# Format: comma-separated `user:path` pairs. Sidecar applies it
# to auth_users on boot and every 60s, and `mkdir -p`s each
# target subdirectory so PhotoPrism's ACL filter has somewhere to
# point. Leave empty to disable.
# USER_BASEPATHS="test:test, alice:family/alice"
USER_BASEPATHS: ${USER_BASEPATHS:-}
volumes:
# Sidecar mutates originals (rename, folder mutations, heap
# convert) — always rw regardless of PhotoPrism's mount mode.
- "${PHOTO_DIRS:?set PHOTO_DIRS in .env}:/photoprism/originals:rw,Z"
networks: [photoprism-network]
networks:
mulita-network:
photoprism-network:
driver: bridge
volumes:
thumbs_data:
proxies_data:
db_data:
redis_data:
pg_data:
models_data:
pp_mariadb_data:

View File

@@ -1,31 +0,0 @@
# Build stage
FROM node:18-alpine as build
WORKDIR /app
# Copy package files
COPY package*.json ./
# Install dependencies
RUN npm ci
# Copy source code
COPY . .
# Build the application
RUN npm run build
# Production stage
FROM nginx:alpine
# Copy built assets from build stage
COPY --from=build /app/dist /usr/share/nginx/html
# Copy nginx configuration
COPY nginx.conf /etc/nginx/conf.d/default.conf
# Expose port
EXPOSE 80
# Start nginx
CMD ["nginx", "-g", "daemon off;"]

View File

@@ -1,20 +0,0 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "default",
"rsc": false,
"tsx": true,
"tailwind": {
"config": "tailwind.config.js",
"css": "src/index.css",
"baseColor": "zinc",
"cssVariables": false,
"prefix": ""
},
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"hooks": "@/hooks",
"lib": "@/lib"
}
}

View File

@@ -1,15 +0,0 @@
<!doctype html>
<html lang="en" class="dark">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/png" href="/favicon.png" />
<link rel="apple-touch-icon" href="/favicon.png" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="theme-color" content="#0f0f0f" />
<title>Mulimago</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>

View File

@@ -1,54 +0,0 @@
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
# Enable gzip
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/xml+rss application/json;
# API proxy
location /api/ {
proxy_pass http://backend:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket support for real-time updates
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# Thumbnail serving with X-Accel-Redirect
location /internal_thumbs/ {
internal;
alias /data/thumbs/;
}
# SPA routing - serve index.html for all routes
location / {
try_files $uri $uri/ /index.html;
}
# Never cache index.html (or any HTML). The asset filenames are
# content-hashed by Vite, so a fresh index.html is the only thing
# that tells the browser to fetch the new bundle. Without this the
# browser happily serves a stale index.html → stale bundle hash →
# users see the old build until they hard-reload.
location = /index.html {
add_header Cache-Control "no-cache, no-store, must-revalidate";
add_header Pragma "no-cache";
expires 0;
}
# Cache static assets (filenames are content-hashed, so 1y is safe)
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,73 +0,0 @@
{
"name": "mulita-frontend",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"preview": "vite preview",
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0"
},
"dependencies": {
"@radix-ui/react-accordion": "^1.1.2",
"@radix-ui/react-alert-dialog": "^1.0.5",
"@radix-ui/react-checkbox": "^1.0.4",
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-context-menu": "^2.1.5",
"@radix-ui/react-dialog": "^1.0.5",
"@radix-ui/react-dropdown-menu": "^2.0.6",
"@radix-ui/react-label": "^2.0.2",
"@radix-ui/react-popover": "^1.0.7",
"@radix-ui/react-radio-group": "^1.3.8",
"@radix-ui/react-scroll-area": "^1.0.5",
"@radix-ui/react-select": "^2.0.0",
"@radix-ui/react-separator": "^1.0.3",
"@radix-ui/react-slider": "^1.1.2",
"@radix-ui/react-slot": "^1.2.4",
"@radix-ui/react-switch": "^1.0.3",
"@radix-ui/react-tabs": "^1.0.4",
"@radix-ui/react-toast": "^1.1.5",
"@radix-ui/react-toggle": "^1.1.10",
"@radix-ui/react-toggle-group": "^1.1.11",
"@radix-ui/react-tooltip": "^1.0.7",
"@tanstack/react-query": "^5.17.0",
"@tanstack/react-virtual": "^3.0.1",
"axios": "^1.6.5",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.0",
"cmdk": "^1.1.1",
"date-fns": "^3.2.0",
"framer-motion": "^10.18.0",
"leaflet": "^1.9.4",
"lucide-react": "^0.303.0",
"react": "^18.2.0",
"react-day-picker": "^8.10.1",
"react-dom": "^18.2.0",
"react-hotkeys-hook": "^4.4.3",
"react-intersection-observer": "^9.5.3",
"react-leaflet": "^4.2.1",
"react-leaflet-cluster": "^2.1.0",
"sonner": "^2.0.7",
"tailwind-merge": "^2.2.0",
"tailwindcss-animate": "^1.0.7",
"zustand": "^4.4.7"
},
"devDependencies": {
"@tanstack/react-query-devtools": "^5.96.2",
"@types/leaflet": "^1.9.8",
"@types/react": "^18.2.46",
"@types/react-dom": "^18.2.18",
"@typescript-eslint/eslint-plugin": "^6.17.0",
"@typescript-eslint/parser": "^6.17.0",
"@vitejs/plugin-react": "^4.2.1",
"autoprefixer": "^10.4.16",
"eslint": "^8.56.0",
"eslint-plugin-react-hooks": "^4.6.0",
"eslint-plugin-react-refresh": "^0.4.5",
"postcss": "^8.4.33",
"tailwindcss": "^3.4.0",
"typescript": "^5.3.3",
"vite": "^5.0.10"
}
}

View File

@@ -1,6 +0,0 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}

View File

@@ -1,174 +0,0 @@
import { useEffect, useRef, useState } from 'react'
import { Timeline } from './components/timeline/Timeline'
import { DuplicatesView } from './components/duplicates/DuplicatesView'
import { MapView } from './components/map/MapView'
import { MemoriesView } from './components/memories/MemoriesView'
import { TagsView } from './components/tags/TagsView'
import { ColorsView } from './components/colors/ColorsView'
import { RatedView } from './components/rated/RatedView'
import { LeftSidebar } from './components/layout/LeftSidebar'
import { RightSidebar } from './components/layout/RightSidebar'
import { TopBar } from './components/layout/TopBar'
import { ScanProgress } from './components/ScanProgress'
import { ToastContainer } from './components/ToastContainer'
import { KeyboardHints } from './components/KeyboardHints'
import { PreviewView } from './components/preview/PreviewView'
import { FilterBar } from './components/filter/FilterBar'
import { DiscardActionBar } from './components/discard/DiscardActionBar'
import { SettingsPage } from './components/dialogs/SettingsDialog'
import { usePhotoStore } from './store/photoStore'
import { useFilterStore } from './store/filterStore'
import { useKeyboardShortcuts } from './hooks/useKeyboardShortcuts'
import { useFilterUrlSync } from './hooks/useFilterUrlSync'
import { usePhotosQuery } from './hooks/usePhotosQuery'
import { AuthProvider, useAuth } from './contexts/AuthContext'
import { LoginPage } from './components/auth/LoginPage'
import { SetupPage } from './components/auth/SetupPage'
import { TooltipProvider } from '@/components/ui/tooltip'
function MainApp() {
const [leftSidebarOpen, setLeftSidebarOpen] = useState(true)
const [rightSidebarOpen, setRightSidebarOpen] = useState(true)
const viewMode = usePhotoStore((state) => state.viewMode)
const activePhotoId = usePhotoStore((state) => state.activePhotoId)
const currentSection = useFilterStore((s) => s.currentSection)
// Close the metadata panel when the user switches between sections so
// it doesn't carry over a now-irrelevant selection. It re-opens once a
// photo gains focus in the new section (effect below).
const prevSectionRef = useRef(currentSection)
useEffect(() => {
if (prevSectionRef.current !== currentSection) {
prevSectionRef.current = currentSection
setRightSidebarOpen(false)
}
}, [currentSection])
useEffect(() => {
setRightSidebarOpen(!!activePhotoId)
}, [activePhotoId])
// Bidirectional sync of filter store with URL query params.
useFilterUrlSync()
// Subscribe to the same photos query the Timeline uses, so the keyboard
// "open preview on first photo" path can read from the live cache regardless
// of what filter key it's stored under.
const { data: allPhotos } = usePhotosQuery()
// Set up global keyboard shortcuts
useKeyboardShortcuts({
onToggleLeftSidebar: () => setLeftSidebarOpen(!leftSidebarOpen),
onToggleRightSidebar: () => setRightSidebarOpen(!rightSidebarOpen),
getFirstPhotoId: () => allPhotos?.[0]?.id ?? null,
})
// Settings page is a full-page section — hide filter bar, right sidebar,
// and keyboard hints when it's active.
const isSettings = currentSection === 'settings'
// Right sidebar stays open by default and shows whatever's selected
// (or an empty state if nothing is). User can still toggle it manually.
const showRightSidebar = rightSidebarOpen && !isSettings
return (
<TooltipProvider delayDuration={300}>
<div className="flex flex-col h-screen bg-bg text-text">
<TopBar />
<div className="flex flex-1 overflow-hidden">
{/* Left Sidebar */}
<div
className={`transition-all duration-200 ${
leftSidebarOpen ? 'w-60' : 'w-0'
} overflow-hidden border-r border-border bg-surface`}
>
<LeftSidebar />
</div>
{/* Main column — filter bar, discard bar, timeline. Lives to the
* right of the left sidebar so the filter row doesn't bleed
* across the sidebar. relative so the KeyboardHints overlay
* centers against this column, not the viewport. */}
<div className="relative flex min-w-0 flex-1 flex-col">
{!isSettings && (
<FilterBar
leftSidebarOpen={leftSidebarOpen}
rightSidebarOpen={showRightSidebar}
onToggleLeftSidebar={() => setLeftSidebarOpen(!leftSidebarOpen)}
onToggleRightSidebar={() => setRightSidebarOpen(!rightSidebarOpen)}
/>
)}
{!isSettings && <DiscardActionBar />}
<div className="flex-1 overflow-auto">
{currentSection === 'settings' ? (
<SettingsPage />
) : currentSection === 'map' ? (
<MapView />
) : currentSection === 'memories' ? (
<MemoriesView />
) : currentSection === 'duplicates' ? (
<DuplicatesView />
) : currentSection === 'tags' ? (
<TagsView />
) : currentSection === 'colors' ? (
<ColorsView />
) : currentSection === 'rated' ? (
<RatedView />
) : (
<Timeline />
)}
</div>
{!isSettings && viewMode !== 'preview' && <KeyboardHints />}
</div>
{/* Right Sidebar */}
<div
className={`transition-all duration-200 ${
showRightSidebar ? 'w-72' : 'w-0'
} overflow-hidden border-l border-border bg-surface`}
>
<RightSidebar />
</div>
</div>
{/* Scan Progress Indicator */}
<ScanProgress />
{/* Toast Notifications */}
<ToastContainer />
{/* Preview overlay — covers TopBar when active */}
{viewMode === 'preview' && <PreviewView />}
</div>
</TooltipProvider>
)
}
/** Auth-gated shell: shows setup, login, or the main app. */
function App() {
return (
<AuthProvider>
<AuthGate />
</AuthProvider>
)
}
function AuthGate() {
const { user, isLoading, needsSetup } = useAuth()
if (isLoading) {
return (
<div className="flex min-h-screen items-center justify-center bg-bg">
<div className="text-text-muted">Loading&hellip;</div>
</div>
)
}
if (needsSetup) return <SetupPage />
if (!user) return <LoginPage />
return <MainApp />
}
export default App

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 821 KiB

View File

@@ -1,143 +0,0 @@
import { useEffect, useState } from 'react'
import { useHotkeys } from 'react-hotkeys-hook'
import { ChevronDown, ChevronUp } from 'lucide-react'
import clsx from 'clsx'
import { usePhotoStore } from '../store/photoStore'
import { useFilterStore } from '../store/filterStore'
const STORAGE_KEY = 'keyboard-hints-collapsed'
interface Hint {
key: string
action: string
}
/** Build the hint list for the current context. Returns an empty array
* when no shortcuts apply, which lets the caller hide the panel
* entirely instead of rendering an empty pill. */
function getHints(opts: {
selectedCount: number
currentSection: string
viewMode: string
}): Hint[] {
const { selectedCount, currentSection, viewMode } = opts
// Preview mode: culling shortcuts apply to the photo on screen, plus
// arrow nav between photos and Esc to close.
if (viewMode === 'preview') {
const preview: Hint[] = [
{ key: '←→', action: 'Navigate' },
{ key: '1-5', action: 'Rate' },
{ key: 'S', action: 'Select → heap' },
]
if (currentSection === 'discarded') {
preview.push({ key: 'U', action: 'Restore' })
} else {
preview.push({ key: 'X', action: 'Discard' })
}
preview.push(
{ key: 'I', action: 'Info panel' },
{ key: 'Space', action: 'Close' },
{ key: 'Esc', action: 'Close' }
)
return preview
}
if (selectedCount > 0) {
const base: Hint[] = [
{ key: '1-5', action: 'Rate' },
{ key: 'S', action: 'Select → heap' },
]
if (currentSection === 'discarded') {
base.push({ key: 'U', action: 'Restore' })
} else {
base.push({ key: 'X', action: 'Discard' })
}
base.push(
{ key: 'Space', action: 'Preview' },
{ key: 'I', action: 'Info panel' },
{ key: 'Esc', action: 'Deselect' }
)
return base
}
return [
{ key: '↑↓←→', action: 'Navigate' },
{ key: 'Space', action: 'Preview' },
{ key: 'Tab', action: 'Library panel' },
{ key: 'I', action: 'Info panel' },
{ key: '/', action: 'Search' },
]
}
export function KeyboardHints() {
const selectedCount = usePhotoStore((s) => s.selectedPhotos.length)
const viewMode = usePhotoStore((s) => s.viewMode)
const currentSection = useFilterStore((s) => s.currentSection)
const [collapsed, setCollapsed] = useState(
() => typeof window !== 'undefined' && localStorage.getItem(STORAGE_KEY) === '1'
)
useEffect(() => {
localStorage.setItem(STORAGE_KEY, collapsed ? '1' : '0')
}, [collapsed])
// `H` toggles the panel. `?` (shift+/) collides with the global `/`
// search shortcut, so we use a plain letter instead.
useHotkeys('h', () => setCollapsed((c) => !c), { preventDefault: true })
const hints = getHints({ selectedCount, currentSection, viewMode })
// Nothing relevant to show — hide entirely.
if (hints.length === 0) return null
return (
<div className="pointer-events-none absolute bottom-0 left-1/2 z-30 -translate-x-1/2 pb-4">
{collapsed ? (
// Collapsed handle: a small pill peeking from the bottom so the
// user can re-open the panel without remembering the shortcut.
<button
type="button"
onClick={() => setCollapsed(false)}
className="pointer-events-auto flex items-center gap-1.5 rounded-full border border-white/15 bg-black/80 px-3 py-1 text-[11px] text-white/80 shadow-xl backdrop-blur-md transition-colors hover:bg-black/90 hover:text-white"
title="Show shortcuts (H)"
>
<ChevronUp className="h-3 w-3" />
Shortcuts
<kbd className="rounded bg-white/15 px-1 py-0.5 text-[10px] font-medium text-white">
H
</kbd>
</button>
) : (
<div
className={clsx(
'pointer-events-auto flex items-center gap-3 whitespace-nowrap rounded-full border border-white/15 bg-black/80 px-4 py-1.5 shadow-xl ring-1 ring-black/40 backdrop-blur-md'
)}
>
{hints.map((hint, i) => (
<div key={i} className="flex items-center gap-1.5">
<kbd className="rounded bg-white/15 px-1.5 py-0.5 text-[11px] font-medium text-white shadow-sm">
{hint.key}
</kbd>
<span className="whitespace-nowrap text-xs text-white/85">
{hint.action}
</span>
<span className="ml-1 text-white/30"></span>
</div>
))}
<button
type="button"
onClick={() => setCollapsed(true)}
className="-mr-1 flex items-center gap-1 rounded-full px-1.5 py-0.5 text-[11px] text-white/60 transition-colors hover:bg-white/10 hover:text-white"
title="Hide shortcuts (H)"
>
<kbd className="rounded bg-white/15 px-1 py-0.5 text-[10px] font-medium text-white">
H
</kbd>
<ChevronDown className="h-3 w-3" />
</button>
</div>
)}
</div>
)
}

View File

@@ -1,101 +0,0 @@
import { useEffect, useRef } from 'react'
import { useQuery, useQueryClient } from '@tanstack/react-query'
import { library, WorkerStatus } from '../services/api'
interface ScanStatus {
is_scanning: boolean
current_folder?: string
processed_files: number
total_files: number
errors: string[]
}
type Phase = 'idle' | 'scanning' | 'processing' | 'done'
/**
* Headless background-activity orchestrator. Polls scan + worker status
* and invalidates affected query caches when a scan/processing pass
* completes. The visible status indicator now lives inline in the
* LeftSidebar (small spinner next to the FOLDERS header / specific
* folder rows) — see useScanActivity.
*/
export function ScanProgress() {
const queryClient = useQueryClient()
const wasScanningRef = useRef(false)
const wasProcessingRef = useRef(false)
const { data: scanStatus } = useQuery<ScanStatus>({
queryKey: ['scan-status'],
queryFn: () => library.scanStatus(),
refetchInterval: (query) =>
query.state.data?.is_scanning ? 2000 : 10000,
enabled: true,
})
const isScanning = scanStatus?.is_scanning ?? false
// Poll worker status to track vision queue activity.
// Fast polling (3s) while processing, slow (15s) otherwise.
const { data: workerStatus } = useQuery<WorkerStatus>({
queryKey: ['worker-status-progress'],
queryFn: () => library.maintenance.workerStatus(),
refetchInterval: (query) => {
const q = totalQueued(query.state.data)
return q > 0 ? 3000 : 15000
},
enabled: true,
})
const totalActive = totalQueued(workerStatus)
const phase: Phase = isScanning
? 'scanning'
: totalActive > 0
? 'processing'
: 'idle'
useEffect(() => {
if (phase === 'scanning') {
wasScanningRef.current = true
wasProcessingRef.current = false
} else if (phase === 'processing') {
wasProcessingRef.current = true
if (wasScanningRef.current) {
wasScanningRef.current = false
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: ['folders'] })
queryClient.invalidateQueries({ queryKey: ['folders', 'tree'] })
queryClient.invalidateQueries({ queryKey: ['heaps'] })
queryClient.invalidateQueries({ queryKey: ['tags'] })
queryClient.invalidateQueries({ queryKey: ['library', 'stats'] })
}
} else if (phase === 'idle') {
if (wasScanningRef.current) {
wasScanningRef.current = false
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: ['folders'] })
queryClient.invalidateQueries({ queryKey: ['folders', 'tree'] })
queryClient.invalidateQueries({ queryKey: ['heaps'] })
queryClient.invalidateQueries({ queryKey: ['tags'] })
queryClient.invalidateQueries({ queryKey: ['library', 'stats'] })
}
if (wasProcessingRef.current) {
wasProcessingRef.current = false
queryClient.invalidateQueries({ queryKey: ['tags'] })
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: ['library', 'stats'] })
}
}
}, [phase, queryClient])
return null
}
function totalQueued(ws: WorkerStatus | undefined): number {
if (!ws) return 0
const queued = Object.values(ws.queues ?? {}).reduce((a, b) => a + b, 0)
const active = ws.workers?.reduce(
(sum, w) => sum + (w.active ?? 0) + (w.reserved ?? 0), 0
) ?? 0
return queued + active
}

View File

@@ -1,34 +0,0 @@
import { toast as sonnerToast } from 'sonner'
import { Toaster } from '@/components/ui/sonner'
export interface ToastAction {
label: string
onClick: () => void
}
/** Shim that preserves the legacy `(title, message?, action?)` call
* shape used throughout the codebase while delegating to sonner for
* actual rendering. Call sites don't need to change. Actions auto-
* extend the toast duration to 8s so users have time to hit Undo. */
const build = (message?: string, action?: ToastAction, duration = 5000) => ({
description: message,
action: action && { label: action.label, onClick: action.onClick },
duration: action ? Math.max(duration, 8000) : duration,
})
export const toast = {
success: (title: string, message?: string, action?: ToastAction) =>
sonnerToast.success(title, build(message, action)),
error: (title: string, message?: string, action?: ToastAction) =>
sonnerToast.error(title, build(message, action)),
info: (title: string, message?: string, action?: ToastAction) =>
sonnerToast.info(title, build(message, action)),
warning: (title: string, message?: string, action?: ToastAction) =>
sonnerToast.warning(title, build(message, action)),
}
/** Mounted once near the App root. Delegates to sonner's `<Toaster />`
* with palette-matched class overrides (see `@/components/ui/sonner`). */
export function ToastContainer() {
return <Toaster />
}

View File

@@ -1,371 +0,0 @@
import { useState, useEffect, useCallback } from 'react'
import { Plus, Pencil, UserX, Shield, User as UserIcon } from 'lucide-react'
import { admin, type AdminUser } from '../../services/api'
import {
Dialog,
DialogContent,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select'
import { Alert, AlertDescription } from '@/components/ui/alert'
import { ConfirmDialog } from '../dialogs/ConfirmDialog'
export function UserManagement() {
const [users, setUsers] = useState<AdminUser[]>([])
const [loading, setLoading] = useState(true)
const [showCreate, setShowCreate] = useState(false)
const [editingUser, setEditingUser] = useState<AdminUser | null>(null)
const [deactivatingUser, setDeactivatingUser] = useState<AdminUser | null>(null)
const [error, setError] = useState<string | null>(null)
const fetchUsers = useCallback(async () => {
try {
const data = await admin.listUsers()
setUsers(data.users)
} catch {
setError('Failed to load users.')
} finally {
setLoading(false)
}
}, [])
useEffect(() => {
fetchUsers()
}, [fetchUsers])
const handleDeactivate = async () => {
if (!deactivatingUser) return
try {
await admin.deleteUser(deactivatingUser.id)
setDeactivatingUser(null)
fetchUsers()
} catch (err: any) {
setError(err.response?.data?.detail ?? 'Failed to deactivate user.')
setDeactivatingUser(null)
}
}
if (loading) {
return <div className="p-4 text-sm text-text-muted">Loading users&hellip;</div>
}
return (
<div className="space-y-4">
<div className="flex items-center justify-between">
<h3 className="text-sm font-semibold text-text">Users</h3>
<Button size="sm" onClick={() => setShowCreate(true)}>
<Plus className="mr-1 h-3 w-3" />
Add User
</Button>
</div>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
<table className="w-full text-xs">
<thead>
<tr className="border-b border-border text-left text-text-muted">
<th className="pb-1 pr-4">Username</th>
<th className="pb-1 pr-4">Role</th>
<th className="pb-1 pr-4">Photos</th>
<th className="pb-1 pr-4">Status</th>
<th className="pb-1">Actions</th>
</tr>
</thead>
<tbody>
{users.map((u) => (
<tr key={u.id} className="border-b border-border/50">
<td className="py-1.5 pr-4">
<div className="flex items-center gap-1.5">
{u.role === 'admin' ? (
<Shield className="h-3 w-3 text-accent" />
) : (
<UserIcon className="h-3 w-3 text-text-muted" />
)}
<span className="text-text">{u.username}</span>
</div>
</td>
<td className="py-1.5 pr-4 text-text-muted">{u.role}</td>
<td className="py-1.5 pr-4 text-text-muted">
{u.photo_count.toLocaleString()}
</td>
<td className="py-1.5 pr-4">
<span
className={
u.is_active
? 'text-green-400'
: 'text-red-400'
}
>
{u.is_active ? 'Active' : 'Inactive'}
</span>
</td>
<td className="py-1.5">
<div className="flex gap-1">
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
onClick={() => setEditingUser(u)}
title="Edit user"
>
<Pencil className="h-3 w-3" />
</Button>
{u.is_active && (
<Button
variant="ghost"
size="icon"
className="h-6 w-6 hover:text-red-400"
onClick={() => setDeactivatingUser(u)}
title="Deactivate user"
>
<UserX className="h-3 w-3" />
</Button>
)}
</div>
</td>
</tr>
))}
</tbody>
</table>
<CreateUserModal
open={showCreate}
onClose={() => setShowCreate(false)}
onCreated={() => {
setShowCreate(false)
fetchUsers()
}}
/>
<EditUserModal
user={editingUser}
onClose={() => setEditingUser(null)}
onSaved={() => {
setEditingUser(null)
fetchUsers()
}}
/>
<ConfirmDialog
isOpen={!!deactivatingUser}
title={`Deactivate "${deactivatingUser?.username}"?`}
message="Their photos will be preserved."
confirmLabel="Deactivate"
destructive
onConfirm={handleDeactivate}
onClose={() => setDeactivatingUser(null)}
/>
</div>
)
}
// ── Create User Modal ──────────────────────────────────────────────────
function CreateUserModal({
open,
onClose,
onCreated,
}: {
open: boolean
onClose: () => void
onCreated: () => void
}) {
const [username, setUsername] = useState('')
const [password, setPassword] = useState('')
const [role, setRole] = useState<'user' | 'admin'>('user')
const [error, setError] = useState<string | null>(null)
const [loading, setLoading] = useState(false)
useEffect(() => {
if (open) {
setUsername('')
setPassword('')
setRole('user')
setError(null)
setLoading(false)
}
}, [open])
const handleSubmit = async () => {
setError(null)
setLoading(true)
try {
await admin.createUser({ username: username.trim(), password, role })
onCreated()
} catch (err: any) {
setError(err.response?.data?.detail ?? 'Failed to create user.')
} finally {
setLoading(false)
}
}
return (
<Dialog open={open} onOpenChange={(o) => !o && onClose()}>
<DialogContent className="max-w-sm">
<DialogHeader>
<DialogTitle>Add User</DialogTitle>
</DialogHeader>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
<div className="space-y-3">
<div className="space-y-1">
<Label htmlFor="new-username">Username</Label>
<Input
id="new-username"
value={username}
onChange={(e) => setUsername(e.target.value)}
autoFocus
/>
</div>
<div className="space-y-1">
<Label htmlFor="new-password">Password</Label>
<Input
id="new-password"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
/>
</div>
<div className="space-y-1">
<Label>Role</Label>
<Select
value={role}
onValueChange={(v) => setRole(v as 'user' | 'admin')}
>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="user">User</SelectItem>
<SelectItem value="admin">Admin</SelectItem>
</SelectContent>
</Select>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={onClose}>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={loading}>
{loading ? 'Creating\u2026' : 'Create'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
)
}
// ── Edit User Modal ────────────────────────────────────────────────────
function EditUserModal({
user,
onClose,
onSaved,
}: {
user: AdminUser | null
onClose: () => void
onSaved: () => void
}) {
const [role, setRole] = useState<'user' | 'admin'>('user')
const [newPassword, setNewPassword] = useState('')
const [error, setError] = useState<string | null>(null)
const [loading, setLoading] = useState(false)
useEffect(() => {
if (user) {
setRole(user.role as 'user' | 'admin')
setNewPassword('')
setError(null)
setLoading(false)
}
}, [user])
const handleSubmit = async () => {
if (!user) return
setError(null)
setLoading(true)
try {
const data: { role?: string; new_password?: string } = {}
if (role !== user.role) data.role = role
if (newPassword) data.new_password = newPassword
if (Object.keys(data).length > 0) {
await admin.updateUser(user.id, data)
}
onSaved()
} catch (err: any) {
setError(err.response?.data?.detail ?? 'Failed to update user.')
} finally {
setLoading(false)
}
}
return (
<Dialog open={!!user} onOpenChange={(o) => !o && onClose()}>
<DialogContent className="max-w-sm">
<DialogHeader>
<DialogTitle>Edit: {user?.username}</DialogTitle>
</DialogHeader>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
<div className="space-y-3">
<div className="space-y-1">
<Label>Role</Label>
<Select
value={role}
onValueChange={(v) => setRole(v as 'user' | 'admin')}
>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="user">User</SelectItem>
<SelectItem value="admin">Admin</SelectItem>
</SelectContent>
</Select>
</div>
<div className="space-y-1">
<Label htmlFor="edit-password">
New Password (leave blank to keep current)
</Label>
<Input
id="edit-password"
type="password"
value={newPassword}
onChange={(e) => setNewPassword(e.target.value)}
placeholder="Unchanged"
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={onClose}>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={loading}>
{loading ? 'Saving\u2026' : 'Save'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
)
}

View File

@@ -1,75 +0,0 @@
import { useState, type FormEvent } from 'react'
import { useAuth } from '../../contexts/AuthContext'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import { Alert, AlertDescription } from '@/components/ui/alert'
export function LoginPage() {
const { login } = useAuth()
const [username, setUsername] = useState('')
const [password, setPassword] = useState('')
const [error, setError] = useState<string | null>(null)
const [loading, setLoading] = useState(false)
const handleSubmit = async (e: FormEvent) => {
e.preventDefault()
setError(null)
setLoading(true)
try {
await login(username, password)
} catch (err: any) {
setError(
err.response?.data?.detail ?? 'Unable to sign in. Check your credentials.',
)
} finally {
setLoading(false)
}
}
return (
<div className="flex min-h-screen items-center justify-center bg-bg px-4">
<form
onSubmit={handleSubmit}
className="w-full max-w-sm space-y-5 rounded-lg border border-border bg-surface p-8 shadow-xl"
>
<h1 className="text-center text-xl font-semibold text-text">
Sign in to Mulita
</h1>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
<div className="space-y-1.5">
<Label htmlFor="login-user">Username</Label>
<Input
id="login-user"
type="text"
value={username}
onChange={(e) => setUsername(e.target.value)}
required
autoFocus
/>
</div>
<div className="space-y-1.5">
<Label htmlFor="login-pass">Password</Label>
<Input
id="login-pass"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
required
/>
</div>
<Button type="submit" disabled={loading} className="w-full">
{loading ? 'Signing in\u2026' : 'Sign In'}
</Button>
</form>
</div>
)
}

View File

@@ -1,110 +0,0 @@
import { useState, type FormEvent } from 'react'
import { useAuth } from '../../contexts/AuthContext'
import api from '../../services/api'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import { Alert, AlertDescription } from '@/components/ui/alert'
export function SetupPage() {
const { onSetupComplete } = useAuth()
const [username, setUsername] = useState('')
const [password, setPassword] = useState('')
const [confirmPassword, setConfirmPassword] = useState('')
const [error, setError] = useState<string | null>(null)
const [loading, setLoading] = useState(false)
const handleSubmit = async (e: FormEvent) => {
e.preventDefault()
setError(null)
if (password !== confirmPassword) {
setError('Passwords do not match.')
return
}
if (password.length < 6) {
setError('Password must be at least 6 characters.')
return
}
if (username.trim().length < 2) {
setError('Username must be at least 2 characters.')
return
}
setLoading(true)
try {
const res = await api.post('/auth/setup', {
username: username.trim(),
password,
})
const { access_token, refresh_token } = res.data
await onSetupComplete(access_token, refresh_token)
} catch (err: any) {
setError(
err.response?.data?.detail ?? 'Setup failed. Please try again.',
)
} finally {
setLoading(false)
}
}
return (
<div className="flex min-h-screen items-center justify-center bg-bg px-4">
<form
onSubmit={handleSubmit}
className="w-full max-w-sm space-y-5 rounded-lg border border-border bg-surface p-8 shadow-xl"
>
<div className="space-y-1 text-center">
<h1 className="text-xl font-semibold text-text">Welcome to Mulita</h1>
<p className="text-sm text-text-muted">
Create your admin account to get started.
</p>
</div>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
<div className="space-y-1.5">
<Label htmlFor="setup-user">Username</Label>
<Input
id="setup-user"
type="text"
value={username}
onChange={(e) => setUsername(e.target.value)}
required
autoFocus
/>
</div>
<div className="space-y-1.5">
<Label htmlFor="setup-pass">Password</Label>
<Input
id="setup-pass"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
required
/>
</div>
<div className="space-y-1.5">
<Label htmlFor="setup-confirm">Confirm Password</Label>
<Input
id="setup-confirm"
type="password"
value={confirmPassword}
onChange={(e) => setConfirmPassword(e.target.value)}
required
/>
</div>
<Button type="submit" disabled={loading} className="w-full">
{loading ? 'Creating account\u2026' : 'Create Admin Account'}
</Button>
</form>
</div>
)
}

View File

@@ -1,187 +0,0 @@
import { useState, useMemo, useCallback } from 'react'
import { Palette, ArrowLeft, Loader2 } from 'lucide-react'
import clsx from 'clsx'
import { Button } from '@/components/ui/button'
import { photos as photosApi } from '../../services/api'
import { useFilterStore } from '../../store/filterStore'
import { usePhotosQuery } from '../../hooks/usePhotosQuery'
import { COLOR_LABEL_OPTIONS, type ColorLabel } from '../../constants/colorLabels'
import { useCardGridNav } from '../../hooks/useCardGridNav'
import { Timeline } from '../timeline/Timeline'
import type { Photo } from '../../types/photo'
interface ColorGroup {
label: string
value: ColorLabel | null
className: string
count: number
representative: Photo | null
}
/**
* Colors view — two states:
* 1. Grid of color label cards (default) — arrow keys + Enter to browse
* 2. Detail view showing a color's photos in the full Timeline — Esc to go back
*/
export function ColorsView() {
const { data: allPhotos = [], isLoading } = usePhotosQuery()
const setColorLabel = useFilterStore((s) => s.setColorLabel)
const [selectedGroup, setSelectedGroup] = useState<ColorGroup | null>(null)
const groups = useMemo(() => {
const buckets = new Map<string, Photo[]>()
const uncolored: Photo[] = []
for (const photo of allPhotos) {
if (photo.color_label) {
const arr = buckets.get(photo.color_label) ?? []
arr.push(photo)
buckets.set(photo.color_label, arr)
} else {
uncolored.push(photo)
}
}
const result: ColorGroup[] = []
for (const { value, className } of COLOR_LABEL_OPTIONS) {
const photos = buckets.get(value) ?? []
if (photos.length === 0) continue
result.push({
label: value.charAt(0).toUpperCase() + value.slice(1),
value,
className,
count: photos.length,
representative: photos[0],
})
}
if (uncolored.length > 0) {
result.push({
label: 'Uncolored',
value: null,
className: 'bg-neutral-400',
count: uncolored.length,
representative: uncolored[0],
})
}
result.sort((a, b) => b.count - a.count)
return result
}, [allPhotos])
const enterDetail = useCallback(
(group: ColorGroup) => {
setColorLabel((group.value ?? 'none') as ColorLabel)
setSelectedGroup(group)
},
[setColorLabel]
)
const exitDetail = useCallback(() => {
setColorLabel(null)
setSelectedGroup(null)
}, [setColorLabel])
const { activeIndex, gridRef } = useCardGridNav({
items: groups,
inDetail: selectedGroup !== null,
onEnter: enterDetail,
onExit: exitDetail,
})
if (selectedGroup) {
return (
<div className="flex h-full flex-col overflow-hidden">
<div className="flex items-center gap-3 border-b border-border px-4 py-3">
<Button
variant="ghost"
size="icon"
className="h-7 w-7 text-text-muted"
onClick={exitDetail}
title="Back to colors"
>
<ArrowLeft className="h-4 w-4" />
</Button>
<div className="flex items-center gap-2">
<span className={`inline-block h-3 w-3 rounded-full ${selectedGroup.className}`} />
<h2 className="text-sm font-semibold text-text">{selectedGroup.label}</h2>
</div>
</div>
<div className="flex-1 overflow-hidden">
<Timeline />
</div>
</div>
)
}
if (isLoading) {
return (
<div className="flex h-full items-center justify-center text-text-muted">
<Loader2 className="mr-2 h-5 w-5 animate-spin" />
Loading colors...
</div>
)
}
if (groups.length === 0) {
return (
<div className="flex h-full flex-col items-center justify-center gap-3 text-text-muted">
<Palette className="h-12 w-12 opacity-40" />
<p className="text-sm">No color labels assigned yet</p>
<p className="max-w-xs text-center text-xs opacity-70">
Color labels will appear here once you assign them to photos.
</p>
</div>
)
}
return (
<div className="h-full overflow-auto p-4 pb-20">
<div className="mb-4 flex items-center gap-2 text-text-muted">
<Palette className="h-4 w-4" />
<span className="text-sm font-medium">
{groups.length} {groups.length === 1 ? 'color' : 'colors'}
</span>
</div>
<div
ref={gridRef}
className="grid grid-cols-[repeat(auto-fill,minmax(140px,1fr))] gap-3"
>
{groups.map((group, i) => (
<div
key={group.label}
className={clsx(
'group cursor-pointer overflow-hidden rounded-lg border bg-surface transition-all hover:border-primary/50 hover:shadow-md',
i === activeIndex
? 'border-primary ring-1 ring-primary/30'
: 'border-border'
)}
onClick={() => enterDetail(group)}
>
<div className="relative aspect-square overflow-hidden bg-surface-2">
{group.representative ? (
<img
src={photosApi.getThumbnailUrl(group.representative.id, 'small')}
alt={group.label}
className="h-full w-full object-cover"
/>
) : (
<div className="flex h-full w-full items-center justify-center">
<Palette className="h-10 w-10 text-text-muted/30" />
</div>
)}
<span className="absolute bottom-1.5 right-1.5 rounded-full bg-black/60 px-2 py-0.5 text-[11px] font-medium text-white backdrop-blur-sm">
{group.count}
</span>
</div>
<div className="flex items-center gap-1.5 px-2 py-1.5">
<span className={`inline-block h-2.5 w-2.5 rounded-full ${group.className}`} />
<p className="truncate text-xs font-medium text-text">{group.label}</p>
</div>
</div>
))}
</div>
</div>
)
}

View File

@@ -1,61 +0,0 @@
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog'
import { Button } from '@/components/ui/button'
interface ConfirmDialogProps {
isOpen: boolean
title: string
message: React.ReactNode
confirmLabel?: string
cancelLabel?: string
/** When true, the confirm button uses the destructive accent. */
destructive?: boolean
onConfirm: () => void
onClose: () => void
}
/**
* Modal confirmation dialog. Built on the shadcn Dialog primitive:
* Radix handles focus trap, portal, overlay-click dismissal, Esc, and
* animations. We only supply title, body, and the two action buttons.
*/
export function ConfirmDialog({
isOpen,
title,
message,
confirmLabel = 'Confirm',
cancelLabel = 'Cancel',
destructive = false,
onConfirm,
onClose,
}: ConfirmDialogProps) {
return (
<Dialog open={isOpen} onOpenChange={(o) => !o && onClose()}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>{title}</DialogTitle>
<DialogDescription asChild>
<div>{message}</div>
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button variant="outline" onClick={onClose}>
{cancelLabel}
</Button>
<Button
variant={destructive ? 'destructive' : 'default'}
onClick={onConfirm}
>
{confirmLabel}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
)
}

View File

@@ -1,166 +0,0 @@
import { useEffect, useState } from 'react'
import clsx from 'clsx'
import { Trash2, Archive } from 'lucide-react'
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog'
import { Button } from '@/components/ui/button'
import { RadioGroup, RadioGroupItem } from '@/components/ui/radio-group'
import { Label } from '@/components/ui/label'
interface DeleteFolderDialogProps {
isOpen: boolean
folderName: string
/** Number of photos under this folder, including descendants. Surfaced
* in the dialog copy so the user understands the blast radius. */
photoCount?: number
onClose: () => void
/** Called with the chosen mode when the user confirms. */
onConfirm: (mode: 'discard' | 'permanent') => void
}
type Mode = 'discard' | 'permanent'
/**
* Two-mode folder delete dialog built on the shadcn Dialog + RadioGroup
* primitives (instead of the old custom fixed-inset backdrop). Esc
* closes, overlay click closes, focus is trapped by Radix.
*/
export function DeleteFolderDialog({
isOpen,
folderName,
photoCount,
onClose,
onConfirm,
}: DeleteFolderDialogProps) {
const [mode, setMode] = useState<Mode>('discard')
// Reset mode when re-opening so the safe option is always the default.
useEffect(() => {
if (isOpen) setMode('discard')
}, [isOpen])
const photoBlurb =
photoCount === undefined
? 'photos in this folder'
: photoCount === 0
? 'this empty folder'
: `${photoCount} photo${photoCount === 1 ? '' : 's'} in this folder`
return (
<Dialog open={isOpen} onOpenChange={(o) => !o && onClose()}>
<DialogContent className="max-w-[420px]">
<DialogHeader>
<DialogTitle>Delete folder "{folderName}"?</DialogTitle>
<DialogDescription>
What should happen to {photoBlurb}?
</DialogDescription>
</DialogHeader>
<RadioGroup
value={mode}
onValueChange={(v) => setMode(v as Mode)}
className="gap-2"
>
<ModeCard
id="folder-delete-discard"
value="discard"
icon={<Archive className="h-4 w-4" />}
title="Move photos to discard pile"
description="Photos can be restored later from Discarded. The folder and files stay on disk."
selected={mode === 'discard'}
/>
<ModeCard
id="folder-delete-permanent"
value="permanent"
icon={<Trash2 className="h-4 w-4" />}
title="Permanently delete folder and photos"
description="Removes the folder, every photo inside it, and the directory from disk. This cannot be undone."
selected={mode === 'permanent'}
destructive
/>
</RadioGroup>
<DialogFooter>
<Button variant="outline" onClick={onClose}>
Cancel
</Button>
<Button
variant={mode === 'permanent' ? 'destructive' : 'default'}
onClick={() => onConfirm(mode)}
>
{mode === 'permanent' ? 'Delete forever' : 'Move to discard pile'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
)
}
/** Radio-group item rendered as a full-width descriptive card. */
function ModeCard({
id,
value,
icon,
title,
description,
selected,
destructive = false,
}: {
id: string
value: string
icon: React.ReactNode
title: string
description: string
selected: boolean
destructive?: boolean
}) {
return (
<Label
htmlFor={id}
className={clsx(
'flex w-full cursor-pointer gap-3 rounded-lg border p-3 text-left transition-colors',
selected
? destructive
? 'border-reject/60 bg-reject/10'
: 'border-primary/60 bg-primary/10'
: 'border-border bg-surface-2 hover:bg-surface-offset'
)}
>
<RadioGroupItem id={id} value={value} className="mt-0.5" />
<div
className={clsx(
'mt-0.5 flex-shrink-0',
selected
? destructive
? 'text-reject'
: 'text-primary'
: 'text-text-muted'
)}
>
{icon}
</div>
<div className="flex-1">
<div
className={clsx(
'text-sm font-medium',
selected
? destructive
? 'text-reject'
: 'text-primary'
: 'text-text'
)}
>
{title}
</div>
<div className="mt-0.5 text-xs text-text-muted">{description}</div>
</div>
</Label>
)
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,192 +0,0 @@
import { useState } from 'react'
import { RotateCcw, Trash2 } from 'lucide-react'
import { useMutation, useQueryClient } from '@tanstack/react-query'
import { usePhotoStore } from '../../store/photoStore'
import { useFilterStore } from '../../store/filterStore'
import { usePhotosQuery, stripPhotosFromCache } from '../../hooks/usePhotosQuery'
import { discard as discardApi, photos as photosApi } from '../../services/api'
import { toast } from '../ToastContainer'
import { ConfirmDialog } from '../dialogs/ConfirmDialog'
import { Button } from '@/components/ui/button'
import { registerUndoable } from '../../store/undoStore'
import { LIBRARY_STATS_QUERY_KEY } from '../../hooks/useLibraryStatsQuery'
/**
* Top-of-timeline bar visible only when the discarded filter is active.
* Shows a count, lets the user restore the current selection, and lets them
* permanently empty the discard pile (with confirmation).
*/
export function DiscardActionBar() {
const flag = useFilterStore((s) => s.flag)
const selectedPhotos = usePhotoStore((s) => s.selectedPhotos)
const clearSelection = usePhotoStore((s) => s.clearSelection)
const queryClient = useQueryClient()
const { data: photos = [] } = usePhotosQuery()
const [confirmOpen, setConfirmOpen] = useState(false)
const [deleteSelectedOpen, setDeleteSelectedOpen] = useState(false)
const restoreMutation = useMutation({
mutationFn: (ids: string[]) => discardApi.restore(ids),
// Pull the restored ids out of the discard view immediately. The
// user is sitting on flag=discarded so they should disappear from
// sight the moment the click lands; the onSuccess invalidate still
// reconciles with server truth shortly after.
onMutate: (ids) => {
usePhotoStore.getState().removePhotosFromTimeline(ids)
stripPhotosFromCache(queryClient, ids)
},
onSuccess: (_, ids) => {
registerUndoable(
`Restored ${ids.length} photo${ids.length === 1 ? '' : 's'}`,
async () => {
await photosApi.bulkDiscard(ids)
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
}
)
clearSelection()
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
},
onError: (e: any) => toast.error('Restore failed', e.message || 'Unknown error'),
})
const deleteSelectedMutation = useMutation({
mutationFn: (ids: string[]) => discardApi.deletePermanent(ids),
onMutate: (ids) => {
usePhotoStore.getState().removePhotosFromTimeline(ids)
stripPhotosFromCache(queryClient, ids)
},
onSuccess: (data: any) => {
const count = data?.deleted ?? 0
const errors = data?.file_errors ?? 0
if (errors > 0) {
toast.error(
`Deleted with ${errors} error${errors > 1 ? 's' : ''}`,
`${count} record${count === 1 ? '' : 's'} deleted; some files could not be removed`
)
} else {
toast.success(
'Permanently deleted',
`${count} photo${count === 1 ? '' : 's'} removed from disk`
)
}
clearSelection()
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
setDeleteSelectedOpen(false)
},
onError: (e: any) =>
toast.error('Delete failed', e.message || 'Unknown error'),
})
const emptyMutation = useMutation({
mutationFn: () => discardApi.empty(),
onSuccess: (data: any) => {
const count = data?.deleted ?? 0
const errors = data?.file_errors ?? 0
if (errors > 0) {
toast.error(
`Emptied with ${errors} error${errors > 1 ? 's' : ''}`,
`${count} record${count > 1 ? 's' : ''} deleted; some files could not be removed`
)
} else {
toast.success('Discard pile emptied', `${count} photo${count > 1 ? 's' : ''} permanently deleted`)
}
clearSelection()
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
setConfirmOpen(false)
},
onError: (e: any) => toast.error('Empty failed', e.message || 'Unknown error'),
})
if (flag !== 'discarded') return null
const total = photos.length
const selected = selectedPhotos.length
return (
<>
<div className="flex items-center justify-between gap-3 border-b border-border bg-reject/10 px-4 py-2 text-sm">
<div className="flex items-center gap-2 text-text">
<Trash2 className="h-4 w-4 text-reject" />
<span className="font-medium">Discarded</span>
<span className="text-text-muted">
{total} photo{total === 1 ? '' : 's'}
</span>
</div>
<div className="flex items-center gap-2">
{selected > 0 && (
<>
<Button
variant="secondary"
size="sm"
onClick={() => restoreMutation.mutate(selectedPhotos)}
disabled={restoreMutation.isPending}
title="Restore selected (U)"
>
<RotateCcw className="mr-1.5 h-3.5 w-3.5" />
Restore {selected}
</Button>
<Button
size="sm"
onClick={() => setDeleteSelectedOpen(true)}
disabled={deleteSelectedMutation.isPending}
className="bg-reject/20 text-reject hover:bg-reject/30"
title="Permanently delete selected"
>
<Trash2 className="mr-1.5 h-3.5 w-3.5" />
Delete {selected}
</Button>
</>
)}
<Button
size="sm"
onClick={() => setConfirmOpen(true)}
disabled={total === 0 || emptyMutation.isPending}
className="bg-reject/20 text-reject hover:bg-reject/30"
title="Permanently delete all discarded photos and files"
>
<Trash2 className="mr-1.5 h-3.5 w-3.5" />
Empty discard pile
</Button>
</div>
</div>
<ConfirmDialog
isOpen={deleteSelectedOpen}
title={`Delete ${selected} photo${selected === 1 ? '' : 's'}?`}
message={
<>
This will <strong className="text-text">permanently delete</strong>{' '}
{selected} photo{selected === 1 ? '' : 's'} and remove the file
{selected === 1 ? '' : 's'} from disk. This cannot be undone.
</>
}
confirmLabel="Delete"
destructive
onConfirm={() => deleteSelectedMutation.mutate(selectedPhotos)}
onClose={() => setDeleteSelectedOpen(false)}
/>
<ConfirmDialog
isOpen={confirmOpen}
title="Empty discard pile?"
message={
<>
This will <strong className="text-text">permanently delete</strong>{' '}
{total} photo{total === 1 ? '' : 's'} and remove the file
{total === 1 ? '' : 's'} from disk. This cannot be undone.
</>
}
confirmLabel="Empty pile"
destructive
onConfirm={() => emptyMutation.mutate()}
onClose={() => setConfirmOpen(false)}
/>
</>
)
}

View File

@@ -1,432 +0,0 @@
import { useMemo, useState, useEffect, useCallback } from 'react'
import { Copy, Layers, Sparkles, Trash2, Loader2, Info, Crown } from 'lucide-react'
import clsx from 'clsx'
import { useMutation, useQueryClient } from '@tanstack/react-query'
import { Button } from '@/components/ui/button'
import {
useDuplicateGroupsQuery,
DUPLICATE_GROUPS_QUERY_KEY,
} from '../../hooks/useDuplicateGroupsQuery'
import {
photos as photosApi,
type DuplicateGroup,
type DuplicateGroupMember,
} from '../../services/api'
import {
PhotoThumbnail,
THUMB_BADGE_BASE,
THUMB_BADGE_ICON,
THUMB_BADGE_NEUTRAL,
THUMB_BADGE_PICK,
} from '../timeline/PhotoThumbnail'
import { usePhotoStore } from '../../store/photoStore'
import { registerUndoable } from '../../store/undoStore'
import { LIBRARY_STATS_QUERY_KEY } from '../../hooks/useLibraryStatsQuery'
import { toast } from '../ToastContainer'
import type { Photo } from '../../types/photo'
/**
* Sectioned grid view of duplicate clusters. Replaces the old flat
* "is_duplicate=true" timeline. Each section is one cluster the
* regroup_duplicates task identified — header on top with a count and a
* "keep best, discard rest" button, members rendered as PhotoThumbnail
* cards below.
*
* Mounted from App.tsx in place of <Timeline /> when the user is in the
* duplicates section. Touches no filter store state.
*/
export function DuplicatesView() {
const { data, isLoading, isError, error } = useDuplicateGroupsQuery()
const queryClient = useQueryClient()
const openPreview = usePhotoStore((s) => s.openPreview)
const selectPhoto = usePhotoStore((s) => s.selectPhoto)
const selectedPhotos = usePhotoStore((s) => s.selectedPhotos)
const activePhotoId = usePhotoStore((s) => s.activePhotoId)
// Bulk discard with the same undoable wrapper the timeline uses, so
// Cmd+Z restores the discarded copies. invalidate ['library', 'duplicates']
// so the group disappears from the view immediately.
const discardMutation = useMutation({
mutationFn: (ids: string[]) => photosApi.bulkDiscard(ids),
onSuccess: (_, ids) => {
registerUndoable(
`Discarded ${ids.length} duplicate${ids.length === 1 ? '' : 's'}`,
async () => {
await photosApi.bulkRestore(ids)
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: DUPLICATE_GROUPS_QUERY_KEY })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
}
)
queryClient.invalidateQueries({ queryKey: ['photos'] })
queryClient.invalidateQueries({ queryKey: DUPLICATE_GROUPS_QUERY_KEY })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
},
onError: (e: any) =>
toast.error('Discard failed', e?.message || 'Unknown error'),
})
// Hooks below this point must run on every render — rules of hooks
// forbid early returns above any useState/useEffect/useMemo. The early
// loading/error/empty branches sit AFTER the hook block.
const groups = data?.groups ?? []
// Flat sequence of member ids in visual order. Drives both preview
// navigation and the in-grid keyboard walker. useMemo so the keyboard
// effect doesn't tear down on every render.
const allMemberIds = useMemo(
() => groups.flatMap((g) => g.members.map((m) => m.id)),
[groups]
)
// Track the rendered column count of the duplicates grid so ↑/↓ can
// skip a row instead of jumping a single cell. The grid uses
// `repeat(auto-fill, minmax(180px, 1fr))` so columns = floor(width/180).
// We measure the FIRST section's grid container — every section uses
// the same auto-fill rule so any one is representative.
const [columns, setColumns] = useState(4)
const sampleGridRef = useCallback((el: HTMLDivElement | null) => {
if (!el) return
const measure = () => {
const cols = Math.max(1, Math.floor(el.clientWidth / 180))
setColumns(cols)
}
measure()
const ro = new ResizeObserver(measure)
ro.observe(el)
// Caller doesn't get the cleanup hook but ResizeObserver disconnects
// when the element unmounts, which is fine for our lifecycle.
}, [])
// Window-level keyboard nav. Mirrors Timeline's handler but walks
// `allMemberIds` directly — duplicate groups don't have a uniform row
// grid so we approximate ↑/↓ via the measured `columns` count and
// wrap ←/→ across group boundaries.
useEffect(() => {
if (allMemberIds.length === 0) return
const onKeyDown = (e: KeyboardEvent) => {
const target = e.target as HTMLElement | null
if (target && (target.tagName === 'INPUT' || target.tagName === 'TEXTAREA')) {
return
}
const key = e.key
if (
key !== 'ArrowLeft' &&
key !== 'ArrowRight' &&
key !== 'ArrowUp' &&
key !== 'ArrowDown'
) {
return
}
e.preventDefault()
const currentIdx = activePhotoId ? allMemberIds.indexOf(activePhotoId) : -1
const startIdx = currentIdx >= 0 ? currentIdx : 0
let nextIdx = startIdx
if (key === 'ArrowLeft') nextIdx = startIdx - 1
else if (key === 'ArrowRight') nextIdx = startIdx + 1
else if (key === 'ArrowUp') nextIdx = startIdx - columns
else if (key === 'ArrowDown') nextIdx = startIdx + columns
// Clamp to bounds — we don't wrap on out-of-range vertical moves
// since the grid is partitioned into groups and a "wrap" would
// skip across visually unrelated content.
nextIdx = Math.max(0, Math.min(allMemberIds.length - 1, nextIdx))
const nextId = allMemberIds[nextIdx]
if (!nextId) return
selectPhoto(nextId)
// Scroll the now-active cell into view if it's off-screen. The
// PhotoThumbnail wrapper carries data-dup-id so we can find it
// without threading refs through every cell.
const el = document.querySelector<HTMLElement>(
`[data-dup-id="${nextId}"]`
)
el?.scrollIntoView({ block: 'nearest', inline: 'nearest' })
}
window.addEventListener('keydown', onKeyDown)
return () => window.removeEventListener('keydown', onKeyDown)
}, [allMemberIds, activePhotoId, columns, selectPhoto])
if (isLoading) {
return (
<div className="flex h-full items-center justify-center text-text-muted">
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
Loading duplicate groups
</div>
)
}
if (isError) {
return (
<div className="flex h-full items-center justify-center text-text-muted">
Could not load duplicate groups: {(error as any)?.message ?? 'unknown error'}
</div>
)
}
if (groups.length === 0) {
return (
<div className="flex h-full flex-col items-center justify-center gap-2 px-6 text-center text-text-muted">
<Sparkles className="h-8 w-8" />
<div className="text-sm font-medium text-text">No duplicates found</div>
<p className="max-w-sm text-xs">
Nothing in your library matches another photo at the perceptual-hash
level. If you've just added new photos, give the worker a minute and
re-run "Re-detect duplicates" from Settings.
</p>
</div>
)
}
return (
<div className="h-full overflow-auto bg-bg p-4 pb-20">
<div className="mb-4 flex items-center gap-2 text-xs text-text-muted">
<Info className="h-3.5 w-3.5" />
<span>
{data?.total_groups} group{data?.total_groups === 1 ? '' : 's'} ·{' '}
{data?.total_members} photo{data?.total_members === 1 ? '' : 's'}.
Click "Keep best" to auto-discard all but the highest-resolution
copy of each group. Cmd+Z to undo.
</span>
</div>
<div className="space-y-6">
{groups.map((group, idx) => (
<DuplicateGroupSection
key={group.group_id}
group={group}
onKeepBest={(discardIds) => discardMutation.mutate(discardIds)}
onPreviewMember={(memberId) => openPreview(memberId, allMemberIds)}
onSelectMember={(memberId) => selectPhoto(memberId)}
selectedPhotos={selectedPhotos}
isPending={discardMutation.isPending}
// Hand the column-measurement ref to the first section only
// — every section's grid uses the same auto-fill rule so any
// one is representative of the rendered column count.
gridRef={idx === 0 ? sampleGridRef : undefined}
/>
))}
</div>
</div>
)
}
interface DuplicateGroupSectionProps {
group: DuplicateGroup
onKeepBest: (discardIds: string[]) => void
onPreviewMember: (memberId: string) => void
onSelectMember: (memberId: string) => void
selectedPhotos: string[]
isPending: boolean
/** Optional callback ref attached to this section's grid container.
* Used by DuplicatesView to measure the rendered column count for
* ↑/↓ keyboard navigation. Only the first section gets one. */
gridRef?: (el: HTMLDivElement | null) => void
}
function DuplicateGroupSection({
group,
onKeepBest,
onPreviewMember,
onSelectMember,
selectedPhotos,
isPending,
gridRef,
}: DuplicateGroupSectionProps) {
// Auto-pick "best" copy: highest pixel count, ties broken by file_size,
// then earliest taken_at, then id for determinism. This is just the
// default — the user can override it by clicking the crown button on
// any other thumbnail (see `manualBestId`).
const autoBest = useMemo(() => pickBestMember(group.members), [group.members])
// When the user clicks "make this the best" on a non-default thumb,
// we override the auto-pick. Local to the section so different groups
// remember independent overrides; resets if the group itself changes.
const [manualBestId, setManualBestId] = useState<string | null>(null)
const bestId =
manualBestId && group.members.some((m) => m.id === manualBestId)
? manualBestId
: autoBest.id
const best = group.members.find((m) => m.id === bestId) ?? autoBest
const discardCount = group.member_count - 1
const isExact = group.reason === 'exact'
return (
<section className="rounded-lg border border-border bg-surface">
<header className="flex items-center justify-between gap-3 border-b border-border px-3 py-2">
<div className="flex items-center gap-2 text-sm">
{isExact ? (
<Copy className="h-4 w-4 text-text-muted" />
) : (
<Layers className="h-4 w-4 text-text-muted" />
)}
<span className="font-medium text-text">
{group.member_count} {isExact ? 'exact' : 'similar'} photos
</span>
<span className="text-xs text-text-faint">
keeping: {formatDimensions(best)}
{best.file_size != null && ` · ${formatBytes(best.file_size)}`}
{manualBestId && manualBestId !== autoBest.id && (
<span className="ml-1 text-text-muted">(manual)</span>
)}
</span>
</div>
<Button
variant="outline"
size="sm"
onClick={() => {
const discardIds = group.members
.filter((m) => m.id !== best.id)
.map((m) => m.id)
if (discardIds.length === 0) return
onKeepBest(discardIds)
}}
disabled={isPending}
className="hover:border-reject/50 hover:bg-reject/10 hover:text-reject"
title="Keep the highest-resolution copy and discard the rest"
>
<Trash2 className="mr-1.5 h-3 w-3" />
Keep best, discard {discardCount}
</Button>
</header>
<div
ref={gridRef}
className="grid gap-1 p-2"
style={{
gridTemplateColumns:
'repeat(auto-fill, minmax(180px, 1fr))',
}}
>
{group.members.map((member) => {
const isBest = member.id === best.id
return (
<div
key={member.id}
data-dup-id={member.id}
className="group/dup relative"
>
<PhotoThumbnail
photo={memberToPhoto(member)}
size={180}
fill
isSelected={selectedPhotos.includes(member.id)}
onClick={() => onSelectMember(member.id)}
onDoubleClick={() => onPreviewMember(member.id)}
/>
{/* BEST pill — top-right, pick-coloured. Composes the same
* THUMB_BADGE_* family used by PhotoThumbnail so the full
* set of ornaments (selection, rating, flags, BEST) reads
* as one consistent chip system. Inset 1.5 (6px) rather
* than 1 (4px) because these are SIBLINGS of the thumbnail,
* not inside its overflow-hidden box, so they need
* clearance from PhotoThumbnail's outer selection ring. */}
{isBest && (
<span
className={clsx(
'pointer-events-none absolute right-1.5 top-1.5 z-10 uppercase',
THUMB_BADGE_BASE,
THUMB_BADGE_PICK
)}
>
<Crown className={THUMB_BADGE_ICON} />
Best
</span>
)}
{/* "Keep this" — shown on hover for non-best thumbnails.
* Mirrors BEST's placement so the eye doesn't retarget
* while scanning. Starts neutral and shifts to pick on
* hover as a preview of the state it'll set. */}
{!isBest && (
<button
onClick={(e) => {
e.stopPropagation()
setManualBestId(member.id)
}}
className={clsx(
'absolute right-1.5 top-1.5 z-10 hidden uppercase transition hover:bg-pick group-hover/dup:inline-flex',
THUMB_BADGE_BASE,
THUMB_BADGE_NEUTRAL
)}
title="Keep this one instead"
>
<Crown className={THUMB_BADGE_ICON} />
Keep this
</button>
)}
{/* Dimensions chip — bottom-LEFT. Neutral metadata variant
* matches the family. Rare collision with a manual rating
* (also bottom-left) is tolerated: rated duplicates are
* uncommon in practice. */}
<div
className={clsx(
'pointer-events-none absolute bottom-1 left-1 z-10 font-mono',
THUMB_BADGE_BASE,
THUMB_BADGE_NEUTRAL
)}
>
{formatDimensions(member)}
</div>
</div>
)
})}
</div>
</section>
)
}
// ── Helpers ──────────────────────────────────────────────────────────────
/** Score a member by (pixels, file_size, -taken_at) and return the winner.
* Larger pixel count wins; ties broken by file_size; final tie by earliest
* taken_at (more likely the original capture). */
function pickBestMember(members: DuplicateGroupMember[]): DuplicateGroupMember {
return members.reduce((best, m) => {
const bestPixels = (best.width ?? 0) * (best.height ?? 0)
const mPixels = (m.width ?? 0) * (m.height ?? 0)
if (mPixels !== bestPixels) return mPixels > bestPixels ? m : best
const bestSize = best.file_size ?? 0
const mSize = m.file_size ?? 0
if (mSize !== bestSize) return mSize > bestSize ? m : best
// Earliest taken_at wins (treat null as far-future).
const bestTaken = best.taken_at ?? '9999'
const mTaken = m.taken_at ?? '9999'
if (mTaken !== bestTaken) return mTaken < bestTaken ? m : best
return best
})
}
function formatDimensions(m: DuplicateGroupMember): string {
if (!m.width || !m.height) return '?'
const mp = (m.width * m.height) / 1_000_000
if (mp >= 1) return `${mp.toFixed(1)}MP`
return `${m.width}×${m.height}`
}
function formatBytes(n: number): string {
if (n >= 1024 * 1024) return `${(n / (1024 * 1024)).toFixed(1)}MB`
if (n >= 1024) return `${(n / 1024).toFixed(0)}KB`
return `${n}B`
}
/** Adapt a DuplicateGroupMember (the slim API shape) to a Photo, which
* is what PhotoThumbnail expects. We deliberately set is_duplicate=false
* on the synthetic Photo so the duplicate badge isn't drawn on every
* cell — the entire view is duplicates, the badge would be redundant. */
function memberToPhoto(m: DuplicateGroupMember): Photo {
return {
id: m.id,
filepath: m.filename, // good enough for the RAW/video extension regex
filename: m.filename,
media_type: m.media_type,
width: m.width,
height: m.height,
taken_at: m.taken_at,
rating: 0,
is_discarded: false,
is_duplicate: false,
file_hash: m.file_hash ?? '',
folder_id: m.folder_id,
added_at: null,
thumb_small: m.thumb_small ?? undefined,
}
}

View File

@@ -1,596 +0,0 @@
import { useEffect, useMemo, useRef, useState } from 'react'
import {
Star,
X,
ArrowDown,
ArrowUp,
Search,
PanelLeftOpen,
PanelLeftClose,
PanelRightOpen,
PanelRightClose,
} from 'lucide-react'
import clsx from 'clsx'
import {
useFilterStore,
hasActiveFilters,
type MediaType,
type SortField,
} from '../../store/filterStore'
import { useTagsQuery } from '../../hooks/useTagsQuery'
import { usePhotosQuery } from '../../hooks/usePhotosQuery'
import { FilterPill } from './FilterPill'
import { COLOR_LABEL_OPTIONS } from '../../constants/colorLabels'
import { Input } from '@/components/ui/input'
import { Button } from '@/components/ui/button'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select'
import { MultiSelect } from '@/components/ui/multi-select'
import { Calendar } from '@/components/ui/calendar'
const SEARCH_DEBOUNCE_MS = 300
const MEDIA_TYPES: { value: MediaType; label: string }[] = [
{ value: 'photo', label: 'Photo' },
{ value: 'video', label: 'Video' },
{ value: 'raw', label: 'RAW' },
{ value: 'heic', label: 'HEIC' },
]
const SORT_OPTIONS: { value: SortField; label: string }[] = [
{ value: 'taken_at', label: 'Date taken' },
{ value: 'added_at', label: 'Date added' },
{ value: 'filename', label: 'Filename' },
{ value: 'file_size', label: 'File size' },
{ value: 'rating', label: 'Rating' },
]
/**
* Compact, always-visible filter toolbar built out of FilterPill primitives.
* Each pill represents a filter category, opens a popover with the
* underlying control, and shows a short value summary inline when active.
* Replaces the old expandable FilterBar + ActiveFilterChips combo.
*/
interface FilterBarProps {
leftSidebarOpen: boolean
rightSidebarOpen: boolean
onToggleLeftSidebar: () => void
onToggleRightSidebar: () => void
}
export function FilterBar({
leftSidebarOpen,
rightSidebarOpen,
onToggleLeftSidebar,
onToggleRightSidebar,
}: FilterBarProps) {
const filterState = useFilterStore()
const dateFrom = useFilterStore((s) => s.dateFrom)
const dateTo = useFilterStore((s) => s.dateTo)
const mediaTypes = useFilterStore((s) => s.mediaTypes)
const ratingMin = useFilterStore((s) => s.ratingMin)
const colorLabel = useFilterStore((s) => s.colorLabel)
const flag = useFilterStore((s) => s.flag)
const sortBy = useFilterStore((s) => s.sortBy)
const sortOrder = useFilterStore((s) => s.sortOrder)
const tagIds = useFilterStore((s) => s.tagIds)
const needsReview = useFilterStore((s) => s.needsReview)
const setNeedsReview = useFilterStore((s) => s.setNeedsReview)
const currentSection = useFilterStore((s) => s.currentSection)
// Only the Flag pill is hidden inside the Discarded section. Flag has
// exactly two values and the section locks one of them, so the pill
// would only ever toggle the section off — useless. Rating + Tags
// pills stay visible in their sections because the user can refine
// them further (ratingMin >= 3, restrict to specific tag ids).
const hideFlagPill = currentSection === 'discarded'
const setDateFrom = useFilterStore((s) => s.setDateFrom)
const setDateTo = useFilterStore((s) => s.setDateTo)
const toggleMediaType = useFilterStore((s) => s.toggleMediaType)
const setRatingMin = useFilterStore((s) => s.setRatingMin)
const setColorLabel = useFilterStore((s) => s.setColorLabel)
const setFlag = useFilterStore((s) => s.setFlag)
const setTagIds = useFilterStore((s) => s.setTagIds)
const setSortBy = useFilterStore((s) => s.setSortBy)
const toggleSortOrder = useFilterStore((s) => s.toggleSortOrder)
const clearAll = useFilterStore((s) => s.clearAll)
const { data: allTags = [] } = useTagsQuery()
// Search box. Local state mirrors the store so typing stays responsive
// while we debounce store writes (each store write triggers a re-fetch).
const storeQ = useFilterStore((s) => s.q)
const setStoreQ = useFilterStore((s) => s.setQ)
const [searchQuery, setSearchQuery] = useState(storeQ)
useEffect(() => {
setSearchQuery(storeQ)
}, [storeQ])
const debounceRef = useRef<number | null>(null)
useEffect(() => {
if (searchQuery === storeQ) return
if (debounceRef.current) window.clearTimeout(debounceRef.current)
debounceRef.current = window.setTimeout(() => {
setStoreQ(searchQuery)
}, SEARCH_DEBOUNCE_MS)
return () => {
if (debounceRef.current) window.clearTimeout(debounceRef.current)
}
}, [searchQuery, storeQ, setStoreQ])
// Pre-compute pill values + active flags so the JSX stays terse.
const dateActive = dateFrom !== null || dateTo !== null
const dateValue = dateActive
? `${dateFrom ?? '…'}${dateTo ?? '…'}`
: null
const typeActive = mediaTypes.length > 0
const typeValue = typeActive
? mediaTypes.map((t) => t.toUpperCase()).join(', ')
: null
const ratingActive = ratingMin > 0
const ratingValue = ratingActive ? `${ratingMin}` : null
const colorActive = colorLabel !== null
const colorValue = colorActive ? colorLabel : null
const flagActive = flag !== 'any' || needsReview
const flagValue = needsReview
? 'needs review'
: flag !== 'any'
? flag === 'date_warning'
? 'date issues'
: flag
: null
const tagActive = tagIds.length > 0
const activeTagNames = allTags
.filter((t) => tagIds.includes(t.id))
.map((t) => t.name)
const tagValue = tagActive
? activeTagNames.length <= 2
? activeTagNames.join(', ')
: `${activeTagNames.slice(0, 2).join(', ')} +${activeTagNames.length - 2}`
: null
const sortLabel = SORT_OPTIONS.find((o) => o.value === sortBy)?.label ?? sortBy
const sortValue = `${sortLabel} ${sortOrder === 'desc' ? '↓' : '↑'}`
const anyActive = hasActiveFilters(filterState)
return (
<div className="flex h-9 items-center gap-3 border-b border-border bg-surface px-3 py-0">
{/* Left sidebar toggle — pinned to the far-left edge of the bar so it
* sits flush against the panel it controls (or the viewport edge
* when collapsed). */}
<button
onClick={onToggleLeftSidebar}
className="flex-shrink-0 rounded p-1 text-text-muted hover:bg-surface-2 hover:text-text"
title={leftSidebarOpen ? 'Collapse panel (Tab)' : 'Expand panel (Tab)'}
aria-label={leftSidebarOpen ? 'Collapse left panel' : 'Expand left panel'}
>
{leftSidebarOpen ? (
<PanelLeftClose className="h-3.5 w-3.5" />
) : (
<PanelLeftOpen className="h-3.5 w-3.5" />
)}
</button>
{/* Pills — left side, scroll horizontally if they overflow. */}
<div className="flex min-w-0 flex-1 items-center gap-1.5 overflow-x-auto">
{/* Date */}
<FilterPill
label="Date"
value={dateValue}
isActive={dateActive}
onClear={() => {
setDateFrom(null)
setDateTo(null)
}}
>
<DateRangePicker
from={dateFrom}
to={dateTo}
onFromChange={setDateFrom}
onToChange={setDateTo}
/>
</FilterPill>
{/* Type */}
<FilterPill
label="Type"
value={typeValue}
isActive={typeActive}
onClear={() => mediaTypes.forEach((t) => toggleMediaType(t))}
>
<MultiSelect
searchable={false}
pinSelected={false}
options={MEDIA_TYPES.map(({ value, label }) => ({
value,
label,
}))}
values={mediaTypes}
onChange={(next) => {
// Diff against current selection — store uses per-item toggles.
for (const t of MEDIA_TYPES.map((m) => m.value)) {
const wasOn = mediaTypes.includes(t)
const nowOn = next.includes(t as MediaType)
if (wasOn !== nowOn) toggleMediaType(t)
}
}}
/>
</FilterPill>
{/* Rating */}
<FilterPill
label="Rating"
value={ratingValue}
isActive={ratingActive}
onClear={() => setRatingMin(0)}
>
<div>
<p className="mb-1 text-[11px] text-text-muted">Minimum</p>
<div className="flex gap-1">
{[1, 2, 3, 4, 5].map((n) => (
<button
key={n}
onClick={() => setRatingMin(ratingMin === n ? 0 : n)}
className="p-0.5"
title={`At least ${n} star${n > 1 ? 's' : ''}`}
>
<Star
className={clsx(
'h-5 w-5 transition-colors',
n <= ratingMin
? 'fill-star text-star'
: 'text-text-muted hover:text-star'
)}
/>
</button>
))}
</div>
</div>
</FilterPill>
{/* Color */}
<FilterPill
label="Color"
value={colorValue}
isActive={colorActive}
onClear={() => setColorLabel(null)}
>
<div className="flex items-center gap-1.5">
{COLOR_LABEL_OPTIONS.map(({ value, className }) => {
const active = colorLabel === value
return (
<button
key={value}
onClick={() => setColorLabel(active ? null : value)}
className={clsx(
'h-5 w-5 rounded-full ring-offset-2 ring-offset-surface transition-all',
className,
active ? 'ring-2 ring-primary' : 'opacity-60 hover:opacity-100'
)}
title={value}
/>
)
})}
{colorLabel && (
<button
onClick={() => setColorLabel(null)}
className="ml-1 rounded p-0.5 text-text-muted hover:bg-surface-2 hover:text-text"
title="Clear color"
>
<X className="h-3 w-3" />
</button>
)}
</div>
</FilterPill>
{/* Flag — hidden in the Discarded section, where the flag is
* pinned to "discarded" by the section preset. The Needs review
* option lives here too: it sets a different store field
* (`needsReview`) but is mutually exclusive with the other flag
* values from the user's perspective. */}
{!hideFlagPill && (
<FilterPill
label="Flag"
value={flagValue}
isActive={flagActive}
onClear={() => {
setFlag('any')
setNeedsReview(false)
}}
>
<MultiSelect
searchable={false}
pinSelected={false}
options={[
{ value: 'discarded', label: 'Discarded' },
{ value: 'date_warning', label: 'Date issues' },
{ value: 'needs_review', label: 'Needs review' },
]}
values={
needsReview
? ['needs_review']
: flag !== 'any'
? [flag]
: []
}
onChange={(next) => {
// Flag state is mutually exclusive in the store; treat
// the just-added value as the new single selection, or
// clear everything when the user unchecks the current.
const added = next.find(
(v) =>
v !==
(needsReview ? 'needs_review' : flag !== 'any' ? flag : '')
)
if (!added) {
setFlag('any')
setNeedsReview(false)
return
}
if (added === 'needs_review') {
setFlag('any')
setNeedsReview(true)
} else {
setFlag(added as 'discarded' | 'date_warning')
setNeedsReview(false)
}
}}
/>
</FilterPill>
)}
{/* Tags */}
{allTags.length > 0 && (
<FilterPill
label="Tags"
value={tagValue}
isActive={tagActive}
onClear={() => setTagIds([])}
>
<MultiSelect
className="w-60"
searchPlaceholder="Search tags…"
emptyMessage="No tags match"
options={allTags
.slice()
.sort((a, b) => {
if (b.photo_count !== a.photo_count)
return b.photo_count - a.photo_count
return a.name.localeCompare(b.name)
})
.map((t) => ({
value: t.id,
label: t.name,
meta: t.photo_count,
}))}
values={tagIds}
onChange={setTagIds}
/>
</FilterPill>
)}
{/* Sort — always present, never "active/inactive" since there's
always a value. */}
<FilterPill label="Sort" value={sortValue} isActive>
<div className="space-y-2">
<Select
value={sortBy}
onValueChange={(v) => setSortBy(v as SortField)}
>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
<SelectContent>
{SORT_OPTIONS.map((opt) => (
<SelectItem key={opt.value} value={opt.value}>
{opt.label}
</SelectItem>
))}
</SelectContent>
</Select>
<Button
variant="secondary"
size="sm"
onClick={toggleSortOrder}
className="w-full"
>
{sortOrder === 'desc' ? (
<>
<ArrowDown className="h-3.5 w-3.5" />
Descending
</>
) : (
<>
<ArrowUp className="h-3.5 w-3.5" />
Ascending
</>
)}
</Button>
</div>
</FilterPill>
{/* Clear-all — borderless text affordance pinned next to the pill
* cluster on the right. Lives inside the pills container so it
* shares the same flex group and gap and reads as "another
* pill". Only renders when any filter is active. */}
{anyActive && (
<button
onClick={clearAll}
className="ml-1 flex h-7 flex-shrink-0 items-center whitespace-nowrap px-1 text-xs text-text-muted underline-offset-2 hover:text-text hover:underline"
title="Clear all filters in this section"
>
Clear all
</button>
)}
</div>
{/* Search — pinned to the right edge of the bar. Same id as before
* so the global "/" focus shortcut still finds it. */}
<div className="relative w-56 flex-shrink-0">
<Search className="pointer-events-none absolute left-2.5 top-1/2 h-3.5 w-3.5 -translate-y-1/2 text-text-muted" />
<Input
id="topbar-search"
type="text"
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Escape') {
setSearchQuery('')
setStoreQ('')
e.currentTarget.blur()
}
}}
placeholder="Search photos…"
className="h-7 w-full rounded-full bg-surface-2 pl-8 pr-7 text-xs"
/>
{searchQuery && (
<button
onClick={() => {
setSearchQuery('')
setStoreQ('')
}}
className="absolute right-1.5 top-1/2 -translate-y-1/2 rounded-full p-0.5 text-text-muted hover:bg-surface-offset hover:text-text"
title="Clear search (Esc)"
aria-label="Clear search"
>
<X className="h-3 w-3" />
</button>
)}
</div>
{/* Right sidebar toggle — pinned to the far-right edge. */}
<button
onClick={onToggleRightSidebar}
className="flex-shrink-0 rounded p-1 text-text-muted hover:bg-surface-2 hover:text-text"
title={rightSidebarOpen ? 'Collapse panel (I)' : 'Expand panel (I)'}
aria-label={rightSidebarOpen ? 'Collapse right panel' : 'Expand right panel'}
>
{rightSidebarOpen ? (
<PanelRightClose className="h-3.5 w-3.5" />
) : (
<PanelRightOpen className="h-3.5 w-3.5" />
)}
</button>
</div>
)
}
/** Date range picker used inside the Date FilterPill. A single
* shadcn/react-day-picker Calendar in `range` mode — first click picks
* the start, second click picks the end. Writes both bounds to the
* store as yyyy-mm-dd ISO strings.
*
* Days without matching photos are disabled + visually dimmed
* ("booked days" pattern from shadcn docs), so the user can see at a
* glance which dates are worth clicking. The booked set is derived
* from the currently-cached photos — reflects every other active
* filter, which is the intended UX: "which days have 5★ photos of
* screenshots" etc. The caption uses the dropdown layout so the user
* can jump across months/years without clicking the arrows. */
function DateRangePicker({
from,
to,
onFromChange,
onToChange,
}: {
from: string | null
to: string | null
onFromChange: (v: string | null) => void
onToChange: (v: string | null) => void
}) {
// Unique yyyy-mm-dd keys of every cached photo's taken_at. Derived
// from the shared photos cache so opening the calendar doesn't fire
// another network request.
const { data: photos = [] } = usePhotosQuery()
const bookedSet = useMemo(() => {
const s = new Set<string>()
for (const p of photos) {
if (!p.taken_at) continue
const d = new Date(p.taken_at)
if (Number.isNaN(d.getTime())) continue
s.add(formatIsoDate(d))
}
return s
}, [photos])
const currentYear = new Date().getFullYear()
const earliestYear = useMemo(() => {
let min = currentYear
for (const k of bookedSet) {
const y = parseInt(k.slice(0, 4), 10)
if (!Number.isNaN(y) && y < min) min = y
}
return min
}, [bookedSet, currentYear])
const selected = {
from: from ? parseIsoDate(from) : undefined,
to: to ? parseIsoDate(to) : undefined,
}
const handleSelect = (range: { from?: Date; to?: Date } | undefined) => {
onFromChange(range?.from ? formatIsoDate(range.from) : null)
onToChange(range?.to ? formatIsoDate(range.to) : null)
}
const label =
from && to
? from === to
? from
: `${from}${to}`
: from
? `From ${from}`
: to
? `Until ${to}`
: 'Click to pick a start date, then an end date'
return (
<div className="space-y-2">
<div className="text-[11px] text-text-muted">{label}</div>
<Calendar
mode="range"
selected={selected}
onSelect={handleSelect}
numberOfMonths={1}
defaultMonth={selected.from ?? selected.to ?? new Date()}
captionLayout="dropdown"
fromYear={earliestYear}
toYear={currentYear + 1}
modifiers={{
booked: (d) => bookedSet.has(formatIsoDate(d)),
}}
modifiersClassNames={{
// Days that DO have matching photos get a small primary dot
// below the number; unmatched days stay visible and
// clickable so the user can still pick any bound they want.
booked:
'relative font-semibold text-text after:absolute after:bottom-0.5 after:left-1/2 after:h-1 after:w-1 after:-translate-x-1/2 after:rounded-full after:bg-primary after:content-[""]',
}}
className="rounded-md border border-border bg-bg p-2"
/>
</div>
)
}
function parseIsoDate(s: string): Date {
// yyyy-mm-dd → Date with local components so a user's "2024-01-15"
// never rolls back to 2024-01-14 in a pacific timezone.
const [y, m, d] = s.split('-').map(Number)
return new Date(y, (m ?? 1) - 1, d ?? 1)
}
function formatIsoDate(d: Date): string {
const y = d.getFullYear()
const m = String(d.getMonth() + 1).padStart(2, '0')
const day = String(d.getDate()).padStart(2, '0')
return `${y}-${m}-${day}`
}

Some files were not shown because too many files have changed in this diff Show More