Commit Graph

71 Commits

Author SHA1 Message Date
fc8dd370c2 feat: "On this day" memories — photos from previous years
Add a Memories view that surfaces photos taken on the current date in
previous years (like Google Photos / Immich). Only uses EXIF-sourced
dates to avoid false matches from filesystem timestamps.

- Backend: GET /api/v1/photos/memories returns groups by year, up to
  12 photos each, filtered to non-discarded/non-hidden EXIF dates
- Frontend: MemoriesView with year-grouped thumbnail grid
- Sidebar: new "Memories" nav item with clock icon

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 22:18:28 +02:00
bbb8e4850c feat: GPU acceleration support for ONNX Runtime inference
Centralize execution provider selection in providers.py with
auto-detection and graceful fallback. All ONNX sessions (embedder,
detector, face processor, recognizer) now use the configured providers.

- New VISION_EXECUTION_PROVIDERS env var: "auto" for GPU auto-detect,
  or explicit "CUDAExecutionProvider,CPUExecutionProvider"
- Provider priority: CUDA > ROCm > OpenVINO > CPU (when set to "auto")
- docker-compose.yml includes commented-out NVIDIA GPU deploy section
- Supports onnxruntime-gpu as a drop-in replacement for onnxruntime

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 22:12:21 +02:00
94c07b1d0d feat: upgrade to SigLIP2 ViT-B/16 for semantic search
Replace OpenCLIP ViT-B/32 (512-d, ~78% recall) with SigLIP2 ViT-B/16
(768-d, ~84% recall) as the default embedding model for significantly
better image-text retrieval quality.

- New SigLIP2Embedder class with 384px input and SigLIP normalization
- ONNX export pipeline for SigLIP2 visual + textual encoders
- Migration 0010: resize embeddings.vector from 512 to 768 dimensions
- Config-driven model selection: "siglip2_vitb16" (default) or
  "openclip_vitb32" (legacy) — both models can coexist
- Content classifier follows the configured embedder family
- Existing embeddings cleared on migration; vision backfill regenerates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 22:09:16 +02:00
c7dd03ade2 feat: CLIP-powered incremental duplicate detection
Replace O(N²) pHash-only duplicate detection with a hybrid approach:
- pHash Hamming distance for exact/near-exact copies
- CLIP embedding cosine similarity via pgvector HNSW for visually
  similar photos (crops, format changes, screenshots)

Post-scan now uses incremental mode: only newly added photos are
compared against the full library — O(new × log N) via HNSW index
instead of O(N²). Full regroup remains available from Settings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 22:03:04 +02:00
348e9c3585 feat: multi-user auth with per-user media isolation
Introduce username/password authentication with admin and user roles.
Each user gets their own media directory under /photos/{username}/ with
isolated photos, folders, heaps, and tags. Admins manage users and
observe the full library from a dedicated Settings page.

Backend:
- User model with bcrypt passwords and JWT access/refresh tokens
- Auth router (login, refresh, setup, change-password, status)
- Admin router (user CRUD with last-admin protection)
- user_id FK added to photos, folders, source_roots, heaps, tags
- All data routers scoped by authenticated user
- Scanner inherits user_id from source root owner
- Thumbnails stored under user-prefixed paths for isolation
- Library endpoints accept ?scope=global for admin cross-user view
- Alembic migration 0009 with data migration for existing installs
- Defensive bootstrap.py handles fresh vs existing DB startup

Frontend:
- AuthContext with token lifecycle, auto-refresh, login/logout
- Login page, first-run setup page, auth gate in App.tsx
- Bearer token interceptor on all API requests
- User identity + logout in left sidebar
- Admin-only Settings page with Library Management and Users tabs
- UserManagement panel (add, edit role, reset password, deactivate)
- Settings shows global stats across all users for admin
- Filter bar, right sidebar, keyboard hints hidden on settings page

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:47:43 +02:00
root
03a4c75e3e fix: reduce db pool size to prevent connection exhaustion, sort grouped views by count
Pool was 20+10 overflow per engine; uvicorn --reload leaked pools until
Postgres hit max_connections=100. Reduced to 5+5 with pool_pre_ping.

Grouped views (tags, people, colors, rated) now sort cards largest-first.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:32:05 +02:00
root
d933ea3842 feat: auto-chain vision backfill + face recluster after scan, show progress in UI
Scan now automatically queues backfill_vision (+90s) and recluster_faces
(+300s) after dispatching folder scans. Face extraction also schedules a
debounced recluster via Redis so incremental file-watcher imports get
clustered without manual intervention.

The ScanProgress widget now tracks worker queue activity beyond the scan
phase, showing a "Processing Photos" indicator with vision queue counts
while background tasks (embeddings, faces, tags, OCR) are running.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:54:27 +02:00
38d975e354 feat: highlight matched metadata on thumbnails during search
Renders a thin top-of-cell banner on each photo while a text search
is active, labelling which metadata field matched (filename, title,
note, tag, or EXIF key name) and showing a short excerpt with the
exact matched substring highlighted in amber. Extends the list
endpoint's search to also match photos whose tag names contain the
query so tags show up alongside filename/title/notes/EXIF hits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 21:38:27 +02:00
30d03d8d4d feat: editable taken_at + folder-based date repair and filter
Lets operators fix corrupted capture dates at scale. Adds an editable
Date Taken field with a folder/filename-derived suggestion hint, a bulk
Date Taken section in the multi-select sidebar that either applies one
date to the whole selection or infers a per-photo date from each path,
a warning badge on thumbnails whose stored date disagrees with the
path, and a "Date issues" filter pill so suspicious photos can be
surfaced and fixed as a group. Edits are written back to EXIF on disk
so rescans don't clobber the fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:48:55 +02:00
root
339e1be510 feat: hide-from-views flag on folders
Adds a per-folder "hide from views" toggle so noisy subtrees
(screenshots, WhatsApp dumps, work archives) can be excluded from
cross-cutting views without losing indexing. Photos under a hidden
folder are still scanned, thumbnailed, embedded, OCR'd, face-
extracted — they just stop appearing in All Photos, Rated, Colors,
Map, Tags, People, Search, Duplicates, and the sidebar counts.
Navigating directly into the folder still shows every photo.

Schema (migration 0007_folder_hidden):
- folders.is_hidden   user-set toggle, default false
- photos.is_hidden    denormalized effective flag (true iff any
                      ancestor folder is hidden), indexed so cross-
                      cutting queries stay on the existing planner
                      paths

The denorm is maintained by two paths:
- The scanner walks the ancestry chain on insert, with a per-scan
  memoized cache so each folder is resolved once per scan.
- POST /api/v1/folders/{id}/hide flips folders.is_hidden and runs a
  WITH RECURSIVE CTE to recompute every folder's effective state in
  one query, then bulk-updates photos WHERE IS DISTINCT FROM. Runs
  in ~10 ms on a 13k-photo library.

Filters added (cross-cutting queries):
- /library/stats — every sidebar badge via a shared `visible` filter
- /photos (list) — only when neither folder_id nor heap_id is set;
  folder browse and heap browse always show everything
- /photos/map
- /library/duplicates/groups
- /folders/tree photo_count subquery
- /tags count_subq (drives Tags + People sidebar counts)
- services/duplicates.regroup_duplicates (so hidden dupes never
  contaminate the Duplicates view)
- services/search.hybrid_search — both semantic (pgvector) and FTS
  legs join photos so rankings don't include hidden results

Intentionally NOT filtered:
- /photos?folder_id=X and /photos?heap_id=X (user-intentional browse)
- /library/maintenance/pipeline-stats (tracks real worker state)
- cleanup service (disk-level ops, not views)

Frontend:
- sourceFolders.setHidden(id, hidden) API client method
- FolderTreeNode.is_hidden carried through the tree into TreeItem
- LeftSidebar kebab menu: "Hide from views" / "Show in views" with a
  mutation that invalidates folders, photos, stats, and tags caches
- Hidden folder rows swap the Folder icon for EyeOff and render the
  label italic/muted so the state is visible at a glance

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 10:55:25 +02:00
root
07b1e5e02a feat: split celery workers, fix asyncpg-in-fork, add pipeline progress UI
Three overlapping fixes so the ingestion pipeline actually runs and the
user can see what it's doing:

Pipeline recovery
- app/database.py: use NullPool when MULITA_CELERY_WORKER=1 so each
  Celery task opens a fresh asyncpg connection on its own event loop.
  Fixes "another operation in progress" and "Future attached to a
  different loop" errors that were dropping ~every thumbnail +
  extract_metadata task on the floor.
- app/tasks/thumbs.py: initialize photo=None before the try and rollback
  on error so a transport failure in the initial SELECT doesn't raise
  UnboundLocalError in the except block and leak rows stuck in 'pending'.
- app/services/vision/bootstrap_models.py: on missing model files,
  invoke export_models automatically instead of just warning. First
  boot of a fresh install now self-heals.
- app/services/vision/export_models.py: shutil.move instead of
  Path.rename so the YOLO export survives the /app → /data/models
  cross-volume hop.
- requirements.txt: add ultralytics so export works in a stock image.

Worker topology
- docker-compose.yml: replace the single worker with worker-light
  (default/high/low queues, c=2, IO-bound) and worker-vision (vision
  queue, c=5, OMP_NUM_THREADS=1 to avoid oversubscription on 6 cores).
  Vision is pinned to ≤5 parallel inferences so ONNX doesn't each
  spawn an all-cores intra-op pool.
- .env / .env.example: CELERYD_CONCURRENCY replaced with
  CELERY_LIGHT_CONCURRENCY + CELERY_VISION_CONCURRENCY.
- Backfill queries in thumbs / scan / vision now ORDER BY taken_at
  DESC NULLS LAST so newest photos finish first — the library fills
  in top-down in the UI instead of arbitrary insertion order.

Settings visibility
- routers/library.py: new GET /maintenance/pipeline-stats returning
  done/total per stage (thumbnails, exif, gps, phash, embeddings,
  tags, ocr, faces, face clusters, duplicate groups). Worker-status
  now also reports the `vision` queue depth, which was missing.
- services/api.ts: PipelineStats / PipelineStage / ScanStatus types
  and the matching client call.
- components/dialogs/SettingsDialog.tsx:
  - new Pipeline Progress card with one progress bar per stage
  - inline scan banner (processed/total/current folder) inside the
    Library section while a scan is running
  - Tasks/min throughput computed by diffing worker processed counters
    between polls
  - Workers section calls out the vision queue and documents the
    CELERY_LIGHT/VISION_CONCURRENCY + docker compose up -d scale path

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 10:06:45 +02:00
4bc6dc1dc8 feat: refactor grouping views into card-grid browse pattern
Replace Timeline-based grouped views (tags, colors, rated) with
dedicated card-grid components that drill into Timeline detail views
on click/Enter. Adds shared useCardGridNav hook for arrow-key
navigation across all four card grids (tags, colors, rated, people).

- TagsView, ColorsView, RatedView: card grid → inline Timeline detail
- PeopleView: migrated to same pattern (Timeline replaces custom grid)
- Tags endpoint: fall back to first associated photo for representative
- Filter store: add ratingMax for exact rating filtering in RatedView
- Timeline: remove tag/rating/color grouping; skip date headers when
  groupBy != 'date' so detail views render flat grids
- SettingsDialog: bump z-index above Leaflet map layers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 15:48:50 +02:00
fa9b21856f feat: replace face pipeline with InsightFace, add content classifier
Face detection/recognition:
- Replace YuNet + SFace with InsightFace buffalo_l (RetinaFace + ArcFace)
- 512-d ArcFace embeddings (was 128-d SFace), migration 0006 resizes column
- Remove YOLO person-bbox workaround — RetinaFace is accurate enough
- Detection threshold 0.65 cleanly separates real faces (0.72+) from
  false positives on dogs/paintings (0.56-0.61)

Content-type classification:
- CLIP zero-shot classifier using native PyTorch text encoder + ONNX
  image encoder for high-quality text-image similarity
- Categories: photograph, screenshot, document, receipt, meme, artwork
- Writes Tag(kind=content_type) per photo via photo_tags
- Margin-based confidence: top-1 vs top-2 score difference
- New ClassifierSettings in config (enabled, min_confidence)
- Wired into vision_fanout pipeline

Tested: 6 real faces from 4 photos (zero false positives), 11/13 photos
classified (8 photograph, 2 artwork, 1 meme).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 13:49:02 +02:00
f48e099bd2 fix: verify faces against YOLO person detections for precision
Cross-reference face detections with YOLO 'person' bounding boxes —
only keep faces that overlap >= 50% with a detected human body. This
eliminates false positives on dogs, paintings, and cartoons without
needing an aggressive score threshold.

Lower face detection threshold back to 0.6 since the person-overlap
check is now the primary precision filter.

Tested: 6 verified faces from 4 photos, zero false positives.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 11:13:01 +02:00
40d570f2c2 fix: raise face detection threshold to 0.85, tighten clustering
Eliminates false positives (dog faces, painting faces) by requiring
score >= 0.85. Tighten cluster eps to 0.25 for better separation.
Tested: 4 real faces from 2 photos, no false positives.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 11:09:43 +02:00
5f3fa5240e fix: search endpoint — correct column name and DISTINCT/ORDER BY
- Fix Photo.created_at → Photo.added_at (column doesn't exist)
- Fix Postgres DISTINCT + ORDER BY conflict by using a subquery for
  tag_id filtering instead of JOIN + DISTINCT on the outer query

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 10:55:55 +02:00
229611b4c3 fix: face clusters write photo_tags, raise detection threshold to 0.7
- recluster_faces now writes photo_tags rows for each face cluster so
  the tag count and tag_ids filter work (previously count was always 0)
- Old cluster tags and photo_tags are cleaned up before re-clustering
- Raise face detection threshold from 0.4 to 0.7 to reduce false
  positives (was detecting dog faces as people)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 10:48:19 +02:00
db20cbb7d8 fix: face detection — use OpenCV FaceDetectorYN and full-res originals
- Rewrite faces.py to use cv2.FaceDetectorYN instead of raw ONNX
  (handles multi-scale anchor decoding and NMS internally)
- Load original photo files at up to 4000px for face detection instead
  of 240px thumbnails — faces were too small to detect at thumbnail res
- Falls back to thumbnail if original is unavailable

Tested: 33 faces extracted from 13 photos, clustered into 1 person.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 10:43:29 +02:00
7558aeb5e6 fix: sync DB sessions in Celery, letterbox YuNet, dedupe detections
- Rewrite all vision tasks to use sync psycopg2 sessions instead of
  asyncpg — fixes 'another operation in progress' and event loop errors
  when Celery forks workers sharing the async connection pool
- Letterbox-pad images to exactly 640x640 for YuNet face detector
  (was crashing on non-square thumbnails)
- Deduplicate object detections per label per photo — keep highest
  confidence only to avoid photo_tags PK violation on multiple
  detections of the same class
- Add all queues (-Q default,high,low,vision) to worker command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 10:28:17 +02:00
2a6661f779 fix: model weights setup — export scripts, ORT compat, bootstrap
- Add export_models.py for OpenCLIP ViT-B/32 and YOLOv8n ONNX export
- Fix ArgMax(13) ORT ARM64 incompatibility by passing eot_indices as a
  separate ONNX input (computed outside the graph in embed.py)
- Use legacy TorchScript exporter (dynamo=False) for IR version 9 compat
- Upgrade onnxruntime to 1.18.1
- Rewrite bootstrap_models.py with clear separation of auto-downloadable
  models (YuNet, SFace) vs manually-exported ones (OpenCLIP, YOLOv8n)
- Wire bootstrap into worker CMD (runs before Celery)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:41:25 +02:00
ad007e4cd4 feat: add face detection, recognition, and clustering
- Create face_embeddings table with pgvector Vector(128) + HNSW index
- Implement extract_faces task (YuNet detection + SFace recognition)
- Implement recluster_faces task (DBSCAN clustering → Tag(kind=face_cluster))
- Clusters are named "Person N" and get representative_photo_id
- cluster_id FK → tags.id, SET NULL on delete for merge/rename support

Migration 0005 creates the face_embeddings table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:13:41 +02:00
1ebc4bfe73 feat: add YOLO object detection writing to unified Tag model
Implement detect_objects Celery task:
- Runs YOLOv8n on 640px thumbnail via ONNX Runtime
- Creates Tag(kind=object) rows for each COCO class detected
- Writes photo_tags associations with confidence, bbox, and source
- Wipes previous detections per source model on re-run

No new tables/migrations — uses the unified Tag model from PR3.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:11:26 +02:00
842a4fc864 feat: add OCR text extraction and Postgres full-text search
- Create ocr_text table for storing per-region OCR results
- Add tsvector search_vector column to photos with GIN index and
  auto-update trigger on filename/user_title/user_notes
- Implement ocr_photo Celery task using rapidocr-onnxruntime
- Add FTS leg to hybrid search: queries photos.search_vector and
  ocr_text via UNION, fused with semantic results via RRF (k=60)

Migration 0004 backfills search_vector for existing rows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:10:14 +02:00
649437dc85 feat: add embeddings pipeline and semantic search endpoint
Wire the full embedding flow:
- Rewrite Embedding model to use pgvector Vector(512) with HNSW index
- Add embed_photo, vision_fanout, backfill_vision Celery tasks on
  dedicated `vision` queue
- Hook vision_fanout into generate_thumbnails completion
- Add POST /api/v1/photos/search with hybrid RRF ranking (semantic-only
  for now; FTS leg added in PR5)
- Stub ocr_photo, detect_objects, extract_faces tasks for later PRs

Migration 0003 drops/recreates the embeddings table (was never populated).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:07:32 +02:00
b1c2bdf7f0 feat: extend Tag model for unified ML tagging
Unify object detections, scene labels, and face clusters with user tags
via new columns on the existing Tag model:
- kind (user|object|scene|face_cluster), source, representative_photo_id
- photo_tags gains confidence, bbox (JSONB), source per-association
- Uniqueness moves from (name) to (name, kind) so ML labels coexist
  with user tags without collision

Add Alembic migration 0002 with defensive IF NOT EXISTS guards.

Update tags router: kind filter on GET, merge endpoint for combining
auto-detected clusters/objects, include kind/source in list response.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:02:32 +02:00
9282a5c734 feat: add vision pipeline scaffolding with ONNX backend
Introduce the app/services/vision/ module with ABC interfaces, ONNX
Runtime backend, model registry, and per-task implementations:
- OpenCLIP ViT-B/32 embedder (image + text, 512-d)
- RapidOCR engine (PP-OCRv4 via ONNX, no PaddlePaddle)
- YOLOv8n object detector (raw ONNX, no ultralytics runtime)
- YuNet + SFace face processor (Apache 2.0, opencv_zoo, 128-d)
- DBSCAN face clustering helper

Add VisionSettings to config (mulita.yml + Pydantic), bootstrap_models.py
for first-boot weight downloads, models_data Docker volume, and ROCm
backend stub for future GPU acceleration.

No Celery tasks wired yet — models load but nothing invokes them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:00:06 +02:00
dea04ceed9 feat: migrate to Postgres + pgvector with Alembic scaffolding
Switch the default database from SQLite to Postgres + pgvector (via
pgvector/pgvector:pg16 Docker image) to support the upcoming vision
pipeline (embeddings, OCR, object detection, face clustering).

- Add `db` service to docker-compose.yml with healthcheck
- Wire `alembic upgrade head` into backend CMD before uvicorn
- Bootstrap empty 0001_baseline revision (schema still owned by create_all)
- Guard SQLite-only PRAGMAs and inline ALTERs behind _is_sqlite flag
- Run `CREATE EXTENSION IF NOT EXISTS vector` on Postgres init
- Add asyncpg, psycopg2-binary, pgvector to requirements
- Provide docker-compose.sqlite.yml escape hatch for legacy SQLite mode

Fresh DB + rescan assumed — no SQLite→Postgres data migration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:46:20 +02:00
7cf546af7a feat: map view with GPS extraction fix
Adds a new Map sidebar entry that plots photos by their EXIF GPS
coordinates on a clustered Leaflet map. While wiring this up, the
metadata extractor was reading unprefixed GPS keys that never exist
in `exiftool -G -j` output AND assumed coordinates were already
floats — every photo silently lost its GPS. The new extract_gps
helper handles Composite/EXIF group prefixes and parses DMS strings,
and lat/lon are stored as first-class indexed columns so the map
can query them without parsing exif_json on every request.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 23:44:29 +02:00
1c6383e5a0 feat: colored count badge on Colors sidebar entry
Adds a `colored` field to /library/stats counting non-discarded
photos with a color_label set, mirroring how `rated` is exposed.
The Colors sidebar entry now shows the live count and refreshes
through the existing LIBRARY_STATS_QUERY_KEY invalidations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:43:19 +02:00
733c16bf82 feat: perceptual-hash duplicate detection + grouped picker view
The Duplicates section was useless: SHA-256-only detection only caught
byte-identical files, not the actual duplicates a real library
accumulates (re-encoded JPEGs, screenshots, resized exports), and the
view was a flat date-sorted list with no grouping or actions. This
replaces the whole flow.

Detection
- New phash + duplicate_group_id columns on Photo, added via an
  idempotent ALTER TABLE pass in init_db (the project has no Alembic).
- Thumbs worker computes a 64-bit pHash from the original-resolution
  decoded frame just before the destructive thumbnail loop. Falls back
  silently — phash is nice-to-have, not a blocker for thumbnails.
- backfill_phashes Celery task fills in phashes for photos that
  predated the column, reading the existing thumb_large rather than
  re-decoding the original.
- regroup_duplicates service runs union-find over Hamming distance
  (threshold 6), persists duplicate_group_id, and maintains is_duplicate
  as derived state so existing badges/counts keep working. Chained
  after scan_all_source_roots with a 60s countdown.

API
- GET /library/duplicates/groups returns all groups with members,
  bucketed in Python from one query. Each group has a reason ("exact"
  iff every member shares a SHA-256, "similar" otherwise).
- POST /library/maintenance/{regroup-duplicates,backfill-phashes}.

Frontend
- New DuplicatesView (sectioned grid, one section per cluster) replaces
  the timeline when the user is in the duplicates section. Each section
  shows a "Keep best, discard N" button that picks the highest-pixel
  copy and reuses the existing undoable bulk-discard so Cmd+Z works.
- Manual best override: hover any non-best thumbnail and click "Keep
  this" (Crown icon, top-right) to override the auto-pick. The header
  annotates "(manual)" so it's obvious which copy will be kept.
- Keyboard nav within the duplicates view walks the flat member list,
  with ↑/↓ jumping by the measured column count and scrollIntoView on
  every move. Timeline's keyboard handler now early-returns in the
  duplicates section so the two don't fight.
- BEST pill / Keep-this button live at top-right with a ring outline so
  they don't collide visually with the cyan selection ring around a
  selected cell. Dimensions chip moved to bottom-left to free both
  right corners for the keep affordances.
- New "Duplicates" section in SettingsDialog: shows group/member counts
  and exposes both backfill + re-detect actions, sharing a query cache
  with DuplicatesView via DUPLICATE_GROUPS_QUERY_KEY.
- PhotoInfoPanel "Basic Info" section now shows the photo's full file
  path in monospace below the size/dimensions/date grid.
- New imagehash==4.3.1 dep in requirements.txt.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 17:19:08 +02:00
e51b93d59e perf: speed up settings dialog + relocate settings entry point
- Parallelize the six celery inspect.*() calls in /library/maintenance/
  worker-status via asyncio.gather + to_thread, and drop per-call
  timeout from 1.0s to 0.5s. Endpoint goes from ~6.1s to ~0.54s — it
  was the sole bottleneck on opening the Settings dialog.
- SettingsDialog now fetches through React Query with enabled:isOpen,
  so reopening shows cached data instantly while a background refetch
  updates. Worker polling moved to refetchInterval. Loading spinners
  only show when there's no cached data yet, so background refetches
  don't keep them spinning.
- Move the Settings entry point from the TopBar to a pinned row at the
  bottom of the LeftSidebar so it sits alongside the other library
  controls. TopBar no longer takes onOpenSettings.
- Remove the "Scan all folders" bottom action from LeftSidebar — the
  same control already lives in Settings → Library → Re-scan.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 16:31:52 +02:00
root
d27ec1af2e fix: prune orphaned folder rows alongside photo rows
prune_missing_photos() previously only deleted Photo rows whose files
were gone, leaving every folder row from the old library in the DB —
which made the sidebar tree wildly out of sync with the on-disk
structure (still showing /photos/2024/, /photos/2026/03/, etc. that
no longer exist).

Now also drops Folder rows whose path doesn't resolve under a mounted
source root, with the same defensive "skip if source root unmounted"
guard. The Settings orphan card surfaces both counts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 10:50:32 +02:00
root
697343646a feat: prune orphaned photo rows + retry-pending action
Adds /api/v1/library/maintenance/{missing-stats,prune-missing} backed
by a new cleanup helper that deletes Photo rows whose files no longer
exist on disk under a *mounted* source root. Skips photos under
unmounted roots so a temporarily-disconnected drive doesn't get
silently nuked.

Settings panel surfaces the orphan count with a destructive Prune
button, plus a "Kick pending" action that re-queues photos stuck in
processing_status='pending' (typically left behind when the scanner
created the row but the worker never picked up the thumbnail task).

Common trigger: PHOTO_DIRS in .env was repointed at a different
library root, leaving every old row dangling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 10:47:06 +02:00
root
3df8add3b6 fix: stop scan crashing on duplicate file hashes
scan.py used scalar_one_or_none() to test whether any other photo
shared the same file_hash, but that helper raises MultipleResultsFound
the moment 2+ rows match — i.e. exactly the duplicate case it was
trying to flag. Every file beyond the second copy bombed out with
"Multiple rows were found when one or none was required" and was
left in the failed bucket. Replace with a COUNT(*) > 0 check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 10:14:40 +02:00
root
42250aa16e feat: worker diagnostics in settings panel
Adds /api/v1/library/maintenance/worker-status (Celery inspect + queue
depths + recent failed photos) and a Workers section in the Settings
dialog so users can debug stuck queues and task failures without
tailing container logs. Auto-polls every 5s while open.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 10:11:33 +02:00
9ed577f40c feat: settings panel + thumbnail pipeline fixes
- photos.py: stop crashing in FileResponse when a thumb hasn't been
  generated; return a clean 404 with Retry-After so the frontend can
  back off.
- thumbs.py: fix process_video_thumbnail (overwrite_output, robust
  duration probe across stream/format, eager frame load + temp cleanup)
  so videos stop ending up as the gray placeholder.
- library.py: new /maintenance/* endpoints — thumbnail-stats,
  regenerate-thumbnails (with media_type / only_failed filters), and a
  manual data-integrity cleanup trigger.
- Frontend Settings panel (gear in TopBar) surfacing those endpoints
  plus a re-scan button and live thumbnail status counts.
- PhotoThumbnail: stretch the auto-retry schedule for slow RAW jobs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 09:24:08 +02:00
f05ae77ef0 feat: folder CRUD with discard-or-delete dialog
The left sidebar can now create, rename, and delete folders. Each
operation is mirrored to disk through the backend.

Backend (folders router):
- POST /folders { name, parent_id } — create a sub-folder under an
  existing Folder row, mkdir on disk, insert the row, return it. Names
  are validated (no separators, no traversal).
- PATCH /folders/{id} extended — still does the display-only rename for
  SourceRoot ids, but for Folder ids it now actually moves the directory
  on disk and rewrites every descendant Folder.path + Photo.filepath
  that lived under the old prefix in a single transaction. Refuses to
  rename the source-root mount itself.
- DELETE /folders/{id}?mode=discard|permanent —
    discard: set is_discarded on every photo whose filepath lives under
             this folder. The folder, descendants, and on-disk dir are
             left intact. Recoverable from the discard pile.
    permanent: unlink each file, remove rows, rmtree the directory.
- Refuses to delete the source-root mount in either mode.

Frontend:
- New DeleteFolderDialog: two-card mode picker (Move to discard pile /
  Permanently delete) with destructive accent on the latter. Esc and
  backdrop click cancel.
- LeftSidebar: hover-revealed kebab menu on every folder row with
  New sub-folder, Rename, and Delete folder…  Inline create input
  appears below the parent row when "New sub-folder" is picked.
  All mutations invalidate ['folders'], ['photos'], and the library
  stats query so the sidebar counts stay live.
- api.ts: sourceFolders.create + sourceFolders.delete wrappers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:51:23 +02:00
bd904aca36 fix: assorted UI polish from review pass
- FilterPill: drop the inline value text from the active state. Pills
  now stay the same width whether or not a filter is set; the popover
  is the canonical place to read the value, and the title attribute
  surfaces it on hover.
- TopBar: remove the search input — search lives in the filter bar now.
- FilterBar: add a search input on the left, with the pill cluster
  centered between it and a flex-shrink-0 Clear-all on the right.
- LeftSidebar / HeapsPanel: count badges use a fixed-width slot
  (h-5 min-w-[24px], tabular-nums) so counts line up in the same
  visual column across rows. Empty rows reserve the slot.
- LeftSidebar: pull section counts (All Photos, Rated, Duplicates,
  Discarded) from a new useLibraryStatsQuery hook backed by the
  expanded /library/stats endpoint. Tags count was already wired.
- backend/library: stats endpoint returns per-section counts that
  match the filter the sidebar applies on click.
- Stats invalidation hooked into the standard photo-mutation paths.
- RightSidebar header: h-12 to match TopBar height.
- Timeline sticky date overlay: only show once the natural in-grid
  header has scrolled OUT of the viewport. Avoids the duplicate-label
  flash when both labels would be visible.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:45:30 +02:00
696477eefd feat: heap row kebab menu with rename + duplicate
The heap row used to fan out three small icon buttons (set active, convert
to folder, delete) on hover, which crowded the row and didn't leave room
for new actions. Collapse the destructive / occasional ones into a kebab
menu and add the missing operations.

- Right-aligned action cluster: active indicator → count badge → target
  toggle (when not active) → kebab menu, all flex-shrink-0 so the name
  truncates first.
- Kebab menu items: Rename, Duplicate, Move to folder…, Delete. Outside
  click and Escape close the popover; the trigger has aria-haspopup +
  aria-expanded. Delete still confirms via window.confirm.
- Inline rename: double-click a heap row OR pick Rename from the menu
  to edit the name in place. Enter commits, Escape cancels. Mirrors the
  folder rename pattern in LeftSidebar.
- backend: new POST /heaps/{id}/duplicate creates a copy with the same
  membership ("{name} (copy)") via INSERT...SELECT on heap_photos.
  Never marks the new heap as active so duplicating doesn't quietly
  steal the user's T-key destination.
- api.ts: heaps.duplicate wrapper.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:36:00 +02:00
fc63b1f69d config: env-driven CORS, ports, log level, timezone
The CORS allowed-origins list, host port mappings, log level, container
timezone, and worker concurrency are now all driven by environment
variables with sane defaults. Same-origin access through the nginx
proxy keeps working with no config; direct cross-origin backend
access can be locked down via ALLOWED_ORIGINS.

- backend/config: ALLOWED_ORIGINS env (comma-separated, "*" for any)
  exposed via settings.cors_origins. LOG_LEVEL too.
- backend/main: build the CORS middleware from settings.cors_origins,
  auto-disable allow_credentials when origins is wildcard (CORS spec
  forbids credentials + "*").
- docker-compose: parameterize FRONTEND_PORT, BACKEND_PORT, REDIS_PORT,
  CELERYD_CONCURRENCY, LOG_LEVEL, and TZ via ${VAR:-default} so each
  has a working fallback if the .env entry is missing.
- .env.example: new template documenting every knob with examples.
- .env: pruned to only the values that diverge from .env.example;
  removed dead VITE_API_URL.
- README: configuration knobs table + "accessing from another machine"
  section explaining the same-origin proxy story.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:31:59 +02:00
ac5b18b60c fix: cross-machine access + center filter bar + floating hints
- api.ts: switch baseURL from http://localhost:8001/api/v1 to relative
  /api/v1. Both nginx (prod) and vite (dev) already proxy /api/ to the
  backend, so requests become same-origin and the app works from any
  host (LAN IP, reverse proxy, another machine) with no CORS dance.
- backend CORS: open to "*" as a fallback for the rare direct-hit case;
  the normal flow is same-origin via the proxy and never touches CORS.
- App layout: move FilterBar and DiscardActionBar inside the main
  content column (right of the left sidebar) so the filter row no
  longer bleeds across the sidebar.
- FilterBar: justify-center the pills so they sit centered above the
  timeline. Clear-all uses ml-2 instead of ml-auto.
- KeyboardHints: convert to a floating, glassy pill pinned bottom-
  center (fixed positioning + backdrop-blur + ring) instead of a flat
  toolbar row. Removed from the column layout — now mounted as an
  overlay sibling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:28:04 +02:00
a5b4054a71 feat: bulk tag add/remove on multi-select right sidebar
The bulk action panel previously covered rating, color, flag, and pick
but had no way to apply tags across a multi-photo selection — the only
path was to tag photos one at a time via the single-photo PhotoInfoPanel.
Add it.

- backend: extend the existing /photos/bulk action endpoint with
  add_tags and remove_tags actions. add_tags is idempotent (computes
  the new (photo_id, tag_id) pair set against existing rows and inserts
  only the missing ones); remove_tags is a single DELETE WHERE IN.
- api.ts: bulkAddTags / bulkRemoveTags wrappers.
- RightSidebar: new BulkTagsEditor below the bulk flag row. Filters /
  searches the existing tag list, lets the user click any chip to apply
  it to the whole selection or X to remove it. Typing a name with no
  exact match shows a "Create and apply" button that creates the tag
  via tagsApi.create and immediately attaches it to every selected
  photo. All three mutations invalidate both the photo and tag caches
  so the FilterBar tag count + sidebar Tags section stay fresh.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:25:18 +02:00
a55839d9a2 fix: more audit findings — perf, types, and a11y polish
- backend/photos: collapse the per-tag subquery loop in the tag filter
  into a single GROUP BY ... HAVING COUNT(DISTINCT) = N subquery so the
  cost is independent of how many tags the user is filtering on.
- useFilterUrlSync: type the parseUrl return value as
  Partial<FilterState> & { currentSection?: string } so the section field
  doesn't need an (out as any) cast.
- Timeline sticky header: bump opacity, padding, and border so it reads
  more clearly against the underlying grid.
- FilterPill clear: convert the nested <button> (invalid HTML — buttons
  cannot nest) to a span with role=button + keyboard handler, with a
  larger hit area.
- RightSidebar: add aria-label to the close-X buttons so screen readers
  announce them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:22:23 +02:00
749e836617 fix: high-severity findings from code audit
- backend/photos: whitelist sortable columns instead of getattr(Photo, sort).
  Previously any client-supplied string was passed to SQLAlchemy, exposing
  every Photo attribute (filepath, file_hash, etc.) as a sort target.
- App: move the auto-show-right-sidebar logic out of the render body and
  into a useEffect. The previous version called setState during render,
  causing extra re-render passes the audit caught.
- types/photo: add added_at and tighten folder_id from optional to nullable.
  Drops a (photo as any).added_at cast in Timeline.
- constants/colorLabels: extract a single COLOR_LABEL_OPTIONS used by
  FilterBar, RightSidebar, and PhotoInfoPanel. filterStore re-exports the
  ColorLabel type so existing imports keep working.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:18:19 +02:00
b870084be0 feat: per-photo permanent delete + discarded thumbnail treatment
Discarded photos now look discarded in the grid (50% opacity + grayscale)
with a red trash badge in the corner instead of a bare icon. The discard
action bar gains a "Delete N" button that permanently deletes only the
current selection, complementing the existing "Empty discard pile".

Backend: new DELETE /discard endpoint accepting {photo_ids: [...]} that
permanently removes only listed photos. Skips ids that aren't in the
discard pile so it can never bypass the soft-delete safety net.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 13:12:11 +02:00
8413b112ee fix: folder tree counts computed from photos (not stale folder.photo_count)
The sidebar showed Juno=7 and sub=blank because the scanner's
folder.photo_count bookkeeping is broken end-to-end:

  for root, dirs, files in os.walk(folder_path):
      folder = await get_or_create_folder(...)
      ...
      processed_files += 1     # global counter

  # AFTER the loop:
  folder.last_scanned = datetime.utcnow()
  folder.photo_count = processed_files   # only the LAST folder

processed_files is the running total across the whole walk, not
per-folder; and the assignment runs once after the loop, only on
whichever folder os.walk happened to visit last. Result: that folder
gets the grand total, every other folder gets nothing (or stale).

Rather than fix the scanner's bookkeeping (which has leaked into
two production scans already), the tree endpoint now computes
counts on demand from the photos table:

- One GROUP BY per source root: photo.folder_id → COUNT, excluding
  discarded
- Each node starts with its DIRECT count
- A post-order walk accumulates descendants so every node reports
  recursive count — i.e. clicking the row gives you that number of
  photos because the photos query also expands descendants

The stored Folder.photo_count column is now unused by the API. A
future cleanup could drop it from the model entirely.

Verified on the dev DB: Library=7 (4 direct + Juno=2 + sub=1),
Juno=2, sub=1.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 12:11:22 +02:00
db37be902e fix: disable folder watcher on startup to unblock scan_folder
Diagnosis: every backend restart was dispatching watch_folders.delay()
unconditionally. watch_folders is an infinite-loop celery task
(for changes in watch(*paths)). With CELERYD_CONCURRENCY=4 and several
restarts during dev, all four worker slots ended up pinned by stale
watch_folders instances, leaving zero workers free for scan_folder.
The result: clicking "Scan all folders" successfully queued a task
that then sat in the queue forever, the new /photos/sub folder was
never walked, and the user's newly added photo never appeared.

The watcher was only opportunistically useful and the user already
triggers scans manually. Disabling it removes the foot-gun. Re-
enabling needs:
  - a Redis lock so only one watcher runs at a time
  - or a dedicated long-running container with concurrency=1
  - or a celery beat schedule with a singleton flag

Until then, manual scans work. Cleared the backlog by wiping the
redis broker volume so the stale watch_folders tasks are gone.

Verified: post-fix, scan_folder runs in 0.12s and reports
"Processed 7/7 files. Errors: 0", picking up the previously missing
/photos/sub/Samuel_Colman... file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 12:01:19 +02:00
6985026106 feat: tag-grouped timeline view
Reworks the Tags sidebar entry from an expandable list of tags into a
single leaf entry. Clicking it switches the timeline grouping mode to
"tag" — every tag becomes a sticky-headered group, with an "Untagged"
group at the bottom for photos with no tags. A photo with N tags
appears in N groups. Existing filters and sort still apply within each
group.

Backend
- list_photos eagerly loads Photo.tags via selectinload to avoid an
  N+1 round-trip.
- Each photo in the list response now carries a `tags: [{id, name,
  color}]` array. The route stops using PhotoListResponse strict
  validation (returns a plain dict with the same shape plus the new
  field) so we don't have to extend the pydantic schema.

Frontend
- Photo TS type gains an optional tags field plus a PhotoTagSummary
  alias.
- filterStore: new groupBy: 'date' | 'tag' field, default 'date',
  with setGroupBy + URL sync via ?group=tag. clearAll resets it.
- usePhotosQuery threads groupBy through filtersToParams (it's
  client-side only but kept in the params for cache key
  consistency).
- LeftSidebar Tags entry is now a leaf node (no children), shows the
  total tag photo count as the badge, and is highlighted when
  groupBy === 'tag'. Click → setGroupBy('tag') without touching
  other filters. Selecting "All Photos" resets groupBy back to
  'date' via clearAll.
- Timeline.buildItems gets a third "tag" branch:
  - Iterates photos × tags into per-tag buckets
  - Photos with no tags go into an "Untagged" bucket
  - Tag groups sorted alphabetically; Untagged pinned to the end
  - Headers + rows pushed in the same shape the date branch uses,
    so the existing sticky-header overlay works for free
- Selection state is by photo id, so a photo appearing in multiple
  groups stays consistently selected/highlighted across instances.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 11:51:21 +02:00
914eb58ac5 feat: real recursive folder tree in the sidebar
The Folders section in the LeftSidebar previously rendered the flat
list of source roots — actual subdirectories were invisible. Now it
shows the full nested tree, click any node to filter, drop targets
work at every depth.

Backend
- New GET /folders/tree returning a list of root nodes (one per
  active SourceRoot). Each node is { id, name, path, photo_count,
  children: [...] } with children sorted alphabetically at every
  level. Walks Folder rows whose source_root_id matches and whose
  path is at or beneath the source root, then attaches them by
  parent path so partial scans don't break the tree.
- The source root's display label is overlaid on the root folder
  node so the top-level entry reads as "Library" instead of
  "/photos".
- list_photos folder_id filter now does descendant matching: when
  a Folder id is given, it includes the folder itself and every
  Folder whose path is a sep-prefixed descendant. Matches the
  Lightroom mental model: clicking "Library" or any parent folder
  shows everything beneath it. The existing source-root-id branch
  is unchanged.

Frontend
- New types/api.ts FolderTreeNode interface and sourceFolders.tree()
  helper.
- New hooks/useFolderTreeQuery.ts with a 30s staleTime and a
  findFolderInTree() walker for id-based name lookups.
- LeftSidebar drops the flat foldersData list and uses the tree
  query. folderNodeToTreeItem recursively maps backend nodes into
  the existing TreeItem shape; renderTreeItem already knew how to
  recurse into children, so the tree just works at any depth.
  Drop targets, drag-to-move, drag-to-copy, double-click rename,
  and active-state highlighting all carry over to nested folders.
- The renameMutation now also invalidates ['folders', 'tree'] so a
  source-root rename refreshes the tree label immediately.
- ActiveFilterChips switches to the tree query and uses the new
  findFolderInTree walker so the chip label resolves correctly for
  sub-folder filters too — not just top-level source roots.
- The "Scan all folders" button visibility now keys off the tree
  length instead of the flat folders length.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 11:36:01 +02:00
bb7c2b12d6 feat: heap convert can create a subfolder under the target
The previous /heaps/{id}/convert dropped photos directly into a chosen
source root, which is rarely what you want — Lightroom-style behaviour
is "make a folder named after the collection inside the library".
Now the dialog lets you do that.

Backend
- HeapConvertBody gains an optional subfolder_name field. Path
  separators and dot-segments are rejected. When set, the handler
  joins it onto the resolved parent_dir, mkdir's it if missing, and
  uses the resulting path as the move/copy destination. Otherwise
  the parent_dir itself is used (unchanged behaviour).
- The Folder DB row for the destination is created via the existing
  scanner get_or_create_folder helper so dedupe + path normalization
  stay consistent across the codebase.
- The target source root id is propagated through both the source-
  root and folder branches so the new Folder row is correctly
  parented when subfolder_name is set on a folder target too.

Frontend
- HeapConvertDialog grows a "Subfolder name" input that prefills
  with the heap name when the dialog opens. Trimmed empty value
  drops directly into the parent. A live hint below the input
  shows exactly which path will be created (or that the parent
  will be used).
- api.ts heaps.convert() signature accepts an optional
  subfolder_name field; the dialog sends it via mutationFn.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 11:31:14 +02:00