feat: perceptual-hash duplicate detection + grouped picker view

The Duplicates section was useless: SHA-256-only detection only caught
byte-identical files, not the actual duplicates a real library
accumulates (re-encoded JPEGs, screenshots, resized exports), and the
view was a flat date-sorted list with no grouping or actions. This
replaces the whole flow.

Detection
- New phash + duplicate_group_id columns on Photo, added via an
  idempotent ALTER TABLE pass in init_db (the project has no Alembic).
- Thumbs worker computes a 64-bit pHash from the original-resolution
  decoded frame just before the destructive thumbnail loop. Falls back
  silently — phash is nice-to-have, not a blocker for thumbnails.
- backfill_phashes Celery task fills in phashes for photos that
  predated the column, reading the existing thumb_large rather than
  re-decoding the original.
- regroup_duplicates service runs union-find over Hamming distance
  (threshold 6), persists duplicate_group_id, and maintains is_duplicate
  as derived state so existing badges/counts keep working. Chained
  after scan_all_source_roots with a 60s countdown.

API
- GET /library/duplicates/groups returns all groups with members,
  bucketed in Python from one query. Each group has a reason ("exact"
  iff every member shares a SHA-256, "similar" otherwise).
- POST /library/maintenance/{regroup-duplicates,backfill-phashes}.

Frontend
- New DuplicatesView (sectioned grid, one section per cluster) replaces
  the timeline when the user is in the duplicates section. Each section
  shows a "Keep best, discard N" button that picks the highest-pixel
  copy and reuses the existing undoable bulk-discard so Cmd+Z works.
- Manual best override: hover any non-best thumbnail and click "Keep
  this" (Crown icon, top-right) to override the auto-pick. The header
  annotates "(manual)" so it's obvious which copy will be kept.
- Keyboard nav within the duplicates view walks the flat member list,
  with ↑/↓ jumping by the measured column count and scrollIntoView on
  every move. Timeline's keyboard handler now early-returns in the
  duplicates section so the two don't fight.
- BEST pill / Keep-this button live at top-right with a ring outline so
  they don't collide visually with the cyan selection ring around a
  selected cell. Dimensions chip moved to bottom-left to free both
  right corners for the keep affordances.
- New "Duplicates" section in SettingsDialog: shows group/member counts
  and exposes both backfill + re-detect actions, sharing a query cache
  with DuplicatesView via DUPLICATE_GROUPS_QUERY_KEY.
- PhotoInfoPanel "Basic Info" section now shows the photo's full file
  path in monospace below the size/dimensions/date grid.
- New imagehash==4.3.1 dep in requirements.txt.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-09 17:19:08 +02:00
parent e51b93d59e
commit 733c16bf82
14 changed files with 1098 additions and 14 deletions

View File

@@ -58,17 +58,57 @@ async def init_db():
async with engine.begin() as conn:
# Import all models to register them with Base
from app.models import Photo, Folder, SourceRoot, Tag, PhotoTag, Heap, HeapPhoto, Embedding
# Create all tables
# Create all tables. Note: create_all only creates *missing* tables —
# it does NOT add new columns to existing tables when the model gains
# them. Anything new on an existing table needs an explicit ALTER
# below.
await conn.run_sync(Base.metadata.create_all)
# Enable WAL mode for SQLite (better concurrency)
if "sqlite" in settings.database_url:
await conn.execute(text("PRAGMA journal_mode=WAL"))
await conn.execute(text("PRAGMA synchronous=NORMAL"))
await conn.execute(text("PRAGMA cache_size=10000"))
await conn.execute(text("PRAGMA temp_store=MEMORY"))
# ── Idempotent column adds ────────────────────────────────────────
# The project does not use Alembic; we lean on create_all + a small
# set of inline ALTER TABLE statements for the columns we've added
# post-launch. SQLite supports ADD COLUMN but not "IF NOT EXISTS"
# for columns, so introspect via PRAGMA first. Each entry is
# (column_name, ALTER statement). Add new columns at the bottom.
if "sqlite" in settings.database_url:
existing_cols = {
row[1]
for row in (
await conn.execute(text("PRAGMA table_info(photos)"))
).fetchall()
}
pending_alters: list[tuple[str, str]] = [
("phash", "ALTER TABLE photos ADD COLUMN phash VARCHAR(16)"),
(
"duplicate_group_id",
"ALTER TABLE photos ADD COLUMN duplicate_group_id VARCHAR",
),
]
for col_name, alter_sql in pending_alters:
if col_name not in existing_cols:
logger.info(f"Adding photos.{col_name} column")
await conn.execute(text(alter_sql))
# Indexes for the new duplicate-detection columns. CREATE INDEX
# IF NOT EXISTS is supported on SQLite so this is safe to run
# every startup.
await conn.execute(
text("CREATE INDEX IF NOT EXISTS ix_photos_phash ON photos(phash)")
)
await conn.execute(
text(
"CREATE INDEX IF NOT EXISTS ix_photos_duplicate_group_id "
"ON photos(duplicate_group_id)"
)
)
logger.info("Database initialized successfully")
async def create_fts_table():