dtoro
733c16bf82
The Duplicates section was useless: SHA-256-only detection only caught
byte-identical files, not the actual duplicates a real library
accumulates (re-encoded JPEGs, screenshots, resized exports), and the
view was a flat date-sorted list with no grouping or actions. This
replaces the whole flow.
Detection
- New phash + duplicate_group_id columns on Photo, added via an
idempotent ALTER TABLE pass in init_db (the project has no Alembic).
- Thumbs worker computes a 64-bit pHash from the original-resolution
decoded frame just before the destructive thumbnail loop. Falls back
silently — phash is nice-to-have, not a blocker for thumbnails.
- backfill_phashes Celery task fills in phashes for photos that
predated the column, reading the existing thumb_large rather than
re-decoding the original.
- regroup_duplicates service runs union-find over Hamming distance
(threshold 6), persists duplicate_group_id, and maintains is_duplicate
as derived state so existing badges/counts keep working. Chained
after scan_all_source_roots with a 60s countdown.
API
- GET /library/duplicates/groups returns all groups with members,
bucketed in Python from one query. Each group has a reason ("exact"
iff every member shares a SHA-256, "similar" otherwise).
- POST /library/maintenance/{regroup-duplicates,backfill-phashes}.
Frontend
- New DuplicatesView (sectioned grid, one section per cluster) replaces
the timeline when the user is in the duplicates section. Each section
shows a "Keep best, discard N" button that picks the highest-pixel
copy and reuses the existing undoable bulk-discard so Cmd+Z works.
- Manual best override: hover any non-best thumbnail and click "Keep
this" (Crown icon, top-right) to override the auto-pick. The header
annotates "(manual)" so it's obvious which copy will be kept.
- Keyboard nav within the duplicates view walks the flat member list,
with ↑/↓ jumping by the measured column count and scrollIntoView on
every move. Timeline's keyboard handler now early-returns in the
duplicates section so the two don't fight.
- BEST pill / Keep-this button live at top-right with a ring outline so
they don't collide visually with the cyan selection ring around a
selected cell. Dimensions chip moved to bottom-left to free both
right corners for the keep affordances.
- New "Duplicates" section in SettingsDialog: shows group/member counts
and exposes both backfill + re-detect actions, sharing a query cache
with DuplicatesView via DUPLICATE_GROUPS_QUERY_KEY.
- PhotoInfoPanel "Basic Info" section now shows the photo's full file
path in monospace below the size/dimensions/date grid.
- New imagehash==4.3.1 dep in requirements.txt.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
94 lines
4.0 KiB
Python
94 lines
4.0 KiB
Python
"""
|
|
Photo model definition
|
|
"""
|
|
from sqlalchemy import Column, String, Integer, Boolean, DateTime, ForeignKey, Text, Index
|
|
from sqlalchemy.sql import func
|
|
from datetime import datetime
|
|
import uuid
|
|
|
|
from app.database import Base
|
|
|
|
class Photo(Base):
|
|
__tablename__ = 'photos'
|
|
|
|
# Primary key
|
|
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
|
|
|
# File information
|
|
filepath = Column(String, unique=True, nullable=False)
|
|
filename = Column(String, nullable=False)
|
|
folder_id = Column(String, ForeignKey('folders.id'))
|
|
file_hash = Column(String, index=True) # SHA-256 hash for duplicate detection
|
|
|
|
# Media information
|
|
media_type = Column(String, nullable=False) # 'photo' | 'video' | 'raw' | 'heic'
|
|
original_format = Column(String) # 'CR3', 'NEF', 'HEIC', 'MP4', 'JPEG', etc.
|
|
width = Column(Integer)
|
|
height = Column(Integer)
|
|
file_size = Column(Integer)
|
|
|
|
# Timestamps
|
|
taken_at = Column(DateTime) # from EXIF DateTimeOriginal, fallback to file mtime
|
|
taken_at_source = Column(String) # 'exif' | 'filesystem' | 'manual'
|
|
added_at = Column(DateTime, server_default=func.now())
|
|
updated_at = Column(DateTime, onupdate=func.now())
|
|
|
|
# Discard status. The DB column names stay is_trashed/trashed_at to avoid
|
|
# a migration; only the Python attribute name reflects the rename.
|
|
is_discarded = Column('is_trashed', Boolean, default=False)
|
|
discarded_at = Column('trashed_at', DateTime)
|
|
|
|
# Thumbnail paths
|
|
thumb_small = Column(String) # path to 240px thumb
|
|
thumb_medium = Column(String) # path to 640px thumb
|
|
thumb_large = Column(String) # path to 1280px thumb
|
|
|
|
# Processing status
|
|
processing_status = Column(String, default='pending') # 'pending' | 'processing' | 'completed' | 'failed'
|
|
processing_error = Column(Text)
|
|
|
|
# Metadata
|
|
exif_json = Column(Text) # full EXIF/XMP blob as JSON
|
|
|
|
# User-editable fields
|
|
user_title = Column(String)
|
|
user_notes = Column(Text)
|
|
rating = Column(Integer, default=0) # 0-5 stars
|
|
color_label = Column(String) # 'red'|'orange'|'yellow'|'green'|'blue'|'purple'|NULL
|
|
# Note: is_rejected was merged into is_discarded (a single soft "discarded"
|
|
# concept). is_picked was unified with active-heap membership — picking a
|
|
# photo just means adding it to the active heap. Both DB columns may still
|
|
# exist on legacy installs but are no longer read or written.
|
|
|
|
# Duplicate detection.
|
|
#
|
|
# - file_hash (above): SHA-256 of the raw bytes. Catches byte-identical
|
|
# copies but not visually-identical re-encodes / resizes / screenshots.
|
|
# - phash: 16-char hex of a 64-bit perceptual hash, computed by the
|
|
# thumbs worker from the decoded original frame. Robust to resize and
|
|
# re-compression — this is what actually identifies "the same photo
|
|
# saved twice with different JPEG quality".
|
|
# - duplicate_group_id: shared by every photo in the same duplicate
|
|
# cluster. Maintained by app.services.duplicates.regroup_duplicates,
|
|
# not on individual writes — recomputed in batches after scans / on
|
|
# demand from the Settings panel.
|
|
# - is_duplicate: derived boolean (group_id IS NOT NULL). Kept as a real
|
|
# column so the existing PhotoThumbnail badge and /library/stats
|
|
# duplicates count don't have to change.
|
|
is_duplicate = Column(Boolean, default=False)
|
|
phash = Column(String(16), index=True)
|
|
duplicate_group_id = Column(String, index=True)
|
|
|
|
# Live photo support
|
|
live_photo_video_id = Column(String, ForeignKey('photos.id'))
|
|
|
|
# Indexes for performance
|
|
__table_args__ = (
|
|
Index('ix_photos_taken_at', 'taken_at'),
|
|
Index('ix_photos_folder_id', 'folder_id'),
|
|
Index('ix_photos_is_trashed', 'is_trashed'),
|
|
Index('ix_photos_rating', 'rating'),
|
|
Index('ix_photos_color_label', 'color_label'),
|
|
Index('ix_photos_media_type', 'media_type'),
|
|
Index('ix_photos_processing_status', 'processing_status'),
|
|
) |