Files
mule-image/backend/app/models/photos.py
claudio a27267f7ad refactor: drop AI/vision pipeline + plain Postgres + full-refresh script
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
  - backend: app/services/vision/, app/tasks/vision.py,
    app/services/feature_flags.py, app/routers/features.py — all
    deleted; admin AI/feature-flag endpoints and the worker-vision
    bootstrap call gone. Photo.needs_review and its index dropped.
  - frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
    types, "Needs Review" sidebar entry + filter, needs_review filter
    URL param all gone.
  - infra: worker-vision compose service + models_data volume deleted;
    worker-light command no longer runs bootstrap_models; the db
    image switches from pgvector/pgvector:pg16 to postgres:16; backend
    Dockerfile drops the dedicated torch RUN layer; requirements.txt
    drops torch/torchvision/open-clip-torch/onnxruntime.

Alembic 0019_drop_ai_remnants:
  - drops photos.needs_review + ix_photos_needs_review
  - DROP EXTENSION IF EXISTS vector (must run before the image swap;
    the new postgres:16 doesn't ship pgvector)

New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 00:20:38 +02:00

136 lines
6.4 KiB
Python

"""
Photo model definition
"""
from sqlalchemy import Column, String, Integer, Float, Boolean, DateTime, ForeignKey, Text, Index
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from datetime import datetime
import uuid
from app.database import Base
class Photo(Base):
__tablename__ = 'photos'
# Primary key
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
# Owner
user_id = Column(String, ForeignKey('users.id'), nullable=True, index=True)
# Eager-loadable owner relationship. Used by the thumbnail handler so
# one photo lookup also yields the NC creds we need to call the
# preview endpoint, instead of issuing a second SELECT users WHERE
# id=…. No FK change — user_id above already exists.
user = relationship("User", lazy="select")
# File information
filepath = Column(String, unique=True, nullable=False)
filename = Column(String, nullable=False)
folder_id = Column(String, ForeignKey('folders.id'))
file_hash = Column(String, index=True) # SHA-256 hash for duplicate detection
# Nextcloud fileid for the same file. Set by the scanner when the file
# lives under a Nextcloud-rooted SourceRoot. Used by the thumbnail
# endpoint to proxy /index.php/core/preview instead of generating
# and serving thumbs locally — Nextcloud already maintains previews
# for the same source file, and duplicating that work was the bulk
# of `/data/thumbs/*`. NULL on legacy / non-NC paths; the handler
# falls back to on-disk thumbs when this is unset.
nextcloud_fileid = Column(Integer, nullable=True, index=True)
# Media information
media_type = Column(String, nullable=False) # 'photo' | 'video' | 'raw' | 'heic'
original_format = Column(String) # 'CR3', 'NEF', 'HEIC', 'MP4', 'JPEG', etc.
width = Column(Integer)
height = Column(Integer)
file_size = Column(Integer)
# Timestamps
taken_at = Column(DateTime) # from EXIF DateTimeOriginal, fallback to file mtime
taken_at_source = Column(String) # 'exif' | 'filesystem' | 'manual'
added_at = Column(DateTime, server_default=func.now())
updated_at = Column(DateTime, onupdate=func.now())
# Discard status. The DB column names stay is_trashed/trashed_at to avoid
# a migration; only the Python attribute name reflects the rename.
is_discarded = Column('is_trashed', Boolean, default=False)
discarded_at = Column('trashed_at', DateTime)
# "Hidden from views" — materialized from Folder.is_hidden walking
# the ancestry chain. True iff any ancestor folder (including the
# photo's direct folder) is hidden. Cross-cutting queries filter
# `AND NOT is_hidden`; per-folder browses ignore the flag so the
# user can still open a hidden folder and see its contents. The
# column is maintained by two places: the scanner sets it on new
# rows, and POST /folders/{id}/hide recomputes it on toggle.
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# "Capture date is probably wrong" — denormalized from the folder/filename
# date-guesser. Set at scan time and recomputed on every taken_at edit so
# the filter bar can query it directly. See services/date_guess.py for
# the heuristic; kept as a stored column because recomputing on every
# list query would mean running the regex stack across thousands of rows.
has_date_warning = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# Thumbnail paths
thumb_small = Column(String) # path to 240px thumb
thumb_medium = Column(String) # path to 640px thumb
thumb_large = Column(String) # path to 1280px thumb
# Processing status
processing_status = Column(String, default='pending') # 'pending' | 'processing' | 'completed' | 'failed'
processing_error = Column(Text)
# Metadata
exif_json = Column(Text) # full EXIF/XMP blob as JSON
# GPS coordinates extracted from EXIF, in signed decimal degrees
# (S latitude / W longitude are negative). Stored as first-class columns
# so the Map view and any future location filters can query/index them
# without parsing exif_json on every request.
latitude = Column(Float)
longitude = Column(Float)
# User-editable fields
user_title = Column(String)
user_notes = Column(Text)
rating = Column(Integer, default=0) # 0-5 stars
color_label = Column(String) # 'red'|'orange'|'yellow'|'green'|'blue'|'purple'|NULL
# Note: is_rejected was merged into is_discarded (a single soft "discarded"
# concept). is_picked was unified with active-heap membership — picking a
# photo just means adding it to the active heap. Both DB columns may still
# exist on legacy installs but are no longer read or written.
# Duplicate detection.
#
# - file_hash (above): SHA-256 of the raw bytes. Catches byte-identical
# copies but not visually-identical re-encodes / resizes / screenshots.
# - phash: 16-char hex of a 64-bit perceptual hash, computed by the
# thumbs worker from the decoded original frame. Robust to resize and
# re-compression — this is what actually identifies "the same photo
# saved twice with different JPEG quality".
# - duplicate_group_id: shared by every photo in the same duplicate
# cluster. Maintained by app.services.duplicates.regroup_duplicates,
# not on individual writes — recomputed in batches after scans / on
# demand from the Settings panel.
# - is_duplicate: derived boolean (group_id IS NOT NULL). Kept as a real
# column so the existing PhotoThumbnail badge and /library/stats
# duplicates count don't have to change.
is_duplicate = Column(Boolean, default=False)
phash = Column(String(16), index=True)
duplicate_group_id = Column(String, index=True)
# Live photo support
live_photo_video_id = Column(String, ForeignKey('photos.id'))
# Indexes for performance
__table_args__ = (
Index('ix_photos_taken_at', 'taken_at'),
Index('ix_photos_folder_id', 'folder_id'),
Index('ix_photos_is_trashed', 'is_trashed'),
Index('ix_photos_rating', 'rating'),
Index('ix_photos_color_label', 'color_label'),
Index('ix_photos_media_type', 'media_type'),
Index('ix_photos_processing_status', 'processing_status'),
Index('ix_photos_lat_lon', 'latitude', 'longitude'),
)