refactor: drop AI/vision pipeline + plain Postgres + full-refresh script

Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
  - backend: app/services/vision/, app/tasks/vision.py,
    app/services/feature_flags.py, app/routers/features.py — all
    deleted; admin AI/feature-flag endpoints and the worker-vision
    bootstrap call gone. Photo.needs_review and its index dropped.
  - frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
    types, "Needs Review" sidebar entry + filter, needs_review filter
    URL param all gone.
  - infra: worker-vision compose service + models_data volume deleted;
    worker-light command no longer runs bootstrap_models; the db
    image switches from pgvector/pgvector:pg16 to postgres:16; backend
    Dockerfile drops the dedicated torch RUN layer; requirements.txt
    drops torch/torchvision/open-clip-torch/onnxruntime.

Alembic 0019_drop_ai_remnants:
  - drops photos.needs_review + ix_photos_needs_review
  - DROP EXTENSION IF EXISTS vector (must run before the image swap;
    the new postgres:16 doesn't ship pgvector)

New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
claudio
2026-05-14 00:20:38 +02:00
parent 6915c30911
commit a27267f7ad
39 changed files with 265 additions and 1573 deletions

View File

@@ -65,11 +65,6 @@ class Photo(Base):
# rows, and POST /folders/{id}/hide recomputes it on toggle.
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# "Needs review" — set by the content classifier when a photo is
# classified as 'other' (screenshot, document, meme, scan, etc.) so
# the user can page through non-photographs in the UI and triage them.
needs_review = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# "Capture date is probably wrong" — denormalized from the folder/filename
# date-guesser. Set at scan time and recomputed on every taken_at edit so
# the filter bar can query it directly. See services/date_guess.py for
@@ -138,5 +133,4 @@ class Photo(Base):
Index('ix_photos_media_type', 'media_type'),
Index('ix_photos_processing_status', 'processing_status'),
Index('ix_photos_lat_lon', 'latitude', 'longitude'),
Index('ix_photos_needs_review', 'needs_review'),
)

View File

@@ -22,7 +22,7 @@ photo_tags = Table(
# ML metadata — null for user-applied tags
Column('confidence', Float, nullable=True),
Column('bbox', JSONB, nullable=True), # [x1, y1, x2, y2] normalized 0-1
Column('source', String, nullable=True), # e.g. "vision:clip_classifier"
Column('source', String, nullable=True), # null for user-applied tags
Index('ix_photo_tags_photo_id', 'photo_id'),
Index('ix_photo_tags_tag_id', 'tag_id'),
)
@@ -44,9 +44,8 @@ class Tag(Base):
kind = Column(String, nullable=False, default='user', index=True)
# kind values: 'user' | 'content_type'
# Which model produced this tag (null for user-created)
# Which producer wrote this tag (null for user-created)
source = Column(String, nullable=True)
# e.g. "vision:clip_classifier", null
# Relationships
photos = relationship("Photo", secondary=photo_tags, backref="tags")