refactor: drop AI/vision pipeline + plain Postgres + full-refresh script
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
- backend: app/services/vision/, app/tasks/vision.py,
app/services/feature_flags.py, app/routers/features.py — all
deleted; admin AI/feature-flag endpoints and the worker-vision
bootstrap call gone. Photo.needs_review and its index dropped.
- frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
types, "Needs Review" sidebar entry + filter, needs_review filter
URL param all gone.
- infra: worker-vision compose service + models_data volume deleted;
worker-light command no longer runs bootstrap_models; the db
image switches from pgvector/pgvector:pg16 to postgres:16; backend
Dockerfile drops the dedicated torch RUN layer; requirements.txt
drops torch/torchvision/open-clip-torch/onnxruntime.
Alembic 0019_drop_ai_remnants:
- drops photos.needs_review + ix_photos_needs_review
- DROP EXTENSION IF EXISTS vector (must run before the image swap;
the new postgres:16 doesn't ship pgvector)
New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -65,11 +65,6 @@ class Photo(Base):
|
||||
# rows, and POST /folders/{id}/hide recomputes it on toggle.
|
||||
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
|
||||
|
||||
# "Needs review" — set by the content classifier when a photo is
|
||||
# classified as 'other' (screenshot, document, meme, scan, etc.) so
|
||||
# the user can page through non-photographs in the UI and triage them.
|
||||
needs_review = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
|
||||
|
||||
# "Capture date is probably wrong" — denormalized from the folder/filename
|
||||
# date-guesser. Set at scan time and recomputed on every taken_at edit so
|
||||
# the filter bar can query it directly. See services/date_guess.py for
|
||||
@@ -138,5 +133,4 @@ class Photo(Base):
|
||||
Index('ix_photos_media_type', 'media_type'),
|
||||
Index('ix_photos_processing_status', 'processing_status'),
|
||||
Index('ix_photos_lat_lon', 'latitude', 'longitude'),
|
||||
Index('ix_photos_needs_review', 'needs_review'),
|
||||
)
|
||||
@@ -22,7 +22,7 @@ photo_tags = Table(
|
||||
# ML metadata — null for user-applied tags
|
||||
Column('confidence', Float, nullable=True),
|
||||
Column('bbox', JSONB, nullable=True), # [x1, y1, x2, y2] normalized 0-1
|
||||
Column('source', String, nullable=True), # e.g. "vision:clip_classifier"
|
||||
Column('source', String, nullable=True), # null for user-applied tags
|
||||
Index('ix_photo_tags_photo_id', 'photo_id'),
|
||||
Index('ix_photo_tags_tag_id', 'tag_id'),
|
||||
)
|
||||
@@ -44,9 +44,8 @@ class Tag(Base):
|
||||
kind = Column(String, nullable=False, default='user', index=True)
|
||||
# kind values: 'user' | 'content_type'
|
||||
|
||||
# Which model produced this tag (null for user-created)
|
||||
# Which producer wrote this tag (null for user-created)
|
||||
source = Column(String, nullable=True)
|
||||
# e.g. "vision:clip_classifier", null
|
||||
|
||||
# Relationships
|
||||
photos = relationship("Photo", secondary=photo_tags, backref="tags")
|
||||
|
||||
Reference in New Issue
Block a user