refactor: drop AI/vision pipeline + plain Postgres + full-refresh script
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
- backend: app/services/vision/, app/tasks/vision.py,
app/services/feature_flags.py, app/routers/features.py — all
deleted; admin AI/feature-flag endpoints and the worker-vision
bootstrap call gone. Photo.needs_review and its index dropped.
- frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
types, "Needs Review" sidebar entry + filter, needs_review filter
URL param all gone.
- infra: worker-vision compose service + models_data volume deleted;
worker-light command no longer runs bootstrap_models; the db
image switches from pgvector/pgvector:pg16 to postgres:16; backend
Dockerfile drops the dedicated torch RUN layer; requirements.txt
drops torch/torchvision/open-clip-torch/onnxruntime.
Alembic 0019_drop_ai_remnants:
- drops photos.needs_review + ix_photos_needs_review
- DROP EXTENSION IF EXISTS vector (must run before the image swap;
the new postgres:16 doesn't ship pgvector)
New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -48,11 +48,10 @@ THUMB_SIZES = {
|
||||
}
|
||||
|
||||
# Sizes the worker writes to /data/thumbs. Empty set since Phase 4 —
|
||||
# the API serves all sizes via Nextcloud's /core/preview proxy, and
|
||||
# the vision worker also fetches NC previews on demand instead of
|
||||
# reading a local cache. generate_thumbnails still runs the decode-
|
||||
# and-pHash side-effect (perceptual dedup is mule-only and needs the
|
||||
# original-resolution pixels) but no longer touches the disk.
|
||||
# the API serves all sizes via Nextcloud's /core/preview proxy.
|
||||
# generate_thumbnails still runs the decode-and-pHash side-effect
|
||||
# (perceptual dedup is mule-only and needs the original-resolution
|
||||
# pixels) but no longer touches the disk.
|
||||
WORKER_THUMB_SIZES: set[str] = set()
|
||||
|
||||
def get_thumb_path(photo_id: str, size: str, user_id: str = None) -> str:
|
||||
@@ -396,14 +395,6 @@ async def _generate_thumbnails_async(photo_id: str, task):
|
||||
|
||||
logger.info(f"Thumbnails generated for photo {photo_id}")
|
||||
|
||||
# Dispatch vision pipeline only after thumbnails succeeded —
|
||||
# vision tasks need the generated thumbnails to run inference.
|
||||
try:
|
||||
from app.tasks.vision import vision_fanout
|
||||
vision_fanout.delay(photo_id)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not dispatch vision_fanout for {photo_id}: {e}")
|
||||
|
||||
return {'status': 'success', 'photo_id': photo_id}
|
||||
|
||||
except Exception as e:
|
||||
|
||||
Reference in New Issue
Block a user