refactor: drop AI/vision pipeline + plain Postgres + full-refresh script
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
- backend: app/services/vision/, app/tasks/vision.py,
app/services/feature_flags.py, app/routers/features.py — all
deleted; admin AI/feature-flag endpoints and the worker-vision
bootstrap call gone. Photo.needs_review and its index dropped.
- frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
types, "Needs Review" sidebar entry + filter, needs_review filter
URL param all gone.
- infra: worker-vision compose service + models_data volume deleted;
worker-light command no longer runs bootstrap_models; the db
image switches from pgvector/pgvector:pg16 to postgres:16; backend
Dockerfile drops the dedicated torch RUN layer; requirements.txt
drops torch/torchvision/open-clip-torch/onnxruntime.
Alembic 0019_drop_ai_remnants:
- drops photos.needs_review + ix_photos_needs_review
- DROP EXTENSION IF EXISTS vector (must run before the image swap;
the new postgres:16 doesn't ship pgvector)
New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
47
backend/alembic/versions/0019_drop_ai_remnants.py
Normal file
47
backend/alembic/versions/0019_drop_ai_remnants.py
Normal file
@@ -0,0 +1,47 @@
|
||||
"""Drop AI remnants: photos.needs_review and the pgvector extension
|
||||
|
||||
Revision ID: 0019_drop_ai_remnants
|
||||
Revises: 0018_photos_nextcloud_fileid
|
||||
Create Date: 2026-05-14
|
||||
|
||||
The vision pipeline has been removed entirely (no more classifier, no
|
||||
worker-vision service, no torch/onnxruntime/open-clip-torch deps). The
|
||||
`needs_review` boolean and its partial index were populated only by
|
||||
that classifier and have no remaining writers or readers.
|
||||
|
||||
The pgvector extension was originally added by 0003_pgvector_embeddings
|
||||
for the embeddings table that 0012_strip_ai_pipeline dropped; the
|
||||
extension itself is now unused, and the next deploy switches the
|
||||
Postgres image from pgvector/pgvector:pg16 to plain postgres:16. The
|
||||
extension must be dropped *before* that image swap or the new
|
||||
container will fail to load existing CREATE EXTENSION declarations.
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "0019_drop_ai_remnants"
|
||||
down_revision: Union[str, None] = "0018_photos_nextcloud_fileid"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute("DROP INDEX IF EXISTS ix_photos_needs_review")
|
||||
op.execute("ALTER TABLE photos DROP COLUMN IF EXISTS needs_review")
|
||||
op.execute("DROP EXTENSION IF EXISTS vector")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Re-create the column as a no-op (data is gone). The pgvector
|
||||
# extension is intentionally NOT re-added — matches the precedent
|
||||
# set by 0012_strip_ai_pipeline for its dropped tables.
|
||||
op.execute(
|
||||
"ALTER TABLE photos ADD COLUMN IF NOT EXISTS needs_review "
|
||||
"BOOLEAN NOT NULL DEFAULT FALSE"
|
||||
)
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS ix_photos_needs_review "
|
||||
"ON photos(needs_review) WHERE needs_review"
|
||||
)
|
||||
Reference in New Issue
Block a user