refactor: drop AI/vision pipeline + plain Postgres + full-refresh script
Removes the OpenCLIP-on-ONNX classifier and everything that fed or
consumed it:
- backend: app/services/vision/, app/tasks/vision.py,
app/services/feature_flags.py, app/routers/features.py — all
deleted; admin AI/feature-flag endpoints and the worker-vision
bootstrap call gone. Photo.needs_review and its index dropped.
- frontend: AI Settings tab, useFeaturesQuery hook, FeatureFlag
types, "Needs Review" sidebar entry + filter, needs_review filter
URL param all gone.
- infra: worker-vision compose service + models_data volume deleted;
worker-light command no longer runs bootstrap_models; the db
image switches from pgvector/pgvector:pg16 to postgres:16; backend
Dockerfile drops the dedicated torch RUN layer; requirements.txt
drops torch/torchvision/open-clip-torch/onnxruntime.
Alembic 0019_drop_ai_remnants:
- drops photos.needs_review + ix_photos_needs_review
- DROP EXTENSION IF EXISTS vector (must run before the image swap;
the new postgres:16 doesn't ship pgvector)
New scripts/full_refresh.py: one-shot DB ↔ filesystem reconciliation.
Runs cleanup_data_integrity, scans every active SourceRoot inline
(no celery dependency so the worker can be stopped), hard-prunes
photo + folder rows for files that are gone, removes orphan
/data/thumbs/{user}/{photo}/ directories. New helper
prune_orphan_thumbnails in cleanup.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -30,19 +30,6 @@ class PerformanceSettings(BaseModel):
|
||||
db_pool_max_overflow: int = 10
|
||||
db_pool_recycle: int = 3600
|
||||
|
||||
class ClassifierSettings(BaseModel):
|
||||
"""Binary content classifier (photography vs other)."""
|
||||
min_confidence: float = 0.3
|
||||
|
||||
class VisionSettings(BaseModel):
|
||||
"""Vision pipeline — one binary classifier (photography vs other)."""
|
||||
enabled: bool = True
|
||||
backend: str = "onnx"
|
||||
models_dir: str = "/data/models"
|
||||
execution_providers: list[str] = ["CPUExecutionProvider"]
|
||||
classifier: ClassifierSettings = ClassifierSettings()
|
||||
worker_concurrency: int = 2
|
||||
|
||||
class MulitaConfig(BaseModel):
|
||||
"""Main configuration from YAML file. Source roots and the discard
|
||||
workflow are owned by the database now — only operational settings
|
||||
@@ -50,13 +37,12 @@ class MulitaConfig(BaseModel):
|
||||
thumbnails: ThumbnailSettings = ThumbnailSettings()
|
||||
scanner: ScannerSettings = ScannerSettings()
|
||||
performance: PerformanceSettings = PerformanceSettings()
|
||||
vision: VisionSettings = VisionSettings()
|
||||
|
||||
class Settings(BaseSettings):
|
||||
"""Application settings"""
|
||||
# Database — Postgres + pgvector by default. The SQLite escape hatch
|
||||
# remains supported via the docker-compose.sqlite.yml override and by
|
||||
# setting DATABASE_URL=sqlite+aiosqlite:///... in .env for local dev.
|
||||
# Database — plain Postgres. The SQLite escape hatch remains
|
||||
# supported via the docker-compose.sqlite.yml override and by setting
|
||||
# DATABASE_URL=sqlite+aiosqlite:///... in .env for local dev.
|
||||
database_url: str = Field(
|
||||
default="postgresql+asyncpg://mulita:mulita@db:5432/mulita",
|
||||
env="DATABASE_URL"
|
||||
@@ -195,23 +181,6 @@ class Settings(BaseSettings):
|
||||
def performance(self) -> PerformanceSettings:
|
||||
return self.config.performance
|
||||
|
||||
# ONNX Runtime execution providers, overridable via env var.
|
||||
# Comma-separated: "CUDAExecutionProvider,CPUExecutionProvider"
|
||||
# or "auto" for GPU auto-detection.
|
||||
vision_execution_providers: str = Field(
|
||||
default="CPUExecutionProvider",
|
||||
env="VISION_EXECUTION_PROVIDERS",
|
||||
)
|
||||
|
||||
@property
|
||||
def vision(self) -> VisionSettings:
|
||||
v = self.config.vision
|
||||
# Override execution_providers from env if set.
|
||||
providers = [p.strip() for p in self.vision_execution_providers.split(",") if p.strip()]
|
||||
if providers:
|
||||
v.execution_providers = providers
|
||||
return v
|
||||
|
||||
class Config:
|
||||
env_file = ".env"
|
||||
case_sensitive = False
|
||||
|
||||
Reference in New Issue
Block a user