120-mule-images: drop AI pipeline, swap to plain postgres, log refresh

Reflects state after the 2026-05-14 session: the OpenCLIP vision
classifier and worker-vision are gone, the db image is now postgres:16,
and the one-shot full_refresh.py reaped 7982 orphan thumbnail dirs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
claudio
2026-05-14 01:02:48 +02:00
parent 23c7f6a4f1
commit 838052543b

View File

@@ -18,11 +18,12 @@ Hosts `mule-image` / "mulita" — the photos app at `photos.hubris.network`. Aut
| ----------------- | ------ | ----- |
| frontend | 3000 | Reverse-proxied by Caddy |
| backend | 8001 | FastAPI |
| worker-vision | — | ML scan worker |
| worker-light | — | Lightweight worker (+ celery beat scheduler since 2026-05-11) |
| db | (pg) | pgvector |
| worker-light | — | Sole celery worker — runs default/high/low queues + beat scheduler |
| db | (pg) | postgres:16 (was pgvector/pg16 until 2026-05-14) |
| redis | (rd) | queue |
**No vision / ML services.** The OpenCLIP ONNX classifier and its dedicated `worker-vision` were ripped out on 2026-05-14 — see Changelog.
`worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below.
`.env` is **untracked**`git checkout .env` will wipe it. Holds:
@@ -90,6 +91,50 @@ For pushes from inside the LXC, gitea creds at `/etc/mule-deploy/git-credentials
## Changelog
### 2026-05-14 — Drop AI/vision pipeline, plain Postgres, DB↔FS refresh
**AI removal (commits `a27267f` + `423a73a`).** The OpenCLIP-on-ONNX
binary classifier (photography vs other) and all its scaffolding gone:
- Backend: `app/services/vision/`, `app/tasks/vision.py`,
`app/services/feature_flags.py`, `app/routers/features.py` deleted.
Admin `/admin/feature-flags`, `/admin/ai/{backfill,rescan}` endpoints
removed. `Photo.needs_review` column + `ix_photos_needs_review`
index dropped (alembic `0019_drop_ai_remnants`).
- Frontend: AI Settings tab, `useFeaturesQuery`, "Needs Review" sidebar
entry + filter, `needs_review` URL param plumbing, FeatureFlag types
all gone.
- Infra: `worker-vision` compose service + `models_data` volume
deleted. `worker-light` no longer runs `python -m
app.services.vision.bootstrap_models` before celery. Backend
Dockerfile drops the dedicated torch RUN layer. `requirements.txt`
drops `torch`, `torchvision`, `open-clip-torch`, `onnxruntime`.
**Postgres image swap.** `pgvector/pgvector:pg16``postgres:16`. The
0019 migration drops the (now-unused) `vector` extension first; the
SQL was pre-applied via `psql` against the still-pgvector container
and `alembic_version` stamped to `0019` so the new backend's
`bootstrap.py` upgrade-head was a no-op. After the swap surfaced a
collation-version mismatch (Bookworm glibc 2.36 → Trixie glibc 2.41),
the fix was `REINDEX DATABASE mulita` + `ALTER DATABASE … REFRESH
COLLATION VERSION` on `mulita`, `postgres`, `template1`. The data
volume was preserved across the image change.
**One-shot DB ↔ filesystem refresh.** New script
`backend/scripts/full_refresh.py` (run as
`docker exec -w /app mulita-backend python -m scripts.full_refresh
[--dry-run]`). Phases: `cleanup_data_integrity` → inline scan of every
active SourceRoot → `prune_missing_photos(dry_run=False)` → new
`prune_orphan_thumbnails` helper that removes
`/data/thumbs/{user}/{photo}/` dirs for photo_ids that left the DB.
First run: 0 missing photos, 1 stale folder row, **7982 orphan
thumbnail dirs** reaped.
**Manual cleanup post-deploy.** `docker compose up -d --build
--force-recreate` doesn't reap services removed from the YAML, so
`mulita-worker-vision` had to be `docker rm -f`'d by hand. The
`mule-image_models_data` volume was likewise removed manually.
### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2)
Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files.