120-mule-images: drop AI pipeline, swap to plain postgres, log refresh

Reflects state after the 2026-05-14 session: the OpenCLIP vision
classifier and worker-vision are gone, the db image is now postgres:16,
and the one-shot full_refresh.py reaped 7982 orphan thumbnail dirs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
claudio
2026-05-14 01:02:48 +02:00
parent 23c7f6a4f1
commit 838052543b

View File

@@ -18,11 +18,12 @@ Hosts `mule-image` / "mulita" — the photos app at `photos.hubris.network`. Aut
| ----------------- | ------ | ----- | | ----------------- | ------ | ----- |
| frontend | 3000 | Reverse-proxied by Caddy | | frontend | 3000 | Reverse-proxied by Caddy |
| backend | 8001 | FastAPI | | backend | 8001 | FastAPI |
| worker-vision | — | ML scan worker | | worker-light | — | Sole celery worker — runs default/high/low queues + beat scheduler |
| worker-light | — | Lightweight worker (+ celery beat scheduler since 2026-05-11) | | db | (pg) | postgres:16 (was pgvector/pg16 until 2026-05-14) |
| db | (pg) | pgvector |
| redis | (rd) | queue | | redis | (rd) | queue |
**No vision / ML services.** The OpenCLIP ONNX classifier and its dedicated `worker-vision` were ripped out on 2026-05-14 — see Changelog.
`worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below. `worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below.
`.env` is **untracked**`git checkout .env` will wipe it. Holds: `.env` is **untracked**`git checkout .env` will wipe it. Holds:
@@ -90,6 +91,50 @@ For pushes from inside the LXC, gitea creds at `/etc/mule-deploy/git-credentials
## Changelog ## Changelog
### 2026-05-14 — Drop AI/vision pipeline, plain Postgres, DB↔FS refresh
**AI removal (commits `a27267f` + `423a73a`).** The OpenCLIP-on-ONNX
binary classifier (photography vs other) and all its scaffolding gone:
- Backend: `app/services/vision/`, `app/tasks/vision.py`,
`app/services/feature_flags.py`, `app/routers/features.py` deleted.
Admin `/admin/feature-flags`, `/admin/ai/{backfill,rescan}` endpoints
removed. `Photo.needs_review` column + `ix_photos_needs_review`
index dropped (alembic `0019_drop_ai_remnants`).
- Frontend: AI Settings tab, `useFeaturesQuery`, "Needs Review" sidebar
entry + filter, `needs_review` URL param plumbing, FeatureFlag types
all gone.
- Infra: `worker-vision` compose service + `models_data` volume
deleted. `worker-light` no longer runs `python -m
app.services.vision.bootstrap_models` before celery. Backend
Dockerfile drops the dedicated torch RUN layer. `requirements.txt`
drops `torch`, `torchvision`, `open-clip-torch`, `onnxruntime`.
**Postgres image swap.** `pgvector/pgvector:pg16``postgres:16`. The
0019 migration drops the (now-unused) `vector` extension first; the
SQL was pre-applied via `psql` against the still-pgvector container
and `alembic_version` stamped to `0019` so the new backend's
`bootstrap.py` upgrade-head was a no-op. After the swap surfaced a
collation-version mismatch (Bookworm glibc 2.36 → Trixie glibc 2.41),
the fix was `REINDEX DATABASE mulita` + `ALTER DATABASE … REFRESH
COLLATION VERSION` on `mulita`, `postgres`, `template1`. The data
volume was preserved across the image change.
**One-shot DB ↔ filesystem refresh.** New script
`backend/scripts/full_refresh.py` (run as
`docker exec -w /app mulita-backend python -m scripts.full_refresh
[--dry-run]`). Phases: `cleanup_data_integrity` → inline scan of every
active SourceRoot → `prune_missing_photos(dry_run=False)` → new
`prune_orphan_thumbnails` helper that removes
`/data/thumbs/{user}/{photo}/` dirs for photo_ids that left the DB.
First run: 0 missing photos, 1 stale folder row, **7982 orphan
thumbnail dirs** reaped.
**Manual cleanup post-deploy.** `docker compose up -d --build
--force-recreate` doesn't reap services removed from the YAML, so
`mulita-worker-vision` had to be `docker rm -f`'d by hand. The
`mule-image_models_data` volume was likewise removed manually.
### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2) ### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2)
Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files. Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files.