120-mule-images: drop AI pipeline, swap to plain postgres, log refresh
Reflects state after the 2026-05-14 session: the OpenCLIP vision classifier and worker-vision are gone, the db image is now postgres:16, and the one-shot full_refresh.py reaped 7982 orphan thumbnail dirs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -18,11 +18,12 @@ Hosts `mule-image` / "mulita" — the photos app at `photos.hubris.network`. Aut
|
|||||||
| ----------------- | ------ | ----- |
|
| ----------------- | ------ | ----- |
|
||||||
| frontend | 3000 | Reverse-proxied by Caddy |
|
| frontend | 3000 | Reverse-proxied by Caddy |
|
||||||
| backend | 8001 | FastAPI |
|
| backend | 8001 | FastAPI |
|
||||||
| worker-vision | — | ML scan worker |
|
| worker-light | — | Sole celery worker — runs default/high/low queues + beat scheduler |
|
||||||
| worker-light | — | Lightweight worker (+ celery beat scheduler since 2026-05-11) |
|
| db | (pg) | postgres:16 (was pgvector/pg16 until 2026-05-14) |
|
||||||
| db | (pg) | pgvector |
|
|
||||||
| redis | (rd) | queue |
|
| redis | (rd) | queue |
|
||||||
|
|
||||||
|
**No vision / ML services.** The OpenCLIP ONNX classifier and its dedicated `worker-vision` were ripped out on 2026-05-14 — see Changelog.
|
||||||
|
|
||||||
`worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below.
|
`worker-watcher` was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below.
|
||||||
|
|
||||||
`.env` is **untracked** — `git checkout .env` will wipe it. Holds:
|
`.env` is **untracked** — `git checkout .env` will wipe it. Holds:
|
||||||
@@ -90,6 +91,50 @@ For pushes from inside the LXC, gitea creds at `/etc/mule-deploy/git-credentials
|
|||||||
|
|
||||||
## Changelog
|
## Changelog
|
||||||
|
|
||||||
|
### 2026-05-14 — Drop AI/vision pipeline, plain Postgres, DB↔FS refresh
|
||||||
|
|
||||||
|
**AI removal (commits `a27267f` + `423a73a`).** The OpenCLIP-on-ONNX
|
||||||
|
binary classifier (photography vs other) and all its scaffolding gone:
|
||||||
|
|
||||||
|
- Backend: `app/services/vision/`, `app/tasks/vision.py`,
|
||||||
|
`app/services/feature_flags.py`, `app/routers/features.py` deleted.
|
||||||
|
Admin `/admin/feature-flags`, `/admin/ai/{backfill,rescan}` endpoints
|
||||||
|
removed. `Photo.needs_review` column + `ix_photos_needs_review`
|
||||||
|
index dropped (alembic `0019_drop_ai_remnants`).
|
||||||
|
- Frontend: AI Settings tab, `useFeaturesQuery`, "Needs Review" sidebar
|
||||||
|
entry + filter, `needs_review` URL param plumbing, FeatureFlag types
|
||||||
|
all gone.
|
||||||
|
- Infra: `worker-vision` compose service + `models_data` volume
|
||||||
|
deleted. `worker-light` no longer runs `python -m
|
||||||
|
app.services.vision.bootstrap_models` before celery. Backend
|
||||||
|
Dockerfile drops the dedicated torch RUN layer. `requirements.txt`
|
||||||
|
drops `torch`, `torchvision`, `open-clip-torch`, `onnxruntime`.
|
||||||
|
|
||||||
|
**Postgres image swap.** `pgvector/pgvector:pg16` → `postgres:16`. The
|
||||||
|
0019 migration drops the (now-unused) `vector` extension first; the
|
||||||
|
SQL was pre-applied via `psql` against the still-pgvector container
|
||||||
|
and `alembic_version` stamped to `0019` so the new backend's
|
||||||
|
`bootstrap.py` upgrade-head was a no-op. After the swap surfaced a
|
||||||
|
collation-version mismatch (Bookworm glibc 2.36 → Trixie glibc 2.41),
|
||||||
|
the fix was `REINDEX DATABASE mulita` + `ALTER DATABASE … REFRESH
|
||||||
|
COLLATION VERSION` on `mulita`, `postgres`, `template1`. The data
|
||||||
|
volume was preserved across the image change.
|
||||||
|
|
||||||
|
**One-shot DB ↔ filesystem refresh.** New script
|
||||||
|
`backend/scripts/full_refresh.py` (run as
|
||||||
|
`docker exec -w /app mulita-backend python -m scripts.full_refresh
|
||||||
|
[--dry-run]`). Phases: `cleanup_data_integrity` → inline scan of every
|
||||||
|
active SourceRoot → `prune_missing_photos(dry_run=False)` → new
|
||||||
|
`prune_orphan_thumbnails` helper that removes
|
||||||
|
`/data/thumbs/{user}/{photo}/` dirs for photo_ids that left the DB.
|
||||||
|
First run: 0 missing photos, 1 stale folder row, **7982 orphan
|
||||||
|
thumbnail dirs** reaped.
|
||||||
|
|
||||||
|
**Manual cleanup post-deploy.** `docker compose up -d --build
|
||||||
|
--force-recreate` doesn't reap services removed from the YAML, so
|
||||||
|
`mulita-worker-vision` had to be `docker rm -f`'d by hand. The
|
||||||
|
`mule-image_models_data` volume was likewise removed manually.
|
||||||
|
|
||||||
### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2)
|
### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2)
|
||||||
|
|
||||||
Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files.
|
Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files.
|
||||||
|
|||||||
Reference in New Issue
Block a user