Files
mule-image/docker-compose.yml
root 07b1e5e02a feat: split celery workers, fix asyncpg-in-fork, add pipeline progress UI
Three overlapping fixes so the ingestion pipeline actually runs and the
user can see what it's doing:

Pipeline recovery
- app/database.py: use NullPool when MULITA_CELERY_WORKER=1 so each
  Celery task opens a fresh asyncpg connection on its own event loop.
  Fixes "another operation in progress" and "Future attached to a
  different loop" errors that were dropping ~every thumbnail +
  extract_metadata task on the floor.
- app/tasks/thumbs.py: initialize photo=None before the try and rollback
  on error so a transport failure in the initial SELECT doesn't raise
  UnboundLocalError in the except block and leak rows stuck in 'pending'.
- app/services/vision/bootstrap_models.py: on missing model files,
  invoke export_models automatically instead of just warning. First
  boot of a fresh install now self-heals.
- app/services/vision/export_models.py: shutil.move instead of
  Path.rename so the YOLO export survives the /app → /data/models
  cross-volume hop.
- requirements.txt: add ultralytics so export works in a stock image.

Worker topology
- docker-compose.yml: replace the single worker with worker-light
  (default/high/low queues, c=2, IO-bound) and worker-vision (vision
  queue, c=5, OMP_NUM_THREADS=1 to avoid oversubscription on 6 cores).
  Vision is pinned to ≤5 parallel inferences so ONNX doesn't each
  spawn an all-cores intra-op pool.
- .env / .env.example: CELERYD_CONCURRENCY replaced with
  CELERY_LIGHT_CONCURRENCY + CELERY_VISION_CONCURRENCY.
- Backfill queries in thumbs / scan / vision now ORDER BY taken_at
  DESC NULLS LAST so newest photos finish first — the library fills
  in top-down in the UI instead of arbitrary insertion order.

Settings visibility
- routers/library.py: new GET /maintenance/pipeline-stats returning
  done/total per stage (thumbnails, exif, gps, phash, embeddings,
  tags, ocr, faces, face clusters, duplicate groups). Worker-status
  now also reports the `vision` queue depth, which was missing.
- services/api.ts: PipelineStats / PipelineStage / ScanStatus types
  and the matching client call.
- components/dialogs/SettingsDialog.tsx:
  - new Pipeline Progress card with one progress bar per stage
  - inline scan banner (processed/total/current folder) inside the
    Library section while a scan is running
  - Tasks/min throughput computed by diffing worker processed counters
    between polls
  - Workers section calls out the vision queue and documents the
    CELERY_LIGHT/VISION_CONCURRENCY + docker compose up -d scale path

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 10:06:45 +02:00

200 lines
7.2 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
services:
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
container_name: mulita-frontend
ports:
# Host port is configurable via FRONTEND_PORT in .env so multiple
# instances / other services on the same host don't collide.
- "${FRONTEND_PORT:-3000}:80"
depends_on:
- backend
networks:
- mulita-network
restart: unless-stopped
backend:
build:
context: ./backend
dockerfile: Dockerfile
container_name: mulita-backend
ports:
# Direct backend access on the host is rarely needed (the frontend
# talks to it through the nginx /api proxy on the same network),
# but it's exposed for debugging / curl. Override with BACKEND_PORT.
- "${BACKEND_PORT:-8001}:8000"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
# The single host → container mount for your photo library. Set
# PHOTO_DIRS in .env to your library root. Mounted :rw because file
# operations (rename, move, empty discard pile) need to mutate the
# filesystem; flip to :ro for a strict read-only library and the
# write endpoints will return EROFS.
- ${PHOTO_DIRS:-./photos}:/photos:rw
- thumbs_data:/data/thumbs
- proxies_data:/data/proxies
- db_data:/data/db # retained so the docker-compose.sqlite.yml override has somewhere to put mulita.db
# Run Alembic migrations before starting uvicorn. On a fresh Postgres
# the empty 0001 baseline is a no-op stamp; create_all in init_db then
# builds the schema.
command: sh -c "python -c \"import asyncio; from app.database import init_db; asyncio.run(init_db())\" && alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload"
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-*}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
depends_on:
redis:
condition: service_started
db:
condition: service_healthy
networks:
- mulita-network
restart: unless-stopped
# ── Celery workers ─────────────────────────────────────────────────────
#
# The ingestion pipeline is split across two worker services so CPU-heavy
# vision tasks (embed / detect / OCR / faces / classify) cannot starve
# the fast IO-bound tasks (scan / thumbnails / EXIF / phash / duplicates).
#
# worker-light listens on default,high,low — IO-bound, cheap
# worker-vision listens on vision — CPU-bound, loads ONNX
#
# Both share the same image, photo volume, and model cache, so there's
# no disk duplication and model weights are loaded lazily only by
# worker-vision. Each service has its own concurrency knob; both
# workers ship their heartbeat to the same Redis broker so the
# Settings > Workers panel lists them side-by-side.
#
# Sizing defaults target a 6-core / 16 GB host:
# CELERY_LIGHT_CONCURRENCY=2 (enough for parallel thumbnail + EXIF)
# CELERY_VISION_CONCURRENCY=5 (5 × ~2GB ONNX = ~10GB RAM, 5/6 cores)
# Raise these in .env and run `docker compose up -d worker-light worker-vision`
# to scale. Keep light under ~4 and vision under your physical core
# count; more just thrashes.
worker-light:
build:
context: ./backend
dockerfile: Dockerfile
image: mule-image-worker
container_name: mulita-worker-light
command: sh -c "python -m app.services.vision.bootstrap_models && celery -A app.tasks.celery worker --loglevel=${LOG_LEVEL:-info} --concurrency=${CELERY_LIGHT_CONCURRENCY:-2} -Q default,high,low -n light@%h"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
- ${PHOTO_DIRS:-./photos}:/photos:rw
- thumbs_data:/data/thumbs
- proxies_data:/data/proxies
- db_data:/data/db
- models_data:/data/models
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
# NullPool — see app/database.py for rationale.
- MULITA_CELERY_WORKER=1
depends_on:
redis:
condition: service_started
backend:
condition: service_started
db:
condition: service_healthy
networks:
- mulita-network
restart: unless-stopped
worker-vision:
build:
context: ./backend
dockerfile: Dockerfile
image: mule-image-worker
container_name: mulita-worker-vision
command: sh -c "python -m app.services.vision.bootstrap_models && celery -A app.tasks.celery worker --loglevel=${LOG_LEVEL:-info} --concurrency=${CELERY_VISION_CONCURRENCY:-5} -Q vision -n vision@%h"
volumes:
- ./mulita.yml:/app/config/mulita.yml:ro
- ${PHOTO_DIRS:-./photos}:/photos:rw
- thumbs_data:/data/thumbs
- proxies_data:/data/proxies
- db_data:/data/db
- models_data:/data/models
environment:
- DATABASE_URL=postgresql+asyncpg://mulita:mulita@db:5432/mulita
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
- MULITA_CELERY_WORKER=1
# Pin each ONNX session to one intra-op thread so N prefork children
# × default-all-cores doesn't oversubscribe the box. With
# concurrency=5 and OMP=1, vision peaks at 5 busy cores, leaving
# one for worker-light + system. These env vars cover the three
# threading runtimes ONNX Runtime might pick up on first use.
- OMP_NUM_THREADS=1
- OPENBLAS_NUM_THREADS=1
- MKL_NUM_THREADS=1
depends_on:
redis:
condition: service_started
backend:
condition: service_started
db:
condition: service_healthy
networks:
- mulita-network
restart: unless-stopped
db:
image: pgvector/pgvector:pg16
container_name: mulita-db
environment:
POSTGRES_USER: mulita
POSTGRES_PASSWORD: mulita
POSTGRES_DB: mulita
volumes:
- pg_data:/var/lib/postgresql/data
networks:
- mulita-network
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -U mulita -d mulita"]
interval: 5s
timeout: 5s
retries: 10
redis:
image: redis:7-alpine
container_name: mulita-redis
# Host port exposed only for local debugging; the backend / worker
# reach Redis via the internal mulita-network on its container name.
ports:
- "${REDIS_PORT:-6379}:6379"
volumes:
- redis_data:/data
networks:
- mulita-network
restart: unless-stopped
command: redis-server --appendonly yes
networks:
mulita-network:
driver: bridge
volumes:
thumbs_data:
proxies_data:
db_data:
redis_data:
pg_data:
models_data: