Three overlapping fixes so the ingestion pipeline actually runs and the
user can see what it's doing:
Pipeline recovery
- app/database.py: use NullPool when MULITA_CELERY_WORKER=1 so each
Celery task opens a fresh asyncpg connection on its own event loop.
Fixes "another operation in progress" and "Future attached to a
different loop" errors that were dropping ~every thumbnail +
extract_metadata task on the floor.
- app/tasks/thumbs.py: initialize photo=None before the try and rollback
on error so a transport failure in the initial SELECT doesn't raise
UnboundLocalError in the except block and leak rows stuck in 'pending'.
- app/services/vision/bootstrap_models.py: on missing model files,
invoke export_models automatically instead of just warning. First
boot of a fresh install now self-heals.
- app/services/vision/export_models.py: shutil.move instead of
Path.rename so the YOLO export survives the /app → /data/models
cross-volume hop.
- requirements.txt: add ultralytics so export works in a stock image.
Worker topology
- docker-compose.yml: replace the single worker with worker-light
(default/high/low queues, c=2, IO-bound) and worker-vision (vision
queue, c=5, OMP_NUM_THREADS=1 to avoid oversubscription on 6 cores).
Vision is pinned to ≤5 parallel inferences so ONNX doesn't each
spawn an all-cores intra-op pool.
- .env / .env.example: CELERYD_CONCURRENCY replaced with
CELERY_LIGHT_CONCURRENCY + CELERY_VISION_CONCURRENCY.
- Backfill queries in thumbs / scan / vision now ORDER BY taken_at
DESC NULLS LAST so newest photos finish first — the library fills
in top-down in the UI instead of arbitrary insertion order.
Settings visibility
- routers/library.py: new GET /maintenance/pipeline-stats returning
done/total per stage (thumbnails, exif, gps, phash, embeddings,
tags, ocr, faces, face clusters, duplicate groups). Worker-status
now also reports the `vision` queue depth, which was missing.
- services/api.ts: PipelineStats / PipelineStage / ScanStatus types
and the matching client call.
- components/dialogs/SettingsDialog.tsx:
- new Pipeline Progress card with one progress bar per stage
- inline scan banner (processed/total/current folder) inside the
Library section while a scan is running
- Tasks/min throughput computed by diffing worker processed counters
between polls
- Workers section calls out the vision queue and documents the
CELERY_LIGHT/VISION_CONCURRENCY + docker compose up -d scale path
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The CORS allowed-origins list, host port mappings, log level, container
timezone, and worker concurrency are now all driven by environment
variables with sane defaults. Same-origin access through the nginx
proxy keeps working with no config; direct cross-origin backend
access can be locked down via ALLOWED_ORIGINS.
- backend/config: ALLOWED_ORIGINS env (comma-separated, "*" for any)
exposed via settings.cors_origins. LOG_LEVEL too.
- backend/main: build the CORS middleware from settings.cors_origins,
auto-disable allow_credentials when origins is wildcard (CORS spec
forbids credentials + "*").
- docker-compose: parameterize FRONTEND_PORT, BACKEND_PORT, REDIS_PORT,
CELERYD_CONCURRENCY, LOG_LEVEL, and TZ via ${VAR:-default} so each
has a working fallback if the .env entry is missing.
- .env.example: new template documenting every knob with examples.
- .env: pruned to only the values that diverge from .env.example;
removed dead VITE_API_URL.
- README: configuration knobs table + "accessing from another machine"
section explaining the same-origin proxy story.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The frontend AddSourceFolderDialog let users register source roots
from inside the app, but with the bootstrap auto-creating one for
the /photos mount on first boot, the dialog was redundant in the
common case and confusing in every other (users had to know which
container path corresponded to their host directory). Going
config-driven matches Plex/Photoprism/Immich and matches the
mental model "the docker mount IS the library".
Frontend
- Deleted components/dialogs/AddSourceFolderDialog.tsx entirely.
- LeftSidebar drops the "+ Add Source Folder" button + bottom-bar
layout, the addFolderMutation, the dead Plus action button on
the (no-longer-existing) folders/heaps tree headers, and the
Plus icon import.
- api.ts: removed sourceFolders.add(), library.browse(), and the
BrowseChild / BrowseResponse types. The remaining sourceFolders
surface is read-only (list + manual scan).
- LeftSidebar bottom strip is now just the "Scan all folders"
button when there's at least one source root.
Backend
- Dropped POST /folders (no consumers) along with FolderCreate /
FolderResponse pydantic models. The folders router header now
documents the config-driven approach.
- Dropped GET /library/browse (no consumers). Removed the unused
os/HTTPException/SourceRoot imports it brought in.
- cleanup_data_integrity now also walks the source roots and logs
a warning for any whose path is missing on disk. Doesn't auto-
delete (a missing path could be a temporarily unmounted drive)
but surfaces enough hint to fix it. Returns the count in the
summary dict alongside merged-duplicates.
Docs
- README "How libraries are managed" section rewritten to spell
out that mounts ARE source roots, edit .env + restart, no UI for
managing source roots. New "Changing or adding libraries"
section walks through the typical edit-restart loop including
the optional volume-nuke for a clean slate.
- "Adding more libraries" subsection covers multi-mount via
edited compose with a note that auto-registration is roadmap.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cleans up the maze of overlapping ways folders entered the app, plus
removes the dead trash plumbing left over from the soft-discard
refactor.
Setup model (now)
- ONE env var: PHOTO_DIRS in .env, set to the host path of your
library. Compose mounts that at /photos. That's the entire setup.
- On first boot, the backend auto-creates a SourceRoot row named
"Library" pointing at /photos so the user sees their photos
immediately without configuring anything.
- Source roots and discard live in the database; mulita.yml only
carries operational settings (thumbnails, scanner, performance).
- The "Add Source Folder" dialog is now a directory browser
restricted server-side to /photos and any existing source root —
the user clicks through actual mounted directories instead of
typing container paths they can't possibly know.
Backend
- New services/scanner.bootstrap_default_source_root(): if no
SourceRoot rows exist and /photos is mounted, create one. Wired
into the lifespan handler before cleanup + initial scan.
- New GET /library/browse?path= returning the immediate child
directories of `path`, validated to live under one of the allowed
roots (default mount + every active SourceRoot). Hidden entries
are filtered. Children are tagged with is_existing_root so the UI
can show an "Added" badge. Returns parent path for up-nav, or
null when at the top of the allowed scope.
- scan_all_source_roots now reads from the DB instead of the YAML
config so DB-managed source roots are honoured by initial scan.
- Dropped the placeholder source_roots block from mulita.yml — the
paths /photos/main and /photos/iphone never existed and just
produced startup warnings.
- Dropped TrashSettings, settings.trash, settings.source_roots,
and the SourceRoot pydantic model from config.py. Soft discard
has owned this for a while; it was dead code.
Compose
- Single ${PHOTO_DIRS:-./photos}:/photos:rw mount in both backend
and worker.
- Removed the hardcoded ~/Pictures:/host/Pictures:rw mount — the
PHOTO_DIRS variable is the single source of truth now.
- Removed the trash_data named volume + mounts (no consumers).
- backend/Dockerfile no longer creates /data/trash; it now creates
/data/proxies (which the proxy endpoint actually uses).
Frontend
- AddSourceFolderDialog rewritten as a directory tree picker:
loads /library/browse on open, lets the user navigate up via a
ChevronUp button or down by clicking subfolders, shows the
current path inline, and adds whatever directory is currently
shown. Existing source roots are tagged "Added" so the user
knows what's already registered. Errors from the backend (e.g.
trying to navigate outside the allowed scope) surface inline.
- New library.browse() helper + BrowseChild / BrowseResponse types
in services/api.ts.
Docs
- README Quick Start rewritten around the single PHOTO_DIRS env
var, with macOS/Linux/Windows examples.
- New "How mounted folders and source folders relate" section that
spells out the two-layer model (mount = visibility, source root
= scanning) so the most common confusion is addressed up front.
- Added a "Read-only libraries" subsection that lists exactly which
endpoints fail under :ro.
- "Configuration" section reframed: source roots are managed by the
UI/API now, mulita.yml is operational settings only.
- .env file now has examples for the common host paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
