Commit Graph

12 Commits

Author SHA1 Message Date
root
afe420c620 fix: use CPU-only PyTorch and run init_db before alembic migrations
PyTorch default install pulls ~7GB of CUDA libs, exceeding disk on small
VMs. Switching to CPU-only saves ~6GB. Also run create_all before alembic
so migrations find existing tables on a fresh Postgres.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:23:32 +02:00
7558aeb5e6 fix: sync DB sessions in Celery, letterbox YuNet, dedupe detections
- Rewrite all vision tasks to use sync psycopg2 sessions instead of
  asyncpg — fixes 'another operation in progress' and event loop errors
  when Celery forks workers sharing the async connection pool
- Letterbox-pad images to exactly 640x640 for YuNet face detector
  (was crashing on non-square thumbnails)
- Deduplicate object detections per label per photo — keep highest
  confidence only to avoid photo_tags PK violation on multiple
  detections of the same class
- Add all queues (-Q default,high,low,vision) to worker command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 10:28:17 +02:00
2a6661f779 fix: model weights setup — export scripts, ORT compat, bootstrap
- Add export_models.py for OpenCLIP ViT-B/32 and YOLOv8n ONNX export
- Fix ArgMax(13) ORT ARM64 incompatibility by passing eot_indices as a
  separate ONNX input (computed outside the graph in embed.py)
- Use legacy TorchScript exporter (dynamo=False) for IR version 9 compat
- Upgrade onnxruntime to 1.18.1
- Rewrite bootstrap_models.py with clear separation of auto-downloadable
  models (YuNet, SFace) vs manually-exported ones (OpenCLIP, YOLOv8n)
- Wire bootstrap into worker CMD (runs before Celery)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:41:25 +02:00
9282a5c734 feat: add vision pipeline scaffolding with ONNX backend
Introduce the app/services/vision/ module with ABC interfaces, ONNX
Runtime backend, model registry, and per-task implementations:
- OpenCLIP ViT-B/32 embedder (image + text, 512-d)
- RapidOCR engine (PP-OCRv4 via ONNX, no PaddlePaddle)
- YOLOv8n object detector (raw ONNX, no ultralytics runtime)
- YuNet + SFace face processor (Apache 2.0, opencv_zoo, 128-d)
- DBSCAN face clustering helper

Add VisionSettings to config (mulita.yml + Pydantic), bootstrap_models.py
for first-boot weight downloads, models_data Docker volume, and ROCm
backend stub for future GPU acceleration.

No Celery tasks wired yet — models load but nothing invokes them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:00:06 +02:00
dea04ceed9 feat: migrate to Postgres + pgvector with Alembic scaffolding
Switch the default database from SQLite to Postgres + pgvector (via
pgvector/pgvector:pg16 Docker image) to support the upcoming vision
pipeline (embeddings, OCR, object detection, face clustering).

- Add `db` service to docker-compose.yml with healthcheck
- Wire `alembic upgrade head` into backend CMD before uvicorn
- Bootstrap empty 0001_baseline revision (schema still owned by create_all)
- Guard SQLite-only PRAGMAs and inline ALTERs behind _is_sqlite flag
- Run `CREATE EXTENSION IF NOT EXISTS vector` on Postgres init
- Add asyncpg, psycopg2-binary, pgvector to requirements
- Provide docker-compose.sqlite.yml escape hatch for legacy SQLite mode

Fresh DB + rescan assumed — no SQLite→Postgres data migration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:46:20 +02:00
fc63b1f69d config: env-driven CORS, ports, log level, timezone
The CORS allowed-origins list, host port mappings, log level, container
timezone, and worker concurrency are now all driven by environment
variables with sane defaults. Same-origin access through the nginx
proxy keeps working with no config; direct cross-origin backend
access can be locked down via ALLOWED_ORIGINS.

- backend/config: ALLOWED_ORIGINS env (comma-separated, "*" for any)
  exposed via settings.cors_origins. LOG_LEVEL too.
- backend/main: build the CORS middleware from settings.cors_origins,
  auto-disable allow_credentials when origins is wildcard (CORS spec
  forbids credentials + "*").
- docker-compose: parameterize FRONTEND_PORT, BACKEND_PORT, REDIS_PORT,
  CELERYD_CONCURRENCY, LOG_LEVEL, and TZ via ${VAR:-default} so each
  has a working fallback if the .env entry is missing.
- .env.example: new template documenting every knob with examples.
- .env: pruned to only the values that diverge from .env.example;
  removed dead VITE_API_URL.
- README: configuration knobs table + "accessing from another machine"
  section explaining the same-origin proxy story.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:31:59 +02:00
204d2bf2a8 feat: simplify folder setup — single mount, auto bootstrap, browser dialog
Cleans up the maze of overlapping ways folders entered the app, plus
removes the dead trash plumbing left over from the soft-discard
refactor.

Setup model (now)
- ONE env var: PHOTO_DIRS in .env, set to the host path of your
  library. Compose mounts that at /photos. That's the entire setup.
- On first boot, the backend auto-creates a SourceRoot row named
  "Library" pointing at /photos so the user sees their photos
  immediately without configuring anything.
- Source roots and discard live in the database; mulita.yml only
  carries operational settings (thumbnails, scanner, performance).
- The "Add Source Folder" dialog is now a directory browser
  restricted server-side to /photos and any existing source root —
  the user clicks through actual mounted directories instead of
  typing container paths they can't possibly know.

Backend
- New services/scanner.bootstrap_default_source_root(): if no
  SourceRoot rows exist and /photos is mounted, create one. Wired
  into the lifespan handler before cleanup + initial scan.
- New GET /library/browse?path= returning the immediate child
  directories of `path`, validated to live under one of the allowed
  roots (default mount + every active SourceRoot). Hidden entries
  are filtered. Children are tagged with is_existing_root so the UI
  can show an "Added" badge. Returns parent path for up-nav, or
  null when at the top of the allowed scope.
- scan_all_source_roots now reads from the DB instead of the YAML
  config so DB-managed source roots are honoured by initial scan.
- Dropped the placeholder source_roots block from mulita.yml — the
  paths /photos/main and /photos/iphone never existed and just
  produced startup warnings.
- Dropped TrashSettings, settings.trash, settings.source_roots,
  and the SourceRoot pydantic model from config.py. Soft discard
  has owned this for a while; it was dead code.

Compose
- Single ${PHOTO_DIRS:-./photos}:/photos:rw mount in both backend
  and worker.
- Removed the hardcoded ~/Pictures:/host/Pictures:rw mount — the
  PHOTO_DIRS variable is the single source of truth now.
- Removed the trash_data named volume + mounts (no consumers).
- backend/Dockerfile no longer creates /data/trash; it now creates
  /data/proxies (which the proxy endpoint actually uses).

Frontend
- AddSourceFolderDialog rewritten as a directory tree picker:
  loads /library/browse on open, lets the user navigate up via a
  ChevronUp button or down by clicking subfolders, shows the
  current path inline, and adds whatever directory is currently
  shown. Existing source roots are tagged "Added" so the user
  knows what's already registered. Errors from the backend (e.g.
  trying to navigate outside the allowed scope) surface inline.
- New library.browse() helper + BrowseChild / BrowseResponse types
  in services/api.ts.

Docs
- README Quick Start rewritten around the single PHOTO_DIRS env
  var, with macOS/Linux/Windows examples.
- New "How mounted folders and source folders relate" section that
  spells out the two-layer model (mount = visibility, source root
  = scanning) so the most common confusion is addressed up front.
- Added a "Read-only libraries" subsection that lists exactly which
  endpoints fail under :ro.
- "Configuration" section reframed: source roots are managed by the
  UI/API now, mulita.yml is operational settings only.
- .env file now has examples for the common host paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 00:27:21 +02:00
9729391dcc chore(docker): mount ~/Pictures read-write so file ops work; document
The first phase-11 file op (inline rename) returns EROFS today
because docker-compose mounts ~/Pictures read-only by default.
Lightroom-style file operations (rename, move, discard-pile empty)
all need to mutate the filesystem, so the right default is :rw.

Flips both the backend and worker mounts to :rw with an inline
comment explaining the trade-off, and adds a "Photo directory
mounts and permissions" section to the README that:
- States the default is now :rw
- Explains exactly which endpoints fail under :ro (rename, empty
  discard pile, future move/copy)
- Notes the implication: Mulita has full write access to whatever
  host directory ends up at /host/Pictures, same trust model as
  Lightroom's catalog folder

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:56:27 +02:00
f7bf22db29 chore(docker): add proxies_data volume for /proxy endpoint cache
The /photos/{id}/proxy endpoint (added in 1096854) caches transcoded
RAW/HEIC WebPs at /data/proxies/{id}.webp, but the compose file had no
volume mount for that path — files would be lost on every container
restart, forcing repeated full-resolution decodes. Adding a named
volume to both backend and worker so the cache survives restarts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 22:15:43 +02:00
78e12e8309 feat: timeline and folder import 2026-04-07 00:42:22 +02:00
6d1b227fb9 feat: structure 2 2026-04-07 00:15:00 +02:00
46a0d7aba8 feat: structure 2026-04-06 23:30:19 +02:00