oikos/knowledge/wiki/containers/120-mule-images.md

# 120 — `mule-images`

Hosts `mule-image` — the photos app at `photos.hubris.network`. PhotoPrism + Go sidecar + SvelteKit, replacing the legacy FastAPI/Celery stack as of 2026-05-22 (see Changelog). Auto-deploys from `dtoro/mule-image` on `git push origin main`.

## At a glance
- **Hostname:** `mule-images`
- **IP:** `192.168.8.136`
- **Privilege:** privileged
- **Resources:** 6 cores / 12 GiB RAM / 60 GiB rootfs / 2 GiB swap
- **Mounts:** `/mnt/library` ↔ `/mnt/library`; `/dev/dri/{card0,renderD128}` passed through for VA-API video accel on the AMD Phoenix1 iGPU.
- **Public hostname:** [`photos.hubris.network`](../infrastructure/dns.md) → [caddy](121-caddy.md) → path-routed to PhotoPrism `:2342` / sidecar `:8000` / nginx-static `:3000`.

## Stack (`/opt/mule-image`)

`/opt/mule-image` IS the working tree of `dtoro/mule-image`. Compose stack: base `docker-compose.yml` + LAN-port-rebind `docker-compose.override.yml` (untracked) + VA-API `docker-compose.gpu.yml`. The SvelteKit frontend is built on the host and served as a static bundle by nginx — no `vite dev` in prod.

| Service / process        | Port (LAN)            | Notes |
| ------------------------ | --------------------- | ----- |
| pp-mariadb               | (internal 3306)       | MariaDB 11; holds PhotoPrism's `photoprism` DB + sidecar's `mule_sidecar.*` |
| pp-app                   | `0.0.0.0:2342`        | PhotoPrism `:latest`; `PHOTOPRISM_FFMPEG_ENCODER=h264_vaapi` |
| pp-sidecar               | `0.0.0.0:8000`        | Go service (file rename / folder ops / heap convert / dup scan / per-photo marks); also reconciles `USER_BASEPATHS` into PhotoPrism's `auth_users` every 60 s |
| nginx (host process)     | `0.0.0.0:3000`        | Serves the SvelteKit static bundle from `web/build/` |

**Per-user scoping** is driven by `USER_BASEPATHS=admin:admin/files, muli:muli/files` in `.env`. Sidecar applies it to PhotoPrism's `auth_users` table on boot + every 60s, `mkdir -p`s each target so PhotoPrism's ACL filter has somewhere to point.

`.env` is **untracked** — `git reset --hard` won't touch it, but `git clean -fdx` would. Holds PP_*, MariaDB passwords, SIDECAR_DB_PASSWORD, OIDC_* (existing `mule-image` Authentik app, redirect URI now `/api/v1/oidc/redirect`), `USER_BASEPATHS`, and `PP_FFMPEG_ENCODER=vaapi`.

`docker-compose.override.yml` is also untracked — it `!override`s the upstream `127.0.0.1:port` mappings to `0.0.0.0:port` so cross-host Caddy on [LXC 121](121-caddy.md) can reach pp-app + sidecar.

## Library access

PhotoPrism reads the library **directly off the bind-mounted filesystem** — no Nextcloud webhook integration in the new stack. The base path is `/mnt/library/homecloud` and per-user scoping comes from `USER_BASEPATHS` (see Stack section above).

- `dtoro` is mapped to NC user `admin` → `/mnt/library/homecloud/admin/files/`
- `muli` is mapped to NC user `muli` → `/mnt/library/homecloud/muli/files/`

PhotoPrism's container user is `33:10000` (`www-data:media`), matching the host ownership of the NC tree. The legacy `oc_webhook_listeners` rows + `NEXTCLOUD_WEBHOOK_SECRET` are gone — when NC writes via WebDAV (still its own primary surface), PhotoPrism picks up the new file on its next index pass.

## Authentication

OIDC via Authentik. App slug `mule-image`, redirect URI `https://photos.hubris.network/api/v1/oidc/redirect` (PhotoPrism's auto-derived path; PhotoPrism builds it from `PHOTOPRISM_SITE_URL`). `OIDC_REGISTER=true` auto-creates a PhotoPrism user at role `user` on first SSO login. The sidecar's basepath reconciler then assigns their scoped folder.

The `mule-image` Authentik app's redirect URI was migrated from the legacy FastAPI `/api/v1/auth/oidc/callback` path on 2026-05-22 — same client ID/secret were reused. The separate `mule-photos-new` Authentik app was deleted in the same operation.

## Auto-deploy

Push to `dtoro/mule-image` `main` → gitea webhook → `http://192.168.8.136:9797/deploy` → `mule-deploy-webhook.service`:
- Validates HMAC against `/etc/mule-deploy/secret`
- Filters to `refs/heads/main`
- Runs `/opt/mule-deploy/deploy.sh` in a daemon thread (returns 202 immediately — docker builds exceed gitea's request timeout)
- `git fetch && git reset --hard origin/main`, force-recreates **only the sidecar** image (so PhotoPrism's `Session` HMAC key in `pp/storage/config/hub.yml` doesn't rotate and invalidate every in-flight OIDC state cookie), reconciles `pp-app` + `mariadb` in place, `cd web && npm ci && npm run build`, `systemctl reload nginx`.
- Conditionally layers `docker-compose.gpu.yml` when `/dev/dri/renderD128` exists, and always layers `docker-compose.override.yml` (the LAN-port rebind) when present.

Deploy tooling is **outside** the app repo: `/opt/mule-deploy/{deploy.sh,webhook.py}`, secret at `/etc/mule-deploy/secret`. Gitea webhook id 6. `app.ini` `ALLOWED_HOST_LIST` on [gitea](104-gitea.md) includes `192.168.8.136`.

Logs: `pct exec 120 -- journalctl -u mule-deploy-webhook -f`.
Manual deploy: `pct exec 120 -- /opt/mule-deploy/deploy.sh`.

For pushes from inside the LXC, gitea creds at `/etc/mule-deploy/git-credentials` (mode 600) — same token as `/etc/caddy-deploy/git-credentials` on [caddy](121-caddy.md).

## Related
- [Nextcloud (114)](114-nextcloud.md) — source of truth for photo libraries
- [Authentik (124)](106-auth-outpost.md)
- [Caddy (121)](121-caddy.md)
- [DNS](../infrastructure/dns.md)
- [Auto-deploy](../infrastructure/auto-deploy.md)
- [Gitea (104)](104-gitea.md)

## Changelog

### 2026-05-22 — Cutover to PhotoPrism stack (Mulimage 2.0)

The `new` branch of `dtoro/mule-image` was merged into `main` as commit
`70dc1b6`. The merge replaces the legacy FastAPI + PostgreSQL + Celery
+ React stack with PhotoPrism + Go sidecar + MariaDB + SvelteKit, lifted
in from the M0 evaluation on (now-destroyed) [LXC 127](#).

Cutover on 120 (this session):
- Bumped to 6 cores / 2 GB swap. `/dev/dri/{card0,renderD128}` already
  passed through; added an opt-in `docker-compose.gpu.yml` overlay that
  the deploy script layers in when the device is present. PhotoPrism now
  uses `h264_vaapi` instead of `libx264`.
- Installed host nginx + a small `photos.conf` vhost serving the SvelteKit
  static bundle from `/opt/mule-image/web/build/` on `:3000` — no Vite
  dev server in prod. SvelteKit's `adapter-static` produces a real SPA
  bundle.
- Replaced `/opt/mule-deploy/deploy.sh` with the 127-style multi-compose
  deploy (force-recreates only sidecar to preserve PhotoPrism's Session
  HMAC key; reconciles pp-app + mariadb in place; rebuilds web/ and
  reloads nginx).
- `/opt/mule-image/.env` was rewritten to the PhotoPrism shape. Reused
  passwords from `/root/mule-photos-new-secrets.txt`. `USER_BASEPATHS=
  admin:admin/files, muli:muli/files` exposes both libraries.
- `docker-compose.override.yml` (untracked) `!override`s pp-app +
  sidecar ports to `0.0.0.0` so cross-LXC caddy on [121](121-caddy.md)
  can reach them.
- Authentik: `mule-image` app's redirect URI updated to `/api/v1/oidc/redirect`;
  the separate `mule-photos-new` app deleted.
- Caddyfile (`dtoro/caddy-conf`): `photos.hubris.network` switched from
  plain passthrough to path-matchers (PhotoPrism API + sidecar API +
  nginx static catch-all), and `photos-new.hubris.network` was removed
  entirely.
- Cleanups: 4 Nextcloud `oc_webhook_listeners` rows for the legacy
  `/api/v1/internal/nc-webhook` endpoint deleted; gitea webhook id 9
  (the `refs/heads/new` deploy hook for 127) deleted; `192.168.8.181`
  removed from gitea's `ALLOWED_HOST_LIST`; `photos-new.hubris.network`
  removed from dnsmasq.
- LXC 127 destroyed via `pct destroy 127` after the new stack passed
  curl verification end-to-end (PP `/api/v1/status` returns
  `{"status":"operational"}` through caddy; sidecar `/api/sidecar/healthz`
  returns `{"ok":true,...}`).
- Rollback safety net: legacy `mulita` postgres dump at
  `/root/backups/mulita-legacy-final-20260521-212036.sql.gz` on hubris
  (10 MB, 16,155 photo rows). The `mule-image_mule_db_data` docker
  volume on 120 retains the on-disk postgres data for at least 24 h
  before housekeeping removes it.

### 2026-05-14 — Drop AI/vision pipeline, plain Postgres, DB↔FS refresh

**AI removal (commits `a27267f` + `423a73a`).** The OpenCLIP-on-ONNX
binary classifier (photography vs other) and all its scaffolding gone:

- Backend: `app/services/vision/`, `app/tasks/vision.py`,
  `app/services/feature_flags.py`, `app/routers/features.py` deleted.
  Admin `/admin/feature-flags`, `/admin/ai/{backfill,rescan}` endpoints
  removed. `Photo.needs_review` column + `ix_photos_needs_review`
  index dropped (alembic `0019_drop_ai_remnants`).
- Frontend: AI Settings tab, `useFeaturesQuery`, "Needs Review" sidebar
  entry + filter, `needs_review` URL param plumbing, FeatureFlag types
  all gone.
- Infra: `worker-vision` compose service + `models_data` volume
  deleted. `worker-light` no longer runs `python -m
  app.services.vision.bootstrap_models` before celery. Backend
  Dockerfile drops the dedicated torch RUN layer. `requirements.txt`
  drops `torch`, `torchvision`, `open-clip-torch`, `onnxruntime`.

**Postgres image swap.** `pgvector/pgvector:pg16` → `postgres:16`. The
0019 migration drops the (now-unused) `vector` extension first; the
SQL was pre-applied via `psql` against the still-pgvector container
and `alembic_version` stamped to `0019` so the new backend's
`bootstrap.py` upgrade-head was a no-op. After the swap surfaced a
collation-version mismatch (Bookworm glibc 2.36 → Trixie glibc 2.41),
the fix was `REINDEX DATABASE mulita` + `ALTER DATABASE … REFRESH
COLLATION VERSION` on `mulita`, `postgres`, `template1`. The data
volume was preserved across the image change.

**One-shot DB ↔ filesystem refresh.** New script
`backend/scripts/full_refresh.py` (run as
`docker exec -w /app mulita-backend python -m scripts.full_refresh
[--dry-run]`). Phases: `cleanup_data_integrity` → inline scan of every
active SourceRoot → `prune_missing_photos(dry_run=False)` → new
`prune_orphan_thumbnails` helper that removes
`/data/thumbs/{user}/{photo}/` dirs for photo_ids that left the DB.
First run: 0 missing photos, 1 stale folder row, **7982 orphan
thumbnail dirs** reaped.

**Manual cleanup post-deploy.** `docker compose up -d --build
--force-recreate` doesn't reap services removed from the YAML, so
`mulita-worker-vision` had to be `docker rm -f`'d by hand. The
`mule-image_models_data` volume was likewise removed manually.

### 2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2)

Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files.

**Phase 1 — thumbnail proxy (commits `576b0c2`, `28738ac`).** Photos table gains `nextcloud_fileid` (migration `0018`). `GET /api/v1/photos/{id}/thumb/{size}` proxies NC's `/index.php/core/preview` keyed by that fileid, authenticated with the owner's encrypted app password (small=240, medium=640, large=1280). Worker now only writes the `medium` size to `/data/thumbs` (the vision worker still loads it from disk); `small` + `large` no longer touch disk. Disk fallback stays wired for legacy rows and the rare 404 from NC (iPhone JPEGs mis-extensioned as `.DNG` — verified). Existing 17,591 photos backfilled via `backend/scripts/backfill_nextcloud_fileid.py`. Tested with real CR2, real DNG, fake DNG, JPEG, HEIC — all green.

**Range support for `/original` (commit `18dce33`).** Pre-existing bug surfaced by Phase 1 testing: `<video>` couldn't play `.mov` because `FileResponse` was returning 200 with the whole body and no `Accept-Ranges` header, so browsers reported "format not supported." Now parses `Range: bytes=START-END`, returns 206 with `Content-Range`, streams in 1 MB chunks.

**Fix: backfill_gps was the actual CPU drain, not the watcher (commit `d24c64e`).** `_scan_all_source_roots_async` auto-queued `backfill_gps` 30 s after every container boot, which then re-queued `extract_metadata` for every photo with `latitude IS NULL` — ~60k pointless tasks per deploy, pinning worker-light at 180+% CPU for ~30 min. Killed the auto-trigger; manual `POST /api/v1/library/backfill-gps` still works.

**Phase 2 — webhook receiver replaces watchfiles (commits `362fbc6`, `f657e2c`).** New endpoint `POST /api/v1/internal/nc-webhook` (auth via `Authorization: Bearer $NEXTCLOUD_WEBHOOK_SECRET`, `hmac.compare_digest`) handles `NodeCreated|NodeWritten|NodeDeleted|NodeRenamed`. Maps NC's `/admin/files/...` path to the bind-mounted `/nextcloud-users/admin/files/...` and dispatches the same `scan_folder` / `handle_file_deletion` machinery the watcher used. Registered against NC via `backend/scripts/register_nc_webhooks.py` (idempotent: deletes existing webhooks targeting the same URL first). `mulita-worker-watcher` container retired; celery `--beat` folded into `worker-light` so `discard_missing_photos_beat` still fires. End-to-end verified by uploading a test JPEG through WebDAV — both `NodeCreated` and `NodeWritten` fire, mule receives + dispatches, 200 OK.

**NC cron tightened to `*/1` so webhook delivery latency drops from up to 5 min to ~60 s** (`crontab -u www-data` on LXC 114). NC dispatches webhook calls through its background-job queue; the cron interval = the worst-case latency. `*/5` was the default; `*/1` is the upstream recommendation anyway.

**Post-Phase-2 delete-roundtrip patches (commit `9408825`).** End-to-end testing of the NC↔mule deletion paths surfaced two real gaps:

1. *Folder delete* — NC fires one `NodeDeletedEvent` for the folder, not one per child. The webhook handler bailed with "unsupported extension" and photos under the deleted folder kept `is_discarded=false` until the 30-min reconcile sweep. Fix: `handle_directory_deletion()` does a single `UPDATE photos SET is_discarded=true WHERE filepath LIKE 'dir/%'` when the deleted path has no supported image extension.
2. *Resurrect on rewrite* — PUT-overwrite of a previously-discarded file fired `NodeWrittenEvent` → `scan_folder`, but scan_folder's "photo exists, skip" branch left `is_discarded=true`. Fix: when the existing row is discarded, flip `is_discarded=false`, clear `discarded_at`, re-queue `extract_metadata`.

Verified end-to-end on `Photos/MuleTestFolder/{test1,test2,nested/test3}.jpg`: `DELETE Photos/MuleTestFolder/` discarded all 3 in one shot; `PUT test1.jpg` back resurrected only that one.

**Known remaining gap — trashbin restore.** Moving a file out of `/dav/trashbin/.../trash/foo.jpg.dXXXX` back to `/files/...` via WebDAV MOVE fires **no event mule subscribes to**. NC's trashbin app emits its own internal event class that isn't in the `OCP\Files\Events\Node\*` set we registered with `webhook_listeners`. Workaround: re-upload via PUT (covered by the resurrect-on-rewrite fix above) or wait for the 30-min reconcile sweep.

**Folder rename round-trip (commits `f4a03b6` + `f27f3cb`).** Both directions now work:

- *NC-side rename → mule:* webhook's "renamed" branch detects directory rename (neither path has a supported extension) and calls new `handle_directory_rename(old, new)` in `scan.py`. The helper iterates matching rows in Python and prefix-rewrites `Photo.filepath`, `Folder.path`, `SourceRoot.path` in one transaction. Cross-source-root case (rare) discards the old subtree and lets scan_folder add fresh rows under the new root.
- *Mule-side rename → NC:* the existing `PATCH /api/v1/folders/{id}` endpoint (`folders.py:202`) already does WebDAV MOVE via `nextcloud_dav.move_for_user` and rewrites mule's DB inline. The NodeRenamedEvent that bounces back through the webhook hits `handle_directory_rename`, which finds 0 rows under the old prefix and is a no-op — feedback loop is idempotent.

**asyncpg gotcha**: the original implementation of `handle_directory_rename` used raw SQL with `SUBSTRING(filepath FROM LENGTH(:old_prefix) + 1)` so the offset would be computed server-side. asyncpg's type inference miscategorises the `LENGTH()` result and rejects the parameter as "$2: int (expected str)". The fix iterates in Python (same pattern as the existing PATCH endpoint). Lesson: avoid passing `LENGTH(:x)+1` as an argument to `SUBSTRING(...)` via asyncpg + sqlalchemy `text()`.

**Phase 3 — Memories-backed `extract_metadata` (commit `2a5759c`).** Memories app re-enabled on NC. `extract_metadata` now tries `GET /index.php/apps/memories/api/image/info/{fileid}` (auth: Basic + `OCS-APIRequest: true` header to bypass CSRF) before falling back to ExifTool. Replaces ~80 ms of subprocess with ~1-2 ms HTTP for ongoing imports. New helpers: `nextcloud_dav.fetch_memories_info_async()` + `metadata._apply_memories_metadata()`. We kept mule's full date-fallback chain (SubSec → DateTimeOriginal → CreateDate → MediaCreateDate → TrackCreateDate → filename heuristic → mtime) because 35% of the library (`taken_at_source='path'`) depends on the filename heuristic, and Memories alone would silently mis-date those photos to mtime. PhotoInfoPanel reads `exif.Make/Model/ISO/FNumber` from `photos.exif_json` — Memories' `exif` blob uses those exact plain key names, so no frontend adapter was needed. ExifTool subprocess is still in place as the fallback for brand-new photos racing the NC scan, non-NC photos, and any NC HTTP failure.

**Phase 4 — retire `/data/thumbs` (commits `5a67ed7` + `7a1c6b6`).** Vision worker now fetches NC's 640px preview via a new sync helper `nextcloud_dav.get_preview_bytes()` instead of reading `/data/thumbs/{id}/medium.webp`. `thumbs.WORKER_THUMB_SIZES = set()` — `generate_thumbnails` still computes pHash on the original-res pixels (perceptual dedup is mule-only) but stops writing files. All ~22k `medium.webp` purged after verification; `/data/thumbs` shrank from 4.1 GB → 94 MB residual.

**SECRET_KEY bug found mid-deploy (commit `7a1c6b6`).** `docker-compose.yml` only set `SECRET_KEY` on the `backend` service, not on the workers. Workers' Fernet-based `decrypt(nextcloud_app_password_enc)` silently returned empty and `_credentials_for()` raised `NextcloudCredentialsMissing`. This meant Phase 3's `extract_metadata` in worker-light had been silently falling back to ExifTool *the entire time*, and Phase 4's vision worker couldn't fetch NC previews at all. Fix replicates `SECRET_KEY=${SECRET_KEY:-...}` to all worker services. After any compose edit, run `docker exec mulita-<svc> env | grep SECRET_KEY` to confirm propagation.

**Caveats worth knowing for Phase 4:**
- Vision is `disabled` in production (`redis GET mulita:flags:vision.enabled == "false"`). The Phase 4 vision-from-NC path is correct but unexercised by live traffic; it'll matter whenever vision is re-enabled.
- NC's preview generator hasn't covered the whole library at the 640px+ tier. A sample of 6 photos: 4 with proper sizes, 1 stuck at 160px (`IMG_4954.DNG`), 1 unfetchable (`IMG_0193.DNG`). If vision re-enables and quality matters, run `pct exec 114 -- sudo -u www-data php occ preview:generate-all` to backfill.
- The HTTP `/thumb/{size}` endpoint's inline-regen disk fallback still writes one WebP per NC-404 event (mis-extensioned RAW etc). `/data/thumbs` will grow back very slowly from that path.

### 2026-05-11 — Duplicates view shows parent folder + full-path tooltip

`GET /api/v1/library/duplicates/groups` now includes `filepath` in each member payload. `DuplicatesView` renders a dark bottom-strip on every thumbnail showing the last two folder segments — `…/<grandparent>/<parent>` (e.g. `…/files/Phone Photos starting 01-26`) via a `duplicatePathLabel` helper. Going up two levels avoids the failure mode where two copies sit under matching parent names (e.g. duplicate `2023/` subfolders under different archives) and the label would collapse. Full filepath surfaces through the native `title` tooltip on hover. Dimensions chip moves from bottom-left to top-left so the path strip can run edge-to-edge. `memberToPhoto` finally stops faking `filepath = filename` — the synthetic Photo it hands to `PhotoThumbnail` now carries the real path.

**Infra side-effect during the deploy:** two consecutive `--force-recreate` cycles (deploy.sh and a manual `down/up`) raced and left **orphan duplicate worker containers** (`<id>_mulita-worker-light` + the named `mulita-worker-light`, same for vision) both pulling from the same Redis queue. Worker concurrency was effectively doubled, host load climbed past 120, and `pct exec` into LXC 120 hung for several minutes. Recovery: `docker rm -f` on both orphan IDs. The deploy-gotcha entry below covers the pattern; the new contribution from this session is "look for renamed `<id>_<service>` containers and remove them if you ever see load spike + pct hang after a deploy".

### 2026-05-11 — Drop Title field, add bulk Notes editor

- Removed the `Title` (user_title) row from `PhotoInfoPanel.tsx`: the form field, its `titleDraft`/`setTitleDraft` state, the `commitTitle` handler, and the `photo?.user_title` watcher in the draft-sync `useEffect`. Column stays on `photos` and on the backend model — only the UI affordance went away.
- New bulk action `set_notes` in `backend/app/routers/photos.py` `bulk_action`: validates the value is a string (or null/empty to clear), then sets `user_notes` on every photo in the selection in one transaction.
- Frontend wiring: `bulkSetNotes` in `services/api.ts`, `bulkNotes` mutation in `hooks/useBulkPhotoMutations.ts` (optimistic patch with empty-string → null collapse, full rollback on error), surfaced in `RightSidebar.tsx`'s multi-photo bulk panel as a 2-row Textarea + Clear + Apply pair below the Tags section. Doesn't auto-fire on every keystroke — clicking Apply is the explicit commit (otherwise every keystroke would PATCH N rows).
- Shared `Photo` interface in `types/photo.ts` now includes `user_notes?: string | null` so the optimistic-patch typing accepts the field; previously it only existed on `PhotoInfoPanel`'s local `PhotoDetails` shape.

### 2026-05-10 — Right sidebar compact rebuild

Right sidebar previously had `<ActiveHeapCard />` + `<Header />` + a scroll region holding two parallel collapsibles ("Edit" + "Metadata"), with three nested `<Section>` sub-collapsibles (Basic Info / Camera / Location) inside Metadata. A lot of chrome for a per-photo form.

- **Removed** `frontend/src/components/heaps/ActiveHeapCard.tsx` entirely (no remaining importers; active-heap state stays in the store, Select/Discard buttons still consult it).
- `RightSidebar.tsx` single-photo branch also drops its `<Header />` strip — the new "METADATA" collapsible trigger inside `PhotoInfoPanel` is the visible title. Multi-photo branch keeps the Header for "N Photos Selected".
- `PhotoInfoPanel.tsx` is now two stacked collapsibles:
  - **Metadata** (default expanded): readonly fields (Size / Dimensions grid, Path, GPS inlined when present), a thin `<hr>`, then editable form (Filename, Title, Date Taken, Notes, Tags, Rating + Color on one row, Flag).
  - **Camera** (default expanded): isolated EXIF block (make+model, lens, 2×2 ISO/Aperture/Shutter/Focal grid).
- Compact density: Notes `rows=3 → 2`, stars/swatches `h-5 → h-4`, `space-y-2.5 → 2`, Flag buttons `text-sm → text-xs`, grid `gap-2 → gap-x-2 gap-y-1`, empty "No GPS data" chip dropped (now hidden when there are no coordinates).
- Local `Section` helper deleted from `PhotoInfoPanel.tsx` (no longer used).

`PreviewView` reuses `RightSidebar` inside its overlay aside, so the change applies in both grid and preview.

### 2026-05-10 — Timeline scroll-anchor on section switch

Clicking a folder in the left sidebar (or any other navigation that changed `currentSection` — All Photos, Rated, Discarded, etc.) didn't reset the timeline's scroll position. If the user was deep in All Photos and clicked a folder with fewer rows, the new section loaded with the previous scroll offset preserved, often landing on empty space below the last row.

`Timeline.tsx` already had a section-change effect that cleared selection and reset the auto-focus guard, and a separate auto-focus effect that selects `photos[0]` once the new query resolves. The latter has an "ensure visible" scrollTo, but it only fires if the cell is out of view AND only after the next render — by then the user has already seen the wrong scroll position. Added `parentRef.current.scrollTop = 0` synchronously inside the section-change effect so the first paint of the new section anchors at the top; the auto-focus selectPhoto then highlights photo[0] as before.

### 2026-05-10 — Filter bar 422-on-date-from

User reported "all filters broken, no photos shown" right after the metadata-extraction overhaul shipped. Tracing actual API traffic on a fresh session showed `GET /api/v1/photos?per_page=100&date_from=2026-04-10&sort=taken_at&order=desc` returning **422 Unprocessable Entity**: pydantic v2's `datetime` parser rejects bare-date strings (`"2026-04-10"`) for `Optional[datetime]` query params.

The frontend's `filtersToParams` in `store/filterStore.ts` had been padding `date_to` with `T23:59:59` for inclusive end-of-day, but `date_from` went out as a bare `YYYY-MM-DD` — so every date-range filter request 422'd, and TanStack Query's failure-state shows an empty grid. From the user's perspective it looked like "filters return nothing across the board"; from the backend it was a single endpoint signature problem.

Fix on both sides:
- **Frontend**: pad `date_from` with `T00:00:00` symmetric to `date_to`'s `T23:59:59`.
- **Backend**: switch `date_from` / `date_to` to `Optional[str]` and parse with `datetime.fromisoformat` inside the handler. `fromisoformat` accepts both bare dates (→ midnight) and full ISO strings, so any older client / curl that sends a date-only value still works. Tz-aware values get coerced to naive UTC, matching the `taken_at` column shape and the same fix applied to `PATCH /photos/{id}` earlier in the session. Bad input now returns 400 with a clear message instead of pydantic's 422.

### 2026-05-10 — Date extraction overhaul

User reported wrong "Date Taken" on stills (JPEG / HEIC / DNG). Tracing the pipeline showed the trusted-EXIF list at `services/metadata.py:230-244` ended in `EXIF:ModifyDate`, which is set every time a file is re-saved (Lightroom export, batch resize, EXIF strip), so any photo that lost its original capture metadata during editing was being labeled `taken_at_source='exif'` with the *edit* timestamp.

Pipeline rewrite:

- New trusted-EXIF priority: `SubSecDateTimeOriginal` → `DateTimeOriginal` → `CreateDate` → `QuickTime:MediaCreateDate` → `QuickTime:CreateDate`. **`ModifyDate` removed entirely.**
- Fall back to `guess_date_from_path()` (the same heuristic that already powers `has_date_warning`) when no trusted EXIF date is found. New `taken_at_source='path'` value with a "PATH" badge in the info panel; tooltip explains the date came from filename / folder rather than real EXIF.
- Skip the date-write block entirely when `photo.taken_at_source == 'manual'` so a rescan can't clobber a user correction. (Previous behavior overwrote manual edits.)
- `parse_exif_datetime` accepts tz-aware variants (`%z`), normalizes to naive UTC, and rejects the `0000:00:00` placeholder.

Backfill: new `backfill_taken_at` celery task + `POST /api/v1/library/maintenance/backfill-taken-at` endpoint. Re-enqueues `extract_metadata` for every non-manual, non-trashed photo so the new rules apply across the existing library. Snapshot before the sweep started: 11,223 `exif` + 6,039 `filesystem` + 10 `path`. Will reshape over the next ~45 min.

Side note: the default celery queue had ~209k pending tasks at the time we fired the backfill — the watcher's 5-minute restart loop (since fixed) had been re-enqueuing scans, and tasks for the 4,158 photos we hard-deleted earlier today were still sitting around. Most fail fast (`Photo not found`, ~24ms each); real work runs at ~100ms. Decided to let it drain instead of flushing — safer.

### 2026-05-10 — Two cascade-delete + datetime fixes after the hard-remove shipped

The first cut of `delete_nextcloud_source_root` blew up with `folders_parent_id_fkey` violations on the first real test (`Taco and Muli - 2024 onward`, 35 folders / 4,158 photos). Two iterations to get it right:

1. **NULL `parent_id` before deleting Folders.** Folders have a self-referential `parent_id` FK with no `ON DELETE` rule; postgres checks the constraint per row regardless of insertion order, so a flat `DELETE FROM folders WHERE id IN (...)` of the whole subtree fails on the parents whose children appear later in the same statement. Fixed with an `UPDATE folders SET parent_id = NULL WHERE id IN (folder_ids)` first.
2. **Widen the NULL UPDATE to cross-source-root children.** A "Leóns 1st Year" SourceRoot at `.../Taco and Muli - 2024 onward/Leóns 1st Year` had its own folder rows whose `parent_id` pointed into the Taco SourceRoot's hierarchy. The first patch only NULLed `parent_id` for folders whose `id` was in the delete set; the Leóns folders weren't in that set so they kept their references and the FK still tripped. Fix: `UPDATE folders SET parent_id = NULL WHERE parent_id IN (folder_ids)` — kills any incoming reference into the delete set, internal or external.

After both fixes: `DELETE /api/v1/nextcloud/source-roots/{id}` for `Taco and Muli - 2024 onward` cleared 4,158 photos and 35 folders in a single request and returned 200. UI swaps the trash icon for a `Loader2` spinner while the request is in flight (`removeRoot.isPending && removeRoot.variables === r.id`) so the multi-second cascade is visible.

Separate fix in the same session: `PATCH /api/v1/photos/{id}` returned 500 with `can't subtract offset-naive and offset-aware datetimes` when the frontend sent a tz-aware ISO string for `taken_at` (the datetime-local input is supposed to be naive but real-world locales / paste flows occasionally include `+02:00`). The DB column is `timestamp without time zone`, so asyncpg refused to bind. Normalize on the server with `astimezone(timezone.utc).replace(tzinfo=None)` if `tzinfo is not None`.

Deploy infra learning: the new `--force-recreate` in `deploy.sh` does NOT reliably recreate containers on every push — saw two consecutive deploys leave the backend at the previous `StartedAt` despite a fresh image. Manual `docker compose up -d --no-deps --force-recreate <svc>` after deploy still races with whatever compose state the auto-deploy left mid-flight, twice landing the stack in a half-broken state (orphaned `<id>_mulita-backend` rename containers, db / redis stopped). Recovery: `docker compose down && docker compose up -d`. Open question — `--force-recreate` may need to be replaced with something more explicit.

### 2026-05-10 — Hard-remove Nextcloud SourceRoot + reliable delete sync

`DELETE /api/v1/nextcloud/source-roots/{id}` was a soft-deactivate (`is_active=false`) — the trash icon in Settings only hid the SourceRoot from active queries while leaving every Folder + Photo row in the DB forever. Re-adding the same path resurrected ghosts; `prune-missing` reported zero deletes for the soft-removed entry because the cleanup code skipped inactive source roots.

Endpoint now hard-deletes the SourceRoot, every Folder under it (chunked at 500), and every Photo in those folders. `photo_tags` and `heap_photos` cascade automatically via `ON DELETE CASCADE` on the join tables. `FolderShare` uses a stringly-typed `folder_id` (no FK) so cleaned manually for both `folder_type='folder'` and `folder_type='source_root'`. Returns `{deleted_photos, deleted_folders}` so the UI can toast a count. Files in Nextcloud are untouched.

Sync side: the `watch_folders` celery task in `backend/app/tasks/scan.py` already detected filesystem deletions and soft-marked photos `is_discarded=true`, but the global `task_soft_time_limit=300` in `app/tasks/celery.py` was killing the watcher every five minutes and dropping every FS event during the restart window. The `soft_time_limit=None` on the decorator was being interpreted as "use worker default" rather than "unlimited". Override to `soft_time_limit=0, time_limit=0` (Celery convention for unbounded). Backstop: a new `discard_missing_photos()` in `app/services/cleanup.py` — soft variant of `prune_missing_photos`, walks every `_sr_state == 'present'` source root and flips `is_discarded=true` on Photo rows whose file is gone. Wired as `discard_missing_photos_beat` celery task scheduled every 30 minutes via `beat_schedule` on the celery app. Beat runs in-process on `worker-watcher` (`--beat` flag in `docker-compose.yml`) — the watcher is already a Redis-locked singleton so no need for a separate beat container.

Manual `POST /api/v1/library/maintenance/prune-missing` remains the hard-delete path for when the user wants to permanently drop orphan rows; the new beat job only soft-discards (file gone → mule-image trash, restorable).

### 2026-05-10 — OIDC auto-redirect on LoginPage

`OIDC_ENABLED=true` was already set in `.env`, so the LoginPage rendered a "Sign in with Authentik" button next to the password form. With a single trusted IdP and a logged-in Authentik session, that extra click was friction without upside.

`LoginPage.tsx` now reads `/auth/config` on mount and, if OIDC is on, immediately navigates to the OIDC login URL. Authentik recognizes the existing session and bounces back through the callback with no user interaction. Two escape hatches: `?password=1` in the URL forces the password form, and a `skipAutoSso` sessionStorage flag (set by `AuthContext.logout` and by the OIDC callback's error branch) suppresses the next auto-redirect so logouts actually log out and OIDC failures surface their error instead of looping. While the redirect is in flight the page shows "Signing in with Authentik..." plus a "Use password instead" link.

### 2026-05-10 — right sidebar restructure (heap pinned, single scroll, collapsible Metadata)

The right sidepanel had three stacked flex regions: `<ActiveHeapCard />`, `<Header />`, and `<PhotoInfoPanel />` — with `PhotoInfoPanel` carrying its own internal scroll. That left the editable fields (filename, title, notes, rating, color, flag) stuck above the readonly metadata scroll, effectively two scroll boundaries on one sidebar.

- Moved the scroll boundary up to `RightSidebar.tsx`: only `ActiveHeapCard` and `Header` stay pinned now. Editable + readonly content scroll together in one `flex-1 overflow-y-auto` region beneath them.
- `PhotoInfoPanel.tsx` dropped its `h-full` / inner `flex-1 overflow-y-auto`.
- The four readonly sections (Tags / Basic Info / Camera / Location) are now wrapped in a single outer "Metadata" `Collapsible`. Default expanded, one click hides the whole block. Sub-sections stay individually collapsible.
- Second pass: the editable form (filename / title / notes / rating / color / flag) got the same treatment under an outer "Edit" `Collapsible` so the panel is now two equal collapsible groups below the title strip. Dropped the X (clear-selection) button from the Header; Esc and grid-empty-area-click still clear.
- Third pass: split editable vs read-only between the two groups consistently. `TagsEditor` and `TakenAtEditor` had been buried inside the readonly Metadata sub-sections — `Tags` as its own Section, taken-at wedged into `Basic Info` between size/dims and the filepath. Moved both into the Edit collapsible, ordered identification → description → categorization: Filename · Title · Date Taken · Notes · Tags · Rating · Color · Flag. Metadata now holds only readonly: `Basic Info` (size, dims, path), `Camera`, `Location`.

`PreviewView` reuses `RightSidebar` under an `<aside class="overflow-hidden">`, so the change applies in both the grid sidebar and the preview overlay.

### 2026-05-10 — `photos.hubris.network` perf sweep

User reported slow load. Five fixes shipped, in order:

1. **Backend out of dev mode.** `docker-compose.yml` `command:` was running `uvicorn … --reload` (single worker, file-watcher). Page loads fan out ~15 parallel API calls; they all serialized on one event loop. → `--workers 2 --proxy-headers`. Two uvicorn worker procs now.
2. **iPhone Apple ProRAW / Linear DNG decode fixed.** rawpy 0.26.1 + LibRaw 0.22 rejects Apple Linear DNG (`Photometric Interpretation: Linear Raw`, 8-bit) as "Unsupported file format or not RAW file". Those files are TIFF containers with developed RGB inside and have no embedded preview to fall back to. Added PIL `Image.open(src_path)` as the last fallback in both `_generate_proxy_webp` (routers/photos.py) and `process_raw_image` (tasks/thumbs.py). ~1,300 iPhone DNGs were 415-ing on every detail view; now decode in <1s via PIL.
3. **Reconcile DB with disk (renamed-folder case).** `prune_missing_photos` was skipping all 4,154 orphaned photo rows under SourceRoot `Taco and Muli - 2024 onward` (renamed in Nextcloud to `Photo Archive 2004-2024`) because the leaf path didn't resolve and the code assumed "drive unmounted → must skip". Added `_sr_state()` to classify as `present` / `renamed` (parent mount fine, leaf gone) / `unmounted` (parent inaccessible). Only `unmounted` still skips. Two stale source roots logged with a clearer hint pointing at `POST /api/v1/library/maintenance/prune-missing`. User has not yet been asked to run that — endpoint is ready when they are.
4. **Frontend page size + idle polling.** `usePhotosQuery.ts` was first-fetching `per_page=500` (slow paint + 500 thumb requests at once). Split into `PER_PAGE_INITIAL=100` for first paint, `PER_PAGE_BACKGROUND=500` for the cursor-chain prefetch. Idle polling for `scan-status` and `worker-status` (`useScanActivity.ts`, `ScanProgress.tsx`) bumped from 10s/15s to 30s/30s while idle; active cadence (2s/3s) unchanged.
5. **Partial index on `photos`.** Default list query `WHERE NOT is_trashed AND NOT is_hidden ORDER BY taken_at DESC NULLS LAST, id DESC LIMIT N` was doing a seq-scan + top-N heapsort (~25ms standalone, worse under concurrency). Added migration `0017_photos_list_index` creating `ix_photos_list_visible` (partial index on the sort key, restricted to visible rows). EXPLAIN now shows an Index Only Scan → **24.7ms → 0.097ms**, ~250× speedup.

**Deploy gotcha — fixed (with caveat).** The original `docker compose up -d --build` in `deploy.sh` did not reliably recreate containers when only runtime config (`command:`, env-only) or migration files changed; image hash would change but compose treated the existing container as "current enough". Bit three times this session before `/opt/mule-deploy/deploy.sh` was updated to `docker compose up -d --build --force-recreate`. Trade-off accepted: an extra restart cycle on deploys where nothing user-visible changed.

**Caveat:** the first auto-deploy after the flag flip raced with my own earlier manual `docker compose up -d --no-deps --force-recreate frontend` and landed the stack in a half-broken state — `mulita-frontend` got stuck under a renamed temp container, several services dropped off `mule-image_mulita-network`, frontend nginx restarted in a loop with `host not found in upstream "backend"`. Fixed by `docker compose down && docker compose up -d`. Don't issue a manual `--force-recreate` on a single service while the auto-deploy webhook is also expected to fire — let the deploy own the lifecycle.

**Data drift still outstanding.** 4,154 photo rows + 1 unregistered folder (`Photo Archive 2004-2024`) on disk that's not a SourceRoot. The reconcile endpoints now work — user decides when to call them. The new folder needs to be added as a SourceRoot via the Settings UI before its files will be indexed.

**Proxy cache still empty** (`mule-image_proxies_data` volume is 4 KB). Pre-generating ~500–1,500 WebP proxies for non-web-safe formats would make first-open of every RAW/HEIC photo instant. Deferred — needs a one-shot script and the disk-space tradeoff isn't worth it until the data-drift reconcile happens first.

### 2026-04-28 — wiki entry created
Initial documentation.

### 2026-04-26 — Nextcloud-rooted libraries shipped
Bind `/mnt/library/homecloud` into the workers, reads via filesystem, writes via WebDAV. `users.nextcloud_username` override field added; `dtoro → admin` mapping. Surviving SourceRoots cleaned up to NC paths.

### 2026-04-22 — native OIDC via Authentik
Authlib-based code in `backend/app/auth_oidc.py`. `extra_hosts` override for `auth.hubris.network` in compose override (gitignored).

### 2026-04-21 — auto-deploy pipeline shipped
Webhook receiver at `:9797`, async deploy returning 202. Mirrors caddy-conf / gitea-customizations.