Files
oikos/containers/120-mule-images.md
claudio 599715a447 120 mule-images: folder rename round-trip both ways
- handle_directory_rename in scan.py covers NC-side renames via the
  webhook. Iterates rows in Python (asyncpg int-type quirk on raw
  SUBSTRING+LENGTH).
- Existing PATCH /folders/{id} handles mule-side renames; webhook
  feedback hits the same helper and is a 0-row no-op (idempotent).
2026-05-11 13:19:05 +02:00

33 KiB
Raw Blame History

120 — mule-images

Hosts mule-image / "mulita" — the photos app at photos.hubris.network. Auto-deploys from dtoro/mule-image on git push origin main.

At a glance

  • Hostname: mule-images
  • IP: 192.168.8.136
  • Privilege: privileged
  • Resources: 4 cores / 8 GiB RAM / 60 GiB rootfs
  • Mounts: /mnt/library/mnt/library
  • Public hostname: photos.hubris.networkcaddy:3000 (frontend)

Stack (/opt/mule-image)

/opt/mule-image IS the working tree of dtoro/mule-image. Compose at /opt/mule-image/docker-compose.yml. Services:

Service Port Notes
frontend 3000 Reverse-proxied by Caddy
backend 8001 FastAPI
worker-vision ML scan worker
worker-light Lightweight worker (+ celery beat scheduler since 2026-05-11)
db (pg) pgvector
redis (rd) queue

worker-watcher was retired 2026-05-11 — file events come from NC webhook_listeners instead of watchfiles inotify. See "Nextcloud integration" below.

.env is untrackedgit checkout .env will wipe it. Holds:

  • PHOTO_DIRS=/mnt/library/images/
  • NEXTCLOUD_USERS_HOST_PATH=/mnt/library/homecloud
  • NEXTCLOUD_BASE_URL=https://cloud.hubris.network
  • OIDC client secret + scopes
  • SECRET_KEY (generated)
  • NEXTCLOUD_WEBHOOK_SECRET (since 2026-05-11; bearer token NC presents on every webhook call to /api/v1/internal/nc-webhook)

Nextcloud integration (since 2026-05-11)

Phase 1 + 2 of dropping the duplicated indexing work between mule and Nextcloud — see ~/.claude/projects/-root/memory/project_mule_nc_integration.md for the full detail.

Thumbnails (Phase 1, commit 576b0c2): GET /api/v1/photos/{id}/thumb/{size} proxies https://cloud.hubris.network/index.php/core/preview keyed by photos.nextcloud_fileid (alembic 0018), authenticated with the photo owner's encrypted NC app password. NC handles JPEG/HEIC/TIFF/RAW/DNG/Movie previews natively. X-Mule-Thumb-Source: nextcloud|disk on every response. Worker now only writes the medium size to /data/thumbs/{user}/{photo}/medium.webp (the vision worker still loads it from disk); small + large are NC-proxied. Backfill script: backend/scripts/backfill_nextcloud_fileid.py — 17,591/17,591 photos filled.

File events (Phase 2, commits 362fbc6 + f657e2c): mule no longer runs watchfiles. NC's webhook_listeners app (already installed, registered via backend/scripts/register_nc_webhooks.py) POSTs every NodeCreated|NodeWritten|NodeDeleted|NodeRenamed event to http://192.168.8.136:8001/api/v1/internal/nc-webhook with Authorization: Bearer $NEXTCLOUD_WEBHOOK_SECRET. The endpoint dispatches the same scan_folder / handle_file_deletion machinery the watcher used. NC dispatches webhooks asynchronously via its background-job queuecrontab -u www-data on LXC 114 runs cron.php every minute (*/1), so end-to-end latency is ≲60 s. Re-running the registrar script is idempotent (it deletes existing webhooks pointing at the same URL first).

Nextcloud-rooted libraries (since 2026-04-26)

Photo libraries live under each user's Nextcloud files/ tree, NOT in /mnt/library/images/*.

  • /mnt/library/homecloud is bind-mounted into backend, worker-light, worker-watcher, worker-vision as /nextcloud-users. Each NC user is /nextcloud-users/<nc_user>/files/.
  • Reads use that bind directly.
  • Mutations (upload, delete, rename, move) dispatch through services/nextcloud_dav.py (HTTP Basic auth, per-user app password Fernet-encrypted in users.nextcloud_app_password_enc) so Nextcloud's oc_filecache, trashbin, comments, and desktop-sync clients stay coherent.
  • Photo copy + cross-system moves return 501 with a "use Nextcloud's web UI" hint — defer until needed.
  • users.nextcloud_username overrides the default OIDC preferred_username. dtoro (mule-image) maps to admin (Nextcloud) — don't assume username equality.
  • Surviving SourceRoots in DB: Photos/nextcloud-users/admin/files/Photos; Memories/nextcloud-users/admin/files/Memories (both owned by dtoro). User muli has nextcloud_username=muli backfilled but no SourceRoot yet.
  • Pre-migration DB dump: /root/snapshots/mulita-pre-nc-migration-20260426-075132.dump (11 MB) on the host.

Authentication (since 2026-04-22)

Native OIDC via Authentik. Code in backend/app/auth_oidc.py, routes /api/v1/auth/oidc/{login,callback}. Authentik side:

  • OAuth2/OIDC Provider, client ID fCuHew48ONTskDjUKnMTZjFbVXuHwvQqTScQRNQ1
  • App slug mule-image
  • Redirect URI: https://photos.hubris.network/api/v1/auth/oidc/callback

Backend container needs extra_hosts: auth.hubris.network:192.168.8.175 via docker-compose.override.yml (gitignored). Otherwise Authlib's metadata fetch fails with SSL: CERTIFICATE_VERIFY_FAILED: self-signed certificate (it ends up at a random public host because LXC DNS resolves the public IONOS A record).

Caddyfile stays plain reverse_proxy 192.168.8.136:3000 — no forward-auth, no /api/* bypass needed.

Auto-deploy

Push to dtoro/mule-image main → gitea webhook → http://192.168.8.136:9797/deploymule-deploy-webhook.service:

  • Validates HMAC against /etc/mule-deploy/secret
  • Filters to refs/heads/main
  • Runs /opt/mule-deploy/deploy.sh in a daemon thread (returns 202 immediately — docker builds exceed gitea's request timeout)
  • git pull --ff-only + docker compose up -d --build + docker image prune -f

Deploy tooling is outside the app repo: /opt/mule-deploy/{deploy.sh,webhook.py}, secret at /etc/mule-deploy/secret, unit at /etc/systemd/system/mule-deploy-webhook.service. Same shape as the Caddy + Artifacto pipelines. Gitea webhook id 6.

app.ini ALLOWED_HOST_LIST on gitea includes 192.168.8.136.

Logs: pct exec 120 -- journalctl -u mule-deploy-webhook -f. Manual deploy: pct exec 120 -- /opt/mule-deploy/deploy.sh.

For pushes from inside the LXC, gitea creds at /etc/mule-deploy/git-credentials (mode 600) — same token as /etc/caddy-deploy/git-credentials on caddy.

Changelog

2026-05-11 — Stop duplicating Nextcloud's indexing (Phase 1 + 2)

Two big architectural shifts in one session, both aimed at killing work mule was doing that NC already does for the same source files.

Phase 1 — thumbnail proxy (commits 576b0c2, 28738ac). Photos table gains nextcloud_fileid (migration 0018). GET /api/v1/photos/{id}/thumb/{size} proxies NC's /index.php/core/preview keyed by that fileid, authenticated with the owner's encrypted app password (small=240, medium=640, large=1280). Worker now only writes the medium size to /data/thumbs (the vision worker still loads it from disk); small + large no longer touch disk. Disk fallback stays wired for legacy rows and the rare 404 from NC (iPhone JPEGs mis-extensioned as .DNG — verified). Existing 17,591 photos backfilled via backend/scripts/backfill_nextcloud_fileid.py. Tested with real CR2, real DNG, fake DNG, JPEG, HEIC — all green.

Range support for /original (commit 18dce33). Pre-existing bug surfaced by Phase 1 testing: <video> couldn't play .mov because FileResponse was returning 200 with the whole body and no Accept-Ranges header, so browsers reported "format not supported." Now parses Range: bytes=START-END, returns 206 with Content-Range, streams in 1 MB chunks.

Fix: backfill_gps was the actual CPU drain, not the watcher (commit d24c64e). _scan_all_source_roots_async auto-queued backfill_gps 30 s after every container boot, which then re-queued extract_metadata for every photo with latitude IS NULL — ~60k pointless tasks per deploy, pinning worker-light at 180+% CPU for ~30 min. Killed the auto-trigger; manual POST /api/v1/library/backfill-gps still works.

Phase 2 — webhook receiver replaces watchfiles (commits 362fbc6, f657e2c). New endpoint POST /api/v1/internal/nc-webhook (auth via Authorization: Bearer $NEXTCLOUD_WEBHOOK_SECRET, hmac.compare_digest) handles NodeCreated|NodeWritten|NodeDeleted|NodeRenamed. Maps NC's /admin/files/... path to the bind-mounted /nextcloud-users/admin/files/... and dispatches the same scan_folder / handle_file_deletion machinery the watcher used. Registered against NC via backend/scripts/register_nc_webhooks.py (idempotent: deletes existing webhooks targeting the same URL first). mulita-worker-watcher container retired; celery --beat folded into worker-light so discard_missing_photos_beat still fires. End-to-end verified by uploading a test JPEG through WebDAV — both NodeCreated and NodeWritten fire, mule receives + dispatches, 200 OK.

NC cron tightened to */1 so webhook delivery latency drops from up to 5 min to ~60 s (crontab -u www-data on LXC 114). NC dispatches webhook calls through its background-job queue; the cron interval = the worst-case latency. */5 was the default; */1 is the upstream recommendation anyway.

Post-Phase-2 delete-roundtrip patches (commit 9408825). End-to-end testing of the NC↔mule deletion paths surfaced two real gaps:

  1. Folder delete — NC fires one NodeDeletedEvent for the folder, not one per child. The webhook handler bailed with "unsupported extension" and photos under the deleted folder kept is_discarded=false until the 30-min reconcile sweep. Fix: handle_directory_deletion() does a single UPDATE photos SET is_discarded=true WHERE filepath LIKE 'dir/%' when the deleted path has no supported image extension.
  2. Resurrect on rewrite — PUT-overwrite of a previously-discarded file fired NodeWrittenEventscan_folder, but scan_folder's "photo exists, skip" branch left is_discarded=true. Fix: when the existing row is discarded, flip is_discarded=false, clear discarded_at, re-queue extract_metadata.

Verified end-to-end on Photos/MuleTestFolder/{test1,test2,nested/test3}.jpg: DELETE Photos/MuleTestFolder/ discarded all 3 in one shot; PUT test1.jpg back resurrected only that one.

Known remaining gap — trashbin restore. Moving a file out of /dav/trashbin/.../trash/foo.jpg.dXXXX back to /files/... via WebDAV MOVE fires no event mule subscribes to. NC's trashbin app emits its own internal event class that isn't in the OCP\Files\Events\Node\* set we registered with webhook_listeners. Workaround: re-upload via PUT (covered by the resurrect-on-rewrite fix above) or wait for the 30-min reconcile sweep.

Folder rename round-trip (commits f4a03b6 + f27f3cb). Both directions now work:

  • NC-side rename → mule: webhook's "renamed" branch detects directory rename (neither path has a supported extension) and calls new handle_directory_rename(old, new) in scan.py. The helper iterates matching rows in Python and prefix-rewrites Photo.filepath, Folder.path, SourceRoot.path in one transaction. Cross-source-root case (rare) discards the old subtree and lets scan_folder add fresh rows under the new root.
  • Mule-side rename → NC: the existing PATCH /api/v1/folders/{id} endpoint (folders.py:202) already does WebDAV MOVE via nextcloud_dav.move_for_user and rewrites mule's DB inline. The NodeRenamedEvent that bounces back through the webhook hits handle_directory_rename, which finds 0 rows under the old prefix and is a no-op — feedback loop is idempotent.

asyncpg gotcha: the original implementation of handle_directory_rename used raw SQL with SUBSTRING(filepath FROM LENGTH(:old_prefix) + 1) so the offset would be computed server-side. asyncpg's type inference miscategorises the LENGTH() result and rejects the parameter as "$2: int (expected str)". The fix iterates in Python (same pattern as the existing PATCH endpoint). Lesson: avoid passing LENGTH(:x)+1 as an argument to SUBSTRING(...) via asyncpg + sqlalchemy text().

Open: Phase 3. Enable the Nextcloud Memories app (currently installed but disabled) and read taken_at / lat / lon / w / h from oc_memories instead of running mule's own extract_metadata. Keep pHash + file_hash for dedup. When that lands, the vision worker can also stop needing on-disk medium thumbs and the whole /data/thumbs pipeline retires.

2026-05-11 — Duplicates view shows parent folder + full-path tooltip

GET /api/v1/library/duplicates/groups now includes filepath in each member payload. DuplicatesView renders a dark bottom-strip on every thumbnail showing the last two folder segments — …/<grandparent>/<parent> (e.g. …/files/Phone Photos starting 01-26) via a duplicatePathLabel helper. Going up two levels avoids the failure mode where two copies sit under matching parent names (e.g. duplicate 2023/ subfolders under different archives) and the label would collapse. Full filepath surfaces through the native title tooltip on hover. Dimensions chip moves from bottom-left to top-left so the path strip can run edge-to-edge. memberToPhoto finally stops faking filepath = filename — the synthetic Photo it hands to PhotoThumbnail now carries the real path.

Infra side-effect during the deploy: two consecutive --force-recreate cycles (deploy.sh and a manual down/up) raced and left orphan duplicate worker containers (<id>_mulita-worker-light + the named mulita-worker-light, same for vision) both pulling from the same Redis queue. Worker concurrency was effectively doubled, host load climbed past 120, and pct exec into LXC 120 hung for several minutes. Recovery: docker rm -f on both orphan IDs. The deploy-gotcha entry below covers the pattern; the new contribution from this session is "look for renamed <id>_<service> containers and remove them if you ever see load spike + pct hang after a deploy".

2026-05-11 — Drop Title field, add bulk Notes editor

  • Removed the Title (user_title) row from PhotoInfoPanel.tsx: the form field, its titleDraft/setTitleDraft state, the commitTitle handler, and the photo?.user_title watcher in the draft-sync useEffect. Column stays on photos and on the backend model — only the UI affordance went away.
  • New bulk action set_notes in backend/app/routers/photos.py bulk_action: validates the value is a string (or null/empty to clear), then sets user_notes on every photo in the selection in one transaction.
  • Frontend wiring: bulkSetNotes in services/api.ts, bulkNotes mutation in hooks/useBulkPhotoMutations.ts (optimistic patch with empty-string → null collapse, full rollback on error), surfaced in RightSidebar.tsx's multi-photo bulk panel as a 2-row Textarea + Clear + Apply pair below the Tags section. Doesn't auto-fire on every keystroke — clicking Apply is the explicit commit (otherwise every keystroke would PATCH N rows).
  • Shared Photo interface in types/photo.ts now includes user_notes?: string | null so the optimistic-patch typing accepts the field; previously it only existed on PhotoInfoPanel's local PhotoDetails shape.

2026-05-10 — Right sidebar compact rebuild

Right sidebar previously had <ActiveHeapCard /> + <Header /> + a scroll region holding two parallel collapsibles ("Edit" + "Metadata"), with three nested <Section> sub-collapsibles (Basic Info / Camera / Location) inside Metadata. A lot of chrome for a per-photo form.

  • Removed frontend/src/components/heaps/ActiveHeapCard.tsx entirely (no remaining importers; active-heap state stays in the store, Select/Discard buttons still consult it).
  • RightSidebar.tsx single-photo branch also drops its <Header /> strip — the new "METADATA" collapsible trigger inside PhotoInfoPanel is the visible title. Multi-photo branch keeps the Header for "N Photos Selected".
  • PhotoInfoPanel.tsx is now two stacked collapsibles:
    • Metadata (default expanded): readonly fields (Size / Dimensions grid, Path, GPS inlined when present), a thin <hr>, then editable form (Filename, Title, Date Taken, Notes, Tags, Rating + Color on one row, Flag).
    • Camera (default expanded): isolated EXIF block (make+model, lens, 2×2 ISO/Aperture/Shutter/Focal grid).
  • Compact density: Notes rows=3 → 2, stars/swatches h-5 → h-4, space-y-2.5 → 2, Flag buttons text-sm → text-xs, grid gap-2 → gap-x-2 gap-y-1, empty "No GPS data" chip dropped (now hidden when there are no coordinates).
  • Local Section helper deleted from PhotoInfoPanel.tsx (no longer used).

PreviewView reuses RightSidebar inside its overlay aside, so the change applies in both grid and preview.

2026-05-10 — Timeline scroll-anchor on section switch

Clicking a folder in the left sidebar (or any other navigation that changed currentSection — All Photos, Rated, Discarded, etc.) didn't reset the timeline's scroll position. If the user was deep in All Photos and clicked a folder with fewer rows, the new section loaded with the previous scroll offset preserved, often landing on empty space below the last row.

Timeline.tsx already had a section-change effect that cleared selection and reset the auto-focus guard, and a separate auto-focus effect that selects photos[0] once the new query resolves. The latter has an "ensure visible" scrollTo, but it only fires if the cell is out of view AND only after the next render — by then the user has already seen the wrong scroll position. Added parentRef.current.scrollTop = 0 synchronously inside the section-change effect so the first paint of the new section anchors at the top; the auto-focus selectPhoto then highlights photo[0] as before.

2026-05-10 — Filter bar 422-on-date-from

User reported "all filters broken, no photos shown" right after the metadata-extraction overhaul shipped. Tracing actual API traffic on a fresh session showed GET /api/v1/photos?per_page=100&date_from=2026-04-10&sort=taken_at&order=desc returning 422 Unprocessable Entity: pydantic v2's datetime parser rejects bare-date strings ("2026-04-10") for Optional[datetime] query params.

The frontend's filtersToParams in store/filterStore.ts had been padding date_to with T23:59:59 for inclusive end-of-day, but date_from went out as a bare YYYY-MM-DD — so every date-range filter request 422'd, and TanStack Query's failure-state shows an empty grid. From the user's perspective it looked like "filters return nothing across the board"; from the backend it was a single endpoint signature problem.

Fix on both sides:

  • Frontend: pad date_from with T00:00:00 symmetric to date_to's T23:59:59.
  • Backend: switch date_from / date_to to Optional[str] and parse with datetime.fromisoformat inside the handler. fromisoformat accepts both bare dates (→ midnight) and full ISO strings, so any older client / curl that sends a date-only value still works. Tz-aware values get coerced to naive UTC, matching the taken_at column shape and the same fix applied to PATCH /photos/{id} earlier in the session. Bad input now returns 400 with a clear message instead of pydantic's 422.

2026-05-10 — Date extraction overhaul

User reported wrong "Date Taken" on stills (JPEG / HEIC / DNG). Tracing the pipeline showed the trusted-EXIF list at services/metadata.py:230-244 ended in EXIF:ModifyDate, which is set every time a file is re-saved (Lightroom export, batch resize, EXIF strip), so any photo that lost its original capture metadata during editing was being labeled taken_at_source='exif' with the edit timestamp.

Pipeline rewrite:

  • New trusted-EXIF priority: SubSecDateTimeOriginalDateTimeOriginalCreateDateQuickTime:MediaCreateDateQuickTime:CreateDate. ModifyDate removed entirely.
  • Fall back to guess_date_from_path() (the same heuristic that already powers has_date_warning) when no trusted EXIF date is found. New taken_at_source='path' value with a "PATH" badge in the info panel; tooltip explains the date came from filename / folder rather than real EXIF.
  • Skip the date-write block entirely when photo.taken_at_source == 'manual' so a rescan can't clobber a user correction. (Previous behavior overwrote manual edits.)
  • parse_exif_datetime accepts tz-aware variants (%z), normalizes to naive UTC, and rejects the 0000:00:00 placeholder.

Backfill: new backfill_taken_at celery task + POST /api/v1/library/maintenance/backfill-taken-at endpoint. Re-enqueues extract_metadata for every non-manual, non-trashed photo so the new rules apply across the existing library. Snapshot before the sweep started: 11,223 exif + 6,039 filesystem + 10 path. Will reshape over the next ~45 min.

Side note: the default celery queue had ~209k pending tasks at the time we fired the backfill — the watcher's 5-minute restart loop (since fixed) had been re-enqueuing scans, and tasks for the 4,158 photos we hard-deleted earlier today were still sitting around. Most fail fast (Photo not found, ~24ms each); real work runs at ~100ms. Decided to let it drain instead of flushing — safer.

2026-05-10 — Two cascade-delete + datetime fixes after the hard-remove shipped

The first cut of delete_nextcloud_source_root blew up with folders_parent_id_fkey violations on the first real test (Taco and Muli - 2024 onward, 35 folders / 4,158 photos). Two iterations to get it right:

  1. NULL parent_id before deleting Folders. Folders have a self-referential parent_id FK with no ON DELETE rule; postgres checks the constraint per row regardless of insertion order, so a flat DELETE FROM folders WHERE id IN (...) of the whole subtree fails on the parents whose children appear later in the same statement. Fixed with an UPDATE folders SET parent_id = NULL WHERE id IN (folder_ids) first.
  2. Widen the NULL UPDATE to cross-source-root children. A "Leóns 1st Year" SourceRoot at .../Taco and Muli - 2024 onward/Leóns 1st Year had its own folder rows whose parent_id pointed into the Taco SourceRoot's hierarchy. The first patch only NULLed parent_id for folders whose id was in the delete set; the Leóns folders weren't in that set so they kept their references and the FK still tripped. Fix: UPDATE folders SET parent_id = NULL WHERE parent_id IN (folder_ids) — kills any incoming reference into the delete set, internal or external.

After both fixes: DELETE /api/v1/nextcloud/source-roots/{id} for Taco and Muli - 2024 onward cleared 4,158 photos and 35 folders in a single request and returned 200. UI swaps the trash icon for a Loader2 spinner while the request is in flight (removeRoot.isPending && removeRoot.variables === r.id) so the multi-second cascade is visible.

Separate fix in the same session: PATCH /api/v1/photos/{id} returned 500 with can't subtract offset-naive and offset-aware datetimes when the frontend sent a tz-aware ISO string for taken_at (the datetime-local input is supposed to be naive but real-world locales / paste flows occasionally include +02:00). The DB column is timestamp without time zone, so asyncpg refused to bind. Normalize on the server with astimezone(timezone.utc).replace(tzinfo=None) if tzinfo is not None.

Deploy infra learning: the new --force-recreate in deploy.sh does NOT reliably recreate containers on every push — saw two consecutive deploys leave the backend at the previous StartedAt despite a fresh image. Manual docker compose up -d --no-deps --force-recreate <svc> after deploy still races with whatever compose state the auto-deploy left mid-flight, twice landing the stack in a half-broken state (orphaned <id>_mulita-backend rename containers, db / redis stopped). Recovery: docker compose down && docker compose up -d. Open question — --force-recreate may need to be replaced with something more explicit.

2026-05-10 — Hard-remove Nextcloud SourceRoot + reliable delete sync

DELETE /api/v1/nextcloud/source-roots/{id} was a soft-deactivate (is_active=false) — the trash icon in Settings only hid the SourceRoot from active queries while leaving every Folder + Photo row in the DB forever. Re-adding the same path resurrected ghosts; prune-missing reported zero deletes for the soft-removed entry because the cleanup code skipped inactive source roots.

Endpoint now hard-deletes the SourceRoot, every Folder under it (chunked at 500), and every Photo in those folders. photo_tags and heap_photos cascade automatically via ON DELETE CASCADE on the join tables. FolderShare uses a stringly-typed folder_id (no FK) so cleaned manually for both folder_type='folder' and folder_type='source_root'. Returns {deleted_photos, deleted_folders} so the UI can toast a count. Files in Nextcloud are untouched.

Sync side: the watch_folders celery task in backend/app/tasks/scan.py already detected filesystem deletions and soft-marked photos is_discarded=true, but the global task_soft_time_limit=300 in app/tasks/celery.py was killing the watcher every five minutes and dropping every FS event during the restart window. The soft_time_limit=None on the decorator was being interpreted as "use worker default" rather than "unlimited". Override to soft_time_limit=0, time_limit=0 (Celery convention for unbounded). Backstop: a new discard_missing_photos() in app/services/cleanup.py — soft variant of prune_missing_photos, walks every _sr_state == 'present' source root and flips is_discarded=true on Photo rows whose file is gone. Wired as discard_missing_photos_beat celery task scheduled every 30 minutes via beat_schedule on the celery app. Beat runs in-process on worker-watcher (--beat flag in docker-compose.yml) — the watcher is already a Redis-locked singleton so no need for a separate beat container.

Manual POST /api/v1/library/maintenance/prune-missing remains the hard-delete path for when the user wants to permanently drop orphan rows; the new beat job only soft-discards (file gone → mule-image trash, restorable).

2026-05-10 — OIDC auto-redirect on LoginPage

OIDC_ENABLED=true was already set in .env, so the LoginPage rendered a "Sign in with Authentik" button next to the password form. With a single trusted IdP and a logged-in Authentik session, that extra click was friction without upside.

LoginPage.tsx now reads /auth/config on mount and, if OIDC is on, immediately navigates to the OIDC login URL. Authentik recognizes the existing session and bounces back through the callback with no user interaction. Two escape hatches: ?password=1 in the URL forces the password form, and a skipAutoSso sessionStorage flag (set by AuthContext.logout and by the OIDC callback's error branch) suppresses the next auto-redirect so logouts actually log out and OIDC failures surface their error instead of looping. While the redirect is in flight the page shows "Signing in with Authentik..." plus a "Use password instead" link.

2026-05-10 — right sidebar restructure (heap pinned, single scroll, collapsible Metadata)

The right sidepanel had three stacked flex regions: <ActiveHeapCard />, <Header />, and <PhotoInfoPanel /> — with PhotoInfoPanel carrying its own internal scroll. That left the editable fields (filename, title, notes, rating, color, flag) stuck above the readonly metadata scroll, effectively two scroll boundaries on one sidebar.

  • Moved the scroll boundary up to RightSidebar.tsx: only ActiveHeapCard and Header stay pinned now. Editable + readonly content scroll together in one flex-1 overflow-y-auto region beneath them.
  • PhotoInfoPanel.tsx dropped its h-full / inner flex-1 overflow-y-auto.
  • The four readonly sections (Tags / Basic Info / Camera / Location) are now wrapped in a single outer "Metadata" Collapsible. Default expanded, one click hides the whole block. Sub-sections stay individually collapsible.
  • Second pass: the editable form (filename / title / notes / rating / color / flag) got the same treatment under an outer "Edit" Collapsible so the panel is now two equal collapsible groups below the title strip. Dropped the X (clear-selection) button from the Header; Esc and grid-empty-area-click still clear.
  • Third pass: split editable vs read-only between the two groups consistently. TagsEditor and TakenAtEditor had been buried inside the readonly Metadata sub-sections — Tags as its own Section, taken-at wedged into Basic Info between size/dims and the filepath. Moved both into the Edit collapsible, ordered identification → description → categorization: Filename · Title · Date Taken · Notes · Tags · Rating · Color · Flag. Metadata now holds only readonly: Basic Info (size, dims, path), Camera, Location.

PreviewView reuses RightSidebar under an <aside class="overflow-hidden">, so the change applies in both the grid sidebar and the preview overlay.

2026-05-10 — photos.hubris.network perf sweep

User reported slow load. Five fixes shipped, in order:

  1. Backend out of dev mode. docker-compose.yml command: was running uvicorn … --reload (single worker, file-watcher). Page loads fan out ~15 parallel API calls; they all serialized on one event loop. → --workers 2 --proxy-headers. Two uvicorn worker procs now.
  2. iPhone Apple ProRAW / Linear DNG decode fixed. rawpy 0.26.1 + LibRaw 0.22 rejects Apple Linear DNG (Photometric Interpretation: Linear Raw, 8-bit) as "Unsupported file format or not RAW file". Those files are TIFF containers with developed RGB inside and have no embedded preview to fall back to. Added PIL Image.open(src_path) as the last fallback in both _generate_proxy_webp (routers/photos.py) and process_raw_image (tasks/thumbs.py). ~1,300 iPhone DNGs were 415-ing on every detail view; now decode in <1s via PIL.
  3. Reconcile DB with disk (renamed-folder case). prune_missing_photos was skipping all 4,154 orphaned photo rows under SourceRoot Taco and Muli - 2024 onward (renamed in Nextcloud to Photo Archive 2004-2024) because the leaf path didn't resolve and the code assumed "drive unmounted → must skip". Added _sr_state() to classify as present / renamed (parent mount fine, leaf gone) / unmounted (parent inaccessible). Only unmounted still skips. Two stale source roots logged with a clearer hint pointing at POST /api/v1/library/maintenance/prune-missing. User has not yet been asked to run that — endpoint is ready when they are.
  4. Frontend page size + idle polling. usePhotosQuery.ts was first-fetching per_page=500 (slow paint + 500 thumb requests at once). Split into PER_PAGE_INITIAL=100 for first paint, PER_PAGE_BACKGROUND=500 for the cursor-chain prefetch. Idle polling for scan-status and worker-status (useScanActivity.ts, ScanProgress.tsx) bumped from 10s/15s to 30s/30s while idle; active cadence (2s/3s) unchanged.
  5. Partial index on photos. Default list query WHERE NOT is_trashed AND NOT is_hidden ORDER BY taken_at DESC NULLS LAST, id DESC LIMIT N was doing a seq-scan + top-N heapsort (~25ms standalone, worse under concurrency). Added migration 0017_photos_list_index creating ix_photos_list_visible (partial index on the sort key, restricted to visible rows). EXPLAIN now shows an Index Only Scan → 24.7ms → 0.097ms, ~250× speedup.

Deploy gotcha — fixed (with caveat). The original docker compose up -d --build in deploy.sh did not reliably recreate containers when only runtime config (command:, env-only) or migration files changed; image hash would change but compose treated the existing container as "current enough". Bit three times this session before /opt/mule-deploy/deploy.sh was updated to docker compose up -d --build --force-recreate. Trade-off accepted: an extra restart cycle on deploys where nothing user-visible changed.

Caveat: the first auto-deploy after the flag flip raced with my own earlier manual docker compose up -d --no-deps --force-recreate frontend and landed the stack in a half-broken state — mulita-frontend got stuck under a renamed temp container, several services dropped off mule-image_mulita-network, frontend nginx restarted in a loop with host not found in upstream "backend". Fixed by docker compose down && docker compose up -d. Don't issue a manual --force-recreate on a single service while the auto-deploy webhook is also expected to fire — let the deploy own the lifecycle.

Data drift still outstanding. 4,154 photo rows + 1 unregistered folder (Photo Archive 2004-2024) on disk that's not a SourceRoot. The reconcile endpoints now work — user decides when to call them. The new folder needs to be added as a SourceRoot via the Settings UI before its files will be indexed.

Proxy cache still empty (mule-image_proxies_data volume is 4 KB). Pre-generating ~5001,500 WebP proxies for non-web-safe formats would make first-open of every RAW/HEIC photo instant. Deferred — needs a one-shot script and the disk-space tradeoff isn't worth it until the data-drift reconcile happens first.

2026-04-28 — wiki entry created

Initial documentation.

2026-04-26 — Nextcloud-rooted libraries shipped

Bind /mnt/library/homecloud into the workers, reads via filesystem, writes via WebDAV. users.nextcloud_username override field added; dtoro → admin mapping. Surviving SourceRoots cleaned up to NC paths.

2026-04-22 — native OIDC via Authentik

Authlib-based code in backend/app/auth_oidc.py. extra_hosts override for auth.hubris.network in compose override (gitignored).

2026-04-21 — auto-deploy pipeline shipped

Webhook receiver at :9797, async deploy returning 202. Mirrors caddy-conf / gitea-customizations.