feat(nextcloud): hard-delete SourceRoot + reliable delete sync

Two related fixes for the Nextcloud library lifecycle.

1. DELETE /api/v1/nextcloud/source-roots/{id} now actually deletes
   the SourceRoot, every Folder under it, and every Photo in those
   folders (Nextcloud files untouched). Was a soft-deactivate
   (is_active=false) that left the rows around forever, so re-adding
   the same path resurrected ghosts and prune-missing reported zero.
   Returns {deleted_photos, deleted_folders}; the Settings UI toasts
   the count and invalidates photos/folders/stats so cached lists
   don't show ghosts. photo_tags and heap_photos already cascade via
   ON DELETE CASCADE; FolderShare uses a stringly-typed folder_id
   with no FK so cleaned up explicitly.

2. The watcher (watch_folders task) was getting killed every five
   minutes by the global task_soft_time_limit=300 in app/tasks/celery.py
   despite passing soft_time_limit=None on the decorator (None falls
   back to the worker default in this Celery version). Override with
   soft_time_limit=0, time_limit=0 (= unlimited) so the watch loop
   actually stays alive. The 'Soft time limit (300s) exceeded' /
   'Worker exited prematurely' lines should stop in worker-watcher
   logs.

3. Added discard_missing_photos() in services/cleanup.py — a soft
   variant of prune_missing_photos that walks every present source
   root, checks os.path.exists for each non-discarded Photo, and
   flips is_discarded=true on the missing ones (UPDATE not DELETE).
   Wired as discard_missing_photos_beat in tasks/scan.py and
   scheduled every 30 min via celery beat. Beat runs in-process on
   worker-watcher (--beat flag in compose) — there's only ever one
   watcher and we don't need a separate container.

Hard delete remains manual via prune-missing for users who want to
review before committing. The beat catch-up only soft-discards (file
gone -> mule-image trash, restorable).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claudio
2026-05-10 22:19:58 +02:00
parent 99d504842e
commit 09a00f7419
7 changed files with 174 additions and 17 deletions

View File

@@ -474,7 +474,7 @@ WATCHER_LOCK_KEY = "mulita:watch_folders:lock"
WATCHER_LOCK_TTL = 60 # 1 min — renewed every event batch via wall-clock check
@shared_task(name='watch_folders', bind=True, soft_time_limit=None, time_limit=None)
@shared_task(name='watch_folders', bind=True, soft_time_limit=0, time_limit=0)
def watch_folders(self):
"""
Watch folders for changes using watchfiles. Long-running task that
@@ -615,4 +615,18 @@ async def _backfill_gps_async():
extract_metadata.delay(pid)
logger.info(f"backfill_gps: queued extract_metadata for {len(photo_ids)} photos")
return {'queued': len(photo_ids)}
return {'queued': len(photo_ids)}
@shared_task(name='discard_missing_photos_beat')
def discard_missing_photos_beat():
"""Periodic catch-up for filesystem deletions the watcher missed
(e.g. while the worker was restarting). Walks every active source
root that is currently mounted and present, and soft-discards any
Photo whose file is gone. Hard-deletion stays manual via
POST /api/v1/library/maintenance/prune-missing.
Wired to a 30-minute beat schedule in app/tasks/celery.py.
"""
from app.services.cleanup import discard_missing_photos
return asyncio.run(discard_missing_photos())