feat(nextcloud): hard-delete SourceRoot + reliable delete sync

Two related fixes for the Nextcloud library lifecycle.

1. DELETE /api/v1/nextcloud/source-roots/{id} now actually deletes
   the SourceRoot, every Folder under it, and every Photo in those
   folders (Nextcloud files untouched). Was a soft-deactivate
   (is_active=false) that left the rows around forever, so re-adding
   the same path resurrected ghosts and prune-missing reported zero.
   Returns {deleted_photos, deleted_folders}; the Settings UI toasts
   the count and invalidates photos/folders/stats so cached lists
   don't show ghosts. photo_tags and heap_photos already cascade via
   ON DELETE CASCADE; FolderShare uses a stringly-typed folder_id
   with no FK so cleaned up explicitly.

2. The watcher (watch_folders task) was getting killed every five
   minutes by the global task_soft_time_limit=300 in app/tasks/celery.py
   despite passing soft_time_limit=None on the decorator (None falls
   back to the worker default in this Celery version). Override with
   soft_time_limit=0, time_limit=0 (= unlimited) so the watch loop
   actually stays alive. The 'Soft time limit (300s) exceeded' /
   'Worker exited prematurely' lines should stop in worker-watcher
   logs.

3. Added discard_missing_photos() in services/cleanup.py — a soft
   variant of prune_missing_photos that walks every present source
   root, checks os.path.exists for each non-discarded Photo, and
   flips is_discarded=true on the missing ones (UPDATE not DELETE).
   Wired as discard_missing_photos_beat in tasks/scan.py and
   scheduled every 30 min via celery beat. Beat runs in-process on
   worker-watcher (--beat flag in compose) — there's only ever one
   watcher and we don't need a separate container.

Hard delete remains manual via prune-missing for users who want to
review before committing. The beat catch-up only soft-discards (file
gone -> mule-image trash, restorable).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claudio
2026-05-10 22:19:58 +02:00
parent 99d504842e
commit 09a00f7419
7 changed files with 174 additions and 17 deletions

View File

@@ -299,6 +299,71 @@ async def prune_missing_photos(dry_run: bool = True) -> dict:
raise
async def discard_missing_photos() -> dict:
"""Soft variant of prune_missing_photos for the periodic beat
catch-up. Walks every active source root that is currently
`present` (not 'renamed' — the user-driven manual flow handles
those — and not 'unmounted'), and for each Photo whose file is
gone from disk, sets is_discarded=True so it shows up in the
in-app trash. Idempotent: skips photos that are already
discarded.
Hard-deletion stays manual via prune_missing_photos so users
can review the list before committing.
"""
async with AsyncSessionLocal() as session:
try:
sr_rows = (await session.execute(select(SourceRoot))).scalars().all()
present_sr_ids = {
sr.id for sr in sr_rows if _sr_state(sr.path) == 'present'
}
if not present_sr_ids:
return {"discarded": 0, "checked": 0}
folder_to_sr = {
fid: srid
for fid, _path, srid in (await session.execute(
select(Folder.id, Folder.path, Folder.source_root_id)
)).all()
}
photo_rows = (await session.execute(
select(Photo.id, Photo.filepath, Photo.folder_id)
.where(Photo.is_discarded.is_(False))
)).all()
missing_ids: list[str] = []
checked = 0
for pid, fp, folder_id in photo_rows:
sr_id = folder_to_sr.get(folder_id)
if sr_id not in present_sr_ids:
continue
checked += 1
if not os.path.exists(fp):
missing_ids.append(pid)
if missing_ids:
CHUNK = 500
now = datetime.utcnow()
for i in range(0, len(missing_ids), CHUNK):
await session.execute(
update(Photo)
.where(Photo.id.in_(missing_ids[i:i + CHUNK]))
.values(is_discarded=True, discarded_at=now)
)
await session.commit()
logger.info(
f"discard_missing_photos: discarded {len(missing_ids)} "
f"of {checked} photos under {len(present_sr_ids)} present source roots"
)
return {"discarded": len(missing_ids), "checked": checked}
except Exception as e:
logger.error(f"discard_missing_photos failed: {e}")
await session.rollback()
raise
async def cleanup_data_integrity() -> dict:
"""Top-level entry point. Runs the dedupe + count refresh in a single
transaction. Returns a small summary dict for logging."""