feat: retire the watchfiles watcher in favour of NC webhooks

End-to-end webhook flow is proven on this NC instance (NodeCreated +
NodeWritten both fired and dispatched scan_folder on a PUT test), so
the watchfiles-based polling layer is no longer needed.

- scanner.start_initial_scan no longer queues watch_folders on boot.
- scan.watch_folders kept as a one-line no-op shim so any leftover
  apply_async in flight from the previous deploy doesn't crash a
  worker. Will be deleted entirely after the queue drains.
- celery.py reroutes watch_folders to the `default` queue (worker-light)
  so the no-op shim actually completes — the `watcher` queue is dead.
- docker-compose drops the mulita-worker-watcher service. Its celery
  --beat responsibility (firing discard_missing_photos_beat every 30
  min) moves to worker-light's command.

Latency note: NC dispatches webhook events through its background-job
queue, currently run by cron */5. After this commit lands you'll want
to tighten cron to */1 so new uploads land in mule within ~60s instead
of up to 5 min.
This commit is contained in:
Claudio
2026-05-11 12:28:36 +02:00
parent 362fbc6d83
commit f657e2c0ba
4 changed files with 31 additions and 157 deletions

View File

@@ -78,26 +78,16 @@ async def bootstrap_default_source_root() -> None:
async def start_initial_scan():
"""Start the initial library scan and optionally the file watcher.
"""Start the initial library scan.
The file watcher uses a Redis lock to ensure only one instance runs
across all workers, so it's safe to dispatch on every startup — only
the first one will actually watch, the rest exit immediately.
The watchfiles-based watcher has been retired in favour of Nextcloud
`webhook_listeners` (see `app.routers.nc_webhook`). NC POSTs every
file event directly to mule, so we no longer keep a long-running
inotify task. The periodic `discard_missing_photos_beat` Celery job
is still there as a safety net for deletions a webhook might miss.
"""
try:
scan_all_source_roots.delay()
logger.info("Initial scan queued successfully")
except Exception as e:
logger.error(f"Failed to start initial scan: {e}")
# Start the file watcher if enabled in config.
from app.config import settings
if settings.scanner.watch:
try:
from app.tasks.scan import watch_folders
# Countdown gives the initial scan time to register source roots
# before the watcher tries to load them.
watch_folders.apply_async(countdown=10)
logger.info("File watcher queued (Redis-locked, single instance)")
except Exception as e:
logger.warning(f"Could not queue file watcher: {e}")