feat: retire the watchfiles watcher in favour of NC webhooks

End-to-end webhook flow is proven on this NC instance (NodeCreated +
NodeWritten both fired and dispatched scan_folder on a PUT test), so
the watchfiles-based polling layer is no longer needed.

- scanner.start_initial_scan no longer queues watch_folders on boot.
- scan.watch_folders kept as a one-line no-op shim so any leftover
  apply_async in flight from the previous deploy doesn't crash a
  worker. Will be deleted entirely after the queue drains.
- celery.py reroutes watch_folders to the `default` queue (worker-light)
  so the no-op shim actually completes — the `watcher` queue is dead.
- docker-compose drops the mulita-worker-watcher service. Its celery
  --beat responsibility (firing discard_missing_photos_beat every 30
  min) moves to worker-light's command.

Latency note: NC dispatches webhook events through its background-job
queue, currently run by cron */5. After this commit lands you'll want
to tighten cron to */1 so new uploads land in mule within ~60s instead
of up to 5 min.
This commit is contained in:
Claudio
2026-05-11 12:28:36 +02:00
parent 362fbc6d83
commit f657e2c0ba
4 changed files with 31 additions and 157 deletions

View File

@@ -52,8 +52,11 @@ celery_app.conf.update(
'scan_folder': {'queue': 'low'},
'scan_all_source_roots': {'queue': 'low'},
'backfill_gps': {'queue': 'low'},
# Dedicated watcher queue
'watch_folders': {'queue': 'watcher'},
# `watch_folders` is retired (file events come from NC webhooks)
# but the task definition still exists as a no-op shim for any
# in-flight apply_async. Route it to the default queue so the
# remaining worker actually drains it.
'watch_folders': {'queue': 'default'},
'discard_missing_photos_beat': {'queue': 'low'},
},
task_default_queue='default',