feat: auto-start file watcher with Redis lock for live import

Re-enable the watchfiles-based folder watcher with a Redis lock to
prevent multiple instances from stacking up across restarts. The
watcher is now automatically dispatched on startup when scanner.watch
is true (default), and only one instance runs at a time.

- Redis lock (SETNX + TTL renewal) ensures single-instance execution
- Graceful exit if another watcher holds the lock
- New POST /maintenance/start-watcher endpoint for manual control
- Fix: use settings.scanner/vision properties instead of mulita_config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-12 22:24:38 +02:00
parent fc8dd370c2
commit d693569f59
4 changed files with 112 additions and 68 deletions

View File

@@ -78,17 +78,26 @@ async def bootstrap_default_source_root() -> None:
async def start_initial_scan():
"""Start the initial library scan.
"""Start the initial library scan and optionally the file watcher.
NOTE: the folder watcher (watch_folders task) is intentionally NOT
dispatched here. It's an infinite loop celery task and every backend
restart was queuing a new instance, eventually pinning every worker
and starving scan_folder dispatches. Re-enabling it needs a Redis
lock or a dedicated long-running container — until then the user
triggers scans manually via "Scan all folders".
The file watcher uses a Redis lock to ensure only one instance runs
across all workers, so it's safe to dispatch on every startup — only
the first one will actually watch, the rest exit immediately.
"""
try:
scan_all_source_roots.delay()
logger.info("Initial scan queued successfully")
except Exception as e:
logger.error(f"Failed to start initial scan: {e}")
# Start the file watcher if enabled in config.
from app.config import settings
if settings.scanner.watch:
try:
from app.tasks.scan import watch_folders
# Countdown gives the initial scan time to register source roots
# before the watcher tries to load them.
watch_folders.apply_async(countdown=10)
logger.info("File watcher queued (Redis-locked, single instance)")
except Exception as e:
logger.warning(f"Could not queue file watcher: {e}")