feat: auto-start file watcher with Redis lock for live import
Re-enable the watchfiles-based folder watcher with a Redis lock to prevent multiple instances from stacking up across restarts. The watcher is now automatically dispatched on startup when scanner.watch is true (default), and only one instance runs at a time. - Redis lock (SETNX + TTL renewal) ensures single-instance execution - Graceful exit if another watcher holds the lock - New POST /maintenance/start-watcher endpoint for manual control - Fix: use settings.scanner/vision properties instead of mulita_config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -78,17 +78,26 @@ async def bootstrap_default_source_root() -> None:
|
||||
|
||||
|
||||
async def start_initial_scan():
|
||||
"""Start the initial library scan.
|
||||
"""Start the initial library scan and optionally the file watcher.
|
||||
|
||||
NOTE: the folder watcher (watch_folders task) is intentionally NOT
|
||||
dispatched here. It's an infinite loop celery task and every backend
|
||||
restart was queuing a new instance, eventually pinning every worker
|
||||
and starving scan_folder dispatches. Re-enabling it needs a Redis
|
||||
lock or a dedicated long-running container — until then the user
|
||||
triggers scans manually via "Scan all folders".
|
||||
The file watcher uses a Redis lock to ensure only one instance runs
|
||||
across all workers, so it's safe to dispatch on every startup — only
|
||||
the first one will actually watch, the rest exit immediately.
|
||||
"""
|
||||
try:
|
||||
scan_all_source_roots.delay()
|
||||
logger.info("Initial scan queued successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start initial scan: {e}")
|
||||
|
||||
# Start the file watcher if enabled in config.
|
||||
from app.config import settings
|
||||
if settings.scanner.watch:
|
||||
try:
|
||||
from app.tasks.scan import watch_folders
|
||||
# Countdown gives the initial scan time to register source roots
|
||||
# before the watcher tries to load them.
|
||||
watch_folders.apply_async(countdown=10)
|
||||
logger.info("File watcher queued (Redis-locked, single instance)")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not queue file watcher: {e}")
|
||||
|
||||
Reference in New Issue
Block a user