fix: harden pipeline — retries, acks_late, time limits, session safety

Addresses 16 robustness, transparency, and performance issues across
the Celery media processing pipeline:

Critical:
- Singleton DB engine in vision tasks (was leaking one per task call)
- acks_late + task_reject_on_worker_lost so crashed workers don't lose tasks
- Global soft/hard time limits (5/10 min) to prevent hung worker slots
- Thumbnail copy-before-resize (in-place mutation degraded larger sizes)
- backfill_vision now checks each task type independently (OCR, faces, etc.)
- Parameterized LIMIT in backfill_vision (was f-string SQL injection)

High:
- try/except + retry(max=3) on all vision inference tasks
- extract_metadata writes processing_error on exiftool failure
- PIL Image handles closed in _load_thumb/_load_original
- Scan progress Redis keys auto-expire after 1 hour
- Watcher lock renewal is wall-clock based (30s) not event-count based
- worker_process_init signal warms up vision models on startup

Medium:
- Explicit task_routes for every task name (wildcards never matched)
- app.services.metadata added to Celery include list
- POST /maintenance/recover-stuck endpoint for photos stuck in processing
- Docker healthchecks for worker-light, worker-vision, and Redis
- Task ID in vision log lines for distributed tracing
- Bare except:pass narrowed to specific exceptions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

backend/app/tasks/thumbs.py

@@ -218,23 +218,28 @@ def auto_rotate_image(image: Image.Image) -> Image.Image:
             
             if orientation in rotation_map:
                 image = image.rotate(rotation_map[orientation], expand=True)
-    except:
+    except (AttributeError, KeyError, TypeError):
         pass  # No orientation data available
     
     return image
 
 def generate_thumbnail(image: Image.Image, size: int, output_path: str):
-    """Generate a thumbnail of the specified size"""
-    # Maintain aspect ratio
-    image.thumbnail((size, size), Image.Resampling.LANCZOS)
-    
-    # Save as WebP with specified quality
-    image.save(
+    """Generate a thumbnail of the specified size.
+
+    Works on a copy so the caller's image is never mutated — this is
+    critical because the thumbnail loop iterates multiple sizes and
+    in-place shrinking would degrade later (larger) sizes.
+    """
+    img = image.copy()
+    img.thumbnail((size, size), Image.Resampling.LANCZOS)
+
+    img.save(
         output_path,
         'WEBP',
         quality=settings.thumbnails.quality,
         method=4  # Balance between speed and compression
     )
+    img.close()
 
 @shared_task(bind=True, name='generate_thumbnails')
 def generate_thumbnails(self, photo_id: str):
@@ -325,11 +330,11 @@ async def _generate_thumbnails_async(photo_id: str, task):
             photo.processing_status = 'completed'
             photo.processing_error = None
             await session.commit()
-            
+
             logger.info(f"Thumbnails generated for photo {photo_id}")
 
-            # Dispatch vision pipeline (embedding, OCR, detection, faces)
-            # after thumbs are ready so vision tasks have images to read.
+            # Dispatch vision pipeline only after thumbnails succeeded —
+            # vision tasks need the generated thumbnails to run inference.
             try:
                 from app.tasks.vision import vision_fanout
                 vision_fanout.delay(photo_id)