feat: add vision pipeline scaffolding with ONNX backend

Introduce the app/services/vision/ module with ABC interfaces, ONNX
Runtime backend, model registry, and per-task implementations:
- OpenCLIP ViT-B/32 embedder (image + text, 512-d)
- RapidOCR engine (PP-OCRv4 via ONNX, no PaddlePaddle)
- YOLOv8n object detector (raw ONNX, no ultralytics runtime)
- YuNet + SFace face processor (Apache 2.0, opencv_zoo, 128-d)
- DBSCAN face clustering helper

Add VisionSettings to config (mulita.yml + Pydantic), bootstrap_models.py
for first-boot weight downloads, models_data Docker volume, and ROCm
backend stub for future GPU acceleration.

No Celery tasks wired yet — models load but nothing invokes them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-10 09:00:06 +02:00
parent dea04ceed9
commit 9282a5c734
16 changed files with 856 additions and 2 deletions

View File

@@ -29,6 +29,42 @@ class PerformanceSettings(BaseModel):
db_pool_size: int = 20
db_pool_recycle: int = 3600
class EmbedderSettings(BaseModel):
"""CLIP / SigLIP embedding model settings"""
name: str = "openclip_vitb32"
batch_size: int = 8
class OCRSettings(BaseModel):
"""PaddleOCR / rapidocr settings"""
enabled: bool = True
languages: list[str] = ["en"]
min_confidence: float = 0.5
class DetectorSettings(BaseModel):
"""YOLOv8n object detection settings"""
enabled: bool = True
min_confidence: float = 0.35
max_detections: int = 50
class FacesSettings(BaseModel):
"""YuNet + SFace face detection/recognition settings"""
enabled: bool = True
min_face_size: int = 40
recognition_threshold: float = 0.4
cluster_eps: float = 0.35
class VisionSettings(BaseModel):
"""AI vision pipeline settings. Disabled when running on SQLite
(pgvector is required for embedding storage)."""
enabled: bool = True
backend: str = "onnx" # "onnx" | "rocm" (future)
models_dir: str = "/data/models"
embedder: EmbedderSettings = EmbedderSettings()
ocr: OCRSettings = OCRSettings()
detector: DetectorSettings = DetectorSettings()
faces: FacesSettings = FacesSettings()
worker_concurrency: int = 2
class MulitaConfig(BaseModel):
"""Main configuration from YAML file. Source roots and the discard
workflow are owned by the database now — only operational settings
@@ -36,6 +72,7 @@ class MulitaConfig(BaseModel):
thumbnails: ThumbnailSettings = ThumbnailSettings()
scanner: ScannerSettings = ScannerSettings()
performance: PerformanceSettings = PerformanceSettings()
vision: VisionSettings = VisionSettings()
class Settings(BaseSettings):
"""Application settings"""
@@ -127,6 +164,10 @@ class Settings(BaseSettings):
def performance(self) -> PerformanceSettings:
return self.config.performance
@property
def vision(self) -> VisionSettings:
return self.config.vision
class Config:
env_file = ".env"
case_sensitive = False