dtoro
fa9b21856f
Face detection/recognition: - Replace YuNet + SFace with InsightFace buffalo_l (RetinaFace + ArcFace) - 512-d ArcFace embeddings (was 128-d SFace), migration 0006 resizes column - Remove YOLO person-bbox workaround — RetinaFace is accurate enough - Detection threshold 0.65 cleanly separates real faces (0.72+) from false positives on dogs/paintings (0.56-0.61) Content-type classification: - CLIP zero-shot classifier using native PyTorch text encoder + ONNX image encoder for high-quality text-image similarity - Categories: photograph, screenshot, document, receipt, meme, artwork - Writes Tag(kind=content_type) per photo via photo_tags - Margin-based confidence: top-1 vs top-2 score difference - New ClassifierSettings in config (enabled, min_confidence) - Wired into vision_fanout pipeline Tested: 6 real faces from 4 photos (zero false positives), 11/13 photos classified (8 photograph, 2 artwork, 1 meme). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
42 lines
1.5 KiB
Python
42 lines
1.5 KiB
Python
"""
|
|
ONNX Runtime backend — default CPU inference for all vision models.
|
|
|
|
Each create_* method returns a concrete implementation of the
|
|
corresponding ABC from base.py. Models are loaded from ONNX files
|
|
under settings.vision.models_dir, downloaded on first boot by
|
|
bootstrap_models.py.
|
|
"""
|
|
import logging
|
|
|
|
from app.config import VisionSettings
|
|
from app.services.vision.base import Embedder, OCREngine, ObjectDetector, FaceProcessor, ContentClassifier
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
class ONNXBackend:
|
|
"""Factory for ONNX Runtime-based vision model instances."""
|
|
|
|
def __init__(self, vision_settings: VisionSettings):
|
|
self._settings = vision_settings
|
|
|
|
def create_embedder(self) -> Embedder:
|
|
from app.services.vision.embed import OpenCLIPEmbedder
|
|
return OpenCLIPEmbedder(self._settings)
|
|
|
|
def create_ocr(self) -> OCREngine:
|
|
from app.services.vision.ocr import RapidOCREngine
|
|
return RapidOCREngine(self._settings)
|
|
|
|
def create_detector(self) -> ObjectDetector:
|
|
from app.services.vision.detect import YOLOv8Detector
|
|
return YOLOv8Detector(self._settings)
|
|
|
|
def create_face_processor(self) -> FaceProcessor:
|
|
from app.services.vision.insightface_processor import InsightFaceProcessor
|
|
return InsightFaceProcessor(self._settings)
|
|
|
|
def create_classifier(self) -> ContentClassifier:
|
|
from app.services.vision.classify import CLIPContentClassifier
|
|
return CLIPContentClassifier(self._settings)
|