fa9b21856f
feat: replace face pipeline with InsightFace, add content classifier
...
Face detection/recognition:
- Replace YuNet + SFace with InsightFace buffalo_l (RetinaFace + ArcFace)
- 512-d ArcFace embeddings (was 128-d SFace), migration 0006 resizes column
- Remove YOLO person-bbox workaround — RetinaFace is accurate enough
- Detection threshold 0.65 cleanly separates real faces (0.72+) from
false positives on dogs/paintings (0.56-0.61)
Content-type classification:
- CLIP zero-shot classifier using native PyTorch text encoder + ONNX
image encoder for high-quality text-image similarity
- Categories: photograph, screenshot, document, receipt, meme, artwork
- Writes Tag(kind=content_type) per photo via photo_tags
- Margin-based confidence: top-1 vs top-2 score difference
- New ClassifierSettings in config (enabled, min_confidence)
- Wired into vision_fanout pipeline
Tested: 6 real faces from 4 photos (zero false positives), 11/13 photos
classified (8 photograph, 2 artwork, 1 meme).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-10 13:49:02 +02:00
2a6661f779
fix: model weights setup — export scripts, ORT compat, bootstrap
...
- Add export_models.py for OpenCLIP ViT-B/32 and YOLOv8n ONNX export
- Fix ArgMax(13) ORT ARM64 incompatibility by passing eot_indices as a
separate ONNX input (computed outside the graph in embed.py)
- Use legacy TorchScript exporter (dynamo=False) for IR version 9 compat
- Upgrade onnxruntime to 1.18.1
- Rewrite bootstrap_models.py with clear separation of auto-downloadable
models (YuNet, SFace) vs manually-exported ones (OpenCLIP, YOLOv8n)
- Wire bootstrap into worker CMD (runs before Celery)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-10 09:41:25 +02:00
9282a5c734
feat: add vision pipeline scaffolding with ONNX backend
...
Introduce the app/services/vision/ module with ABC interfaces, ONNX
Runtime backend, model registry, and per-task implementations:
- OpenCLIP ViT-B/32 embedder (image + text, 512-d)
- RapidOCR engine (PP-OCRv4 via ONNX, no PaddlePaddle)
- YOLOv8n object detector (raw ONNX, no ultralytics runtime)
- YuNet + SFace face processor (Apache 2.0, opencv_zoo, 128-d)
- DBSCAN face clustering helper
Add VisionSettings to config (mulita.yml + Pydantic), bootstrap_models.py
for first-boot weight downloads, models_data Docker volume, and ROCm
backend stub for future GPU acceleration.
No Celery tasks wired yet — models load but nothing invokes them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-10 09:00:06 +02:00
