574d71371f
refactor: strip AI pipeline to binary photo/other classifier
...
Drops face recognition, OCR, object detection, and semantic embeddings.
The sole remaining vision task is a CLIP-based binary classifier
(photography vs other); photos in "other" get needs_review=true so
screenshots, documents, memes and scans can be triaged from a new
filter pill in the UI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-14 22:27:17 +02:00
fa9b21856f
feat: replace face pipeline with InsightFace, add content classifier
...
Face detection/recognition:
- Replace YuNet + SFace with InsightFace buffalo_l (RetinaFace + ArcFace)
- 512-d ArcFace embeddings (was 128-d SFace), migration 0006 resizes column
- Remove YOLO person-bbox workaround — RetinaFace is accurate enough
- Detection threshold 0.65 cleanly separates real faces (0.72+) from
false positives on dogs/paintings (0.56-0.61)
Content-type classification:
- CLIP zero-shot classifier using native PyTorch text encoder + ONNX
image encoder for high-quality text-image similarity
- Categories: photograph, screenshot, document, receipt, meme, artwork
- Writes Tag(kind=content_type) per photo via photo_tags
- Margin-based confidence: top-1 vs top-2 score difference
- New ClassifierSettings in config (enabled, min_confidence)
- Wired into vision_fanout pipeline
Tested: 6 real faces from 4 photos (zero false positives), 11/13 photos
classified (8 photograph, 2 artwork, 1 meme).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-10 13:49:02 +02:00
9282a5c734
feat: add vision pipeline scaffolding with ONNX backend
...
Introduce the app/services/vision/ module with ABC interfaces, ONNX
Runtime backend, model registry, and per-task implementations:
- OpenCLIP ViT-B/32 embedder (image + text, 512-d)
- RapidOCR engine (PP-OCRv4 via ONNX, no PaddlePaddle)
- YOLOv8n object detector (raw ONNX, no ultralytics runtime)
- YuNet + SFace face processor (Apache 2.0, opencv_zoo, 128-d)
- DBSCAN face clustering helper
Add VisionSettings to config (mulita.yml + Pydantic), bootstrap_models.py
for first-boot weight downloads, models_data Docker volume, and ROCm
backend stub for future GPU acceleration.
No Celery tasks wired yet — models load but nothing invokes them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-10 09:00:06 +02:00
