feat: GPU acceleration support for ONNX Runtime inference

Centralize execution provider selection in providers.py with
auto-detection and graceful fallback. All ONNX sessions (embedder,
detector, face processor, recognizer) now use the configured providers.

- New VISION_EXECUTION_PROVIDERS env var: "auto" for GPU auto-detect,
  or explicit "CUDAExecutionProvider,CPUExecutionProvider"
- Provider priority: CUDA > ROCm > OpenVINO > CPU (when set to "auto")
- docker-compose.yml includes commented-out NVIDIA GPU deploy section
- Supports onnxruntime-gpu as a drop-in replacement for onnxruntime

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-12 22:12:21 +02:00
parent 94c07b1d0d
commit bbb8e4850c
7 changed files with 138 additions and 20 deletions

View File

@@ -120,12 +120,10 @@ class YOLOv8Detector(ObjectDetector):
def __init__(self, settings: VisionSettings):
model_path = Path(settings.models_dir) / "detect" / "yolov8n.onnx"
opts = ort.SessionOptions()
opts.inter_op_num_threads = 2
opts.intra_op_num_threads = 2
from app.services.vision.providers import create_session
logger.info("Loading YOLOv8n from %s", model_path)
self._session = ort.InferenceSession(str(model_path), opts, providers=["CPUExecutionProvider"])
self._session = create_session(str(model_path), configured_providers=settings.execution_providers)
self._conf_threshold = settings.detector.min_confidence
self._max_detections = settings.detector.max_detections