feat: GPU acceleration support for ONNX Runtime inference
Centralize execution provider selection in providers.py with auto-detection and graceful fallback. All ONNX sessions (embedder, detector, face processor, recognizer) now use the configured providers. - New VISION_EXECUTION_PROVIDERS env var: "auto" for GPU auto-detect, or explicit "CUDAExecutionProvider,CPUExecutionProvider" - Provider priority: CUDA > ROCm > OpenVINO > CPU (when set to "auto") - docker-compose.yml includes commented-out NVIDIA GPU deploy section - Supports onnxruntime-gpu as a drop-in replacement for onnxruntime Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -120,12 +120,10 @@ class YOLOv8Detector(ObjectDetector):
|
||||
def __init__(self, settings: VisionSettings):
|
||||
model_path = Path(settings.models_dir) / "detect" / "yolov8n.onnx"
|
||||
|
||||
opts = ort.SessionOptions()
|
||||
opts.inter_op_num_threads = 2
|
||||
opts.intra_op_num_threads = 2
|
||||
from app.services.vision.providers import create_session
|
||||
|
||||
logger.info("Loading YOLOv8n from %s", model_path)
|
||||
self._session = ort.InferenceSession(str(model_path), opts, providers=["CPUExecutionProvider"])
|
||||
self._session = create_session(str(model_path), configured_providers=settings.execution_providers)
|
||||
self._conf_threshold = settings.detector.min_confidence
|
||||
self._max_detections = settings.detector.max_detections
|
||||
|
||||
|
||||
Reference in New Issue
Block a user