CUDAExecutionProvider

[tnil25]

Im recieving this error while generating. This is after a fresh install:

onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider
names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'

The generation continues but it seems like it may be using the CPU? Is anyone else experiencing this?

[dancemanUK]

same me

(venv) G:\animatediff-cli-prompt-travel-other>pip show onnxruntime
Name: onnxruntime
Version: 1.16.1
Summary: ONNX Runtime is a runtime accelerator for Machine Learning models
Home-page: https://onnxruntime.ai
Author: Microsoft Corporation
Author-email: [email protected]
License: MIT License
Location: g:\animatediff-cli-prompt-travel-other\venv\lib\site-packages
Requires: coloredlogs, flatbuffers, numpy, packaging, protobuf, sympy
Required-by: rembg

(venv) G:\animatediff-cli-prompt-travel-other>pip show onnxruntime-gpu
Name: onnxruntime-gpu
Version: 1.16.1
Summary: ONNX Runtime is a runtime accelerator for Machine Learning models
Home-page: https://onnxruntime.ai
Author: Microsoft Corporation
Author-email: [email protected]
License: MIT License
Location: g:\animatediff-cli-prompt-travel-other\venv\lib\site-packages
Requires: coloredlogs, flatbuffers, numpy, packaging, protobuf, sympy
Required-by:

[dancemanUK]

(venv) G:\animatediff-cli-prompt-travel-other>pip show protobuf
Name: protobuf
Version: 3.20.3
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author:
Author-email:
License: BSD-3-Clause
Location: g:\animatediff-cli-prompt-travel-other\venv\lib\site-packages
Requires:
Required-by: mediapipe, onnxruntime, onnxruntime-gpu

[s9roll7]

https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
Unfortunately, it doesn't appear to be compatible with newer versions of cuda.

[dancemanUK]

https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html Unfortunately, it doesn't appear to be compatible with newer versions of cuda.

ONNX Runtime CUDA cuDNN Notes
1.15
1.16 11.8 8.2.4 (Linux)
8.5.0.96 (Windows) Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2.4 up to 8.7.0

[deeplearn-art]

It's a well known issue. They released recently a nightly build of onnxruntime-gpu that was supposed to work with cuda12, but it didn't, however as they are working on it, I believe it will be remedied in near future

microsoft/onnxruntime#18850

[dancemanUK]

G:\animatediff-cli-prompt-travel-other\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider'
is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
warnings.warn(
Preprocessing images (controlnet_openpose) 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 159/159 [ 0:00:41 < 0:00:00 , 4 it/s ]
Saving Preprocessed images (controlnet_openpose) 99% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺ 158/159 [ 0:00:01 < 0:00:01 , 92 it/s ]

[tnil25]

Rolling back CUDA to 11.8 seems to fix the error. You have to make sure to uninstall all CUDA elements first though.

[dancemanUK]

将 CUDA 回滚到 11.8 似乎可以修复该错误。不过，您必须确保先卸载所有 CUDA 元素。

Sun Dec 24 18:32:53 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 546.33 Driver Version: 546.33 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 WDDM | 00000000:0B:00.0 On | Off |
| 0% 35C P8 24W / 450W | 8389MiB / 24564MiB | 4% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 6444 C+G C:\Windows\explorer.exe N/A |
| 0 N/A N/A 6548 C+G ...al\Discord\app-1.0.9028\Discord.exe N/A |
| 0 N/A N/A 8816 C+G ....Search_cw5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 14176 C+G ...CBS_cw5n1h2txyewy\TextInputHost.exe N/A |
| 0 N/A N/A 15684 C+G ...9\extracted\runtime\WeChatAppEx.exe N/A |
| 0 N/A N/A 17120 C+G ...ekyb3d8bbwe\PhoneExperienceHost.exe N/A |
| 0 N/A N/A 17800 C+G ...siveControlPanel\SystemSettings.exe N/A |
| 0 N/A N/A 18256 C+G ...GeForce Experience\NVIDIA Share.exe N/A |
| 0 N/A N/A 19860 C+G ...crosoft\Edge\Application\msedge.exe N/A |
| 0 N/A N/A 20096 C+G ....0_x64__8wekyb3d8bbwe\HxOutlook.exe N/A |
| 0 N/A N/A 23444 C ...imateDiff\python_embeded\python.exe N/A |
| 0 N/A N/A 25276 C+G ...12.0_x64__8wekyb3d8bbwe\GameBar.exe N/A |
| 0 N/A N/A 26104 C+G ...5n1h2txyewy\ShellExperienceHost.exe N/A |
+---------------------------------------------------------------------------------------+

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

[JojoYay]

oh i just forcefully update CUDAExecutionProvider to AzureExecutionProvider. not sure that was okay but so far it seems fine...
was it a problem?

[dancemanUK]

import torch
import onnxruntime as rt
rt.get_device()
'CPU'
rt.get_available_providers()
['AzureExecutionProvider', 'CPUExecutionProvider']

[dancemanUK]

rembg[gpu] need onnxruntime, onnxruntime-gpu.
so,onnxruntime get_available_providers() ['AzureExecutionProvider', 'CPUExecutionProvider']

[dancemanUK]

1. pip uninstall rembg
2. pip uninstall onnxruntime
3. pip uninstall onnxruntime-gpu
4. pip install rembg[cpu] or pip install rembg[cpu,cli]
5. pip install onnxruntime
6. pip install onnxruntime-gpu

if ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
animatediff 0.1.dev236+gdbf40d1.d20231218 requires pydantic<2.0.0,>=1.10.0, but you have pydantic 2.5.3 which is incompatible.

pip install pydantic==1.10.0

test
type python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import onnxruntime as rt
rt.get_device()
'GPU'
rt.get_available_providers()
['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']