Enabling use of GPU

[teebu]

I assumed the recent update enabled gpu use, but the only way I was able to get it to start using the GPU is set an env var.

export ONNX_PROVIDER=CUDAExecutionProvider

Is it supposed to auto detect this or is this the proper way to enable use of gpu?

[nazdridoy]

does uninstalling "kokoro-onnx" and installing "kokoro-onnx[gpu]" fix your issue?

uv remove kokoro-onnx
uv add kokoro-onnx[gpu]'

or

pip uninstall kokoro-onnx
pip install kokoro-onnx[gpu]'
``

[teebu]

Didn't make a difference for me.

[teebu]

running uv sync

i see it creates venv with

 + onnxruntime==1.20.1
 + onnxruntime-gpu==1.20.1

[nazdridoy]

its being addressed Upstream

so far i can't reproduce your issue, i'll look into it soon

[fastfading]

uv remove kokoro-onnx
uv add kokoro-onnx[gpu]'

does it support Mac M chips ?

I get errors
uv add kokoro-onnx[gpu]
zsh: no matches found: kokoro-onnx[gpu]

[fastfading]

@teebu how to use this on mac

I assumed the recent update enabled gpu use, but the only way I was able to get it to start using the GPU is set an env var.
export ONNX_PROVIDER=CUDAExecutionProvider

[MeanMan]

There might be some issue with onnxruntime module. microsoft/onnxruntime#7748
If onnxruntime_gpu and onnxruntime are both loaded then CUDAExecutionProvider is not available. Installing module kokoro-onnx[gpu] seem to install both those libraries.

I have not been able to use GPU.

The versions might have an effect according to some note I found but Im not getting it to work.

[nazdridoy]

uv remove kokoro-onnx uv add kokoro-onnx[gpu]'
does it support Mac M chips ?

I get errors uv add kokoro-onnx[gpu] zsh: no matches found: kokoro-onnx[gpu]

uv add 'kokoro-onnx[gpu]'

[nazdridoy]

There might be some issue with onnxruntime module. microsoft/onnxruntime#7748 If onnxruntime_gpu and onnxruntime are both loaded then CUDAExecutionProvider is not available. Installing module kokoro-onnx[gpu] seem to install both those libraries.

I have not been able to use GPU.

The versions might have an effect according to some note I found but Im not getting it to work.

yes, upstream kokoro-onnx is addressing this issue, as kokoro-onnx[gpu] pulls onnxruntime_gpu as well as onnxruntime.

there is a work around, (i'm not a fan of it)

uv pip install kokoro-onnx[gpu]
uv pip uninstall onnxruntime
uv pip install onnxruntime_gpu

This was reported to be working.

[fastfading]

uv remove kokoro-onnx uv add kokoro-onnx[gpu]'
does it support Mac M chips ?
I get errors uv add kokoro-onnx[gpu] zsh: no matches found: kokoro-onnx[gpu]

uv add 'kokoro-onnx[gpu]'

can it support MAC
do I need to do some export like this
export ONNX_PROVIDER=CUDAExecutionProvider

can some one provide a guide for mac ?

[Dave1475]

uv remove kokoro-onnx uv add kokoro-onnx[gpu]'
does it support Mac M chips ?
I get errors uv add kokoro-onnx[gpu] zsh: no matches found: kokoro-onnx[gpu]

uv add 'kokoro-onnx[gpu]'

can it support MAC do I need to do some export like this export ONNX_PROVIDER=CUDAExecutionProvider

can some one provide a guide for mac ?

ChatGPT had this to say:

The error occurs because the zsh shell interprets the square brackets [] as special characters for glob patterns. To fix this issue, you can escape the brackets or quote the entire package name.

pip install kokoro-onnx[gpu]
or
pip install "kokoro-onnx[gpu]"

export ONNX_PROVIDER=CoreMLExecutionProvider