Unable to use onnxruntime for amd rocm

[zixianwang2022]

[Enter feedback here]

---

Document Details

• Title: AMD - ROCm
• Page: https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html
• Page Source: ROCm-ExecutionProvider.md

`Python 3.8.18 (default, Sep 11 2023, 13:40:15)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import onnxruntime as ort
model_path = "build/data/bert_tf_v1_1_large_fp32_384_v2/bert_large_v1_1_fake_quant.onnx"
providers = [
... 'ROCMExecutionProvider',
... 'CPUExecutionProvider',
... ]

session = ort.InferenceSession(model_path, providers=providers)
/home/ziw081/miniconda3/envs/test_mlperf_1/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'ROCMExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'`

>>> ort.__version__ '1.13.1'

rocm version 5.6

[cloudhan]

ORT 1.13.1 was released on Oct 25, 2022 and ROCm was released on Jun 28, 2023.

You should use a new release (recommended) or build the old version yourself (you might need to patch for new rocm, there is breaking changes in between).

[zixianwang2022]

ORT 1.13.1 was released on Oct 25, 2022 and ROCm was released on Jun 28, 2023.

You should use a new release (recommended) or build the old version yourself (you might need to patch for new rocm, there is breaking changes in between).

I pip uninstall onnxruntime and pip install onnxruntime-gpu again, now the version is onnxruntime-1.16.1, but I still had the same issue.
/home/ziw081/miniconda3/envs/test_mlperf_1/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'ROCMExecutionProvider' is not in available provider names.Available providers: 'TensorrtExecutionProvider, CUDAExecutionProvider, AzureExecutionProvider, CPUExecutionProvider'

Then I looked into the build for ROCM here: https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html.

I downloaded the onnxruntime github and run ./build.sh --config Release --use_rocm --rocm_home opt/rocm

But it output me that I failed 1 test. I am not sure how to resolve this. I thought the build might complete even though it fails a test, but apparently it did because I still get the above error.

While I was in the testing phase of the build, I saw many warnings about ROCMExecutionProvider were not able to run the model, screenshot attached below.

[cloudhan]

pip package onnxruntime-gpu is for CUDA, IIRC, you should use onnxruntime-rocm for it.

[zixianwang2022]

But there is no such a pip install onnxruntime-rocm, the package doesn't exit?

pip install onnxruntime-rocm
ERROR: Could not find a version that satisfies the requirement onnxruntime-rocm (from versions: none)
ERROR: No matching distribution found for onnxruntime-rocm```

[cloudhan]

@PeixuanZuo

[PeixuanZuo]

ROCm package didn't upload to Pypi, Please use packages from here: https://download.onnxruntime.ai/

[zixianwang2022]

Thank you cloudhan and Peixuan,

Is there a specific version that I should use? I assume I should use onnxruntime_nightly_rocm56.html because my system version is rocm5.6.

There are many links inside (https://download.onnxruntime.ai/onnxruntime_nightly_rocm56.html), should I wget and download all of them or is there a specific one that I should use?

[PeixuanZuo]

You can use stable version from https://download.onnxruntime.ai/onnxruntime_stable_rocm56.html or nightly version from
https://download.onnxruntime.ai/onnxruntime_nightly_rocm56.html, please install a specific one.

[Mershl]

ROCm package didn't upload to Pypi, Please use packages from here: https://download.onnxruntime.ai/

Hi @PeixuanZuo, is an upload to Pypi still planned or is the direct download the long-term recommended method?

[PeixuanZuo]

ROCm package didn't upload to Pypi, Please use packages from here: https://download.onnxruntime.ai/

Hi @PeixuanZuo, is an upload to Pypi still planned or is the direct download the long-term recommended method?

Hi @Mershl, We don't have a plan to upload it to PyPI. Please use the directly download way.