Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] cuda and cudnn requirements for 1.18.1 gpu packages #21377

Merged
merged 5 commits into from
Jul 17, 2024
+36 −22
Merged

[Doc] cuda and cudnn requirements for 1.18.1 gpu packages #21377

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d60ddfc
update cuda/cudnn requirements
tianleiwu Jul 16, 2024
cd32038
format
tianleiwu Jul 16, 2024
f85b052
format
tianleiwu Jul 16, 2024
2859819
update comment
tianleiwu Jul 16, 2024
a64a2b4
update comment
tianleiwu Jul 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 36 additions & 22 deletions docs/execution-providers/CUDA-ExecutionProvider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,28 +29,42 @@ See [Build instructions](../build/eps.html#cuda).
## Requirements

Please reference table below for official GPU packages dependencies for the ONNX Runtime inferencing package. Note that
ONNX Runtime Training is aligned with PyTorch CUDA versions; refer to the Training tab
on [onnxruntime.ai](https://onnxruntime.ai/) for supported versions.

Note: Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility), ONNX Runtime built with CUDA 11.8 should be compatible with any CUDA 11.x version; ONNX Runtime built with CUDA 12.2 should be compatible with any CUDA 12.x version.

ONNX Runtime built with cuDNN 8.x are not compatible with cuDNN 9.x.

| ONNX Runtime | CUDA | cuDNN | Notes |
|--------------------------|--------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1.18 | 12.4 | 8.9.2.26 (Linux)<br/>8.9.2.26 (Windows) | The default CUDA version for ORT 1.18 is CUDA 11.8. To install CUDA 12 package, please look at [Install ORT](../install). Java CUDA 12 support is back for release 1.18 |
| 1.18 | 11.8 | 8.9.2.26 (Linux)<br/>8.9.2.26 (Windows) | |
| 1.17 | 12.2 | 8.9.2.26 (Linux)<br/>8.9.2.26 (Windows) | The default CUDA version for ORT 1.17 is CUDA 11.8. To install CUDA 12 package, please look at [Install ORT](../install).<br>Due to low demand on Java GPU package, only C++/C# Nuget and Python packages are released with CUDA 12.2 |
| 1.15<br>1.16<br>1.17 | 11.8 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2.4 up to 8.9.0 |
| 1.14<br/>1.13.1<br/>1.13 | 11.6 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
| 1.12<br/>1.11 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
| 1.10 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.1.51<br/>libcublas 11.6.1.51<br/>libcudnn 8.2.4 |
| 1.9 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.1.51<br/>libcublas 11.6.1.51<br/>libcudnn 8.2.4 |
| 1.8 | 11.0.3 | 8.0.4 (Linux)<br/>8.0.2.39 (Windows) | libcudart 11.0.221<br/>libcufft 10.2.1.245<br/>libcurand 10.2.1.245<br/>libcublasLt 11.2.0.252<br/>libcublas 11.2.0.252<br/>libcudnn 8.0.4 |
| 1.7 | 11.0.3 | 8.0.4 (Linux)<br/>8.0.2.39 (Windows) | libcudart 11.0.221<br/>libcufft 10.2.1.245<br/>libcurand 10.2.1.245<br/>libcublasLt 11.2.0.252<br/>libcublas 11.2.0.252<br/>libcudnn 8.0.4 |
| 1.5-1.6 | 10.2 | 8.0.3 | CUDA 11 can be built from source |
| 1.2-1.4 | 10.1 | 7.6.5 | Requires cublas10-10.2.1.243; cublas 10.1.x will not work |
| 1.0-1.1 | 10.0 | 7.6.4 | CUDA versions from 9.1 up to 10.1, and cuDNN versions from 7.1 up to 7.4 should also work with Visual Studio 2017 |
ONNX Runtime Training is aligned with PyTorch CUDA versions; refer to the Optimize Training tab on [onnxruntime.ai](https://onnxruntime.ai/getting-started) for supported versions.

Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility), ONNX Runtime built with CUDA 11.8 are compatible with any CUDA 11.x version; ONNX Runtime built with CUDA 12.x are compatible with any CUDA 12.x version.

ONNX Runtime built with cuDNN 8.x is not compatible with cuDNN 9.x, and vice versa. You can choose the package based on CUDA and cuDNN major versions that match your runtime environment (For example, PyTorch 2.3 uses cuDNN 8.x, while PyTorch 2.4 or later used cuDNN 9.x).

### CUDA 12.x

To install CUDA 12 package, please look at [Install ORT](../install).

| ONNX Runtime | CUDA | cuDNN | Notes |
|---------------|--------|-------|----------------------------------------------------------------------|
| 1.18.1 | 12.x | 9.x | cuDNN 9 is required. No Java package. |
| 1.18.0 | 12.x | 8.x | Java package is added. |
| 1.17.x | 12.x | 8.x | Only C++/C# Nuget and Python packages are released. No Java package. |

### CUDA 11.x

| ONNX Runtime | CUDA | cuDNN | Notes |
|----------------------|--------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| 1.18.x | 11.8 | 8.x | |
| 1.17<br>1.16<br>1.15 | 11.8 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2 up to 8.9 |
| 1.14<br>1.13 | 11.6 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
| 1.12<br>1.11 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
| 1.10 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.1.51<br/>libcublas 11.6.1.51<br/>libcudnn 8.2.4 |
| 1.9 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.1.51<br/>libcublas 11.6.1.51<br/>libcudnn 8.2.4 |
| 1.8 | 11.0.3 | 8.0.4 (Linux)<br/>8.0.2.39 (Windows) | libcudart 11.0.221<br/>libcufft 10.2.1.245<br/>libcurand 10.2.1.245<br/>libcublasLt 11.2.0.252<br/>libcublas 11.2.0.252<br/>libcudnn 8.0.4 |
| 1.7 | 11.0.3 | 8.0.4 (Linux)<br/>8.0.2.39 (Windows) | libcudart 11.0.221<br/>libcufft 10.2.1.245<br/>libcurand 10.2.1.245<br/>libcublasLt 11.2.0.252<br/>libcublas 11.2.0.252<br/>libcudnn 8.0.4 |

### CUDA 10.x

| ONNX Runtime | CUDA | cuDNN | Notes |
|--------------|--------|-------|-------------------------------------------------------------------------------------------------------------------|
| 1.5-1.6 | 10.2 | 8.0.3 | CUDA 11 can be built from source |
| 1.2-1.4 | 10.1 | 7.6.5 | Requires cublas10-10.2.1.243; cublas 10.1.x will not work |
| 1.0-1.1 | 10.0 | 7.6.4 | CUDA versions from 9.1 up to 10.1, and cuDNN versions from 7.1 up to 7.4 should also work with Visual Studio 2017 |

For older versions, please reference the readme and build pages on the release branch.

Expand Down
Loading