Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TensorRT EP] [documentation] Set cuda 12 as default for ORT 1.19 #21758

Merged
merged 1 commit into from
Sep 24, 2024
Merged

[TensorRT EP] [documentation] Set cuda 12 as default for ORT 1.19 #21758

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/execution-providers/CUDA-ExecutionProvider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/dep

ONNX Runtime built with cuDNN 8.x is not compatible with cuDNN 9.x, and vice versa. You can choose the package based on CUDA and cuDNN major versions that match your runtime environment (For example, PyTorch 2.3 uses cuDNN 8.x, while PyTorch 2.4 or later used cuDNN 9.x).

Note: starting ORT 1.19, **CUDA 12.x** becomes default version when distributing ONNX Runtime GPU packages.

tianleiwu marked this conversation as resolved.
Show resolved Hide resolved
### CUDA 12.x

To install CUDA 12 package, please look at [Install ORT](../install).
Expand Down
33 changes: 18 additions & 15 deletions docs/execution-providers/TensorRT-ExecutionProvider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,21 +27,24 @@ See [Build instructions](../build/eps.md#tensorrt).

## Requirements

| ONNX Runtime | TensorRT | CUDA |
| :----------- | :------- | :--------- |
| 1.18-main | 10.0 | 11.8, 12.2 |
| 1.17 | 8.6 | 11.8, 12.2 |
| 1.16 | 8.6 | 11.8 |
| 1.15 | 8.6 | 11.8 |
| 1.14 | 8.5 | 11.6 |
| 1.12-1.13 | 8.4 | 11.4 |
| 1.11 | 8.2 | 11.4 |
| 1.10 | 8.0 | 11.4 |
| 1.9 | 8.0 | 11.4 |
| 1.7-1.8 | 7.2 | 11.0.3 |
| 1.5-1.6 | 7.1 | 10.2 |
| 1.2-1.4 | 7.0 | 10.1 |
| 1.0-1.1 | 6.0 | 10.0 |
Note: starting ORT 1.19, **CUDA 12** becomes default version when distributing ONNX Runtime GPU packages.

| ONNX Runtime | TensorRT | CUDA |
| :----------- | :------- | :------------- |
| 1.19-main | 10.2 | **12.2**, 11.8 |
tianleiwu marked this conversation as resolved.
Show resolved Hide resolved
| 1.18 | 10.0 | 11.8, 12.2 |
| 1.17 | 8.6 | 11.8, 12.2 |
| 1.16 | 8.6 | 11.8 |
| 1.15 | 8.6 | 11.8 |
| 1.14 | 8.5 | 11.6 |
| 1.12-1.13 | 8.4 | 11.4 |
| 1.11 | 8.2 | 11.4 |
| 1.10 | 8.0 | 11.4 |
| 1.9 | 8.0 | 11.4 |
| 1.7-1.8 | 7.2 | 11.0.3 |
| 1.5-1.6 | 7.1 | 10.2 |
| 1.2-1.4 | 7.0 | 10.1 |
| 1.0-1.1 | 6.0 | 10.0 |

For more details on CUDA/cuDNN versions, please see [CUDA EP requirements](./CUDA-ExecutionProvider.md#requirements).

Expand Down
10 changes: 5 additions & 5 deletions docs/get-started/with-python.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,20 @@ Use the CPU package if you are running on Arm CPUs and/or macOS.
pip install onnxruntime
```

### Install ONNX Runtime GPU (CUDA 11.x)
### Install ONNX Runtime GPU (CUDA 12.x)

The default CUDA version for ORT is 11.8.
The default CUDA version for ORT is 12.2.
tianleiwu marked this conversation as resolved.
Show resolved Hide resolved

```bash
pip install onnxruntime-gpu
```

### Install ONNX Runtime GPU (CUDA 12.x)
### Install ONNX Runtime GPU (CUDA 11.8)

For Cuda 12.x, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12/PyPI/onnxruntime-gpu/overview)
For Cuda 11.8, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-11/PyPI/onnxruntime-gpu/overview)

```bash
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/
```

## Install ONNX for model export
Expand Down
10 changes: 5 additions & 5 deletions docs/install/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,16 @@ pip install skl2onnx
dotnet add package Microsoft.ML.OnnxRuntime
```

#### Install ONNX Runtime GPU (CUDA 11.x)
#### Install ONNX Runtime GPU (CUDA 12.x)

The default CUDA version for ORT is 11.8
The default CUDA version for ORT is 12.2
tianleiwu marked this conversation as resolved.
Show resolved Hide resolved

```bash
# GPU
dotnet add package Microsoft.ML.OnnxRuntime.Gpu
```

#### Install ONNX Runtime GPU (CUDA 12.x)
#### Install ONNX Runtime GPU (CUDA 11.8)

1. Project Setup

Expand All @@ -116,8 +116,8 @@ a nuget.config file to your project in the same directory as your .csproj file.
<configuration>
<packageSources>
<clear/>
<add key="onnxruntime-cuda-12"
value="https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/nuget/v3/index.json"/>
<add key="onnxruntime-cuda-11"
value="https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/nuget/v3/index.json"/>
</packageSources>
</configuration>
```
Expand Down
Loading