Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TensorRT EP] Update doc #20857

Merged
merged 5 commits into from
May 30, 2024
Merged

[TensorRT EP] Update doc #20857

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a5de5c1
Add `trt_engine_hw_compatible` and reorganize table
yf711 May 29, 2024
bc91ce5
update jetson instruction
yf711 May 29, 2024
5f4688e
nit
yf711 May 29, 2024
e3bed64
visual adjustment
yf711 May 29, 2024
675b334
add hyperlinks
yf711 May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions docs/build/eps.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Dockerfile instructions are available [here](https://github.com/microsoft/onnxru
### Build Instructions
{: .no_toc }

These instructions are for the latest [JetPack SDK 6.0](https://developer.nvidia.com/embedded/jetpack-sdk-60dp) for Jetson Orin.
These instructions are for the latest [JetPack SDK 6](https://developer.nvidia.com/embedded/jetpack) for Jetson Orin.

1. Clone the ONNX Runtime repo on the Jetson host

Expand All @@ -161,11 +161,13 @@ These instructions are for the latest [JetPack SDK 6.0](https://developer.nvidia

2. Specify the CUDA compiler, or add its location to the PATH.

1. Starting with **CUDA 11.8**, Jetson users on **JetPack 5.0+** can upgrade to the latest CUDA release without updating the JetPack version or Jetson Linux BSP (Board Support Package). CUDA version 11.8 with JetPack 5.1.2 has been tested on Jetson when building ONNX Runtime 1.16.
1. Starting with **CUDA 11.8**, Jetson users on **JetPack 5.0+** can upgrade to the latest CUDA release without updating the JetPack version or Jetson Linux BSP (Board Support Package).

1. For JetPack 5.x users, CUDA 11.8 and GCC 11 are required to be updated, in order to build latest ONNX Runtime locally.

1. Check [this official blog](https://developer.nvidia.com/blog/simplifying-cuda-upgrades-for-nvidia-jetson-users/) for CUDA upgrade instruction.
2. Check [this official blog](https://developer.nvidia.com/blog/simplifying-cuda-upgrades-for-nvidia-jetson-users/) for CUDA upgrade instruction.

2. CUDA 12.x is only available to Jetson Orin and newer series (CUDA compute capability >= 8.7). Check [here](https://developer.nvidia.com/cuda-gpus#collapse5) for compute capability datasheet.
3. CUDA 12.x is only available to Jetson Orin and newer series (CUDA compute capability >= 8.7). Check [here](https://developer.nvidia.com/cuda-gpus#collapse5) for compute capability datasheet.
JetPack 6.0 comes preinstalled with CUDA 12.2

2. CMake can't automatically find the correct `nvcc` if it's not in the `PATH`. `nvcc` can be added to `PATH` via:
Expand All @@ -180,6 +182,14 @@ These instructions are for the latest [JetPack SDK 6.0](https://developer.nvidia
export CUDACXX="/usr/local/cuda/bin/nvcc"
```

3. Update TensorRT libraries

1. Jetpack 5.x supports up to TensorRT 8.5. Jetpack 6.0 is equipped with TensorRT 8.6 and can support TensorRT 10.

2. Jetpack 6.0 users can download latest TensorRT 10 TAR package for jetpack on [TensorRT SDK website](https://developer.nvidia.com/tensorrt/download/10x).

3. Check [here](../execution-providers/TensorRT-ExecutionProvider.md#requirements) for TensorRT/CUDA support matrix among all ONNX Runtime versions.

3. Install the ONNX Runtime build dependencies on the Jetpack host:

```bash
Expand All @@ -194,9 +204,9 @@ These instructions are for the latest [JetPack SDK 6.0](https://developer.nvidia
and follow [https://cmake.org/install/](https://cmake.org/install/) to build from source.
2. (Ubuntu) Install deb package via apt repository: e.g [https://apt.kitware.com/](https://apt.kitware.com/)

5. Build the ONNX Runtime Python wheel (update path to CUDA/CUDNN/TensorRT libraries if necessary):
5. Build the ONNX Runtime Python wheel:

1. Build `onnxruntime-gpu` wheel with CUDA and TensorRT support:
1. Build `onnxruntime-gpu` wheel with CUDA and TensorRT support (update paths to CUDA/CUDNN/TensorRT libraries if necessary):

```bash
./build.sh --config Release --update --build --parallel --build_wheel \
Expand Down
Loading
Loading