Skip to content

Commit

Permalink
[Docs] Adding CUDA Graph docs for TRT (#19064)
Browse files Browse the repository at this point in the history
### Description

Adding docs for the flag `trt_cuda_graph_enable`.
  • Loading branch information
gedoensmax authored Jan 10, 2024
1 parent 7720b66 commit e394a8e
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions docs/execution-providers/TensorRT-ExecutionProvider.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ There are two ways to configure TensorRT settings, either by **TensorRT Executio
| trt_profile_min_shapes | ORT_TENSORRT_PROFILE_MIN_SHAPES | string |
| trt_profile_max_shapes | ORT_TENSORRT_PROFILE_MAX_SHAPES | string |
| trt_profile_opt_shapes | ORT_TENSORRT_PROFILE_OPT_SHAPES | string |
| trt_cuda_graph_enable | ORT_TENSORRT_CUDA_GRAPH_ENABLE | bool |

> Note: for bool type options, assign them with **True**/**False** in python, or **1**/**0** in C++.
Expand Down Expand Up @@ -179,6 +180,8 @@ TensorRT configurations can be set by execution provider options. It's useful wh

* `trt_build_heuristics_enable`: Build engine using heuristics to reduce build time.

* `trt_cuda_graph_enable`: This will capture a [CUDA graph](https://developer.nvidia.com/blog/cuda-graphs/) which can drastically help for a network with many small layers as it reduces launch overhead on the CPU.

* `trt_sparsity_enable`: Control if sparsity can be used by TRT.
* Check `--sparsity` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).

Expand Down

0 comments on commit e394a8e

Please sign in to comment.