From ed44977d252fdeae3fb3b72d5a6d59f373cfe422 Mon Sep 17 00:00:00 2001 From: yf711 Date: Wed, 31 Jan 2024 20:08:37 -0800 Subject: [PATCH 1/2] Add description for new option trt_engine_cache_prefix --- docs/execution-providers/TensorRT-ExecutionProvider.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/execution-providers/TensorRT-ExecutionProvider.md b/docs/execution-providers/TensorRT-ExecutionProvider.md index ecdf53716a057..218fe203b7214 100644 --- a/docs/execution-providers/TensorRT-ExecutionProvider.md +++ b/docs/execution-providers/TensorRT-ExecutionProvider.md @@ -84,6 +84,7 @@ There are two ways to configure TensorRT settings, either by **TensorRT Executio | trt_dla_core | ORT_TENSORRT_DLA_CORE | int | | trt_engine_cache_enable | ORT_TENSORRT_ENGINE_CACHE_ENABLE | bool | | trt_engine_cache_path | ORT_TENSORRT_CACHE_PATH | string | +| trt_engine_cache_prefix | ORT_TENSORRT_CACHE_PREFIX | string | | trt_dump_subgraphs | ORT_TENSORRT_DUMP_SUBGRAPHS | bool | | trt_force_sequential_engine_build | ORT_TENSORRT_FORCE_SEQUENTIAL_ENGINE_BUILD | bool | | trt_context_memory_sharing_enable | ORT_TENSORRT_CONTEXT_MEMORY_SHARING_ENABLE | bool | @@ -162,6 +163,10 @@ TensorRT configurations can be set by execution provider options. It's useful wh * `trt_engine_cache_path`: Specify path for TensorRT engine and profile files if `trt_engine_cache_enable` is `True`, or path for INT8 calibration table file if `trt_int8_enable` is `True`. +* `trt_engine_cache_prefix`: Customize engine cache prefix when `trt_engine_cache_enable` is `True` (available on ORT 1.17). + + * ORT-TRT will only reuse existing engine cache with customized prefix if the same prefix is assigned in `trt_engine_cache_prefix`. If this option is empty, new engine cache with default prefix will be generated. + * `trt_dump_subgraphs`: Dumps the subgraphs that are transformed into TRT engines in onnx format to the filesystem. * This can help debugging subgraphs, e.g. by using `trtexec --onnx my_model.onnx` and check the outputs of the parser. From 6dac184ba38da940e484d994eafe84c881fab320 Mon Sep 17 00:00:00 2001 From: yf711 Date: Thu, 1 Feb 2024 12:09:21 -0800 Subject: [PATCH 2/2] update --- docs/execution-providers/TensorRT-ExecutionProvider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/execution-providers/TensorRT-ExecutionProvider.md b/docs/execution-providers/TensorRT-ExecutionProvider.md index 218fe203b7214..7a9b371b60eff 100644 --- a/docs/execution-providers/TensorRT-ExecutionProvider.md +++ b/docs/execution-providers/TensorRT-ExecutionProvider.md @@ -163,7 +163,7 @@ TensorRT configurations can be set by execution provider options. It's useful wh * `trt_engine_cache_path`: Specify path for TensorRT engine and profile files if `trt_engine_cache_enable` is `True`, or path for INT8 calibration table file if `trt_int8_enable` is `True`. -* `trt_engine_cache_prefix`: Customize engine cache prefix when `trt_engine_cache_enable` is `True` (available on ORT 1.17). +* `trt_engine_cache_prefix`: Customize engine cache prefix when `trt_engine_cache_enable` is `True`. * ORT-TRT will only reuse existing engine cache with customized prefix if the same prefix is assigned in `trt_engine_cache_prefix`. If this option is empty, new engine cache with default prefix will be generated.