Skip to content

Commit

Permalink
[TRT EP] Fix ProviderOptions functions (microsoft#17567)
Browse files Browse the repository at this point in the history
### Description
When trying to use the TRT EP option trt_extra_plugin_lib_paths I
noticed that my custom op library was not being loaded by the EP. After
some digging I found that code was missing to update this option when
UpdateTensorRTProviderOptions() is used to set it.

At the same time I noticed that char arrays were allocated in that
function and wondered where they are de-allocated. When I found it was
done in ReleaseTensorRTProviderOptions(), I noticed that a few
de-allocations were missing.

### Motivation and Context
This PR fixes the problems described above.
  • Loading branch information
simonjub authored Sep 17, 2023
1 parent 705f8a3 commit c969237
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 3 deletions.
14 changes: 14 additions & 0 deletions onnxruntime/core/providers/tensorrt/tensorrt_provider_factory.cc
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,20 @@ struct Tensorrt_Provider : Provider {
trt_options.trt_tactic_sources = (const char*)dest;
}

str_size = internal_options.extra_plugin_lib_paths.size();
if (str_size == 0) {
trt_options.trt_extra_plugin_lib_paths = nullptr;
} else {
dest = new char[str_size + 1];
#ifdef _MSC_VER
strncpy_s(dest, str_size + 1, internal_options.extra_plugin_lib_paths.c_str(), str_size);
#else
strncpy(dest, internal_options.extra_plugin_lib_paths.c_str(), str_size);
#endif
dest[str_size] = '\0';
trt_options.trt_extra_plugin_lib_paths = (const char*)dest;
}

str_size = internal_options.profile_min_shapes.size();
if (str_size == 0) {
trt_options.trt_profile_min_shapes = nullptr;
Expand Down
11 changes: 8 additions & 3 deletions onnxruntime/core/session/provider_bridge_ort.cc
Original file line number Diff line number Diff line change
Expand Up @@ -1930,9 +1930,14 @@ ORT_API_STATUS_IMPL(OrtApis::GetTensorRTProviderOptionsByName,
ORT_API(void, OrtApis::ReleaseTensorRTProviderOptions, _Frees_ptr_opt_ OrtTensorRTProviderOptionsV2* ptr) {
#ifdef USE_TENSORRT
if (ptr != nullptr) {
delete ptr->trt_int8_calibration_table_name;
delete ptr->trt_engine_cache_path;
delete ptr->trt_engine_decryption_lib_path;
delete[] ptr->trt_int8_calibration_table_name;
delete[] ptr->trt_engine_cache_path;
delete[] ptr->trt_engine_decryption_lib_path;
delete[] ptr->trt_tactic_sources;
delete[] ptr->trt_extra_plugin_lib_paths;
delete[] ptr->trt_profile_min_shapes;
delete[] ptr->trt_profile_max_shapes;
delete[] ptr->trt_profile_opt_shapes;
}

std::unique_ptr<OrtTensorRTProviderOptionsV2> p(ptr);
Expand Down

0 comments on commit c969237

Please sign in to comment.