Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TensorRT EP] Update deprecated TRT api #18834

Merged
merged 4 commits into from
Dec 18, 2023
Merged

[TensorRT EP] Update deprecated TRT api #18834

merged 4 commits into from
Dec 18, 2023

Conversation

yf711
Copy link
Contributor

@yf711 yf711 commented Dec 15, 2023 •
edited
Loading

Description

Update deprecated TRT api:

  1. setMaxWorkspaceSize(max_workspace_size_)-------->setMemoryPoolLimit(nvinfer1::MemoryPoolType::kWORKSPACE, max_workspace_size_)
  2. kENABLE_TACTIC_HEURISTIC-------->supersede by trt builder optimization level 2

Perf & warning log comparison

TRT EP options User will see corresponding warning logs: Average inference time cost (FRCNN on A100)
trt_build_heuristics_enable|true [TensorRT EP] trt_build_heuristics_enable is deprecated on TRT 8.6 onwards. Please set builder optimization level as 2 to enable builder heuristics. ~300ms
trt_build_heuristics_enable|true   trt_builder_optimization_level|2 [TensorRT EP] Builder heuristics are enabled automatically by builder optimization level 2. trt_build_heuristics_enable is deprecated on TRT 8.6 onwards. ~275ms
trt_builder_optimization_level|2   ~275ms

Motivation and Context

Prepare for upcoming TRT 10

@yf711 yf711 marked this pull request as ready for review December 16, 2023 20:45
@yf711 yf711 merged commit 9426bd5 into main Dec 18, 2023
92 of 100 checks passed
@yf711 yf711 deleted the yifanl/trt_deprecated_api branch December 18, 2023 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jywu-msft jywu-msft jywu-msft approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yf711 @jywu-msft