New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

集成TensorRT相关问题 #287

Open

BaiStone2017 opened this issue Jun 1, 2024 · 1 comment

BaiStone2017 commented Jun 1, 2024

大佬，打扰问下一些问题。

device=GPU，集成TensorRT是否支持qwen、baichuan等模型的推理？能否运行tools中的benchmark？
MindSpore集成TensorRT，在环境配置时，官方文档里提到“当前版本适配[CUDA 10.1]和[TensorRT 6.0.1.5]以及 [CUDA 11.1]和 [TensorRT 8.5.1].”，是否可以适配更高版本的cuda，比如12.3

wyq-Git commented Jul 29, 2024

尊敬的开发者您好，此模型TensorRT推理在GPU暂未适配，可以尝试MindFormers+Ascend https://gitee.com/mindspore/mindformers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment