Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement basic cache and support resnet in tvm backend #76

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

zrr1999
Copy link
Contributor

@zrr1999 zrr1999 commented Sep 29, 2023

本 PR 包含以下内容:

  1. 实现基础的 cache 机制,目前支持输入为 tensor 的情况(只判断shape和dtype是否一致)
  2. 为 TVM 后端添加多 tuple 返回值支持
  3. 修复子图打断中,多个 tensor 只会返回第一个的问题。
  4. 完善tvm后端,支持resnet

@zrr1999 zrr1999 changed the title implement basic cache implement basic cache and support resnet in tvm backend Oct 2, 2023
@zrr1999
Copy link
Contributor Author

zrr1999 commented Oct 3, 2023

目前已经可以实现 ResNet 的端到端推理(tvm后端),目前速度比 paddle动态图慢了很多,大概耗时是20倍,不优化的 tvm.tir 看起来速度非常慢
@jzhang533 @gglin001

@jzhang533
Copy link
Contributor

对比速度的脚本是哪个呀,我可以看一看。

@zrr1999
Copy link
Contributor Author

zrr1999 commented Oct 7, 2023

对比速度的脚本是哪个呀,我可以看一看。

我对比速度现在是用的viztracer直接把每行代码的速度都追踪一下,然后大概看的时间



class TVMCompiler(CompilerBase):
def gen_compiled_func(self, symbol_table: SymbolTable[te.Tensor, paddle.Tensor]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

对于tvm,不知道有没有可能利用现有的tvm的这个API:tvm.relay.frontend.from_paddle
也就是, 先把捕获到的子图,转成paddle的静态图,然后用tvm.relay.frontend.from_paddle给加载进来。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我这几天尝试一下,看起来是可以的。不过这样的话就感觉主要工作量都在动转静和tvm的tvm.relay.frontend.from_paddle这个接口的适配了,看起来 paddlefx 的内容很少

Copy link
Contributor

@jzhang533 jzhang533 Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

嗯,可以探索看看。好处是,可以复用现有的tvm里已有的,成熟的,paddle IR到tvm IR的转换,理论上,能很快实现较多的模型,lower到tvm来运行。
如果从头做fx graph到tvm IR的转换的话,感觉工作量是不少的。

@zrr1999 zrr1999 mentioned this pull request Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants