Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TPU backend for general_perf and llm_perf #119

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Yi-sir
Copy link

@Yi-sir Yi-sir commented Nov 6, 2024

  1. 提交general_perf TPU后端,包含编译、运行时及README。除TPU后端外,还包括如下改动:
    1.1 general_perf/backends/CPU/calculate_cpu_diff.sh 判断后端是否为TPU,若是则跳过venv
    1.2 general_perf/launch.py 调用上述sh时增加传参hardware_type
    1.3 general_perf/launch.py 当compile_only为true时,修复传参错误导致解析workload失败的问题
    1.4 general_perf/core/perf_engine.py 修改了更安全的获取cpu型号代码。旧代码在某些情况下获取失败时会导致程序崩溃
    1.5 general_perf/backends/CPU/ 后端文件将import torch提到import tensorflow之前,因为在sophgo/tpuc_dev中按照旧顺序import时进程会卡死
    1.6 general_perf/backends/runtime_backend.py 修复了load()方法的注释
    目前只测试了fp32 1b的resnet50和yolov5,后续会补充测试内容及完善README。
  2. 提交了llm_perf的TPU后端,包含调度部分,暂未包含modeling_xxx.py。模型部分近期将提交。

@CLAassistant
Copy link

CLAassistant commented Nov 6, 2024

CLA assistant check
All committers have signed the CLA.

@Yi-sir
Copy link
Author

Yi-sir commented Nov 13, 2024

  1. 支持了resnet50-torch-fp32 int8量化,需要交互设置量化参数。支持双芯异步perf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants