[Badcase]: qwen2.5-72b 在昇腾910推理结果不符合预期

[tianshiyisi]

Model Series

Qwen2.5

What are the models used?

qwen2.5-72b-instruct

What is the scenario where the problem happened?

qwen2.5-72b-instruct 在昇腾910b上推理异常

Is this badcase known and can it be solved using avaiable techniques?

• I have followed the GitHub README.
• I have checked the Qwen documentation and cannot find a solution there.
• I have checked the documentation of the related framework and cannot find useful information.
• I have searched the issues and there is not a similar one.

Information about environment

OS: Ubuntu 22.04
Python: Python 3.10.9
GPUs: 4 x 910B3
NPU driver: 24.1.rc1 (from npu-smi)
mindie version: 1.0.T59
mindietorch : 1.0-t59-torch2.1.0.abi0
torch-npu : 2.1.0.post3-20240523

Description

qwen2.5-72b-instruct

qwen2-72b-instruct

[paul-yangmy]

mark

[qiangruoyu]

请问下这个配置文件是咋修改的，模型初始化的时候就报错了。

[940910941]

需要多大的显存？需要什么样的显卡才能运行？