使用昇腾910进行ChatGLM单机多卡推理报错

[BellMeng]

使用PyTorch/built-in/foundation/ChatGLM-6B/utils.py中的样例执行
发生异常: TypeError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
new() received an invalid combination of arguments - got (Tensor, requires_grad=bool), but expected one of:

• (*, torch.device device)
  didn't match because some of the keywords were incorrect: requires_grad
• (torch.Storage storage)
• (Tensor other)
• (tuple of ints size, *, torch.device device)
• (object data, *, torch.device device)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/utils/modeling.py", line 368, in set_module_tensor_to_device
  new_value = param_cls(new_value, requires_grad=old_value.requires_grad).to(device)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 253, in init_hook
  set_module_tensor_to_device(module, name, self.execution_device)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 155, in add_hook_to_module
  module = hook.init_hook(module)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 350, in attach_execution_device_hook
  add_hook_to_module(module, AlignDevicesHook(execution_device, skip_keys=skip_keys))
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 357, in attach_execution_device_hook
  attach_execution_device_hook(child, execution_device)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 517, in attach_align_device_hook_on_blocks
  attach_execution_device_hook(module, execution_device[module_name])
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 546, in attach_align_device_hook_on_blocks
  attach_align_device_hook_on_blocks(
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 546, in attach_align_device_hook_on_blocks
  attach_align_device_hook_on_blocks(
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/hooks.py", line 546, in attach_align_device_hook_on_blocks
  attach_align_device_hook_on_blocks(
  [Previous line repeated 2 more times]
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/accelerate/big_modeling.py", line 399, in dispatch_model
  attach_align_device_hook_on_blocks(
  File "/home/ma-user/work/test.py", line 44, in load_model_on_gpus
  model = dispatch_model(model, device_map=device_map)
  File "/home/ma-user/work/test.py", line 51, in
  model = load_model_on_gpus("/home/ma-user/work/chatglm3-6b", 2)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/runpy.py", line 87, in _run_code
  exec(code, run_globals)
  File "/home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/runpy.py", line 197, in _run_module_as_main (Current frame)
  return _run_code(code, main_globals, None,
  TypeError: new() received an invalid combination of arguments - got (Tensor, requires_grad=bool), but expected one of:
• (*, torch.device device)
  didn't match because some of the keywords were incorrect: requires_grad
• (torch.Storage storage)
• (Tensor other)
• (tuple of ints size, *, torch.device device)
• (object data, *, torch.device device)