请问支持官网的Qwen1.5 32B awq模型吗? #2363
Answered
by
lvhan028
lai-serena
asked this question in
Q&A
-
Beta Was this translation helpful? Give feedback.
Answered by
lvhan028
Aug 23, 2024
Replies: 1 comment
-
the quant_config of qwen1.5-32b is listed above. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
lvhan028
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
the quant_config of qwen1.5-32b is listed above.
The group_size is 32, which is not supported by lmdeploy.
Currently, lmdeploy only supports group_size 128