制作了gptq 4bit版本 #107
MrToy
started this conversation in
Show and tell
Replies: 3 comments 3 replies
-
咋用的啊 |
Beta Was this translation helpful? Give feedback.
1 reply
-
感谢热心分享,由于不属于 |
Beta Was this translation helpful? Give feedback.
2 replies
-
感谢分享,亲测可用,大约占用 8G 显存。附上 GPTQ-for-LLaMa CUDA 命令
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
https://huggingface.co/mrtoy/chinese-llama-13b-4bit-128g
效果还行,和fp16输出的内容差不多
Beta Was this translation helpful? Give feedback.
All reactions