New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

如何多卡部署 #480

Open

longcheng183 opened this issue Jul 22, 2024 · 1 comment

longcheng183 commented Jul 22, 2024

./main -p fastllm_int4g_70B.flm --eos_token "<|eot_id|>" 博主这个命令后面加多卡部署命令（--device "['cuda:0', 'cuda:1']"），不好使，求赐教

Owner

ztxz16 commented Jul 23, 2024

main程序现在没写这个多卡部署的参数.. 用ftllm.chat的方式可以设置

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment