New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openai api部署glm-4-9b-chat如何设置多gpu #536

Open

2 tasks

desertfoxfj opened this issue Sep 3, 2024 · 1 comment

Assignees

desertfoxfj commented Sep 3, 2024

System Info / 系統信息

ubuntu18.04

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

The official example scripts / 官方的示例脚本
My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

我是两张 3060（一张 12G 显存）通过 openai_api 代码部署 glm-4-9b-chat，一直报显存不足，也没有招待代码中设置多 GPU 的地方
glm-4/basic_demo/openai_api_server.py

Expected behavior / 期待表现

这种场景应该如何配置多GPU

zhipuch self-assigned this

Collaborator

zhipuch commented Sep 3, 2024

https://zhipu-ai.feishu.cn/wiki/VpncwoWn4i2ZsDkWSqYcbtwSnWd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment