New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev分支的vllm支持流式输出么 #757

Open

jack-rao opened this issue Sep 17, 2024 · 1 comment

Labels

jack-rao commented Sep 17, 2024

测试了下dev的vllm加速推理，似乎不支持流式输出。

Member

fumiama commented Sep 17, 2024

流式正由社区开发中，详见PR列表。

fumiama added the documentation label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment