New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llm perf with vllm server #104

Open

At1a8 opened this issue Sep 2, 2024 · 0 comments

At1a8 commented Sep 2, 2024

It seems that mlperf can use the server process in vllm server mode as a server to accept requests from client.

Can this be done in llm perf? If so, how to implement it in ByteMLPerf?

Thanks so much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment