New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Usage]: 请问是否支持batch 推理？ #3

Open

pandaGst opened this issue Jul 8, 2024 · 1 comment

pandaGst commented Jul 8, 2024 •

edited

Loading

是否支持在minicpm_example.py 传入[prompt, ..., prompt], [image, image, image] ，从而实现batch推理。目前的默认脚本，看起来是一条一条的送进去推理的，推完一条再处理下一条？

Collaborator

HwwwwwwwH commented Jul 23, 2024

暂不支持，不过最近应该会更新 Batch 推理

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment