New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请教一下，如何使用openai兼容格式的大模型作为评估模型来进行两模型在自定义数据集上的评估任务呢？ #115

Open

EvilCalf opened this issue Aug 26, 2024 · 3 comments

EvilCalf commented Aug 26, 2024

No description provided.

Collaborator

Yunnglin commented Aug 26, 2024

今天会更新相关的文档

Collaborator

wangxingjun778 commented Aug 26, 2024 •

edited

Loading

Refer to:

custom benchmark: https://evalscope.readthedocs.io/zh-cn/latest/advanced_guides/custom_dataset.html
vllm/swift/lmdeploy serving:
(LLM) https://evalscope.readthedocs.io/zh-cn/latest/user_guides/opencompass_backend.html
(VLM) https://evalscope.readthedocs.io/zh-cn/latest/user_guides/vlmevalkit_backend.html

Author

EvilCalf commented Aug 27, 2024 •

edited

Loading

那这样构建的数据集也可以进行竞技场模式和model based的模型测评么。gpt可以自定义url么，感谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment