New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Question about testing CONCODE dataset in text-to-code (code generation) #188

Open

waroad opened this issue Sep 10, 2024 · 0 comments

waroad commented Sep 10, 2024

I'm new to this field and want to test my fine tuned model's capability to generate the code, using this CONCODE dataset.
However, I found that there is no ground truth for test.json (https://github.com/microsoft/CodeXGLUE/blob/main/Text-Code/text-to-code/dataset/concode/test.json).
How can I obtain the model's score after inference?
Is there a hidden server for evaluation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment