We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A number of parametrized test specs from SWE-Bench_Verified are incorrect as they are missing parameters.
For example issue pylint-7277 references PASS_TO_PASS test tests/test_self.py::TestRunTC::test_stdin[/mymodule.py]
PASS_TO_PASS
tests/test_self.py::TestRunTC::test_stdin[/mymodule.py]
The dataset entry is here: https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified/viewer/default/test?q=pylint+684a1d6aa0a6791e20078bc524f97c8906332390
There is a single /mymodule.py parameter in the test spec. This is incorrect as test_stdin requires 3 parameters, as can be seen here: https://github.com/pylint-dev/pylint/blob/684a1d6aa0a6791e20078bc524f97c8906332390/tests/test_self.py#L497
/mymodule.py
test_stdin
Review the links included in the bug description.
Correct test specs including the required parameters.
Spec tests are missing the required parameters.
No response
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Describe the bug
A number of parametrized test specs from SWE-Bench_Verified are incorrect as they are missing parameters.
For example issue pylint-7277 references
PASS_TO_PASS
testtests/test_self.py::TestRunTC::test_stdin[/mymodule.py]
The dataset entry is here: https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified/viewer/default/test?q=pylint+684a1d6aa0a6791e20078bc524f97c8906332390
There is a single
/mymodule.py
parameter in the test spec. This is incorrect astest_stdin
requires 3 parameters, as can be seen here:https://github.com/pylint-dev/pylint/blob/684a1d6aa0a6791e20078bc524f97c8906332390/tests/test_self.py#L497
Steps/Code to Reproduce
Review the links included in the bug description.
Expected Results
Correct test specs including the required parameters.
Actual Results
Spec tests are missing the required parameters.
System Information
No response
The text was updated successfully, but these errors were encountered: