Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image build error running gold_patch #250

Closed
jintheman opened this issue Nov 13, 2024 · 1 comment
Closed

Image build error running gold_patch #250

jintheman opened this issue Nov 13, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@jintheman
Copy link

Describe the bug

Running

python3 -m swebench.harness.run_evaluation \
    --dataset_name princeton-nlp/SWE-bench \
    --predictions_path gold \
    --max_workers 1 \
    --instance_ids pylint-dev__pylint-8898  \
    --run_id validate-gold \
;

Gives

Error building image pylint-dev__pylint-8898: 400 Client Error for http+docker://localhost/v1.46/containers/create?name=sweb.eval.pylint-dev__pylint-8898.validate-gold&platform=linux%2Fx86_64: Bad Request ("NanoCPUs can not be set, as your kernel does not support CPU CFS scheduler or the cgroup is not mounted")

This also happens to the following task instances

pylint-dev__pylint-6196 pylint-dev__pylint-6386 pylint-dev__pylint-6517 pylint-dev__pylint-6556 pylint-dev__pylint-6937 pylint-dev__pylint-7114 pylint-dev__pylint-7993 pylint-dev__pylint-8281 pylint-dev__pylint-8757 pylint-dev__pylint-8898

Steps/Code to Reproduce

git clone [email protected]:princeton-nlp/SWE-bench.git
cd SWE-bench
pip install -e .

python3 -m swebench.harness.run_evaluation \
    --dataset_name princeton-nlp/SWE-bench \
    --predictions_path gold \
    --max_workers 1 \
    --instance_ids pylint-dev__pylint-6196 pylint-dev__pylint-6386 pylint-dev__pylint-6517 pylint-dev__pylint-6556 pylint-dev__pylint-6937 pylint-dev__pylint-7114 pylint-dev__pylint-7993 pylint-dev__pylint-8281 pylint-dev__pylint-8757 pylint-dev__pylint-8898  \
    --run_id validate-gold \
;

Expected Results

Gold patches pass without error.

Actual Results

Error building image pylint-dev__pylint-8898: 400 Client Error for http+docker://localhost/v1.46/containers/create?name=sweb.eval.pylint-dev__pylint-8898.validate-gold&platform=linux%2Fx86_64: Bad Request ("NanoCPUs can not be set, as your kernel does not support CPU CFS scheduler or the cgroup is not mounted")

System Information

Ubuntu 22.04.4, Python 3.10, swebench 2.1.2.

@jintheman jintheman added the bug Something isn't working label Nov 13, 2024
@john-b-yang
Copy link
Member

Just ran it, everything looks fine on my side. I think it's possible that some prior commit fixed it.

This line, which sets nano_cpus for certain versions of pylint, should fix this issue.

$ ./test.sh
/opt/miniconda3/envs/sweb/lib/python3.10/runpy.py:126: RuntimeWarning: 'swebench.harness.run_evaluation' found in sys.modules after import of package 'swebench.harness', but prior to execution of 'swebench.harness.run_evaluation'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Using gold predictions - ignoring predictions_path
Running 1 unevaluated instances...
Base image sweb.base.x86_64:latest already exists, skipping build.
Base images built successfully.
Total environment images to build: 1
1 ran successfully, 0 failed: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [01:16<00:00, 76.18s/it]
All environment images built successfully.
Running 1 instances...
1 ran successfully, 0 failed: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:29<00:00, 29.03s/it]
All instances run.
Cleaning cached images...
Removed 0 images.
Total instances: 1
Instances submitted: 1
Instances completed: 1
Instances incomplete: 0
Instances resolved: 1
Instances unresolved: 0
Instances with empty patches: 0
Instances with errors: 0
Unstopped containers: 0
Unremoved images: 0
Report written to gold.gold.json

$ cat test.sh
python3 -m swebench.harness.run_evaluation \
    --dataset_name princeton-nlp/SWE-bench \
    --predictions_path gold \
    --max_workers 1 \
    --instance_ids pylint-dev__pylint-8898  \
    --run_id gold \

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants