Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failed to build env images #283

Open
jianzzz opened this issue Jan 10, 2025 · 1 comment
Open

failed to build env images #283

jianzzz opened this issue Jan 10, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@jianzzz
Copy link

jianzzz commented Jan 10, 2025

Describe the bug

Describe the bug and reproduction steps

I use OpenHands to run SWE-bench_Verified evaluation, when I run evaluation using the following script:

./evaluation/benchmarks/swe_bench/scripts/eval_infer.sh evaluation/evaluation_outputs/outputs/princeton-nlp__SWE-bench_Verified-test/CodeActAgent/gpt-4o_maxiter_30_N_v0.18.0a0-no-hint-run_1/output.jsonl "" princeton-nlp/SWE-bench_Verified

It fails with this error:

Running SWE-bench evaluation on the whole input file...
<frozen runpy>:128: RuntimeWarning: 'swebench.harness.run_evaluation' found in sys.modules after import of package 'swebench.harness', but prior to execution of 'swebench.harness.run_evaluation'; this may result in unpredictable behaviour
Running 485 unevaluated instances...
Base image sweb.base.x86_64:latest already exists, skipping build.
Base images built successfully.
Total environment images to build: 17
Building environment images:   6%|████▏                                                                   | 1/17 [00:31<08:16, 31.04s/it]BuildImageError sweb.env.x86_64.2baaea72acc974f6c02079:latest
Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/openhands-ai-w5nP72r4-py3.12/lib/python3.12/site-packages/swebench/harness/docker_build.py", line 145, in build_image
    raise docker.errors.BuildError(
docker.errors.BuildError: The command '/bin/sh -c /bin/bash -c "source ~/.bashrc && /root/setup_env.sh"' returned a non-zero code: 137

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/openhands-ai-w5nP72r4-py3.12/lib/python3.12/site-packages/swebench/harness/docker_build.py", line 314, in build_env_images
    future.result()
  File "/root/.conda/envs/openhands/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/envs/openhands/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/root/.conda/envs/openhands/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/openhands-ai-w5nP72r4-py3.12/lib/python3.12/site-packages/swebench/harness/docker_build.py", line 151, in build_image
    raise BuildImageError(image_name, str(e), logger) from e
swebench.harness.docker_build.BuildImageError: Error building image sweb.env.x86_64.2baaea72acc974f6c02079:latest: The command '/bin/sh -c /bin/bash -c "source ~/.bashrc && /root/setup_env.sh"' returned a non-zero code: 137
Check (logs/build_images/env/sweb.env.x86_64.2baaea72acc974f6c02079__latest/build_image.log) for more information.

A total of 17 env images failed to build.

Steps/Code to Reproduce

./evaluation/benchmarks/swe_bench/scripts/eval_infer.sh evaluation/evaluation_outputs/outputs/princeton-nlp__SWE-bench_Verified-test/CodeActAgent/gpt-4o_maxiter_30_N_v0.18.0a0-no-hint-run_1/output.jsonl "" princeton-nlp/SWE-bench_Verified

Actually it will run:

    poetry run python -m swebench.harness.run_evaluation \
        --dataset_name "$DATASET_NAME" \
        --split "$SPLIT" \
        --predictions_path $SWEBENCH_FORMAT_JSONL \
        --timeout 1800 \
        --cache_level instance \
        --max_workers $N_PROCESS \
        --run_id $RUN_ID

Expected Results

Successfully build all env images, or if there is an instance image, can I skip the env image build (currently the code does not support this).

Actual Results

A total of 17 env images failed to build.

System Information

No response

@jianzzz jianzzz added the bug Something isn't working label Jan 10, 2025
@carlosejimenez
Copy link
Member

Can you provide the content of the logs/build_images/env/sweb.env.x86_64.2baaea72acc974f6c02079__latest/build_image.log file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants