Skip to content

Commit

Permalink
[SPARK-50394][PYTHON][INFRA][FOLLOW-UP] Reduce parallelism further in…
Browse files Browse the repository at this point in the history
… Pure Python library builds

### What changes were proposed in this pull request?

This PR is a followup of #48932 that reduces parallelism further

### Why are the changes needed?

In order to make the tests more robust:

https://github.com/apache/spark/actions/workflows/build_python_connect35.yml

It fails because of OOM.

FWIW, https://github.com/apache/spark/actions/workflows/build_python_connect.yml is fixed by the original PR.

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

Will monitor the build.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #48948 from HyukjinKwon/SPARK-50394-followup.

Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
HyukjinKwon authored and dongjoon-hyun committed Nov 25, 2024
1 parent d9757b5 commit 4f8d1f5
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion .github/workflows/build_python_connect35.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ jobs:
# Run branch-3.5 tests
./python/run-tests --parallelism=1 --python-executables=python3 --modules pyspark-connect
# None of tests are dependent on each other in Pandas API on Spark so run them in parallel
./python/run-tests --parallelism=2 --python-executables=python3 --modules pyspark-pandas-connect,pyspark-pandas-slow-connect
./python/run-tests --parallelism=1 --python-executables=python3 --modules pyspark-pandas-connect,pyspark-pandas-slow-connect
- name: Upload test results to report
if: always()
uses: actions/upload-artifact@v4
Expand Down

0 comments on commit 4f8d1f5

Please sign in to comment.