[SPARK-50394][PYTHON][INFRA][FOLLOW-UP] Reduce parallelism further in…

… Pure Python library builds ### What changes were proposed in this pull request? This PR is a followup of #48932 that reduces parallelism further ### Why are the changes needed? In order to make the tests more robust: https://github.com/apache/spark/actions/workflows/build_python_connect.yml It fails because of OOM. ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Will monitor the build. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #48977 from HyukjinKwon/SPARK-50394-followup2. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
apache · Nov 27, 2024 · 5425d45 · 5425d45
1 parent 4ec9ebf
commit 5425d45
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/.github/workflows/build_python_connect.yml b/.github/workflows/build_python_connect.yml
@@ -93,7 +93,7 @@ jobs:
           # Several tests related to catalog requires to run them sequencially, e.g., writing a table in a listener.
           ./python/run-tests --parallelism=1 --python-executables=python3 --modules pyspark-connect,pyspark-ml-connect
           # None of tests are dependent on each other in Pandas API on Spark so run them in parallel
-          ./python/run-tests --parallelism=2 --python-executables=python3 --modules pyspark-pandas-connect-part0,pyspark-pandas-connect-part1,pyspark-pandas-connect-part2,pyspark-pandas-connect-part3
+          ./python/run-tests --parallelism=1 --python-executables=python3 --modules pyspark-pandas-connect-part0,pyspark-pandas-connect-part1,pyspark-pandas-connect-part2,pyspark-pandas-connect-part3
 
           # Stop Spark Connect server.
           ./sbin/stop-connect-server.sh