From bdd34647c42a14b5856d74c7cb76c7b93d26079f Mon Sep 17 00:00:00 2001 From: Dongjoon Hyun Date: Thu, 17 Oct 2024 11:26:19 -0700 Subject: [PATCH] [SPARK-50014][INFRA] Use `grpcio*` 1.67.0 in Python 3.13 image ### What changes were proposed in this pull request? This PR aims to use `grpcio` and `grpcio-status` `1.67.0` for Python 3.13 tests in order to reveal the remaining test failures after installing the official `grpcio` in Python 3.13 environment. ### Why are the changes needed? `grpcio` added Python 3.13 support since 1.66.2. - https://pypi.org/project/grpcio/1.67.0/ - https://pypi.org/project/grpcio/1.66.2/ ### Does this PR introduce _any_ user-facing change? No, this is an infra change for test coverage. Currently, `pyspark-connect` module test fails due to the missing required package, `grpc`, like the following. - https://github.com/apache/spark/actions/runs/11372942311/job/31638495254 ``` ModuleNotFoundError: No module named 'grpc' ``` ### How was this patch tested? Manual check the generated image of this PR builder. ``` $ docker run -it --rm ghcr.io/dongjoon-hyun/apache-spark-ci-image:master-11389776259 python3.13 -m pip list | grep grpcio grpcio 1.67.0 grpcio-status 1.67.0 ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #48522 from dongjoon-hyun/SPARK-50014. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun --- dev/infra/base/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev/infra/base/Dockerfile b/dev/infra/base/Dockerfile index 1edeed775880b..4313a61db5bf2 100644 --- a/dev/infra/base/Dockerfile +++ b/dev/infra/base/Dockerfile @@ -148,7 +148,7 @@ RUN apt-get update && apt-get install -y \ RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.13 # TODO(SPARK-49862) Add BASIC_PIP_PKGS and CONNECT_PIP_PKGS to Python 3.13 image when it supports Python 3.13 RUN python3.13 -m pip install --ignore-installed blinker>=1.6.2 # mlflow needs this -RUN python3.13 -m pip install lxml numpy>=2.1 && \ +RUN python3.13 -m pip install grpcio==1.67.0 grpcio-status==1.67.0 lxml numpy>=2.1 && \ python3.13 -m pip cache purge # Remove unused installation packages to free up disk space