Skip to content

Commit

Permalink
[SPARK-50014][INFRA] Use grpcio* 1.67.0 in Python 3.13 image
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to use `grpcio` and `grpcio-status` `1.67.0` for Python 3.13 tests in order to reveal the remaining test failures after installing the official `grpcio` in Python 3.13 environment.

### Why are the changes needed?

`grpcio` added Python 3.13 support since 1.66.2.
- https://pypi.org/project/grpcio/1.67.0/
- https://pypi.org/project/grpcio/1.66.2/

### Does this PR introduce _any_ user-facing change?

No, this is an infra change for test coverage. Currently, `pyspark-connect` module test fails due to the missing required package, `grpc`, like the following.

- https://github.com/apache/spark/actions/runs/11372942311/job/31638495254
```
ModuleNotFoundError: No module named 'grpc'
```

### How was this patch tested?

Manual check the generated image of this PR builder.
```
$ docker run -it --rm ghcr.io/dongjoon-hyun/apache-spark-ci-image:master-11389776259 python3.13 -m pip list | grep grpcio
grpcio                   1.67.0
grpcio-status            1.67.0
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #48522 from dongjoon-hyun/SPARK-50014.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Oct 17, 2024
1 parent 91becf1 commit bdd3464
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion dev/infra/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ RUN apt-get update && apt-get install -y \
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.13
# TODO(SPARK-49862) Add BASIC_PIP_PKGS and CONNECT_PIP_PKGS to Python 3.13 image when it supports Python 3.13
RUN python3.13 -m pip install --ignore-installed blinker>=1.6.2 # mlflow needs this
RUN python3.13 -m pip install lxml numpy>=2.1 && \
RUN python3.13 -m pip install grpcio==1.67.0 grpcio-status==1.67.0 lxml numpy>=2.1 && \
python3.13 -m pip cache purge

# Remove unused installation packages to free up disk space
Expand Down

0 comments on commit bdd3464

Please sign in to comment.