From 2cb5e3bcef527f51b1b19a8d485c95343389c8a0 Mon Sep 17 00:00:00 2001
From: Ewe Zi Yi <36802364+deadlycoconuts@users.noreply.github.com>
Date: Mon, 23 Sep 2024 12:26:23 +0800
Subject: [PATCH] fix(batch-predictor, pyfunc-server): Pin setuptools max
 version (#608)

# Description
It seems like from the 20th of September 2024, CICD tasks that require
building a python environment started
[failing](https://github.com/caraml-dev/merlin/actions/runs/10938668959/job/30411359538#step:7:2657)
with the following error when the ancient `mlflow` package tries to log
artifacts:
```
ModuleNotFoundError: No module named 'distutils._modified'
```

This seems to be an error
[reported](https://github.com/pyinstaller/pyinstaller/issues/8793)
recently by other people in the `pyinstaller` repo, with some users
mentioning that `distutils` is used by `setuptools`, which just had a
new version released recently. Since it doesn't seem like we're
importing `distutils` directly, I've decided to simply pin down the max
version of `setuptools` to `<75`. Note that the version of `setuputils`
somehow needs to be specified in the variable passed to this argument
`install_requires` here


https://github.com/caraml-dev/merlin/blob/a9184ede2126d71c84f6458bcc44515dd78b0d03/python/pyfunc-server/setup.py#L49

As such, I've added `setuptools<75` in both the batch predictor and
pyfunc server's requirements.txt file.

Note that this PR technically does not fix the root cause of the
problem; it's only a band-aid solution for the moment until we migrate
to newer versions of Mlflow **_very shortly_**. When that happens I'll
revert the changes in this PR.

# Modifications
- `.github/workflows/release.yml` - Updated the versions of `setuptools`
in the build jobs for the python-related components
- `python/batch-predictor/requirements.txt` - Added `setuptools<75` to
the requirements of the batch predictor
- `python/pyfunc-server/requirements.txt` - Added `setuptools<75` to the
requirements of the pyfunc server

# Tests
<!-- Besides the existing / updated automated tests, what specific
scenarios should be tested? Consider the backward compatibility of the
changes, whether corner cases are covered, etc. Please describe the
tests and check the ones that have been completed. Eg:
- [x] Deploying new and existing standard models
- [ ] Deploying PyFunc models
-->

# Checklist
- [x] Added PR label
- [ ] Added unit test, integration, and/or e2e tests
- [ ] Tested locally
- [ ] Updated documentation
- [ ] Update Swagger spec if the PR introduce API changes
- [ ] Regenerated Golang and Python client if the PR introduces API
changes

# Release Notes
<!--
Does this PR introduce a user-facing change?
If no, just write "NONE" in the release-note block below.
If yes, a release note is required. Enter your extended release note in
the block below.
If the PR requires additional action from users switching to the new
release, include the string "action required".

For more information about release notes, see kubernetes' guide here:
http://git.k8s.io/community/contributors/guide/release-notes.md
-->

```release-note
NONE
```
---
 .github/workflows/release.yml           | 6 +++---
 python/batch-predictor/requirements.txt | 1 +
 python/pyfunc-server/requirements.txt   | 1 +
 3 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 7241e8d86..f1ebc95b7 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -28,7 +28,7 @@ jobs:
           cache: pip
       - name: Install dependencies
         working-directory: ./python/sdk
-        run: pip install setuptools>=64 setuptools_scm>=8 twine wheel
+        run: pip install setuptools>=64,<75 setuptools_scm>=8 twine wheel
       - name: Build and publish
         env:
           TWINE_USERNAME: ${{ secrets.pypi_username }}
@@ -53,7 +53,7 @@ jobs:
           cache: pip
       - name: Install dependencies
         working-directory: ./python/pyfunc-server
-        run: pip install setuptools>=64 setuptools_scm>=8 twine wheel
+        run: pip install setuptools>=64,<75 setuptools_scm>=8 twine wheel
       - name: Build and publish
         env:
           TWINE_USERNAME: ${{ secrets.pypi_username }}
@@ -78,7 +78,7 @@ jobs:
           cache: pip
       - name: Install dependencies
         working-directory: ./python/batch-predictor
-        run: pip install setuptools>=64 setuptools_scm>=8 twine wheel
+        run: pip install setuptools>=64,<75 setuptools_scm>=8 twine wheel
       - name: Build and publish
         env:
           TWINE_USERNAME: ${{ secrets.pypi_username }}
diff --git a/python/batch-predictor/requirements.txt b/python/batch-predictor/requirements.txt
index 1a839e89f..4c41868a2 100644
--- a/python/batch-predictor/requirements.txt
+++ b/python/batch-predictor/requirements.txt
@@ -4,3 +4,4 @@ merlin-sdk==0.44.0
 mlflow==1.26.1
 pyarrow>=0.14.1,<11.0.0
 pyspark==3.0.1
+setuptools<75
\ No newline at end of file
diff --git a/python/pyfunc-server/requirements.txt b/python/pyfunc-server/requirements.txt
index 93f841942..d1af983ec 100644
--- a/python/pyfunc-server/requirements.txt
+++ b/python/pyfunc-server/requirements.txt
@@ -8,5 +8,6 @@ merlin-sdk==0.44.0
 numpy>=1.8.2
 orjson>=2.6.8
 prometheus-client
+setuptools<75
 tornado
 uvloop