Bumped minimum version to 3.8 #1163
Conversation
|
@dimitri-yatsenko @ethho Can we update the Python versioning to drop support for end-of-life versions of Python (3.7)? |
|
Thanks for the PR @tabedzki! Agreed that we should drop 3.7 in favor of 3.12. Can you also change the CI so that we test Python3.12 + MySQL 8.0, and drop the 3.7 tests? I can't make the change on your fork: datajoint-python/.github/workflows/development.yaml Lines 60 to 68 in 685b42c EDIT: Seems that https://github.com/datajoint/datajoint-python/actions/runs/9976604874/job/28053793748?pr=1163 is failing for 3.7 because of this change, as expected. Let's remove 3.7 from the PY_VER matrix. |
|
Also please update the CHANGELOG.md |
|
Done! Let me know if there are any other changes I should make. |
|
CI job failed because there is no Docker image We need to add 3.12 and build the image from the datajoint/djbase-docker repository (has CI, looks like it's never been used), similar to the addition of 3.11 in this PR. EDIT: just saw datajoint/djtest-docker#7, running the build now. |
|
Depends on datajoint/djbase-docker#36 |
|
To summarize: we need to add python 3.12 to every image in the dependency tree, in this order:
|
|
@ethho Since Python 3.13 will be released on October 1st of this year, is there any prep work we can do now or should this be broached in October? Python 3.8 also goes end of life that same month. |
@tabedzki We install Python in the image via conda, so let's wait until a stable 3.13 release is available via conda channels before updating. This whole dependency tree of Docker images is cumbersome; we should simplify it by maintaining a minimal set of images either in this repository or |
|
I took a look at the datajoint/miniconda3-docker and datajoint/dj-base repositories to get an understanding of the workflow. I saw that the code is tested on both alpine and distros and packaged with
Please let me know if there are any consideration details that I have may missed in these suggestions (i.e., if conda installs more than just python/if docker images are used elsewhere for testing and debugging purposes). |
@tabedzki AFAIK,
You are correct. Unfortunately, images such as IMO the best approach now is to manually build a 3.12 miniconda image, with the Dockerfile (pseudocode): It doesn't make sense to maintain separate miniconda images for different python versions anyway. |
|
@ethho thanks for giving me a fuller picture. I didn't realize how heavily dependent the system was on those docker images. Therefore, moving away from the conda installation and using the github actions, in the current setup, doesn't make sense. Looking forward to the Python 3.12 support when it happens. Thank you for your effort on this. |
|
@ethho since there is more work to add 3.12 than there is to remove 3.7 support, I've removed aspects of the PR that would be more burdensome and only focused on removing things that tied the repo to <3.8 in order to merge this request. Additionally, I've updated the tags used for a couple of github actions since I noticed they were out of the date. I can open a separate issue that PR that would focus on 3.12 support. |
@tabedzki Please do make a separate issue for this, thanks! The docker builds need some work; I'll make another ticket to remove dependency on these images as noted in my previous comment. CI runs such as this one now failing due to use of |
|
@ethho I just learned about SPEC 0 — Minimum Supported Dependencies, a coordinated effort endorsed by the major scientific packages (SciPy, NumPy, scikit-image, matplotlib, etc) to adopt a common time-based policy for dropping dependencies after 36 months from the feature release (e.g., 3.9.0, not 3.9.x). Based off this recommendations, the oldest Python version that should be supported is Python 3.10, which would be dropped according to SPEC 0. Likewise, As we are already updating which versions of Python |
|
@ethho and @dimitri-yatsenko What are your thoughts on adopting the same support timetable as other scientific Python packages (e.g., numpy, scipy, etc), or adopting the timetable from Scientific Python?
|
@tabedzki Thanks for bringing this to our attention. I see the motivation here: SciPy has historically had a tough time supporting many Python versions. In my opinion, datajoint should continue to support older versions of Python as long as the maintenance cost is minimal. We should deprecate older versions on a case-by-case basis, when we encounter instances where the maintenance cost is non-trivial. This ticket is one example; it seems to be caused by a breaking networkx change. The reasoning is that datajoint occupies a different space in the scientific Python stack than packages like numpy, scipy, and pandas. New features in numpy / scipy are tightly coupled to Python version and target arch. datajoint is not tightly coupled to Python version, and therefore the maintenance cost of supporting legacy versions is relatively small. In other words, new datajoint features are unlikely to use specific Python language features. datajoint is, however, tightly coupled to the database backend. This is why we adopted this approach and deprecated MySQL 5.7 and only test for MySQL 8. |
|
@ethho thank you for the in-depth explanation. I can understand and appreciate the reasoning. @ethho and @dimitri-yatsenko |
To close #1162
Happy to make other contributions as necessary to upgrade the minimum version.