Stop installing setuptools and wheel #243
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joshwlewis
approved these changes
Aug 7, 2024
edmorley
force-pushed
the
pip-capitalisation
branch
from
d5f8522
to
8fb4292
Compare
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
edmorley
force-pushed
the
rm-setuptools-wheel
branch
from
c0b11b6
to
27b15e3
Compare
This was referenced Aug 7, 2024
This was referenced Aug 7, 2024
Merged
heroku-linguist bot
added a commit
to heroku/cnb-builder-images
that referenced
this pull request
Aug 7, 2024
## heroku/python ### Removed - Stopped explicitly installing setuptools and wheel. They will be automatically installed by pip into an isolated build environment if they are required for building a package. ([#243](heroku/buildpacks-python#243)) Co-authored-by: heroku-linguist[bot] <136119646+heroku-linguist[bot]@users.noreply.github.com>
This was referenced Aug 30, 2024
edmorley
added a commit
to heroku/heroku-buildpack-python
that referenced
this pull request
Oct 10, 2024
Add support for Python 3.13, and release 3.13.0. The default Python version remains unchanged (at 3.12.x) for now. Notably for Python 3.13 we now: - No longer install setuptools and wheel - matching what the wider ecosystem has already done for Python 3.12+. (See the Python CNB's removal PR for more details: heroku/buildpacks-python#243) - No longer install the SQLite headers and CLI, as the first step towards dropping that rarely used feature. - Have to disable some of the tests that run during the PGO profiling phase when on Heroku-22 (see PR comments for more details). In addition, for all Python versions we now also remove the `idle3` and `pydoc3` scripts, since they do not work with relocated Python and so have been broken for some time. Their functionality continues to be available by invoking them via their modules instead (e.g. `python -m pydoc`). Release announcement: https://blog.python.org/2024/10/python-3130-final-released.html https://www.python.org/downloads/release/python-3130/ Details on what's new in Python 3.13: https://docs.python.org/3.13/whatsnew/3.13.html Binary builds: https://github.com/heroku/heroku-buildpack-python/actions/runs/11280580537 Python 3.13 readiness status of the top 360 packages on PyPI: https://pyreadiness.org/3.13/ GUS-W-14846826. GUS-W-14846839. GUS-W-16944574.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels.
However:
[build-system]in theirpyproject.toml. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions.pyproject.tomlbuild backend config can still build even if setuptools isn't installed globally. (There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway.)As such, it's no longer necessary for us to install setuptools and wheel globally - and removing their install means we'll match the behaviour of the
venvandensurepipmodules in Python 3.12+, where setuptools and wheel installation has also been removed.In addition, these packages will no longer be accessible by app dependencies after the upcoming switch to venvs (see #253), so there is no point continuing to install them in the user site-packages directory.
This change also improves parity with the default behaviour of Poetry too, whose
install --synccommand removes any implicitly installed packages in the current environment (other than pip).See:
setuptoolsandwheelby default on Python 3.12+ environments pypa/virtualenv#2487GUS-W-16437776.