Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cffi relies on deprecated (now removed) distutils.msvc9compiler module #117

Closed
alex opened this issue Aug 27, 2024 · 7 comments · Fixed by #118
Closed

cffi relies on deprecated (now removed) distutils.msvc9compiler module #117

alex opened this issue Aug 27, 2024 · 7 comments · Fixed by #118

Comments

@alex
Copy link
Contributor

alex commented Aug 27, 2024

In pypa/distutils#287 this module was removed and in pypa/setuptools#4606 that was incorporated into setuptools 74.

As a result, building cffi now fails with:

    Traceback (most recent call last):
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\_shimmed_dist_utils.py", line 33, in <module>
        from distutils.msvc9compiler import MSVCCompiler
    ModuleNotFoundError: No module named 'distutils.msvc9compiler'

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "D:\a\cryptography\cryptography\src\_cffi_src\build_openssl.py", line 60, in <module>
        ffi.emit_c_code(c_file)
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\api.py", line 695, in emit_c_code
        recompile(self, module_name, source,
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\recompiler.py", line 1546, in recompile
        ext = ffiplatform.get_extension(ext_c_file, module_name, **kwds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\ffiplatform.py", line 9, in get_extension
        from cffi._shimmed_dist_utils import Extension
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\_shimmed_dist_utils.py", line 36, in <module>
        raise Exception("This CFFI feature requires setuptools on Python >= 3.12. Please install the setuptools package.") from ex
    Exception: This CFFI feature requires setuptools on Python >= 3.12. Please install the setuptools package.
@MarkCallow
Copy link

MarkCallow commented Sep 3, 2024

Given this issue is breaking my CI builds and the delays getting one or other of the PRs to fix it landed and released, I am looking to workaround the issue temporarily for my project. I did not write our Python interface and I lack Python expertise so I am seeking help here to find a workaround.

My project's requirements.txt has

cffi==1.15.1
setuptools==70.0.0

I would have thought the latter line would get me an older version of setuptools that still has distutils.mscv9compiler but I am getting the ModuleNotFound error.

The setup function in my project's setup.py which does from setuptools import setup has

    cffi_modules=["buildscript.py:ffibuilder"],
    ...
    install_requires=["cffi>=1.15.1"],
    ...
    setup_requires=["cffi>=1.15.1"],

It seems that cffi is unaware of my project's setuptools choice and is using the latest one.

How can I make it use my project's selected setuptools or prevent it using the latest one?

To the maintainers of cffi, please release a fix a.s.a.p.

@Flow86
Copy link

Flow86 commented Sep 3, 2024

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

@arigo
Copy link
Contributor

arigo commented Sep 3, 2024

@nitzmahone see comments above.

@BLACKNOTES
Copy link

BLACKNOTES commented Sep 3, 2024

Pyclarity package depends on cffi and 2024 challenges of clarity is ongoing right now.
Only solution I find was installing pyclarity on Linux and running codes there. As cffi has no problem on Linux but this problem persists on windows.

@MarkCallow
Copy link

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

Thank you @Flow86. This worked for me too. I recommend anyone who is running their build from 'CMake' in GitHub Actions to use a full path for constraint.txt. I used $env:RUNNER_TEMP/constraint.txt.

@LeftMonster
Copy link

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

Same worked for me, Thanks. I'm curious why?

@Flow86
Copy link

Flow86 commented Sep 4, 2024

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

Same worked for me, Thanks. I'm curious why?

you can specify constraint files for the "simple" install (with --constraint file.txt) on the pip install command, but: and thats a big but: its not applied by pip to any build requirements - e.g if the package needs to be build locally since no binary is available, a "venv" is created for the build, and there the constraints do not apply at the moment and would use the broken combination of setuptools and cffi. if you use the environment variable, the venv also picks up the constraints and therefor it works.

lbartoletti pushed a commit to Oslandia/pysfcgal_ci that referenced this issue Sep 4, 2024
In setuptools v74, the `distutils.msvc9compiler` module has been
removed. However, `cffi` still relies on it. Ensure to install
`setuptools` v72 to ensure this still works.

See: python-cffi/cffi#117
lbartoletti pushed a commit to Oslandia/pysfcgal_ci that referenced this issue Sep 4, 2024
In setuptools v74, the `distutils.msvc9compiler` module has been
removed. However, `cffi` still relies on it. Therefore, the generation
of the windows package fails.

By adding a `pyproject.toml` file and declaring the build backend, it
is possible to define the setuptools version used by the command
`python3 - build` which generates the windows wheel.

See: python-cffi/cffi#117
See: https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
lbartoletti pushed a commit to Oslandia/pysfcgal_ci that referenced this issue Sep 5, 2024
In setuptools v74, the `distutils.msvc9compiler` module has been
removed. However, `cffi` still relies on it. Therefore, the generation
of the windows package fails.

By adding a `pyproject.toml` file and declaring the build backend, it
is possible to define the setuptools version used by the command
`python3 - build` which generates the windows wheel.

See: python-cffi/cffi#117
See: https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
MarkCallow added a commit to KhronosGroup/KTX-Software that referenced this issue Sep 6, 2024
An Emscripten Docker image update to Clang 20 and an update to the
GitHub Actions Windows runner image broke our CI builds. This PR
provides fixes or workarounds. While working on the Clang 20 issue it
was noticed that the FP options for `libktx` were not being set as
intended, except for the ASTC encoder. This PR also fixes that problem
so the same FP options are used throughout `libktx`. Lastly it contains
improvements to `build_wasm_docker.sh` made while debugging and fixing
the FP settings issue.

clang 20 has a new "overriding option" warning which is raised if one
float-point option modifies a setting made by another. We are setting
`-ffp-model=precise`, which leaves contraction on, and
`-ffp-contract=off` thus we get the warning. This usage is normal for
cross-platform invariance thus the fix is to disable the spam warning.

The Windows runner image update broke the python build due to removal of
a long deprecated package from the Python/PIP `setuptools` package and
our use of the `cffi` package which has a dependency on `setuptools`
including the deleted package. See
python-cffi/cffi#117. This PR works around the
problem by creating a `constraint.txt` file which is passed to PIP and
tells it to limit the `setuptools` package to a version that still
contains the deleted package.
MarkCallow added a commit to KhronosGroup/KTX-Software that referenced this issue Sep 6, 2024
MarkCallow added a commit to KhronosGroup/KTX-Software that referenced this issue Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants