Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libbeat: make update fails on Python 3.13 #42181

Closed
mauri870 opened this issue Dec 30, 2024 · 2 comments · Fixed by #42183
Closed

libbeat: make update fails on Python 3.13 #42181

mauri870 opened this issue Dec 30, 2024 · 2 comments · Fixed by #42183
Assignees
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@mauri870
Copy link
Member

mauri870 commented Dec 30, 2024

I had to recreate my python environment today and it is failing on the most recent version of Python (3.13).

# Delete your existing python env
$ python -m venv ./build/python-env --clear
# Since venv is a python module, you need to be running python 3.13 (released Oct 7) in your system
$ python -V
Python 3.13.1
# Create a new python env on 3.13
$ python -m venv ./build/python-env
$ make update
Error message
. /home/mauri870/git/elastic/beats/libbeat/build/python-env/bin/activate && pip install  -q --upgrade pip ; \
if [ -a ./tests/system/requirements.txt ] && [ ! ../libbeat/tests/system/requirements.txt -ef ./tests/system/requirements.txt ] ; then \
        . /home/mauri870/git/elastic/beats/libbeat/build/python-env/bin/activate && pip install  -qUr ../libbeat/tests/system/requirements.txt -Ur ./tests/system/requirements.txt ; \
else \
        . /home/mauri870/git/elastic/beats/libbeat/build/python-env/bin/activate && pip install  -qUr ../libbeat/tests/system/requirements.txt ; \
fi
Live child 0x5d274a7a3ab0 (python-env) PID 149871
  error: subprocess-exited-with-error

  × Building wheel for cffi (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [45 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/__init__.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/_imp_emulation.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/_shimmed_dist_utils.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/api.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/backend_ctypes.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/cffi_opcode.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/commontypes.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/cparser.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/error.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/ffiplatform.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/lock.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/model.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/pkgconfig.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/recompiler.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/setuptools_ext.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/vengine_cpy.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/vengine_gen.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/verifier.py -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/_cffi_include.h -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/parse_c_type.h -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/_embedding.h -> build/lib.linux-x86_64-cpython-313/cffi
      copying src/cffi/_cffi_errors.h -> build/lib.linux-x86_64-cpython-313/cffi
      running build_ext
      building '_cffi_backend' extension
      creating build/temp.linux-x86_64-cpython-313/src/c
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/home/mauri870/git/elastic/beats/libbeat/build/python-env/include -I/usr/include/python3.13 -c src/c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-313/src/c/_cffi_backend.o
      src/c/_cffi_backend.c: In function ‘b_do_dlopen’:
      src/c/_cffi_backend.c:4532:22: warning: ‘Py_FileSystemDefaultEncoding’ is deprecated [-Wdeprecated-declarations]
       4532 |                      Py_FileSystemDefaultEncoding, &filename_or_null, &flags))
            |                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      In file included from /usr/include/python3.13/Python.h:99,
                       from src/c/_cffi_backend.c:2:
      /usr/include/python3.13/fileobject.h:22:46: note: declared here
         22 | Py_DEPRECATED(3.12) PyAPI_DATA(const char *) Py_FileSystemDefaultEncoding;
            |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      src/c/_cffi_backend.c: In function ‘_my_PyErr_WriteUnraisable’:
      src/c/_cffi_backend.c:6121:9: error: implicit declaration of function ‘_PyErr_WriteUnraisableMsg’; did you mean ‘PyErr_WriteUnraisable’? [-Wimplicit-function-declaration]
       6121 |         _PyErr_WriteUnraisableMsg(PyText_AS_UTF8(s), NULL);
            |         ^~~~~~~~~~~~~~~~~~~~~~~~~
            |         PyErr_WriteUnraisable
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cffi

The issue seems to be that some libraries (mainly cffi and grpcio) are not updated to work with python 3.13. Simply updating their versions fix the issue:

diff --git a/libbeat/tests/system/requirements.txt b/libbeat/tests/system/requirements.txt
index 00f3914d1a..edf7f66093 100644
--- a/libbeat/tests/system/requirements.txt
+++ b/libbeat/tests/system/requirements.txt
@@ -10,7 +10,7 @@ backports.ssl-match-hostname==3.5.0.1
 bcrypt==4.1.2
 cached-property==1.4.2
 certifi==2024.7.4
-cffi==1.16.0
+cffi==1.17.1
 chardet==3.0.4
 charset-normalizer==3.3.2
 cryptography==43.0.1
@@ -24,7 +24,7 @@ elasticsearch==7.8.1
 enum34==1.1.6
 exceptiongroup==1.2.0
 googleapis-common-protos==1.56.4
-grpcio==1.60.0
+grpcio==1.68.1
 idna==3.7
 importlib-metadata==1.7.0
 iniconfig==1.0.1

Looking at some config files in the repo, it seems we are running on 3.10.9 which is currently only receiving security updates.

I'm not completely sure that this does not break older python versions, I assume our CI is not running python latest for example, since it does not face this problem. I decided to create this issue to make sure nothing breaks when updating this.

@mauri870 mauri870 added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Dec 30, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@mauri870
Copy link
Member Author

Looks like the CI is running on Python 3.9, which will reach EOL in mid to late 2025.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants