variadic function calls on macos arm64 do not work

[rkaminsk]

The calling conventions on MacOSX for x86_64 and arm64 differ for variadic functions.

I have a function

extern "C" bool fun(int x, int *res, ...);

Accessing the third parameter fails because it is probably passed on the stack by cffi instead of by register.

a = ffi.new("int[]", 1)
b = ffi.new("int[]", 1)
lib.fun(0, a, b)

As a test, I managed to circumvent the problem by adding more named parameters to enforce passing the variadic parameters on the stack.

[arigo]

Hi! CFFI itself does not contain any assembly code for all the platforms it supports, and instead delegates to LibFFI. I think that this kind of problem can come from one of three places:

• bad API usage of LibFFI inside CFFI; we had this problem in the past when LibFFI grew new APIs to support new cases on some new platform.
• or it can be that there's a mismatch between the declaration and usage of your function; sometimes mismatches that are perfectly fine on some platforms are not on other platforms.
• or finally, it can be a bug in LibFFI.

Can you paste the whole runnable code for your example?

[rkaminsk]

I personally don't have a Mac and have to use one of a colleague. I'll try to produce some example tomorrow that really fails on the OS. I just want to note that I am using ffi.emit_c_code(). This means that libcffi is not even involved, right?

[arigo]

Actually, variadic function calls still need to go through LibFFI, even with ffi.emit_c_code().

[rkaminsk]

I see. Now that you point it out, this makes sense. Maybe I should also get rid of the variadic function. All arguments (except one int) are pointers anyway and could as well be passed in an array.

I'll still report the example here so that you can identify the problem.

[rkaminsk]

I managed to reproduce the issue on the macos runner. Strangely, this only happens with conda:

• https://github.com/rkaminsk/fun
• using python provided by the host works fine
  • https://github.com/rkaminsk/fun/actions/runs/9908016418/job/27372889488
• using python provided by conda fails
  • https://github.com/rkaminsk/fun/actions/runs/9907998334/job/27372836673
  • it only fails on macos, linux and windows work fine

Here is the list of packages installed in the conda environment:

    bzip2-1.0.8                |       h93a5062_5         119 KB  conda-forge
    ca-certificates-2024.7.4   |       hf0a4a13_0         151 KB  conda-forge
    libexpat-2.6.2             |       hebf3989_0          62 KB  conda-forge
    libffi-3.4.2               |       h3422bc3_5          38 KB  conda-forge
    libsqlite-3.46.0           |       hfb93653_0         811 KB  conda-forge
    libzlib-1.3.1              |       hfb2fe0b_1          46 KB  conda-forge
    ncurses-6.5                |       hb89a1cb_0         776 KB  conda-forge
    openssl-3.3.1              |       hfb2fe0b_1         2.8 MB  conda-forge
    pip-24.0                   |     pyhd8ed1ab_0         1.3 MB  conda-forge
    python-3.11.9              |h932a869_0_cpython        14.0 MB  conda-forge
    readline-8.2               |       h92ec313_1         244 KB  conda-forge
    setuptools-70.3.0          |     pyhd8ed1ab_0         485 KB  conda-forge
    tk-8.6.13                  |       h5083fa2_1         3.0 MB  conda-forge
    tzdata-2024a               |       h0c530f3_0         117 KB  conda-forge
    wheel-0.43.0               |     pyhd8ed1ab_1          57 KB  conda-forge
    xz-5.2.6                   |       h57fd34a_0         230 KB  conda-forge
    cffi-1.16.0                |  py311h4a08483_0         286 KB  conda-forge
    pycparser-2.22             |     pyhd8ed1ab_0         103 KB  conda-forge
    python_abi-3.11            |          4_cp311           6 KB  conda-forge

[rkaminsk]

The culprit might actually be the rather old libffi version on conda-forge from 2021...

I further refined the workflow and the test only fails if the libffi from conda-forge is used:

• https://github.com/rkaminsk/fun/actions/runs/9908410732

We should probably request an update libffi here:

• https://github.com/conda-forge/libffi-feedstock