Update Python interface to fall back on ctypes.util.find_library

[dfm]

Since the default dynamic library suffix is platform dependent, loading libfinufft.so directly with ctypes isn't very portable. This PR uses ctypes.util.find_library as a fallback to discover system libraries if libfinufft.so isn't found. This won't change the behavior for most users, but does allow users on macOS (for example) to use an installed version of libfinufft.

[DiamonDinoia]

I'll have a look

[ahbarnett]

@DiamonDinoia

[ahbarnett]

Thanks! @DiamonDinoia will incorporate this into #429

[ahbarnett]

maybe this should be brought in before we make any bigger shift of python wrapping, @janden ?

[DiamonDinoia]

This should be fixed in master after #429 was merged.

[ahbarnett]

good idea.

…

On Wed, Jul 17, 2024 at 11:45 AM Marco Barbone ***@***.***> wrote: This should be fixed in master after #429 <#429> was merged. — Reply to this email directly, view it on GitHub <#435 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNZRSQASQXSXQDBLM2W3A3ZM2GS3AVCNFSM6AAAAABHEMAC5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZTGYZTIMJVGA> . You are receiving this because you commented.Message ID: ***@***.***>

-- *-------------------------------------------------------------------~^`^~._.~' |\ Alex Barnett Center for Computational Mathematics, Flatiron Institute | \ http://users.flatironinstitute.org/~ahb 646-876-5942

[janden]

So this is actually incorporated for the FINUFFT (it was merged as part of #429). See

finufft/python/finufft/finufft/_finufft.py

Line 45 in 8baaa97

libname = find_library('finufft')

.

However, we should discuss today whether this behavior still makes sense. When a user does pip install for (even from source), the library will be compiled and the .so/.dll/.dylib will be installed along with the Python source. So under no circumstances should the pre-packaged library be missing. As such, there is not really a place for a system fallback.

[dfm]

For conda-forge (https://github.com/conda-forge/finufft-split-feedstock) the library and Python bindings are built separately so this is crucial for that. We can keep patching if strictly necessary.

…

On Tue, Jul 23, 2024 at 12:02 PM Joakim Andén ***@***.***> wrote: So this is actually incorporated for the FINUFFT (it was merged as part of #429 <#429>). See https://github.com/flatironinstitute/finufft/blob/8baaa9776ea8b0a3cb5e67822dae8fff9d5268a5/python/finufft/finufft/_finufft.py#L45 . However, we should discuss today whether this behavior still makes sense. When a user does pip install for (even from source), the library will be compiled and the .so/.dll/.dylib will be installed along with the Python source. So under no circumstances should the pre-packaged library be missing. As such, there is not really a place for a system fallback. — Reply to this email directly, view it on GitHub <#435 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACVQSQWW5TX4AALRAPCTZDZNZ5B3AVCNFSM6AAAAABHEMAC5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVGY2DGNBUGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[DiamonDinoia]

For conda-forge (conda-forge/finufft-split-feedstock) the library and Python bindings are built separately so this is crucial for that. We can keep patching if strictly necessary.
…
On Tue, Jul 23, 2024 at 12:02 PM Joakim Andén @.> wrote: So this is actually incorporated for the FINUFFT (it was merged as part of #429 <#429>). See

finufft/python/finufft/finufft/_finufft.py

Line 45 in 8baaa97

libname = find_library('finufft')

. However, we should discuss today whether this behavior still makes sense. When a user does pip install for (even from source), the library will be compiled and the .so/.dll/.dylib will be installed along with the Python source. So under no circumstances should the pre-packaged library be missing. As such, there is not really a place for a system fallback. — Reply to this email directly, view it on GitHub <#435 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACVQSQWW5TX4AALRAPCTZDZNZ5B3AVCNFSM6AAAAABHEMAC5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVGY2DGNBUGE . You are receiving this because you authored the thread.Message ID: @.>

#429 does the following:

1. search for a .so inside the _finufft.py directory, if found use that
2. otherwise search for finufft.so in the system LIBRARY_PATH and if found use it
3. fail with a meaningful error.

I think this works with conda-forge?

[dfm]

@DiamonDinoia Yes that's all good. @janden was suggesting step 2 should be removed, and I was just asking that it not be.

[DiamonDinoia]

Hi, thanks for pointing it out. We do not plan on removing step 2.

[janden]

Ok we can keep that then. Out of curiosity, do you manually remove the included .so from the built wheel then in conda-forge? Otherwise I don't see how we would ever hit this fallback.

[dfm]

On conda-forge the build is patched so that the Python extension never compiles the core libraries since those are packaged separately.

[janden]

Ok got it.

At any rate, since this is already included in #429, I'm closing this one.

[DiamonDinoia]

@dfm v2.3.0-rc1 is out with these fixes. Could you give us feedback on conda-forge? We switched to pyproject.toml which may cause problems.

[dfm]

Thanks for the heads up @DiamonDinoia! I probably won't have a chance to update the conda-forge recipe until after the final version is out, but I'm sure it'll be no problem to get it running!