Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add h5py #1354

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

Add h5py #1354

wants to merge 12 commits into from

Conversation

talmo
Copy link
Contributor

@talmo talmo commented Sep 29, 2024

  • Addresses h5py package #1352
  • Based off of adding h5py recipe #685
  • Updated recipe to current rattler-build syntax
  • Updated to h5py=3.11.0
  • Using this monkey patch for removing -R flags from emcc calls.
  • Assuming hdf5=1.12.2. I think we have 1.12.3 available in emscripten-forge, but the older version is getting pulled in by the CI and there is a feature flag in an env var that breaks the API if the version isn't set correctly.
  • Added libhdf5=1.12.1 as a shared library rather than the static hdf5 one. This is based on the pyodide recipe.

@talmo talmo mentioned this pull request Sep 29, 2024
@shimwell shimwell mentioned this pull request Sep 29, 2024
@talmo
Copy link
Contributor Author

talmo commented Sep 29, 2024

At this point I think I'm stuck. The package builds, but then we get loads of errors deep in HDF5. There are plenty of suspicious warnings earlier on, but unclear where it's coming from.

Maybe @DerThorsten or @bmaranville have some ideas?

@talmo
Copy link
Contributor Author

talmo commented Sep 29, 2024

Perhaps the hdf5 package in this repo needs to be built as a shared library like libhdf5 over in Pyodide?

@DerThorsten
Copy link
Contributor

At this point I think I'm stuck. The package builds, but then we get loads of errors deep in HDF5. There are plenty of suspicious warnings earlier on, but unclear where it's coming from.

Maybe @DerThorsten or @bmaranville have some ideas?

I might find some time to look into this, but I would also need to do some major debugging / experimentation

@bmaranville
Copy link

Sorry, I can't be much help until I understand how this very specific build process works...

I went through the instructions for doing a local build, but after I build libhdf5 and then h5py, I get an error:

rattler-build build  --recipe recipes/recipes_emscripten/h5py/recipe.yaml  --target-platform=emscripten-wasm32 -c https://repo.mamba.pm/emscripten-forge -c conda-forge -c microsoft -m conda_build_config.yaml
...
Error building package: Failed to resolve dependencies: Cannot solve the request because of: No candidates were found for libhdf5 *

How do you tell the h5py build to use the locally-built libhdf5?

@talmo
Copy link
Contributor Author

talmo commented Oct 1, 2024

I think the issue is that we're building two conda packages successfully (libhdf5 and h5py), but we're not telling the second one (h5py) where to find its dependency (libhdf5).

With conda-build I think you have to set the local build directory as an additional channel to look for locally-built packages, but I'm not so sure how to do that in rattler-build.

Alternatively, we could get libhdf5 merged in and uploaded to the repository so that h5py can find it, but it'd be nice to be able to test it first before uploading.

Maybe @DerThorsten or @wolfv can provide some quick assistance? I think the hard part is done (the packages individually are getting built!).

@DerThorsten
Copy link
Contributor

@talmo I hope I will find some time to debug this, but cannot promise to do It this week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants