Remove trailing slash in HDF5 download url

[viljarjf]

Hi,
I'm getting the following error when trying to install using bootstrap.py:

Installing HDF5...
  log file is /home/vife5188/cctbx_project/base_tmp/HDF5_install_log
  getting package hdf5-1.10.5.tar.bz2...
    downloading from https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.5/src//hdf5-1.10.5.tar.bz2 :     download failed with HTTP Error 404: Not Found    retrying in 3 seconds    download failed with HTTP Error 404: Not Found    retrying in 3 seconds    download failed with HTTP Error 404: Not FoundTraceback (most recent call last):
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 1672, in <module>
    installer(args=sys.argv, log=sys.stdout)
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 257, in __init__
    self.build_dependencies(packages=packages)
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 777, in build_dependencies
    getattr(self, 'build_%s'%i)()
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 1191, in build_hdf5
    hdf5pkg = self.fetch_package(pkg_name=HDF5_PKG, pkg_url=BASE_HDF5_PKG_URL)
  File "/home/vife5188/cctbx_project/modules/cctbx_project/libtbx/auto_build/package_defs.py", line 275, in __call__
    raise RuntimeError("Could not download " + pkg_name)
RuntimeError: Could not download hdf5-1.10.5.tar.bz2
Process failed with return code 1

which seems to be due to a trailing slash in the url in

cctbx_project/libtbx/auto_build/package_defs.py

Line 101 in 3360b22

BASE_HDF5_PKG_URL = "https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.5/src/"

An extra slash is later inserted here:

cctbx_project/libtbx/auto_build/package_defs.py

Line 254 in 3360b22

urls = ["%s/%s" % (pkg_url, pkg_name)]

This makes the full url have an extra slash:

https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.5/src//hdf5-1.10.5.tar.bz2
                                                                        ^

By removing this trailing slash, it seems to work as intended:

Installing HDF5...
  log file is /home/vife5188/cctbx_project/base_tmp/HDF5_install_log
  getting package hdf5-1.10.5.tar.bz2...
    downloading from https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.5/src/hdf5-1.10.5.tar.bz2 : 8.3 MB
    [0%.........20%.........40%.........60%.........80%.........100%]
  installing hdf5-1.10.5.tar.bz2...
  package hdf5 took 127.1s to install

As I am having some (unrelated) trouble installing the software, I can't run the tests myself, but a quick search did not find any tst_*-files containing "hdf5", so maybe this is not tested?

I am running WSL2 Ubuntu 22.04 on Windows 11, using python 3.7. I kept getting errors using python 3.10, so I downgraded and ran into the HDF5 download issue instead.

[bkpoon]

I suggest that you switch to using conda packages for dependencies. What is your bootstrap.py command? I'm in the process of updating our static conda environments and will update bootstrap.py to use conda by default.

Alternatively, if you're using cctbx as a library and not directly modifying cctbx code, you can use the pre-built conda packages. Instructions for installation are here https://github.com/cctbx/cctbx_project?tab=readme-ov-file#install

[viljarjf]

Thank you for the quick reply! Conda worked much better. The command was something like python3 bootstrap.py --builder=cctbx --with-python=python3 with an activated venv.

I'm not sure if I'll be modifying the code or using it just as a library yet, but thank you for the heads up!