Problem to build LAMMPS with cmake(QUIP and ScaLAPACK)

[ctxu-NWPU]

hello, i have a unresolved problem due to unknown reasons. When I want to build LAMMPS with cmake, i face a complex problem. The following is my operations.

1. cd lammps
2. mkdir build cd build
3. cmake -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_Fortran_COMPILER=mpifort -DMPI_C_COMPILER=mpicc -DMPI_CXX_COMPILER=mpicxx -DMPI_Fortran_COMPILER=mpifort -DMPI_LIBRARIES=/home/ctxu/lammps/mpich_install/lib/libmpi.so -DMPI_INCLUDE_DIRS=/home/ctxu/lammps/mpich_install/include -DLAMMPS_MACHINE=mpi -DGPU_API=cuda -DBLAS_LIBRARIES=/home/ctxu/gcc-9.5.0/lib64/libblas.a -DLAPACK_LIBRARIES=/home/ctxu/gcc-9.5.0/lib64/liblapack.a -DSCALAPACK_LIBRARIES=/home/ctxu/scalapack-2.2.0/libscalapack.a -DPKG_ML-QUIP=on -DDOWNLOAD_QUIP=no -DQUIP_LIBRARY=/home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a -DCMAKE_INSTALL_PREFIX=/home/ctxu/lammps/lammps …/cmake
   – Appending :/home/ctxu/lapack-3.12.0 to CMAKE_LIBRARY_PATH: /home/ctxu/gcc-9.5.0/lib64/libscalapack.a;:/home/ctxu/lapack-3.12.0

output:

– Appending :/home/ctxu/lapack-3.12.0 to CMAKE_LIBRARY_PATH: /home/ctxu/gcc-9.5.0/lib64/libscalapack.a;:/home/ctxu/lapack-3.12.0
– Running check for auto-generated files from make-based build system
– Could NOT find FFMPEG (missing: FFMPEG_EXECUTABLE)
– Generating style headers…
– Generating package headers…
– Generating lmpinstalledpkgs.h…
– Could NOT find ClangFormat (missing: ClangFormat_EXECUTABLE) (Required is at least version “8.0”)
– The following tools and libraries have been found and configured:

Git
MPI
OpenMP
LAPACK
BLAS
JPEG
PNG
ZLIB
QUIP
– <<< Build configuration >>>
LAMMPS Version: 20230802
Operating System: Linux CentOS 7
CMake Version: 3.25.2
Build type: RelWithDebInfo
Install path: /home/ctxu/lammps/lammps
Generator: Unix Makefiles using /usr/bin/gmake
– Enabled packages: ML-QUIP
– <<< Compilers and Flags: >>>
– C++ Compiler: /home/ctxu/lammps/mpich_install/bin/mpicxx
Type: GNU
Version: 9.5.0
C++ Flags: -O2 -g -DNDEBUG
Defines: LAMMPS_SMALLBIG;LAMMPS_MEMALIGN=64;LAMMPS_OMP_COMPAT=4;LAMMPS_JPEG;LAMMPS_PNG;LAMMPS_GZIP
– Fortran Compiler: /home/ctxu/lammps/mpich_install/bin/mpifort
Type: GNU
Version: 9.5.0
Fortran Flags: -O2 -g -DNDEBUG
– C compiler: /home/ctxu/lammps/mpich_install/bin/mpicc
Type: GNU
Version: 9.5.0
C Flags: -O2 -g -DNDEBUG
– <<< Linker flags: >>>
– Executable name: lmp_mpi
– Static library flags:
– <<< MPI flags >>>
– MPI_defines: MPICH_SKIP_MPICXX;OMPI_SKIP_MPICXX;_MPICC_H
– MPI includes:
– MPI libraries: ;
– Configuring done
– Generating done
– Build files have been written to: /home/ctxu/lammps/lammps/build

4. cmake --build .
   output:
   [ 0%] Built target fmt_format.h
   – Git Directory: /home/ctxu/lammps/lammps/.git
   – Generating lmpgitversion.h…
   [ 0%] Built target gitversion
   [ 0%] Built target angle.h
   [ 0%] Built target atom.h
   [ 0%] Built target bond.h
   [ 0%] Built target citeme.h
   [ 0%] Built target comm.h
   [ 0%] Built target command.h
   [ 1%] Built target compute.h
   [ 1%] Built target dihedral.h
   [ 1%] Built target domain.h
   [ 1%] Built target error.h
   [ 2%] Built target exceptions.h
   [ 2%] Built target fix.h
   [ 2%] Built target force.h
   [ 2%] Built target group.h
   [ 3%] Built target improper.h
   [ 3%] Built target input.h
   [ 3%] Built target info.h
   [ 3%] Built target kspace.h
   [ 3%] Built target lammps.h
   [ 3%] Built target lattice.h
   [ 4%] Built target library.h
   [ 4%] Built target lmppython.h
   [ 4%] Built target lmptype.h
   [ 5%] Built target memory.h
   [ 5%] Built target modify.h
   [ 5%] Built target neighbor.h
   [ 5%] Built target neigh_list.h
   [ 6%] Built target output.h
   [ 6%] Built target pair.h
   [ 6%] Built target platform.h
   [ 6%] Built target pointers.h
   [ 6%] Built target region.h
   [ 7%] Built target timer.h
   [ 7%] Built target universe.h
   [ 7%] Built target update.h
   [ 7%] Built target utils.h
   [ 8%] Built target variable.h
   [ 9%] Built target fmt_core.h
   [100%] Built target lammps
   [100%] Linking CXX executable lmp_mpi
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_get_lwork_pdormqr_i32o64': ScaLAPACK.f95:(.text+0x852): undefined reference to indxg2p’
   ScaLAPACK.f95:(.text+0x879): undefined reference to indxg2p' ScaLAPACK.f95:(.text+0x8aa): undefined reference to numroc_’
   ScaLAPACK.f95:(.text+0x8e1): undefined reference to numroc_' ScaLAPACK.f95:(.text+0x97e): undefined reference to indxg2p_’
   ScaLAPACK.f95:(.text+0x9a8): undefined reference to numroc_' ScaLAPACK.f95:(.text+0x9bb): undefined reference to ilcm_’
   ScaLAPACK.f95:(.text+0x9ec): undefined reference to numroc_' ScaLAPACK.f95:(.text+0xa0f): undefined reference to numroc_’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_get_lwork_pdgeqrf_i32o64': ScaLAPACK.f95:(.text+0xb3d): undefined reference to indxg2p’
   ScaLAPACK.f95:(.text+0xb5f): undefined reference to indxg2p' ScaLAPACK.f95:(.text+0xb96): undefined reference to numroc_’
   ScaLAPACK.f95:(.text+0xbbe): undefined reference to numroc_' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_to_array2d’:
   ScaLAPACK.f95:(.text+0xd2f): undefined reference to descinit_' ScaLAPACK.f95:(.text+0xeae): undefined reference to pdgeadd’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_to_array1d': ScaLAPACK.f95:(.text+0x106d): undefined reference to descinit’
   ScaLAPACK.f95:(.text+0x11c9): undefined reference to pdgeadd' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_pdtrmr2d_wrapper’:
   ScaLAPACK.f95:(.text+0x1533): undefined reference to pdtrmr2d' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_pdgemr2d_wrapper’:
   ScaLAPACK.f95:(.text+0x1849): undefined reference to pdgemr2d' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_pdtrtrs_wrapper’:
   ScaLAPACK.f95:(.text+0x1af2): undefined reference to pdtrtrs' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_pdormqr_wrapper’:
   ScaLAPACK.f95:(.text+0x1e82): undefined reference to pdormqr_' ScaLAPACK.f95:(.text+0x20c4): undefined reference to pdormqr’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_pdgeqrf_wrapper': ScaLAPACK.f95:(.text+0x22d4): undefined reference to pdgeqrf’
   ScaLAPACK.f95:(.text+0x23ed): undefined reference to pdgeqrf' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_matrix_product_sub_zzz’:
   ScaLAPACK.f95:(.text+0x2d07): undefined reference to pzgemm' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_matrix_product_sub_ddd’:
   ScaLAPACK.f95:(.text+0x3334): undefined reference to pdgemm_' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_diagonalise_gen_c’:
   ScaLAPACK.f95:(.text+0x3ce9): undefined reference to pzhegvx_' ScaLAPACK.f95:(.text+0x40a5): undefined reference to pzhegvx’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_diagonalise_gen_r': ScaLAPACK.f95:(.text+0x5446): undefined reference to pdsygvx’
   ScaLAPACK.f95:(.text+0x5778): undefined reference to pdsygvx' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_diagonalise_c’:
   ScaLAPACK.f95:(.text+0x683c): undefined reference to pzheevx' ScaLAPACK.f95:(.text+0x6be1): undefined reference to pzheevx’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_diagonalise_r': ScaLAPACK.f95:(.text+0x7bbf): undefined reference to pdsyevx’
   ScaLAPACK.f95:(.text+0x7ee6): undefined reference to pdsyevx' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_inverse_c’:
   ScaLAPACK.f95:(.text+0x8b11): undefined reference to numroc' ScaLAPACK.f95:(.text+0x8b34): undefined reference to numroc’
   ScaLAPACK.f95:(.text+0x8b7f): undefined reference to numroc_' ScaLAPACK.f95:(.text+0x8bc9): undefined reference to pzgetrf’
   ScaLAPACK.f95:(.text+0x8cb3): undefined reference to pzgetri_' ScaLAPACK.f95:(.text+0x8dcc): undefined reference to pzgetri’
   ScaLAPACK.f95:(.text+0x8f24): undefined reference to numroc_' ScaLAPACK.f95:(.text+0x8f7f): undefined reference to numroc’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_inverse_r': ScaLAPACK.f95:(.text+0x959f): undefined reference to numroc’
   ScaLAPACK.f95:(.text+0x95c2): undefined reference to numroc' ScaLAPACK.f95:(.text+0x960d): undefined reference to numroc’
   ScaLAPACK.f95:(.text+0x9657): undefined reference to pdgetrf' ScaLAPACK.f95:(.text+0x973f): undefined reference to pdgetri’
   ScaLAPACK.f95:(.text+0x9858): undefined reference to pdgetri_' ScaLAPACK.f95:(.text+0x99b4): undefined reference to numroc’
   ScaLAPACK.f95:(.text+0x9a0f): undefined reference to numroc_' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_scalapack_init_matrix_desc’:
   ScaLAPACK.f95:(.text+0xa4c0): undefined reference to numroc_' ScaLAPACK.f95:(.text+0xa4e5): undefined reference to numroc’
   ScaLAPACK.f95:(.text+0xa597): undefined reference to descinit_' /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function scalapack_module_MOD_matrix_scalapack_info_coords_local_to_global’:
   ScaLAPACK.f95:(.text+0xa5fc): undefined reference to indxl2g_' ScaLAPACK.f95:(.text+0xa61b): undefined reference to indxl2g’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function __scalapack_module_MOD_matrix_scalapack_info_coords_global_to_local': ScaLAPACK.f95:(.text+0xc34a): undefined reference to infog2l’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function __scalapack_module_MOD_scalapack_finalise': ScaLAPACK.f95:(.text+0xd3a5): undefined reference to blacs_gridexit’
   /home/ctxu/lammps/lammps/lib/quip/QUIP/build/linux_x86_64_gfortran_openmpi+openmp/libquip.a(ScaLAPACK.o): In function __scalapack_module_MOD_scalapack_initialise': ScaLAPACK.f95:(.text+0xd5d5): undefined reference to blacs_gridinit’
   ScaLAPACK.f95:(.text+0xd5f9): undefined reference to `blacs_gridinfo’
   collect2: error: ld returned 1 exit status
   gmake[2]: *** [lmp_mpi] Error 1
   gmake[1]: *** [CMakeFiles/lmp.dir/all] Error 2
   gmake: *** [all] Error 2

As you can see, although I tried to build scalapack when cmake, it still had a lot of problems.
I want to know how to gap the error. I'm sorry to bother you. @max-veit @Montmorency @jameskermode @bernstei

[bernstei]

I've never compiled LAMMPS with ScaLAPACK, so I don't know why you'd want to do that. Regardless, it looks like, for some reason, doing it causes LAMMPS to enable ScaLAPACK inside QUIP, which is wrong because LAMMPS needs a completely serial QUIP library. Also, it appears that the link line that cmake generates doesn't actually link in scalapack for some reason, but even if it did, this could be bad, because you don't want ScaLAPACK active inside QUIP.

If you don't really need scalapack for some LAMMPS reason, don't enable it. If you do, we may have to dig into how LAMMPS compiles QUIP and stop ScaLAPACK in LAMMPS from turning on ScaLAPACK in QUIP.

[ctxu-NWPU]

I've never compiled LAMMPS with ScaLAPACK, so I don't know why you'd want to do that. Regardless, it looks like, for some reason, doing it causes LAMMPS to enable ScaLAPACK inside QUIP, which is wrong because LAMMPS needs a completely serial QUIP library. Also, it appears that the link line that cmake generates doesn't actually link in scalapack for some reason, but even if it did, this could be bad, because you don't want ScaLAPACK active inside QUIP.

If you don't really need scalapack for some LAMMPS reason, don't enable it. If you do, we may have to dig into how LAMMPS compiles QUIP and stop ScaLAPACK in LAMMPS from turning on ScaLAPACK in QUIP.

Thanks for your suggestion, I'll try to fix this by excluding ScaLAPACK in quip. My purpose is to reproduce the simulation of silicon in the past paper with quip/lammps.

[bernstei]

You definitely don't need any of the lapack/scalapack flags passed to cmake. All I routinely use (pretty sure cmake is depending on MKLROOT env var to find MKL for fftw, but if you don't have MKL, you can just remove the -D FFT flag altogether) is the following. Pretty sure essentially none of those options are even needed if all you plan to run is QUIP.

cmake -D BUILD_SHARED_LIBS=ON  \
      -D PKG_CLASS2=ON \
      -D PKG_KSPACE=ON \
      -D PKG_MANYBODY=ON \
      -D PKG_MEAM=ON \
      -D PKG_MISC=ON \
      -D PKG_ML-QUIP=ON \
      -D PKG_MOLECULE=ON \
      -D PKG_REAXFF=ON \
      -D YAML_BUILD_SHARED_LIBS=OFF \
      -D FFT=MKL \
      ../cmake