C2SM Guidelines for Spack
-Spack enables users to install software in a very user-friendly way, -for example by allowing different versions or specifications -of the same package to be installed simultaneously, or installing -without having to “manually” download the source code. This comes at -the expense of potentially losing control over the exact -version/specification being installed. For this reason, C2SM has -the following guidelines for building, running and installing your -libraries and executables.
-Building
-There are two possible ways of building software with Spack:
-spack install and spack dev-build.
-Both are good, but have some special features that need to be taken into account.
Option 1: spack install
-Every spack install command needs a version suffix,
-i.e. spack install <package>@<version-suffix>
-This version-suffix can have different meanings:
-
-
branch in the git repository
-one out of several git repositories defined in the package
-a specifc version (git-tag) with a corresponding git-hash hardcoded in the package
-
Only for the last item in the list above you will always fetch and
-compile the same code. The two other items can lead to different
-codes in case the HEAD of this specific branch/repository has received some
-additional commits in the meantime.
Especially for production it is very important to know which version of a code is actually used.
-For example, there is a variety of different version suffixes for the cosmo package:
git = 'git@github.com:COSMO-ORG/cosmo.git'
-apngit = 'git@github.com:MeteoSwiss-APN/cosmo.git'
-c2smgit = 'git@github.com:C2SM-RCM/cosmo.git'
-
-version('org-master', branch='master', get_full_repo=True)
-version('apn-mch', git=apngit, branch='mch', get_full_repo=True)
-version('c2sm-master', git=c2smgit, branch='master', get_full_repo=True)
-version('c2sm-features', git=c2smgit, branch='c2sm-features', get_full_repo=True)
-version(5.09, git=c2sm, tag=5.09, get_full_repo=True)
-Here, there are three different git repositories available for the cosmo package:
-
-
COSMO-ORG/cosmo.git: version-suffix
org-master
-MeteoSwiss-APN/cosmo.git: version-suffix
apn-mch
-C2SM-RCM/cosmo.git: version-suffix
c2sm-master
-C2SM-RCM/cosmo.git: version-suffix
c2sm-features
-
It is clear that only using spack install <package>@5.09 will
-always result in the same code, all other version only point to a
-HEAD of a git branch.
The list of versions, including tagged versions, is provided by spack
-info <package_name>.
Attention
-Always use a valid git tag as a version-suffix when building
-software with spack install for production!
Option 2: spack dev-build
-In order to install software with spack dev-build, one needs a
-local source code. Spack will then compile the code as it is locally
-present. Contrary to spack install, the version suffix
-(@master, @v2.7.9, etc.) does not have any effect on the code version compiled,
-since Spack will always use the local source.
Nonetheless, it is important to use the correct version suffix, i.e. c2sm-master
-for all your COSMO versions that build on top of the c2sm-master branch.
-The same applies to c2sm-features etc.
-The reason for this is that there are some patches Spack applies based on the version suffix.
-Using the wrong version suffix may break your code.
Attention
-You may run into a conflict if you try to install the same spec using dev-build.
-Spack will tell you that this particular spec is already installed.
Tip
-To circumvent the above conflict, you can keep your executable alive in your source folder,
-but omit the installation phase using spack dev-build --until build <your_spec>.
Running
-When used properly, Spack is able to manage many different -configurations of a package along with the corresponding -run environment.
-Load run environment of a package
-Spack provides the command spack load to load an installation into your environment.
-This could either be a Python installation or required variables that
-a specific binary needs to run correctly. There are two different ways of using it
-(both of them are fine), which are presented here.
-For more information consider reading the
-official Spack docs.
$ spack load <package>@<version>%<compiler> +<variants>
-The executable now has the correct environment to run in your current shell.
-The other possibility is use spack load to print the required
-shell commands and store them in a file that can be sourced at a later
-stage:
$ spack load --sh <package>@<version>%<compiler> +<variants> > run_package.env
-An example output of spack load --sh for COSMO could look as follows:
export LIBRARY_PATH=/opt/cray/pe/mpt/7.7.15/gni/mpich-pgi/20.1/lib:/project/s903/juckerj/spack-install/daint/eccodes/2.19.0/pgi/ccigv3uvkdl5h3d2jtb6blxvvv4qsdpc/lib64:/apps/daint/UES/xalt/xalt2/software/xalt/2.8.10/lib64:/apps/daint/UES/xalt/xalt2/software/xalt/2.8.10/lib;
-export LD_LIBRARY_PATH=/opt/cray/pe/mpt/7.7.15/gni/mpich-pgi/20.1/lib:/project/s903/juckerj/spack-install/daint/eccodes/2.19.0/pgi/ccigv3uvkdl5h3d2jtb6blxvvv4qsdpc/lib64:/opt/cray/pe/gcc-libs:/apps/daint/UES/xalt/xalt2/software/xalt/2.8.10/lib64:/apps/daint/UES/xalt/xalt2/software/xalt/2.8.10/lib:/opt/cray/pe/papi/6.0.0.4/lib64:/opt/cray/job/2.2.4-7.0.2.1_2.86__g36b56f4.ari/lib64;
-export GRIB_SAMPLES_PATH=/project/s903/juckerj/spack-install/daint/cosmo-eccodes-definitions/2.19.0.5/pgi/egf6fp466u2cl3ckkmhpemzf4hz7loqr/cosmoDefinitions/samples;
-export GRIB_DEFINITION_PATH=/project/s903/juckerj/spack-install/daint/cosmo-eccodes-definitions/2.19.0.5/pgi/egf6fp466u2cl3ckkmhpemzf4hz7loqr/cosmoDefinitions/definitions/:/project/s903/juckerj/spack-install/daint/eccodes/2.19.0/pgi/ccigv3uvkdl5h3d2jtb6blxvvv4qsdpc/share/eccodes/definitions;
-Tip
-Always load the run environment provided by Spack prior to any -executions of an executable installed by Spack!
-Spack in scripts
-The Spack commands are rather tailored for interacive use. For example,
-it is very possible for commands such as spack find or spack
-location to complain about multiple potential installed SPECS satisfying
-the command line input. For this reason, it is advisable to
-avoid spack commands in scripts. However, for spack find and
-spack location, this should not be aproblem. For spack load, we rather
-recommend to use it from the login nodes before submitting jobs, inheriting
-the environment of the running job from the environment at submission time.
-