Skip to content

Commit

Permalink
Merge pull request #164 from xylar/drop-alpha-from-spack-env-name
Browse files Browse the repository at this point in the history
Use only Polaris release version is spack env name
  • Loading branch information
xylar authored Jan 17, 2024
2 parents 0c67247 + cf29ebc commit 260489c
Show file tree
Hide file tree
Showing 3 changed files with 27 additions and 8 deletions.
2 changes: 1 addition & 1 deletion configure_polaris_envs.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def setup_install_env(env_name, activate_base, use_local, logger, recreate,
channels = '--use-local'
else:
channels = ''
packages = f'progressbar2 jinja2 {mache}'
packages = f'jinja2 {mache} packaging progressbar2'
if recreate or not os.path.exists(env_path):
print('Setting up a conda environment for installing polaris\n')
commands = f'{activate_base} && ' \
Expand Down
5 changes: 4 additions & 1 deletion deploy/bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from mache import MachineInfo
from mache import discover_machine as mache_discover_machine
from mache.spack import get_spack_script, make_spack_env
from packaging import version
from shared import (
check_call,
get_conda_base,
Expand Down Expand Up @@ -210,7 +211,9 @@ def get_env_setup(args, config, machine, compiler, mpi, env_type, source_path,
check_supported('petsc', machine, compiler, mpi, source_path)

if env_type == 'dev':
spack_env = f'dev_polaris_{polaris_version}{env_suffix}'
ver = version.parse(polaris_version)
release_version = '.'.join(str(vr) for vr in ver.release)
spack_env = f'dev_polaris_{release_version}{env_suffix}'
elif env_type == 'test_release':
spack_env = f'test_polaris_{polaris_version}{env_suffix}'
else:
Expand Down
28 changes: 22 additions & 6 deletions docs/developers_guide/deploying_spack.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ If system modules change in E3SM, we try to stay in sync:
- pnetcdf
- mkl (or other linear algebra libs)

When we update the `mache` version in Polaris, we also need to bump the
Polaris version (typically either the major or the minor version) and then
re-deploy shared spack environments on each supported machine.

### Spack

Spack is for libraries used by MPAS and tools that need system MPI:
Expand All @@ -25,13 +29,21 @@ Spack is for libraries used by MPAS and tools that need system MPI:
- PETSc
- Netlib LAPACK

When we update the versions of any of these libraries in Polaris, we also need
to bump the Polaris version (typically either the major or the minor version)
and then re-deploy shared spack environments on each supported machine.

### Conda

Conda (via conda-forge) is used for python packages and related dependencies
that don’t need system MPI. Conda environments aren’t shared between
developers because the polaris you’re developing is part of the conda
environment.

When we update the constraints on conda dependencies, we also need to bump the
Polaris alpha, beta or rc version. We do not need to re-deploy spack
environments on share machines because they remain unaffected.

## Mache

A brief tour of mache.
Expand Down Expand Up @@ -100,14 +112,18 @@ by polaris. Here, we will use `<fork>` as a stand-in for the fork of mache
to use (e.g. `E3SM-Project/mache`) and `<branch>` as the stand-in for a branch on
that fork (e.g. `main`).

We also need to make sure there is a spack branch for the version of mache.
This is a branch off of the develop branch on
We also need to make sure there is a spack branch for the version of Polaris.
The spack branch is a branch off of the develop branch on
[E3SM’s spack repo](https://github.com/E3SM-Project/spack) that has any
updates to packages required for this version of mache and polaris. The
spack branch will omit any alpha, beta or rc suffix on the mache version
because it is intended to be the spack branch we will use once the `mache`
release happens. In this example, we will work with the branch
uupdates to packages required for this version of mache. The remote branch
is named after the release version of mache (omitting any alpha, beta or rc
suffix because it is intended to be the spack branch we will use once the
``mache`` release happens). In this example, we will work with the branch
[spack_for_mache_1.12.0](https://github.com/E3SM-Project/spack/tree/spack_for_mache_1.12.0).
The local clone is instead named after the Polaris version (again any omitting
alpha, beta or rc) plus the compiler and MPI library because we have discovered
two users cannot make modifications to the same git clone. Giving each clone
of the spack branch a unique name ensures that they are independent.

Here's how to get a branch of polaris we're testing (`simplify_local_mache`
in this case) as a local worktree:
Expand Down

0 comments on commit 260489c

Please sign in to comment.