Build modules in new project folder on Frontier #119
Conversation
There was a problem hiding this comment.
Since we are at this, I suggest:
- We build everything with at least ROCm >= 5.6
- Build Ginkgo >= 1.7.0; the experimental branch is not supported and will likely be deleted.
- Consider moving location of modules to world shared. This will allow collaborators outside the project to build our code (e.g. at hackathons)
| module load hip/5.2.0-clang-14.0.0-rocm5.2.0-mixed-6ftqihk | ||
| # hsa-rocr-dev@=5.2.0%clang@=14.0.0-rocm5.2.0-mixed~asan+image+shared build_system=cmake build_type=Release generator=make patches=9267179 arch=linux-sles15-zen3 |
There was a problem hiding this comment.
We need ROCm >= 5.6. Also, I suggest using Frontier modules rather than building our own.
There was a problem hiding this comment.
See #89 regarding rocm >=5.6.
For the rocm modules, I am using the system modules (see https://github.com/pnnl/ExaGO/blob/nicholson/frontier-eng145/buildsystem/spack/crusher/spack.yaml#L208), but Spack likes to add a wrapper to it.
There was a problem hiding this comment.
Thanks for clarifying these are Spack wrappers.
As for #89, this looks like a bug in PETSc Spack package.
There was a problem hiding this comment.
There is a workaround for the PETSc error. I am more concerned about the failing tests, especially in Debug mode.
| module load ginkgo/1.5.0.glu_experimental-clang-14.0.0-rocm5.2.0-mixed-6bembpq | ||
| # gmake@=4.4.1%gcc@=12.2.0-mixed~guile build_system=generic arch=linux-sles15-zen3 |
There was a problem hiding this comment.
We need Ginkgo >= 1.7. I think glu_experimental branch will be deleted soon anyways.
There was a problem hiding this comment.
I am not aware that HiOp's interface to [email protected] has been updated.
There was a problem hiding this comment.
It has not. However, glu_experimental is unsupported Ginkgo branch incompatible with mainstream Ginkgo, so it is not clear which solution is worse.
There was a problem hiding this comment.
Perhaps we should build Ginkgo >= 1.7, but build HiOp without Ginkgo support enabled. This will at least create an environment in which developers could complete HiOp interface updates. Just my $0.02 tip.
| module use -a /lustre/orion/eng145/proj-shared/nkouk/spack-install/modules/linux-sles15-zen3 | ||
| # exago@=develop%clang@=14.0.0-rocm5.2.0-mixed~cuda+hiop~ipo+ipopt+logging+mpi~python+raja+rocm amdgpu_target=gfx90a build_system=cmake build_type=Release dev_path=/lustre/orion/scratch/nkouk/eng145/ExaGO generator=make arch=linux-sles15-zen3 |
There was a problem hiding this comment.
Could we install these modules in "world shared" folder?
There was a problem hiding this comment.
I'll make a new pull request.
Merge request type
Relates to
This MR updates
Summary
New build of ExaGO and dependencies on Frontier in the new project shared folder.