Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

podman native build #122

Merged
merged 12 commits into from
Mar 27, 2024
Merged

podman native build #122

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
180b67b
podman build
edoapra Mar 27, 2024
8ec2626
podman build
edoapra Mar 27, 2024
23cf80b
podman build
edoapra Mar 27, 2024
ef2fe52
podman build
edoapra Mar 27, 2024
c4cf2f5
cuda args
edoapra Mar 27, 2024
fbf4e87
cuda args
edoapra Mar 27, 2024
a07871d
cuda args
edoapra Mar 27, 2024
e403f81
cuda args
edoapra Mar 27, 2024
9e89244
cuda args
edoapra Mar 27, 2024
d6726fa
cuda args
edoapra Mar 27, 2024
04f0214
fix test step
edoapra Mar 27, 2024
036b616
svn gone
edoapra Mar 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 185 additions & 0 deletions .github/workflows/nwchem-podman-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
name: nwchem_podman_build

on:
repository_dispatch:
types: [backend_automation]
workflow_dispatch:

jobs:
shifter_schedule:
if: (github.event_name == 'workflow_dispatch') || (github.event_name == 'schedule')
strategy:
fail-fast: false
matrix:
folder:
- nwchem-dev.mpipr.nersc_podman.mpich4
branch: [ master, hotfix/release-7-2-0]
openmp: [ Y, N]
fc: [ gfortran]
armci_network: [ MPI-PR ]
include:
- folder: nwchem-dev.mpipr.nersc_podman.mpich4
armci_network: MPI-PR
openmp: N
branch: master
fc: nvfortran
- folder: nwchem-dev.mpipr.nersc_podman.mpich4
armci_network: MPI-PR
openmp: N
branch: release-7-2-0
fc: nvfortran
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: FC tag
id: fc-tag
run: |
if [[ ${{ matrix.fc }} == 'gfortran' ]]; then
echo "fc=" >> $GITHUB_ENV
else
echo "fc=$(echo .${{matrix.fc }} )" >> $GITHUB_ENV
fi
- name: cache key
run: |
echo "cache_key=${{ matrix.folder }}-${{ matrix.branch }}${{ env.fc }}-cache-v005" >> $GITHUB_ENV
- name: Setup cache
id: setup-cache
uses: actions/cache@v4
with:
path: |
~/cache
key: ${{ env.cache_key }}
- name: Login to GitHub Container Registry
if: ${{ github.actor == github.repository_owner }}
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# username: ${{ secrets.DOCKERHUB_USERNAME }}
# password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: fetch cache
if: (steps.setup-cache.outputs.cache-hit == 'true')
run: |
cd ${{ matrix.folder }}
ls -lrta
if [[ -f ~/cache/libext.tar.bz2 ]]; then \
echo "libext cache present" ; \
mkdir -p cache|| true ; \
rsync -av ~/cache/libext* cache/. ; \
echo "libext cache fetched" ; \
fi
if [[ -f ~/cache/simint.tar.bz2 ]]; then \
echo "simint cache present" ; \
mkdir -p cache|| true ; \
rsync -av ~/cache/simint* cache/. ; \
echo "simint cache fetched" ; \
fi
if [[ -f ~/cache/mpich.tar.bz2 ]]; then \
echo "mpich cache present" ; \
mkdir -p cache|| true ; \
rsync -av ~/cache/mpich* cache/. ; \
echo "mpich cache fetched" ; \
ls -l cache/ ; \
echo "======" ; \
fi
- name: openmp tag
id: openmp-tag
run: |
if [[ ${{ matrix.openmp }} == 'N' ]]; then
echo "omp_tag=_no_openmp" >> $GITHUB_ENV
else
echo "omp_tag=" >> $GITHUB_ENV
fi
shell: bash
- name: armci_net tag
id: armci_net
run: |
echo "armci_net=.$(echo ${{ matrix.armci_network }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
shell: bash
- name: nwchem version tag
run: |
if [[ ${{ matrix.branch }} == master ]]; then
echo "longnwversion=$(echo ${{ matrix.folder }}| sed -e 's|.mpipr||')" >> $GITHUB_ENV
elif [[ ${{ matrix.branch }} == hotfix/release-7-2-0 ]]; then
echo "longnwversion=$(echo ${{ matrix.folder }} | sed -e 's|.mpipr||' | sed -e 's|dev|720|')" >> $GITHUB_ENV
else
echo "longnwversion=$(echo ${{ matrix.folder }} | sed -e 's|.mpipr||' | sed -e 's|dev|${{ matrix.branch }}|')" >> $GITHUB_ENV
fi
shell: bash
- name: docker tag
id: docker-tag
run: |
echo "tag=ghcr.io/${{ github.actor }}/${{ env.longnwversion}}${{ env.omp_tag }}${{ env.armci_net }}${{ env.fc }}" >> $GITHUB_ENV
shell: bash
- name: builddate
id: build-date
run: |
echo "date=$(TZ=America/Los_Angeles date +%Y%m%d_%H%M%S)" >> $GITHUB_ENV
- name: Install podman
run: |
sudo apt-get install -y podman
which podman
podman version
- name: build podman
run: |
cd ${{ matrix.folder }}
podman build \
--build-arg FC=${{ matrix.fc }} \
--build-arg USE_OPENMP_VALUE=${{ matrix.openmp }} \
--build-arg NWCHEM_BRANCH=${{ matrix.branch}} \
--build-arg ARMCI_NETWORK=${{ matrix.armci_network}} \
--build-arg CACHE_HIT=${{ env.cache_hit }} \
--build-arg GITHUB_REPOSITORY_OWNER=${{ github.repository_owner }} \
--build-arg CUDA_VERSION_MAJOR=11 \
--build-arg CUDA_VERSION_MINOR=7 \
--format docker \
-t ${{ env.tag }}:latest .
- name: push to ghcr.io
run: |
echo ${{ secrets.GITHUB_TOKEN }} | podman login -u ${{ github.actor }} --password-stdin ghcr.io
podman push ${{ env.tag }}:latest ${{ env.tag }}:${{ env.date }}
podman push ${{ env.tag }}:latest ${{ env.tag }}:latest
- name: load and test image
if: ( github.actor == github.repository_owner )
run: |
sleep 30s
podman pull ${{ env.tag }}
export MYLDPATH=/usr/local/lib:/opt/lib
if [[ ${{ matrix.fc }} == nvfortran ]]; then
podman run -e LD_LIBRARY_PATH="$MYLDPATH" --entrypoint=ldd --rm \
${{ env.tag }} /opt/nwchem/bin/LINUX64/nwchem
else
cd /tmp
umask u=rwx,g=rwx,o=rwx
umask -S
git clone --no-checkout https://github.com/nwchemgit/nwchem
cd nwchem
git sparse-checkout init --cone
git sparse-checkout set QA
git checkout
cd QA
podman run --rm -e LD_LIBRARY_PATH="$MYLDPATH" \
-e HWLOC_HIDE_ERRORS=2 -w /opt/nwchem/QA \
-v `pwd`:/opt/nwchem/QA --entrypoint='/opt/nwchem/QA/runtests.mpi.unix' \
${{ env.tag }} \
procs 2 h2o_opt prop_mep_gcube
fi
- name: store cache
if: ( github.actor == github.repository_owner )
run: |
mkdir -p ~/cache/
podman run --privileged --user 0 --rm \
--entrypoint='/bin/cp' -v ~/cache:/data \
${{ env.tag }} \
-rp /opt/nwchem/src/libext/libext.tar.bz2 /data/. || true
podman run --privileged --user 0 --rm \
--entrypoint='/bin/cp' -v ~/cache:/data \
${{ env.tag }} \
-rp /opt/nwchem/src/libext/simint.tar.bz2 /data/. || true
podman run --privileged --user 0 --rm \
--entrypoint='/bin/cp' -v ~/cache:/data \
${{ env.tag }} \
-rp /usr/local/mpich.tar.bz2 /data/. || true
ls -Ralrt ~/cache || true
8 changes: 4 additions & 4 deletions nwchem-dev.mpipr.nersc_podman.mpich4/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ ENV GITHUB_REPOSITORY_OWNER=$GITHUB_REPOSITORY_OWNER
ENV ARMCI_NETWORK=$ARMCI_NETWORK
ARG mpich=4.0.2
ARG mpich_prefix=mpich-$mpich
ARG CUDA_VERSION_MAJOR=11 \
CUDA_VERSION_MINOR=7
ARG PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/cuda-"$CUDA_VERSION_MAJOR"."$CUDA_VERSION_MINOR"/bin:/opt/nwchem/bin:/usr/local/bin \
LD_LIBRARY_PATH=/usr/local/cuda-"$CUDA_VERSION_MAJOR"."$CUDA_VERSION_MINOR"/lib64:/usr/local/lib
ARG CUDA_VERSION_MAJOR=11
ARG CUDA_VERSION_MINOR=7
ARG PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/cuda-"$CUDA_VERSION_MAJOR"."$CUDA_VERSION_MINOR"/bin:/opt/nwchem/bin:/usr/local/bin
ARG LD_LIBRARY_PATH=/usr/local/cuda-"$CUDA_VERSION_MAJOR"."$CUDA_VERSION_MINOR"/lib64:/usr/local/lib
#get OpenBLAS and compile with 64-bit integers
WORKDIR /opt
ENV NWCHEM_TOP="/opt/nwchem"
Expand Down
Loading